From patchwork Thu Oct 10 18:23:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830833 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B5C22CFC5EC for ; Thu, 10 Oct 2024 18:33:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:Content-Type:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=Om/x5O1cFoCu45xFQlZl7muX4OGvvwFhFfu/OqGssvQ=; b=nIOKDqwS+ogDD3 HILxH/OSKjK/f3EH0+qFwezhDm86XefG6nj2qQfo3aKSSGuQaMvhDgvU+VMnw+GF0pnTvEpo7BiC2 JN5kCrTAGouPheoM3eVsD8jXdpRIyf5rjUb8FP3M3c/DoMs6pWUCN+rwAYlapTQygyJZJjynrDuKo KBYQYSBqB2f9eBMEqhYymHCr1D9njd9u4WPri5GlriKA4O/rAiq/LMmDYMudlinTolEWjT/UmZYgN 0ThBj+ZKymKRw8znKxSfGK56tyv3E1Lee+GxJrq6FgIQsQ3WphN1PudE/qcphew5qfyY+tSNObdbt 72yL2gcDvPXPSYwZdiGw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syxyE-0000000DuQh-03tI; Thu, 10 Oct 2024 18:33:06 +0000 Received: from mail-pj1-x1049.google.com ([2607:f8b0:4864:20::1049]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxqH-0000000Dpc3-174y for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:24:54 +0000 Received: by mail-pj1-x1049.google.com with SMTP id 98e67ed59e1d1-2e2de6496easo620779a91.1 for ; Thu, 10 Oct 2024 11:24:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584691; x=1729189491; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=Om/x5O1cFoCu45xFQlZl7muX4OGvvwFhFfu/OqGssvQ=; b=tCsBk8HaNFJgE7k6da8O+927jDUhAdilLbD6Ct0HgJr6ILzIqoH7HqDbdGX3904isM mgjRlaLXsMfuyAoxkw1WPhJdK3pB5VO/dxbxxKMIal5c9Z8EWR5FbmnyBwNxrl584hj8 llhunBIIUNFcrX/3VjyNPRQ3o0OsEX7z8X0VE4Bgz/2ziMssp34F8oGWo+58KaUGxXnp GgSspRdccjVn7EAlku3QQ/kf2DmnFRd4xHkQRA0RG2GhInoi/RZeZJ1lmT5zInJExN8C nubiDEnlewvl8u2EFKmo4455ddHS15hNF7FMoQlwgmnzw50YNrRY+bHWh95QruI1FQ0a A0vA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584691; x=1729189491; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=Om/x5O1cFoCu45xFQlZl7muX4OGvvwFhFfu/OqGssvQ=; b=jmaKdBZYIR3ZV0V9iwogbUZHa+Ld29lT/hi0qdTdtcc+6T3djcb4BK0Le36giUXW09 uHaw1XRf36kFkMwLlsrkNLteKfxt9Q+3JKpty76fr3CnTUea1x2gCEectV9S2LA8LX+1 ZM50B6F/yTfMdm8tT5Rua9srFx/Hzzvi4+oGe168JaQomEAfKSfDtSNbWQU6TmBKHzQU V8l1PWNch6Pbzo5V2itGuM0x8pj6qFKurDUHEajJtyXKf+nMNhImOP0ZZVmVDs5hhERa qMAc+5eQlhLAHCzCjpTNlcuLCWFQNMWzhRc5BxC3x1TAXPpo7DQh5+chXL/0Y5Yr3dar Qw1g== X-Forwarded-Encrypted: i=1; AJvYcCXUZZKxy/yrwr4l8qLCgvQU23l3R/a1yVFGPycm4y18eGXYrhMgOM7+jKRmMCLIGozD3cBwrqFyAgt5Jhjd30+6@lists.infradead.org X-Gm-Message-State: AOJu0YzydlIFvzPoJWTbCFOgVHu/+9cxsLrT//fD7UykfD7rCh2z2HnC B9hJDykUBnHbh65kqdZegIfMhKv4s9AglVkFxILWCM9BSOfOEAY7L2+o9sWA09zDA3FMBPn2RXh BFQ== X-Google-Smtp-Source: AGHT+IG89v4lIFRhnNqxe9hFvIIxqDvldHGwH//6ib2L4odxYyk8SJA2JGOjXIHuxBeh8ylp7HgA6abaBbM= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:90a:eac4:b0:2e2:8f4d:457 with SMTP id 98e67ed59e1d1-2e2c80b5015mr7266a91.2.1728584689990; Thu, 10 Oct 2024 11:24:49 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:03 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-2-seanjc@google.com> Subject: [PATCH v13 01/85] KVM: Drop KVM_ERR_PTR_BAD_PAGE and instead return NULL to indicate an error From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112453_384894_7BA1492F X-CRM114-Status: GOOD ( 13.83 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Remove KVM_ERR_PTR_BAD_PAGE and instead return NULL, as "bad page" is just a leftover bit of weirdness from days of old when KVM stuffed a "bad" page into the guest instead of actually handling missing pages. See commit cea7bb21280e ("KVM: MMU: Make gfn_to_page() always safe"). Reviewed-by: Alex Bennée Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/powerpc/kvm/book3s_pr.c | 2 +- arch/powerpc/kvm/book3s_xive_native.c | 2 +- arch/s390/kvm/vsie.c | 2 +- arch/x86/kvm/lapic.c | 2 +- include/linux/kvm_host.h | 7 ------- virt/kvm/kvm_main.c | 15 ++++++--------- 6 files changed, 10 insertions(+), 20 deletions(-) diff --git a/arch/powerpc/kvm/book3s_pr.c b/arch/powerpc/kvm/book3s_pr.c index 7b8ae509328f..d7721297b9b6 100644 --- a/arch/powerpc/kvm/book3s_pr.c +++ b/arch/powerpc/kvm/book3s_pr.c @@ -645,7 +645,7 @@ static void kvmppc_patch_dcbz(struct kvm_vcpu *vcpu, struct kvmppc_pte *pte) int i; hpage = gfn_to_page(vcpu->kvm, pte->raddr >> PAGE_SHIFT); - if (is_error_page(hpage)) + if (!hpage) return; hpage_offset = pte->raddr & ~PAGE_MASK; diff --git a/arch/powerpc/kvm/book3s_xive_native.c b/arch/powerpc/kvm/book3s_xive_native.c index 6e2ebbd8aaac..d9bf1bc3ff61 100644 --- a/arch/powerpc/kvm/book3s_xive_native.c +++ b/arch/powerpc/kvm/book3s_xive_native.c @@ -654,7 +654,7 @@ static int kvmppc_xive_native_set_queue_config(struct kvmppc_xive *xive, } page = gfn_to_page(kvm, gfn); - if (is_error_page(page)) { + if (!page) { srcu_read_unlock(&kvm->srcu, srcu_idx); pr_err("Couldn't get queue page %llx!\n", kvm_eq.qaddr); return -EINVAL; diff --git a/arch/s390/kvm/vsie.c b/arch/s390/kvm/vsie.c index 89cafea4c41f..763a070f5955 100644 --- a/arch/s390/kvm/vsie.c +++ b/arch/s390/kvm/vsie.c @@ -661,7 +661,7 @@ static int pin_guest_page(struct kvm *kvm, gpa_t gpa, hpa_t *hpa) struct page *page; page = gfn_to_page(kvm, gpa_to_gfn(gpa)); - if (is_error_page(page)) + if (!page) return -EINVAL; *hpa = (hpa_t)page_to_phys(page) + (gpa & ~PAGE_MASK); return 0; diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c index 2098dc689088..20526e4d6c62 100644 --- a/arch/x86/kvm/lapic.c +++ b/arch/x86/kvm/lapic.c @@ -2664,7 +2664,7 @@ int kvm_alloc_apic_access_page(struct kvm *kvm) } page = gfn_to_page(kvm, APIC_DEFAULT_PHYS_BASE >> PAGE_SHIFT); - if (is_error_page(page)) { + if (!page) { ret = -EFAULT; goto out; } diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index db567d26f7b9..ee186a1fbaad 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -153,13 +153,6 @@ static inline bool kvm_is_error_gpa(gpa_t gpa) return gpa == INVALID_GPA; } -#define KVM_ERR_PTR_BAD_PAGE (ERR_PTR(-ENOENT)) - -static inline bool is_error_page(struct page *page) -{ - return IS_ERR(page); -} - #define KVM_REQUEST_MASK GENMASK(7,0) #define KVM_REQUEST_NO_WAKEUP BIT(8) #define KVM_REQUEST_WAIT BIT(9) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 05cbb2548d99..4b659a649dfa 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -3078,19 +3078,14 @@ EXPORT_SYMBOL_GPL(gfn_to_page_many_atomic); */ struct page *gfn_to_page(struct kvm *kvm, gfn_t gfn) { - struct page *page; kvm_pfn_t pfn; pfn = gfn_to_pfn(kvm, gfn); if (is_error_noslot_pfn(pfn)) - return KVM_ERR_PTR_BAD_PAGE; + return NULL; - page = kvm_pfn_to_refcounted_page(pfn); - if (!page) - return KVM_ERR_PTR_BAD_PAGE; - - return page; + return kvm_pfn_to_refcounted_page(pfn); } EXPORT_SYMBOL_GPL(gfn_to_page); @@ -3184,7 +3179,8 @@ static void kvm_set_page_accessed(struct page *page) void kvm_release_page_clean(struct page *page) { - WARN_ON(is_error_page(page)); + if (WARN_ON(!page)) + return; kvm_set_page_accessed(page); put_page(page); @@ -3208,7 +3204,8 @@ EXPORT_SYMBOL_GPL(kvm_release_pfn_clean); void kvm_release_page_dirty(struct page *page) { - WARN_ON(is_error_page(page)); + if (WARN_ON(!page)) + return; kvm_set_page_dirty(page); kvm_release_page_clean(page); From patchwork Thu Oct 10 18:23:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830834 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A9D0CCFC5EB for ; Thu, 10 Oct 2024 18:34:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:Content-Type:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=yUMe47Ku2d+lU53n+UV6eA4G9zH9pOaBnB7PUsg7n0E=; b=CbrSm7vohtEieR ccOGV1I/m2I99WSx+5iqG4MszpezUafv/uMRzKMVdLA54OKvYCby712vd1a8SGfPwqZzwSmvXpA+l OIuQfFntqjArVT2v0jp+n5d+AoCBe27BZl/BmXexGU+iNttnXveu/Upc63MHth2KM1iOIRkoMJgT1 yQLmouuu1SqjZuN8d7GxfCeVqTvl6BrikJ8jh4znJgyfNTRYdsh4GIkBxYwERDuttPcnAMFESDUAi O6y07YQjX1hlCKvl4E2XhaGyMAe6OdZdkdWosHyAz+5JiRnFZdBkOmjOOG6XBMWE1vUBhTRuda75M SJLVUGjff3o6uzW+r+/A==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syxzr-0000000DvDk-0G3Q; Thu, 10 Oct 2024 18:34:47 +0000 Received: from mail-pl1-x64a.google.com ([2607:f8b0:4864:20::64a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxqI-0000000Dpct-3HHF for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:24:56 +0000 Received: by mail-pl1-x64a.google.com with SMTP id d9443c01a7336-20b6144cc2aso12164395ad.0 for ; Thu, 10 Oct 2024 11:24:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584693; x=1729189493; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=yUMe47Ku2d+lU53n+UV6eA4G9zH9pOaBnB7PUsg7n0E=; b=shU7ea7RlpvjkLopneCrUBkZlJ/eYg9PruxHXuYYqoC41aPvYR8MviFIZwfv2Bs/38 V6DM4Rhjol1hLNMqyclecumybgC3FqZubBkU0Ging5vsyF/ycStxP1eNRZ8RqYcwUO6+ QIFFjFEsNwFdIFpKQOMcyhdiOrIwnUVQ0aczRiDG3U5DVsHpbLHR2gdJH5XPGINnkyOW FA4jebfmXxekNvf9FQp5hWO5lvvjLPmzuDXreg+TSJZ7IRbakavxEH2Txn31TG40lqKU +aVDrvuoXBNGaoduvnd40hFU1e+OVk+YXrp2xtTayTEAIRuw2evu92DsRJUkYmp4M0rX Kp8g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584693; x=1729189493; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=yUMe47Ku2d+lU53n+UV6eA4G9zH9pOaBnB7PUsg7n0E=; b=bG01pyhpzd7eD43tdt7B9KFz/XHr+b+43U2A3cWnT5AvcyVst9ph06AD6TgFa1bUER Dg8eU5Qq78f9ZsoKeXbf4nxTK+QyAAmKspMwZXmeDwJkKuqP1OlQqNvZb4l6qD21QuLp wH66Tar7IsYkGkBOKSdfppdld12HCIkYsthy1K47su1f4YcMbGrr/kzPnZgeYvabzPd0 Vgoq6n4zHkrUOsyAEFS1m2VGZGWChHjgg+zthyoyFx/22EPCZOVvAOd5do+PF0yIvcEh c4KG9YGLjTcVeE2eSFFYlyGvMtaM6xOA+gaO4XbJXL9ULPlSSONH0+dvrfDXAmM0M3kC Ysfw== X-Forwarded-Encrypted: i=1; AJvYcCXFpV5qrIlNQI2EWu0FIs8N4Ll+fg0ocX8gUzD6Flh+tlsADFY76iZWgWS1ZoT8Ha79RPFWiwOUxGebo/yJyCaR@lists.infradead.org X-Gm-Message-State: AOJu0YxMKOutdZgi9MmrHtETog6BsJLYCFx9zfwUMdHcC9xpAT/hLlqW Eaf9AVB2Mz7gyu6gKVIi8Ls6PLwYsVUVPCCVdC/PGh7g9ZQUaub2+8eQd5C5Rmox6RRZ39ps1IV HdA== X-Google-Smtp-Source: AGHT+IHgBvu4/6UpOEaDnS659CK1Rn1/E5r8mTk00ZsUtMRUIXJafj8Ignnkw5WxXqtPYIrr50vXY4UIGHU= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:902:ff02:b0:205:58ee:1567 with SMTP id d9443c01a7336-20c80362278mr43175ad.0.1728584692824; Thu, 10 Oct 2024 11:24:52 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:04 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-3-seanjc@google.com> Subject: [PATCH v13 02/85] KVM: Allow calling kvm_release_page_{clean,dirty}() on a NULL page pointer From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112454_833611_6ED62C7B X-CRM114-Status: GOOD ( 11.68 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Allow passing a NULL @page to kvm_release_page_{clean,dirty}(), there's no tangible benefit to forcing the callers to pre-check @page, and it ends up generating a lot of duplicate boilerplate code. Reviewed-by: Alex Bennée Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- virt/kvm/kvm_main.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 4b659a649dfa..2032292df0b0 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -3179,7 +3179,7 @@ static void kvm_set_page_accessed(struct page *page) void kvm_release_page_clean(struct page *page) { - if (WARN_ON(!page)) + if (!page) return; kvm_set_page_accessed(page); @@ -3204,7 +3204,7 @@ EXPORT_SYMBOL_GPL(kvm_release_pfn_clean); void kvm_release_page_dirty(struct page *page) { - if (WARN_ON(!page)) + if (!page) return; kvm_set_page_dirty(page); From patchwork Thu Oct 10 18:23:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830839 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C2E98CFC5EC for ; Thu, 10 Oct 2024 18:36:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:Content-Type:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=8PiI9IzFtQVAZLsCzTZ7ta7WGdR9XnSo4A5BSh1uxRg=; b=0Wzk2fmWj8nKod yT1TTXmVde6ji2aDBQZKzR+9712Bxukx6FKkEWIIpMoXHv4pRN6Us78eKRm+Tu9Pnz7a4dm2lpWDD c++91Ds7+u5NbDr9miO/74VovYta6g7zfyiZHp6SM6mDSUkgSP2lhfbpHra4raqSt4syMS/47FDNK jG4bqMeJRLBle6469nAFFuAFMllbi9Z2oAC3AD4lddpZg+H/g5w2+YfAnCMNBX8dVu42zGCwCEopE 41PqEU4Z99M7I2ioUTbAAczSS4cEUUr7VgXrJVPJvKbIkuTeMnpoj1j3o5T6/jpoYA628HdFN2cQ+ 1T3cRo1994ZMG1qmxBrw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syy1F-0000000Dvey-3vND; Thu, 10 Oct 2024 18:36:13 +0000 Received: from mail-yw1-x1149.google.com ([2607:f8b0:4864:20::1149]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxqK-0000000Dpe8-0UkE for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:24:58 +0000 Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-6e284982a31so22414797b3.3 for ; Thu, 10 Oct 2024 11:24:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584695; x=1729189495; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=8PiI9IzFtQVAZLsCzTZ7ta7WGdR9XnSo4A5BSh1uxRg=; b=3tpcWmDhC2p5JkkW8aB29EYcT5Tc9T3eDSrIYpr4d6VdBPy/WHXsR4WShFJm6c9Rgp IwMeB1Akf45dADb/TeQKAfku3V7l/dzk41yiqET2SfY9fONopnJnvP4hE5Hi/kSiofPx 7m6lAJganol+7NKovRFL2zZudjYJRd+Ve4FMWnQ8wtVVq2/6aCMfJM9zEq5h12F9kJoc EXxzXUN61izp/gYQafC5ezdxs0Hmt2qr6ciEnfI2pE51ck2xY2nfnsRpO0j2R6TmnFtR tvKzRl3/fD9ufDJ5YS//pg6k3swCPzJTaPvEKExRNguqGGAS0pSD+9d7EBjVWsvevfaN /u3A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584695; x=1729189495; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=8PiI9IzFtQVAZLsCzTZ7ta7WGdR9XnSo4A5BSh1uxRg=; b=IxLqY7aeY71EEfhn/cKpF12UpqOT26h1378zsw+u2XciQph8gp4mTUG+7w162MKkGl K4VIAYVQObYnsd8m/hR8C7PAqHjT+D8ROe+ghRLonVFUojiIlWzOwegXCr49MBcxjS1a 8Ig4I0Pu3CtSF751cLx30fkB4nGLKgHVMzyKcQ0LkEwHamLj5fbUvLZxAp+d4v1BztPb KuhigMi7D97VyPsn06SnWKX4FXHZ2snvNvPkGLS2vGF4rkMQyx7aiP/o4TgO1WjnB6Tl JB+cCDMVS4PsK1cvPvdb1iDNm6yKvOD6OQu8zttEcfgavIur7QjEw2WfGDt32o4M4OUQ OJqg== X-Forwarded-Encrypted: i=1; AJvYcCXssNOC3KXlhBSSZs8CMorLafZ0/dKvHGUwOq0kTlFDk15iT6YKA70m7pR2KgqmHknx9HgBtJ9WiVwzJA1tNNF4@lists.infradead.org X-Gm-Message-State: AOJu0YyhzisJxQ87F7NixOEkQRddLissEB6UX77D7zu53wlljGxJR+sa 94sHEizL95nZvX77aoCN44kXJvbhCXM99Kb1GouPb/QC7+wQB0wm2ZyAIc16hkiLmShwBIgWd6t f5A== X-Google-Smtp-Source: AGHT+IGz6mODxZTH5agCMYiGZey+8PwFJu50se4JqpXBNG0SwyKrN9TWGGQJqqmZYCdc2Fnc7eka1IGyLLE= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a81:ad16:0:b0:6c1:298e:5a7 with SMTP id 00721157ae682-6e3224662ffmr1632867b3.5.1728584694780; Thu, 10 Oct 2024 11:24:54 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:05 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-4-seanjc@google.com> Subject: [PATCH v13 03/85] KVM: Add kvm_release_page_unused() API to put pages that KVM never consumes From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112456_201885_577E4E6F X-CRM114-Status: GOOD ( 10.02 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Add an API to release an unused page, i.e. to put a page without marking it accessed or dirty. The API will be used when KVM faults-in a page but bails before installing the guest mapping (and other similar flows). Reviewed-by: Alex Bennée Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- include/linux/kvm_host.h | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index ee186a1fbaad..ab4485b2bddc 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -1216,6 +1216,15 @@ unsigned long gfn_to_hva_prot(struct kvm *kvm, gfn_t gfn, bool *writable); unsigned long gfn_to_hva_memslot(struct kvm_memory_slot *slot, gfn_t gfn); unsigned long gfn_to_hva_memslot_prot(struct kvm_memory_slot *slot, gfn_t gfn, bool *writable); + +static inline void kvm_release_page_unused(struct page *page) +{ + if (!page) + return; + + put_page(page); +} + void kvm_release_page_clean(struct page *page); void kvm_release_page_dirty(struct page *page); From patchwork Thu Oct 10 18:23:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830840 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 020B8CFC5F0 for ; Thu, 10 Oct 2024 18:37:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:Content-Type:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=X4CpCR2xmh5xLGpBn2Bl3P0Y4RCeyU3sCMc2ynGYEmk=; b=jxW7uER1jDhEn8 341qklki9xvztKXeK4RfYuk8QJpT6zDsVA3Mp5S2D+lwi5SUAF6hzNsAPVyv3ruA/duJhztQcqOga QMkYFrGWQUpRzLnhPtdPvUKzV23JAymcSdSaNdAd5zPuP8b+ajZKRF4pLImdyvtXwQeZgHv+FNgsh tyviEgosrN1LZgSfbvLf+bewDkdYHFoN7quXuNmqk+qFlGybfkevmlYBJDTttVZD3s1pMIniKSuBM wkVhH0YO3uV+rclYEjZhGUOnNrcQLf6qHpT/i9UV/PULpqaRvx7+1rMnoHy3KJRr+88ztwOfGJQ8B iONBfWJPYWUFuLfju1Uw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syy2d-0000000Dw3T-1LRe; Thu, 10 Oct 2024 18:37:39 +0000 Received: from mail-yw1-x114a.google.com ([2607:f8b0:4864:20::114a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxqL-0000000DpfC-46t3 for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:24:59 +0000 Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-6e26ba37314so24841897b3.0 for ; Thu, 10 Oct 2024 11:24:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584697; x=1729189497; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=X4CpCR2xmh5xLGpBn2Bl3P0Y4RCeyU3sCMc2ynGYEmk=; b=M7Snr5N22pikPCYYXMJYsOlgIEBf4+x3Uqg6sVxeZVadYTICrG4ulpFRgPtXVG0X0n JZ2/nX9phcaEtKWHj1yPru+AarfrxgYbtzXswb2v92ZPGKNmtdah06jEFW/mllb+oibh CBKgB1cMUdcgVGm+aTjLPjjrmXe2Hfze6XuIV0ZeaSFds53FKVgiaPzajxfWki+VusBG 0ZxzebIV31ZFXuzuGmx7TJqF8oZ/P8kgCd05nBiQQ6We33hwrDG5iyqppNKEN7BXJoyY U1tRYYoYlyeYl0xm2A4lPSen0EvpIggWblDdQu+MQKURLXfflhNdB/DNtQyuAg89xkyJ 7Jtw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584697; x=1729189497; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=X4CpCR2xmh5xLGpBn2Bl3P0Y4RCeyU3sCMc2ynGYEmk=; b=BtVr5cyHmr71nDrfDfA225ncOmiYBalFYQ3w6vhrSS0T6EyQu3AbPF4wamnPE6bnoQ KGeW/6qKgFKyFcjgAf5Kko+FQ5fNiIr6ZQCpPK1XKY7mr48prR4lQzijhvfnggmSTRrl iVkAwjuFaeL/bz206r0LX+LEuAjJ4tLw03ImSgci/5/4Ddyn1IxBbi6TuPtiBwZ37LA8 VRS6Nkrj/LzsWASsmYJCyJqui2Ckot3eqysLIJFsAyeUbal3MUliiHIxWZO8o3/oub00 m8Orfl5PIW0QBw1pt/a7sGXd/PKf5iZQTpv3HC8MtO1iEQfaVd7UnMkG8+JaWUn7lEri X+gg== X-Forwarded-Encrypted: i=1; AJvYcCWBLCE3bINxMS4N5E8yKa1gg6Kl2YwcT09KJUsFbwTz9dcUT+0VLRQ4C00G9ZJHOYQpHAl41XGsIX4GmuJDiHvt@lists.infradead.org X-Gm-Message-State: AOJu0YyWy2cCvpbnCZkKokwdR6tGpotlVVVmk2iVa4enB1EYzRppDRgr O1dxxBBnkHBbH8wOalNSEqZjvtXsZBCPK6heg4zmzESczclucPEXlGVtqRs3U3Y0KM2l15HJQfv ZvQ== X-Google-Smtp-Source: AGHT+IGrULPD5dT5NWC9dFAfgTsoL9JIWGzHU9t8+6e7flVyO5QHPCRlr/jDUZCu+nhXBHYl+WGzejB3KuQ= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a0d:e486:0:b0:6e2:371f:4aef with SMTP id 00721157ae682-6e322168931mr204147b3.3.1728584696651; Thu, 10 Oct 2024 11:24:56 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:06 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-5-seanjc@google.com> Subject: [PATCH v13 04/85] KVM: x86/mmu: Skip the "try unsync" path iff the old SPTE was a leaf SPTE From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112458_044660_D6147033 X-CRM114-Status: GOOD ( 13.76 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Apply make_spte()'s optimization to skip trying to unsync shadow pages if and only if the old SPTE was a leaf SPTE, as non-leaf SPTEs in direct MMUs are always writable, i.e. could trigger a false positive and incorrectly lead to KVM creating a SPTE without write-protecting or marking shadow pages unsync. This bug only affects the TDP MMU, as the shadow MMU only overwrites a shadow-present SPTE when synchronizing SPTEs (and only 4KiB SPTEs can be unsync). Specifically, mmu_set_spte() drops any non-leaf SPTEs *before* calling make_spte(), whereas the TDP MMU can do a direct replacement of a page table with the leaf SPTE. Opportunistically update the comment to explain why skipping the unsync stuff is safe, as opposed to simply saying "it's someone else's problem". Cc: stable@vger.kernel.org Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/spte.c | 18 +++++++++++++----- 1 file changed, 13 insertions(+), 5 deletions(-) diff --git a/arch/x86/kvm/mmu/spte.c b/arch/x86/kvm/mmu/spte.c index 8f7eb3ad88fc..5521608077ec 100644 --- a/arch/x86/kvm/mmu/spte.c +++ b/arch/x86/kvm/mmu/spte.c @@ -226,12 +226,20 @@ bool make_spte(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp, spte |= PT_WRITABLE_MASK | shadow_mmu_writable_mask; /* - * Optimization: for pte sync, if spte was writable the hash - * lookup is unnecessary (and expensive). Write protection - * is responsibility of kvm_mmu_get_page / kvm_mmu_sync_roots. - * Same reasoning can be applied to dirty page accounting. + * When overwriting an existing leaf SPTE, and the old SPTE was + * writable, skip trying to unsync shadow pages as any relevant + * shadow pages must already be unsync, i.e. the hash lookup is + * unnecessary (and expensive). + * + * The same reasoning applies to dirty page/folio accounting; + * KVM will mark the folio dirty using the old SPTE, thus + * there's no need to immediately mark the new SPTE as dirty. + * + * Note, both cases rely on KVM not changing PFNs without first + * zapping the old SPTE, which is guaranteed by both the shadow + * MMU and the TDP MMU. */ - if (is_writable_pte(old_spte)) + if (is_last_spte(old_spte, level) && is_writable_pte(old_spte)) goto out; /* From patchwork Thu Oct 10 18:23:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830841 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C989DCFC5EB for ; Thu, 10 Oct 2024 18:39:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Type:Cc:To: From:Subject:Message-ID:References:Mime-Version:In-Reply-To:Date: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=pPAj0F/7+N3G0fgp8rSBmKqwV8uR3Qu6XgX2gyD+fUo=; b=SAZNWrhc2NSut9l6W24c2snB9c Iheubn3VUeFvfGCo8+kX0eIz9a//L63En5Hzg13HvJlbQCJZzEw7Hb439c+sZgh1459Or7DDx7g/h Q+7WYlpAQ0PeWwyRH7YXqkd3fYMjt7pvQX7+kiZ+2ANoeOMemxThnB6BJwBB82CtN6/159dHJvzO9 cEk9yb7WR35uPpGv5izMO3V2aNzGtfp9C4PC369u3GHt198nMWxCqEOiJKJmnz7YZb33+g5S8atoc UhTbE1eUlayg+TssbafrV30/eps/uGPDKGLUtiDJf98q27x3qkz7czvRFIVPhiWy0FOgBYiYUdRjn ndlIk8wg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syy41-0000000DwVK-2isT; Thu, 10 Oct 2024 18:39:05 +0000 Received: from mail-yb1-xb49.google.com ([2607:f8b0:4864:20::b49]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxqR-0000000Dphv-2HQD for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:25:06 +0000 Received: by mail-yb1-xb49.google.com with SMTP id 3f1490d57ef6-e2904a54a10so1670019276.3 for ; Thu, 10 Oct 2024 11:25:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584701; x=1729189501; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=pPAj0F/7+N3G0fgp8rSBmKqwV8uR3Qu6XgX2gyD+fUo=; b=uho4rH1Ltyn1x521PuC9kj2h8uorL+rNfA819qT8YPYqGmkARzx2HIJ5RX8b2Scx4o pw/8G2HLJn/wBNNPeQ6BOz9/y0sV2YGDI5/YkZ17dMebLtTQ+0YWwHuVk4SmdTx5MAgN FHZZCWw/6icoy28KbaiYRQKg61SsybK9ggVaoLwVN3TJAUYXjGF9WxagSKJzPcFWeglj f9BIxNwTn3XGX8Vv/64aL2hbxWVunNr959SYvURuL3nwI+eRAI4hyJWuaoB/ScpM0t93 kQuWM2/o9Ivo3R1Z7sbhcoTCwzmOGAQTziNvlKTG8eLizopDKY05CWMPLITK8cJOPl76 waBQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584701; x=1729189501; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=pPAj0F/7+N3G0fgp8rSBmKqwV8uR3Qu6XgX2gyD+fUo=; b=BSeLfDrGY1Ys+3cLXA1Ut4PnVL4ZeRZGTxdmNScrJer88YN4Dzw9xuhKfr5R93xV8C WxrFJVhI7RNoNu6CkdQpWUtwezq+C5xp2MkCnoxFiq7r0htvxFR/yIxfvXRUJddMy226 oSQN0q8XUNTsN7tCb6vAuNoNCFUSFnyLXgtZOdQL2D7249CUmv/dBwVB+OSjHYHQ0QU5 So0dItwAEqgbgdoEa6/VZfTJQLR29+CP0HA10JYMFByizvxzZHHj97hPKbJjMu138cnw IQftgVxd01hmlldIuQ7APD/jKnHb+EdEO+sbBBmDX/4jxXUrvQNqvLL9l9Ff3yHgTejF nxmg== X-Forwarded-Encrypted: i=1; AJvYcCV5SHdfAUUr0a3ZpIlX517AhJGtqjv1c4Gv135KkQ5xqgKklkMJW2Qq2Se/4sDs+c6lS0SUwJWW3GsPk5QppS9s@lists.infradead.org X-Gm-Message-State: AOJu0YzZnoDD38UpDxl4Z+IjhN6dehv2THt4bt3GvZq79Vu4MIFylAV4 aAvZc1hKG9RB3NlM7581wcLd78uh33LexPiBZp50YcALGPhlXN4A2+696qx8jAQGmiXBHv0SFOV vbw== X-Google-Smtp-Source: AGHT+IHQgMXiDY3ifHkkle2NgOnYNSR5nXtws6LnwLsJ4J6Fqip1dtdD5j5lAqBgp/4MJdCcEXFALInKEMI= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a25:69c3:0:b0:e1a:9ed2:67f4 with SMTP id 3f1490d57ef6-e28fe4313ffmr4949276.2.1728584700890; Thu, 10 Oct 2024 11:25:00 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:07 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-6-seanjc@google.com> Subject: [PATCH v13 05/85] KVM: x86/mmu: Don't overwrite shadow-present MMU SPTEs when prefaulting From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112503_724944_E63F7B26 X-CRM114-Status: GOOD ( 13.43 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Treat attempts to prefetch/prefault MMU SPTEs as spurious if there's an existing shadow-present SPTE, as overwriting a SPTE that may have been create by a "real" fault is at best confusing, and at worst potentially harmful. E.g. mmu_try_to_unsync_pages() doesn't unsync when prefetching, which creates a scenario where KVM could try to replace a Writable SPTE with a !Writable SPTE, as sp->unsync is checked prior to acquiring mmu_unsync_pages_lock. Note, this applies to three of the four flavors of "prefetch" in KVM: - KVM_PRE_FAULT_MEMORY - Async #PF (host or PV) - Prefetching The fourth flavor, SPTE synchronization, i.e. FNAME(sync_spte), _only_ overwrites shadow-present SPTEs when calling make_spte(). But SPTE synchronization specifically uses mmu_spte_update(), and so naturally avoids the @prefetch check in mmu_set_spte(). Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/mmu.c | 3 +++ arch/x86/kvm/mmu/tdp_mmu.c | 3 +++ 2 files changed, 6 insertions(+) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index a9a23e058555..a8c64069aa89 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -2919,6 +2919,9 @@ static int mmu_set_spte(struct kvm_vcpu *vcpu, struct kvm_memory_slot *slot, } if (is_shadow_present_pte(*sptep)) { + if (prefetch) + return RET_PF_SPURIOUS; + /* * If we overwrite a PTE page pointer with a 2MB PMD, unlink * the parent of the now unreachable PTE. diff --git a/arch/x86/kvm/mmu/tdp_mmu.c b/arch/x86/kvm/mmu/tdp_mmu.c index 3b996c1fdaab..3c6583468742 100644 --- a/arch/x86/kvm/mmu/tdp_mmu.c +++ b/arch/x86/kvm/mmu/tdp_mmu.c @@ -1026,6 +1026,9 @@ static int tdp_mmu_map_handle_target_level(struct kvm_vcpu *vcpu, if (WARN_ON_ONCE(sp->role.level != fault->goal_level)) return RET_PF_RETRY; + if (fault->prefetch && is_shadow_present_pte(iter->old_spte)) + return RET_PF_SPURIOUS; + if (unlikely(!fault->slot)) new_spte = make_mmio_spte(vcpu, iter->gfn, ACC_ALL); else From patchwork Thu Oct 10 18:23:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830847 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 7A400CFC5EC for ; Thu, 10 Oct 2024 18:40:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Type:Cc:To: From:Subject:Message-ID:References:Mime-Version:In-Reply-To:Date: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=zmCOkT+wuJeT9H1MjXHCrAkGhLQoiMWK376kHF4kwCg=; b=uc60mfbwdlM7nw6hbsrjmp8tnB O1xK8eqCx24oI+rEXO0vw7iKbX9fwc4tiOFN01XhxkxadywnixYgImlBmCOtX7y6k8lOlhOfFjS4l Vnd4QQOjRhuAn0pdK/20fTQg5YWUrzTgbH2Xqhah4t67s7cnI0efBY0UpB0ioJVtjqectTkmIdCoU gDU6f8V0A/U2KiAGhjKBS5PpIXHROFCBfa/2KxSPfGu0cGbW8LekCl+YCnFEaDCBw35m5bpl347uo JU9VHK4D0BqBdkq7tkQPxbFDkaXMSFZIQZQ2a36z8cLF0Y98T+djlJj+NXJonWmgUNnvU8QKGF5Dw F8lNQ22Q==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syy5M-0000000DwrV-3TXo; Thu, 10 Oct 2024 18:40:28 +0000 Received: from mail-yb1-xb4a.google.com ([2607:f8b0:4864:20::b4a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxqT-0000000Dpjy-0JQP for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:25:08 +0000 Received: by mail-yb1-xb4a.google.com with SMTP id 3f1490d57ef6-e28fc8902e6so1776706276.0 for ; Thu, 10 Oct 2024 11:25:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584704; x=1729189504; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=zmCOkT+wuJeT9H1MjXHCrAkGhLQoiMWK376kHF4kwCg=; b=Kq1WOn3ILTCesOLo5TwG98ce4X4S2wBk+Zed3w07pth0mmR27m6+zcIyUDuJ90K+tm H+FQy9IEhYfF0RVk1sN+uPV9VcfkmlC388brN9oIUwZNBfslQrpSLQpjPEekeKwAECAW 2yGteXV6Zs9Ai7X0iKtDFBT6EI5+bcL0iQSZm+JwuYIL/GZ6kqokrCisnWfZI2cp/w52 DNf/U/JvuC2w3tIklSm+kpLK8DDwtz7HtLUb62f7F49qPLWneNZV6W3Chjdfb5rxikNY YWlXE+zrdqJb3OTa3cM6srJ+EqsaVuKwpF+4ltKq8VJKibN0Sh4+lO62nT1rXvo2yqGY OoRg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584704; x=1729189504; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=zmCOkT+wuJeT9H1MjXHCrAkGhLQoiMWK376kHF4kwCg=; b=SeoQPaEgXXjWr3K3y2tTz3PIt4LkHmKR226hZxm/GUM6jJASPd8XHJp55H/SBYNf/T Q5B6eg37xwzyjdS8xZEFB/rKOcLcPVBCEcnjPx8RUbQqR/AVIrOSvCzNpAqF5rBEOI/D eY3HSvQmOW/RUSR7j+rRYOItUORKLVGGYcQNt/2hiRhzCg1I/E84BajZbwJh0OZX9ouu WDz2pfPf2Tg8lR3Yk7ZLXXGB/b9Pat9I+k+E2ryOJn8SEzcciRozHrO2Z55rjUo1dlcd JwEGmamwTPMHYG3mcQXt2417xBBLepRFHb6Vk09n/0AYm+ZmeCDcYfIc/Psb5mN65sMZ X/gQ== X-Forwarded-Encrypted: i=1; AJvYcCW1bFhB/rnc0P+B3FEcGfK1V8l/eFOIsJwyWlAhCSkOmOiBPTsIleTdYwWQhyinCU5hXrhJ+vUrfsnRJ3k1lqEX@lists.infradead.org X-Gm-Message-State: AOJu0YyS+COdRDD5zseuH5iKgPTMgN7IRM6tq7YhWiUqjGHYyroglqng bXBk1kxU46e09dSrRncuXzQrxK1gKe8E9tkOymKehppCh6X4apYVtD2PvQ9YbteJo2yF7Ru6a6y 6nQ== X-Google-Smtp-Source: AGHT+IHLS32ZnUy4bY0gdVqRjJSlR5N1zmb3UxYk5drKGwVtxhq/Y8NIZIwMp1duovDzYwXPQsOfO5WBu4o= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:6902:1812:b0:e24:9f58:dd17 with SMTP id 3f1490d57ef6-e28fe32f042mr66754276.1.1728584702932; Thu, 10 Oct 2024 11:25:02 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:08 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-7-seanjc@google.com> Subject: [PATCH v13 06/85] KVM: x86/mmu: Invert @can_unsync and renamed to @synchronizing From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112505_256081_8A690218 X-CRM114-Status: GOOD ( 17.05 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Invert the polarity of "can_unsync" and rename the parameter to "synchronizing" to allow a future change to set the Accessed bit if KVM is synchronizing an existing SPTE. Querying "can_unsync" in that case is nonsensical, as the fact that KVM can't unsync SPTEs doesn't provide any justification for setting the Accessed bit. Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/mmu.c | 12 ++++++------ arch/x86/kvm/mmu/mmu_internal.h | 2 +- arch/x86/kvm/mmu/paging_tmpl.h | 2 +- arch/x86/kvm/mmu/spte.c | 4 ++-- arch/x86/kvm/mmu/spte.h | 2 +- arch/x86/kvm/mmu/tdp_mmu.c | 4 ++-- 6 files changed, 13 insertions(+), 13 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index a8c64069aa89..0f21d6f76cab 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -2795,7 +2795,7 @@ static void kvm_unsync_page(struct kvm *kvm, struct kvm_mmu_page *sp) * be write-protected. */ int mmu_try_to_unsync_pages(struct kvm *kvm, const struct kvm_memory_slot *slot, - gfn_t gfn, bool can_unsync, bool prefetch) + gfn_t gfn, bool synchronizing, bool prefetch) { struct kvm_mmu_page *sp; bool locked = false; @@ -2810,12 +2810,12 @@ int mmu_try_to_unsync_pages(struct kvm *kvm, const struct kvm_memory_slot *slot, /* * The page is not write-tracked, mark existing shadow pages unsync - * unless KVM is synchronizing an unsync SP (can_unsync = false). In - * that case, KVM must complete emulation of the guest TLB flush before - * allowing shadow pages to become unsync (writable by the guest). + * unless KVM is synchronizing an unsync SP. In that case, KVM must + * complete emulation of the guest TLB flush before allowing shadow + * pages to become unsync (writable by the guest). */ for_each_gfn_valid_sp_with_gptes(kvm, sp, gfn) { - if (!can_unsync) + if (synchronizing) return -EPERM; if (sp->unsync) @@ -2941,7 +2941,7 @@ static int mmu_set_spte(struct kvm_vcpu *vcpu, struct kvm_memory_slot *slot, } wrprot = make_spte(vcpu, sp, slot, pte_access, gfn, pfn, *sptep, prefetch, - true, host_writable, &spte); + false, host_writable, &spte); if (*sptep == spte) { ret = RET_PF_SPURIOUS; diff --git a/arch/x86/kvm/mmu/mmu_internal.h b/arch/x86/kvm/mmu/mmu_internal.h index c98827840e07..4da83544c4e1 100644 --- a/arch/x86/kvm/mmu/mmu_internal.h +++ b/arch/x86/kvm/mmu/mmu_internal.h @@ -164,7 +164,7 @@ static inline gfn_t gfn_round_for_level(gfn_t gfn, int level) } int mmu_try_to_unsync_pages(struct kvm *kvm, const struct kvm_memory_slot *slot, - gfn_t gfn, bool can_unsync, bool prefetch); + gfn_t gfn, bool synchronizing, bool prefetch); void kvm_mmu_gfn_disallow_lpage(const struct kvm_memory_slot *slot, gfn_t gfn); void kvm_mmu_gfn_allow_lpage(const struct kvm_memory_slot *slot, gfn_t gfn); diff --git a/arch/x86/kvm/mmu/paging_tmpl.h b/arch/x86/kvm/mmu/paging_tmpl.h index ae7d39ff2d07..6e7bd8921c6f 100644 --- a/arch/x86/kvm/mmu/paging_tmpl.h +++ b/arch/x86/kvm/mmu/paging_tmpl.h @@ -963,7 +963,7 @@ static int FNAME(sync_spte)(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp, int host_writable = spte & shadow_host_writable_mask; slot = kvm_vcpu_gfn_to_memslot(vcpu, gfn); make_spte(vcpu, sp, slot, pte_access, gfn, - spte_to_pfn(spte), spte, true, false, + spte_to_pfn(spte), spte, true, true, host_writable, &spte); return mmu_spte_update(sptep, spte); diff --git a/arch/x86/kvm/mmu/spte.c b/arch/x86/kvm/mmu/spte.c index 5521608077ec..0e47fea1a2d9 100644 --- a/arch/x86/kvm/mmu/spte.c +++ b/arch/x86/kvm/mmu/spte.c @@ -157,7 +157,7 @@ bool spte_has_volatile_bits(u64 spte) bool make_spte(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp, const struct kvm_memory_slot *slot, unsigned int pte_access, gfn_t gfn, kvm_pfn_t pfn, - u64 old_spte, bool prefetch, bool can_unsync, + u64 old_spte, bool prefetch, bool synchronizing, bool host_writable, u64 *new_spte) { int level = sp->role.level; @@ -248,7 +248,7 @@ bool make_spte(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp, * e.g. it's write-tracked (upper-level SPs) or has one or more * shadow pages and unsync'ing pages is not allowed. */ - if (mmu_try_to_unsync_pages(vcpu->kvm, slot, gfn, can_unsync, prefetch)) { + if (mmu_try_to_unsync_pages(vcpu->kvm, slot, gfn, synchronizing, prefetch)) { wrprot = true; pte_access &= ~ACC_WRITE_MASK; spte &= ~(PT_WRITABLE_MASK | shadow_mmu_writable_mask); diff --git a/arch/x86/kvm/mmu/spte.h b/arch/x86/kvm/mmu/spte.h index 2cb816ea2430..c81cac9358e0 100644 --- a/arch/x86/kvm/mmu/spte.h +++ b/arch/x86/kvm/mmu/spte.h @@ -499,7 +499,7 @@ bool spte_has_volatile_bits(u64 spte); bool make_spte(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp, const struct kvm_memory_slot *slot, unsigned int pte_access, gfn_t gfn, kvm_pfn_t pfn, - u64 old_spte, bool prefetch, bool can_unsync, + u64 old_spte, bool prefetch, bool synchronizing, bool host_writable, u64 *new_spte); u64 make_huge_page_split_spte(struct kvm *kvm, u64 huge_spte, union kvm_mmu_page_role role, int index); diff --git a/arch/x86/kvm/mmu/tdp_mmu.c b/arch/x86/kvm/mmu/tdp_mmu.c index 3c6583468742..76bca7a726c1 100644 --- a/arch/x86/kvm/mmu/tdp_mmu.c +++ b/arch/x86/kvm/mmu/tdp_mmu.c @@ -1033,8 +1033,8 @@ static int tdp_mmu_map_handle_target_level(struct kvm_vcpu *vcpu, new_spte = make_mmio_spte(vcpu, iter->gfn, ACC_ALL); else wrprot = make_spte(vcpu, sp, fault->slot, ACC_ALL, iter->gfn, - fault->pfn, iter->old_spte, fault->prefetch, true, - fault->map_writable, &new_spte); + fault->pfn, iter->old_spte, fault->prefetch, + false, fault->map_writable, &new_spte); if (new_spte == iter->old_spte) ret = RET_PF_SPURIOUS; From patchwork Thu Oct 10 18:23:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831005 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 70442D24447 for ; Thu, 10 Oct 2024 19:46:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Type:Cc:To: From:Subject:Message-ID:References:Mime-Version:In-Reply-To:Date: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=GZoCfd92gqYHwqsSYOai9KQfQ7K6dxh+JWKccmiJTPU=; b=2dHgaAWDSC6N914CukRQDypX8z jv2+bYRZAcllexKWEV5tR7/pjtehkYwFyI02iv5yxu+FE1xctwTQVdZAUM0SIerkXdTk7myBhukR4 DY2MzQN83NANxjMbIimLd82yvZb6T7SPRWZRfaZ02GjcomnCrvajH+t2OhfPy+Ld3f6pwt7DnYExQ amXoMmB4O7u/i2xeNWaaJylPJNOFq/6cxSe7CO17SSOElEwgFd4kCrj8o0oL2fPcWMCT5lSzMfH3i AmjND4UVXK0N54Pcell6NDDDNu3KW+hpKtnMje0MUFCKSZFFnT0OrBErlzITEMthx14mP+LUr7GEP BFLWW96A==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syz75-0000000EAjG-1TCC; Thu, 10 Oct 2024 19:46:19 +0000 Received: from mail-pj1-x104a.google.com ([2607:f8b0:4864:20::104a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxqV-0000000DpmH-3CVR for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:25:09 +0000 Received: by mail-pj1-x104a.google.com with SMTP id 98e67ed59e1d1-2e2ca403f5dso1037019a91.1 for ; Thu, 10 Oct 2024 11:25:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584707; x=1729189507; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=GZoCfd92gqYHwqsSYOai9KQfQ7K6dxh+JWKccmiJTPU=; b=h8kgmq5KMhiBhtsbuTJw6kU2LJ3f2iQP13uN81By+PdbuW2nwg9GE7cWKJK4SIKxe9 aqQXNd0DGyY4m+d5ZAQ6AwjC5GPE0e8hdo5ZQn8POJbht1P2W0EOFPEf/SqHVj9VIEPG mTauj/V2pKKa8GfckVKm8AicpL2PyF2yZCj6/iuFPJChYGGr7qKDGMKW02z+t4+bIiJY 2yCoRqfFPRISvLKxNhzo+zUQG2ARex/YqnovbrM3qvS88K07YwkBncJVeTp3tRusFkom BIOx4xdIwVY9lgLVDyE6HEqvbD5bvi0UkH09epNU5s/AYlbnV/veBkjfvxbSlpss22U0 qsAg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584707; x=1729189507; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=GZoCfd92gqYHwqsSYOai9KQfQ7K6dxh+JWKccmiJTPU=; b=QSUy4ZdwKDUcct1ELIcOubnNQDOzoyXzHsVOPVCCNUec3EyR0axrs/AlOWUFub6y2B W5fVPEEwzEvP9ifmasI15ZU4QWWSjkB2I/XHTUQNTYGEUp8pyg4ll4KDXDb5sx4+I+ND G/C0EzfIEy2O1ZmgF5YzcPyZ4wtgfvXnBS61MBuOFebNuyNHSx6en48doErFx7WMOLRs CCnTllGJTUHDVaSxASYIMKU9eaWgb31LBq2r9YZ2JotVkznjYlgBqq9H8rbOLMT6HFue oysEE3aPPVVhacLpaxRZR6PylMhc6drwwy/d6ZdeTD+3f6gmMnUcTjPVuTD/KBpfPBlB tTvw== X-Forwarded-Encrypted: i=1; AJvYcCWC9RvKly2T5dMVHyq3bOhcoomzbQIWGLqfjiNkDxufNSu1dz1GKpWv2yUS9f73xrP4f5uNY3Av7B3MIsAs+zBL@lists.infradead.org X-Gm-Message-State: AOJu0Yxc/GXrCOVUtU4FE/ddvc6N3A2CBIUq7aYwI6xSG/NRhJ396fil O9cyoceD6z2IhPW5YOjIzD6AuYG9Pi5jeOGOOAtiTb5PcW02opi3pLkUW4pr6+GO5HdEynUJo/5 OGA== X-Google-Smtp-Source: AGHT+IH8UDk7ruln14ZDsSWqbj1uDcx7028dIle1wpc1As3ec3XNWN4cizl8I1jAsp9rsgJdiTjtCNc1iNw= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:90a:fa84:b0:2e2:da81:40c6 with SMTP id 98e67ed59e1d1-2e2f0a524d4mr44a91.2.1728584705592; Thu, 10 Oct 2024 11:25:05 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:09 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-8-seanjc@google.com> Subject: [PATCH v13 07/85] KVM: x86/mmu: Mark new SPTE as Accessed when synchronizing existing SPTE From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112507_927927_E9B4904F X-CRM114-Status: GOOD ( 14.52 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Set the Accessed bit when making a "new" SPTE during SPTE synchronization, as _clearing_ the Accessed bit is counter-productive, and even if the Accessed bit wasn't set in the old SPTE, odds are very good the guest will access the page in the near future, as the most common case where KVM synchronizes a shadow-present SPTE is when the guest is making the gPTE read-only for Copy-on-Write (CoW). Preserving the Accessed bit will allow dropping the logic that propagates the Accessed bit to the underlying struct page when overwriting an existing SPTE, without undue risk of regressing page aging. Note, KVM's current behavior is very deliberate, as SPTE synchronization was the only "speculative" access type as of commit 947da5383069 ("KVM: MMU: Set the accessed bit on non-speculative shadow ptes"). But, much has changed since 2008, and more changes are on the horizon. Spurious clearing of the Accessed (and Dirty) was mitigated by commit e6722d9211b2 ("KVM: x86/mmu: Reduce the update to the spte in FNAME(sync_spte)"), which changed FNAME(sync_spte) to only overwrite SPTEs if the protections are actually changing. I.e. KVM is already preserving Accessed information for SPTEs that aren't dropping protections. And with the aforementioned future change to NOT mark the page/folio as accessed, KVM's SPTEs will become the "source of truth" so to speak, in which case clearing the Accessed bit outside of page aging becomes very undesirable. Suggested-by: Yan Zhao Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/spte.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/arch/x86/kvm/mmu/spte.c b/arch/x86/kvm/mmu/spte.c index 0e47fea1a2d9..618059b30b8b 100644 --- a/arch/x86/kvm/mmu/spte.c +++ b/arch/x86/kvm/mmu/spte.c @@ -178,7 +178,7 @@ bool make_spte(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp, spte |= SPTE_TDP_AD_WRPROT_ONLY; spte |= shadow_present_mask; - if (!prefetch) + if (!prefetch || synchronizing) spte |= spte_shadow_accessed_mask(spte); /* @@ -259,7 +259,7 @@ bool make_spte(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp, spte |= spte_shadow_dirty_mask(spte); out: - if (prefetch) + if (prefetch && !synchronizing) spte = mark_spte_for_access_track(spte); WARN_ONCE(is_rsvd_spte(&vcpu->arch.mmu->shadow_zero_check, spte, level), From patchwork Thu Oct 10 18:23:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830848 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 91B8CCFC5EC for ; Thu, 10 Oct 2024 18:43:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:Content-Type:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=e3d7DBw/sg5cuh+IKhcM0BkpGN0EJ0d/2DHG844pD6k=; b=zIVe+a/+ScFQDH 0GMOPqkKVK9fPHcH8nXkxcwB+gmLd1Ii1D1zGedhr1E1SA3C6b4+qzASG1H1oUZQs+qnIGSiRnRuq acyBPq7eigssQFvwehA9PmPXglQXczaMD9eHvNdVQEiZDOWtCw+q3B1N4rtwlmFndnVMdhoCp2sPg rIPZA64enWaWghUeTrzdNVyjYKEuHnygUXL4QdMfvLkNt17WAr5srEtqxfcwpYQqWyr8l7JHn/m4X bHe3kcymZtc2MQWX63hvSSyaQSmY7iNbs320ceofHrYJl2npVVZ28YQsXuxxILPNbK776YnO4Z/E6 izom4shnihAFYsSvoIyA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syy81-0000000Dxau-0TB9; Thu, 10 Oct 2024 18:43:13 +0000 Received: from mail-pj1-x104a.google.com ([2607:f8b0:4864:20::104a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxqY-0000000Dpoc-45m6 for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:25:14 +0000 Received: by mail-pj1-x104a.google.com with SMTP id 98e67ed59e1d1-2e2eba0efc6so171329a91.3 for ; Thu, 10 Oct 2024 11:25:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584709; x=1729189509; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=e3d7DBw/sg5cuh+IKhcM0BkpGN0EJ0d/2DHG844pD6k=; b=hYflAaWqfkt/9sjlBCfgQhFqXMVl+3HoIOZuTqZBbZVlTAMz/0CpZvy1EZ+Xt5U4Vk l3zYswVa/pShqe5WMq/UNsp7ryiz3zV1sn09NLFFufXCrWgkQS652k3DpQJX4VjkxCmw f/tHxYugYCdPNcfMSFwFC6865MJNcPOMzPRfnym2tadGXORr9qCJV/KqXgLE5MpBA/aA 7soIPZ+JxQvzUIgQhY/STcnxBzIl/cT2FagOECDlV+N7zK+0n1/lzQF+/Hv80tKiESJ6 fucd/zt5i39oJGbDFBQH6RdIvBQtZD0pVk7EQrmh8wU9eSrwNsW/5H/j6r3BZvIfa2hZ DjCA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584709; x=1729189509; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=e3d7DBw/sg5cuh+IKhcM0BkpGN0EJ0d/2DHG844pD6k=; b=xT0oE2igggSeBEU/XcOrbrDsbDOyBlnLtWXg9Y7tb4L+WE+cqvGH5EHloxLum5Of/F ZIybENbu4Xqo/jfo/jwAdTTSVyCDH81E6ReZ6ZDwEzte/RXg0r6EtOtPMHJ/zbDk5ZRm wbeY/dSEorizmwfwItIYQaG4oVSV3eotbI3HlqMJc533VrHgtSyR6drk2F1wo07fFo9Y rom55qgMd4X7WVzJR53YcXuDAfkEqtbBdEe2XOzVzOqVoYTBca6DVQtEEB0OqHr0U1O1 7QTtHIFIzza9nNrYw2JtLpk3fgLfaZCJ4Y2o0N+PnEbgzI5GO29Ax8R6ErC35yxbKh37 a7yQ== X-Forwarded-Encrypted: i=1; AJvYcCVAolLvwcHaUhp8b3egfIKe+aqXss9z55422rUWxaa3h3CuyNDFXNYbTSECb2kkn2e9UcBYuGpxdRBnamcjzBNV@lists.infradead.org X-Gm-Message-State: AOJu0YwEOSQfAZ/ADQCYdr/p6NoSGwNEddXHi/NKuvHeuI9I/jkXTEbW hTIN2gFVax/spL9LifsqVSPsF1VGcOVVKb3yoFSjulyZK7REWV8yYnRV8/JMeujc7WE/m/N1b5z /Mw== X-Google-Smtp-Source: AGHT+IFvOj1YmixVIm2giqrxikolLV70YR7/zWjxlJNmhfeSXdlwSACqKd64r4Is1alzH3q275ZQuugRXZo= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:90b:153:b0:2e2:af66:c33e with SMTP id 98e67ed59e1d1-2e2f0ae73f3mr37a91.1.1728584708343; Thu, 10 Oct 2024 11:25:08 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:10 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-9-seanjc@google.com> Subject: [PATCH v13 08/85] KVM: x86/mmu: Mark folio dirty when creating SPTE, not when zapping/modifying From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112511_135134_250BCF99 X-CRM114-Status: GOOD ( 30.06 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Mark pages/folios dirty when creating SPTEs to map PFNs into the guest, not when zapping or modifying SPTEs, as marking folios dirty when zapping or modifying SPTEs can be extremely inefficient. E.g. when KVM is zapping collapsible SPTEs to reconstitute a hugepage after disbling dirty logging, KVM will mark every 4KiB pfn as dirty, even though _at least_ 512 pfns are guaranteed to be in a single folio (the SPTE couldn't potentially be huge if that weren't the case). The problem only becomes worse for 1GiB HugeTLB pages, as KVM can mark a single folio dirty 512*512 times. Marking a folio dirty when mapping is functionally safe as KVM drops all relevant SPTEs in response to an mmu_notifier invalidation, i.e. ensures that the guest can't dirty a folio after access has been removed. And because KVM already marks folios dirty when zapping/modifying SPTEs for KVM reasons, i.e. not in response to an mmu_notifier invalidation, there is no danger of "prematurely" marking a folio dirty. E.g. if a filesystems cleans a folio without first removing write access, then there already exists races where KVM could mark a folio dirty before remote TLBs are flushed, i.e. before guest writes are guaranteed to stop. Furthermore, x86 is literally the only architecture that marks folios dirty on the backend; every other KVM architecture marks folios dirty at map time. x86's unique behavior likely stems from the fact that x86's MMU predates mmu_notifiers. Long, long ago, before mmu_notifiers were added, marking pages dirty when zapping SPTEs was logical, and perhaps even necessary, as KVM held references to pages, i.e. kept a page's refcount elevated while the page was mapped into the guest. At the time, KVM's rmap_remove() simply did: if (is_writeble_pte(*spte)) kvm_release_pfn_dirty(pfn); else kvm_release_pfn_clean(pfn); i.e. dropped the refcount and marked the page dirty at the same time. After mmu_notifiers were introduced, commit acb66dd051d0 ("KVM: MMU: don't hold pagecount reference for mapped sptes pages") removed the refcount logic, but kept the dirty logic, i.e. converted the above to: if (is_writeble_pte(*spte)) kvm_release_pfn_dirty(pfn); And for KVM x86, that's essentially how things have stayed over the last ~15 years, without anyone revisiting *why* KVM marks pages/folios dirty at zap/modification time, e.g. the behavior was blindly carried forward to the TDP MMU. Practically speaking, the only downside to marking a folio dirty during mapping is that KVM could trigger writeback of memory that was never actually written. Except that can't actually happen if KVM marks folios dirty if and only if a writable SPTE is created (as done here), because KVM always marks writable SPTEs as dirty during make_spte(). See commit 9b51a63024bd ("KVM: MMU: Explicitly set D-bit for writable spte."), circa 2015. Note, KVM's access tracking logic for prefetched SPTEs is a bit odd. If a guest PTE is dirty and writable, KVM will create a writable SPTE, but then mark the SPTE for access tracking. Which isn't wrong, just a bit odd, as it results in _more_ precise dirty tracking for MMUs _without_ A/D bits. To keep things simple, mark the folio dirty before access tracking comes into play, as an access-tracked SPTE can be restored in the fast page fault path, i.e. without holding mmu_lock. While writing SPTEs and accessing memslots outside of mmu_lock is safe, marking a folio dirty is not. E.g. if the fast path gets interrupted _just_ after setting a SPTE, the primary MMU could theoretically invalidate and free a folio before KVM marks it dirty. Unlike the shadow MMU, which waits for CPUs to respond to an IPI, the TDP MMU only guarantees the page tables themselves won't be freed (via RCU). Opportunistically update a few stale comments. Cc: David Matlack Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/mmu.c | 30 ++++-------------------------- arch/x86/kvm/mmu/paging_tmpl.h | 6 +++--- arch/x86/kvm/mmu/spte.c | 20 ++++++++++++++++++-- arch/x86/kvm/mmu/tdp_mmu.c | 12 ------------ 4 files changed, 25 insertions(+), 43 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 0f21d6f76cab..1ae823ebd12b 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -547,10 +547,8 @@ static bool mmu_spte_update(u64 *sptep, u64 new_spte) kvm_set_pfn_accessed(spte_to_pfn(old_spte)); } - if (is_dirty_spte(old_spte) && !is_dirty_spte(new_spte)) { + if (is_dirty_spte(old_spte) && !is_dirty_spte(new_spte)) flush = true; - kvm_set_pfn_dirty(spte_to_pfn(old_spte)); - } return flush; } @@ -593,9 +591,6 @@ static u64 mmu_spte_clear_track_bits(struct kvm *kvm, u64 *sptep) if (is_accessed_spte(old_spte)) kvm_set_pfn_accessed(pfn); - if (is_dirty_spte(old_spte)) - kvm_set_pfn_dirty(pfn); - return old_spte; } @@ -1250,16 +1245,6 @@ static bool spte_clear_dirty(u64 *sptep) return mmu_spte_update(sptep, spte); } -static bool spte_wrprot_for_clear_dirty(u64 *sptep) -{ - bool was_writable = test_and_clear_bit(PT_WRITABLE_SHIFT, - (unsigned long *)sptep); - if (was_writable && !spte_ad_enabled(*sptep)) - kvm_set_pfn_dirty(spte_to_pfn(*sptep)); - - return was_writable; -} - /* * Gets the GFN ready for another round of dirty logging by clearing the * - D bit on ad-enabled SPTEs, and @@ -1275,7 +1260,8 @@ static bool __rmap_clear_dirty(struct kvm *kvm, struct kvm_rmap_head *rmap_head, for_each_rmap_spte(rmap_head, &iter, sptep) if (spte_ad_need_write_protect(*sptep)) - flush |= spte_wrprot_for_clear_dirty(sptep); + flush |= test_and_clear_bit(PT_WRITABLE_SHIFT, + (unsigned long *)sptep); else flush |= spte_clear_dirty(sptep); @@ -1628,14 +1614,6 @@ static bool kvm_rmap_age_gfn_range(struct kvm *kvm, clear_bit((ffs(shadow_accessed_mask) - 1), (unsigned long *)sptep); } else { - /* - * Capture the dirty status of the page, so that - * it doesn't get lost when the SPTE is marked - * for access tracking. - */ - if (is_writable_pte(spte)) - kvm_set_pfn_dirty(spte_to_pfn(spte)); - spte = mark_spte_for_access_track(spte); mmu_spte_update_no_track(sptep, spte); } @@ -3415,7 +3393,7 @@ static bool fast_pf_fix_direct_spte(struct kvm_vcpu *vcpu, * harm. This also avoids the TLB flush needed after setting dirty bit * so non-PML cases won't be impacted. * - * Compare with set_spte where instead shadow_dirty_mask is set. + * Compare with make_spte() where instead shadow_dirty_mask is set. */ if (!try_cmpxchg64(sptep, &old_spte, new_spte)) return false; diff --git a/arch/x86/kvm/mmu/paging_tmpl.h b/arch/x86/kvm/mmu/paging_tmpl.h index 6e7bd8921c6f..fbaae040218b 100644 --- a/arch/x86/kvm/mmu/paging_tmpl.h +++ b/arch/x86/kvm/mmu/paging_tmpl.h @@ -892,9 +892,9 @@ static gpa_t FNAME(gva_to_gpa)(struct kvm_vcpu *vcpu, struct kvm_mmu *mmu, /* * Using the information in sp->shadowed_translation (kvm_mmu_page_get_gfn()) is - * safe because: - * - The spte has a reference to the struct page, so the pfn for a given gfn - * can't change unless all sptes pointing to it are nuked first. + * safe because SPTEs are protected by mmu_notifiers and memslot generations, so + * the pfn for a given gfn can't change unless all SPTEs pointing to the gfn are + * nuked first. * * Returns * < 0: failed to sync spte diff --git a/arch/x86/kvm/mmu/spte.c b/arch/x86/kvm/mmu/spte.c index 618059b30b8b..8e8d6ee79c8b 100644 --- a/arch/x86/kvm/mmu/spte.c +++ b/arch/x86/kvm/mmu/spte.c @@ -232,8 +232,8 @@ bool make_spte(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp, * unnecessary (and expensive). * * The same reasoning applies to dirty page/folio accounting; - * KVM will mark the folio dirty using the old SPTE, thus - * there's no need to immediately mark the new SPTE as dirty. + * KVM marked the folio dirty when the old SPTE was created, + * thus there's no need to mark the folio dirty again. * * Note, both cases rely on KVM not changing PFNs without first * zapping the old SPTE, which is guaranteed by both the shadow @@ -266,12 +266,28 @@ bool make_spte(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp, "spte = 0x%llx, level = %d, rsvd bits = 0x%llx", spte, level, get_rsvd_bits(&vcpu->arch.mmu->shadow_zero_check, spte, level)); + /* + * Mark the memslot dirty *after* modifying it for access tracking. + * Unlike folios, memslots can be safely marked dirty out of mmu_lock, + * i.e. in the fast page fault handler. + */ if ((spte & PT_WRITABLE_MASK) && kvm_slot_dirty_track_enabled(slot)) { /* Enforced by kvm_mmu_hugepage_adjust. */ WARN_ON_ONCE(level > PG_LEVEL_4K); mark_page_dirty_in_slot(vcpu->kvm, slot, gfn); } + /* + * If the page that KVM got from the primary MMU is writable, i.e. if + * it's host-writable, mark the page/folio dirty. As alluded to above, + * folios can't be safely marked dirty in the fast page fault handler, + * and so KVM must (somewhat) speculatively mark the folio dirty even + * though it isn't guaranteed to be written as KVM won't mark the folio + * dirty if/when the SPTE is made writable. + */ + if (host_writable) + kvm_set_pfn_dirty(pfn); + *new_spte = spte; return wrprot; } diff --git a/arch/x86/kvm/mmu/tdp_mmu.c b/arch/x86/kvm/mmu/tdp_mmu.c index 76bca7a726c1..517b384473c1 100644 --- a/arch/x86/kvm/mmu/tdp_mmu.c +++ b/arch/x86/kvm/mmu/tdp_mmu.c @@ -511,10 +511,6 @@ static void handle_changed_spte(struct kvm *kvm, int as_id, gfn_t gfn, if (is_leaf != was_leaf) kvm_update_page_stats(kvm, level, is_leaf ? 1 : -1); - if (was_leaf && is_dirty_spte(old_spte) && - (!is_present || !is_dirty_spte(new_spte) || pfn_changed)) - kvm_set_pfn_dirty(spte_to_pfn(old_spte)); - /* * Recursively handle child PTs if the change removed a subtree from * the paging structure. Note the WARN on the PFN changing without the @@ -1249,13 +1245,6 @@ static bool age_gfn_range(struct kvm *kvm, struct tdp_iter *iter, iter->level); new_spte = iter->old_spte & ~shadow_accessed_mask; } else { - /* - * Capture the dirty status of the page, so that it doesn't get - * lost when the SPTE is marked for access tracking. - */ - if (is_writable_pte(iter->old_spte)) - kvm_set_pfn_dirty(spte_to_pfn(iter->old_spte)); - new_spte = mark_spte_for_access_track(iter->old_spte); iter->old_spte = kvm_tdp_mmu_write_spte(iter->sptep, iter->old_spte, new_spte, @@ -1596,7 +1585,6 @@ static void clear_dirty_pt_masked(struct kvm *kvm, struct kvm_mmu_page *root, trace_kvm_tdp_mmu_spte_changed(iter.as_id, iter.gfn, iter.level, iter.old_spte, iter.old_spte & ~dbit); - kvm_set_pfn_dirty(spte_to_pfn(iter.old_spte)); } rcu_read_unlock(); From patchwork Thu Oct 10 18:23:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831019 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E4D7BD24442 for ; Thu, 10 Oct 2024 19:59:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:Content-Type:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=0ix/Xs4/Qn/2DvOT3Bh000Q598gzWwOGIiO61AJ4Bto=; b=ec00OFV/jgFZL4 NIxA7iQFjFTiXKwKPSJrqWQyhNHAnuhxQvPwr5TlyjR7gyWUvMZMjavbRpk7MQYSjo0Cq0IY2ePf8 3CS6f7inxkAjSvyuLbDsNkzGiCxiQyb5xch74TAU0c7O375oK57P4ydbW3a+pYsVESJZM/s7MgzyK BYc9S8iBxmNgMWBDNNzyQhj7tGDaWnInr2DQFAYQx06epGmEamxKPAnadBLpbnUXFhVzAkXlzVv8d vTDpD78tRm16muu4/5w2+wA30mIz13WAW0pPykd4aPbLCs/e+p3csKnMIVPrpUzdbXQKcMl79pr+W z2lg5iGs6kXTtWzc+OXw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syzJl-0000000EDKt-0Z7H; Thu, 10 Oct 2024 19:59:25 +0000 Received: from mail-yw1-x114a.google.com ([2607:f8b0:4864:20::114a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxqa-0000000Dppy-2lZt for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:25:16 +0000 Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-6e2b049b64aso20288117b3.3 for ; Thu, 10 Oct 2024 11:25:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584711; x=1729189511; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=0ix/Xs4/Qn/2DvOT3Bh000Q598gzWwOGIiO61AJ4Bto=; b=wg6l/e68EEItXXcfocqTbRUpJ/oGTThYQm/2n+G9Mw+E0+rgd/O8ALeeTI0zso3low A4IuPTpimz0NQuRMW8Uj3CXmwznPneCME+/ePXkyHtXW57BceBMbIX5zxGj2TGtYC1iA LVFdYAaz40Ujvh2KG/1U9Je1xNngtGgYPesb2oS8XSfpalOoYSptDzyZkgRIHj1XxtS7 jfP03JdayC4nQ8fEyV8m4KHGF+R0vtK1thWL3EAbusKhT/mQEQYcHogc/b3dLaeVXhMW KAfRABQIpN8xmuoHrALjhVLDeEQTIydOsBSvlnSmtEpSpnJBoskxtDWeYkSylb8yh/GR bXCw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584711; x=1729189511; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=0ix/Xs4/Qn/2DvOT3Bh000Q598gzWwOGIiO61AJ4Bto=; b=YM2ODS8l4Fw/8WUxsmzHdwRfwHzq16S/mP16LmKie6o1+wbPqo1FPjZEvpWpLc0noI 4v4RFBwK+6gQ8xRjOAFazLtPxVLxyTyxJMlRXLL41FJG2lHvu4oYMN5PjjNIj+jeOc3/ +at3B0Q6KivnbZ8Tc9R/xguRuZXPHKZhwBy4Y6A0UTyFZnTEhXnuGFyFEd40whIrcHAe xbCdnSAwPyI4SUpu6u5aZPD8QiliHbNg/X6YMxikc10TCQI2Aj6tZYyVfN+WWSyPD1Mv XyK89Xk0xi97ZWGBdN+IYTK/gnOgkoOgW+rJ+KF03i/BAkv+/fPTfgvWh6wZNWB9g8oK tnzA== X-Forwarded-Encrypted: i=1; AJvYcCWszrNo4bx1925odLrq8BESr4yiLXBQN+hPr7mu4e5aYCwHnIPsphlR1cAoqYr8AN1Qd/FUCwmCE21SIPuC9xxA@lists.infradead.org X-Gm-Message-State: AOJu0YxytByHaYeB0kmdva/It1LbRcigPak1IqE7lZ5JRym/DS/rAZKN ru+uLpP6iD2hnmYGdZyBdT2S3ZYdsCbM61s40L6xs4uKNeDe/p8vR3p+ov/nT673N5U14oHDckQ YoA== X-Google-Smtp-Source: AGHT+IGD2TfP2dloLIlDYfRlI1lvmWPD7YUtu8EYW3dxRXY7JxJbZQIAJnuRF8jcBfCYGhk25T1qJs7AAuY= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:690c:5b41:b0:6db:c6ac:62a0 with SMTP id 00721157ae682-6e322305467mr243067b3.5.1728584711093; Thu, 10 Oct 2024 11:25:11 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:11 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-10-seanjc@google.com> Subject: [PATCH v13 09/85] KVM: x86/mmu: Mark page/folio accessed only when zapping leaf SPTEs From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112512_758569_83BE59A1 X-CRM114-Status: GOOD ( 18.93 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Now that KVM doesn't clobber Accessed bits of shadow-present SPTEs, e.g. when prefetching, mark folios as accessed only when zapping leaf SPTEs, which is a rough heuristic for "only in response to an mmu_notifier invalidation". Page aging and LRUs are tolerant of false negatives, i.e. KVM doesn't need to be precise for correctness, and re-marking folios as accessed when zapping entire roots or when zapping collapsible SPTEs is expensive and adds very little value. E.g. when a VM is dying, all of its memory is being freed; marking folios accessed at that time provides no known value. Similarly, because KVM marks folios as accessed when creating SPTEs, marking all folios as accessed when userspace happens to delete a memslot doesn't add value. The folio was marked access when the old SPTE was created, and will be marked accessed yet again if a vCPU accesses the pfn again after reloading a new root. Zapping collapsible SPTEs is a similar story; marking folios accessed just because userspace disable dirty logging is a side effect of KVM behavior, not a deliberate goal. As an intermediate step, a.k.a. bisection point, towards *never* marking folios accessed when dropping SPTEs, mark folios accessed when the primary MMU might be invalidating mappings, as such zappings are not KVM initiated, i.e. might actually be related to page aging and LRU activity. Note, x86 is the only KVM architecture that "double dips"; every other arch marks pfns as accessed only when mapping into the guest, not when mapping into the guest _and_ when removing from the guest. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- Documentation/virt/kvm/locking.rst | 76 +++++++++++++++--------------- arch/x86/kvm/mmu/mmu.c | 4 +- arch/x86/kvm/mmu/tdp_mmu.c | 7 ++- 3 files changed, 43 insertions(+), 44 deletions(-) diff --git a/Documentation/virt/kvm/locking.rst b/Documentation/virt/kvm/locking.rst index 20a9a37d1cdd..3d8bf40ca448 100644 --- a/Documentation/virt/kvm/locking.rst +++ b/Documentation/virt/kvm/locking.rst @@ -147,49 +147,51 @@ Then, we can ensure the dirty bitmaps is correctly set for a gfn. 2) Dirty bit tracking -In the origin code, the spte can be fast updated (non-atomically) if the +In the original code, the spte can be fast updated (non-atomically) if the spte is read-only and the Accessed bit has already been set since the Accessed bit and Dirty bit can not be lost. But it is not true after fast page fault since the spte can be marked writable between reading spte and updating spte. Like below case: -+------------------------------------------------------------------------+ -| At the beginning:: | -| | -| spte.W = 0 | -| spte.Accessed = 1 | -+------------------------------------+-----------------------------------+ -| CPU 0: | CPU 1: | -+------------------------------------+-----------------------------------+ -| In mmu_spte_clear_track_bits():: | | -| | | -| old_spte = *spte; | | -| | | -| | | -| /* 'if' condition is satisfied. */| | -| if (old_spte.Accessed == 1 && | | -| old_spte.W == 0) | | -| spte = 0ull; | | -+------------------------------------+-----------------------------------+ -| | on fast page fault path:: | -| | | -| | spte.W = 1 | -| | | -| | memory write on the spte:: | -| | | -| | spte.Dirty = 1 | -+------------------------------------+-----------------------------------+ -| :: | | -| | | -| else | | -| old_spte = xchg(spte, 0ull) | | -| if (old_spte.Accessed == 1) | | -| kvm_set_pfn_accessed(spte.pfn);| | -| if (old_spte.Dirty == 1) | | -| kvm_set_pfn_dirty(spte.pfn); | | -| OOPS!!! | | -+------------------------------------+-----------------------------------+ ++-------------------------------------------------------------------------+ +| At the beginning:: | +| | +| spte.W = 0 | +| spte.Accessed = 1 | ++-------------------------------------+-----------------------------------+ +| CPU 0: | CPU 1: | ++-------------------------------------+-----------------------------------+ +| In mmu_spte_update():: | | +| | | +| old_spte = *spte; | | +| | | +| | | +| /* 'if' condition is satisfied. */ | | +| if (old_spte.Accessed == 1 && | | +| old_spte.W == 0) | | +| spte = new_spte; | | ++-------------------------------------+-----------------------------------+ +| | on fast page fault path:: | +| | | +| | spte.W = 1 | +| | | +| | memory write on the spte:: | +| | | +| | spte.Dirty = 1 | ++-------------------------------------+-----------------------------------+ +| :: | | +| | | +| else | | +| old_spte = xchg(spte, new_spte);| | +| if (old_spte.Accessed && | | +| !new_spte.Accessed) | | +| flush = true; | | +| if (old_spte.Dirty && | | +| !new_spte.Dirty) | | +| flush = true; | | +| OOPS!!! | | ++-------------------------------------+-----------------------------------+ The Dirty bit is lost in this case. diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 1ae823ebd12b..04228a7da69a 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -542,10 +542,8 @@ static bool mmu_spte_update(u64 *sptep, u64 new_spte) * to guarantee consistency between TLB and page tables. */ - if (is_accessed_spte(old_spte) && !is_accessed_spte(new_spte)) { + if (is_accessed_spte(old_spte) && !is_accessed_spte(new_spte)) flush = true; - kvm_set_pfn_accessed(spte_to_pfn(old_spte)); - } if (is_dirty_spte(old_spte) && !is_dirty_spte(new_spte)) flush = true; diff --git a/arch/x86/kvm/mmu/tdp_mmu.c b/arch/x86/kvm/mmu/tdp_mmu.c index 517b384473c1..8aa0d7a7602b 100644 --- a/arch/x86/kvm/mmu/tdp_mmu.c +++ b/arch/x86/kvm/mmu/tdp_mmu.c @@ -520,10 +520,6 @@ static void handle_changed_spte(struct kvm *kvm, int as_id, gfn_t gfn, if (was_present && !was_leaf && (is_leaf || !is_present || WARN_ON_ONCE(pfn_changed))) handle_removed_pt(kvm, spte_to_child_pt(old_spte, level), shared); - - if (was_leaf && is_accessed_spte(old_spte) && - (!is_present || !is_accessed_spte(new_spte) || pfn_changed)) - kvm_set_pfn_accessed(spte_to_pfn(old_spte)); } static inline int __must_check __tdp_mmu_set_spte_atomic(struct tdp_iter *iter, @@ -865,6 +861,9 @@ static bool tdp_mmu_zap_leafs(struct kvm *kvm, struct kvm_mmu_page *root, tdp_mmu_iter_set_spte(kvm, &iter, SHADOW_NONPRESENT_VALUE); + if (is_accessed_spte(iter.old_spte)) + kvm_set_pfn_accessed(spte_to_pfn(iter.old_spte)); + /* * Zappings SPTEs in invalid roots doesn't require a TLB flush, * see kvm_tdp_mmu_zap_invalidated_roots() for details. From patchwork Thu Oct 10 18:23:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831155 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 60625D24455 for ; Thu, 10 Oct 2024 21:16:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:Content-Type:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=k3/xcOX0JmpZSdxUAeDWhRU0v4mqVA9cM67JfDYoNsA=; b=A/MLAD2DUuatCr /YevtBfVj966HugWDzylmzm/fnPgJFA0/AJ6UfBztHvfbaxQqd1uU99mC4ysmkOvyt7EcogS80J1z hojgceIljejrDNXd62weVlqfVdlX8bWQb9zBB4u02DtZZTHuq7b5Mh39EaIeT6NOhFeoNJL0LMejp FAPgdTMfLfxQDlQQxku0UWAVDnI3lNzuVRlIDp0UySf1snlbzsWFpP8Wcd/cVULbcgJwdbaL7XTgM 1HjVdhlPoWmd2nJ7K+Qp4HYhemW+lTk2z3yrM/ud257T69jIF8hy8qCH+/kzVo2dV41hgXtBqABp4 aVn1QtSZyXMXrzIcF6tg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1sz0WJ-0000000ENWX-0gSC; Thu, 10 Oct 2024 21:16:27 +0000 Received: from desiato.infradead.org ([2001:8b0:10b:1:d65d:64ff:fe57:4e05]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxqm-0000000Dpyw-04W6 for linux-arm-kernel@bombadil.infradead.org; Thu, 10 Oct 2024 18:25:24 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Content-Transfer-Encoding:Content-Type :Cc:To:From:Subject:Message-ID:References:Mime-Version:In-Reply-To:Date: Reply-To:Sender:Content-ID:Content-Description; bh=k3/xcOX0JmpZSdxUAeDWhRU0v4mqVA9cM67JfDYoNsA=; b=e4kC1GsnMc3fVWx150TWZkk4mJ W9GNOR/3K/bSrB94o8PL5T2UF2z018oVq2U9wburO+xK0rj9KecRNrOr51jKS8cees/SQmWs2coEj 7crRYNlk34ft5lpW5zYx/MnM6XBac6uKJh18ThElAGKnMITSu8ZIa8Gh5cMsRsIIIxblVx3UigMYs iAbVZTmnMYnL2Y6fMigROyCInmASG71VBBNm6gAAO0cFKJesKDIXnoCFyxTAJdHYyD0qCpj+8oei8 LTGP04cXknc8RAep1genntbr4rvg1hxzEVqdvHjwnBgnVDFVjDNe1fqQbPBXfZyaH0xArGfw0QFIR I4LJ7ATQ==; Received: from mail-pj1-x1049.google.com ([2607:f8b0:4864:20::1049]) by desiato.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxqh-00000005UVL-1Ga9 for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:25:22 +0000 Received: by mail-pj1-x1049.google.com with SMTP id 98e67ed59e1d1-2e2a9577037so1354367a91.1 for ; Thu, 10 Oct 2024 11:25:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584713; x=1729189513; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=k3/xcOX0JmpZSdxUAeDWhRU0v4mqVA9cM67JfDYoNsA=; b=25I4FmM9q8/zakq/tajBcDwOrvaX/2yJF+noHaaqm7+nhtCEEA/vHiU69MABElw42H B0+z7J4m0eAxHcEysEV5vjKCvS07QkxRcPK+VkmZpmuUDp6NfwO3Hvjtsgw809Xkf7cK xEWsOB6J1lk8cjv54hI8ebnqvdiE0V8pyzSHiEy5o33n4d83JNM2vXUxpVVLfIIBy2Gk vnelAIZWHr1D4pc4Ntp/6Ckp76P4LiMNNaMPcqx8leW2ZC1NVk0dvQPNTg2yv+KCnTwe kjkpA8Kzuy4XL/IX5EnjFjgQ3ax4KyRzS75vIqG1SBagCjy18+WtNgYR9ZX90VOYzF7L 2vtA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584713; x=1729189513; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=k3/xcOX0JmpZSdxUAeDWhRU0v4mqVA9cM67JfDYoNsA=; b=Bc+cdr+qY4bSow+iUI3FP2sDafjySebZzEnhYeS5rMtKnWyMuyZxRkkn5FHtvCUBsY KV1FTeX+EkAh0LGbQSlzzyqzZHYmvtHQGlwmDIOwrn7dpIl7/HO43sQWgbtV5RY+987w FO9osKujVlbvX7jPbEK2OeOWc3TBJxjC/BxR/si94tx92nuaFiwNAc6GtLoQVaYdU5NH l5aGdAPlT0t76DYRXv5X/HcmCITNxQk254EsoFcz0zPULZxFYxMO5AeStCXVfqQT+1+b BNtKH3CCaK5DtQFzOV+c5M3iYcAo4ypwk+D2mDyITpUbyYiaiwd/CdPqVKu2djl5/ryk MiZg== X-Forwarded-Encrypted: i=1; AJvYcCXt+dtIwA2keRsiqO+EKarBm0SzuWRy7a72rG2LiSzVnfs2gc7QzAaKVxa537wMEsanMT3gGmqKDCqP1DVk7evb@lists.infradead.org X-Gm-Message-State: AOJu0YyL+83AsAoHTjoykJ4ivsj7BvLJavZpj9Jl0Sw0obtYVwRIcMgj K8eAas73I9jikjFFoCRZW7njpzn2uFefTAiqqZ5IPUc+OZqNEE1gwva37KL8aJxTJMqeD1ap0o5 ajQ== X-Google-Smtp-Source: AGHT+IEzZdX7U43FtBqfYwOSCvhQrFRs7MoNj08p1am6l1FbD4QaVhgMc/3KfB78OlyTOzXtpjGPG3KIosU= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:90a:ea0e:b0:2e2:d843:4880 with SMTP id 98e67ed59e1d1-2e2f0c82a2fmr30a91.5.1728584713017; Thu, 10 Oct 2024 11:25:13 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:12 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-11-seanjc@google.com> Subject: [PATCH v13 10/85] KVM: x86/mmu: Use gfn_to_page_many_atomic() when prefetching indirect PTEs From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_192519_452165_3C5259FA X-CRM114-Status: GOOD ( 10.39 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Use gfn_to_page_many_atomic() instead of gfn_to_pfn_memslot_atomic() when prefetching indirect PTEs (direct_pte_prefetch_many() already uses the "to page" APIS). Functionally, the two are subtly equivalent, as the "to pfn" API short-circuits hva_to_pfn() if hva_to_pfn_fast() fails, i.e. is just a wrapper for get_user_page_fast_only()/get_user_pages_fast_only(). Switching to the "to page" API will allow dropping the @atomic parameter from the entire hva_to_pfn() callchain. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/paging_tmpl.h | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/arch/x86/kvm/mmu/paging_tmpl.h b/arch/x86/kvm/mmu/paging_tmpl.h index fbaae040218b..36b2607280f0 100644 --- a/arch/x86/kvm/mmu/paging_tmpl.h +++ b/arch/x86/kvm/mmu/paging_tmpl.h @@ -535,8 +535,8 @@ FNAME(prefetch_gpte)(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp, { struct kvm_memory_slot *slot; unsigned pte_access; + struct page *page; gfn_t gfn; - kvm_pfn_t pfn; if (FNAME(prefetch_invalid_gpte)(vcpu, sp, spte, gpte)) return false; @@ -549,12 +549,11 @@ FNAME(prefetch_gpte)(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp, if (!slot) return false; - pfn = gfn_to_pfn_memslot_atomic(slot, gfn); - if (is_error_pfn(pfn)) + if (gfn_to_page_many_atomic(slot, gfn, &page, 1) != 1) return false; - mmu_set_spte(vcpu, slot, spte, pte_access, gfn, pfn, NULL); - kvm_release_pfn_clean(pfn); + mmu_set_spte(vcpu, slot, spte, pte_access, gfn, page_to_pfn(page), NULL); + kvm_release_page_clean(page); return true; } From patchwork Thu Oct 10 18:23:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831153 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B9D9DD2444F for ; Thu, 10 Oct 2024 21:16:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:Content-Type:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=WDoARMTIexYqUKDMQLAC2OQWFTeknat7PpvqQx990/M=; b=msqY7uBXRGTy0M AGoCTeXnmBBUiahKjHv0GFu13VXPMbV+FTv875kVYBmMSusMWatDM9h4Q9IZsUjA2QOPmw5vQS1/U 6dDgmWRdJJ4rAs+pD6mtsf8TbMZMvWrz2uKiPur3XI0kHh7Jahy0BoT3CQxdOE9GRgPaz8xE5SMPM ij+QC3YA0/72aoTBsCtckC7UyzjaBpJG/P8Yi6oDLhDqclE/wlwx1dTIZ88Irh+f/Kf/kNVz3UuBj ooz0LPOmHK2RhoZjBpO5hCHa8pNkB/Tym6txT42Wt7duSWfAHVap8hP6gziHT5OAz0emZaBEAx8yM d94vxI9YZJ8c0ewWqkRg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1sz0WI-0000000ENVr-220X; Thu, 10 Oct 2024 21:16:26 +0000 Received: from desiato.infradead.org ([2001:8b0:10b:1:d65d:64ff:fe57:4e05]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxql-0000000Dpyr-3Na3 for linux-arm-kernel@bombadil.infradead.org; Thu, 10 Oct 2024 18:25:24 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Content-Transfer-Encoding:Content-Type :Cc:To:From:Subject:Message-ID:References:Mime-Version:In-Reply-To:Date: Reply-To:Sender:Content-ID:Content-Description; bh=WDoARMTIexYqUKDMQLAC2OQWFTeknat7PpvqQx990/M=; b=WC5mBfewmhr0oTCmlmNwSCj/Pp pyZFqDuD9jdf1OyDYyY6Z+780w+rFPt1b3N6St2vNJy6Xo40msLkaeO0EXEzw2hy7pUSmSAtbFbcn hjyMvRQdYxXlrbYo4u93w7sohWmmtcarVfPKc9k6P8/iq1ezmx6/6EsW18ui6lMOw+0wsex7g+rY7 ORB9rl6uaqhLFjFxYHGp+CeCpakIH5y9O2nNXDlUnukB927Pt5FvlmrGt/kv5ygzT3VgqD+aRUKNZ c7Qzsc6v9S3hoMLhyRe4zPTS/zgMsLfZO0IiDygb8bqQuljmk+J7sfJgdBUAHj/9RZAcZykdL7jkn 6LbSo/Pg==; Received: from mail-pl1-x649.google.com ([2607:f8b0:4864:20::649]) by desiato.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxqg-00000005UVP-2UeM for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:25:22 +0000 Received: by mail-pl1-x649.google.com with SMTP id d9443c01a7336-20b921fa133so13291605ad.1 for ; Thu, 10 Oct 2024 11:25:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584715; x=1729189515; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=WDoARMTIexYqUKDMQLAC2OQWFTeknat7PpvqQx990/M=; b=l0w9lnO34VRhpw9k4OCEu+5qWOSo/Odt1wTWLT30hnPFDBw9mjJcx13yjIxqVQ+ozn RZRohhTr//PIMU5+d6u0p2Ybm0pluL9Trct+kP4ZdpaBMtyn+vjVDJ4xtjrW3W5rr4J1 xQqPMgmVruN1GprlW/XdHvx+FDJGuSIERXlP2nBikfRbtTWoVBUOgV9j0JftuApvZJ19 qVe6ULQk5PLsMp/4wZJBa8kk3LNbuqSLiixvIPjkPnYXNjwzO/ralU4fWg9M65kFhe3i Fgza66jUadX1Vy06tXYPpkdLyYki95qkQQQ/etSfZX5Hb2a4gT4fVGxo1ccleuS6iOy4 dVNg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584715; x=1729189515; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=WDoARMTIexYqUKDMQLAC2OQWFTeknat7PpvqQx990/M=; b=MgzwdfWfevaMCWkC3YBShgh7Iw81jcPwPPhX+FiBewlJOeP/h0x1sd7vShKKJM1+XF 9vKFj6dP4GcsMwshM88poPJ05cC4dA+rHqEXmU0Hld7pYvVgzshPa8PWg/+JSG9TVzHK BxliBOBhHAX+1QTDE9UWeBUf6Cxt3+cmtt71ydz7PaclO8gub925qVzlhp8rr4CbIOBF iElVxTg7WUn2GaC5UjKr0wmKKDAZ5YwIvDmjvgVjV+2eK6l1na3GC5Xq6kkfwhKUXLY+ uZ0PNNWy1h5nFO2M+U0QGeKa6sxGhoiXPDEoeCnhVIuPw2DBuZ4DKSvBX9v06FVMfnls TfTw== X-Forwarded-Encrypted: i=1; AJvYcCUcI4IAdUpgRqVNha4zPQuxlC0xSAN6WZVX/m8FB1wiMKjHlZYrGF6g0GWBpr7//d3zTSctIg2SHsNUJpilWloD@lists.infradead.org X-Gm-Message-State: AOJu0YywaK8zMbjIUHF95Ga4yKKr02wogZg9cSBkL+2nAsG+bvGR01At lwEpoOlh4xLJidZUmPkMHkSA2bucFZ9ROKfOGGuPdvWq+HeMrHX88uaNX9YiHKty/vXsojBmNBd sqA== X-Google-Smtp-Source: AGHT+IFdLsiMiQR9ssEjYgvfJ+TOiOlvv8h202qp7bAbI4MCMFrC9IRsz+IufRcGyfoerq3Ms7MkoMKw8Uo= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:902:f143:b0:20c:5beb:9c6a with SMTP id d9443c01a7336-20c80510505mr43225ad.4.1728584715291; Thu, 10 Oct 2024 11:25:15 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:13 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-12-seanjc@google.com> Subject: [PATCH v13 11/85] KVM: Rename gfn_to_page_many_atomic() to kvm_prefetch_pages() From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_192518_991859_F30966E4 X-CRM114-Status: GOOD ( 12.97 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Rename gfn_to_page_many_atomic() to kvm_prefetch_pages() to try and communicate its true purpose, as the "atomic" aspect is essentially a side effect of the fact that x86 uses the API while holding mmu_lock. E.g. even if mmu_lock weren't held, KVM wouldn't want to fault-in pages, as the goal is to opportunistically grab surrounding pages that have already been accessed and/or dirtied by the host, and to do so quickly. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/mmu.c | 2 +- arch/x86/kvm/mmu/paging_tmpl.h | 2 +- include/linux/kvm_host.h | 4 ++-- virt/kvm/kvm_main.c | 6 +++--- 4 files changed, 7 insertions(+), 7 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 04228a7da69a..5fe45ab0e818 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -2958,7 +2958,7 @@ static int direct_pte_prefetch_many(struct kvm_vcpu *vcpu, if (!slot) return -1; - ret = gfn_to_page_many_atomic(slot, gfn, pages, end - start); + ret = kvm_prefetch_pages(slot, gfn, pages, end - start); if (ret <= 0) return -1; diff --git a/arch/x86/kvm/mmu/paging_tmpl.h b/arch/x86/kvm/mmu/paging_tmpl.h index 36b2607280f0..143b7e9f26dc 100644 --- a/arch/x86/kvm/mmu/paging_tmpl.h +++ b/arch/x86/kvm/mmu/paging_tmpl.h @@ -549,7 +549,7 @@ FNAME(prefetch_gpte)(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp, if (!slot) return false; - if (gfn_to_page_many_atomic(slot, gfn, &page, 1) != 1) + if (kvm_prefetch_pages(slot, gfn, &page, 1) != 1) return false; mmu_set_spte(vcpu, slot, spte, pte_access, gfn, page_to_pfn(page), NULL); diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index ab4485b2bddc..56e7cde8c8b8 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -1207,8 +1207,8 @@ void kvm_arch_flush_shadow_all(struct kvm *kvm); void kvm_arch_flush_shadow_memslot(struct kvm *kvm, struct kvm_memory_slot *slot); -int gfn_to_page_many_atomic(struct kvm_memory_slot *slot, gfn_t gfn, - struct page **pages, int nr_pages); +int kvm_prefetch_pages(struct kvm_memory_slot *slot, gfn_t gfn, + struct page **pages, int nr_pages); struct page *gfn_to_page(struct kvm *kvm, gfn_t gfn); unsigned long gfn_to_hva(struct kvm *kvm, gfn_t gfn); diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 2032292df0b0..957b4a6c9254 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -3053,8 +3053,8 @@ kvm_pfn_t kvm_vcpu_gfn_to_pfn(struct kvm_vcpu *vcpu, gfn_t gfn) } EXPORT_SYMBOL_GPL(kvm_vcpu_gfn_to_pfn); -int gfn_to_page_many_atomic(struct kvm_memory_slot *slot, gfn_t gfn, - struct page **pages, int nr_pages) +int kvm_prefetch_pages(struct kvm_memory_slot *slot, gfn_t gfn, + struct page **pages, int nr_pages) { unsigned long addr; gfn_t entry = 0; @@ -3068,7 +3068,7 @@ int gfn_to_page_many_atomic(struct kvm_memory_slot *slot, gfn_t gfn, return get_user_pages_fast_only(addr, nr_pages, FOLL_WRITE, pages); } -EXPORT_SYMBOL_GPL(gfn_to_page_many_atomic); +EXPORT_SYMBOL_GPL(kvm_prefetch_pages); /* * Do not use this helper unless you are absolutely certain the gfn _must_ be From patchwork Thu Oct 10 18:23:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831156 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D6DC7D24457 for ; Thu, 10 Oct 2024 21:16:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:Content-Type:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=IMk6Ksq+DkJF8Cmr0W3poBl1lc0GSthNwEC5P5onZMQ=; b=qPYEOKwxqrFmZP 94rDZKhRnvV+FeiocWWFjPiUwhJoQ7V4zL+faFF7EgSEdX9WidmF8RrAbTQKkdtC1DIT9+wvy/ux7 WsroEISd0wy4xUtdiILdhQcC1yBPk2gshg6jA+kdF5XxXsO2HIcWWa3qoxjxGf/UmX9WUOr9SNz7Z qA5q6nERRP7UAOFLQBKLFXRLmG/u2PULc6veJqmQCo+9ksbBaugc1XW/PiVhKRh0UfI0tlN1qe5DI TYgPOlpSRUgVZZcIMhq51arCj1Fb3g+jTbzT3oXnsXoXy/ad4SU9ttzk8BOj2dyA9Hrj7B0FRao/l ZigBkjsMwmt0FnzQXZCA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1sz0WH-0000000ENVH-3iy0; Thu, 10 Oct 2024 21:16:25 +0000 Received: from mail-pf1-x449.google.com ([2607:f8b0:4864:20::449]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxqh-0000000DpuP-35II for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:25:23 +0000 Received: by mail-pf1-x449.google.com with SMTP id d2e1a72fcca58-71def9ed5e2so1434524b3a.3 for ; Thu, 10 Oct 2024 11:25:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584717; x=1729189517; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=IMk6Ksq+DkJF8Cmr0W3poBl1lc0GSthNwEC5P5onZMQ=; b=sYztD1QUHkHCxhJUevPDoJF4UDpBqIWxrLp4Fsc5G3u67OwUqDTyKbqhAKCZPzJ0d8 qRK1Fa9bB06nko7fzVcGEhCI+y1w5VdhwVp/LLs1a6pVEGIvhu5ntvEhQJ1kNfZZGWFv iGi2Nwxi8R1PVsHxD+9i09eW1iQvDjX1p7b4InpT4z5RV0k9HvhqonQpuuYXH69vwNdu ZY6CSpu+KVE8A5eaTeR4l5pqAT5Fuhdgy0cYd8NxvfQdZ3nRB4FbkRSE5MYs4z/KR6yb I+ApWd+1p8f6iY4xfplNq8eqYOJv97lfMldu9Wvpdnu8LB+vryaM+CIWcGjlL47ZHZuP auGg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584717; x=1729189517; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=IMk6Ksq+DkJF8Cmr0W3poBl1lc0GSthNwEC5P5onZMQ=; b=PeMG3X1ISLOHIHFI0ATY2WGPDaUJ4QA7rT1oKo6wrQHaC2nmoKA+hSGvP5RMasOOSq iPVoWLtoO1KN3PRjguulT7MAWfmU54BXSHluf2Ld6Oz27di4t4x3xMIKp+Ufqgah4C2+ jAw9D2ilkZ4ANW4TIPrPx8Ge0fz9usnR81kkMwVHU+44GwKEFJbmOZwUi8fyv1l5ig7b IIkBM+g7jSIhM+6hBAbhL1db7T6/bMCsxsKn1nbr9w8GnB4wrtKq0sQQG45JiBQRPrvR drjO0lrAVPM2smrpGK82iq0T1jGJwiAAvkc+QARwm4QAXYtW96aAcGf9hI+dbQ+ucfT1 lRVQ== X-Forwarded-Encrypted: i=1; AJvYcCWngm5zWD+tXrnKntItuJtoMvTG8ze6Tph26/t6fo+yX9pWEkKE3mQYhCBgXCE1GPHLi4SsBaHH1Wrr8VLdL9Y+@lists.infradead.org X-Gm-Message-State: AOJu0YzjphL27csoVzAEbe8kgio6LaOsu7wskumRSBf0SxvMll2/VbNw xtXtygRSkJ5o9480w41zfvEL+JsFDtluXny8LjhT2hFUOr66z/J9avdgPj9ZH/RLH8i5PcXzACp J6Q== X-Google-Smtp-Source: AGHT+IEgBAPk5NggidzeRIz4/KASJ69N4B17cGxFjG2BQTaLRrpMtimTBEcirbwLQ+H7ew0HtHwcTgFgXoI= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:6a00:2d8a:b0:71d:fc7c:4a34 with SMTP id d2e1a72fcca58-71e1dbff219mr24371b3a.6.1728584717336; Thu, 10 Oct 2024 11:25:17 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:14 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-13-seanjc@google.com> Subject: [PATCH v13 12/85] KVM: Drop @atomic param from gfn=>pfn and hva=>pfn APIs From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112519_903780_2CA478C3 X-CRM114-Status: GOOD ( 18.52 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Drop @atomic from the myriad "to_pfn" APIs now that all callers pass "false", and remove a comment blurb about KVM running only the "GUP fast" part in atomic context. No functional change intended. Reviewed-by: Alex Bennée Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- Documentation/virt/kvm/locking.rst | 4 +-- arch/arm64/kvm/mmu.c | 2 +- arch/powerpc/kvm/book3s_64_mmu_hv.c | 2 +- arch/powerpc/kvm/book3s_64_mmu_radix.c | 2 +- arch/x86/kvm/mmu/mmu.c | 12 ++++---- include/linux/kvm_host.h | 4 +-- virt/kvm/kvm_main.c | 39 ++++++-------------------- virt/kvm/kvm_mm.h | 4 +-- virt/kvm/pfncache.c | 2 +- 9 files changed, 23 insertions(+), 48 deletions(-) diff --git a/Documentation/virt/kvm/locking.rst b/Documentation/virt/kvm/locking.rst index 3d8bf40ca448..f463ac42ac7a 100644 --- a/Documentation/virt/kvm/locking.rst +++ b/Documentation/virt/kvm/locking.rst @@ -135,8 +135,8 @@ We dirty-log for gfn1, that means gfn2 is lost in dirty-bitmap. For direct sp, we can easily avoid it since the spte of direct sp is fixed to gfn. For indirect sp, we disabled fast page fault for simplicity. -A solution for indirect sp could be to pin the gfn, for example via -kvm_vcpu_gfn_to_pfn_atomic, before the cmpxchg. After the pinning: +A solution for indirect sp could be to pin the gfn before the cmpxchg. After +the pinning: - We have held the refcount of pfn; that means the pfn can not be freed and be reused for another gfn. diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c index a509b63bd4dd..a6e62cc9015c 100644 --- a/arch/arm64/kvm/mmu.c +++ b/arch/arm64/kvm/mmu.c @@ -1569,7 +1569,7 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, mmu_seq = vcpu->kvm->mmu_invalidate_seq; mmap_read_unlock(current->mm); - pfn = __gfn_to_pfn_memslot(memslot, gfn, false, false, NULL, + pfn = __gfn_to_pfn_memslot(memslot, gfn, false, NULL, write_fault, &writable, NULL); if (pfn == KVM_PFN_ERR_HWPOISON) { kvm_send_hwpoison_signal(hva, vma_shift); diff --git a/arch/powerpc/kvm/book3s_64_mmu_hv.c b/arch/powerpc/kvm/book3s_64_mmu_hv.c index 1b51b1c4713b..8cd02ca4b1b8 100644 --- a/arch/powerpc/kvm/book3s_64_mmu_hv.c +++ b/arch/powerpc/kvm/book3s_64_mmu_hv.c @@ -613,7 +613,7 @@ int kvmppc_book3s_hv_page_fault(struct kvm_vcpu *vcpu, write_ok = true; } else { /* Call KVM generic code to do the slow-path check */ - pfn = __gfn_to_pfn_memslot(memslot, gfn, false, false, NULL, + pfn = __gfn_to_pfn_memslot(memslot, gfn, false, NULL, writing, &write_ok, NULL); if (is_error_noslot_pfn(pfn)) return -EFAULT; diff --git a/arch/powerpc/kvm/book3s_64_mmu_radix.c b/arch/powerpc/kvm/book3s_64_mmu_radix.c index 408d98f8a514..26a969e935e3 100644 --- a/arch/powerpc/kvm/book3s_64_mmu_radix.c +++ b/arch/powerpc/kvm/book3s_64_mmu_radix.c @@ -852,7 +852,7 @@ int kvmppc_book3s_instantiate_page(struct kvm_vcpu *vcpu, unsigned long pfn; /* Call KVM generic code to do the slow-path check */ - pfn = __gfn_to_pfn_memslot(memslot, gfn, false, false, NULL, + pfn = __gfn_to_pfn_memslot(memslot, gfn, false, NULL, writing, upgrade_p, NULL); if (is_error_noslot_pfn(pfn)) return -EFAULT; diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 5fe45ab0e818..0e235f276ee5 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4380,9 +4380,9 @@ static int __kvm_faultin_pfn(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault return kvm_faultin_pfn_private(vcpu, fault); async = false; - fault->pfn = __gfn_to_pfn_memslot(fault->slot, fault->gfn, false, false, - &async, fault->write, - &fault->map_writable, &fault->hva); + fault->pfn = __gfn_to_pfn_memslot(fault->slot, fault->gfn, false, &async, + fault->write, &fault->map_writable, + &fault->hva); if (!async) return RET_PF_CONTINUE; /* *pfn has correct page already */ @@ -4402,9 +4402,9 @@ static int __kvm_faultin_pfn(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault * to wait for IO. Note, gup always bails if it is unable to quickly * get a page and a fatal signal, i.e. SIGKILL, is pending. */ - fault->pfn = __gfn_to_pfn_memslot(fault->slot, fault->gfn, false, true, - NULL, fault->write, - &fault->map_writable, &fault->hva); + fault->pfn = __gfn_to_pfn_memslot(fault->slot, fault->gfn, true, NULL, + fault->write, &fault->map_writable, + &fault->hva); return RET_PF_CONTINUE; } diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 56e7cde8c8b8..2faafc7a56ae 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -1232,9 +1232,8 @@ kvm_pfn_t gfn_to_pfn(struct kvm *kvm, gfn_t gfn); kvm_pfn_t gfn_to_pfn_prot(struct kvm *kvm, gfn_t gfn, bool write_fault, bool *writable); kvm_pfn_t gfn_to_pfn_memslot(const struct kvm_memory_slot *slot, gfn_t gfn); -kvm_pfn_t gfn_to_pfn_memslot_atomic(const struct kvm_memory_slot *slot, gfn_t gfn); kvm_pfn_t __gfn_to_pfn_memslot(const struct kvm_memory_slot *slot, gfn_t gfn, - bool atomic, bool interruptible, bool *async, + bool interruptible, bool *async, bool write_fault, bool *writable, hva_t *hva); void kvm_release_pfn_clean(kvm_pfn_t pfn); @@ -1315,7 +1314,6 @@ void mark_page_dirty(struct kvm *kvm, gfn_t gfn); struct kvm_memslots *kvm_vcpu_memslots(struct kvm_vcpu *vcpu); struct kvm_memory_slot *kvm_vcpu_gfn_to_memslot(struct kvm_vcpu *vcpu, gfn_t gfn); -kvm_pfn_t kvm_vcpu_gfn_to_pfn_atomic(struct kvm_vcpu *vcpu, gfn_t gfn); kvm_pfn_t kvm_vcpu_gfn_to_pfn(struct kvm_vcpu *vcpu, gfn_t gfn); int kvm_vcpu_map(struct kvm_vcpu *vcpu, gpa_t gpa, struct kvm_host_map *map); void kvm_vcpu_unmap(struct kvm_vcpu *vcpu, struct kvm_host_map *map, bool dirty); diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 957b4a6c9254..0bc077213d3e 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -2756,8 +2756,7 @@ static inline int check_user_page_hwpoison(unsigned long addr) /* * The fast path to get the writable pfn which will be stored in @pfn, - * true indicates success, otherwise false is returned. It's also the - * only part that runs if we can in atomic context. + * true indicates success, otherwise false is returned. */ static bool hva_to_pfn_fast(unsigned long addr, bool write_fault, bool *writable, kvm_pfn_t *pfn) @@ -2922,7 +2921,6 @@ static int hva_to_pfn_remapped(struct vm_area_struct *vma, /* * Pin guest page in memory and return its pfn. * @addr: host virtual address which maps memory to the guest - * @atomic: whether this function is forbidden from sleeping * @interruptible: whether the process can be interrupted by non-fatal signals * @async: whether this function need to wait IO complete if the * host page is not in the memory @@ -2934,22 +2932,16 @@ static int hva_to_pfn_remapped(struct vm_area_struct *vma, * 2): @write_fault = false && @writable, @writable will tell the caller * whether the mapping is writable. */ -kvm_pfn_t hva_to_pfn(unsigned long addr, bool atomic, bool interruptible, - bool *async, bool write_fault, bool *writable) +kvm_pfn_t hva_to_pfn(unsigned long addr, bool interruptible, bool *async, + bool write_fault, bool *writable) { struct vm_area_struct *vma; kvm_pfn_t pfn; int npages, r; - /* we can do it either atomically or asynchronously, not both */ - BUG_ON(atomic && async); - if (hva_to_pfn_fast(addr, write_fault, writable, &pfn)) return pfn; - if (atomic) - return KVM_PFN_ERR_FAULT; - npages = hva_to_pfn_slow(addr, async, write_fault, interruptible, writable, &pfn); if (npages == 1) @@ -2986,7 +2978,7 @@ kvm_pfn_t hva_to_pfn(unsigned long addr, bool atomic, bool interruptible, } kvm_pfn_t __gfn_to_pfn_memslot(const struct kvm_memory_slot *slot, gfn_t gfn, - bool atomic, bool interruptible, bool *async, + bool interruptible, bool *async, bool write_fault, bool *writable, hva_t *hva) { unsigned long addr = __gfn_to_hva_many(slot, gfn, NULL, write_fault); @@ -3008,39 +3000,24 @@ kvm_pfn_t __gfn_to_pfn_memslot(const struct kvm_memory_slot *slot, gfn_t gfn, writable = NULL; } - return hva_to_pfn(addr, atomic, interruptible, async, write_fault, - writable); + return hva_to_pfn(addr, interruptible, async, write_fault, writable); } EXPORT_SYMBOL_GPL(__gfn_to_pfn_memslot); kvm_pfn_t gfn_to_pfn_prot(struct kvm *kvm, gfn_t gfn, bool write_fault, bool *writable) { - return __gfn_to_pfn_memslot(gfn_to_memslot(kvm, gfn), gfn, false, false, - NULL, write_fault, writable, NULL); + return __gfn_to_pfn_memslot(gfn_to_memslot(kvm, gfn), gfn, false, NULL, + write_fault, writable, NULL); } EXPORT_SYMBOL_GPL(gfn_to_pfn_prot); kvm_pfn_t gfn_to_pfn_memslot(const struct kvm_memory_slot *slot, gfn_t gfn) { - return __gfn_to_pfn_memslot(slot, gfn, false, false, NULL, true, - NULL, NULL); + return __gfn_to_pfn_memslot(slot, gfn, false, NULL, true, NULL, NULL); } EXPORT_SYMBOL_GPL(gfn_to_pfn_memslot); -kvm_pfn_t gfn_to_pfn_memslot_atomic(const struct kvm_memory_slot *slot, gfn_t gfn) -{ - return __gfn_to_pfn_memslot(slot, gfn, true, false, NULL, true, - NULL, NULL); -} -EXPORT_SYMBOL_GPL(gfn_to_pfn_memslot_atomic); - -kvm_pfn_t kvm_vcpu_gfn_to_pfn_atomic(struct kvm_vcpu *vcpu, gfn_t gfn) -{ - return gfn_to_pfn_memslot_atomic(kvm_vcpu_gfn_to_memslot(vcpu, gfn), gfn); -} -EXPORT_SYMBOL_GPL(kvm_vcpu_gfn_to_pfn_atomic); - kvm_pfn_t gfn_to_pfn(struct kvm *kvm, gfn_t gfn) { return gfn_to_pfn_memslot(gfn_to_memslot(kvm, gfn), gfn); diff --git a/virt/kvm/kvm_mm.h b/virt/kvm/kvm_mm.h index 715f19669d01..a3fa86f60d6c 100644 --- a/virt/kvm/kvm_mm.h +++ b/virt/kvm/kvm_mm.h @@ -20,8 +20,8 @@ #define KVM_MMU_UNLOCK(kvm) spin_unlock(&(kvm)->mmu_lock) #endif /* KVM_HAVE_MMU_RWLOCK */ -kvm_pfn_t hva_to_pfn(unsigned long addr, bool atomic, bool interruptible, - bool *async, bool write_fault, bool *writable); +kvm_pfn_t hva_to_pfn(unsigned long addr, bool interruptible, bool *async, + bool write_fault, bool *writable); #ifdef CONFIG_HAVE_KVM_PFNCACHE void gfn_to_pfn_cache_invalidate_start(struct kvm *kvm, diff --git a/virt/kvm/pfncache.c b/virt/kvm/pfncache.c index f0039efb9e1e..58c706a610e5 100644 --- a/virt/kvm/pfncache.c +++ b/virt/kvm/pfncache.c @@ -198,7 +198,7 @@ static kvm_pfn_t hva_to_pfn_retry(struct gfn_to_pfn_cache *gpc) } /* We always request a writeable mapping */ - new_pfn = hva_to_pfn(gpc->uhva, false, false, NULL, true, NULL); + new_pfn = hva_to_pfn(gpc->uhva, false, NULL, true, NULL); if (is_error_noslot_pfn(new_pfn)) goto out_error; From patchwork Thu Oct 10 18:23:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831024 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5F5A1D2444A for ; Thu, 10 Oct 2024 20:00:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:Content-Type:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=F0+KYw/6HzWsmGATvSxbQuKdqCcRS3waqcwxHrAen8Y=; b=pjcqYxRy1T1KOr TTmdQXtbcDQU+vrDnwL6tFpSCcrYTI/Y6c2GlFMB9bSQn3PYVaR9TW7fcprOMkXWO60dnpWcwphLy YuTVGsufCesD91PAIWcQlgch1aJrCb2fhIw8U4xFqGJSBrUkfScNAxmk6MAbdLaMp0GE6kb1nd0jC mOT93fbpIj6WEwbbuR0xGW/w51LUYR/brJAEnEZ69YIXVatjsx+R/u8O2O7MHa0ZMjWOE8Ed5ed2A sXSMeNaUGR32nG+uNyHD9M+noo3AkOQb7Uqhqm8EBYnnh7q1DBUy/fZSq5GOu4mMyH74nk0wTCMrC Zg+odOanpGz47CX1/tPw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syzL4-0000000EDRD-12gq; Thu, 10 Oct 2024 20:00:46 +0000 Received: from mail-yw1-x114a.google.com ([2607:f8b0:4864:20::114a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxqi-0000000DpwJ-2qnY for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:25:22 +0000 Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-6e2261adfdeso21181207b3.2 for ; Thu, 10 Oct 2024 11:25:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584719; x=1729189519; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=F0+KYw/6HzWsmGATvSxbQuKdqCcRS3waqcwxHrAen8Y=; b=xcx2ikl1tbRLIGweKlp3E/XnLAXCMQRCQXm9TtYfNf9DGLbpCrsIFndqhSqsrjmSat qbIwnWEGdAd/nLXZAcHBPN1kvhcatnmCn/IV2PUU4hvQAUC0GtQOG/uesI9giBA68Vdl P8krWUymc0mzim/EeROqSoFIeSbbx3oMeiXZ0tVASZm3sdADhAR1QaWws2RKxwvjQLlU QuZptrSMsSy1XH5GGusvv0oBwN4fcKmhVFZQJ9iPkBVYw0QATlMg251iOM1Fe+gpWxFP XM+6I7qL7tGGMZT5M5+DeqRp9HNjhyKl/Q1MkapmvaxgyFylhv2ThxWRjBfsgRywaBUZ W1aA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584719; x=1729189519; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=F0+KYw/6HzWsmGATvSxbQuKdqCcRS3waqcwxHrAen8Y=; b=E/vZ4kUcY9AjoMUFpP14PNqkOzRH2Yfa8EeaMdRRKxOsyF1kaOER2Ipb8q8Xnxek1h Ri81YZUj0RxBlkscfpFbqI5ZNLNUJjMT3Xn74+IL09svxRVuP4anFuHkZyT7X89wFO+a AsY9JXuj4P41I7nA13CxLKctmox6Fu/b6nBUeiSU+33ib+S4gdDWWoSqfnDLH88XOrLO t/GEXVbLmRHRDb1sAvXvxeNHr1cZIr4uCyglqVIixNc8b4WavtcWznPzjuzqKVJ2o2k6 3vYAly5TqXX4Qiho6ZTPjK9LtZspvUAWiChbYaRcSOOVKIklNqInGsXGj3cDLdF+rL+n unIg== X-Forwarded-Encrypted: i=1; AJvYcCX4zE4Oq2cYbmwyLt5iJqm40sBo9CgecTddfT+1nkfcT598f5fxUVXJwvVsRFEyP/ZJd2MQp5kkGdjfe/0BEXc7@lists.infradead.org X-Gm-Message-State: AOJu0Yxzw7gF1MrxiP059pTLLapMnCEd2vWgnQ43PxRTNqY2lEXcJN9M wwKgQjMhDv7Kw7Cep8i6+bGKadgRlEq7nBP/I04ftE+xGnkwzirZ2TcbYvW2BDRqNGiktbor0ch kAA== X-Google-Smtp-Source: AGHT+IEeBu92qjGJ6/8zutZUBi8WOanw4UupdtztXpUfLE+vAeLHZHNkVuFOksmpHMU427MoGYN+Q9YZlwU= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:690c:3204:b0:62c:f976:a763 with SMTP id 00721157ae682-6e322185b22mr897567b3.1.1728584719334; Thu, 10 Oct 2024 11:25:19 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:15 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-14-seanjc@google.com> Subject: [PATCH v13 13/85] KVM: Annotate that all paths in hva_to_pfn() might sleep From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112520_758822_1C8D01E3 X-CRM114-Status: GOOD ( 11.39 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Now that hva_to_pfn() no longer supports being called in atomic context, move the might_sleep() annotation from hva_to_pfn_slow() to hva_to_pfn(). Reviewed-by: Alex Bennée Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- virt/kvm/kvm_main.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 0bc077213d3e..17acc75990a5 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -2804,8 +2804,6 @@ static int hva_to_pfn_slow(unsigned long addr, bool *async, bool write_fault, struct page *page; int npages; - might_sleep(); - if (writable) *writable = write_fault; @@ -2939,6 +2937,8 @@ kvm_pfn_t hva_to_pfn(unsigned long addr, bool interruptible, bool *async, kvm_pfn_t pfn; int npages, r; + might_sleep(); + if (hva_to_pfn_fast(addr, write_fault, writable, &pfn)) return pfn; From patchwork Thu Oct 10 18:23:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831157 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id BC6F1D24454 for ; Thu, 10 Oct 2024 21:16:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Type:Cc:To: From:Subject:Message-ID:References:Mime-Version:In-Reply-To:Date: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=RdgAxVyxp31c7WibHLiyIF43paER00mxZOAIHhsdwhk=; b=3/C0AnfUrEBWYY4HUYqa3EfOMq 5pSAVZGfB64Z4LqPTQWLhp2jnH21PwnJtX2IXlE2DjeckBAaqZeRvdjhMPllhIizCh3Ntl/+kMjZU tYB8EodRCyB04H8O6itAKwS6MPiUty7pLXSUzuTyvNT3byYxdPROar0PviGtirMy+9bQvKLkxDno7 PWiBu1tXCY9lsxMvn5pUHpweri2vTs5VN2/gd8DGybJ/DTzkBzI8wLocc1mpAjDBbwYZnf/s3Z9o9 Q5QJ1UXo6rXHpxzzbxUCfez5DYwwuXx65yhAfLFxTLLfAMDoEQ1iEzygtdD1AaSIrgQ86vC0w5ruM 5zBDG2Yw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1sz0WJ-0000000ENWr-3P7V; Thu, 10 Oct 2024 21:16:27 +0000 Received: from mail-pj1-x104a.google.com ([2607:f8b0:4864:20::104a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxql-0000000DpyZ-3XZh for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:25:26 +0000 Received: by mail-pj1-x104a.google.com with SMTP id 98e67ed59e1d1-2e2ca4fb175so1247620a91.3 for ; Thu, 10 Oct 2024 11:25:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584722; x=1729189522; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=RdgAxVyxp31c7WibHLiyIF43paER00mxZOAIHhsdwhk=; b=b7p6IY4ClkV+QsXomlwvoGqMh+SB8UDjT7mX9uoM5ZhyMGTHNoOTrGpBZfJp2lwLVX 0x23jdq8biX8z0ZIidDU79yQDeDe6dBrdMgsah/xF71aJn4U5mr4VKqCbENkqredNlpT 0tfmwqY+XQFrBOrUvncPhvTCE4/rbEy5HZJ/ggCCaLBScKOKMlUvJ3pQP7qNuXb3HndF Zsxc8oRV34Z1rzEDTCj2Q4Ps4jWYqYy79NUHweHrj8V03+aXAOEnxXwmcoZQhpeNOYhO B+38kYQySh0koQBeoNmkx1In/nwDJQ3xu5Ei0inEsydspQsXYJq9uOlExrXPKZG/AZXL dJmg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584722; x=1729189522; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=RdgAxVyxp31c7WibHLiyIF43paER00mxZOAIHhsdwhk=; b=U6W0AjmRbDyjKC5TMHwtxPNKBIt5lsMaYeY0p6WMShTDK19IdGU7cN1LUIx8f6BaKn IwHHPxkRdqk7RqocqhiIRDiYCnff/HwTbkoA3eogMLIFs44m6nD3x0dEqM+19U0G6atb cZhg29bZ7Sd5sEDjKvdpHheKm9tyEQ9jP7GFhgd/Ai9rpvoP6dbvQI32K4/POiCiKUxx aP6K1Vz7CWsP5PmgjTyKttSsLfOYT8sYkVZEqZmS2vhctZGSAb3q55w2vsBLLkXZQ1cm udMsi+bZSTBqjTbQPQ9R+rXuyeiCSdPd4RwVw04E3ofiDKcBcMofLx55c4+eBCRCTxmx WKhw== X-Forwarded-Encrypted: i=1; AJvYcCXVBimMk+vvfnsLeZVcIDrvRNhqkUItx+qkCfcoJvGObtvbz58N06YwU/GGKFhQF/n7kZyK6wL1rT54ufF+eR/F@lists.infradead.org X-Gm-Message-State: AOJu0Yz3v5CMTcr2YFAyi/sAdL6Exh9dgpqzbGBwoki9lmzKJoCDKOSX qlFN5tGadO3zgvaDHh7BkWDqudUWpqvXVC3wxgZxjzDXyKFGowtkt5sYShZpbZMRGWSPCMUV6g4 dIg== X-Google-Smtp-Source: AGHT+IH031wjBK6zBugJiyQwbgM1S3WkZumhuZG0/47H7WWzE24O4tbIqX0eqBdD0TzKWzHgCchY3qP4ics= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:90a:9a94:b0:2e2:da81:40c1 with SMTP id 98e67ed59e1d1-2e2f09f2280mr107a91.1.1728584720854; Thu, 10 Oct 2024 11:25:20 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:16 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-15-seanjc@google.com> Subject: [PATCH v13 14/85] KVM: Return ERR_SIGPENDING from hva_to_pfn() if GUP returns -EGAIN From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112523_975585_601D5DC4 X-CRM114-Status: GOOD ( 11.50 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Treat an -EAGAIN return from GUP the same as -EINTR and immediately report to the caller that a signal is pending. GUP only returns -EAGAIN if the _initial_ mmap_read_lock_killable() fails, which in turn onnly fails if a signal is pending Note, rwsem_down_read_slowpath() actually returns -EINTR, so GUP is really just making life harder than it needs to be. And the call to mmap_read_lock_killable() in the retry path returns its -errno verbatim, i.e. GUP (and thus KVM) is already handling locking failure this way, but only some of the time. Suggested-by: Paolo Bonzini Signed-off-by: Sean Christopherson --- virt/kvm/kvm_main.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 17acc75990a5..ebba5d22db2d 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -2946,7 +2946,7 @@ kvm_pfn_t hva_to_pfn(unsigned long addr, bool interruptible, bool *async, writable, &pfn); if (npages == 1) return pfn; - if (npages == -EINTR) + if (npages == -EINTR || npages == -EAGAIN) return KVM_PFN_ERR_SIGPENDING; mmap_read_lock(current->mm); From patchwork Thu Oct 10 18:23:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 34F4BD2444B for ; Thu, 10 Oct 2024 20:07:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Type:Cc:To: From:Subject:Message-ID:References:Mime-Version:In-Reply-To:Date: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=86tNdlyX8HXR/WHFYCj/131p34DPDBwWgQrhOuSjETM=; b=JOax6v6pCEzbbK4+H2rmPc6FEX qZ/1GCBvq1RjiW6puj9k1xSolwaDq7eN1n6JI+ZN8Kg/JCCgesqi5V93fsWLKIEkOS9zu6TDhkmnG KuMqPiiOMsYT5GVrrWbAGnErytM7/GvhrcAiVBGOE5xmZyGxLc/3TA1+joyw29PVtjrZrQ2mWyQTQ QYKW9Lc+0fzaIb9BZeSN22HZ0jm+DPzcR/lKpRNsoNu0n2aeUHF+dx7XDIPjKpTcUyyO/uyseMhRs iZAzN6bivkqPE1PLfNkONtVFFTUly7ZF7pNiZL+X1W83lr0jSxn4C/Ri8ipnhqCLTeR7tN3+IGovE 4ESWiV6g==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syzRb-0000000EDzf-1zNu; Thu, 10 Oct 2024 20:07:31 +0000 Received: from mail-yw1-f201.google.com ([209.85.128.201]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxqo-0000000Dq0U-12Pa for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:25:27 +0000 Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-6e2270a147aso21755297b3.0 for ; Thu, 10 Oct 2024 11:25:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584724; x=1729189524; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=86tNdlyX8HXR/WHFYCj/131p34DPDBwWgQrhOuSjETM=; b=N4gnYeeGDWzC78e3rSacwyrZQ1zS2o0yOL0hz9MvZ+3DIzi+69xBA6zHHWOtGMJ2o4 +J6iT1n1ARNhvB5RJoFHzuv+22xh20DwcLqiRfU5Zh/1hVlsG256CSfzaJoTeA5X+f+O UPmmk1Ezb4cnGxfvJW/RfYmICQgZLA1KjSgMaAfaIwOXS+Pv6DWInlDP9pObpC3C/KlX qpCk6StQ5PCtNMk2Jmk2O/6M9v2d2dl4d70qhqpBoXa+gnO5zGGVilZrXdQ2IvvB5vYN MmC7jNDHog6vNvFcJPnquCCHzR84iug+bqTRoJkTuXfv6KQUKKv6mk+mlcXTwCkqfzBN kuvQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584724; x=1729189524; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=86tNdlyX8HXR/WHFYCj/131p34DPDBwWgQrhOuSjETM=; b=Z75oJ+p2C6+UDUsVnq8f85gm4h3KwVhs6OKycZgwJAjQxPYLgVHtY6Pcjd4Z8eF6AD qZU1CObSARwGkjCBsQU4La3gdmW6ZJ30k2UHkRek1a8Zv3IFdWjR2VpTiQcf653FP1Rc 4Qjk93WUcY90wTjW8II5q3Z8q+V3pM5bI2+6pgMcV0mudlUZVdojryWmCbEbj5lRUb/6 BZHTFm332/4vyApo6N0xdqucsuhTtm20ljGfQNBF3s/wEtLZnzGNJoiJ50YtDH7T8Zq5 +QCXqftRYIfuEF8/LD1zHAQUx9BZ/pFg8OJxejUEJ1enKKXbrWEtANdOGN5w0m5oyTSf Qrdw== X-Forwarded-Encrypted: i=1; AJvYcCW7slV1DuLPXMEI7N7ArK8wMSpAB7dvxLesWHSsf4e7IJD0zK92XX5Q1Ny3ghOsYFmj4v8OvNl12pU8e9S+V6yx@lists.infradead.org X-Gm-Message-State: AOJu0Yz77VIUL2DDu+NbEviX3WQr344wzNij3BRytr9hjAbeSSyacyHX CwKC3nu4H9X0bioI4aJo/wYi8pO5+18a4xkSTI7dFzjXBMgBpr6M0FW/G5lkqkZILrLRdiZsFCc g+w== X-Google-Smtp-Source: AGHT+IGf82b/hxj2IWFe54/TAnEDS4XyhakmYGUWxryxrCLTzb83+cUU5J1DZhSXbLBNZQvipdjBbhmVkR8= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a25:8e09:0:b0:e28:ef25:5f13 with SMTP id 3f1490d57ef6-e28fe0e7352mr200743276.0.1728584723790; Thu, 10 Oct 2024 11:25:23 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:17 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-16-seanjc@google.com> Subject: [PATCH v13 15/85] KVM: Drop extra GUP (via check_user_page_hwpoison()) to detect poisoned page From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112526_381772_4BCAF763 X-CRM114-Status: GOOD ( 12.56 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Remove check_user_page_hwpoison() as it's effectively dead code. Prior to commit 234b239bea39 ("kvm: Faults which trigger IO release the mmap_sem"), hva_to_pfn_slow() wasn't actually a slow path in all cases, i.e. would do get_user_pages_fast() without ever doing slow GUP with FOLL_HWPOISON. Now that hva_to_pfn_slow() is a straight shot to get_user_pages_unlocked(), and unconditionally passes FOLL_HWPOISON, it is impossible for hva_to_pfn() to get an -errno that needs to be morphed to -EHWPOISON. There are essentially four cases in KVM: - npages == 0, then FOLL_NOWAIT, a.k.a. @async, must be true, and thus check_user_page_hwpoison() will not be called - npages == 1 || npages == -EHWPOISON, all good - npages == -EINTR || npages == -EAGAIN, bail early, all good - everything else, including -EFAULT, can go down the vma_lookup() path, as npages < 0 means KVM went through hva_to_pfn_slow() which passes FOLL_HWPOISON Suggested-by: Paolo Bonzini Signed-off-by: Sean Christopherson --- virt/kvm/kvm_main.c | 17 ++--------------- 1 file changed, 2 insertions(+), 15 deletions(-) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index ebba5d22db2d..87f81e74cbc0 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -2746,14 +2746,6 @@ unsigned long kvm_vcpu_gfn_to_hva_prot(struct kvm_vcpu *vcpu, gfn_t gfn, bool *w return gfn_to_hva_memslot_prot(slot, gfn, writable); } -static inline int check_user_page_hwpoison(unsigned long addr) -{ - int rc, flags = FOLL_HWPOISON | FOLL_WRITE; - - rc = get_user_pages(addr, 1, flags, NULL); - return rc == -EHWPOISON; -} - /* * The fast path to get the writable pfn which will be stored in @pfn, * true indicates success, otherwise false is returned. @@ -2948,14 +2940,10 @@ kvm_pfn_t hva_to_pfn(unsigned long addr, bool interruptible, bool *async, return pfn; if (npages == -EINTR || npages == -EAGAIN) return KVM_PFN_ERR_SIGPENDING; + if (npages == -EHWPOISON) + return KVM_PFN_ERR_HWPOISON; mmap_read_lock(current->mm); - if (npages == -EHWPOISON || - (!async && check_user_page_hwpoison(addr))) { - pfn = KVM_PFN_ERR_HWPOISON; - goto exit; - } - retry: vma = vma_lookup(current->mm, addr); @@ -2972,7 +2960,6 @@ kvm_pfn_t hva_to_pfn(unsigned long addr, bool interruptible, bool *async, *async = true; pfn = KVM_PFN_ERR_FAULT; } -exit: mmap_read_unlock(current->mm); return pfn; } From patchwork Thu Oct 10 18:23:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E99E2D2444C for ; Thu, 10 Oct 2024 20:09:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:Content-Type:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=H+RZpPtsk3Pe/RqqhWN1oE7ZGzOesPmt7slwmkU09yw=; b=XjbanuYEeuPuHR sG4hLAVJaXhIj3QhOIMKLkwveVlnTW1vvAYzcXxjq1IZOIa46Bg4oQ5PS4gO3Y5oevfl3Je7W/puO IKrJ29Ib8066Ih0O1aXCbLgAEyQKzmxDdmu05/4mLZdjOBwHfWrL1cH9NA+UHagTAYeNzbldBpYHH d73EokD7sQWqKVZjyFx3+gt4OFcmlH1oiSLPvYcbTtaz8IvxbD5wAlRORVKk3dPFY7gqUlgS7zCiT AbNORGZ+eKp68U8xcBsz4q67EKtQL6cp6IzTVLX1ggIWrZ+RRxmkdqg5AcPIfCIG/aoHT47T1VVn5 mYsTbTS+RRvHQpUoDUlQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syzSu-0000000EE8L-3iKh; Thu, 10 Oct 2024 20:08:52 +0000 Received: from mail-pf1-x44a.google.com ([2607:f8b0:4864:20::44a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxqp-0000000Dq2A-2spD for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:25:30 +0000 Received: by mail-pf1-x44a.google.com with SMTP id d2e1a72fcca58-7190c5e73cdso1782842b3a.0 for ; Thu, 10 Oct 2024 11:25:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584726; x=1729189526; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=H+RZpPtsk3Pe/RqqhWN1oE7ZGzOesPmt7slwmkU09yw=; b=Qjec5b1CwJ9qFJlW4vKnn33w59PK4W9zPFcydX7ZQSuN5EmpIcv79lHfTUVVhozZOs xhYIm58mnWXsUk2OpErOSahqb/ggw6yIcx6eWAgTsaOV+112SciknFzibBcZVB3lSkD6 UfnSN8hm21A7i4UvrvshfZ4Rjp4g1LQuKUmwfXiqDVoacgirXDTYJ4/5bXcw7T47czT0 5vNPZCH3B6swmxQhV1DEi8N3Qn8EaFYT10ZKoiyEkhOaTeWlFf2ocOBVtL+NORRkUTzz sXkQaXKAtNNNW/wNu8MGtO4ATj8rSFarU89qqAWxDNcP+QjhDL92o7YFs4Kg/sl02I7o oJWw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584726; x=1729189526; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=H+RZpPtsk3Pe/RqqhWN1oE7ZGzOesPmt7slwmkU09yw=; b=ASXy8pAKKF1OG5h+xH1rj9HlrBNtgL2RJbOevALIZXkX4WK/toPbCX//qoP+kC1XFW yZQOhyW9gXOL73okowZXnZmxjyXvwdQUOlSLYrlP0+4QYsq1CZAiSiAAqGitXB8nAED1 vrbmnTgD9+JyQknY0fEjtQomyOVAyD+Bejnjb8aJPhMpG/YpNDGQY6g4dCVQugYOYTGE xvv5pDKS5typ1N2hv9uycatfBEDbLO2nyQvB6PA/++8lmd7cxwPggKfEbYHf6RUNvRh/ rQWOKeIuNGF8lGjf9OFU7c/ev8HmAoNU5cF3bK9SJj2jum89bdkrWklBQez+KVxcEQKX a1oQ== X-Forwarded-Encrypted: i=1; AJvYcCUrcOLV+qgsYcmG2rZv/1unOwdmqEpcAqFnkUhSrevTak7dG2/nyHVsqwAE4SAXA/mKYok5RMAmTx73Az+WosSp@lists.infradead.org X-Gm-Message-State: AOJu0YzDQrudl8ehLvdUxIfkKp2EtsZoc5PilBV3e/Litt5wtzGr8u4X BOr9sCZtPlA1ofAcmrWzngWZnpbcyBrKVANEdTrU42V9h4HkHEKJM2dRuwk94wsvfmKMdk0HWwo GYg== X-Google-Smtp-Source: AGHT+IEgY3dn5eo0Xr6pp3rAg0uVd1WuwbDag/Dk23UVvT7MMuxyKVAtWc2T42TlMp1H8D9GJ1Oltmo921o= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:6a00:8c17:b0:71d:f452:ee99 with SMTP id d2e1a72fcca58-71e1dbcac44mr6866b3a.3.1728584725637; Thu, 10 Oct 2024 11:25:25 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:18 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-17-seanjc@google.com> Subject: [PATCH v13 16/85] KVM: Replace "async" pointer in gfn=>pfn with "no_wait" and error code From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112527_835732_5749C9AE X-CRM114-Status: GOOD ( 22.30 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: David Stevens Add a pfn error code to communicate that hva_to_pfn() failed because I/O was needed and disallowed, and convert @async to a constant @no_wait boolean. This will allow eliminating the @no_wait param by having callers pass in FOLL_NOWAIT along with other FOLL_* flags. Tested-by: Alex Bennée Signed-off-by: David Stevens Co-developed-by: Sean Christopherson Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/mmu.c | 18 +++++++++++------- include/linux/kvm_host.h | 3 ++- virt/kvm/kvm_main.c | 27 ++++++++++++++------------- virt/kvm/kvm_mm.h | 2 +- virt/kvm/pfncache.c | 4 ++-- 5 files changed, 30 insertions(+), 24 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 0e235f276ee5..fa8f3fb7c14b 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4374,17 +4374,21 @@ static int kvm_faultin_pfn_private(struct kvm_vcpu *vcpu, static int __kvm_faultin_pfn(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault) { - bool async; - if (fault->is_private) return kvm_faultin_pfn_private(vcpu, fault); - async = false; - fault->pfn = __gfn_to_pfn_memslot(fault->slot, fault->gfn, false, &async, + fault->pfn = __gfn_to_pfn_memslot(fault->slot, fault->gfn, false, true, fault->write, &fault->map_writable, &fault->hva); - if (!async) - return RET_PF_CONTINUE; /* *pfn has correct page already */ + + /* + * If resolving the page failed because I/O is needed to fault-in the + * page, then either set up an asynchronous #PF to do the I/O, or if + * doing an async #PF isn't possible, retry with I/O allowed. All + * other failures are terminal, i.e. retrying won't help. + */ + if (fault->pfn != KVM_PFN_ERR_NEEDS_IO) + return RET_PF_CONTINUE; if (!fault->prefetch && kvm_can_do_async_pf(vcpu)) { trace_kvm_try_async_get_page(fault->addr, fault->gfn); @@ -4402,7 +4406,7 @@ static int __kvm_faultin_pfn(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault * to wait for IO. Note, gup always bails if it is unable to quickly * get a page and a fatal signal, i.e. SIGKILL, is pending. */ - fault->pfn = __gfn_to_pfn_memslot(fault->slot, fault->gfn, true, NULL, + fault->pfn = __gfn_to_pfn_memslot(fault->slot, fault->gfn, true, true, fault->write, &fault->map_writable, &fault->hva); return RET_PF_CONTINUE; diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 2faafc7a56ae..071a0a1f1c60 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -97,6 +97,7 @@ #define KVM_PFN_ERR_HWPOISON (KVM_PFN_ERR_MASK + 1) #define KVM_PFN_ERR_RO_FAULT (KVM_PFN_ERR_MASK + 2) #define KVM_PFN_ERR_SIGPENDING (KVM_PFN_ERR_MASK + 3) +#define KVM_PFN_ERR_NEEDS_IO (KVM_PFN_ERR_MASK + 4) /* * error pfns indicate that the gfn is in slot but faild to @@ -1233,7 +1234,7 @@ kvm_pfn_t gfn_to_pfn_prot(struct kvm *kvm, gfn_t gfn, bool write_fault, bool *writable); kvm_pfn_t gfn_to_pfn_memslot(const struct kvm_memory_slot *slot, gfn_t gfn); kvm_pfn_t __gfn_to_pfn_memslot(const struct kvm_memory_slot *slot, gfn_t gfn, - bool interruptible, bool *async, + bool interruptible, bool no_wait, bool write_fault, bool *writable, hva_t *hva); void kvm_release_pfn_clean(kvm_pfn_t pfn); diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 87f81e74cbc0..dd5839abef6c 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -2778,7 +2778,7 @@ static bool hva_to_pfn_fast(unsigned long addr, bool write_fault, * The slow path to get the pfn of the specified host virtual address, * 1 indicates success, -errno is returned if error is detected. */ -static int hva_to_pfn_slow(unsigned long addr, bool *async, bool write_fault, +static int hva_to_pfn_slow(unsigned long addr, bool no_wait, bool write_fault, bool interruptible, bool *writable, kvm_pfn_t *pfn) { /* @@ -2801,7 +2801,7 @@ static int hva_to_pfn_slow(unsigned long addr, bool *async, bool write_fault, if (write_fault) flags |= FOLL_WRITE; - if (async) + if (no_wait) flags |= FOLL_NOWAIT; if (interruptible) flags |= FOLL_INTERRUPTIBLE; @@ -2912,8 +2912,8 @@ static int hva_to_pfn_remapped(struct vm_area_struct *vma, * Pin guest page in memory and return its pfn. * @addr: host virtual address which maps memory to the guest * @interruptible: whether the process can be interrupted by non-fatal signals - * @async: whether this function need to wait IO complete if the - * host page is not in the memory + * @no_wait: whether or not this function need to wait IO complete if the + * host page is not in the memory * @write_fault: whether we should get a writable host page * @writable: whether it allows to map a writable host page for !@write_fault * @@ -2922,7 +2922,7 @@ static int hva_to_pfn_remapped(struct vm_area_struct *vma, * 2): @write_fault = false && @writable, @writable will tell the caller * whether the mapping is writable. */ -kvm_pfn_t hva_to_pfn(unsigned long addr, bool interruptible, bool *async, +kvm_pfn_t hva_to_pfn(unsigned long addr, bool interruptible, bool no_wait, bool write_fault, bool *writable) { struct vm_area_struct *vma; @@ -2934,7 +2934,7 @@ kvm_pfn_t hva_to_pfn(unsigned long addr, bool interruptible, bool *async, if (hva_to_pfn_fast(addr, write_fault, writable, &pfn)) return pfn; - npages = hva_to_pfn_slow(addr, async, write_fault, interruptible, + npages = hva_to_pfn_slow(addr, no_wait, write_fault, interruptible, writable, &pfn); if (npages == 1) return pfn; @@ -2956,16 +2956,17 @@ kvm_pfn_t hva_to_pfn(unsigned long addr, bool interruptible, bool *async, if (r < 0) pfn = KVM_PFN_ERR_FAULT; } else { - if (async && vma_is_valid(vma, write_fault)) - *async = true; - pfn = KVM_PFN_ERR_FAULT; + if (no_wait && vma_is_valid(vma, write_fault)) + pfn = KVM_PFN_ERR_NEEDS_IO; + else + pfn = KVM_PFN_ERR_FAULT; } mmap_read_unlock(current->mm); return pfn; } kvm_pfn_t __gfn_to_pfn_memslot(const struct kvm_memory_slot *slot, gfn_t gfn, - bool interruptible, bool *async, + bool interruptible, bool no_wait, bool write_fault, bool *writable, hva_t *hva) { unsigned long addr = __gfn_to_hva_many(slot, gfn, NULL, write_fault); @@ -2987,21 +2988,21 @@ kvm_pfn_t __gfn_to_pfn_memslot(const struct kvm_memory_slot *slot, gfn_t gfn, writable = NULL; } - return hva_to_pfn(addr, interruptible, async, write_fault, writable); + return hva_to_pfn(addr, interruptible, no_wait, write_fault, writable); } EXPORT_SYMBOL_GPL(__gfn_to_pfn_memslot); kvm_pfn_t gfn_to_pfn_prot(struct kvm *kvm, gfn_t gfn, bool write_fault, bool *writable) { - return __gfn_to_pfn_memslot(gfn_to_memslot(kvm, gfn), gfn, false, NULL, + return __gfn_to_pfn_memslot(gfn_to_memslot(kvm, gfn), gfn, false, false, write_fault, writable, NULL); } EXPORT_SYMBOL_GPL(gfn_to_pfn_prot); kvm_pfn_t gfn_to_pfn_memslot(const struct kvm_memory_slot *slot, gfn_t gfn) { - return __gfn_to_pfn_memslot(slot, gfn, false, NULL, true, NULL, NULL); + return __gfn_to_pfn_memslot(slot, gfn, false, false, true, NULL, NULL); } EXPORT_SYMBOL_GPL(gfn_to_pfn_memslot); diff --git a/virt/kvm/kvm_mm.h b/virt/kvm/kvm_mm.h index a3fa86f60d6c..51f3fee4ca3f 100644 --- a/virt/kvm/kvm_mm.h +++ b/virt/kvm/kvm_mm.h @@ -20,7 +20,7 @@ #define KVM_MMU_UNLOCK(kvm) spin_unlock(&(kvm)->mmu_lock) #endif /* KVM_HAVE_MMU_RWLOCK */ -kvm_pfn_t hva_to_pfn(unsigned long addr, bool interruptible, bool *async, +kvm_pfn_t hva_to_pfn(unsigned long addr, bool interruptible, bool no_wait, bool write_fault, bool *writable); #ifdef CONFIG_HAVE_KVM_PFNCACHE diff --git a/virt/kvm/pfncache.c b/virt/kvm/pfncache.c index 58c706a610e5..32dc61f48c81 100644 --- a/virt/kvm/pfncache.c +++ b/virt/kvm/pfncache.c @@ -197,8 +197,8 @@ static kvm_pfn_t hva_to_pfn_retry(struct gfn_to_pfn_cache *gpc) cond_resched(); } - /* We always request a writeable mapping */ - new_pfn = hva_to_pfn(gpc->uhva, false, NULL, true, NULL); + /* We always request a writable mapping */ + new_pfn = hva_to_pfn(gpc->uhva, false, false, true, NULL); if (is_error_noslot_pfn(new_pfn)) goto out_error; From patchwork Thu Oct 10 18:23:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831061 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4ADDAD2444C for ; Thu, 10 Oct 2024 20:10:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:Content-Type:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=5N9i4qcLR1M4w0icoJwy+0qPwifBeFycVmWqg2S4YD0=; b=RR0WsQh9u4zsSQ VzmVCSQrTzOamN1RidwFgod2y5PJIuqILib532fP1JeSrW6JQQT1TffTKB0+1j9RR9/3UgNWzB4Ro OB7YOPWZCJ0QHuv43MtaZpyx0S3U9inxWq0ybC6l557WzgHswt6JPImsNHzvWqQS753MUM1bLECHj Nk8d7/8DY1bJULodD1qD/FdYUrJIZEiolAUkxrGRwJbqwTjAei6Gxdh90vaO8m3/9IGFKrhoy3o9y 7lbqP8rDJLamoT60sSup/tdS8ubSnYTr7K05Am8ge/1Gii3KiHWmr506scIN6c2et/KOhNyGUX+yk SL48OA93OMSl5yTHO9nQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syzUE-0000000EEJ7-1cn9; Thu, 10 Oct 2024 20:10:14 +0000 Received: from mail-pl1-x649.google.com ([2607:f8b0:4864:20::649]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxqs-0000000Dq3j-0YiT for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:25:32 +0000 Received: by mail-pl1-x649.google.com with SMTP id d9443c01a7336-20c7ea6b060so15661865ad.1 for ; Thu, 10 Oct 2024 11:25:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584728; x=1729189528; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=5N9i4qcLR1M4w0icoJwy+0qPwifBeFycVmWqg2S4YD0=; b=uTLUXAD9ftwOAWUh4FjktczO89FGu52aC2rgPp245/lEj1s72YqNR/aIDgmovpnjTw HC3jnAk/7L5Fa0FkPdQPqU2qvNP/FCS9jm5l9QtDKCjIYexrNMlJ6rHK9Qh90mNdQ0gz pQWmLtmO5UuB7V2fnKZL8l7u3utIoX9wgdz/yAcJ39cWN1i86vKz12XVoNvrTXzGldva +W7MUjeaG2WCz28YEVABAqFpH18qx4ouR3U+t7I7QQtA6vEwM/KTV5fdtXtHIk2RZ3nO 0lFQqywKjsqrELmkNAOSZ4dPDOMCodBq3cSpkUx60Gsb4fkH4CB5l/Bi/Eiawj8f3uVP IPLw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584728; x=1729189528; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=5N9i4qcLR1M4w0icoJwy+0qPwifBeFycVmWqg2S4YD0=; b=XZxspBzGwetPQ4PPoEmOxwz9h3ndo+txpcFVxL+2iSEhaHc1DqqCAyL5hWnNn6rfrG ZKo5dtxiu5j7/06zncfIfLxR+aVQsjJmmz9nNAuRM6qP5FxtLM9GYiMjO9vb1GVqmqbh pqLjaIUwZYTstql2KpgZnexA4dbEACoFQXp79o6gag9aW1e553xrydCVG1H4iZfAvT8H ggDQHtlUA/EVspo6FvuGSTcwX8/V/e/PEdmIjSIfuBSUZYbof03IZh7reHLy44kmqjLM yCug8opVsGpQbQ/Rjmyf1kwjtr+ZvQtbHihlW/MeDMh4ZHszWU39bSQAEHYGSGaKEAl5 vjVA== X-Forwarded-Encrypted: i=1; AJvYcCUyyGpSPmxllEVHdKP6JXE4ARBamprKjOBRmqvxeYxtNodE2reS0zAuYU3p7WgRJ9++k5C625dP9xEQFso4s1tg@lists.infradead.org X-Gm-Message-State: AOJu0YypbA1AKyDnV+QhFG7sheBplsbzArR2K6We2FFal3LixuLeZApp M9xNFPRf0vw/2sQqUoadV70lagXXJ9bnhzPLoTaX0MvscxSDGVAlYn5kqiYs5sEO3Tsdj0PaLGw BJA== X-Google-Smtp-Source: AGHT+IHx3wOCIkhIacFyS0KkUCfogOtyOnmQIp9NOdA3rn83blMwYV/7exbhoH/uHIfky6alS3vCfOjmfpM= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:902:d502:b0:20b:7ece:3225 with SMTP id d9443c01a7336-20c6358fdd3mr1169885ad.0.1728584727745; Thu, 10 Oct 2024 11:25:27 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:19 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-18-seanjc@google.com> Subject: [PATCH v13 17/85] KVM: x86/mmu: Drop kvm_page_fault.hva, i.e. don't track intermediate hva From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112530_379552_54376496 X-CRM114-Status: GOOD ( 12.90 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Remove kvm_page_fault.hva as it is never read, only written. This will allow removing the @hva param from __gfn_to_pfn_memslot(). Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/mmu.c | 5 ++--- arch/x86/kvm/mmu/mmu_internal.h | 2 -- 2 files changed, 2 insertions(+), 5 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index fa8f3fb7c14b..c67228b46bd5 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -3294,7 +3294,6 @@ static int kvm_handle_noslot_fault(struct kvm_vcpu *vcpu, fault->slot = NULL; fault->pfn = KVM_PFN_NOSLOT; fault->map_writable = false; - fault->hva = KVM_HVA_ERR_BAD; /* * If MMIO caching is disabled, emulate immediately without @@ -4379,7 +4378,7 @@ static int __kvm_faultin_pfn(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault fault->pfn = __gfn_to_pfn_memslot(fault->slot, fault->gfn, false, true, fault->write, &fault->map_writable, - &fault->hva); + NULL); /* * If resolving the page failed because I/O is needed to fault-in the @@ -4408,7 +4407,7 @@ static int __kvm_faultin_pfn(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault */ fault->pfn = __gfn_to_pfn_memslot(fault->slot, fault->gfn, true, true, fault->write, &fault->map_writable, - &fault->hva); + NULL); return RET_PF_CONTINUE; } diff --git a/arch/x86/kvm/mmu/mmu_internal.h b/arch/x86/kvm/mmu/mmu_internal.h index 4da83544c4e1..633aedec3c2e 100644 --- a/arch/x86/kvm/mmu/mmu_internal.h +++ b/arch/x86/kvm/mmu/mmu_internal.h @@ -238,7 +238,6 @@ struct kvm_page_fault { /* Outputs of kvm_faultin_pfn. */ unsigned long mmu_seq; kvm_pfn_t pfn; - hva_t hva; bool map_writable; /* @@ -313,7 +312,6 @@ static inline int kvm_mmu_do_page_fault(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa, .is_private = err & PFERR_PRIVATE_ACCESS, .pfn = KVM_PFN_ERR_FAULT, - .hva = KVM_HVA_ERR_BAD, }; int r; From patchwork Thu Oct 10 18:23:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830858 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E2402CFC5F2 for ; Thu, 10 Oct 2024 18:46:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:Content-Type:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=KxFe7qyo4eHIzhTxczhq6OF5aBNVD8CbU/vq6HHGwtQ=; b=XGtZJMT3G95XPo B8WP7GM/ePblS1iaUxk/HCIHNZdyRdEmemtUyg/Gr+hS4afUMYBF2459uvlXrzJKg7mtBU6d0mkxf KqxwGWgoCdm/UDXg98gfvZpDD1/haH6SCnwfaqU5IstDZ8GWoSxzFxFh3mj+fwgSQnVTdqbB5EHtK ZewcdP4LtTEMx1BR1Qqu5yqgcEtBVg67XP7nJ2sRQUOSgb3e5VmAH+mo3LPFNRb/au9W88EiuyIqP sf8XLrxRpGOqyertr7i6mcm1CDBrfbYzmKEMjELGwbGSkkhMr+a5K99VmWhsDgrN1CUOHFJazsr3Y 67dqazUGVniRHElTcCzQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syyB6-0000000DyxI-02lk; Thu, 10 Oct 2024 18:46:24 +0000 Received: from mail-pj1-x1049.google.com ([2607:f8b0:4864:20::1049]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxqu-0000000Dq6V-2fot for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:25:34 +0000 Received: by mail-pj1-x1049.google.com with SMTP id 98e67ed59e1d1-2e29facda92so1239608a91.1 for ; Thu, 10 Oct 2024 11:25:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584731; x=1729189531; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=KxFe7qyo4eHIzhTxczhq6OF5aBNVD8CbU/vq6HHGwtQ=; b=awdZis4GuboE2s97liIxH2HBWuUUAZFAC1GR5LT8PbnEmXHC9BSez0m6ZP3mLwxKXM lQ1MevXxzALb1FlxY1y77fDO1z/yeM3kTyp/oML38Xu6q7A/SmpXpmWj9OE6Ic2oUvpP G6PcEUHqZ9s82F1e4DgelmyGmWvUWCcrjUMNuBi8hYHZqkdUNNzyDmqRS5imgnmSEsOe Hwlj5C1c37+lD1ZhDMZabp8YATzJcOtF5IR98HGQw71sfkFR7KqlyyliLqBRHmcynxCc mZz4ZlNs/pnaJQjzxt1mqttaEDYAWQ4/MDsqxIKf+BKKwPrQIijYa3xjhuw1Sz7YKV2J 6Ajg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584731; x=1729189531; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=KxFe7qyo4eHIzhTxczhq6OF5aBNVD8CbU/vq6HHGwtQ=; b=Fvxv3z1gnR2cB8hGe5v73+PMyLvTaGY9lST7CMaqKsnyz0rMNyMG692vzUrBt7y/0i gj9n0D6D4e8/Q1Ewsfh4DKDZvTCWjqDTAWkBHL5J24qw4VdrvF5LW8uubS/2oS7tMI3u 6uu2fOlRqK6IwbSoKwwLqMDZn3ymhiqQkZfLwiE1u3QVUuFQ/f4U+m+GoZZsGvcBRMXp F2WZgaJ/zWP8eikHHlyDoyonY1AYQjeVNH1u7KNzqSBWCROLzo4lniZtOmGpV8iND93Z QiKhfwdPVHPppcfDtOzCt+3oHd25i3y/Jpho5aX1lYCxMk0G20aOXrvpxdNFCmh0mwTC +a/Q== X-Forwarded-Encrypted: i=1; AJvYcCUxJzkzsTEWc4iifZJNRxFJ+VJgXIbpjTN24WbXweupJ5BBvJ/Vx3hLcXG0puj7/hH2WKdUwzMerU7YtS9z6dTL@lists.infradead.org X-Gm-Message-State: AOJu0YzJWnlxuARTjFILvtKVWgCA1Qet8hqwoADcDK251e8sgPGkRR8C mR8DjTJlXGFJhATaTHGdvMT0H12Os3s81FdZrOb1llQ2VdXLwUSPClViE+VSuuLpuuhdN3Da9u3 moQ== X-Google-Smtp-Source: AGHT+IGFsZ8G+8oqHMXYnd6l/b9oSo3/s957bEBlaeuh+YsQIWv/Tzia2RQhXQw4+rIMPn/3epGKWmgCn2Q= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:90a:5107:b0:2e2:ca3e:10fe with SMTP id 98e67ed59e1d1-2e2f0f88a54mr20a91.8.1728584729847; Thu, 10 Oct 2024 11:25:29 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:20 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-19-seanjc@google.com> Subject: [PATCH v13 18/85] KVM: Drop unused "hva" pointer from __gfn_to_pfn_memslot() From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112532_799682_B209ED9A X-CRM114-Status: GOOD ( 13.22 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Drop @hva from __gfn_to_pfn_memslot() now that all callers pass NULL. No functional change intended. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/arm64/kvm/mmu.c | 2 +- arch/powerpc/kvm/book3s_64_mmu_hv.c | 2 +- arch/powerpc/kvm/book3s_64_mmu_radix.c | 2 +- arch/x86/kvm/mmu/mmu.c | 6 ++---- include/linux/kvm_host.h | 2 +- virt/kvm/kvm_main.c | 9 +++------ 6 files changed, 9 insertions(+), 14 deletions(-) diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c index a6e62cc9015c..dd221587fcca 100644 --- a/arch/arm64/kvm/mmu.c +++ b/arch/arm64/kvm/mmu.c @@ -1570,7 +1570,7 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, mmap_read_unlock(current->mm); pfn = __gfn_to_pfn_memslot(memslot, gfn, false, NULL, - write_fault, &writable, NULL); + write_fault, &writable); if (pfn == KVM_PFN_ERR_HWPOISON) { kvm_send_hwpoison_signal(hva, vma_shift); return 0; diff --git a/arch/powerpc/kvm/book3s_64_mmu_hv.c b/arch/powerpc/kvm/book3s_64_mmu_hv.c index 8cd02ca4b1b8..2f1d58984b41 100644 --- a/arch/powerpc/kvm/book3s_64_mmu_hv.c +++ b/arch/powerpc/kvm/book3s_64_mmu_hv.c @@ -614,7 +614,7 @@ int kvmppc_book3s_hv_page_fault(struct kvm_vcpu *vcpu, } else { /* Call KVM generic code to do the slow-path check */ pfn = __gfn_to_pfn_memslot(memslot, gfn, false, NULL, - writing, &write_ok, NULL); + writing, &write_ok); if (is_error_noslot_pfn(pfn)) return -EFAULT; page = NULL; diff --git a/arch/powerpc/kvm/book3s_64_mmu_radix.c b/arch/powerpc/kvm/book3s_64_mmu_radix.c index 26a969e935e3..8304b6f8fe45 100644 --- a/arch/powerpc/kvm/book3s_64_mmu_radix.c +++ b/arch/powerpc/kvm/book3s_64_mmu_radix.c @@ -853,7 +853,7 @@ int kvmppc_book3s_instantiate_page(struct kvm_vcpu *vcpu, /* Call KVM generic code to do the slow-path check */ pfn = __gfn_to_pfn_memslot(memslot, gfn, false, NULL, - writing, upgrade_p, NULL); + writing, upgrade_p); if (is_error_noslot_pfn(pfn)) return -EFAULT; page = NULL; diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index c67228b46bd5..28f2b842d6ca 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4377,8 +4377,7 @@ static int __kvm_faultin_pfn(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault return kvm_faultin_pfn_private(vcpu, fault); fault->pfn = __gfn_to_pfn_memslot(fault->slot, fault->gfn, false, true, - fault->write, &fault->map_writable, - NULL); + fault->write, &fault->map_writable); /* * If resolving the page failed because I/O is needed to fault-in the @@ -4406,8 +4405,7 @@ static int __kvm_faultin_pfn(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault * get a page and a fatal signal, i.e. SIGKILL, is pending. */ fault->pfn = __gfn_to_pfn_memslot(fault->slot, fault->gfn, true, true, - fault->write, &fault->map_writable, - NULL); + fault->write, &fault->map_writable); return RET_PF_CONTINUE; } diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 071a0a1f1c60..cbc7b9c04c14 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -1235,7 +1235,7 @@ kvm_pfn_t gfn_to_pfn_prot(struct kvm *kvm, gfn_t gfn, bool write_fault, kvm_pfn_t gfn_to_pfn_memslot(const struct kvm_memory_slot *slot, gfn_t gfn); kvm_pfn_t __gfn_to_pfn_memslot(const struct kvm_memory_slot *slot, gfn_t gfn, bool interruptible, bool no_wait, - bool write_fault, bool *writable, hva_t *hva); + bool write_fault, bool *writable); void kvm_release_pfn_clean(kvm_pfn_t pfn); void kvm_release_pfn_dirty(kvm_pfn_t pfn); diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index dd5839abef6c..10071f31b2ca 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -2967,13 +2967,10 @@ kvm_pfn_t hva_to_pfn(unsigned long addr, bool interruptible, bool no_wait, kvm_pfn_t __gfn_to_pfn_memslot(const struct kvm_memory_slot *slot, gfn_t gfn, bool interruptible, bool no_wait, - bool write_fault, bool *writable, hva_t *hva) + bool write_fault, bool *writable) { unsigned long addr = __gfn_to_hva_many(slot, gfn, NULL, write_fault); - if (hva) - *hva = addr; - if (kvm_is_error_hva(addr)) { if (writable) *writable = false; @@ -2996,13 +2993,13 @@ kvm_pfn_t gfn_to_pfn_prot(struct kvm *kvm, gfn_t gfn, bool write_fault, bool *writable) { return __gfn_to_pfn_memslot(gfn_to_memslot(kvm, gfn), gfn, false, false, - write_fault, writable, NULL); + write_fault, writable); } EXPORT_SYMBOL_GPL(gfn_to_pfn_prot); kvm_pfn_t gfn_to_pfn_memslot(const struct kvm_memory_slot *slot, gfn_t gfn) { - return __gfn_to_pfn_memslot(slot, gfn, false, false, true, NULL, NULL); + return __gfn_to_pfn_memslot(slot, gfn, false, false, true, NULL); } EXPORT_SYMBOL_GPL(gfn_to_pfn_memslot); From patchwork Thu Oct 10 18:23:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831136 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 8BAB4D24451 for ; Thu, 10 Oct 2024 21:01:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:Content-Type:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=GUnoK8hes+aZNWzzso7nvh51KinEVhA6U4mspQ9USyg=; b=3Ivy8iPfJmuwqF 5EJMUx8P37Z4zngj+CzSTXsvY1bi4jusSqyN8kzxVXPNtp3D0cvq5tnXVnssrjrD+y198gLwYh91A /3NjTd0hSeB2+3lW1828W0qjf6em3e4ZAGarzaJ85L7VzbKfIrkKCZKVN+/hZfP6+1RZFsSlWXwk8 ZIb9+DVFWjlBAa/jszqpcWBX6FIqAQ1VJjKWsenxLJhgZMAirJgpHno32BhgrnmqzrS6Vah68JY5L pEuKHg19RdKmFaKp5glZ6o581MuIh03o7xKaL2Xekj98tgywRIhAPR9xTRJaENdaPf6Sb94GYJuqc TU1uJuYfnmBM78suO5oQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1sz0Hj-0000000ELi3-2Wz9; Thu, 10 Oct 2024 21:01:23 +0000 Received: from mail-yw1-x1149.google.com ([2607:f8b0:4864:20::1149]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxqw-0000000Dq80-2dH8 for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:25:37 +0000 Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-6e31d9c8efcso25656197b3.0 for ; Thu, 10 Oct 2024 11:25:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584733; x=1729189533; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=GUnoK8hes+aZNWzzso7nvh51KinEVhA6U4mspQ9USyg=; b=QSA5Sg21GOvgjD1Ji6QRzFZ+R3PP2MPakGWPOI/tPT+1CrRvbx7U8z/Sd1b+fnqA1Q A0AGN19YkH/fcahh2w+MAOQ5qRjYuS47eqno2btw34H/K2nX8nOBoKOz/cGEAqBE9smh CwybwU7CMSzF+dGRGdbQrsgrn0xWROTjrGsCh1QntjZ5OJc9TYkDEDJOpd3AJrIM5x3K 9ulB5CWefMEweMtJTo88ludJG4EwG25DjsKKZ7LdZhrPq8zrlUiAxCon5NbZZrBUmVrS eWGebW8A5hj0X6W7yGnBGdOB59w2+X6x/xS7t1zsL8gPrO4kWpWjcDCH59ZBXYsVwpEt vxyg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584733; x=1729189533; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=GUnoK8hes+aZNWzzso7nvh51KinEVhA6U4mspQ9USyg=; b=KV64Zrs8DbxChy6UR3Sa9jzHriVnbKyAOqjWKeyUqwsESBLCfYgeJ1CqmvA8E/RtRd sR2TNmE6q1eYrxsuMgIDGx4TFXlHG5DNJAmPn9kR8+QEjnpzfkW99SuF7gmV5eI9fGVm 9n1Gcb3kf9YxT+m8Nj83VZrK5mLTqogT3Ch7PCmpxMmw8aUPjR21guNVAZ9KHX91bS2Y wqKBSs79OMWppH48kAXhJjlK55RxIJrSA69p0bboz0sDXTSoWjKAvAzVfKQX8oq1rmgV MEFv7qV3zx/KV8N+pKV1rqJhrIV4ok4RXkiT+lFVLLWSfqbbvUD9k/P3CRal1XgvlFgl BuJg== X-Forwarded-Encrypted: i=1; AJvYcCWg93Hmb4BNLDBDD7UJTcMlNeumvq4qJj22DdLBijwtmsX76w8PxMOOJmOt77zxQVOnPffUSApjrGKBhWwCPVCI@lists.infradead.org X-Gm-Message-State: AOJu0YwhYot3vChybalJsvNCwtJN++pnfShLciXAfKyb1tH8SAYH8mM3 zrHAnnaRYMddmMShLCRU62YvRHvK0WNjr28PdROT1EfDbKb4PQ22JLV7YgeQJjWBbxRAku4G3CH kzg== X-Google-Smtp-Source: AGHT+IHJGx3vYA/ciscOnZoyMGBDqN0A5fNCXYXCqQ4jImN8v4VZ+ud/0qiiFEbLMXTR4U3Rg3+GOt3HIs4= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:690c:2c01:b0:6db:d257:b98 with SMTP id 00721157ae682-6e32215253bmr563917b3.3.1728584733164; Thu, 10 Oct 2024 11:25:33 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:21 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-20-seanjc@google.com> Subject: [PATCH v13 19/85] KVM: Introduce kvm_follow_pfn() to eventually replace "gfn_to_pfn" APIs From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112534_876085_8D182742 X-CRM114-Status: GOOD ( 26.50 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: David Stevens Introduce kvm_follow_pfn() to eventually supplant the various "gfn_to_pfn" APIs, albeit by adding more wrappers. The primary motivation of the new helper is to pass a structure instead of an ever changing set of parameters, e.g. so that tweaking the behavior, inputs, and/or outputs of the "to pfn" helpers doesn't require churning half of KVM. In the more distant future, the APIs exposed to arch code could also follow suit, e.g. by adding something akin to x86's "struct kvm_page_fault" when faulting in guest memory. But for now, the goal is purely to clean up KVM's "internal" MMU code. As part of the conversion, replace the write_fault, interruptible, and no-wait boolean flags with FOLL_WRITE, FOLL_INTERRUPTIBLE, and FOLL_NOWAIT respectively. Collecting the various FOLL_* flags into a single field will again ease the pain of passing new flags. Tested-by: Alex Bennée Signed-off-by: David Stevens Co-developed-by: Sean Christopherson Signed-off-by: Sean Christopherson --- virt/kvm/kvm_main.c | 162 +++++++++++++++++++++++--------------------- virt/kvm/kvm_mm.h | 20 +++++- virt/kvm/pfncache.c | 9 ++- 3 files changed, 109 insertions(+), 82 deletions(-) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 10071f31b2ca..52629ac26119 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -2750,8 +2750,7 @@ unsigned long kvm_vcpu_gfn_to_hva_prot(struct kvm_vcpu *vcpu, gfn_t gfn, bool *w * The fast path to get the writable pfn which will be stored in @pfn, * true indicates success, otherwise false is returned. */ -static bool hva_to_pfn_fast(unsigned long addr, bool write_fault, - bool *writable, kvm_pfn_t *pfn) +static bool hva_to_pfn_fast(struct kvm_follow_pfn *kfp, kvm_pfn_t *pfn) { struct page *page[1]; @@ -2760,14 +2759,13 @@ static bool hva_to_pfn_fast(unsigned long addr, bool write_fault, * or the caller allows to map a writable pfn for a read fault * request. */ - if (!(write_fault || writable)) + if (!((kfp->flags & FOLL_WRITE) || kfp->map_writable)) return false; - if (get_user_page_fast_only(addr, FOLL_WRITE, page)) { + if (get_user_page_fast_only(kfp->hva, FOLL_WRITE, page)) { *pfn = page_to_pfn(page[0]); - - if (writable) - *writable = true; + if (kfp->map_writable) + *kfp->map_writable = true; return true; } @@ -2778,8 +2776,7 @@ static bool hva_to_pfn_fast(unsigned long addr, bool write_fault, * The slow path to get the pfn of the specified host virtual address, * 1 indicates success, -errno is returned if error is detected. */ -static int hva_to_pfn_slow(unsigned long addr, bool no_wait, bool write_fault, - bool interruptible, bool *writable, kvm_pfn_t *pfn) +static int hva_to_pfn_slow(struct kvm_follow_pfn *kfp, kvm_pfn_t *pfn) { /* * When a VCPU accesses a page that is not mapped into the secondary @@ -2792,34 +2789,30 @@ static int hva_to_pfn_slow(unsigned long addr, bool no_wait, bool write_fault, * Note that get_user_page_fast_only() and FOLL_WRITE for now * implicitly honor NUMA hinting faults and don't need this flag. */ - unsigned int flags = FOLL_HWPOISON | FOLL_HONOR_NUMA_FAULT; - struct page *page; + unsigned int flags = FOLL_HWPOISON | FOLL_HONOR_NUMA_FAULT | kfp->flags; + struct page *page, *wpage; int npages; - if (writable) - *writable = write_fault; - - if (write_fault) - flags |= FOLL_WRITE; - if (no_wait) - flags |= FOLL_NOWAIT; - if (interruptible) - flags |= FOLL_INTERRUPTIBLE; - - npages = get_user_pages_unlocked(addr, 1, &page, flags); + npages = get_user_pages_unlocked(kfp->hva, 1, &page, flags); if (npages != 1) return npages; + if (!kfp->map_writable) + goto out; + + if (kfp->flags & FOLL_WRITE) { + *kfp->map_writable = true; + goto out; + } + /* map read fault as writable if possible */ - if (unlikely(!write_fault) && writable) { - struct page *wpage; - - if (get_user_page_fast_only(addr, FOLL_WRITE, &wpage)) { - *writable = true; - put_page(page); - page = wpage; - } + if (get_user_page_fast_only(kfp->hva, FOLL_WRITE, &wpage)) { + *kfp->map_writable = true; + put_page(page); + page = wpage; } + +out: *pfn = page_to_pfn(page); return npages; } @@ -2846,10 +2839,10 @@ static int kvm_try_get_pfn(kvm_pfn_t pfn) } static int hva_to_pfn_remapped(struct vm_area_struct *vma, - unsigned long addr, bool write_fault, - bool *writable, kvm_pfn_t *p_pfn) + struct kvm_follow_pfn *kfp, kvm_pfn_t *p_pfn) { - struct follow_pfnmap_args args = { .vma = vma, .address = addr }; + struct follow_pfnmap_args args = { .vma = vma, .address = kfp->hva }; + bool write_fault = kfp->flags & FOLL_WRITE; kvm_pfn_t pfn; int r; @@ -2860,7 +2853,7 @@ static int hva_to_pfn_remapped(struct vm_area_struct *vma, * not call the fault handler, so do it here. */ bool unlocked = false; - r = fixup_user_fault(current->mm, addr, + r = fixup_user_fault(current->mm, kfp->hva, (write_fault ? FAULT_FLAG_WRITE : 0), &unlocked); if (unlocked) @@ -2878,8 +2871,8 @@ static int hva_to_pfn_remapped(struct vm_area_struct *vma, goto out; } - if (writable) - *writable = args.writable; + if (kfp->map_writable) + *kfp->map_writable = args.writable; pfn = args.pfn; /* @@ -2908,22 +2901,7 @@ static int hva_to_pfn_remapped(struct vm_area_struct *vma, return r; } -/* - * Pin guest page in memory and return its pfn. - * @addr: host virtual address which maps memory to the guest - * @interruptible: whether the process can be interrupted by non-fatal signals - * @no_wait: whether or not this function need to wait IO complete if the - * host page is not in the memory - * @write_fault: whether we should get a writable host page - * @writable: whether it allows to map a writable host page for !@write_fault - * - * The function will map a writable host page for these two cases: - * 1): @write_fault = true - * 2): @write_fault = false && @writable, @writable will tell the caller - * whether the mapping is writable. - */ -kvm_pfn_t hva_to_pfn(unsigned long addr, bool interruptible, bool no_wait, - bool write_fault, bool *writable) +kvm_pfn_t hva_to_pfn(struct kvm_follow_pfn *kfp) { struct vm_area_struct *vma; kvm_pfn_t pfn; @@ -2931,11 +2909,10 @@ kvm_pfn_t hva_to_pfn(unsigned long addr, bool interruptible, bool no_wait, might_sleep(); - if (hva_to_pfn_fast(addr, write_fault, writable, &pfn)) + if (hva_to_pfn_fast(kfp, &pfn)) return pfn; - npages = hva_to_pfn_slow(addr, no_wait, write_fault, interruptible, - writable, &pfn); + npages = hva_to_pfn_slow(kfp, &pfn); if (npages == 1) return pfn; if (npages == -EINTR || npages == -EAGAIN) @@ -2945,18 +2922,19 @@ kvm_pfn_t hva_to_pfn(unsigned long addr, bool interruptible, bool no_wait, mmap_read_lock(current->mm); retry: - vma = vma_lookup(current->mm, addr); + vma = vma_lookup(current->mm, kfp->hva); if (vma == NULL) pfn = KVM_PFN_ERR_FAULT; else if (vma->vm_flags & (VM_IO | VM_PFNMAP)) { - r = hva_to_pfn_remapped(vma, addr, write_fault, writable, &pfn); + r = hva_to_pfn_remapped(vma, kfp, &pfn); if (r == -EAGAIN) goto retry; if (r < 0) pfn = KVM_PFN_ERR_FAULT; } else { - if (no_wait && vma_is_valid(vma, write_fault)) + if ((kfp->flags & FOLL_NOWAIT) && + vma_is_valid(vma, kfp->flags & FOLL_WRITE)) pfn = KVM_PFN_ERR_NEEDS_IO; else pfn = KVM_PFN_ERR_FAULT; @@ -2965,41 +2943,69 @@ kvm_pfn_t hva_to_pfn(unsigned long addr, bool interruptible, bool no_wait, return pfn; } +static kvm_pfn_t kvm_follow_pfn(struct kvm_follow_pfn *kfp) +{ + kfp->hva = __gfn_to_hva_many(kfp->slot, kfp->gfn, NULL, + kfp->flags & FOLL_WRITE); + + if (kfp->hva == KVM_HVA_ERR_RO_BAD) + return KVM_PFN_ERR_RO_FAULT; + + if (kvm_is_error_hva(kfp->hva)) + return KVM_PFN_NOSLOT; + + if (memslot_is_readonly(kfp->slot) && kfp->map_writable) { + *kfp->map_writable = false; + kfp->map_writable = NULL; + } + + return hva_to_pfn(kfp); +} + kvm_pfn_t __gfn_to_pfn_memslot(const struct kvm_memory_slot *slot, gfn_t gfn, bool interruptible, bool no_wait, bool write_fault, bool *writable) { - unsigned long addr = __gfn_to_hva_many(slot, gfn, NULL, write_fault); - - if (kvm_is_error_hva(addr)) { - if (writable) - *writable = false; - - return addr == KVM_HVA_ERR_RO_BAD ? KVM_PFN_ERR_RO_FAULT : - KVM_PFN_NOSLOT; - } - - /* Do not map writable pfn in the readonly memslot. */ - if (writable && memslot_is_readonly(slot)) { - *writable = false; - writable = NULL; - } - - return hva_to_pfn(addr, interruptible, no_wait, write_fault, writable); + struct kvm_follow_pfn kfp = { + .slot = slot, + .gfn = gfn, + .map_writable = writable, + }; + + if (write_fault) + kfp.flags |= FOLL_WRITE; + if (no_wait) + kfp.flags |= FOLL_NOWAIT; + if (interruptible) + kfp.flags |= FOLL_INTERRUPTIBLE; + + return kvm_follow_pfn(&kfp); } EXPORT_SYMBOL_GPL(__gfn_to_pfn_memslot); kvm_pfn_t gfn_to_pfn_prot(struct kvm *kvm, gfn_t gfn, bool write_fault, bool *writable) { - return __gfn_to_pfn_memslot(gfn_to_memslot(kvm, gfn), gfn, false, false, - write_fault, writable); + struct kvm_follow_pfn kfp = { + .slot = gfn_to_memslot(kvm, gfn), + .gfn = gfn, + .flags = write_fault ? FOLL_WRITE : 0, + .map_writable = writable, + }; + + return kvm_follow_pfn(&kfp); } EXPORT_SYMBOL_GPL(gfn_to_pfn_prot); kvm_pfn_t gfn_to_pfn_memslot(const struct kvm_memory_slot *slot, gfn_t gfn) { - return __gfn_to_pfn_memslot(slot, gfn, false, false, true, NULL); + struct kvm_follow_pfn kfp = { + .slot = slot, + .gfn = gfn, + .flags = FOLL_WRITE, + }; + + return kvm_follow_pfn(&kfp); } EXPORT_SYMBOL_GPL(gfn_to_pfn_memslot); diff --git a/virt/kvm/kvm_mm.h b/virt/kvm/kvm_mm.h index 51f3fee4ca3f..d5a215958f06 100644 --- a/virt/kvm/kvm_mm.h +++ b/virt/kvm/kvm_mm.h @@ -20,8 +20,24 @@ #define KVM_MMU_UNLOCK(kvm) spin_unlock(&(kvm)->mmu_lock) #endif /* KVM_HAVE_MMU_RWLOCK */ -kvm_pfn_t hva_to_pfn(unsigned long addr, bool interruptible, bool no_wait, - bool write_fault, bool *writable); + +struct kvm_follow_pfn { + const struct kvm_memory_slot *slot; + const gfn_t gfn; + + unsigned long hva; + + /* FOLL_* flags modifying lookup behavior, e.g. FOLL_WRITE. */ + unsigned int flags; + + /* + * If non-NULL, try to get a writable mapping even for a read fault. + * Set to true if a writable mapping was obtained. + */ + bool *map_writable; +}; + +kvm_pfn_t hva_to_pfn(struct kvm_follow_pfn *kfp); #ifdef CONFIG_HAVE_KVM_PFNCACHE void gfn_to_pfn_cache_invalidate_start(struct kvm *kvm, diff --git a/virt/kvm/pfncache.c b/virt/kvm/pfncache.c index 32dc61f48c81..067daf9ad6ef 100644 --- a/virt/kvm/pfncache.c +++ b/virt/kvm/pfncache.c @@ -159,6 +159,12 @@ static kvm_pfn_t hva_to_pfn_retry(struct gfn_to_pfn_cache *gpc) kvm_pfn_t new_pfn = KVM_PFN_ERR_FAULT; void *new_khva = NULL; unsigned long mmu_seq; + struct kvm_follow_pfn kfp = { + .slot = gpc->memslot, + .gfn = gpa_to_gfn(gpc->gpa), + .flags = FOLL_WRITE, + .hva = gpc->uhva, + }; lockdep_assert_held(&gpc->refresh_lock); @@ -197,8 +203,7 @@ static kvm_pfn_t hva_to_pfn_retry(struct gfn_to_pfn_cache *gpc) cond_resched(); } - /* We always request a writable mapping */ - new_pfn = hva_to_pfn(gpc->uhva, false, false, true, NULL); + new_pfn = hva_to_pfn(&kfp); if (is_error_noslot_pfn(new_pfn)) goto out_error; From patchwork Thu Oct 10 18:23:22 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830859 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A79FFCFC5F3 for ; Thu, 10 Oct 2024 18:49:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:Content-Type:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=LNPEWcjaz+dEwpcBQF1ZcKP3HAxUc/LzxURpjzN7wyY=; b=w4NBikKynckdtQ k/d547lXkMdHQn0FLJbTy4VdvDrIiifFHMl2eSvNYSUKln3IBbNcfu2Grs0dKOKa7BZnT/VCs364x P0MGfVC/qLijzMVK/BUq5YLLQc/NoRoiBcExwHFZ6wMxiPBneKwnKaGpMILHSF0J3dvbdpVRN6k2u 8K1JmDfMajPG0Oas1HqpeYymINaCEpyZHs01QpBd7dl7c9Z1AeMfj6OtCOxRlnbbyI7jtQ0Ofepgf /kaajRiGFC1T/crinMNtGXBc6KLhDGe7SJLNZ5FrjEn0X4rZ5v18VEAQ3YNEfZ/lqOD7TTS1jrs/1 QDys8rmO802aNWRyr4GQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syyDl-0000000DzTY-0t0M; Thu, 10 Oct 2024 18:49:09 +0000 Received: from mail-pj1-x1049.google.com ([2607:f8b0:4864:20::1049]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxqy-0000000Dq9k-15LO for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:25:38 +0000 Received: by mail-pj1-x1049.google.com with SMTP id 98e67ed59e1d1-2e2dcea0f03so997753a91.1 for ; Thu, 10 Oct 2024 11:25:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584735; x=1729189535; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=LNPEWcjaz+dEwpcBQF1ZcKP3HAxUc/LzxURpjzN7wyY=; b=cQVMlHY01P5GUGimI1EwUwV1gMXDWO1YhRNQgRRZRp00tcXkR3VxYVS8SFNZdGUHSM +EnHGdT46sR1K/IGTWajJMmoGBn0sNYOYBl9jKHyXHiRccxDMQz2SiMM8mPBIPgc36qO Po18p81ZhZKkcywvYLdtTn92U0KOy4krcXQi8HumWxBuK6fva1EYc44wJf8gFuc/2fd6 KE22Z93ZbqXtSHgYYN/h+5IQDFU3/qaj+6FgQnJ80jSJFdyR2wG6J4Jnw+f04poLAqTP dsiF1M/Sh525nq3aLtvs5dKB47INji3eZlodpB9Gp1tMy1RepfdgGfZF+37O8j4+KXr8 aD+w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584735; x=1729189535; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=LNPEWcjaz+dEwpcBQF1ZcKP3HAxUc/LzxURpjzN7wyY=; b=O+WF9wIObXGtOSHBZiAdiNZUFEAxvIvtYSckMT5hYgcxxhkaJEaOM/bLH5B11OHu4Q wVeYDiEsYMzdtAsi5dzVBxhX9QU+LcXEPD5Sivn739YkB/Z8CAU2C4QTXOKqikuT66OG EIWzdS48EaOZ8VW/jqoeN2WcB6bLkgZ5rE0LVCcMBKkgl5fHJBbyMLIRDvBZ+QiHKmdZ LEY4ImkwVfHmKqwEFbCgW2UE0TDw9pER1ueGYQTZIR3k8M/Pru8OAoYgmjVKIsM4VHPc QX1LlMgVXMG18e11x7ZvgT5TFG3mp1GbceogLuddk7rpm/yHn8OHtmjleLDV9e5NXrh2 Pr+w== X-Forwarded-Encrypted: i=1; AJvYcCWHaX5wYhi2DjT1hciLF2uBxYmrBWKFibR1Wthgu3aJK5kW4rpCGxV+Grmk9z1sWGdsmvErikjT734h7bG15Yn/@lists.infradead.org X-Gm-Message-State: AOJu0YyR5Xu6n0eNmDbNnjPx896qH4i9LxvLSy9k0xA4sODONySEXe2y mts1Z449dw8R0fDoCzgQ8reohiWZevnoCJE34R68iC7D6zlEPv+9GKitSeNHK2vmRP8+v3945j6 BJA== X-Google-Smtp-Source: AGHT+IEnZEGZphhdHlQgb0iOcJfNRdGKMtxwSRuFd5MkDGpZIOhDcg8ohYIeF4ySSwO5bqKUu8yK4lYjZ+4= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:90a:8595:b0:2e2:d600:3862 with SMTP id 98e67ed59e1d1-2e2f0d9dbfemr89a91.8.1728584735013; Thu, 10 Oct 2024 11:25:35 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:22 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-21-seanjc@google.com> Subject: [PATCH v13 20/85] KVM: Remove pointless sanity check on @map param to kvm_vcpu_(un)map() From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112536_434301_07919371 X-CRM114-Status: GOOD ( 11.58 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Drop kvm_vcpu_{,un}map()'s useless checks on @map being non-NULL. The map is 100% kernel controlled, any caller that passes a NULL pointer is broken and needs to be fixed, i.e. a crash due to a NULL pointer dereference is desirable (though obviously not as desirable as not having a bug in the first place). Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- virt/kvm/kvm_main.c | 6 ------ 1 file changed, 6 deletions(-) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 52629ac26119..c7691bc40389 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -3071,9 +3071,6 @@ int kvm_vcpu_map(struct kvm_vcpu *vcpu, gfn_t gfn, struct kvm_host_map *map) void *hva = NULL; struct page *page = KVM_UNMAPPED_PAGE; - if (!map) - return -EINVAL; - pfn = gfn_to_pfn(vcpu->kvm, gfn); if (is_error_noslot_pfn(pfn)) return -EINVAL; @@ -3101,9 +3098,6 @@ EXPORT_SYMBOL_GPL(kvm_vcpu_map); void kvm_vcpu_unmap(struct kvm_vcpu *vcpu, struct kvm_host_map *map, bool dirty) { - if (!map) - return; - if (!map->hva) return; From patchwork Thu Oct 10 18:23:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830864 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id CC61ECFC5F4 for ; Thu, 10 Oct 2024 18:50:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:Content-Type:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=IMgO/JDFhWAWu7Wkr7+zBWYfApHhI977J8rKw7ML51Y=; b=zGNSb8RJBtPp6Y z9qrDP4v3bMI3BDXDw++e2DgbQ/9XJnjzQhjCfFnkfUwqR8RkHp7NOeIRegEMMcMKY9ZjSJOgYYzZ E8Ely61Ai2FobIt+9t640HBJRbtN51e/fTDAkv2HeUR2aXvHCsLsBGJdaULmN62VwHrW9buIyH3Y/ HLaUSOwVM9ME1/dYJkr0+WOvMN0CS9s40jI/jC/3f53ApQ0cFDjdxZiZ/xTVv5bYSYaxVQGXtubUc mgQijYvcc3GdbVJd4JvBgkbU1KCF3oXmnSh1XfMFtWOi7urWrgnx1mGw9FBbZf49AkY7MFmUo3aUd xrCXN7cTwePT+90kiz9A==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syyF8-0000000DzwI-32gB; Thu, 10 Oct 2024 18:50:34 +0000 Received: from mail-yb1-xb49.google.com ([2607:f8b0:4864:20::b49]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxr0-0000000DqBP-1tf7 for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:25:42 +0000 Received: by mail-yb1-xb49.google.com with SMTP id 3f1490d57ef6-e290b8b69f8so2137090276.2 for ; Thu, 10 Oct 2024 11:25:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584737; x=1729189537; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=IMgO/JDFhWAWu7Wkr7+zBWYfApHhI977J8rKw7ML51Y=; b=hgOyYvQjo5Iks6jmfsYRhhZ653J3E4BLCYk0mKm4K/P7+ujw/kMs5SrTQfCdhu/xHN Db+HLZdiVwPcx0XNx4loznLbHChfVsgKxjMvnZidzMISqKM77GaWqQSFWintuPV3lTQX 5jNj8pMXtm3F6/AV4ag8agFO4lZ7LmFFe8bcXThkRc/3+NJnsaMCjPEnLChuYTVFTwIB CFbpbYCm8WEOCdmDdlAYfY1rbx6mzZGqqXQreWnDdlkTfbmZbXswSTw+o5ATOZGDgKiM JIQWUJLrwQQWH02X0iNjAYXuGRrfA3GmkUw6L3I4X7spPpuDh3EtI/himbevDZ/o1cjm qkzg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584737; x=1729189537; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=IMgO/JDFhWAWu7Wkr7+zBWYfApHhI977J8rKw7ML51Y=; b=NdP94P1NFPhlSkCALwER2Z55bSJlpC4tzofpgpouu3Xn8uGQCXi3yNWzy2zXvu+n+L yATcNuTijWl3drYGBSY2/MZljYdoC1xYo0sO+9MjlsiNNcAtF7iXy/9VofhkcCO4TFH3 BzP18JiL6OJmQR/l5Q25Tr82UQa5QT+Oaqm1+6VueHvbHqbhoOxEbUlxXBsx1izKYKJ5 8CdpPzDvETgAtYfsfpEakaeA8wQc0zNPd68kKQ++HuitfUuYvaS1vj/q9ifuTh7CrfRX zlmjCukBqFaa/dDZQxpuAskU8lehrx1pa3yESodR8FYQJCkp6hh/SEG78t1VoJWIvEvU jiIA== X-Forwarded-Encrypted: i=1; AJvYcCV/dRhihC9Y7VvOfq6hsoOsiTMhr3PW14us8DT4KwPrZ7czFUSdKr5e0fTd8TRh/SFDHmvcvjHNx8GwGjQUOqd7@lists.infradead.org X-Gm-Message-State: AOJu0YyFoOhzj+fpNudORdXoJ3MH7HjpUZFj3nLYyyL6GFVKkXEu04Fk VWCFk2dOBZvsDuVEEVOSFcjA3dabk9er5NN6uSzS33x9EnSdT2nxplWYF9iUc9IXSw6lZkAYVw0 qSA== X-Google-Smtp-Source: AGHT+IGlSN4X6DbrP5DPPe6+MPgoNlxMoOT3GZzpJV2jr+xnXsAJlzTjTlsnBYkwN+CsyuU9UJbLGW89hxk= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a25:c70a:0:b0:e28:eae7:f84a with SMTP id 3f1490d57ef6-e28fe0feaecmr5507276.0.1728584736880; Thu, 10 Oct 2024 11:25:36 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:23 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-22-seanjc@google.com> Subject: [PATCH v13 21/85] KVM: Explicitly initialize all fields at the start of kvm_vcpu_map() From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112538_686639_6CA201A4 X-CRM114-Status: GOOD ( 11.59 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Explicitly initialize the entire kvm_host_map structure when mapping a pfn, as some callers declare their struct on the stack, i.e. don't zero-initialize the struct, which makes the map->hva in kvm_vcpu_unmap() *very* suspect. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- virt/kvm/kvm_main.c | 40 ++++++++++++++++------------------------ 1 file changed, 16 insertions(+), 24 deletions(-) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index c7691bc40389..f1c9a781315c 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -3067,32 +3067,24 @@ void kvm_release_pfn(kvm_pfn_t pfn, bool dirty) int kvm_vcpu_map(struct kvm_vcpu *vcpu, gfn_t gfn, struct kvm_host_map *map) { - kvm_pfn_t pfn; - void *hva = NULL; - struct page *page = KVM_UNMAPPED_PAGE; - - pfn = gfn_to_pfn(vcpu->kvm, gfn); - if (is_error_noslot_pfn(pfn)) - return -EINVAL; - - if (pfn_valid(pfn)) { - page = pfn_to_page(pfn); - hva = kmap(page); -#ifdef CONFIG_HAS_IOMEM - } else { - hva = memremap(pfn_to_hpa(pfn), PAGE_SIZE, MEMREMAP_WB); -#endif - } - - if (!hva) - return -EFAULT; - - map->page = page; - map->hva = hva; - map->pfn = pfn; + map->page = KVM_UNMAPPED_PAGE; + map->hva = NULL; map->gfn = gfn; - return 0; + map->pfn = gfn_to_pfn(vcpu->kvm, gfn); + if (is_error_noslot_pfn(map->pfn)) + return -EINVAL; + + if (pfn_valid(map->pfn)) { + map->page = pfn_to_page(map->pfn); + map->hva = kmap(map->page); +#ifdef CONFIG_HAS_IOMEM + } else { + map->hva = memremap(pfn_to_hpa(map->pfn), PAGE_SIZE, MEMREMAP_WB); +#endif + } + + return map->hva ? 0 : -EFAULT; } EXPORT_SYMBOL_GPL(kvm_vcpu_map); From patchwork Thu Oct 10 18:23:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831134 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E89D4D2444F for ; Thu, 10 Oct 2024 21:01:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:Content-Type:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=wU+jX/zRq5GV793BchOJ2bBXpO8AjDudJh47DKJg70M=; b=3cIfYzc5hyZdy7 6WKwNwO90lEYzEqxbNF3uZsrW2qvSFYJI+mtBWshUAQ+yk5hMbVcc5Ly1H8ElEnX16Kgt21uI0iCC OgML3PB6vidVG0Hf32kcuXpCeBCiJZ0g4nurXB6RKQxvRzcXuQm0+mmPyAMKPeYfMbwNTHoistdai bwT2LXcWxHU8mCVQ607H25pJGW444oPiYnPpjc7Butu7fOS+78+x6TmwSpEQWQNB1nz4SpwpLYT69 ZxKeWwl5xP8y6GFY8+wabJouhdu1MnuBTR5pBP1VkKCsAF4mD4JRvs3VBXHyaT5/w2SeONdD6aEo8 BF6i0Ba1Wb6ej7YPVoow==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1sz0Hl-0000000ELiZ-0b3L; Thu, 10 Oct 2024 21:01:25 +0000 Received: from mail-yb1-xb4a.google.com ([2607:f8b0:4864:20::b4a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxr2-0000000DqCw-2vHE for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:25:44 +0000 Received: by mail-yb1-xb4a.google.com with SMTP id 3f1490d57ef6-e17bb508bb9so1917773276.2 for ; Thu, 10 Oct 2024 11:25:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584739; x=1729189539; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=wU+jX/zRq5GV793BchOJ2bBXpO8AjDudJh47DKJg70M=; b=FIApe3/ZIx/g/Zfj5ITJmKSVaNpQH8BwtI16v5WF0kzSLo63vWhM9zgGz/f6KZrMod eaM47p1v8g3XI8cRrHCgnlw1lkERt2Es2DkFBS95BMEa8VOtSISuR0JQV/5yE6ZrE9g8 jtFlVPKcjYY26HLTybBZ5JgpE+WFrKho0MxAG6zyMHT/NBJhzkj0OsbHSOdDYe5atDIk dwHJUpgA5PUAp3iX+mX/XwVfXhs7goHJGlBb88Z1FzHwOHL0qZrZ/S3A9GWkwAYPF2Rs cJn+WIYR1eimvF36V83F42MITIu1YXfPQNciFvRCZHqu34NfpiAeRIxNgBjC85XWMPfE z3Uw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584739; x=1729189539; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=wU+jX/zRq5GV793BchOJ2bBXpO8AjDudJh47DKJg70M=; b=BwfLBnqTJdS5UaT4smKS7MY3SKNl4GUuQqyEtQM0w6FXSrVN4AxWeRlCTCwuu/HUb+ eG7bqv4HpAOVdL5obIu0H/WxSBO8vX8VGRBP01T983tqB7XuF3Ab6X2B1h2vWOCvvhc/ o0RlqSNKzxOpiw283xymLwBtgxSW+0bwOOL+MJ9Sbe6iC71nEvaSY/7fKfV/TCM/4k5L 8RncsDzMJ0TAIvSwko0iFmn31hCHry026C77Eq3zSRzqoIE5pv/YVHs75IeSQihxr/XH mYEILAiBkPTTP3YqNQDVIkiERiFIRxYgGF+aZnkL6BxUJLQtyuq6xL9B1WOopd8G+2+w mZOw== X-Forwarded-Encrypted: i=1; AJvYcCVXc5Xn6/YzYxLeiQdqFJdb/sWOWjLP13HQHmvqUSlD3Ds17DyuF1P+iTKOo5h2onI0yMwyGaJVE5f5aoDf/pIf@lists.infradead.org X-Gm-Message-State: AOJu0YyULYYC2eFB1/AzveDZuvMOY3f9gLufXvfP9/Cxy7V3uqPsOIc8 vB63nEZJbIl/tkSyFj3AdjmbNXuo5fD0Bj9N6t9KOGjrJ9Tek3Ym2XlV0TKGwElQLQI1RXVcUyu x8g== X-Google-Smtp-Source: AGHT+IGZ+3mddEgfwOZrB7Mg/yEB+clUKORcl3gwcx/GeFypDRI64NMuGnBtjzJ0gTzObJ64BsofGpKHxtA= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a25:ab6d:0:b0:e28:f8e6:f4c6 with SMTP id 3f1490d57ef6-e28fe465652mr58758276.2.1728584738891; Thu, 10 Oct 2024 11:25:38 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:24 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-23-seanjc@google.com> Subject: [PATCH v13 22/85] KVM: Use NULL for struct page pointer to indicate mremapped memory From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112540_813460_80403EDF X-CRM114-Status: GOOD ( 15.75 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Drop yet another unnecessary magic page value from KVM, as there's zero reason to use a poisoned pointer to indicate "no page". If KVM uses a NULL page pointer, the kernel will explode just as quickly as if KVM uses a poisoned pointer. Never mind the fact that such usage would be a blatant and egregious KVM bug. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- include/linux/kvm_host.h | 4 ---- virt/kvm/kvm_main.c | 4 ++-- 2 files changed, 2 insertions(+), 6 deletions(-) diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index cbc7b9c04c14..e3c01cbbc41a 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -273,16 +273,12 @@ enum { READING_SHADOW_PAGE_TABLES, }; -#define KVM_UNMAPPED_PAGE ((void *) 0x500 + POISON_POINTER_DELTA) - struct kvm_host_map { /* * Only valid if the 'pfn' is managed by the host kernel (i.e. There is * a 'struct page' for it. When using mem= kernel parameter some memory * can be used as guest memory but they are not managed by host * kernel). - * If 'pfn' is not managed by the host kernel, this field is - * initialized to KVM_UNMAPPED_PAGE. */ struct page *page; void *hva; diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index f1c9a781315c..7acb1a8af2e4 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -3067,7 +3067,7 @@ void kvm_release_pfn(kvm_pfn_t pfn, bool dirty) int kvm_vcpu_map(struct kvm_vcpu *vcpu, gfn_t gfn, struct kvm_host_map *map) { - map->page = KVM_UNMAPPED_PAGE; + map->page = NULL; map->hva = NULL; map->gfn = gfn; @@ -3093,7 +3093,7 @@ void kvm_vcpu_unmap(struct kvm_vcpu *vcpu, struct kvm_host_map *map, bool dirty) if (!map->hva) return; - if (map->page != KVM_UNMAPPED_PAGE) + if (map->page) kunmap(map->page); #ifdef CONFIG_HAS_IOMEM else From patchwork Thu Oct 10 18:23:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831135 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 36E2BD2444F for ; Thu, 10 Oct 2024 21:01:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:Content-Type:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=UxQVf2h1B1ZgTBX9bfQB+O4u0WEdBy2E8BNaXtNQtEc=; b=M3Ns+spVHK+wmP BdnBxAjYh6ocUpnaFLsrOXuTO6OwHXQiaLS6j8wkImXS0FnrqXXkeETC0a6DtgHA8iTzCgDA2thip B9aTPH/MYhYnH0Ax8MVLTiAIb9w349OfeBwxDUN+iY9fA42qZ2CZw8CwtTzsN/aVUzADdxccRpro6 RDNgo9UedyBJf2wNaTNflBbuXVbjyJdnY9oT8AY7kThgBRmuUwhcznQ0ogDqEV9hxR+0wv3u/IEge fopQA1XJE22fEkUpp3mnZy/YX3aHUWtreX7uJRC9IIoyjmBrJbGZCO7lCGPkmp1sg8crueydteWIm sVt8Z7OX2NeVkn+d126w==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1sz0Hl-0000000ELiw-3C9D; Thu, 10 Oct 2024 21:01:25 +0000 Received: from mail-pl1-x649.google.com ([2607:f8b0:4864:20::649]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxr4-0000000DqEb-0kz4 for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:25:44 +0000 Received: by mail-pl1-x649.google.com with SMTP id d9443c01a7336-204e310e050so14823115ad.0 for ; Thu, 10 Oct 2024 11:25:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584741; x=1729189541; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=UxQVf2h1B1ZgTBX9bfQB+O4u0WEdBy2E8BNaXtNQtEc=; b=cHCxW9OvZGr1yBq6iVLuom+7EfZOc2EiXI7ygoBxdL22tnMY4Rj8QktPQoH9Zvi7rY qyOcJSNL56bk8CzNNK18Qv8diJFnIWO5chCmwCAkSlGhhfKd2k9jLijI97GeNIR3JSb5 KDlg64BqicYf/HRueE0w1SDL7nCmZA4NdT6OUJajraX/1Si4XFYP/vdmBM232nIzJ/Vg 6B+M47Y5OHDhvCmF/DpY/dTExiOIuOVDanc++9fpYZHB1avwuohYC45mdc78wc8C+odL PzeJTkRQxew5bIRj0u4rP3K84+JAXK/nVdKxY+zRIlu3rMcO5RHJD0zj3yKwqUl7by/Z hxkg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584741; x=1729189541; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=UxQVf2h1B1ZgTBX9bfQB+O4u0WEdBy2E8BNaXtNQtEc=; b=pCtVdxvxtrbDWPETdMDEGvmOGNIIMpfGw94ORA9t6+0yERvdNkapAABz9sh/s9B61Q 7+JJ1s+hcKTeT5fjShKDEbnr74PJkWDSbSpnzwRubvsGcUVU8RyMpgsC9yf7IhVGkP8q mi+oaMUSRw5NaVIB84twMCQ9cJMhdV3YNJZhkv6J/pQjhi+6VY7mjVocdv6vh6VpYaUq 04P+T8z1rRXPLrAHcXMEPOXrOYcnZzEXotFzTnL7yiWLYsjqUf0lcWd2bCvAWRcIgiXi shhkHip8BOymRAUK4dboAK1akobjkln4wSBq4B6Adt2VYiAzcQ0bWMKs7iZvQjUNtg0S MseQ== X-Forwarded-Encrypted: i=1; AJvYcCUKtYXvtrXSAmzbw/C4vYtyOWuS1k64A4euDN5yaFaBXk0Rlas0YhAqwC2/5KPpaAydXP6qF1UVCD8gcPm9pBda@lists.infradead.org X-Gm-Message-State: AOJu0YyzZyg1mePTWrr5WgJ/bel3r5FAPatsWq+xdLbg0LX2C0VQOg0Y owoSsHNpItTMLVHa3YwQhhVYwH+yecon0ZJTjq7DJ4GI6japwhj9KArIVLW8Ihef/ZE/+JC3FEo zDQ== X-Google-Smtp-Source: AGHT+IFSsaj7YGs0xXvL7KvHpo4OHZS23TfW901WnGn1dNBw4UEq8HUBbyPz7tDSJuBwPv672jv+MkN6+4M= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:903:2349:b0:20b:9365:e6e4 with SMTP id d9443c01a7336-20c63782bdcmr1094885ad.9.1728584740809; Thu, 10 Oct 2024 11:25:40 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:25 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-24-seanjc@google.com> Subject: [PATCH v13 23/85] KVM: nVMX: Rely on kvm_vcpu_unmap() to track validity of eVMCS mapping From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112542_344517_7914AC0C X-CRM114-Status: GOOD ( 11.25 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Remove the explicit evmptr12 validity check when deciding whether or not to unmap the eVMCS pointer, and instead rely on kvm_vcpu_unmap() to play nice with a NULL map->hva, i.e. to do nothing if the map is invalid. Note, vmx->nested.hv_evmcs_map is zero-allocated along with the rest of vcpu_vmx, i.e. the map starts out invalid/NULL. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/x86/kvm/vmx/nested.c | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c index a8e7bc04d9bf..e94a25373a59 100644 --- a/arch/x86/kvm/vmx/nested.c +++ b/arch/x86/kvm/vmx/nested.c @@ -231,11 +231,8 @@ static inline void nested_release_evmcs(struct kvm_vcpu *vcpu) struct kvm_vcpu_hv *hv_vcpu = to_hv_vcpu(vcpu); struct vcpu_vmx *vmx = to_vmx(vcpu); - if (nested_vmx_is_evmptr12_valid(vmx)) { - kvm_vcpu_unmap(vcpu, &vmx->nested.hv_evmcs_map, true); - vmx->nested.hv_evmcs = NULL; - } - + kvm_vcpu_unmap(vcpu, &vmx->nested.hv_evmcs_map, true); + vmx->nested.hv_evmcs = NULL; vmx->nested.hv_evmcs_vmptr = EVMPTR_INVALID; if (hv_vcpu) { From patchwork Thu Oct 10 18:23:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831062 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id F005FD24442 for ; Thu, 10 Oct 2024 20:11:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:Content-Type:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=eHAZpPaCzYnE+TRrbh6ni4wh+Yys2pChavpSnHk2YEU=; b=Ds60eQG/YhXTJr w9a3xdAJdCjq6AZKrtOn5Kk/CeLu+DPkiRsX7tHDa/f7TieoKNah5fbx4VcpcTW/+ZaEKL8hchRaJ jnQEwDkkISO/d6In43y+19TZ5vOIB6d42nPyjQ+r5XPYLMUC3gZ2lfoEA0bJc/safr1mF+0XZlnTM 43gAsr6oe2GPWLFwzjIk2j9GSMC5gJPxys+gCvL4lHfAr/fadt2I3UjyVpSvonjQZOPmgMmle9sYj /07pnoVUx+F3LAruToWIzFTpVpF/ldw3U5ALUsL0jRwPdoDNyesyTDR8hWH5JzbgNj9IINk4UvAD2 /ZZKFzi+ErL51scqf26A==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syzVY-0000000EEaP-1LYC; Thu, 10 Oct 2024 20:11:36 +0000 Received: from mail-yb1-xb4a.google.com ([2607:f8b0:4864:20::b4a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxr6-0000000DqGA-1b3M for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:25:46 +0000 Received: by mail-yb1-xb4a.google.com with SMTP id 3f1490d57ef6-e28fdb4f35fso2109757276.0 for ; Thu, 10 Oct 2024 11:25:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584743; x=1729189543; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=eHAZpPaCzYnE+TRrbh6ni4wh+Yys2pChavpSnHk2YEU=; b=Zb0EU3prV44v8rePPhkxu4JaEh7GECx4rDUMV31lO5GUbi2/yQu8jNNiHhLjJrB/2S OXtDhvADkrnimtQeMA+9W0Nz7h1XXpa8MTpyPZWvWureI2+J1i5mAMQ7vU3A0M5bepIA +RpwQGwEWZxr13xXmQeYkgCJkenuOlcASDNr/to7qY1cfgY2h+c4OgaSmVEAAzx8J5Af ZpQx654UMkRAeTlRjIvCxepr15qG8z2MYLnNUHLzyek8s0RMI66Nr33O89odLYjjSxX9 AhGjwOH5ukYEndvRdqJSGnAjfdUuKkw0gWxtU/m+QI8HXgIijYEEPmmVaBGYqA594u03 Cgag== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584743; x=1729189543; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=eHAZpPaCzYnE+TRrbh6ni4wh+Yys2pChavpSnHk2YEU=; b=T8co/xdrwNBCGV/wnTrSS2qieTokyRJSB1aL7U11goiTr5yaz8CSHx6cfl/Ybe5NhK YiCuffbGCfxRIK4Ru9LV03gMGV9vXgUBOw8+cmFQaqDiR7ADRmy+X3rGmqGWXsIsWhaT 4/bOLMQyrXoKQpAz/P+Ps2qkmTU/TYgVajQFvpPVVbs+dnRewSm38hN3TnQ3uCniZpPK 4E4M83ibwwMBiAD6D3ei3wXdOHD8c82JKWLxG6auecFxGPf9+6mSCRkE/gvHo3TzUnyv oP5X/Dj9ehcn7V5wOeczDvmZ1EPO3R5oS67XunvpqGhhuHZgzqdYj+dd4two8Nt9ZS97 TwbA== X-Forwarded-Encrypted: i=1; AJvYcCVIp0JsHILl30UD04e0DfvLFRnPE6FaEKEWD6qYwBYPJRUqSJ0b0W/rhwpRUSjWiHRnW1nwo+iTbm5Q9gj9QukJ@lists.infradead.org X-Gm-Message-State: AOJu0YzLK2tMui4mm9oP222XGzzwVbn6+Lwiost5h2T7AVKfWq/4D1bR GRendC6YHcWlBhAyNzPwZX9JNLLC+afURGaQz7OuNdwsQ+5vff2kWkaSUDeglSlWTv2M7qOQ1Zg jnA== X-Google-Smtp-Source: AGHT+IGUAGxjHNELfO1gnQBoxyShlnhLiinRslmifWPsD1igdN7s2BU3B9tdatRPL4jwPkA1Z0qI9X+JjYQ= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a25:9346:0:b0:e28:fc1b:66bb with SMTP id 3f1490d57ef6-e28fe4f0fddmr4836276.6.1728584742775; Thu, 10 Oct 2024 11:25:42 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:26 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-25-seanjc@google.com> Subject: [PATCH v13 24/85] KVM: nVMX: Drop pointless msr_bitmap_map field from struct nested_vmx From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112544_594958_46575E0F X-CRM114-Status: GOOD ( 13.03 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Remove vcpu_vmx.msr_bitmap_map and instead use an on-stack structure in the one function that uses the map, nested_vmx_prepare_msr_bitmap(). Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/x86/kvm/vmx/nested.c | 8 ++++---- arch/x86/kvm/vmx/vmx.h | 2 -- 2 files changed, 4 insertions(+), 6 deletions(-) diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c index e94a25373a59..fb37658b62c9 100644 --- a/arch/x86/kvm/vmx/nested.c +++ b/arch/x86/kvm/vmx/nested.c @@ -621,7 +621,7 @@ static inline bool nested_vmx_prepare_msr_bitmap(struct kvm_vcpu *vcpu, int msr; unsigned long *msr_bitmap_l1; unsigned long *msr_bitmap_l0 = vmx->nested.vmcs02.msr_bitmap; - struct kvm_host_map *map = &vmx->nested.msr_bitmap_map; + struct kvm_host_map msr_bitmap_map; /* Nothing to do if the MSR bitmap is not in use. */ if (!cpu_has_vmx_msr_bitmap() || @@ -644,10 +644,10 @@ static inline bool nested_vmx_prepare_msr_bitmap(struct kvm_vcpu *vcpu, return true; } - if (kvm_vcpu_map(vcpu, gpa_to_gfn(vmcs12->msr_bitmap), map)) + if (kvm_vcpu_map(vcpu, gpa_to_gfn(vmcs12->msr_bitmap), &msr_bitmap_map)) return false; - msr_bitmap_l1 = (unsigned long *)map->hva; + msr_bitmap_l1 = (unsigned long *)msr_bitmap_map.hva; /* * To keep the control flow simple, pay eight 8-byte writes (sixteen @@ -711,7 +711,7 @@ static inline bool nested_vmx_prepare_msr_bitmap(struct kvm_vcpu *vcpu, nested_vmx_set_intercept_for_msr(vmx, msr_bitmap_l1, msr_bitmap_l0, MSR_IA32_FLUSH_CMD, MSR_TYPE_W); - kvm_vcpu_unmap(vcpu, &vmx->nested.msr_bitmap_map, false); + kvm_vcpu_unmap(vcpu, &msr_bitmap_map, false); vmx->nested.force_msr_bitmap_recalc = false; diff --git a/arch/x86/kvm/vmx/vmx.h b/arch/x86/kvm/vmx/vmx.h index 2325f773a20b..40303b43da6c 100644 --- a/arch/x86/kvm/vmx/vmx.h +++ b/arch/x86/kvm/vmx/vmx.h @@ -200,8 +200,6 @@ struct nested_vmx { struct kvm_host_map virtual_apic_map; struct kvm_host_map pi_desc_map; - struct kvm_host_map msr_bitmap_map; - struct pi_desc *pi_desc; bool pi_pending; u16 posted_intr_nv; From patchwork Thu Oct 10 18:23:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831063 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id AD96CD2444C for ; Thu, 10 Oct 2024 20:13:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:Content-Type:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=xnti8P2TQudE7lvlkZdCl8Wq+TvGW4tpGYbCykNWsXw=; b=0GyazET6KDWSBo gPIRO4IyWJS34q9YZszy2RnGSTWKGRvjh6wdnMujYIHA09bcgn2nUGXK25Lq2sTlS4S0p84hlF5ev KWF/tscp3v3Zd2ik/mqStgoSx0XPdxk1uf1qfiPvpUGfMZLOH4RUjfC6mZ64dexE5/SgFvK+IrbQk dpAccOM0eH6nbKGJRjTlxoOslHue6mVnY+AtocWfqCaa0g93bI6cm9vDp7SZQIVdaQmVl11sv/CMm NWcSrL98B1Hszgv7nJBJ+VKwmSxizCCRLhzviKyX/jHZTdXzKiSBTH5WSsBGe+KG3JSawpSpOUwWX MFI7q4vmgk74cgNgInmA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syzWs-0000000EErL-1TNN; Thu, 10 Oct 2024 20:12:58 +0000 Received: from mail-yw1-x114a.google.com ([2607:f8b0:4864:20::114a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxr8-0000000DqHm-04Pl for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:25:48 +0000 Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-6e25b39871fso27044057b3.0 for ; Thu, 10 Oct 2024 11:25:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584744; x=1729189544; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=xnti8P2TQudE7lvlkZdCl8Wq+TvGW4tpGYbCykNWsXw=; b=iLMIpzBZnXcGg0be6+lfC0lRkhEn5KMOD8i176j/1z6K3vzSEKUgEY2w7Slsux3/sR 3Law3bIdQBGjRc99BGD1QjTrmVGIIdRklEe+BS8USHYkaFZ87LaMNp0wTitj1ABRQEGb +GB2Ly492NrlOCkVLBGlkdVcdkTXT7bd/nxeCgdcefR38PXWMKZfxp+zYbQ2eTwdTLLi Cr0gabDJOvk1eneM9z0RaBxYhv4itP1lyjhdFwUNWGM/9rjT6blS0JrqB6TqwHJeOUuj OXlSTptN9xWHtLzCgYAxhDDTihYLI7dD3QW4kF/L5ZcA16r+jrF0UOiapmRfSZAtHtOM hG6A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584744; x=1729189544; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=xnti8P2TQudE7lvlkZdCl8Wq+TvGW4tpGYbCykNWsXw=; b=ba2Ee8vC9EH7R7ndp0Y5cbsmQfrDq3ZYWJEmJHz+seMGqqRcNlEMyhbZ0baYffkSmS vjy1rzi4EKOZeHgH9BACXdoQ5EIRnrFZpWfZJtxTxp2WaUBmLO2K/D8ilHrFLgh20qXl ytMekmkc/Pl42ndRV75Jcxox/bdcRzcaBWZfIN8ZVbUJPOfvaq9GjqV705PerWqLl/+5 lUcRc9lyp3AFRfFq2G53FlWIDTaNHXj6ieg5f2m/k/VQsZXoHL4Z1A1TJn4p6sTALRE8 hfP/jkvNZtFKhYFHZR2hJ5QST40gK/PgR5XSgyjIXsk9Om+Y+LXf97Qry8KEXGkNKLYW 5amA== X-Forwarded-Encrypted: i=1; AJvYcCXqPXZqHYEwYQRTe6g+DqodSBwAw9J/lCKFpYWljApgCW86z231w67wQzXcnjn42alxJbXfy3SYKVfFwZxPs/Wc@lists.infradead.org X-Gm-Message-State: AOJu0YwHZSv7fCGsA1BFphUXntYyjC2/Nyxplxj8KHgyOEB2wiC0pKr8 gWl6A9E7gJptniG/NNbsnlqAPp9V1XPZv7pnfs2X0xLF71ITWM04qFHarVaenQJJL3nMPRXI8w2 EWQ== X-Google-Smtp-Source: AGHT+IEdX//xUJq0LlzHSFsWnzcNANrEE9KtnndGvFzP5BAu23zXYFVXA6eeBi51F5M5i08FgAsxiLe5HbY= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:690c:300b:b0:6e3:f32:5fc8 with SMTP id 00721157ae682-6e32217bbe0mr502817b3.1.1728584744665; Thu, 10 Oct 2024 11:25:44 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:27 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-26-seanjc@google.com> Subject: [PATCH v13 25/85] KVM: nVMX: Add helper to put (unmap) vmcs12 pages From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112546_207988_CF4BC158 X-CRM114-Status: GOOD ( 13.00 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Add a helper to dedup unmapping the vmcs12 pages. This will reduce the amount of churn when a future patch refactors the kvm_vcpu_unmap() API. No functional change intended. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/x86/kvm/vmx/nested.c | 32 ++++++++++++++++++-------------- 1 file changed, 18 insertions(+), 14 deletions(-) diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c index fb37658b62c9..81865db18e12 100644 --- a/arch/x86/kvm/vmx/nested.c +++ b/arch/x86/kvm/vmx/nested.c @@ -314,6 +314,21 @@ static void vmx_switch_vmcs(struct kvm_vcpu *vcpu, struct loaded_vmcs *vmcs) vcpu->arch.regs_dirty = 0; } +static void nested_put_vmcs12_pages(struct kvm_vcpu *vcpu) +{ + struct vcpu_vmx *vmx = to_vmx(vcpu); + + /* + * Unpin physical memory we referred to in the vmcs02. The APIC access + * page's backing page (yeah, confusing) shouldn't actually be accessed, + * and if it is written, the contents are irrelevant. + */ + kvm_vcpu_unmap(vcpu, &vmx->nested.apic_access_page_map, false); + kvm_vcpu_unmap(vcpu, &vmx->nested.virtual_apic_map, true); + kvm_vcpu_unmap(vcpu, &vmx->nested.pi_desc_map, true); + vmx->nested.pi_desc = NULL; +} + /* * Free whatever needs to be freed from vmx->nested when L1 goes down, or * just stops using VMX. @@ -346,15 +361,8 @@ static void free_nested(struct kvm_vcpu *vcpu) vmx->nested.cached_vmcs12 = NULL; kfree(vmx->nested.cached_shadow_vmcs12); vmx->nested.cached_shadow_vmcs12 = NULL; - /* - * Unpin physical memory we referred to in the vmcs02. The APIC access - * page's backing page (yeah, confusing) shouldn't actually be accessed, - * and if it is written, the contents are irrelevant. - */ - kvm_vcpu_unmap(vcpu, &vmx->nested.apic_access_page_map, false); - kvm_vcpu_unmap(vcpu, &vmx->nested.virtual_apic_map, true); - kvm_vcpu_unmap(vcpu, &vmx->nested.pi_desc_map, true); - vmx->nested.pi_desc = NULL; + + nested_put_vmcs12_pages(vcpu); kvm_mmu_free_roots(vcpu->kvm, &vcpu->arch.guest_mmu, KVM_MMU_ROOTS_ALL); @@ -5010,11 +5018,7 @@ void nested_vmx_vmexit(struct kvm_vcpu *vcpu, u32 vm_exit_reason, vmx_update_cpu_dirty_logging(vcpu); } - /* Unpin physical memory we referred to in vmcs02 */ - kvm_vcpu_unmap(vcpu, &vmx->nested.apic_access_page_map, false); - kvm_vcpu_unmap(vcpu, &vmx->nested.virtual_apic_map, true); - kvm_vcpu_unmap(vcpu, &vmx->nested.pi_desc_map, true); - vmx->nested.pi_desc = NULL; + nested_put_vmcs12_pages(vcpu); if (vmx->nested.reload_vmcs01_apic_access_page) { vmx->nested.reload_vmcs01_apic_access_page = false; From patchwork Thu Oct 10 18:23:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831137 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 91024D24453 for ; Thu, 10 Oct 2024 21:01:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:Content-Type:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=/W6JK9yAcorM7mqEFQq2hRBqvFCoz2C4pz3yOEmiGek=; b=H37gJSh/efGVH2 cgbqjykThNlr00hjekmwc5DN9iHVLLOi3InOd+2uQMtX17O0IcOXKulh5frldt8G2hE5PFPUNlrym S0WRUSfngYu5ik0A1IVQaAVYm6Vp484KJX/lu9ww1nip5YNPJEqf/reNWvpsuAfZGkTt46O6MOA+Y Fpy6L/t3fBO81spdQFdUxEj4mHlw1/RsZYvxKtzxE4Lr4OG8hFU/VPiM6rAgGv5JG2yTXg8zuB6Bd 25IahaoyF4jHp1++i+ElAsEI8u11g6hqL3HTYXlqj+vGWX++sQmhX1sN0CmWq+fzZqG5MM6hfYv0+ oVt6pzFIYrfBpg7dKnRQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1sz0Ho-0000000ELlU-2Whq; Thu, 10 Oct 2024 21:01:28 +0000 Received: from mail-pj1-x1049.google.com ([2607:f8b0:4864:20::1049]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxrA-0000000DqK2-20Ru for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:25:50 +0000 Received: by mail-pj1-x1049.google.com with SMTP id 98e67ed59e1d1-2e2a013a01bso1334463a91.0 for ; Thu, 10 Oct 2024 11:25:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584747; x=1729189547; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=/W6JK9yAcorM7mqEFQq2hRBqvFCoz2C4pz3yOEmiGek=; b=3ljqXwYT9KU7vL/bvKPf9S0Yn/qBntGvThCT2wzCd8Z12f/66N5sIoVWA14cY3wEXx g0HCLPAfoQ28OuiX6nzyAfawBv92qW+HPQQu37BsUa/V7uUuXJiLYMSM601iEwSqsUOf /NM7uP351CB79XT86j42gqhFKlgX/vbHgHz5DTqinJBvK+PcdaEbIaL+QrMPgT0FIbtp HGVX7X1ioRXdTGYtEGA8WTxtd6LOh1q2++W09WQfgTr1feSdFE+UnwE/XERuI/cMc2G4 mKN8Qf+/P7jUbv/vuDxwiZEcUlFeFFt/L98/apl6pTYPYAUmkiq5SNEFXz3+LRw5g5bM ZyYw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584747; x=1729189547; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=/W6JK9yAcorM7mqEFQq2hRBqvFCoz2C4pz3yOEmiGek=; b=MvBbEBeevq2cePxfb6YChK1xsn14X4CTHkUVdH6toZqnw9BkEkjF7CS7uWCa5Ae+f2 m45bdmZatcRMkp+yLspd4ckdEhK4PltoJGGdZmsBVtdd+iXq9sInFER+UVk6w2V6p2tN a7jvhPmqnr1NFZdi27608RF4l7BBf7zodFkXTClXALnfuW653O4u/D7s/cAGiySnrX9W 0Et42sc24/Iyk7aXOPvR9yoD7cyqWXthZgwByyjiocsGx2u31HUnJMB877xp8LhzB4u8 3gB1RE3VsQyAaoCSNxm9Sy545t8vrj8gdTK1FciVPSiZpYEO9IlLnkDwvfqjRaXJhy93 JAqA== X-Forwarded-Encrypted: i=1; AJvYcCVd97oidruXBWQmw7DZteHU1bnNjalYY29fToKGAdJZ/QBUuRDkyhjUBWKPaMQyGOFKZJ/N9qM8PIxcWMA/bkLC@lists.infradead.org X-Gm-Message-State: AOJu0YwOaPoAnRlH7zVnc7xC/ftbVCNj8a5pPTffl3XOfROCcy4JsRn8 gAuY50ysvFTykoweXQqvFXaUubssuHk+NJXKe9KxsNJMn+UU0WZyMBlIphNavHMrYW5hhkzqS/1 vUA== X-Google-Smtp-Source: AGHT+IFGWHaFUEsQRxn4rcRWabj8vzxqpLHdNOXqOmiiQlX4dD6MZqF5BpShV2hVfQIOX/PTyNMVtEMSya8= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:90a:9a94:b0:2e2:da81:40c1 with SMTP id 98e67ed59e1d1-2e2f09f2280mr109a91.1.1728584746508; Thu, 10 Oct 2024 11:25:46 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:28 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-27-seanjc@google.com> Subject: [PATCH v13 26/85] KVM: Use plain "struct page" pointer instead of single-entry array From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112548_761263_0153AC46 X-CRM114-Status: GOOD ( 12.32 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Use a single pointer instead of a single-entry array for the struct page pointer in hva_to_pfn_fast(). Using an array makes the code unnecessarily annoying to read and update. No functional change intended. Reviewed-by: Alex Bennée Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- virt/kvm/kvm_main.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 7acb1a8af2e4..d3e48fcc4fb0 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -2752,7 +2752,7 @@ unsigned long kvm_vcpu_gfn_to_hva_prot(struct kvm_vcpu *vcpu, gfn_t gfn, bool *w */ static bool hva_to_pfn_fast(struct kvm_follow_pfn *kfp, kvm_pfn_t *pfn) { - struct page *page[1]; + struct page *page; /* * Fast pin a writable pfn only if it is a write fault request @@ -2762,8 +2762,8 @@ static bool hva_to_pfn_fast(struct kvm_follow_pfn *kfp, kvm_pfn_t *pfn) if (!((kfp->flags & FOLL_WRITE) || kfp->map_writable)) return false; - if (get_user_page_fast_only(kfp->hva, FOLL_WRITE, page)) { - *pfn = page_to_pfn(page[0]); + if (get_user_page_fast_only(kfp->hva, FOLL_WRITE, &page)) { + *pfn = page_to_pfn(page); if (kfp->map_writable) *kfp->map_writable = true; return true; From patchwork Thu Oct 10 18:23:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830872 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0C687CFC5F6 for ; Thu, 10 Oct 2024 18:56:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:Content-Type:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=txhtY6exXDaGFQAWYGvmt09bjeoYz319F03IR2k/C3M=; b=AgNf7IWRN5GxYO B67zBqnka0qf2n8VdEvemtEdSGeJcjP2tkzac4cbcnP9GJCFM7cW9cHvI772AQtDEzyB5CZSPOW4P bF5wfcTZDr7hVXvJovv1TACzsl9HIvjEA//DOOEI9P2DeJL8UR/9pq/6qYuiOK9mnKKlCBjjfH6xd hySUzvkXj0bN0TaRRRoVVvCeZDSFpMnMKPjE7FJjWBXcyTxUYdD8c7aRPQ8hkZQio3cl8LIov4pZ3 PXjPU8AT7P3W3XRa1Y7XDZ6nOF7APpbTehHbbykgMGmI81srrZMcpdqJD1YcRrpLzxjjpf/V8rKYG x6jiFP7EOacu/Ardmb+w==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syyKm-0000000E0WC-2dXX; Thu, 10 Oct 2024 18:56:24 +0000 Received: from mail-pf1-x44a.google.com ([2607:f8b0:4864:20::44a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxrC-0000000DqM6-2Z58 for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:25:53 +0000 Received: by mail-pf1-x44a.google.com with SMTP id d2e1a72fcca58-71def7a1a8aso1517778b3a.3 for ; Thu, 10 Oct 2024 11:25:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584749; x=1729189549; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=txhtY6exXDaGFQAWYGvmt09bjeoYz319F03IR2k/C3M=; b=QOiWlENQw1uIbr6nUNrqjMhma3ERcOVij9+O8o2BjPvO4mrYl4XsAmuLiusexGR8og wjRSA7WvE5P8YAxpwHCCocnQY3fc2JIYkO+hhKk6wpWzFsfI8BAMyortVof5wkh28VS0 Qt6uIQVR43FVlVhGrf1uAMsyy/6cvsQrPWf167zPqypNnCLXzoYxB2jdJWRtBimBpcWt uoYysw3iFUs8BsYcyGBmeWFo8/VrIQKNDreM/iOY/RkbKvOAdmKLBqSp17mNc5/WUjiD njASJ7YlaTL5HOieVnKQztyEjeiufrz2c0OKSD/CKjo1u2aS16zJOMO10Q+JaDo62pQq 63gg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584749; x=1729189549; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=txhtY6exXDaGFQAWYGvmt09bjeoYz319F03IR2k/C3M=; b=Hw/+gL5jiIn9YLAMDTK2NGUBMCDRobAcO0KdAAKsThSbSXWrqw2kFw3eEbMmaFaN3N gBsCgs3UamCit5HqUwk4MgThfCndE6zmfwkHjYAdRqtoJ0J2m+H7Ld81nOSMAfw63oJu NAZJWpCSYNdo4Id2XJURTdv4rEprPhIibzQWpFVIaQBwEZnfrfoKDE2+jwD91JkR1l+i AqT5k1AGv8UWKk9GWluEBSU3tgWqwAk3WZI1YTQ75EypV/Ty9GQT+3yMQBG0088ZPRNw 4tydOUX5MrxreV3Iz9Fwmke0rYwFDuI0HpNT0gMnF0MM+Wh5C7jNEAnCcHpLdRKi+v06 +1Eg== X-Forwarded-Encrypted: i=1; AJvYcCXPyrCXzN5LfWroiX3JTGQr6yRAJFXAZOruwp+sbA6eqVZEuYGPfdEZzcZMXZ4Q766b6Ihkpbc75SnzUAyfx4J7@lists.infradead.org X-Gm-Message-State: AOJu0Yz3Qu5EylCk4Hemm27FvvDXxfNH4KoFIJwR1Le+BJ81bdiaUNa8 pAd5/657gWweIwiR6bBaTkrFRLXge6fBcA/TyRpMuADPMEyiRi1oVRgYfEBsLhjj0X64lRYwGWw bmA== X-Google-Smtp-Source: AGHT+IE/uLa8aldzd4vJMnq+De69DiThAuVt+3q8zF6rqxlBj2/ZKo7+FHCoJyy45GpCD4mZjJfGV3U05Tk= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:6a00:2d8a:b0:71d:f1f9:b982 with SMTP id d2e1a72fcca58-71e1dc00b9cmr38199b3a.6.1728584749206; Thu, 10 Oct 2024 11:25:49 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:29 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-28-seanjc@google.com> Subject: [PATCH v13 27/85] KVM: Provide refcounted page as output field in struct kvm_follow_pfn From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112550_820920_C9D19A1A X-CRM114-Status: GOOD ( 24.27 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Add kvm_follow_pfn.refcounted_page as an output for the "to pfn" APIs to "return" the struct page that is associated with the returned pfn (if KVM acquired a reference to the page). This will eventually allow removing KVM's hacky kvm_pfn_to_refcounted_page() code, which is error prone and can't detect pfns that are valid, but aren't (currently) refcounted. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- virt/kvm/kvm_main.c | 99 +++++++++++++++++++++------------------------ virt/kvm/kvm_mm.h | 9 +++++ 2 files changed, 56 insertions(+), 52 deletions(-) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index d3e48fcc4fb0..e29f78ed6f48 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -2746,6 +2746,46 @@ unsigned long kvm_vcpu_gfn_to_hva_prot(struct kvm_vcpu *vcpu, gfn_t gfn, bool *w return gfn_to_hva_memslot_prot(slot, gfn, writable); } +static kvm_pfn_t kvm_resolve_pfn(struct kvm_follow_pfn *kfp, struct page *page, + struct follow_pfnmap_args *map, bool writable) +{ + kvm_pfn_t pfn; + + WARN_ON_ONCE(!!page == !!map); + + if (kfp->map_writable) + *kfp->map_writable = writable; + + /* + * FIXME: Remove this once KVM no longer blindly calls put_page() on + * every pfn that points at a struct page. + * + * Get a reference for follow_pte() pfns if they happen to point at a + * struct page, as KVM will ultimately call kvm_release_pfn_clean() on + * the returned pfn, i.e. KVM expects to have a reference. + * + * Certain IO or PFNMAP mappings can be backed with valid struct pages, + * but be allocated without refcounting, e.g. tail pages of + * non-compound higher order allocations. Grabbing and putting a + * reference to such pages would cause KVM to prematurely free a page + * it doesn't own (KVM gets and puts the one and only reference). + * Don't allow those pages until the FIXME is resolved. + */ + if (map) { + pfn = map->pfn; + page = kvm_pfn_to_refcounted_page(pfn); + if (page && !get_page_unless_zero(page)) + return KVM_PFN_ERR_FAULT; + } else { + pfn = page_to_pfn(page); + } + + if (kfp->refcounted_page) + *kfp->refcounted_page = page; + + return pfn; +} + /* * The fast path to get the writable pfn which will be stored in @pfn, * true indicates success, otherwise false is returned. @@ -2763,9 +2803,7 @@ static bool hva_to_pfn_fast(struct kvm_follow_pfn *kfp, kvm_pfn_t *pfn) return false; if (get_user_page_fast_only(kfp->hva, FOLL_WRITE, &page)) { - *pfn = page_to_pfn(page); - if (kfp->map_writable) - *kfp->map_writable = true; + *pfn = kvm_resolve_pfn(kfp, page, NULL, true); return true; } @@ -2797,23 +2835,15 @@ static int hva_to_pfn_slow(struct kvm_follow_pfn *kfp, kvm_pfn_t *pfn) if (npages != 1) return npages; - if (!kfp->map_writable) - goto out; - - if (kfp->flags & FOLL_WRITE) { - *kfp->map_writable = true; - goto out; - } - /* map read fault as writable if possible */ - if (get_user_page_fast_only(kfp->hva, FOLL_WRITE, &wpage)) { - *kfp->map_writable = true; + if (!(flags & FOLL_WRITE) && kfp->map_writable && + get_user_page_fast_only(kfp->hva, FOLL_WRITE, &wpage)) { put_page(page); page = wpage; + flags |= FOLL_WRITE; } -out: - *pfn = page_to_pfn(page); + *pfn = kvm_resolve_pfn(kfp, page, NULL, flags & FOLL_WRITE); return npages; } @@ -2828,22 +2858,11 @@ static bool vma_is_valid(struct vm_area_struct *vma, bool write_fault) return true; } -static int kvm_try_get_pfn(kvm_pfn_t pfn) -{ - struct page *page = kvm_pfn_to_refcounted_page(pfn); - - if (!page) - return 1; - - return get_page_unless_zero(page); -} - static int hva_to_pfn_remapped(struct vm_area_struct *vma, struct kvm_follow_pfn *kfp, kvm_pfn_t *p_pfn) { struct follow_pfnmap_args args = { .vma = vma, .address = kfp->hva }; bool write_fault = kfp->flags & FOLL_WRITE; - kvm_pfn_t pfn; int r; r = follow_pfnmap_start(&args); @@ -2867,37 +2886,13 @@ static int hva_to_pfn_remapped(struct vm_area_struct *vma, } if (write_fault && !args.writable) { - pfn = KVM_PFN_ERR_RO_FAULT; + *p_pfn = KVM_PFN_ERR_RO_FAULT; goto out; } - if (kfp->map_writable) - *kfp->map_writable = args.writable; - pfn = args.pfn; - - /* - * Get a reference here because callers of *hva_to_pfn* and - * *gfn_to_pfn* ultimately call kvm_release_pfn_clean on the - * returned pfn. This is only needed if the VMA has VM_MIXEDMAP - * set, but the kvm_try_get_pfn/kvm_release_pfn_clean pair will - * simply do nothing for reserved pfns. - * - * Whoever called remap_pfn_range is also going to call e.g. - * unmap_mapping_range before the underlying pages are freed, - * causing a call to our MMU notifier. - * - * Certain IO or PFNMAP mappings can be backed with valid - * struct pages, but be allocated without refcounting e.g., - * tail pages of non-compound higher order allocations, which - * would then underflow the refcount when the caller does the - * required put_page. Don't allow those pages here. - */ - if (!kvm_try_get_pfn(pfn)) - r = -EFAULT; + *p_pfn = kvm_resolve_pfn(kfp, NULL, &args, args.writable); out: follow_pfnmap_end(&args); - *p_pfn = pfn; - return r; } diff --git a/virt/kvm/kvm_mm.h b/virt/kvm/kvm_mm.h index d5a215958f06..d3ac1ba8ba66 100644 --- a/virt/kvm/kvm_mm.h +++ b/virt/kvm/kvm_mm.h @@ -35,6 +35,15 @@ struct kvm_follow_pfn { * Set to true if a writable mapping was obtained. */ bool *map_writable; + + /* + * Optional output. Set to a valid "struct page" if the returned pfn + * is for a refcounted or pinned struct page, NULL if the returned pfn + * has no struct page or if the struct page is not being refcounted + * (e.g. tail pages of non-compound higher order allocations from + * IO/PFNMAP mappings). + */ + struct page **refcounted_page; }; kvm_pfn_t hva_to_pfn(struct kvm_follow_pfn *kfp); From patchwork Thu Oct 10 18:23:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830873 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E3AABCFC5F6 for ; Thu, 10 Oct 2024 18:57:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:Content-Type:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=MotdKZcknVHCSnrsAh1XZemF3Qz2FYV2KdHfEEknv98=; b=iJ9ngxIjIIz2t2 vOLoB/akXyRaXJrARbYgwd8pxqTSIeEi+yxe6QUckJzoXPmfbCenwAi3732Jjdoi1EgW0ewmH84P8 W2HYPInCsnOBsZS9WClWc596IvMQmKiL1AldmTZwftdeNPhsd0rnRepF6dv+9+kWbuklRhZ5B12C0 no3t+LYbDzMmmT4sbbSFADKP+lFd8VTRRmPHrsAPv1aF204PnxzSWmCwCJ3vxP6OWELRrI89Z/BpI JRCu9Jkh5MkyDD+BlXmRYuXaO4QJ8j73+IOaHii64yfZKpFjIY+IISeMFEVNkdAEyXkdy+nOBAo4Q UA+s7SxD5KLn2Imv9rmg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syyM6-0000000E0pf-2zI2; Thu, 10 Oct 2024 18:57:46 +0000 Received: from mail-pj1-x104a.google.com ([2607:f8b0:4864:20::104a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxrE-0000000DqNS-2epN for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:25:54 +0000 Received: by mail-pj1-x104a.google.com with SMTP id 98e67ed59e1d1-2e2e2e02817so486101a91.1 for ; Thu, 10 Oct 2024 11:25:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584751; x=1729189551; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=MotdKZcknVHCSnrsAh1XZemF3Qz2FYV2KdHfEEknv98=; b=jtLSLBcVgcASnTqlQ1eKxrFJR2feDfmay3NURsIC0axb8C2ARvmHmzCliBGAueotr+ B5XpHud2uLNFerpPsDj3hQwC9F0YTpa1737w8LQD0y5y7LzFTJWgDzF8hgHAoeB98c9h Ori7lZA+YjrLO+Qvs/uyhUsL+FZdPkU+W0JGTGPrJpQDbrwxwoqqJKLrmYcuj3o8IGb9 kg0swTwVKHW8C2cNgOAq8B0ThHBwdCGytcaZqR7NBsDqfTCj3rwXpBwNRbOpdEi/OhiP Gt1beXFBomlvF6zLw3cKb7dJ4CZuwDOsEzDBVH9U+wi6w0FOeGSWx7Vqbfa8yS3y3aJZ XbHA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584751; x=1729189551; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=MotdKZcknVHCSnrsAh1XZemF3Qz2FYV2KdHfEEknv98=; b=XP2WxdLpbxQmPBgs93CdiSDzIcrACbYxNiUVQdUf6FCG1ajN9Y3W6yl+WqbblnhHJW fn5+KssETPhtwce2y6Tdra40Eaf1/rtzauPoEq3eO/V/GR/AIwDpF6DzTOJ5+AqC3F0n 2wUE/dYw4h8dxjfvaxVT86xWl1rmYXB2V1YjjTQt+V9mlpNqkgbFakiaAlChpxWu0LBX D/uuXrAmfJI7OzBCo5hngm2Y2JNbTxx5d7DJ7fxuzz+JKLRlK+3XZg4eCHJhercteC0E zjRBIh6EeTcWPdxu5RzUkLxnIY+zs0QUIR5NdnrMHraKBaflGTWX0/VaeUo02cvlZft5 VGbg== X-Forwarded-Encrypted: i=1; AJvYcCW09AM77krwiBn7iPGHxZ+/yseiLDO5rgNV74B3b2uddxjrv3TRSqeyg/QwRxeGw+eCmgfGpBuOPzTKlD3yVqyA@lists.infradead.org X-Gm-Message-State: AOJu0YwXpk67u4hO3ApmIzZ4aVLOz41TYgsqSFdE7SV2WhUg6TPoUpca RddOM/oIIZuFluKgf5dRCPQ1zF0dGM7LOwmchijduj0QnNwn6HSwoAowU29uIPJiQmy2HOotz+M 1vQ== X-Google-Smtp-Source: AGHT+IHahb+vMbBUybe9JXpaiwYyoag4sz2flsLCX6vY5d7t+wo3V9TaPVQYV32tHeuijXZHqATezWyezMk= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:90a:d913:b0:2e2:a810:c3e4 with SMTP id 98e67ed59e1d1-2e2c81d4d19mr7321a91.4.1728584751203; Thu, 10 Oct 2024 11:25:51 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:30 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-29-seanjc@google.com> Subject: [PATCH v13 28/85] KVM: Move kvm_{set,release}_page_{clean,dirty}() helpers up in kvm_main.c From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112552_709775_082AF46A X-CRM114-Status: GOOD ( 11.06 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Hoist the kvm_{set,release}_page_{clean,dirty}() APIs further up in kvm_main.c so that they can be used by the kvm_follow_pfn family of APIs. No functional change intended. Reviewed-by: Alex Bennée Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- virt/kvm/kvm_main.c | 82 ++++++++++++++++++++++----------------------- 1 file changed, 41 insertions(+), 41 deletions(-) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index e29f78ed6f48..6cdbd0516d58 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -2746,6 +2746,47 @@ unsigned long kvm_vcpu_gfn_to_hva_prot(struct kvm_vcpu *vcpu, gfn_t gfn, bool *w return gfn_to_hva_memslot_prot(slot, gfn, writable); } +static bool kvm_is_ad_tracked_page(struct page *page) +{ + /* + * Per page-flags.h, pages tagged PG_reserved "should in general not be + * touched (e.g. set dirty) except by its owner". + */ + return !PageReserved(page); +} + +static void kvm_set_page_dirty(struct page *page) +{ + if (kvm_is_ad_tracked_page(page)) + SetPageDirty(page); +} + +static void kvm_set_page_accessed(struct page *page) +{ + if (kvm_is_ad_tracked_page(page)) + mark_page_accessed(page); +} + +void kvm_release_page_clean(struct page *page) +{ + if (!page) + return; + + kvm_set_page_accessed(page); + put_page(page); +} +EXPORT_SYMBOL_GPL(kvm_release_page_clean); + +void kvm_release_page_dirty(struct page *page) +{ + if (!page) + return; + + kvm_set_page_dirty(page); + kvm_release_page_clean(page); +} +EXPORT_SYMBOL_GPL(kvm_release_page_dirty); + static kvm_pfn_t kvm_resolve_pfn(struct kvm_follow_pfn *kfp, struct page *page, struct follow_pfnmap_args *map, bool writable) { @@ -3105,37 +3146,6 @@ void kvm_vcpu_unmap(struct kvm_vcpu *vcpu, struct kvm_host_map *map, bool dirty) } EXPORT_SYMBOL_GPL(kvm_vcpu_unmap); -static bool kvm_is_ad_tracked_page(struct page *page) -{ - /* - * Per page-flags.h, pages tagged PG_reserved "should in general not be - * touched (e.g. set dirty) except by its owner". - */ - return !PageReserved(page); -} - -static void kvm_set_page_dirty(struct page *page) -{ - if (kvm_is_ad_tracked_page(page)) - SetPageDirty(page); -} - -static void kvm_set_page_accessed(struct page *page) -{ - if (kvm_is_ad_tracked_page(page)) - mark_page_accessed(page); -} - -void kvm_release_page_clean(struct page *page) -{ - if (!page) - return; - - kvm_set_page_accessed(page); - put_page(page); -} -EXPORT_SYMBOL_GPL(kvm_release_page_clean); - void kvm_release_pfn_clean(kvm_pfn_t pfn) { struct page *page; @@ -3151,16 +3161,6 @@ void kvm_release_pfn_clean(kvm_pfn_t pfn) } EXPORT_SYMBOL_GPL(kvm_release_pfn_clean); -void kvm_release_page_dirty(struct page *page) -{ - if (!page) - return; - - kvm_set_page_dirty(page); - kvm_release_page_clean(page); -} -EXPORT_SYMBOL_GPL(kvm_release_page_dirty); - void kvm_release_pfn_dirty(kvm_pfn_t pfn) { struct page *page; From patchwork Thu Oct 10 18:23:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830874 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id CF62CCFC5F8 for ; Thu, 10 Oct 2024 18:59:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:Content-Type:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=nXVCtJlFVF227JmXan6PsTRYgw7//koA3Av9lMmMw70=; b=AmgDm92DfALTqv aaHmvZZy5kR6mU/nctInaQTxh6UT/KZK5dwFk2TRdtziXuIiwsgIPL97J8RGawd0RF+/Z2QaMDzIe r1h90sEwrriKKs+lS3hSLA1tlJDaN+38/SM3b0eOqEQVD9GuIZkVtLfdPZERLDNfteAVsAPZDmBWb 8gDyyYV8qTTRrTv3GFoiefmWzaiPSG6kGy4Y4VzuHsMHEK7fHK93mNCGq69Blv69mSLQQCUBnvohV IpBwl5xTcqr8nPoWwqDQofu6Ca5kC2+ADo6/wD8GQfmUewpr11UHZUVn9wfnfigacAw4jbiuwvYaU kAyp2B65YC6eZsiqmVlA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syyNR-0000000E1O0-3Gc3; Thu, 10 Oct 2024 18:59:09 +0000 Received: from mail-yb1-xb4a.google.com ([2607:f8b0:4864:20::b4a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxrH-0000000DqP2-0Rk8 for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:25:57 +0000 Received: by mail-yb1-xb4a.google.com with SMTP id 3f1490d57ef6-e1159fb161fso2214713276.1 for ; Thu, 10 Oct 2024 11:25:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584753; x=1729189553; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=nXVCtJlFVF227JmXan6PsTRYgw7//koA3Av9lMmMw70=; b=ttCXfoQrTUVfgH8jwR8KWzplwegscq+j8FDqssv2oxhDJRt4EEVBLhTF1+GaGNqIQl XvdFls0Il6zSW+CKvmB9tAofDO1Ill0yzaCXijICpQhEHTeRAWZnMb43UbmWmuM0vN2I 9sZWepWrJ+nULqgWGU5XrQ46pWviuTndQVuXa3ndAOlS+s/JEa0YjwCfRXMzAf4tzQHd XqRowaXrWrVUe04WhSnbBr0Qc8BnZMsF8YrEnrHvYviJJkJkNbvK6sB89qGjQdFD7cL9 5GOd+zNYLbrYew8HML/Cnf9qrivbBDIL1+HW1EqU4gsgl4lwPm3tZ7yh7tQzleuhQo16 DdYg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584753; x=1729189553; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=nXVCtJlFVF227JmXan6PsTRYgw7//koA3Av9lMmMw70=; b=JvhNcJpjj4LotJ0MQCLOz7yoNX+TZAaKPR38+EJJ8Zp8HY3SMOdsVeXRRPnMs8qCM1 QOlOyB4xI5rTOCBvReNdFo/WzlbLw3qq0pTdY+jZgSbXxehNVv/4xV59v5NHtYZCR+1n 6rMjo8QT0Wxih7DPuvWocxiV7i7/Lj2kmIItB/tuadZhqn8z9nGiEp+YaP7AXm7Y2DUx j0cEjC3t/VpQsDjY7p0kBP+YU73rpjjgLAPtXCaVYlkKbEBJfYLRiw5zpZnVEqlJ4ctM JFyLvePpXxmflNupgQ226QymGPOZZL4nMqPIkbjtJeZSI+62OaFDie+RmzKvGLrob5dA OCjA== X-Forwarded-Encrypted: i=1; AJvYcCUuv0kuIfCkTxolSaE/4bKIw5Fxs96Rh4TBGG/ZrJfuf7VpwwoZEY0/uKxf5DmJ4iwcjekk7s4MNdCr58FDs6jg@lists.infradead.org X-Gm-Message-State: AOJu0YwQ6kjIeKqXe2Kp0bWVpIGRpACqC6xpjqRhKgzF1N1bHyYbQ1Uh 2RUw3Gk16UPd/AuVJmpawNQHnHieWWsVWO5pzUharFAFjUHv3uCLcoosyxP1YL/x4oVf2l5y/hf 6dA== X-Google-Smtp-Source: AGHT+IFqzRS1xOg2oJDncuDOdOtR7gITdwbfvBFuJ4xWEWgDQEplutdiZUKmbH0mIVak18Oaak6LM032atM= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a25:b205:0:b0:e25:17cb:352e with SMTP id 3f1490d57ef6-e28fe43f3f5mr4095276.9.1728584753336; Thu, 10 Oct 2024 11:25:53 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:31 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-30-seanjc@google.com> Subject: [PATCH v13 29/85] KVM: pfncache: Precisely track refcounted pages From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112555_325836_50D40811 X-CRM114-Status: GOOD ( 12.25 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Track refcounted struct page memory using kvm_follow_pfn.refcounted_page instead of relying on kvm_release_pfn_clean() to correctly detect that the pfn is associated with a struct page. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- virt/kvm/pfncache.c | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/virt/kvm/pfncache.c b/virt/kvm/pfncache.c index 067daf9ad6ef..728d2c1b488a 100644 --- a/virt/kvm/pfncache.c +++ b/virt/kvm/pfncache.c @@ -159,11 +159,14 @@ static kvm_pfn_t hva_to_pfn_retry(struct gfn_to_pfn_cache *gpc) kvm_pfn_t new_pfn = KVM_PFN_ERR_FAULT; void *new_khva = NULL; unsigned long mmu_seq; + struct page *page; + struct kvm_follow_pfn kfp = { .slot = gpc->memslot, .gfn = gpa_to_gfn(gpc->gpa), .flags = FOLL_WRITE, .hva = gpc->uhva, + .refcounted_page = &page, }; lockdep_assert_held(&gpc->refresh_lock); @@ -198,7 +201,7 @@ static kvm_pfn_t hva_to_pfn_retry(struct gfn_to_pfn_cache *gpc) if (new_khva != old_khva) gpc_unmap(new_pfn, new_khva); - kvm_release_pfn_clean(new_pfn); + kvm_release_page_unused(page); cond_resched(); } @@ -218,7 +221,7 @@ static kvm_pfn_t hva_to_pfn_retry(struct gfn_to_pfn_cache *gpc) new_khva = gpc_map(new_pfn); if (!new_khva) { - kvm_release_pfn_clean(new_pfn); + kvm_release_page_unused(page); goto out_error; } @@ -236,11 +239,11 @@ static kvm_pfn_t hva_to_pfn_retry(struct gfn_to_pfn_cache *gpc) gpc->khva = new_khva + offset_in_page(gpc->uhva); /* - * Put the reference to the _new_ pfn. The pfn is now tracked by the + * Put the reference to the _new_ page. The page is now tracked by the * cache and can be safely migrated, swapped, etc... as the cache will * invalidate any mappings in response to relevant mmu_notifier events. */ - kvm_release_pfn_clean(new_pfn); + kvm_release_page_clean(page); return 0; From patchwork Thu Oct 10 18:23:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831237 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id CCDFBD2445C for ; Thu, 10 Oct 2024 22:16:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:Content-Type:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=/prKt8Bu+DRISNBcov5NrkfgoAIYHMahITKzCWpSrCI=; b=gpeci023oL1lwY 9KTlIsRVVjdoCwHGyYsv+2b1bYJg5DuoSuv9IjfhbT9U7/etY1V0U+A3ei+5PYPd1zscSC5julBsp Hzu9DYb/HbTs+jzrOwsconNIb2xRNzMq25rFJKWMdNz41gk0C8jDrFovEiP2RHnAGdjdEEkRmwWSn iuofpjtRL72jIG4t31p54Hdj4jsyNaBRyW1RVgNjbUBkDe8lVL9yOFsE5aVIqrZdePi9tsid2xnAU XJFJuU0Hqj7op/3oB8LVPryLX/AIuw7exHUXh3wRuhkiHow9S44obPxClFi1QHXlPpHP5B13fYZkj Xk9xDe5mOfilrApiDEww==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1sz1SQ-0000000EVui-09j1; Thu, 10 Oct 2024 22:16:30 +0000 Received: from mail-yw1-x114a.google.com ([2607:f8b0:4864:20::114a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxrI-0000000DqR7-47aM for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:25:59 +0000 Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-6e2270a147aso21765637b3.0 for ; Thu, 10 Oct 2024 11:25:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584755; x=1729189555; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=/prKt8Bu+DRISNBcov5NrkfgoAIYHMahITKzCWpSrCI=; b=spm+Iun/rKPn0B8n5jp4vtXW4KoIEaOrEeQhdA6BP3YALJ/n0GskBlDL6e3o6m8fcJ B2FLXrGM9uCwxFoF53aH0EHoH0e8ffaFOJQDwK/LpbkXexqlsNTjFgC4wgFjjWRitsa2 UwACMIiv9Ob9acFXqYVSy594DaZ7OfCpmYZ3aqpwJOcSfJBcl+GOnZoVxguzaWh8MnAj Kpu9mPXvfrTaAVtdQVhqotBwfO2qkl2/kuOdYGCFEGrb3/wC8p5ti+UyIpUO2AHERCmz CJK7plDH++huahh/U/pxEGKc9USpkfmmKt+0HHbSHtgky+x/Xx5DiQah54yFJCS1hted Ighg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584755; x=1729189555; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=/prKt8Bu+DRISNBcov5NrkfgoAIYHMahITKzCWpSrCI=; b=FOnM6MRihMiZ2psgo1ZdwRUVkMkF1th1WTFnD4ajjAPn8Nc9qsi5431Wa011nBn5xw cd8MBk3dOEhcgBM96qtSkpe5HH0PjRfrBsuH0HEffT0C+pVCLm536mAuOcLgCMUxh9lX 46DqRYA0OleX3K+ng6Vu1gztoA8E9dGJEIahEYeH8c5zs6r/RxXo5JmDMGz+IO63xAbw kB7q3CmLJR8O+iIf1sqhiTzieBeZj5C3eux4nj6UyakGq2s0GV90oamSWQkipWojmA7e HEHF34zsXKXQYRSFCcqXLUrJGvhWU26RKfNhUwSZGkuxun+pBVvS9mvvsNcVKjsb6QnG nxDg== X-Forwarded-Encrypted: i=1; AJvYcCURe5tUdNwiz4kM2PvemkUIT+q8U/dXTBxs/3+Mf00DCpeZTqq1XIfdgldkrbZ4ppv5gPKgzQ45/fJNZdplTkJm@lists.infradead.org X-Gm-Message-State: AOJu0Yz0tXE18nNkY4e9j1tMNLnrewzW2+qc2ebtaCZGWqHL4UxZu4b/ mgHbJmd6arqA6p2cGVZMtTtPImI/OGfUMsTrrQUK5x7+KkiUbxFLB2H/CClgsj17FR58K60f9AT 1YQ== X-Google-Smtp-Source: AGHT+IHNZW39HKgOpo4XLx10sqcp5MldEDD35H2r/7z7SkgDatDPvbEuIFyvlgb9BtXWEl9gaR4EDWCFxGs= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:690c:2093:b0:6d4:30f0:3a74 with SMTP id 00721157ae682-6e322171640mr1011347b3.6.1728584755409; Thu, 10 Oct 2024 11:25:55 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:32 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-31-seanjc@google.com> Subject: [PATCH v13 30/85] KVM: Migrate kvm_vcpu_map() to kvm_follow_pfn() From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112557_112234_0742E634 X-CRM114-Status: GOOD ( 16.23 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: David Stevens Migrate kvm_vcpu_map() to kvm_follow_pfn(), and have it track whether or not the map holds a refcounted struct page. Precisely tracking struct page references will eventually allow removing kvm_pfn_to_refcounted_page() and its various wrappers. Signed-off-by: David Stevens [sean: use a pointer instead of a boolean] Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- include/linux/kvm_host.h | 2 +- virt/kvm/kvm_main.c | 26 ++++++++++++++++---------- 2 files changed, 17 insertions(+), 11 deletions(-) diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index e3c01cbbc41a..02ab3a657aa6 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -280,6 +280,7 @@ struct kvm_host_map { * can be used as guest memory but they are not managed by host * kernel). */ + struct page *refcounted_page; struct page *page; void *hva; kvm_pfn_t pfn; @@ -1238,7 +1239,6 @@ void kvm_release_pfn_dirty(kvm_pfn_t pfn); void kvm_set_pfn_dirty(kvm_pfn_t pfn); void kvm_set_pfn_accessed(kvm_pfn_t pfn); -void kvm_release_pfn(kvm_pfn_t pfn, bool dirty); int kvm_read_guest_page(struct kvm *kvm, gfn_t gfn, void *data, int offset, int len); int kvm_read_guest(struct kvm *kvm, gpa_t gpa, void *data, unsigned long len); diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 6cdbd0516d58..b1c1b7e4f33a 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -3093,21 +3093,21 @@ struct page *gfn_to_page(struct kvm *kvm, gfn_t gfn) } EXPORT_SYMBOL_GPL(gfn_to_page); -void kvm_release_pfn(kvm_pfn_t pfn, bool dirty) -{ - if (dirty) - kvm_release_pfn_dirty(pfn); - else - kvm_release_pfn_clean(pfn); -} - int kvm_vcpu_map(struct kvm_vcpu *vcpu, gfn_t gfn, struct kvm_host_map *map) { + struct kvm_follow_pfn kfp = { + .slot = gfn_to_memslot(vcpu->kvm, gfn), + .gfn = gfn, + .flags = FOLL_WRITE, + .refcounted_page = &map->refcounted_page, + }; + + map->refcounted_page = NULL; map->page = NULL; map->hva = NULL; map->gfn = gfn; - map->pfn = gfn_to_pfn(vcpu->kvm, gfn); + map->pfn = kvm_follow_pfn(&kfp); if (is_error_noslot_pfn(map->pfn)) return -EINVAL; @@ -3139,10 +3139,16 @@ void kvm_vcpu_unmap(struct kvm_vcpu *vcpu, struct kvm_host_map *map, bool dirty) if (dirty) kvm_vcpu_mark_page_dirty(vcpu, map->gfn); - kvm_release_pfn(map->pfn, dirty); + if (map->refcounted_page) { + if (dirty) + kvm_release_page_dirty(map->refcounted_page); + else + kvm_release_page_clean(map->refcounted_page); + } map->hva = NULL; map->page = NULL; + map->refcounted_page = NULL; } EXPORT_SYMBOL_GPL(kvm_vcpu_unmap); From patchwork Thu Oct 10 18:23:33 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831159 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 607C5D24458 for ; Thu, 10 Oct 2024 21:16:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:Content-Type:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=dQaiAwtn89tr8uPQzQfay17fvGV3F3WPslUlHA+BgDY=; b=uu5TgBz9eTGaOc V43NmPUnhJJFj94UTXGEGyuBXCSpcm+Lt3ewuwou+t/l0Hl69t0IFC7paYqcZyitm0tArAVRf6Xrk MoC/GGy9B3bcw7bPXcloPgl4NeZLBLP97EkAf85TTQhM6tjU99aMMlSylhmY87csy8NH10i4dC0/9 yxU01U1/UdJfESKcdElKYticUvWN891Te9mGJNFmx+WsHIAIhgVKUoo2UnRRf6F3UYMTttOgAOr+m Dz9Sm0gGq231K6xlskcXQftSHUY79UNPsPhCHqLgeukh0+QTt4H/kyQVUD0Rz3uhd7HfyHkaDcn9s wOrPhC7iIOZtZ1UL/K3A==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1sz0WK-0000000ENXC-1Z9Q; Thu, 10 Oct 2024 21:16:28 +0000 Received: from mail-yb1-xb49.google.com ([2607:f8b0:4864:20::b49]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxrK-0000000DqSy-2fuf for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:26:01 +0000 Received: by mail-yb1-xb49.google.com with SMTP id 3f1490d57ef6-e29135d1d0cso859770276.1 for ; Thu, 10 Oct 2024 11:25:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584757; x=1729189557; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=dQaiAwtn89tr8uPQzQfay17fvGV3F3WPslUlHA+BgDY=; b=OSACH8GjnuyuiXpMfRDQCjz+rzwPNx3aGaepvL099zA8znTQC1egQ2seldu9OeijMU LazKJ2ZGVLT3cJM5U1v/DaYRFVPu0uxfn8zMN/VSc5LNS7kFSdzT1N8jORlYegWBdM/M 6+R11SLiQDzZrRPEGc1qEG0QzXF6NWx2dCA+IXrYAeG9Ro77XlesdpijfZpvz09fesrT q+fr7nWvVHD1GEkg/Q0M4eSGZNQMP2USY0zm3KIrxgzWmSOvWe+t8nR4i6Zd1AWzpjTY FkihywB7cmxcQ55tLY2qp4bZYNfslEZRg29M9xzmTj8+zhGiB5ThGCdBROkdMErWT8rb +SJA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584757; x=1729189557; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=dQaiAwtn89tr8uPQzQfay17fvGV3F3WPslUlHA+BgDY=; b=jh+Wo4f1yDYtoZGUZnTb9tZdELdH2DekGlGcPBB3sqAbBZPJn2L2CIAsaMrdJrjgbv 2pk4RGPpyQ3rPSGguNEgbKisqHDxxwBI2cCbTCK3RJknWgFDksAi73Ulo/4GK4vDgCiN HNngJ9KmeCAb0OcCr6nAK+Qj/P5fWHB78uKDRoToYhZffj6O+us9Dfncw/zoJx3RRwnm RpqA3N0AY5ZmW/2SriPW0xfT3DGM/GC2S4ItaYiDczYHJ5unVfPqw/hN9lHPxAjVqMR1 BO031J33EvVSsJGFgCi/e9BCZDWboaAEryRAqGgk7sL8uevQbyZecz81UgkAArZejJJY 362g== X-Forwarded-Encrypted: i=1; AJvYcCW4U3IDL8ijyCS5qiOCW1X4fqFoXz59W2ZqLpOj34MdvUiMvlhVHDcT5bR4Wu1y375I36yCYSAfETAPAaWA3hxx@lists.infradead.org X-Gm-Message-State: AOJu0Ywj8UFKgYJoocKokNZzl89lNS+v8fKO9W/DJ/2MkaRYFnoMGnvW GZZy+o+pj7IVOOpjHCRL3cP6ec/VqUW5vJXQbmt6W1RHRqa6sn8Rh6OFCaZrpk/puIOcoxPsc8M A3w== X-Google-Smtp-Source: AGHT+IHY0yNiFmgeAXsfVKl7EbEBY2R3q1IXvjRBHtSPoxXhe2HbHq4twyB0Huv0R+/OHvP6V1ZW3nZzpzQ= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a25:80cb:0:b0:e20:2502:be14 with SMTP id 3f1490d57ef6-e28fe410672mr4860276.7.1728584757233; Thu, 10 Oct 2024 11:25:57 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:33 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-32-seanjc@google.com> Subject: [PATCH v13 31/85] KVM: Pin (as in FOLL_PIN) pages during kvm_vcpu_map() From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112559_060228_325ACA44 X-CRM114-Status: GOOD ( 24.18 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Pin, as in FOLL_PIN, pages when mapping them for direct access by KVM. As per Documentation/core-api/pin_user_pages.rst, writing to a page that was gotten via FOLL_GET is explicitly disallowed. Correct (uses FOLL_PIN calls): pin_user_pages() write to the data within the pages unpin_user_pages() INCORRECT (uses FOLL_GET calls): get_user_pages() write to the data within the pages put_page() Unfortunately, FOLL_PIN is a "private" flag, and so kvm_follow_pfn must use a one-off bool instead of being able to piggyback the "flags" field. Link: https://lwn.net/Articles/930667 Link: https://lore.kernel.org/all/cover.1683044162.git.lstoakes@gmail.com Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- include/linux/kvm_host.h | 2 +- virt/kvm/kvm_main.c | 54 +++++++++++++++++++++++++++++----------- virt/kvm/kvm_mm.h | 7 ++++++ 3 files changed, 47 insertions(+), 16 deletions(-) diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 02ab3a657aa6..8739b905d85b 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -280,7 +280,7 @@ struct kvm_host_map { * can be used as guest memory but they are not managed by host * kernel). */ - struct page *refcounted_page; + struct page *pinned_page; struct page *page; void *hva; kvm_pfn_t pfn; diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index b1c1b7e4f33a..40a59526d466 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -2814,9 +2814,12 @@ static kvm_pfn_t kvm_resolve_pfn(struct kvm_follow_pfn *kfp, struct page *page, */ if (map) { pfn = map->pfn; - page = kvm_pfn_to_refcounted_page(pfn); - if (page && !get_page_unless_zero(page)) - return KVM_PFN_ERR_FAULT; + + if (!kfp->pin) { + page = kvm_pfn_to_refcounted_page(pfn); + if (page && !get_page_unless_zero(page)) + return KVM_PFN_ERR_FAULT; + } } else { pfn = page_to_pfn(page); } @@ -2834,16 +2837,24 @@ static kvm_pfn_t kvm_resolve_pfn(struct kvm_follow_pfn *kfp, struct page *page, static bool hva_to_pfn_fast(struct kvm_follow_pfn *kfp, kvm_pfn_t *pfn) { struct page *page; + bool r; /* - * Fast pin a writable pfn only if it is a write fault request - * or the caller allows to map a writable pfn for a read fault - * request. + * Try the fast-only path when the caller wants to pin/get the page for + * writing. If the caller only wants to read the page, KVM must go + * down the full, slow path in order to avoid racing an operation that + * breaks Copy-on-Write (CoW), e.g. so that KVM doesn't end up pointing + * at the old, read-only page while mm/ points at a new, writable page. */ if (!((kfp->flags & FOLL_WRITE) || kfp->map_writable)) return false; - if (get_user_page_fast_only(kfp->hva, FOLL_WRITE, &page)) { + if (kfp->pin) + r = pin_user_pages_fast(kfp->hva, 1, FOLL_WRITE, &page) == 1; + else + r = get_user_page_fast_only(kfp->hva, FOLL_WRITE, &page); + + if (r) { *pfn = kvm_resolve_pfn(kfp, page, NULL, true); return true; } @@ -2872,10 +2883,21 @@ static int hva_to_pfn_slow(struct kvm_follow_pfn *kfp, kvm_pfn_t *pfn) struct page *page, *wpage; int npages; - npages = get_user_pages_unlocked(kfp->hva, 1, &page, flags); + if (kfp->pin) + npages = pin_user_pages_unlocked(kfp->hva, 1, &page, flags); + else + npages = get_user_pages_unlocked(kfp->hva, 1, &page, flags); if (npages != 1) return npages; + /* + * Pinning is mutually exclusive with opportunistically mapping a read + * fault as writable, as KVM should never pin pages when mapping memory + * into the guest (pinning is only for direct accesses from KVM). + */ + if (WARN_ON_ONCE(kfp->map_writable && kfp->pin)) + goto out; + /* map read fault as writable if possible */ if (!(flags & FOLL_WRITE) && kfp->map_writable && get_user_page_fast_only(kfp->hva, FOLL_WRITE, &wpage)) { @@ -2884,6 +2906,7 @@ static int hva_to_pfn_slow(struct kvm_follow_pfn *kfp, kvm_pfn_t *pfn) flags |= FOLL_WRITE; } +out: *pfn = kvm_resolve_pfn(kfp, page, NULL, flags & FOLL_WRITE); return npages; } @@ -3099,10 +3122,11 @@ int kvm_vcpu_map(struct kvm_vcpu *vcpu, gfn_t gfn, struct kvm_host_map *map) .slot = gfn_to_memslot(vcpu->kvm, gfn), .gfn = gfn, .flags = FOLL_WRITE, - .refcounted_page = &map->refcounted_page, + .refcounted_page = &map->pinned_page, + .pin = true, }; - map->refcounted_page = NULL; + map->pinned_page = NULL; map->page = NULL; map->hva = NULL; map->gfn = gfn; @@ -3139,16 +3163,16 @@ void kvm_vcpu_unmap(struct kvm_vcpu *vcpu, struct kvm_host_map *map, bool dirty) if (dirty) kvm_vcpu_mark_page_dirty(vcpu, map->gfn); - if (map->refcounted_page) { + if (map->pinned_page) { if (dirty) - kvm_release_page_dirty(map->refcounted_page); - else - kvm_release_page_clean(map->refcounted_page); + kvm_set_page_dirty(map->pinned_page); + kvm_set_page_accessed(map->pinned_page); + unpin_user_page(map->pinned_page); } map->hva = NULL; map->page = NULL; - map->refcounted_page = NULL; + map->pinned_page = NULL; } EXPORT_SYMBOL_GPL(kvm_vcpu_unmap); diff --git a/virt/kvm/kvm_mm.h b/virt/kvm/kvm_mm.h index d3ac1ba8ba66..acef3f5c582a 100644 --- a/virt/kvm/kvm_mm.h +++ b/virt/kvm/kvm_mm.h @@ -30,6 +30,13 @@ struct kvm_follow_pfn { /* FOLL_* flags modifying lookup behavior, e.g. FOLL_WRITE. */ unsigned int flags; + /* + * Pin the page (effectively FOLL_PIN, which is an mm/ internal flag). + * The page *must* be pinned if KVM will write to the page via a kernel + * mapping, e.g. via kmap(), mremap(), etc. + */ + bool pin; + /* * If non-NULL, try to get a writable mapping even for a read fault. * Set to true if a writable mapping was obtained. From patchwork Thu Oct 10 18:23:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831158 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 7E01AD24459 for ; Thu, 10 Oct 2024 21:16:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:Content-Type:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=VITkHk6wYCk30qmAXnWfpQxFdItqbxje0xl/TrLgm28=; b=cT0+wMfpS6mU3j gINanGpNXfl2mPSke906exyjpzqses+wgs2Qj7L2ROcyp4Al+8J1LQcRvaqEX2378v4PkKDbCP0+K MORF1R4X2c85XWBD8ZzKf/1u3b16grlEZlZDdAuCAHFNmd6PPg+BtJ72j/hQdNnavbCMAxFvwpoSo 1lnckoPzM5bHzwzkgwvGBHbwaichzTnajOHS7+YsUXg9gWBL5E6Kn4jnq9KEHBDVsO36jcHlIOAVW /lk4MRIWYa+LUPBCD6nk5wERW+JCkYUKRRO3ARWdkBuCF23eVlYaO9azfX8GbLHBaNb+hOAtaxstB mLKLYZLFR1Jy5l09kyXQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1sz0WL-0000000ENXk-2s9F; Thu, 10 Oct 2024 21:16:29 +0000 Received: from mail-pg1-x549.google.com ([2607:f8b0:4864:20::549]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxrM-0000000DqU8-1QVA for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:26:02 +0000 Received: by mail-pg1-x549.google.com with SMTP id 41be03b00d2f7-6818fa37eecso1273634a12.1 for ; Thu, 10 Oct 2024 11:25:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584759; x=1729189559; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=VITkHk6wYCk30qmAXnWfpQxFdItqbxje0xl/TrLgm28=; b=DDGV/BA9cByk02/tA4Cb/+AgaPSPEwimwcElwsHI1jiRIZELRkjDSYk3e7ZNGCv0X6 Xi76UfOwi0uHKt2IJK3aT1JjYxjdtbyaBwyAnFT3U3gn/rBV0L9eCzkzxmRa04PKCjLe ArtEmn/ENFhUYin/BrtkhAGwGmHDMtR6KKmWPoh91MCyGM2EwU8EoTsRgoFMiTVWqisT 3pgL2ELhDC4PmAeoYGgmfJl21pGh+O/G6/I+7+8SOGtDtuMlcFEKqLEi0Hi7fzSGVvFB ySrMi7ysavXcWJMDfUQY4qfyFR54JIxt96ot/kZsE2L9OD6Z1BG2yOcBh05vKntTesz9 ibPA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584759; x=1729189559; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=VITkHk6wYCk30qmAXnWfpQxFdItqbxje0xl/TrLgm28=; b=uj259BaZitoX65YwrZu2J8FKfvisNShjWHzq5r5wsf10Mubb+36pRRHBp7tXPdj+bB SqnUgaAl5wSJ+5h6RYQvBvcKWfG07g+gBUAz0TNE+okx0Unp4Cn7al8qt519hM1tC320 tdWc1Kj3tWHiOtTwbk/U6iImd2IkeD6Wk7eBxUC6g1uKc8jGL3h36DQSSBfbo7Iv2pBF 7rb/E0GvLx61AYXcUiiQbNRKhPPC8IVw0sCHBgv5J5x7N7d/lLdqEyLLcMxpMAq7oaI6 N+SBWkQUjWliwjOuLHGJRCeTV1TQyfLcpYG7vS7nIldU43fA4pB9UPyfcCT4kOucb1Tm gCRw== X-Forwarded-Encrypted: i=1; AJvYcCXyXoWfbhpov+IC3ccMt0J9A3T15ox13rlIzzj8o6hAq2t1lv5IKsZ0W9tbI3I6eh90DOsfpeyXewXl0IKzbFOW@lists.infradead.org X-Gm-Message-State: AOJu0YyDez5QvX8a2uarmflxoKfYHocYi9dNxbaCqIP0ngz2s0d0jJvx R7X5TCzl1G6r4sR9OO0LcNDXbuzrZ0vhwLLsZF1Rsf3kLTwoUNdM8+Y+2hNDuHFzeAPF7R6fs4u mhQ== X-Google-Smtp-Source: AGHT+IEekecUM0XVWoBmNjTczqp1a2/ixTMw1fOAmBrSm/Dp9XgHLEkyd54mJG+SlDPSB0szYk/zKOq5dxU= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a65:678f:0:b0:7e9:f98c:e9f7 with SMTP id 41be03b00d2f7-7ea5359ed14mr32a12.10.1728584759044; Thu, 10 Oct 2024 11:25:59 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:34 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-33-seanjc@google.com> Subject: [PATCH v13 32/85] KVM: nVMX: Mark vmcs12's APIC access page dirty when unmapping From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112600_447477_7CD30837 X-CRM114-Status: GOOD ( 12.85 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Mark the APIC access page as dirty when unmapping it from KVM. The fact that the page _shouldn't_ be written doesn't guarantee the page _won't_ be written. And while the contents are likely irrelevant, the values _are_ visible to the guest, i.e. dropping writes would be visible to the guest (though obviously highly unlikely to be problematic in practice). Marking the map dirty will allow specifying the write vs. read-only when *mapping* the memory, which in turn will allow creating read-only maps. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/x86/kvm/vmx/nested.c | 7 +------ 1 file changed, 1 insertion(+), 6 deletions(-) diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c index 81865db18e12..ff83b56fe2fa 100644 --- a/arch/x86/kvm/vmx/nested.c +++ b/arch/x86/kvm/vmx/nested.c @@ -318,12 +318,7 @@ static void nested_put_vmcs12_pages(struct kvm_vcpu *vcpu) { struct vcpu_vmx *vmx = to_vmx(vcpu); - /* - * Unpin physical memory we referred to in the vmcs02. The APIC access - * page's backing page (yeah, confusing) shouldn't actually be accessed, - * and if it is written, the contents are irrelevant. - */ - kvm_vcpu_unmap(vcpu, &vmx->nested.apic_access_page_map, false); + kvm_vcpu_unmap(vcpu, &vmx->nested.apic_access_page_map, true); kvm_vcpu_unmap(vcpu, &vmx->nested.virtual_apic_map, true); kvm_vcpu_unmap(vcpu, &vmx->nested.pi_desc_map, true); vmx->nested.pi_desc = NULL; From patchwork Thu Oct 10 18:23:35 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831160 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id F256FD2444F for ; Thu, 10 Oct 2024 21:16:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:Content-Type:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=+DlXoUFYFzJpPJut8ySNZ/avzuJs4Hk+oSmN6VTWgSk=; b=QHleOL17Xa36ne pdzj/f1O6WkYvuDDE2/slpapGFTJqqU7C/TOE1slof5kBavNc1k/uRqY9yZynMsq1zHU2Gv750s+7 feGUOkyiGl2hULcr7X0bEcDD4uZXMfNrhhbW+D4MlPeiuaLZbJ8Pml2sH6m91EIE+CZq1dxqzOqbz NIYaNCYfjKar2mKJ6Nj3P+4tcEcnl6tsny6tVDnN2nX4sEZjBwfQ+O561+JXU0xQkLIoqVI+2Bvf1 Gqsc0yyrfxeEk8AT8xLQva5Vbita3u1WIXYdFtpBdKTYZNiSdSvBkCR8Kh0ruJGi7TgI7H9M5Vc/E p1qO4FDn4ky0Iw5DzrLg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1sz0WN-0000000ENZR-24WH; Thu, 10 Oct 2024 21:16:31 +0000 Received: from mail-yw1-x1149.google.com ([2607:f8b0:4864:20::1149]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxrO-0000000DqVI-0fjN for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:26:04 +0000 Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-6e26ba37314so24856857b3.0 for ; Thu, 10 Oct 2024 11:26:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584761; x=1729189561; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=+DlXoUFYFzJpPJut8ySNZ/avzuJs4Hk+oSmN6VTWgSk=; b=4EMLSx3Ftaw9gpvv4YrUXwm4Ks7qKUIi5Av2oi+legj4y7IQEeq9x4wlwZS+YWWx8t 0xit1JXqyQbkuGjTIvQTmIh9VVxkJnzZXwCY7NnCxL7Tzx0AQ0WqR/rS8TtjzL7zlSar vqNLhoD9LADMuTHQFqSPrwSjvccSQtNnO1Ox0zkU50RacgMm0RN7WZ6nwppBMtVycT3Q 5VtuL7yKy6hZPQuGehnhFA4hxkc7QeNABSIZ4rdhaX3h1YQOLR+4r+p6RuoQu94V8inI q5M+hfNWjQ/vrjfsKEFYLuelSGw5zlEhMvv7lcNZQL9Y3xwO0lH2zxpMwHOTTMe7gdJJ bUiw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584761; x=1729189561; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=+DlXoUFYFzJpPJut8ySNZ/avzuJs4Hk+oSmN6VTWgSk=; b=r7QKDKRWGwQmR/2dm/mjcJxLFcRNyKI6aaEOYJsUs3Wj+s6E08QEnTLmuWoAcHIOwB AlpaS+neARznQ+LaT0ohOsbWBo41H0SYB3LzUN1+3pIwB0Ab7H43j4cJ85kpRNYvZw9V Lo29COQU8IFjtq6wuoYpC8O7wfKul9Sbib0XphdBXCwJwBw4bm6rdzrfhQsSyVqGKEre fg7nLxkZUhjRhQD3J8IGnRtGQb+8s8U6rb7O9hr5JcXAkEuZUb27pW11pO3KNBWBTV+2 5wYTPPbLMqmsMhrg2Bcjskb3DfODPgkMwop9RMmGKo10g+W9fn3sw2C63pf+aG4c/6gc vXKw== X-Forwarded-Encrypted: i=1; AJvYcCXy5vkOQMOjzpEIAIHtBGZK0lpSxj1Bf2cn7e1NwZy4ePaszC5Dl4IGG2SU7P/xcxsOTaG6WkMBGDdUmPaoRcUe@lists.infradead.org X-Gm-Message-State: AOJu0Yw6eAmRB+qjJKh+q6zXXk6GgH9RYKlhoQXyfHqR6VnGZ9X8xOU/ dyD3zNB8fHAuGBcmOzjSlZ30vDbKnf28O0gJrTwHqaJi8eytNfIJsEfc+xF+hNpS5w5ftGPxXZv Q3A== X-Google-Smtp-Source: AGHT+IHYBe/gylmYYB76Y4JbCGQqBxyU7NvKFk8CzbtQRqqOSH0kk5njqmBLFW7wxt+Okx9MKc9CHg/F7Og= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:690c:4349:b0:6e2:1713:bdb5 with SMTP id 00721157ae682-6e32217cfd3mr118607b3.5.1728584760829; Thu, 10 Oct 2024 11:26:00 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:35 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-34-seanjc@google.com> Subject: [PATCH v13 33/85] KVM: Pass in write/dirty to kvm_vcpu_map(), not kvm_vcpu_unmap() From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112602_355714_FC53276F X-CRM114-Status: GOOD ( 18.02 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Now that all kvm_vcpu_{,un}map() users pass "true" for @dirty, have them pass "true" as a @writable param to kvm_vcpu_map(), and thus create a read-only mapping when possible. Note, creating read-only mappings can be theoretically slower, as they don't play nice with fast GUP due to the need to break CoW before mapping the underlying PFN. But practically speaking, creating a mapping isn't a super hot path, and getting a writable mapping for reading is weird and confusing. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/x86/kvm/svm/nested.c | 4 ++-- arch/x86/kvm/svm/sev.c | 2 +- arch/x86/kvm/svm/svm.c | 8 ++++---- arch/x86/kvm/vmx/nested.c | 16 ++++++++-------- include/linux/kvm_host.h | 20 ++++++++++++++++++-- virt/kvm/kvm_main.c | 12 +++++++----- 6 files changed, 40 insertions(+), 22 deletions(-) diff --git a/arch/x86/kvm/svm/nested.c b/arch/x86/kvm/svm/nested.c index d5314cb7dff4..9f9478bdecfc 100644 --- a/arch/x86/kvm/svm/nested.c +++ b/arch/x86/kvm/svm/nested.c @@ -922,7 +922,7 @@ int nested_svm_vmrun(struct kvm_vcpu *vcpu) nested_svm_vmexit(svm); out: - kvm_vcpu_unmap(vcpu, &map, true); + kvm_vcpu_unmap(vcpu, &map); return ret; } @@ -1126,7 +1126,7 @@ int nested_svm_vmexit(struct vcpu_svm *svm) vmcb12->control.exit_int_info_err, KVM_ISA_SVM); - kvm_vcpu_unmap(vcpu, &map, true); + kvm_vcpu_unmap(vcpu, &map); nested_svm_transition_tlb_flush(vcpu); diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c index 0b851ef937f2..4557ff3804ae 100644 --- a/arch/x86/kvm/svm/sev.c +++ b/arch/x86/kvm/svm/sev.c @@ -3468,7 +3468,7 @@ void sev_es_unmap_ghcb(struct vcpu_svm *svm) sev_es_sync_to_ghcb(svm); - kvm_vcpu_unmap(&svm->vcpu, &svm->sev_es.ghcb_map, true); + kvm_vcpu_unmap(&svm->vcpu, &svm->sev_es.ghcb_map); svm->sev_es.ghcb = NULL; } diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c index 9df3e1e5ae81..c1e29307826b 100644 --- a/arch/x86/kvm/svm/svm.c +++ b/arch/x86/kvm/svm/svm.c @@ -2299,7 +2299,7 @@ static int vmload_vmsave_interception(struct kvm_vcpu *vcpu, bool vmload) svm_copy_vmloadsave_state(vmcb12, svm->vmcb); } - kvm_vcpu_unmap(vcpu, &map, true); + kvm_vcpu_unmap(vcpu, &map); return ret; } @@ -4714,7 +4714,7 @@ static int svm_enter_smm(struct kvm_vcpu *vcpu, union kvm_smram *smram) svm_copy_vmrun_state(map_save.hva + 0x400, &svm->vmcb01.ptr->save); - kvm_vcpu_unmap(vcpu, &map_save, true); + kvm_vcpu_unmap(vcpu, &map_save); return 0; } @@ -4774,9 +4774,9 @@ static int svm_leave_smm(struct kvm_vcpu *vcpu, const union kvm_smram *smram) svm->nested.nested_run_pending = 1; unmap_save: - kvm_vcpu_unmap(vcpu, &map_save, true); + kvm_vcpu_unmap(vcpu, &map_save); unmap_map: - kvm_vcpu_unmap(vcpu, &map, true); + kvm_vcpu_unmap(vcpu, &map); return ret; } diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c index ff83b56fe2fa..259fe445e695 100644 --- a/arch/x86/kvm/vmx/nested.c +++ b/arch/x86/kvm/vmx/nested.c @@ -231,7 +231,7 @@ static inline void nested_release_evmcs(struct kvm_vcpu *vcpu) struct kvm_vcpu_hv *hv_vcpu = to_hv_vcpu(vcpu); struct vcpu_vmx *vmx = to_vmx(vcpu); - kvm_vcpu_unmap(vcpu, &vmx->nested.hv_evmcs_map, true); + kvm_vcpu_unmap(vcpu, &vmx->nested.hv_evmcs_map); vmx->nested.hv_evmcs = NULL; vmx->nested.hv_evmcs_vmptr = EVMPTR_INVALID; @@ -318,9 +318,9 @@ static void nested_put_vmcs12_pages(struct kvm_vcpu *vcpu) { struct vcpu_vmx *vmx = to_vmx(vcpu); - kvm_vcpu_unmap(vcpu, &vmx->nested.apic_access_page_map, true); - kvm_vcpu_unmap(vcpu, &vmx->nested.virtual_apic_map, true); - kvm_vcpu_unmap(vcpu, &vmx->nested.pi_desc_map, true); + kvm_vcpu_unmap(vcpu, &vmx->nested.apic_access_page_map); + kvm_vcpu_unmap(vcpu, &vmx->nested.virtual_apic_map); + kvm_vcpu_unmap(vcpu, &vmx->nested.pi_desc_map); vmx->nested.pi_desc = NULL; } @@ -624,7 +624,7 @@ static inline bool nested_vmx_prepare_msr_bitmap(struct kvm_vcpu *vcpu, int msr; unsigned long *msr_bitmap_l1; unsigned long *msr_bitmap_l0 = vmx->nested.vmcs02.msr_bitmap; - struct kvm_host_map msr_bitmap_map; + struct kvm_host_map map; /* Nothing to do if the MSR bitmap is not in use. */ if (!cpu_has_vmx_msr_bitmap() || @@ -647,10 +647,10 @@ static inline bool nested_vmx_prepare_msr_bitmap(struct kvm_vcpu *vcpu, return true; } - if (kvm_vcpu_map(vcpu, gpa_to_gfn(vmcs12->msr_bitmap), &msr_bitmap_map)) + if (kvm_vcpu_map_readonly(vcpu, gpa_to_gfn(vmcs12->msr_bitmap), &map)) return false; - msr_bitmap_l1 = (unsigned long *)msr_bitmap_map.hva; + msr_bitmap_l1 = (unsigned long *)map.hva; /* * To keep the control flow simple, pay eight 8-byte writes (sixteen @@ -714,7 +714,7 @@ static inline bool nested_vmx_prepare_msr_bitmap(struct kvm_vcpu *vcpu, nested_vmx_set_intercept_for_msr(vmx, msr_bitmap_l1, msr_bitmap_l0, MSR_IA32_FLUSH_CMD, MSR_TYPE_W); - kvm_vcpu_unmap(vcpu, &msr_bitmap_map, false); + kvm_vcpu_unmap(vcpu, &map); vmx->nested.force_msr_bitmap_recalc = false; diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 8739b905d85b..9263375d0362 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -285,6 +285,7 @@ struct kvm_host_map { void *hva; kvm_pfn_t pfn; kvm_pfn_t gfn; + bool writable; }; /* @@ -1312,8 +1313,23 @@ void mark_page_dirty(struct kvm *kvm, gfn_t gfn); struct kvm_memslots *kvm_vcpu_memslots(struct kvm_vcpu *vcpu); struct kvm_memory_slot *kvm_vcpu_gfn_to_memslot(struct kvm_vcpu *vcpu, gfn_t gfn); kvm_pfn_t kvm_vcpu_gfn_to_pfn(struct kvm_vcpu *vcpu, gfn_t gfn); -int kvm_vcpu_map(struct kvm_vcpu *vcpu, gpa_t gpa, struct kvm_host_map *map); -void kvm_vcpu_unmap(struct kvm_vcpu *vcpu, struct kvm_host_map *map, bool dirty); + +int __kvm_vcpu_map(struct kvm_vcpu *vcpu, gpa_t gpa, struct kvm_host_map *map, + bool writable); +void kvm_vcpu_unmap(struct kvm_vcpu *vcpu, struct kvm_host_map *map); + +static inline int kvm_vcpu_map(struct kvm_vcpu *vcpu, gpa_t gpa, + struct kvm_host_map *map) +{ + return __kvm_vcpu_map(vcpu, gpa, map, true); +} + +static inline int kvm_vcpu_map_readonly(struct kvm_vcpu *vcpu, gpa_t gpa, + struct kvm_host_map *map) +{ + return __kvm_vcpu_map(vcpu, gpa, map, false); +} + unsigned long kvm_vcpu_gfn_to_hva(struct kvm_vcpu *vcpu, gfn_t gfn); unsigned long kvm_vcpu_gfn_to_hva_prot(struct kvm_vcpu *vcpu, gfn_t gfn, bool *writable); int kvm_vcpu_read_guest_page(struct kvm_vcpu *vcpu, gfn_t gfn, void *data, int offset, diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 40a59526d466..080740f65061 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -3116,7 +3116,8 @@ struct page *gfn_to_page(struct kvm *kvm, gfn_t gfn) } EXPORT_SYMBOL_GPL(gfn_to_page); -int kvm_vcpu_map(struct kvm_vcpu *vcpu, gfn_t gfn, struct kvm_host_map *map) +int __kvm_vcpu_map(struct kvm_vcpu *vcpu, gfn_t gfn, struct kvm_host_map *map, + bool writable) { struct kvm_follow_pfn kfp = { .slot = gfn_to_memslot(vcpu->kvm, gfn), @@ -3130,6 +3131,7 @@ int kvm_vcpu_map(struct kvm_vcpu *vcpu, gfn_t gfn, struct kvm_host_map *map) map->page = NULL; map->hva = NULL; map->gfn = gfn; + map->writable = writable; map->pfn = kvm_follow_pfn(&kfp); if (is_error_noslot_pfn(map->pfn)) @@ -3146,9 +3148,9 @@ int kvm_vcpu_map(struct kvm_vcpu *vcpu, gfn_t gfn, struct kvm_host_map *map) return map->hva ? 0 : -EFAULT; } -EXPORT_SYMBOL_GPL(kvm_vcpu_map); +EXPORT_SYMBOL_GPL(__kvm_vcpu_map); -void kvm_vcpu_unmap(struct kvm_vcpu *vcpu, struct kvm_host_map *map, bool dirty) +void kvm_vcpu_unmap(struct kvm_vcpu *vcpu, struct kvm_host_map *map) { if (!map->hva) return; @@ -3160,11 +3162,11 @@ void kvm_vcpu_unmap(struct kvm_vcpu *vcpu, struct kvm_host_map *map, bool dirty) memunmap(map->hva); #endif - if (dirty) + if (map->writable) kvm_vcpu_mark_page_dirty(vcpu, map->gfn); if (map->pinned_page) { - if (dirty) + if (map->writable) kvm_set_page_dirty(map->pinned_page); kvm_set_page_accessed(map->pinned_page); unpin_user_page(map->pinned_page); From patchwork Thu Oct 10 18:23:36 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831162 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id F0B19D2444F for ; Thu, 10 Oct 2024 21:16:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:Content-Type:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=w3u38MFKj4Hgu0NiviVwRjtFpk/ioMNBGBDQlt4DAgY=; b=KQhpwrgpGQYmdx O3+2TRO33td14LQoMMIJZVB42ilxAcpHSWh132C61yDGwlUx3BvPE5PVshtAtKGoiLOIrsf8Qy6Uq 2g3lWQC5QIhLu3gtIPIesmf4r2tIzh7lsPZX1zZ9dwXu4wJDEEzL4qCo63LxxTIC4YF5kd6rfoVI+ xOQti6ES3ke7o1hBPwSslZ3zPKWiPxV6WDGomp+BHyKQ/fPNFdaxTcR3+tF3dV4mx77OGFjXjHfMS Rtg2Tq4gprSzpO8PDp+ZOpLX55enRXktpKfNuo2kiaCbnBu4Bu4KPF24gSeoMwiYIbsPG7bFKPd0T 2RZDbYIibDKGlkPLjc/A==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1sz0WU-0000000ENfu-336v; Thu, 10 Oct 2024 21:16:38 +0000 Received: from mail-pf1-x44a.google.com ([2607:f8b0:4864:20::44a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxrQ-0000000DqX4-2CxM for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:26:07 +0000 Received: by mail-pf1-x44a.google.com with SMTP id d2e1a72fcca58-71e00c8adf9so1318776b3a.1 for ; Thu, 10 Oct 2024 11:26:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584763; x=1729189563; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=w3u38MFKj4Hgu0NiviVwRjtFpk/ioMNBGBDQlt4DAgY=; b=wwsVmKdK+SipFg1qIF50TrdP6Oo7jO28ReLYz/VDjQtbPUYfeyWnk0GVNrM0uvKMns yzO1p2PHSyt6/6J6LjJccz4eC9X33n2N2DFqSywzlwGAsAWPs0+b7iefqw5tZpUARPQu 9OG64S1K6L+hc8UcCB61w0V2JL630JzshC0mLEAVFvK8USjgHGCuqzpy7gkmEW0tty66 NRob1YtQ6GGUHdVw6r2QY0qsQ4+DlBD5nwBCrRtcLzLwX9krXorOWvD+nF58Mr7YwjaH LDcNdo4BOKyAvdYIXqY2QFZaKTbi1CdYfH/QTc06gv+b8kc00QbcJRcZEUai+QUcejZU 3v4Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584763; x=1729189563; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=w3u38MFKj4Hgu0NiviVwRjtFpk/ioMNBGBDQlt4DAgY=; b=jNqk27E4X21ji96C1SPbLJusfoFLtFNcV4bMA77rovDSUHP0l13Tqs6XybdflZQyK7 N7SOmPc4CKbjDyRbZaZ+LkEPagdrlh+dAJj+QBYsjm3eeFfWHLY+gl6rgofM84frlugn Di5/HbqW9xL/yVHkX6aJoqMeoMKZyJd/VwNanJ34vSWx6S7pMD6XHHkCWqSdjZfjzQyB ZdBl7gTZTy/X0gdoRK6kXKBBiSJRkKSBhD9U8/Sjyv3MYvLfrYdUXPnkCBiDLVcYQ6bF AJLbNChbgn0hmXcgLuzAetNcQsILlr3gLyhzjz7t4m4iEdLeZ5L40i2Z8kXHHhwfsXOu eFvA== X-Forwarded-Encrypted: i=1; AJvYcCWtmx6CFSS/AqX0K5PqcsP8gG5/W9BaHcbSPvpwi22xd0hfI45joFKOGTdbmEYm6azpF7iVbE9MMRZwhw9R5ILQ@lists.infradead.org X-Gm-Message-State: AOJu0YzP/Sd5/hb5w+YmfZXawO9VGtUkG4jO7tqGVMtKeWr7lCk1UD3u F54gHczrSE+1VsdisZrB0IxyuAzRSd1QyhRrtslOa7XOV6ka13dXnjUBsyYjZISrSx9Pm8hYnIo mCA== X-Google-Smtp-Source: AGHT+IHthU1bxxWE76UmRU/cmSB/2b+tJYVulTyd1SGmZGLYBHQFEbukUNmDJAlwNs2CFth+SfvGFADhDCI= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:6a00:2f51:b0:71e:dd:8f9b with SMTP id d2e1a72fcca58-71e1dbe8750mr6976b3a.5.1728584762866; Thu, 10 Oct 2024 11:26:02 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:36 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-35-seanjc@google.com> Subject: [PATCH v13 34/85] KVM: Get writable mapping for __kvm_vcpu_map() only when necessary From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112604_691212_69C9CFF9 X-CRM114-Status: GOOD ( 11.93 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org When creating a memory map for read, don't request a writable pfn from the primary MMU. While creating read-only mappings can be theoretically slower, as they don't play nice with fast GUP due to the need to break CoW before mapping the underlying PFN, practically speaking, creating a mapping isn't a super hot path, and getting a writable mapping for reading is weird and confusing. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- virt/kvm/kvm_main.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 080740f65061..b845e9252633 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -3122,7 +3122,7 @@ int __kvm_vcpu_map(struct kvm_vcpu *vcpu, gfn_t gfn, struct kvm_host_map *map, struct kvm_follow_pfn kfp = { .slot = gfn_to_memslot(vcpu->kvm, gfn), .gfn = gfn, - .flags = FOLL_WRITE, + .flags = writable ? FOLL_WRITE : 0, .refcounted_page = &map->pinned_page, .pin = true, }; From patchwork Thu Oct 10 18:23:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831163 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5576DD24454 for ; Thu, 10 Oct 2024 21:16:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:Content-Type:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=ByBZxBThzycQ9+5cWv7LHsEIs8PvxVG+sWS1etQw10Y=; b=QzWV1SzfqBdZPM aIJmvwgwtxrM+eZjP57DTx6q1eXwZv/cEk3me61kWp4oxZAjMCB9E6S8WDGrwi5UIgcAss4YMWygS SZ4j54PTJ4lX/Lz136UQXsWLvK5bl3357P2+RLlMzxnc5KrnFdn0f8nVSY0f5Gpmxj4v7a3+6cjJ3 mgssSgXwAd7htn84tnp+/oQHz8H41SpDaqdDxlJJ8rFgoPj6yep67Jbt5XIi3NyZJ7JBko3gbdH9q 5cz0ijqhb6434PcUU5WlU6DJvnyrgdxD8KtqKOEFammVZ9UdKq7/XsEqhgLxwt00Eiyg5rhom1YHB qQ9t9BQlAbgqtuAK3HTQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1sz0Wd-0000000ENoq-0wcZ; Thu, 10 Oct 2024 21:16:47 +0000 Received: from mail-pg1-x54a.google.com ([2607:f8b0:4864:20::54a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxrS-0000000DqYl-0BCj for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:26:08 +0000 Received: by mail-pg1-x54a.google.com with SMTP id 41be03b00d2f7-7ea00becea0so1483358a12.2 for ; Thu, 10 Oct 2024 11:26:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584765; x=1729189565; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=ByBZxBThzycQ9+5cWv7LHsEIs8PvxVG+sWS1etQw10Y=; b=VR8H35J0pZub8PS/JM9vp2UKN2Zg/mExpEpDuTzNIH+FW762bLByCaHMK7vSiwjC4T x8MhTpwSZld0moJyvFJhg/1ohUP78VLZJ2X6yOnBjBPRs6ofdVEEpZ+NJRm6GxM2xpIF vvKfeZIRuv9EOalwyRkzpan94NKDtcJUocISdZUZPJJRPvvy2UgbHhkgYUXorzexJZLT NrdOa2be9cJP1lh0sOESqVIGY224l57JOIbn+rYAnjihqpPZGA1yK9opXvvM7mbxD+LE 3SPXzF81rFiZFm76cK9RPr7dhR2qr/NnlTVK8HwKmkY+DzSCLdf0+4721hzjhid+NXrL 3Crg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584765; x=1729189565; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=ByBZxBThzycQ9+5cWv7LHsEIs8PvxVG+sWS1etQw10Y=; b=dhsCeRy0/PCRKlzezkBBaWg37YbEagsq0isb4QZbr1gA3HTQOFoXo7wSjY4pMKNllV MQypuQRGc0PfshZ2gFucvoyyPNSJV2nMamVe2SdMJD8W8uYixPamvT/76nlizv7GeiTa UF9rEMmN8h45JntfdtWRDJGnHEWlb9eojqVbyJp517qQitTfnwAB9WathiDe0CI7F3gN gijqNv3MSYk+5Yhjk79qIsb/qoZ5/LKW+JSs8PZ0dCFH3/G3qL3ah9ZMYUGzN03SRMrQ qoseGo2DM6258EAc3PcKyUP3tzXVl6J8ccMXzLgZPVoYgcCDd2vzEhAeddVfkIjhSexI XbBw== X-Forwarded-Encrypted: i=1; AJvYcCUHa00S00vHpuWGBwKdxMbBqkhst4N6HqWnjPy8xouhbq3rdpQD7LYNRCQBHtm0SSbCgWHEaIhoNUbN2uj35z9Q@lists.infradead.org X-Gm-Message-State: AOJu0YyWG4T/BaPKXuNm4s7TdxFWSs37wwHVDPXeZJrxwW+B5garsd8B YBfYvIppS2Hcsr4rnOvMa14yGvPFg0gdnmmWyRT45LoxVWrbi2EcYdSCHei47j0fu5P9m6pTh+y C+g== X-Google-Smtp-Source: AGHT+IE23QkhjcWr/3QASJ/S9+dapZUNy7Z4CCepJKqapzuUx8eBu6OhWGfM5laLrlsw47tRhKuFSYugMsQ= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a63:ff61:0:b0:7cb:c8c3:3811 with SMTP id 41be03b00d2f7-7ea5356a658mr52a12.5.1728584764791; Thu, 10 Oct 2024 11:26:04 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:37 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-36-seanjc@google.com> Subject: [PATCH v13 35/85] KVM: Disallow direct access (w/o mmu_notifier) to unpinned pfn by default From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112606_176871_8D8BA1A3 X-CRM114-Status: GOOD ( 16.56 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Add an off-by-default module param to control whether or not KVM is allowed to map memory that isn't pinned, i.e. that KVM can't guarantee won't be freed while it is mapped into KVM and/or the guest. Don't remove the functionality entirely, as there are use cases where mapping unpinned memory is safe (as defined by the platform owner), e.g. when memory is hidden from the kernel and managed by userspace, in which case userspace is already fully trusted to not muck with guest memory mappings. But for more typical setups, mapping unpinned memory is wildly unsafe, and unnecessary. The APIs are used exclusively by x86's nested virtualization support, and there is no known (or sane) use case for mapping PFN-mapped memory a KVM guest _and_ letting the guest use it for virtualization structures. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- virt/kvm/kvm_main.c | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index b845e9252633..6dcb4f0eed3e 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -94,6 +94,13 @@ unsigned int halt_poll_ns_shrink = 2; module_param(halt_poll_ns_shrink, uint, 0644); EXPORT_SYMBOL_GPL(halt_poll_ns_shrink); +/* + * Allow direct access (from KVM or the CPU) without MMU notifier protection + * to unpinned pages. + */ +static bool allow_unsafe_mappings; +module_param(allow_unsafe_mappings, bool, 0444); + /* * Ordering of locks: * @@ -2811,6 +2818,9 @@ static kvm_pfn_t kvm_resolve_pfn(struct kvm_follow_pfn *kfp, struct page *page, * reference to such pages would cause KVM to prematurely free a page * it doesn't own (KVM gets and puts the one and only reference). * Don't allow those pages until the FIXME is resolved. + * + * Don't grab a reference for pins, callers that pin pages are required + * to check refcounted_page, i.e. must not blindly release the pfn. */ if (map) { pfn = map->pfn; @@ -2929,6 +2939,14 @@ static int hva_to_pfn_remapped(struct vm_area_struct *vma, bool write_fault = kfp->flags & FOLL_WRITE; int r; + /* + * Remapped memory cannot be pinned in any meaningful sense. Bail if + * the caller wants to pin the page, i.e. access the page outside of + * MMU notifier protection, and unsafe umappings are disallowed. + */ + if (kfp->pin && !allow_unsafe_mappings) + return -EINVAL; + r = follow_pfnmap_start(&args); if (r) { /* From patchwork Thu Oct 10 18:23:38 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830900 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id DBACFCFC5F8 for ; Thu, 10 Oct 2024 19:08:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Type:Cc:To: From:Subject:Message-ID:References:Mime-Version:In-Reply-To:Date: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=n3l8C52m2RjpLTXcY6dq+UIBo01A1fTC4GBMzDz1kHI=; b=awZur1qmaZ4CYJiZnRanf6DTg+ XQtbjif+Z9jwTIX6VJJGQwbvgEAs7Iz4fqXJhFGoUqd4JJeW0kgIet0HJ/UM5+1vrO/Ua/hR4yOMD HRlYohNAyHovYoGwK+uYK4cffvfSM9pcLuZVeFizuPEksHRfa8GzG1dHuPsybVzzjK9X0rZypPV2t 98r8eANxQyb37ZTWXqai/Ug2od7mwjSKe6t3XfYdUHfiB7h83bdbPsVqH4wjdFswFWclmvXafjuhY b3IlqW4u4oZxjWYml54py49dnmpEkGGEwi5UDRfyc+2Z/fd/UTvYllDOscRDrL2N7yQXG7lcgMPO3 ZkSPDCnA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syyWl-0000000E382-2dGT; Thu, 10 Oct 2024 19:08:47 +0000 Received: from mail-pf1-x449.google.com ([2607:f8b0:4864:20::449]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxrU-0000000DqaO-1B1d for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:26:09 +0000 Received: by mail-pf1-x449.google.com with SMTP id d2e1a72fcca58-71e04c42fecso1289999b3a.0 for ; Thu, 10 Oct 2024 11:26:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584767; x=1729189567; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=n3l8C52m2RjpLTXcY6dq+UIBo01A1fTC4GBMzDz1kHI=; b=hbUUm6huEsIpvBlGvvHRI3wy9VneLaobE+5HtYZY5ER96gyN7mdBt227GEUK5zzVMn 76vbGKJ41cFK+Ysmvcp7yzuv7ili+YPUjYOxE/ctNAbuu4Nnb7yjxtQS1drQiUhAjqUc Ip97sqxSr37TeirbDgYFFRiSaHoci9HOuvGFeZA6n046j+szLpL2tKQh7Ei2hE6WbqkS ntJU0pnF07qDfzHcM1/OBCL3m4W69iEVbt8ImxGUxYS+T3YNq0sWzfDSXc6BIWrgACxA cLY+/1MsX2oQAbg3Kc7OdW631minPkxe5BLmqpeshd1NcocBBwJNRd9/VTPro4cW+odU d97A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584767; x=1729189567; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=n3l8C52m2RjpLTXcY6dq+UIBo01A1fTC4GBMzDz1kHI=; b=RLqwotAOjbB2+fX/uo9vKbbsptgbvTjzSDKOyt1N+7HopUYXmddFpNy3yFDXTAN5W7 7X5WWUssbilBFCaAMhG6ifHTAD4ylqehIVst0/mBWkh2G3jFCeJY+sDv06WOceNSmZyd Uj0zTtjj3sRM1CtTAc7+Bb/bRgCkq4QP5fB77WN4j+adtRwlfvA8/wxMYtj6NjsIKNUJ Kd2polTyax7mDhdQjbMZpwOLijx8v+BvHlOTBrhZBvs2EVKLAvjEMZ8dof1jfM1IZn24 coa2YQWshtAJ0wNH3o+dYzWFmIOCHL73P6Jofq7DJlmL4NQR7fi89NzHIa09hZRMFuoC SkiA== X-Forwarded-Encrypted: i=1; AJvYcCW1Qt6pMPVLbUElktdYzHx4aemmS4VFma7T3QA0hZ543n7kReEY/x9zZl8IV2iH8xDDtrXbE/9/jXNP7GLEwEOX@lists.infradead.org X-Gm-Message-State: AOJu0YwhuDYY0E3iaY9c4t4Ze+xmDCH8bGf10LW4qfJ2Tv/LmFyZcZYy DnezDX7clL9z9eF/28zzFjT0EThBY1X8x0KBbafTG1YvGQdcX+j18gkKEULA70q9P2O3V1fOyiK DPg== X-Google-Smtp-Source: AGHT+IH5y4h19aQ8D3UFZNHEjFs0l+IkJ/F+GwDOdY5CLNZ6WKY+eP+zfpO4DiXt2PfVXCIgSRJOtCb+AeE= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:6a00:6f44:b0:71e:268b:845e with SMTP id d2e1a72fcca58-71e26e53c16mr11733b3a.1.1728584766802; Thu, 10 Oct 2024 11:26:06 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:38 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-37-seanjc@google.com> Subject: [PATCH v13 36/85] KVM: x86: Don't fault-in APIC access page during initial allocation From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112608_380944_9B92B9D2 X-CRM114-Status: GOOD ( 11.72 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Drop the gfn_to_page() lookup when installing KVM's internal memslot for the APIC access page, as KVM doesn't need to immediately fault-in the page now that the page isn't pinned. In the extremely unlikely event the kernel can't allocate a 4KiB page, KVM can just as easily return -EFAULT on the future page fault. Suggested-by: Paolo Bonzini Signed-off-by: Sean Christopherson --- arch/x86/kvm/lapic.c | 12 ------------ 1 file changed, 12 deletions(-) diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c index 20526e4d6c62..65412640cfc7 100644 --- a/arch/x86/kvm/lapic.c +++ b/arch/x86/kvm/lapic.c @@ -2647,7 +2647,6 @@ void kvm_apic_update_apicv(struct kvm_vcpu *vcpu) int kvm_alloc_apic_access_page(struct kvm *kvm) { - struct page *page; void __user *hva; int ret = 0; @@ -2663,17 +2662,6 @@ int kvm_alloc_apic_access_page(struct kvm *kvm) goto out; } - page = gfn_to_page(kvm, APIC_DEFAULT_PHYS_BASE >> PAGE_SHIFT); - if (!page) { - ret = -EFAULT; - goto out; - } - - /* - * Do not pin the page in memory, so that memory hot-unplug - * is able to migrate it. - */ - put_page(page); kvm->arch.apic_access_memslot_enabled = true; out: mutex_unlock(&kvm->slots_lock); From patchwork Thu Oct 10 18:23:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830904 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id AD5BDCFC5F7 for ; Thu, 10 Oct 2024 19:10:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:Content-Type:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=551DSOChXG5A+yDanUyc1gC1jxg9ryEf27npUXDW1CE=; b=V+D4yBpDkEzJT8 yG1xZIlfrIOinyFRCBagQTsqXPp0l5psUJmhJhL4EhMc5v2aVIGzSLB0faXkWp5uvOyIEBvYK+7WF g5T6zph2c/MSoObMkfuLkNKuGUBRLYqy73/s0rsY39jRq+ZUTVbhTRyENMnQlonMev/zSQrt9Wdi6 1vx26uTebtyilm4aA58kUDclWnrYHChEsFPtvq4uFQ6BLmJqdE7EGLSig8kiWJ3Cl2sMxtP1ySoUR L0oEM3AhrrHp5I3dNVZPy1BAwrIFHADql+SWrD7haIi8irkXGllmzdQkNHdxrPIGNOBZvQ74ICFHg E2uGLbi/JJcUIPAG0zNQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syyYE-0000000E3bs-41P3; Thu, 10 Oct 2024 19:10:18 +0000 Received: from mail-pl1-x649.google.com ([2607:f8b0:4864:20::649]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxrW-0000000Dqbx-0Xvf for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:26:11 +0000 Received: by mail-pl1-x649.google.com with SMTP id d9443c01a7336-20c8a799e8dso9119595ad.2 for ; Thu, 10 Oct 2024 11:26:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584769; x=1729189569; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=551DSOChXG5A+yDanUyc1gC1jxg9ryEf27npUXDW1CE=; b=hHRUq768hWMB0snk3BTDf56BfbwlI/wpDGgFhaIx6MEk7ikWWWPixPGn5ajCAfoaG7 0pbQvb2docQGGHhQNGOEELyTo7Vn6Kg+2FK1Ado67d3dpb9nyxslxTWPdbIhCjSBDz55 ZZHh5rAO98OLYQq30LZnkOFZ8o9x0nvVNieVl9SGp8g1LHKYjOx0z8KSEtcXQHgXxI5a ssz873LT3liXqRaru9m8P5P+ghEymZ2dHwIcTCN5DQoQB/SedtOHs1Ul/MYoHZ+tTuqg rAtROSmEeH/8XJgjKuysN/PJsY+C2BYPd5iklkCaHOn7gwmpxw+U9/Pxtu4Mfcnf5wVt B9Hg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584769; x=1729189569; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=551DSOChXG5A+yDanUyc1gC1jxg9ryEf27npUXDW1CE=; b=aEiKtaRfuKrGRVxxduMHQXvb/V/g1Cqk+SnqYTNotkOpuMcnXpuMNHUmDClRWiXWHc RrQjfWFriq+fEQTOp0eWxMvFc473DFe2A3Q8OfFrRojopo3eDwGfJnInLLbPmsFoCYic VCWdNC5At3CHseqfxrOsvVL4skp3r+ZdHFZNUpORL04tmSWLdJxhOZNsB43IBqsftbFZ HhYoUJxqK+B4U95DJaThKip0t0mXZ8PlxtIHHFzidY7soeTOz56uA0dwAQTE8yAQF3C7 8w/Qx7uT+Pl03yPuxt6aQ1lFjhQEdyHvq/gRzff58036pvK7f6O72ZyNT0AiFZ9MqVzn aEng== X-Forwarded-Encrypted: i=1; AJvYcCXVBklvC7aOeST9ZXn6VqooY476eJCjBVKfj8TY0gdgiOHeNkaKO+XGYQ/A2MzXUexOTY8HS+IhdmeLT0Qo2OBq@lists.infradead.org X-Gm-Message-State: AOJu0YwbZpbJSFUlZXpfR7AeGCbtpkSQBsrjZiOLb/KFSwUkGe0VN7Wf V4nZImckzQu52hLuCY3DVT2omhm8eXyHFZ6L/jB0wF2KJs2QhfHJe1YbBQWxB41dEZMD2B/rcg1 q7Q== X-Google-Smtp-Source: AGHT+IGKm7yq47izgr1FmkV+xWCQVhrzT+Qt86SXT8qqzZ7digIfsw6vfVxyVo72E4861BiCHpsXRIV5BzI= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:902:e80f:b0:1fa:2ae7:cc6a with SMTP id d9443c01a7336-20c63722d7fmr648865ad.4.1728584768900; Thu, 10 Oct 2024 11:26:08 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:39 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-38-seanjc@google.com> Subject: [PATCH v13 37/85] KVM: x86/mmu: Add "mmu" prefix fault-in helpers to free up generic names From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112610_266740_26543C2D X-CRM114-Status: GOOD ( 13.63 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Prefix x86's faultin_pfn helpers with "mmu" so that the mmu-less names can be used by common KVM for similar APIs. No functional change intended. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/mmu.c | 19 ++++++++++--------- arch/x86/kvm/mmu/mmu_internal.h | 2 +- arch/x86/kvm/mmu/paging_tmpl.h | 2 +- 3 files changed, 12 insertions(+), 11 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 28f2b842d6ca..e451e1b9a55a 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4347,8 +4347,8 @@ static u8 kvm_max_private_mapping_level(struct kvm *kvm, kvm_pfn_t pfn, return max_level; } -static int kvm_faultin_pfn_private(struct kvm_vcpu *vcpu, - struct kvm_page_fault *fault) +static int kvm_mmu_faultin_pfn_private(struct kvm_vcpu *vcpu, + struct kvm_page_fault *fault) { int max_order, r; @@ -4371,10 +4371,11 @@ static int kvm_faultin_pfn_private(struct kvm_vcpu *vcpu, return RET_PF_CONTINUE; } -static int __kvm_faultin_pfn(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault) +static int __kvm_mmu_faultin_pfn(struct kvm_vcpu *vcpu, + struct kvm_page_fault *fault) { if (fault->is_private) - return kvm_faultin_pfn_private(vcpu, fault); + return kvm_mmu_faultin_pfn_private(vcpu, fault); fault->pfn = __gfn_to_pfn_memslot(fault->slot, fault->gfn, false, true, fault->write, &fault->map_writable); @@ -4409,8 +4410,8 @@ static int __kvm_faultin_pfn(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault return RET_PF_CONTINUE; } -static int kvm_faultin_pfn(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault, - unsigned int access) +static int kvm_mmu_faultin_pfn(struct kvm_vcpu *vcpu, + struct kvm_page_fault *fault, unsigned int access) { struct kvm_memory_slot *slot = fault->slot; int ret; @@ -4493,7 +4494,7 @@ static int kvm_faultin_pfn(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault, if (mmu_invalidate_retry_gfn_unsafe(vcpu->kvm, fault->mmu_seq, fault->gfn)) return RET_PF_RETRY; - ret = __kvm_faultin_pfn(vcpu, fault); + ret = __kvm_mmu_faultin_pfn(vcpu, fault); if (ret != RET_PF_CONTINUE) return ret; @@ -4570,7 +4571,7 @@ static int direct_page_fault(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault if (r) return r; - r = kvm_faultin_pfn(vcpu, fault, ACC_ALL); + r = kvm_mmu_faultin_pfn(vcpu, fault, ACC_ALL); if (r != RET_PF_CONTINUE) return r; @@ -4661,7 +4662,7 @@ static int kvm_tdp_mmu_page_fault(struct kvm_vcpu *vcpu, if (r) return r; - r = kvm_faultin_pfn(vcpu, fault, ACC_ALL); + r = kvm_mmu_faultin_pfn(vcpu, fault, ACC_ALL); if (r != RET_PF_CONTINUE) return r; diff --git a/arch/x86/kvm/mmu/mmu_internal.h b/arch/x86/kvm/mmu/mmu_internal.h index 633aedec3c2e..59e600f6ff9d 100644 --- a/arch/x86/kvm/mmu/mmu_internal.h +++ b/arch/x86/kvm/mmu/mmu_internal.h @@ -235,7 +235,7 @@ struct kvm_page_fault { /* The memslot containing gfn. May be NULL. */ struct kvm_memory_slot *slot; - /* Outputs of kvm_faultin_pfn. */ + /* Outputs of kvm_mmu_faultin_pfn(). */ unsigned long mmu_seq; kvm_pfn_t pfn; bool map_writable; diff --git a/arch/x86/kvm/mmu/paging_tmpl.h b/arch/x86/kvm/mmu/paging_tmpl.h index 143b7e9f26dc..9bd3d6f5db91 100644 --- a/arch/x86/kvm/mmu/paging_tmpl.h +++ b/arch/x86/kvm/mmu/paging_tmpl.h @@ -812,7 +812,7 @@ static int FNAME(page_fault)(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault if (r) return r; - r = kvm_faultin_pfn(vcpu, fault, walker.pte_access); + r = kvm_mmu_faultin_pfn(vcpu, fault, walker.pte_access); if (r != RET_PF_CONTINUE) return r; From patchwork Thu Oct 10 18:23:40 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830905 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5C481CFC5F9 for ; Thu, 10 Oct 2024 19:11:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:Content-Type:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=UQkfEcPm4BCGA2NCPxYbIcRTjXNaFQKnzNzKtIM0PSc=; b=BqNaQW3G2nzwOq zkRyXPzFuZO2QJUk/asdxOXKZCGhsX5l+gy9ffDoBPz9+7GXk6O2+r1ciAscgNYhJkyBmxp0GkAYI Rz0RN8RoWHipV9kWf9Q53faIEu1jEOVb4EJhRrgEvFXX+Oo627ojwGAAbv98xjeQcjCP7OM7GgZHq FjhtCfRfXsF4PPJ5PDMF8wWqiu+OUN0T7IWgPXrw5AZ8DvYwSa9yae4Qab7eI3valr5hUlntFnxr9 N8t4aGDXaiPIBjBWiphevV793RWqsJXcLzlmAThO7kfp8fSPFkLuL2yawvCdFWk6HwTh2ifa3bx+L iQLVl5SQaBrDnsNofM0w==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syyZd-0000000E3zV-27tt; Thu, 10 Oct 2024 19:11:45 +0000 Received: from mail-pg1-x549.google.com ([2607:f8b0:4864:20::549]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxrZ-0000000DqeR-0PWm for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:26:14 +0000 Received: by mail-pg1-x549.google.com with SMTP id 41be03b00d2f7-70ac9630e3aso1195398a12.1 for ; Thu, 10 Oct 2024 11:26:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584772; x=1729189572; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=UQkfEcPm4BCGA2NCPxYbIcRTjXNaFQKnzNzKtIM0PSc=; b=a9tvknbu3mVAwdSk/bWW5Mn6EIDWzmh5YcjYhMVkkIs1GvQLH6PIVzkh/eOSJc+rpn 3ueOHIXlNOai8fANmCvUpo+xVDPEYFJTvnkaiYtt1Lx+b1NC0b6SUoONHAXEP/bKZSui rgPHbPmU/oMAwui5lHJhk4Vj2+lb3sZyIjFzW/OIQ4ntLUfMJmYF7D1ILFmOtChZCw6+ CTNBGGncSEfZ4h5yf5X2BrgnM3j86XxhiCzCWwarWP1oaMehmhQAqRF8CTp4GI62hfIu HitGGHrCddX+1k1ox1ejB/nmPZCOPRAWnV8+5pJzeDfC5Z7i3iI7WxVGBWshaWSaoTMj mfaw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584772; x=1729189572; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=UQkfEcPm4BCGA2NCPxYbIcRTjXNaFQKnzNzKtIM0PSc=; b=P1WhO8k+SaAbizqXEbGVAx+hwPKuy0u/0zLJSTqRwf9V5bL1q+4tdxXHxmqmVsHAIu DEUAzI9CHLSkF1bv1eC3U/3k7FMp5SyoeoaxxH32EWT5K3RBjO5GwAXv5lT/5hblZ+cF q6iCykKWtWk+9xez/sw5y6qui74VT7wbmQ+haF/pRWkzQaCN5xxcmk4nyR2m9X40pRXw BhnA2whTIecxsD0r/UAkpEgD4ay0HOB3LvRmbHlAzhtaxs1HBlgP3lWqWP9ufCa5Rmkv nfUVhOOnSs239aNxQIfMFFsLzl+BVIoIY32WLYmK6X1kDqBr6EZ4VPJh5qvBWkL+Km3p n2JQ== X-Forwarded-Encrypted: i=1; AJvYcCVK5ojvucs5yFb40AgUxv37S/Ew+/mVTDVkcScxqjJL4ydg1kRgJGI9Tg4/pryil+8N/1lup8OA8XDh0SWALDPt@lists.infradead.org X-Gm-Message-State: AOJu0Yz8szHbrgoMoPIiFg11WW6paO4TZKzUN6MBEZ/9SxAzeI9Kl8A3 5e9fJlnPpdiHDDsFXvtmfuekfjRrZ4iHHfBN8tCCUdZZVPu3E6+RASdisspjNVZAwGDcJEUibew nFA== X-Google-Smtp-Source: AGHT+IHq9hr54ORGaKAudrJMY0Hy9bRYjmUvRYz8zAm0RAj3x+bD9YD3f2uWGLgDblXeQYOnnP141duUJXA= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a63:1d03:0:b0:7db:539:893c with SMTP id 41be03b00d2f7-7ea535a65c6mr17a12.9.1728584770873; Thu, 10 Oct 2024 11:26:10 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:40 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-39-seanjc@google.com> Subject: [PATCH v13 38/85] KVM: x86/mmu: Put direct prefetched pages via kvm_release_page_clean() From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112613_235218_C538513A X-CRM114-Status: GOOD ( 12.52 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Use kvm_release_page_clean() to put prefeteched pages instead of calling put_page() directly. This will allow de-duplicating the prefetch code between indirect and direct MMUs. Note, there's a small functional change as kvm_release_page_clean() marks the page/folio as accessed. While it's not strictly guaranteed that the guest will access the page, KVM won't intercept guest accesses, i.e. won't mark the page accessed if it _is_ accessed by the guest (unless A/D bits are disabled, but running without A/D bits is effectively limited to pre-HSW Intel CPUs). Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/mmu.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index e451e1b9a55a..62924f95a398 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -2965,7 +2965,7 @@ static int direct_pte_prefetch_many(struct kvm_vcpu *vcpu, for (i = 0; i < ret; i++, gfn++, start++) { mmu_set_spte(vcpu, slot, start, access, gfn, page_to_pfn(pages[i]), NULL); - put_page(pages[i]); + kvm_release_page_clean(pages[i]); } return 0; From patchwork Thu Oct 10 18:23:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830906 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 30280CFC5F9 for ; Thu, 10 Oct 2024 19:13:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:Content-Type:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=bLMllFEXz5NbCB79T5EDeXxXCOdT0E/lPN9LlBgR2FM=; b=d/bjsRn5vX4P6C 1UvOkTJhTGHRYaVHLsUAL1X1Y448+0ftlgpnaKXeJDpi2C9lkvBArXkf3Kd3qd7H/Yi7TlCdGCiTW KahyqMegSu0jjhMZy6puQsvZeHYAgnVV8g6ITH7hJIj5MzuP/TTUdYcufWgnIdAS7Z53NVEgTJFaH eMVtWBmLrpwmxJgMkXf6CkI53OUq4hW26CUJ967LJ2RytybWBSbTX5NBJfFPTmc7De3vE594uJloN DENljuuYNnwS3c38qAkDz5Hc4De3/ynI/yeLVXo7tOVQq9Nw1NoAScrmf+1CQ0rw0TKno9yb5Mbb+ gQiiQIlA6Sl0pQq9dQow==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syyb4-0000000E4RQ-2Vqw; Thu, 10 Oct 2024 19:13:14 +0000 Received: from mail-pl1-x649.google.com ([2607:f8b0:4864:20::649]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxra-0000000Dqfc-3Xul for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:26:16 +0000 Received: by mail-pl1-x649.google.com with SMTP id d9443c01a7336-20c61e4b890so14347665ad.1 for ; Thu, 10 Oct 2024 11:26:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584774; x=1729189574; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=bLMllFEXz5NbCB79T5EDeXxXCOdT0E/lPN9LlBgR2FM=; b=iqLFJk08lU4sXCqW4q3PDDrFFs5ZbTmdrSdsuuvf3BYW0ezc3qn0B+PT1xdp95wJKP v8vCmQJzR9uGRWWX3xM5EFUHYKi2EyUNEbk9tyfb71n60zeMppgYPZ/Vj4gdSD5N6RvJ FkgKdr3Z6z8/NYvrrP30gumqXo23lNuT93fQVxGcHiqCRnlW4E/mnK7E1UGmNlyGu+b8 Y5W97QXCMKKOXRoadOfj7pRQ5An6esrBRxEqAAv4FVktbDLfZXuA1uVEw7KCqLLpPIOx JbdwK5ygGHHjecSj53zztRzMuX8uHhsU/M+w90oMCOX27vE9+USr78CsT5jhtEz3h4c7 2qkg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584774; x=1729189574; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=bLMllFEXz5NbCB79T5EDeXxXCOdT0E/lPN9LlBgR2FM=; b=pOIJ2HSQASoW8Dc408VZLp5r8REvXvdyQyb7MT9ZevRtBAy69n3vi3KXWb8ehbsbLs qjRWI3CZXUzGMGO+/p96Tv/uGRoeNGj7fOZxibixyv4hP/tgpY6BnifQ/myCNV4sjFoo 4uXHn45BzeadOtKSJkn657Hx1Oi7OsdB00Aw5HebZZ1c2QyS0BA+kSkD6FxIaS616HHy fNQAnLlhLgTGa8b6j8v5+aNhGBSKQ5AB92pSdFZ6Cz5b6aXKLLuq0vuCGr+jPxjX9JGc vpWfU3iQM8Hz5dmoGlu+cHuxIQSuD+79jduPZnQBM2lNf0SCmFDLvO5cFdWTJCR9sZh5 jTUw== X-Forwarded-Encrypted: i=1; AJvYcCVkERDGP9v87mglppXZc6Wg9ljYVyvV93xZyN/BPYq4NTWdmJ89QRWS9kIQT5V6XG7tF8t6Bs5IKVphfR2ALYKA@lists.infradead.org X-Gm-Message-State: AOJu0YzE3QZbMqB+otAJAiJ3pbcyXPx/QRwp5orhTQmYKuIeJVk2ILDG YeQg0UA+2Rwypikf9BD6MUiykubtcuZlYVg8+t75nJ1sNBMiNIsAdPgwv7pDEwJnvk5FG1+566E OFg== X-Google-Smtp-Source: AGHT+IHTeV3dFyfdACUyzit3PmgjtoD4THwZ77SaAB8Cv7KtK6sEvfc4IjBdPT+nDHGdhMcMXcKC7R1qaBw= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:903:1c8:b0:20c:716c:5af with SMTP id d9443c01a7336-20c716c078dmr946605ad.3.1728584773496; Thu, 10 Oct 2024 11:26:13 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:41 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-40-seanjc@google.com> Subject: [PATCH v13 39/85] KVM: x86/mmu: Add common helper to handle prefetching SPTEs From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112614_917732_7126D7F5 X-CRM114-Status: GOOD ( 15.06 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Deduplicate the prefetching code for indirect and direct MMUs. The core logic is the same, the only difference is that indirect MMUs need to prefetch SPTEs one-at-a-time, as contiguous guest virtual addresses aren't guaranteed to yield contiguous guest physical addresses. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/mmu.c | 40 +++++++++++++++++++++------------- arch/x86/kvm/mmu/paging_tmpl.h | 13 +---------- 2 files changed, 26 insertions(+), 27 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 62924f95a398..65d3a602eb2c 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -2943,32 +2943,41 @@ static int mmu_set_spte(struct kvm_vcpu *vcpu, struct kvm_memory_slot *slot, return ret; } -static int direct_pte_prefetch_many(struct kvm_vcpu *vcpu, - struct kvm_mmu_page *sp, - u64 *start, u64 *end) +static bool kvm_mmu_prefetch_sptes(struct kvm_vcpu *vcpu, gfn_t gfn, u64 *sptep, + int nr_pages, unsigned int access) { struct page *pages[PTE_PREFETCH_NUM]; struct kvm_memory_slot *slot; - unsigned int access = sp->role.access; - int i, ret; - gfn_t gfn; + int i; + + if (WARN_ON_ONCE(nr_pages > PTE_PREFETCH_NUM)) + return false; - gfn = kvm_mmu_page_get_gfn(sp, spte_index(start)); slot = gfn_to_memslot_dirty_bitmap(vcpu, gfn, access & ACC_WRITE_MASK); if (!slot) - return -1; + return false; - ret = kvm_prefetch_pages(slot, gfn, pages, end - start); - if (ret <= 0) - return -1; + nr_pages = kvm_prefetch_pages(slot, gfn, pages, nr_pages); + if (nr_pages <= 0) + return false; - for (i = 0; i < ret; i++, gfn++, start++) { - mmu_set_spte(vcpu, slot, start, access, gfn, + for (i = 0; i < nr_pages; i++, gfn++, sptep++) { + mmu_set_spte(vcpu, slot, sptep, access, gfn, page_to_pfn(pages[i]), NULL); kvm_release_page_clean(pages[i]); } - return 0; + return true; +} + +static bool direct_pte_prefetch_many(struct kvm_vcpu *vcpu, + struct kvm_mmu_page *sp, + u64 *start, u64 *end) +{ + gfn_t gfn = kvm_mmu_page_get_gfn(sp, spte_index(start)); + unsigned int access = sp->role.access; + + return kvm_mmu_prefetch_sptes(vcpu, gfn, start, end - start, access); } static void __direct_pte_prefetch(struct kvm_vcpu *vcpu, @@ -2986,8 +2995,9 @@ static void __direct_pte_prefetch(struct kvm_vcpu *vcpu, if (is_shadow_present_pte(*spte) || spte == sptep) { if (!start) continue; - if (direct_pte_prefetch_many(vcpu, sp, start, spte) < 0) + if (!direct_pte_prefetch_many(vcpu, sp, start, spte)) return; + start = NULL; } else if (!start) start = spte; diff --git a/arch/x86/kvm/mmu/paging_tmpl.h b/arch/x86/kvm/mmu/paging_tmpl.h index 9bd3d6f5db91..a476a5428017 100644 --- a/arch/x86/kvm/mmu/paging_tmpl.h +++ b/arch/x86/kvm/mmu/paging_tmpl.h @@ -533,9 +533,7 @@ static bool FNAME(prefetch_gpte)(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp, u64 *spte, pt_element_t gpte) { - struct kvm_memory_slot *slot; unsigned pte_access; - struct page *page; gfn_t gfn; if (FNAME(prefetch_invalid_gpte)(vcpu, sp, spte, gpte)) @@ -545,16 +543,7 @@ FNAME(prefetch_gpte)(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp, pte_access = sp->role.access & FNAME(gpte_access)(gpte); FNAME(protect_clean_gpte)(vcpu->arch.mmu, &pte_access, gpte); - slot = gfn_to_memslot_dirty_bitmap(vcpu, gfn, pte_access & ACC_WRITE_MASK); - if (!slot) - return false; - - if (kvm_prefetch_pages(slot, gfn, &page, 1) != 1) - return false; - - mmu_set_spte(vcpu, slot, spte, pte_access, gfn, page_to_pfn(page), NULL); - kvm_release_page_clean(page); - return true; + return kvm_mmu_prefetch_sptes(vcpu, gfn, spte, 1, pte_access); } static bool FNAME(gpte_changed)(struct kvm_vcpu *vcpu, From patchwork Thu Oct 10 18:23:42 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830907 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 443BFCFC5FC for ; Thu, 10 Oct 2024 19:14:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:Content-Type:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=GAJaAoo95YXsHy4joQlb4gblEl3cKLpcYsG4EH0XckI=; b=zDx0yWQsacfqIn eVLgJsf2Axrpy9/9fSUMZHW/0ECQf5gHz3ChRcuANMvIlddWLn81PnpSCNBnJXXsVkL53zfH9U6Ud Kez8P2sVbNnhGJ5lGqNnWGU8o1DwkL2i81uvrv3BrtpkWvt4StXB7JNo5yKH6JYFzrG13jsM+kTet 4oXim/DXq4XVyVfNFSV2jtC42M+53vToSudjkCc5ogxtCOWlImh9HJ/ZG2cKvbQUVIqtOJFXgb9DO 9pxQyU63xIfOOgcuHQnnleNNb2vzB8nNkSB9LpdftB/uYCMylckeftuWqsJMyb67dUJHcEIBP4VuC 51glTv4FLq0m8wbruQ6g==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syycW-0000000E4sf-14q0; Thu, 10 Oct 2024 19:14:44 +0000 Received: from mail-pf1-x449.google.com ([2607:f8b0:4864:20::449]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxrd-0000000DqhT-22vo for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:26:20 +0000 Received: by mail-pf1-x449.google.com with SMTP id d2e1a72fcca58-71e123d3d22so1428900b3a.3 for ; Thu, 10 Oct 2024 11:26:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584776; x=1729189576; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=GAJaAoo95YXsHy4joQlb4gblEl3cKLpcYsG4EH0XckI=; b=agto7fsVBUrNPJE79ZqKdKpL56Mwp4ggAufpi3aobdfUMqqDLiZePIEDWdZdbEiu8K t8r+5OhPjz9fgq16P3VlDF8eKhXvyvPklbZFKypYRoc2ct12HZqqWkaTgun7QUkTKvO0 dNyxvyCqVBkkl2G1Bynvde5QJxv86Iskrfv+necqUnTYkwOORvwd+LUfpSQyTp0tj4Sn 95J2lIFDA5QKqrewiHle9ipChsR0tw7IP8b/yIlAW8KtmlUhkkMxFmnnZ46aKVXuB3E5 DiwGOd4HYAa8c2P85NmCinCOH65of45ssCv3+8jc/ZLu/J3Rq2BD6TCXI3hQ2zqYzUeN Fidw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584776; x=1729189576; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=GAJaAoo95YXsHy4joQlb4gblEl3cKLpcYsG4EH0XckI=; b=IZ8It0sogXVM/7D/VePorX48Jmar48WhzaBhuAabQhTM2CAIF+lck8hzmHOa5FMRLU 051vxhsCvUn/4i7j1SSN3OLkToEfRLmdyGPR3jfhvQO82BIgLJpVxmzRZFpHNEgNAHY1 cf++mnGWu2PE3+qQBFAObhloDv48U9oXoKyMxh9hdoUgg37Y/O2jDrapT6NMFBIEgJYz Ld3qh4aLRKsvGDBlj9UtkCx1pQAylsYofsc0SVpWORlGigkWZKtTGyBzRXdgeCEZ5uit QLCu6YOd1zIK1ffj7Tsj+rfo042R7bh4xj0aoNXP7D3PEfVn8TyrktNirWVrT4r6rag2 dC6g== X-Forwarded-Encrypted: i=1; AJvYcCVmhxthI3hAWG4X69mB3v+UoxCdyM4+K7ULpVreL8bQdp4QcVuq8FhQHMC3EgTEAzVCiLJnUc5hVG6+AjGxp9xZ@lists.infradead.org X-Gm-Message-State: AOJu0YzI8oJ5KVMQAFNfm6a/qO860ru4biJo/+AC9Ab5+xYKNOR8gT7y oBCBhibzO7V60P8zyCeJv3N0VwmceKYmyGPlEpTbdsG4z/4DWgjWEG7lSR4Zq6o14f6/k87ul7d 5Lw== X-Google-Smtp-Source: AGHT+IEXxr/4vsCtgUsUHHkctm5xGRqb6tsNipBtkL7vgMjj75AzVZauqGsQqnffIf336Hhi8e9e6vZrhMQ= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:aa7:9184:0:b0:71d:f7c9:8cb3 with SMTP id d2e1a72fcca58-71e1dbf1ab5mr7792b3a.5.1728584775458; Thu, 10 Oct 2024 11:26:15 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:42 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-41-seanjc@google.com> Subject: [PATCH v13 40/85] KVM: x86/mmu: Add helper to "finish" handling a guest page fault From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112617_647599_E4B4CFB1 X-CRM114-Status: GOOD ( 11.79 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Add a helper to finish/complete the handling of a guest page, e.g. to mark the pages accessed and put any held references. In the near future, this will allow improving the logic without having to copy+paste changes into all page fault paths. And in the less near future, will allow sharing the "finish" API across all architectures. No functional change intended. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/mmu.c | 12 +++++++++--- arch/x86/kvm/mmu/paging_tmpl.h | 2 +- 2 files changed, 10 insertions(+), 4 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 65d3a602eb2c..31a6ae41a6f4 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4357,6 +4357,12 @@ static u8 kvm_max_private_mapping_level(struct kvm *kvm, kvm_pfn_t pfn, return max_level; } +static void kvm_mmu_finish_page_fault(struct kvm_vcpu *vcpu, + struct kvm_page_fault *fault, int r) +{ + kvm_release_pfn_clean(fault->pfn); +} + static int kvm_mmu_faultin_pfn_private(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault) { @@ -4522,7 +4528,7 @@ static int kvm_mmu_faultin_pfn(struct kvm_vcpu *vcpu, * mmu_lock is acquired. */ if (mmu_invalidate_retry_gfn_unsafe(vcpu->kvm, fault->mmu_seq, fault->gfn)) { - kvm_release_pfn_clean(fault->pfn); + kvm_mmu_finish_page_fault(vcpu, fault, RET_PF_RETRY); return RET_PF_RETRY; } @@ -4598,8 +4604,8 @@ static int direct_page_fault(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault r = direct_map(vcpu, fault); out_unlock: + kvm_mmu_finish_page_fault(vcpu, fault, r); write_unlock(&vcpu->kvm->mmu_lock); - kvm_release_pfn_clean(fault->pfn); return r; } @@ -4685,8 +4691,8 @@ static int kvm_tdp_mmu_page_fault(struct kvm_vcpu *vcpu, r = kvm_tdp_mmu_map(vcpu, fault); out_unlock: + kvm_mmu_finish_page_fault(vcpu, fault, r); read_unlock(&vcpu->kvm->mmu_lock); - kvm_release_pfn_clean(fault->pfn); return r; } #endif diff --git a/arch/x86/kvm/mmu/paging_tmpl.h b/arch/x86/kvm/mmu/paging_tmpl.h index a476a5428017..35d0c3f1a789 100644 --- a/arch/x86/kvm/mmu/paging_tmpl.h +++ b/arch/x86/kvm/mmu/paging_tmpl.h @@ -836,8 +836,8 @@ static int FNAME(page_fault)(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault r = FNAME(fetch)(vcpu, fault, &walker); out_unlock: + kvm_mmu_finish_page_fault(vcpu, fault, r); write_unlock(&vcpu->kvm->mmu_lock); - kvm_release_pfn_clean(fault->pfn); return r; } From patchwork Thu Oct 10 18:23:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830911 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 99A85CFC5F9 for ; Thu, 10 Oct 2024 19:16:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:Content-Type:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=sgK+YcjRpEQ+YhbfYv/o4akq/yPA2qNOXCMP41gc9EY=; b=HKoKG6UqXsrAlm Uk7LxoColpPvZ1ivYTZN9OKLk1d8WKKhAs1cxI93wJqOZERO9keEcK1HKPG3xWjBdf4Rin29F9biW RFGy03NHT5zm3wdW6MRCe5CPhZaKnpqA2pW7wKpTJ5H/Ub81eF9piWc5DlzhfdR+eakbx3iqOJrox OAkunvWrfhUPcRjLXzdwF3LB5uvGVqrcIFnFrYIqQ1gYlLD+2nscmbIkhmcioMLjKEFuwaL1+ytqH FSH/rP7EdM4vYDHmEodqpfq4A+l797oBxpJWvYDuWYNGFyZyYTTS1M9RYdFEzaLmPHoBeedpNoVYJ UMQ9kbPxv6zgcMBzICXQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syydq-0000000E53C-21FP; Thu, 10 Oct 2024 19:16:06 +0000 Received: from mail-pl1-x64a.google.com ([2607:f8b0:4864:20::64a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxrf-0000000DqjQ-01tl for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:26:21 +0000 Received: by mail-pl1-x64a.google.com with SMTP id d9443c01a7336-206b912491eso23155645ad.0 for ; Thu, 10 Oct 2024 11:26:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584778; x=1729189578; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=sgK+YcjRpEQ+YhbfYv/o4akq/yPA2qNOXCMP41gc9EY=; b=iFkBAzBXYFBUziXH1DPHFwz0QuJbVZY9yzLrch8ot8iJUnZFwu0Ch7La+ANEyti4nR AiDSevGke+MAd+fF5+8281cC7OvrCxYOzA8Jn/wVBWgRyeNM1C+8Y4ZOM9b7hYkXpmDz rSxcKYrmZHnrNvGHVOgV5h+nWW41K7iFAYTqArL5ncvyqN2Nrg8M2ksNxs489+ODUdI5 RkxO0PgCZ3IqNb8GULWvgAuWQDd7DbJqF6gmz7/xpgXrONOBNZ8xnCYXKQDivmKtwi++ SkbfWdozQu9XNTOmoT67SyrcEA3ebbCiwMyOSypgPQ2bTRLYaltBc5xINem/udZ5gqOM R5OQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584778; x=1729189578; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=sgK+YcjRpEQ+YhbfYv/o4akq/yPA2qNOXCMP41gc9EY=; b=R+No9rIXKa4EZQUlfF0oQadd9w1kZX7w00DQEjxIcxLA/LwDscC3JgiSjYUvrbKLiK Cx4By4IixZQFbBdV5oYPscTCDGeP8xLFQOiUXNhEtc+sX6Ay5ygAQ3shHrFhThH5YyiO 1mlQ4Zx2GbKlyIAb3cMpQOOytqv+8k2EaQX201n6xHUC1py3C4bwqiYiHk232xjuJXQK VD6KbjdZLHaDi/pT5Znj8ilIJKy8dKYeDWrMKylq9FUywBkrHKkHj5E3zy5aLxP5pCV9 bR8Vft+fUkKSEdmNIwogOvPuJ1dTb2w0l52nfaeNEioIrPzTUNjL5uN4DTGZzcx9qHLr FSpw== X-Forwarded-Encrypted: i=1; AJvYcCVgI/UJnFW9IB7NiAWQnqz3sJ8pyRlyWpZrrPOmTPKLYcyeHm0KnY2WdLYjyfGNP6O2GXVX7T6dEe1afjZIJ2PL@lists.infradead.org X-Gm-Message-State: AOJu0YxcauaxBv4eabRRM+95WP71+Bgl0bAT4NvsbrO2n6iC8eoyM0M1 Y6DvDCM8KqXpo+1hI2cTptxrQq3a7Cg/RpVRz2t7YzBvaRNRv/EordLlJl4XDRfXOZsc6Bjn+6H h0w== X-Google-Smtp-Source: AGHT+IHgES02wxBmUBuw4MXImaMxOgBvLS6HzbS699NHinso/VI3SIlimd+hdhb5F/AgwxmTSyUtUzYKtBo= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:902:ec85:b0:20b:7bfa:ac0f with SMTP id d9443c01a7336-20ca037f212mr485ad.1.1728584777475; Thu, 10 Oct 2024 11:26:17 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:43 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-42-seanjc@google.com> Subject: [PATCH v13 41/85] KVM: x86/mmu: Mark pages/folios dirty at the origin of make_spte() From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112619_160938_E1AA7610 X-CRM114-Status: GOOD ( 18.79 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Move the marking of folios dirty from make_spte() out to its callers, which have access to the _struct page_, not just the underlying pfn. Once all architectures follow suit, this will allow removing KVM's ugly hack where KVM elevates the refcount of VM_MIXEDMAP pfns that happen to be struct page memory. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/mmu.c | 30 ++++++++++++++++++++++++++++-- arch/x86/kvm/mmu/paging_tmpl.h | 5 +++++ arch/x86/kvm/mmu/spte.c | 11 ----------- 3 files changed, 33 insertions(+), 13 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 31a6ae41a6f4..f730870887dd 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -2964,7 +2964,17 @@ static bool kvm_mmu_prefetch_sptes(struct kvm_vcpu *vcpu, gfn_t gfn, u64 *sptep, for (i = 0; i < nr_pages; i++, gfn++, sptep++) { mmu_set_spte(vcpu, slot, sptep, access, gfn, page_to_pfn(pages[i]), NULL); - kvm_release_page_clean(pages[i]); + + /* + * KVM always prefetches writable pages from the primary MMU, + * and KVM can make its SPTE writable in the fast page handler, + * without notifying the primary MMU. Mark pages/folios dirty + * now to ensure file data is written back if it ends up being + * written by the guest. Because KVM's prefetching GUPs + * writable PTEs, the probability of unnecessary writeback is + * extremely low. + */ + kvm_release_page_dirty(pages[i]); } return true; @@ -4360,7 +4370,23 @@ static u8 kvm_max_private_mapping_level(struct kvm *kvm, kvm_pfn_t pfn, static void kvm_mmu_finish_page_fault(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault, int r) { - kvm_release_pfn_clean(fault->pfn); + lockdep_assert_once(lockdep_is_held(&vcpu->kvm->mmu_lock) || + r == RET_PF_RETRY); + + /* + * If the page that KVM got from the *primary MMU* is writable, and KVM + * installed or reused a SPTE, mark the page/folio dirty. Note, this + * may mark a folio dirty even if KVM created a read-only SPTE, e.g. if + * the GFN is write-protected. Folios can't be safely marked dirty + * outside of mmu_lock as doing so could race with writeback on the + * folio. As a result, KVM can't mark folios dirty in the fast page + * fault handler, and so KVM must (somewhat) speculatively mark the + * folio dirty if KVM could locklessly make the SPTE writable. + */ + if (!fault->map_writable || r == RET_PF_RETRY) + kvm_release_pfn_clean(fault->pfn); + else + kvm_release_pfn_dirty(fault->pfn); } static int kvm_mmu_faultin_pfn_private(struct kvm_vcpu *vcpu, diff --git a/arch/x86/kvm/mmu/paging_tmpl.h b/arch/x86/kvm/mmu/paging_tmpl.h index 35d0c3f1a789..f4711674c47b 100644 --- a/arch/x86/kvm/mmu/paging_tmpl.h +++ b/arch/x86/kvm/mmu/paging_tmpl.h @@ -954,6 +954,11 @@ static int FNAME(sync_spte)(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp, int spte_to_pfn(spte), spte, true, true, host_writable, &spte); + /* + * There is no need to mark the pfn dirty, as the new protections must + * be a subset of the old protections, i.e. synchronizing a SPTE cannot + * change the SPTE from read-only to writable. + */ return mmu_spte_update(sptep, spte); } diff --git a/arch/x86/kvm/mmu/spte.c b/arch/x86/kvm/mmu/spte.c index 8e8d6ee79c8b..f1a50a78badb 100644 --- a/arch/x86/kvm/mmu/spte.c +++ b/arch/x86/kvm/mmu/spte.c @@ -277,17 +277,6 @@ bool make_spte(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp, mark_page_dirty_in_slot(vcpu->kvm, slot, gfn); } - /* - * If the page that KVM got from the primary MMU is writable, i.e. if - * it's host-writable, mark the page/folio dirty. As alluded to above, - * folios can't be safely marked dirty in the fast page fault handler, - * and so KVM must (somewhat) speculatively mark the folio dirty even - * though it isn't guaranteed to be written as KVM won't mark the folio - * dirty if/when the SPTE is made writable. - */ - if (host_writable) - kvm_set_pfn_dirty(pfn); - *new_spte = spte; return wrprot; } From patchwork Thu Oct 10 18:23:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831076 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id BF476D2444D for ; Thu, 10 Oct 2024 20:31:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:Content-Type:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=8DwvYXI8CmE5CO0Ycvk+EgdgF52vTMi74FFh8TONwA8=; b=hF62k963t/oa0a nLBSJMcCZ33zUsaQA1wXVk1I0SgtyWy0S9Xca6fQZ1uZxWW5Z416Bx+Uo5U/O2YLLlxzdY0kcCWN1 B4eh/wFGmP0goFvAmmjExHmS23jslMIyXW0ZFzYJOvGrIMIvNepW761uwJY3MOFUkKV41tbPtdKZ4 5ei12GFAoPSC8FGeXV/lCgskcNy3Ci4HiM+qFxq6QNZHIlpq7Sbd9KNl77k7ccdSdgUIfVZQ26Tv4 WOr8Mdjd5kkcAFImS7v4IXge2p70ZeBPSv4uNJ7kAp5Wv716+6okUj0SvTXJyx+WMbl+37u8Iur5O 4U3UESZ0Ijb4axVRIqHA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syzog-0000000EGlD-26Xz; Thu, 10 Oct 2024 20:31:22 +0000 Received: from mail-pf1-x449.google.com ([2607:f8b0:4864:20::449]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxrg-0000000Dqkp-3tJj for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:26:22 +0000 Received: by mail-pf1-x449.google.com with SMTP id d2e1a72fcca58-71e1e989aa2so1655352b3a.3 for ; Thu, 10 Oct 2024 11:26:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584780; x=1729189580; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=8DwvYXI8CmE5CO0Ycvk+EgdgF52vTMi74FFh8TONwA8=; b=4QzPzbH+y3h+qIy3AOczqs89mCTm26A8Xmw0N0dOgewGXg9A6oNEwweMsZb+ojs0A+ /oaXqPla6lblPPbLsCeAkMWE0ipPJUeX7/BMJCeUmSAFOBCsBqR9tEnyG6dtvMhnmUoa icT7EjStSrVntpraIZTjudeYgVzDZgDSb8Ozn3GE9fd2lwS76/qNcWlYT5VV3157N2U3 boorlU+iazI5EA8U+PS7+8dAzQPprqHAb+bDJ23cP+mz4Fe98+JO8wzksaoGlcC6sRxS GoaOsto/Zr0ovDqlpn2iqeKujfmzR03yyd8aPIXxNhya6wLqUx9xXzeU/hX0itI/QzVu Is8Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584780; x=1729189580; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=8DwvYXI8CmE5CO0Ycvk+EgdgF52vTMi74FFh8TONwA8=; b=ktastDrt9cP83Y2xtLJ0FNzRJFF6sH/HG33+swVj9roL1yWAYIX6xmMo2e7n/d/S+3 H+h70eNmyO58ui/j89MIS02NsWPgnpxKC1XENm2oLKCBxo1kYxIh7bE80pyEt09s9sQT HkWhM1qXBMb3NOL3hFbgquBLfAovEGi/3i57S8I6KcQm9uka5foyzXWgLlX2TStbIaGC lyfWsYmvFa6zJAX4Tg5gLmqVDIwUMIOkmas+lJ1wM7LVx4cbNdPLqe3sCVRm6zU/uHNL +pXfc9gysZgZjFOeX4KKGErWy7RpVc72lWTknAbPUuDtBVsNjjpAO7ohxiXqy/KgG6+3 gctw== X-Forwarded-Encrypted: i=1; AJvYcCXmdAhVs65mENACq72jzJFnVT1I/9DdV3izIu47u6Kmop2+jwrNlEb8xAsvG6rR7QG0eg1AGBnj/fDSsWck5s21@lists.infradead.org X-Gm-Message-State: AOJu0YyrIxz4QInbHARLVXHCWK6o2vbuDznCxXkeBYqRo+DQXkx8seUy cnn3lwyxt5HSaUp2dZe6MkVGTNnI3xJEZVPw8PpvEfuLRbImjDkUU+ZmIbNZ0GpyKhUIiCzOXIG LXQ== X-Google-Smtp-Source: AGHT+IG3pPvsekcqYnotExeZVZUXEYlJLq+6e3a1eutYJX4gtVKcKQbhJKMNrxmpaRIGMBxtyW/kZ/ByL14= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:6a00:91a3:b0:71d:fb06:e79b with SMTP id d2e1a72fcca58-71e37c53abamr28b3a.0.1728584779496; Thu, 10 Oct 2024 11:26:19 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:44 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-43-seanjc@google.com> Subject: [PATCH v13 42/85] KVM: Move declarations of memslot accessors up in kvm_host.h From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112621_084216_ACFA7743 X-CRM114-Status: UNSURE ( 9.85 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Move the memslot lookup helpers further up in kvm_host.h so that they can be used by inlined "to pfn" wrappers. No functional change intended. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- include/linux/kvm_host.h | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 9263375d0362..346bfef14e5a 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -1168,6 +1168,10 @@ static inline bool kvm_memslot_iter_is_valid(struct kvm_memslot_iter *iter, gfn_ kvm_memslot_iter_is_valid(iter, end); \ kvm_memslot_iter_next(iter)) +struct kvm_memory_slot *gfn_to_memslot(struct kvm *kvm, gfn_t gfn); +struct kvm_memslots *kvm_vcpu_memslots(struct kvm_vcpu *vcpu); +struct kvm_memory_slot *kvm_vcpu_gfn_to_memslot(struct kvm_vcpu *vcpu, gfn_t gfn); + /* * KVM_SET_USER_MEMORY_REGION ioctl allows the following operations: * - create a new memory slot @@ -1303,15 +1307,13 @@ int kvm_gfn_to_hva_cache_init(struct kvm *kvm, struct gfn_to_hva_cache *ghc, }) int kvm_clear_guest(struct kvm *kvm, gpa_t gpa, unsigned long len); -struct kvm_memory_slot *gfn_to_memslot(struct kvm *kvm, gfn_t gfn); bool kvm_is_visible_gfn(struct kvm *kvm, gfn_t gfn); bool kvm_vcpu_is_visible_gfn(struct kvm_vcpu *vcpu, gfn_t gfn); unsigned long kvm_host_page_size(struct kvm_vcpu *vcpu, gfn_t gfn); void mark_page_dirty_in_slot(struct kvm *kvm, const struct kvm_memory_slot *memslot, gfn_t gfn); void mark_page_dirty(struct kvm *kvm, gfn_t gfn); -struct kvm_memslots *kvm_vcpu_memslots(struct kvm_vcpu *vcpu); -struct kvm_memory_slot *kvm_vcpu_gfn_to_memslot(struct kvm_vcpu *vcpu, gfn_t gfn); + kvm_pfn_t kvm_vcpu_gfn_to_pfn(struct kvm_vcpu *vcpu, gfn_t gfn); int __kvm_vcpu_map(struct kvm_vcpu *vcpu, gpa_t gpa, struct kvm_host_map *map, From patchwork Thu Oct 10 18:23:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831075 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 89AE1D24451 for ; Thu, 10 Oct 2024 20:31:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:Content-Type:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=CHy2j9VNwFKd5G/Ubc3O4QkXvNoIm5ox3M6XZ4oswcY=; b=SyIgY867Xtz4zo HEcs4XPw70ZQxuLx8psEYxYmF9PxbLjXFrWyFbHkSse63YdaMqVZ4Rovr5WWBB7YN1o1SMi+ulDxh Hh1C9lLVjPrnPcC16pK++Oi16WK6yfomtWztJIcCnDkZkcVWYju2vrjWjf4OLeVKqRSOooeBPaxbf jnLmbkkuzwNjfV1stXG00JvKYJ3CPKoJxdcz8QGoLwGa0rRrzAk4ZSZmFaUc3T2/a/s8DsIKVpGbP 8+Is7f242y0oix+2nFU/oqrCUs/qOGyodaIWs41buHk4r0ddaO3X+vCpB/nueP9LtkFc1KsBAPkAY /bjhHvh/yCKsC4cSenSg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syzoi-0000000EGm0-1Wcw; Thu, 10 Oct 2024 20:31:24 +0000 Received: from mail-yw1-x114a.google.com ([2607:f8b0:4864:20::114a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxri-0000000Dqm3-46s1 for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:26:25 +0000 Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-6e32f43c797so11872047b3.1 for ; Thu, 10 Oct 2024 11:26:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584782; x=1729189582; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=CHy2j9VNwFKd5G/Ubc3O4QkXvNoIm5ox3M6XZ4oswcY=; b=PxBQb6V+W9jxog8EtEIgeGjMXgX3DoQlH9csUz21WWACHY5TzBY6Zb0NYKAEDEvhCL EEORQUDWODB5Zlh+y65IhXmZaI61GUcrVHjyExWDVVnYWImh+IDn5v6Rn0iYUtOApd27 lRgtb7DyZK3i73ZvBLazXX+S80VsUBJVrvIewB0whu96yv3ay3gWL2sKc8xdagcNXSsC B5F89XVKv1Hf4taRgzl222K8qjxQHiSGWVj8Yi577akb5HVrn6V1Cuc+beMKFlqGzfAA 9iXv89Gftqp9NkxCMeqn6A/sH+3xcYVnYTU5K9D7AKyhHG0S1pgbqIsLN4HiKAti+VlZ OOVQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584782; x=1729189582; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=CHy2j9VNwFKd5G/Ubc3O4QkXvNoIm5ox3M6XZ4oswcY=; b=wzOfWmmSc8U/MklHrgvMzoLWlN7e0pgYNXwpJZ+YgWiCdf0KdI7njBTwn17Xxo7ky5 pduOfwm+7PCQNT083Hs6zgBbgl2BgxpcwKQvXgzaA2qyoD1g9zHhiDtqBoWW77Hq0woL wiDvNuMCIe34XTVvBaUd26JqP9cVw3KFFNvAnctlTYjl7hzp9oIdji5vsQQmQe7YO+Wd 4rycqrIQ/VHt3Md34s5ITmK1poqH0vuZ6csxXDgitjV3BI9US7QonEMVCxJSoBRUGIfa h1QBYnYjSomE0nvEjPQ98lUktdFGAyZCMbNvqBRV8sQ/jO6WiwbGxSvNbRmPnlBjB8rg 20NQ== X-Forwarded-Encrypted: i=1; AJvYcCWKRA7hBnSM22l/SsPRgtSTSkJheVOdDsycVx9QN8MBmfhks6cd1ohDs0zcteLMGPQ6WRqqzd98FnbNAapDTLCd@lists.infradead.org X-Gm-Message-State: AOJu0YwLCyM08C5ogYoOmsTqoyTqI0Ci+N7ynWXaevQ5JoU5IWfsx3+2 5pBo3aVDewtmbCbS3ZQt/jJfestFmVHyLbERVyP9mrb4nezGvYGnqATTfkxPUtRAkRnbBDg7IJP Paw== X-Google-Smtp-Source: AGHT+IEBXjEpYcXfPYRm3Nf4G9ZN1ucR55aqjscHr2p9QPsSrcRZYI0185ucK64AXquh+1QVcruWQv+kNDM= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:690c:5605:b0:6be:523:af53 with SMTP id 00721157ae682-6e32f2f4e28mr919077b3.3.1728584781575; Thu, 10 Oct 2024 11:26:21 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:45 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-44-seanjc@google.com> Subject: [PATCH v13 43/85] KVM: Add kvm_faultin_pfn() to specifically service guest page faults From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112623_222619_BD3CFBFC X-CRM114-Status: GOOD ( 13.53 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Add a new dedicated API, kvm_faultin_pfn(), for servicing guest page faults, i.e. for getting pages/pfns that will be mapped into the guest via an mmu_notifier-protected KVM MMU. Keep struct kvm_follow_pfn buried in internal code, as having __kvm_faultin_pfn() take "out" params is actually cleaner for several architectures, e.g. it allows the caller to have its own "page fault" structure without having to marshal data to/from kvm_follow_pfn. Long term, common KVM would ideally provide a kvm_page_fault structure, a la x86's struct of the same name. But all architectures need to be converted to a common API before that can happen. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- include/linux/kvm_host.h | 12 ++++++++++++ virt/kvm/kvm_main.c | 22 ++++++++++++++++++++++ 2 files changed, 34 insertions(+) diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 346bfef14e5a..3b9afb40e935 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -1231,6 +1231,18 @@ static inline void kvm_release_page_unused(struct page *page) void kvm_release_page_clean(struct page *page); void kvm_release_page_dirty(struct page *page); +kvm_pfn_t __kvm_faultin_pfn(const struct kvm_memory_slot *slot, gfn_t gfn, + unsigned int foll, bool *writable, + struct page **refcounted_page); + +static inline kvm_pfn_t kvm_faultin_pfn(struct kvm_vcpu *vcpu, gfn_t gfn, + bool write, bool *writable, + struct page **refcounted_page) +{ + return __kvm_faultin_pfn(kvm_vcpu_gfn_to_memslot(vcpu, gfn), gfn, + write ? FOLL_WRITE : 0, writable, refcounted_page); +} + kvm_pfn_t gfn_to_pfn(struct kvm *kvm, gfn_t gfn); kvm_pfn_t gfn_to_pfn_prot(struct kvm *kvm, gfn_t gfn, bool write_fault, bool *writable); diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 6dcb4f0eed3e..696d5e429b3e 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -3098,6 +3098,28 @@ kvm_pfn_t kvm_vcpu_gfn_to_pfn(struct kvm_vcpu *vcpu, gfn_t gfn) } EXPORT_SYMBOL_GPL(kvm_vcpu_gfn_to_pfn); +kvm_pfn_t __kvm_faultin_pfn(const struct kvm_memory_slot *slot, gfn_t gfn, + unsigned int foll, bool *writable, + struct page **refcounted_page) +{ + struct kvm_follow_pfn kfp = { + .slot = slot, + .gfn = gfn, + .flags = foll, + .map_writable = writable, + .refcounted_page = refcounted_page, + }; + + if (WARN_ON_ONCE(!writable || !refcounted_page)) + return KVM_PFN_ERR_FAULT; + + *writable = false; + *refcounted_page = NULL; + + return kvm_follow_pfn(&kfp); +} +EXPORT_SYMBOL_GPL(__kvm_faultin_pfn); + int kvm_prefetch_pages(struct kvm_memory_slot *slot, gfn_t gfn, struct page **pages, int nr_pages) { From patchwork Thu Oct 10 18:23:46 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830912 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id BC6DCCFC5FD for ; Thu, 10 Oct 2024 19:20:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:Content-Type:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=O48RmE1hmLPs0qNEdPIh+k/rpCPu+e8A6GC4+tMXVMw=; b=X1hNorl79/1JH0 D/bxRi3O/jkTKnWZPbp/bmkaNf/zVmnrGjCgbmr2D5gRnUdxWfuiqiotp8bxSox2eSXBu6nLvHxQH LPpYZrO547CAupRAEpsmUx77mI6ntGDEMbl2WlTT3rUsPaubRnPQXSDdINBRhJwY0kAtyytXKKH4a LjQ+viKfUz+33fmN8uca7cwRWvODGwSIndO7vzJKNpfZ1DFNad7SDwMgDY16WxtGrA7g+Zwyk/O2s EDgUcutrS7jD/aWfMMtE7VE0gACiJvZR2+A2KsL8wkusVpXY+OqHPB5YakxbPgtNY4SAsjuZSjCUU MA9JSrPoadNkBA+TpASA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syyhk-0000000E5Jb-2SNz; Thu, 10 Oct 2024 19:20:08 +0000 Received: from mail-pg1-x54a.google.com ([2607:f8b0:4864:20::54a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxrk-0000000Dqnk-3lKZ for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:26:26 +0000 Received: by mail-pg1-x54a.google.com with SMTP id 41be03b00d2f7-7ea05b8ea21so1336012a12.2 for ; Thu, 10 Oct 2024 11:26:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584784; x=1729189584; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=O48RmE1hmLPs0qNEdPIh+k/rpCPu+e8A6GC4+tMXVMw=; b=tUX4W6o8HRVDNg2p6vKzVqKv4QtBGye1jr9W70enPs3WfdHdg/DAX4SGK3PXyvt/PU b7AYI/CL/BlJ+quKCLDQWnwM3l+DwmoJ38wzIZ0AUR/5qG3XbJ6uKyGqIGs7af000tzu Yw5mLr5FCn/tAEl9m+rbmGRIlmBjOVsDrIb/34VVROnkoIFIZoc77UW/hjwsWuFMrKWu /akxlANMKK8Zni3hn6A1as9i+vAkfXKoaefSt+xnuNgimIQ5h/U2840t3CpBs4rUac96 3v9uJ9pPCbO+EtZ7MI416Oy4vh2RW3JLQnyfGXSyS7FkCqEOQLvRezcMeq0XiOM0Uyrt IWbg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584784; x=1729189584; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=O48RmE1hmLPs0qNEdPIh+k/rpCPu+e8A6GC4+tMXVMw=; b=NgMrRhGvR1FqSFwVMjl8Uh6rCrN4QYtf3CAAo/9igT5+34/HRsb32BH76wYDbLXu7c RXVFpqRPUm02rxehGs7/9liR7HYhxUwyRUuaHajQFzB9MwYC7sumBgd5ltAW//MQj2SU Ebrl6zyB9hDfTnenTUlDtwNH0wgR39RCXOGTy4g64wRg0Zwd2f057RtBS+h31C+FplSw /fvmc1TLtt0D41aehW8wru5gV4W6VYcwdVDODDpwPlFTwaiKkkoo55Mkrh4ZRnIui+vZ X0JMJns7Pqxd89vI2adCYNcCjmNN+ZRkWqjyL6/vUMppZD4pnLfwKOQ1kDGhdy4cTWUR MuCw== X-Forwarded-Encrypted: i=1; AJvYcCXEahIjixipxnsgH9XNxwk1eSgf6h8fKvQB/nr4d4hiqpuC7cBET78tGcC1iIL8GI2vtuftVyNIbaH4/NqRe3oW@lists.infradead.org X-Gm-Message-State: AOJu0Yx0qfQB5b93Gx6g4gP9hoBVRdOwjIUi7EaeTIu9dK6e1KbBUIFO ekzNtHHKIedsqefk1QBn7td6EKKbm7mBIEjOOC0lK54kq7wSd39EqjawFB8zFkhtCxJHe+N3GOt jrA== X-Google-Smtp-Source: AGHT+IGY3J0AFIsOlJbVDrqn6majLeEUOkcp3LAakwsbgSwDXT3SnAGgA9L20idMLdLGQvBeoRsHZDJxo50= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a63:2445:0:b0:75d:16f9:c075 with SMTP id 41be03b00d2f7-7ea5359cd2amr19a12.9.1728584783356; Thu, 10 Oct 2024 11:26:23 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:46 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-45-seanjc@google.com> Subject: [PATCH v13 44/85] KVM: x86/mmu: Convert page fault paths to kvm_faultin_pfn() From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112625_016492_C5C28719 X-CRM114-Status: GOOD ( 14.55 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Convert KVM x86 to use the recently introduced __kvm_faultin_pfn(). Opportunstically capture the refcounted_page grabbed by KVM for use in future changes. No functional change intended. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/mmu.c | 14 ++++++++++---- arch/x86/kvm/mmu/mmu_internal.h | 1 + 2 files changed, 11 insertions(+), 4 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index f730870887dd..2e2076287aaf 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4416,11 +4416,14 @@ static int kvm_mmu_faultin_pfn_private(struct kvm_vcpu *vcpu, static int __kvm_mmu_faultin_pfn(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault) { + unsigned int foll = fault->write ? FOLL_WRITE : 0; + if (fault->is_private) return kvm_mmu_faultin_pfn_private(vcpu, fault); - fault->pfn = __gfn_to_pfn_memslot(fault->slot, fault->gfn, false, true, - fault->write, &fault->map_writable); + foll |= FOLL_NOWAIT; + fault->pfn = __kvm_faultin_pfn(fault->slot, fault->gfn, foll, + &fault->map_writable, &fault->refcounted_page); /* * If resolving the page failed because I/O is needed to fault-in the @@ -4447,8 +4450,11 @@ static int __kvm_mmu_faultin_pfn(struct kvm_vcpu *vcpu, * to wait for IO. Note, gup always bails if it is unable to quickly * get a page and a fatal signal, i.e. SIGKILL, is pending. */ - fault->pfn = __gfn_to_pfn_memslot(fault->slot, fault->gfn, true, true, - fault->write, &fault->map_writable); + foll |= FOLL_INTERRUPTIBLE; + foll &= ~FOLL_NOWAIT; + fault->pfn = __kvm_faultin_pfn(fault->slot, fault->gfn, foll, + &fault->map_writable, &fault->refcounted_page); + return RET_PF_CONTINUE; } diff --git a/arch/x86/kvm/mmu/mmu_internal.h b/arch/x86/kvm/mmu/mmu_internal.h index 59e600f6ff9d..fabbea504a69 100644 --- a/arch/x86/kvm/mmu/mmu_internal.h +++ b/arch/x86/kvm/mmu/mmu_internal.h @@ -238,6 +238,7 @@ struct kvm_page_fault { /* Outputs of kvm_mmu_faultin_pfn(). */ unsigned long mmu_seq; kvm_pfn_t pfn; + struct page *refcounted_page; bool map_writable; /* From patchwork Thu Oct 10 18:23:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830913 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 68C23CFC5F9 for ; Thu, 10 Oct 2024 19:21:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Type:Cc:To: From:Subject:Message-ID:References:Mime-Version:In-Reply-To:Date: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=uRLwC7za7TRhyOe/CfUzhvpHD8RlodmrAsAEh9/RrLY=; b=vLN+NUpkePyiYBWrjGaWNhfhuY DFZteWGxugzUMQwbz7+FEYNCX0r6qXCljWJa/kiqo/ecrjyUuyigT1oRmvV3BRvVpZCfpcx17fLGL MjfregWPpmVPVX48NkcgrG0Zvs0H4UvhhMQDmQq30nPIhe4yRqD0TncJhEJ8xJPREzCFq/Z4l1/+P zeerGVlIJGcK7dBh8gB9JdR8RnrDiua5sU5mIeg046FlHgmyW+AiIQqypXPLGM/m26iDH7/eyArWl 0SDD2jt7jR145ffrTM232JkuE4CGYGyNK4p0iAXgfdo5HvRFR8Rjx6PfWaPgNfk/JfQhLWSR71GRk HUWI3kww==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syyj4-0000000E5S3-0zbb; Thu, 10 Oct 2024 19:21:30 +0000 Received: from mail-pf1-x44a.google.com ([2607:f8b0:4864:20::44a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxro-0000000Dqpz-07Sh for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:26:30 +0000 Received: by mail-pf1-x44a.google.com with SMTP id d2e1a72fcca58-71e123d3d22so1429051b3a.3 for ; Thu, 10 Oct 2024 11:26:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584787; x=1729189587; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=uRLwC7za7TRhyOe/CfUzhvpHD8RlodmrAsAEh9/RrLY=; b=PKoD/wJAb5njzUtpsWhaiNM2r+OCsHfWtiV/apSkhgK99vTei83ZnneTKYniaGobtS J45i/TolbRzdp5c0H22Luy6Rs4kKlLJaRGcpt/l3Ev+zc3SHfJ9EHEt2Wsh5CiYgVmOL SGYSWmYIMPQNWdysyBr7Fl4AZNrhGBJq/a7zAXoeKDXdLH9srkAQl32RTPc6myO/6pT+ +U1xZ7bie0Oj+11nv8QY/+JxDm6Kylusn84ikqXYk1Ql5f+RfVxJAVfljbZTD77+jbRg HvSzJHmdhjQLWIuP1P/nY+XLd0JF6+Ao5kyRK9GS8cjA6e6dQHUvz3M5OWWBXCiJa34T JjjA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584787; x=1729189587; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=uRLwC7za7TRhyOe/CfUzhvpHD8RlodmrAsAEh9/RrLY=; b=xCGyDOrj5eEeF+07kXyXWcw7kIrF0ylHTreYvcpxGD1h9DefoxHZ8o+4zM8dqJKN9U uF5dRl3dm4RFDLxg1ypcObuGbdGFMZWjFFnXm/2zSL+hPtUVxy0slb0Tq7Um8pwUNygk ZV15sJ7UDHHFZdmmLUeOJPW/cXWAIuglSMMm++2VvPWPUCk/8pyaQhPvIyzR+R5UUXeK Knt7SJ1KrWK+7ozwLTVMyOH168VPvs5nCRGFdeXogHhxz89kLAc1j05fxHviv2LYjkOY 4vM4iQffnjbeiYigy1XQGptlMNoM1yW/P9jAkYlnNKHJYQwngA2JeinVz57lERru2UUd O4Dg== X-Forwarded-Encrypted: i=1; AJvYcCXgYla93g1d/ZWE0vofl3Y5zXUxljByPOt7nnPyMMmg2qicQ7VGWT+m4luvHU/kZkH4OHd1719+NUN9STCkGAR+@lists.infradead.org X-Gm-Message-State: AOJu0YzpyePFcBfFBS/MveI1I8s7iwXuMwHupAzVIksutqnwpWycDPXe Xt2wBAfW6il18Kt3MiWKjfiYFgX2rjkYXW2Mv3CMd2BY/AlqHooKDExlv6jsHCs7rqgLJgQRGAu YFQ== X-Google-Smtp-Source: AGHT+IHljZntO7iRsDkdAW1llbbIMc26iesy/7PCA9rtG6130gQpp6naB/6JGeYrnZXaXdo41CPgD6F3FKs= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:aa7:9184:0:b0:71d:f7c9:8cb3 with SMTP id d2e1a72fcca58-71e1dbf1ab5mr7795b3a.5.1728584785442; Thu, 10 Oct 2024 11:26:25 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:47 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-46-seanjc@google.com> Subject: [PATCH v13 45/85] KVM: guest_memfd: Pass index, not gfn, to __kvm_gmem_get_pfn() From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112628_111581_0B27B2B1 X-CRM114-Status: GOOD ( 16.44 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Refactor guest_memfd usage of __kvm_gmem_get_pfn() to pass the index into the guest_memfd file instead of the gfn, i.e. resolve the index based on the slot+gfn in the caller instead of in __kvm_gmem_get_pfn(). This will allow kvm_gmem_get_pfn() to retrieve and return the specific "struct page", which requires the index into the folio, without a redoing the index calculation multiple times (which isn't costly, just hard to follow). Opportunistically add a kvm_gmem_get_index() helper to make the copy+pasted code easier to understand. Signed-off-by: Sean Christopherson --- virt/kvm/guest_memfd.c | 20 +++++++++++++------- 1 file changed, 13 insertions(+), 7 deletions(-) diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c index 8f079a61a56d..8a878e57c5d4 100644 --- a/virt/kvm/guest_memfd.c +++ b/virt/kvm/guest_memfd.c @@ -302,6 +302,11 @@ static inline struct file *kvm_gmem_get_file(struct kvm_memory_slot *slot) return get_file_active(&slot->gmem.file); } +static pgoff_t kvm_gmem_get_index(struct kvm_memory_slot *slot, gfn_t gfn) +{ + return gfn - slot->base_gfn + slot->gmem.pgoff; +} + static struct file_operations kvm_gmem_fops = { .open = generic_file_open, .release = kvm_gmem_release, @@ -551,12 +556,11 @@ void kvm_gmem_unbind(struct kvm_memory_slot *slot) } /* Returns a locked folio on success. */ -static struct folio * -__kvm_gmem_get_pfn(struct file *file, struct kvm_memory_slot *slot, - gfn_t gfn, kvm_pfn_t *pfn, bool *is_prepared, - int *max_order) +static struct folio *__kvm_gmem_get_pfn(struct file *file, + struct kvm_memory_slot *slot, + pgoff_t index, kvm_pfn_t *pfn, + bool *is_prepared, int *max_order) { - pgoff_t index = gfn - slot->base_gfn + slot->gmem.pgoff; struct kvm_gmem *gmem = file->private_data; struct folio *folio; @@ -592,6 +596,7 @@ __kvm_gmem_get_pfn(struct file *file, struct kvm_memory_slot *slot, int kvm_gmem_get_pfn(struct kvm *kvm, struct kvm_memory_slot *slot, gfn_t gfn, kvm_pfn_t *pfn, int *max_order) { + pgoff_t index = kvm_gmem_get_index(slot, gfn); struct file *file = kvm_gmem_get_file(slot); struct folio *folio; bool is_prepared = false; @@ -600,7 +605,7 @@ int kvm_gmem_get_pfn(struct kvm *kvm, struct kvm_memory_slot *slot, if (!file) return -EFAULT; - folio = __kvm_gmem_get_pfn(file, slot, gfn, pfn, &is_prepared, max_order); + folio = __kvm_gmem_get_pfn(file, slot, index, pfn, &is_prepared, max_order); if (IS_ERR(folio)) { r = PTR_ERR(folio); goto out; @@ -648,6 +653,7 @@ long kvm_gmem_populate(struct kvm *kvm, gfn_t start_gfn, void __user *src, long for (i = 0; i < npages; i += (1 << max_order)) { struct folio *folio; gfn_t gfn = start_gfn + i; + pgoff_t index = kvm_gmem_get_index(slot, gfn); bool is_prepared = false; kvm_pfn_t pfn; @@ -656,7 +662,7 @@ long kvm_gmem_populate(struct kvm *kvm, gfn_t start_gfn, void __user *src, long break; } - folio = __kvm_gmem_get_pfn(file, slot, gfn, &pfn, &is_prepared, &max_order); + folio = __kvm_gmem_get_pfn(file, slot, index, &pfn, &is_prepared, &max_order); if (IS_ERR(folio)) { ret = PTR_ERR(folio); break; From patchwork Thu Oct 10 18:23:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831064 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 97ED4D2444C for ; Thu, 10 Oct 2024 20:14:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:Content-Type:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=/DfuS3cqNKQbdjipc1jp6gNPynmr8H/oDbYVrbrij3Q=; b=kipkX+VLe39yaz SnpvPRxH1Gah2wy9gVc9NmBK9o56xJPBiaOFCSIWAiMhhKPjiHXWToZAsuqIM5RT6zIWVzU+vgZTG F4S6lj/LtGtVq4Td7FQyWUu+vKL+S8mNeL30IPbfop8Bw9ivT4iLoGTGc9Z3zm/Xe+u7Sct+ArSqx OhyQ0mre69dUnMk3Blulbm5e+BOh+4bnotd/xyaSvISD1XgpLiqScSbQRwc58k+mDbpU/Fa3WUel2 7ObGGK3p1jRKD/IxoZqql1d8fizNYDinPxGr7N0zVdAvWwCzd884MAel08XwNTqZgyiMcSkzUH3im gkzetN2tFLRlVreKo1Qg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syzYC-0000000EF89-04Sj; Thu, 10 Oct 2024 20:14:20 +0000 Received: from mail-yw1-x1149.google.com ([2607:f8b0:4864:20::1149]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxrp-0000000Dqs3-4Bl4 for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:26:32 +0000 Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-6e28b624bfcso20396497b3.2 for ; Thu, 10 Oct 2024 11:26:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584789; x=1729189589; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=/DfuS3cqNKQbdjipc1jp6gNPynmr8H/oDbYVrbrij3Q=; b=bQ9AW2IX1XkoVL4gLFimXEPTP1Mhxriw/jA12X4nUWJqxirBw2m1oV21TXm/Q54OeR 3xmOsY2+fN14ugnmdPBfXy10JZiNvnidlIpFjwC2GEgPzc3S1L2f1a7G1sLYsbXGtr6i 9XRCsT2/Cc4kodHq8DEyH1VOHP5NEgfR88ObjV0h7fThBsu7UaMqpD7iHSjQ0uhYnbpX UTjnPR9wQqePEmjiseiBgcqy85260s+ZN+LEAB38tRq3kQI0DwyghI6k3Keuhi5xIiqt IyWVuhf8iqzQLYFlCsp4W/nAMsCB+r6tSjgae+dSuOCaKKvoIZGnzvWO9SINB0Uprnwn bh7w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584789; x=1729189589; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=/DfuS3cqNKQbdjipc1jp6gNPynmr8H/oDbYVrbrij3Q=; b=RYSqjjl2XAaLDR/dk5JFM7M3XU40UPIWQ77X+s96clp7yejQlFsjCGtl+NDIRdk2Yw 04n4uFRaig7wDC8Tt0Ki6pNrG1ca+9OEqikqm91VkwvQo3Hfyn3Xtz4nYGSJ0UAY45Jt AilDK/7XSKiU5vw/k0wfyxGOGL4U7BfVH2yhaBpJP5jAxESmnoTX4idr7f2Z7WRo3wrX WXHEi/+AlxdDnphdt8lmSMCqZecyPCmp6uSJe1ADWnIyzD5XFZKfv0mf+NIeuIHMutVg NAKnOefKHFoZKBLx37HbrpGGrVfjVfxZgLsdgIdxCFrl5l4pqgcoiKHrw9rfGA/IRofD mjkg== X-Forwarded-Encrypted: i=1; AJvYcCWZcCP9pnsoIQqi8AlGzYck5jXfw5UX4iyonS6S/1wTak/+AuXOenCjqE7FCdCvlGKaiW9wlzXXAH+J4ihDVywB@lists.infradead.org X-Gm-Message-State: AOJu0YwG9P24NNwf5H5KFKsL02bENt36hWKcH+LeGLQgU9eHW2+2cspa rsYyZeMzWGZNeQuMed8CPOGm0JYLSKGB1KORXR696l7FKNxbLd6zYEx2OkthMrxx9XSnvzcFReY 6tg== X-Google-Smtp-Source: AGHT+IHLQFn8WLOIHu4PHZ3K+TrLKBMF9JlmWl30kTXK43TD90b+TNbg3ZepRFQQdmJeel5bU/EhDzobpdQ= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a25:b191:0:b0:e1f:eaf1:2254 with SMTP id 3f1490d57ef6-e28fe41acaamr82536276.8.1728584788687; Thu, 10 Oct 2024 11:26:28 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:48 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-47-seanjc@google.com> Subject: [PATCH v13 46/85] KVM: guest_memfd: Provide "struct page" as output from kvm_gmem_get_pfn() From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112630_322748_E4D27939 X-CRM114-Status: GOOD ( 15.97 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Provide the "struct page" associated with a guest_memfd pfn as an output from __kvm_gmem_get_pfn() so that KVM guest page fault handlers can directly put the page instead of having to rely on kvm_pfn_to_refcounted_page(). Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/mmu.c | 2 +- arch/x86/kvm/svm/sev.c | 10 ++++++---- include/linux/kvm_host.h | 6 ++++-- virt/kvm/guest_memfd.c | 8 ++++++-- 4 files changed, 17 insertions(+), 9 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 2e2076287aaf..a038cde74f0d 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4400,7 +4400,7 @@ static int kvm_mmu_faultin_pfn_private(struct kvm_vcpu *vcpu, } r = kvm_gmem_get_pfn(vcpu->kvm, fault->slot, fault->gfn, &fault->pfn, - &max_order); + &fault->refcounted_page, &max_order); if (r) { kvm_mmu_prepare_memory_fault_exit(vcpu, fault); return r; diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c index 4557ff3804ae..c6c852485900 100644 --- a/arch/x86/kvm/svm/sev.c +++ b/arch/x86/kvm/svm/sev.c @@ -3849,6 +3849,7 @@ static int __sev_snp_update_protected_guest_state(struct kvm_vcpu *vcpu) if (VALID_PAGE(svm->sev_es.snp_vmsa_gpa)) { gfn_t gfn = gpa_to_gfn(svm->sev_es.snp_vmsa_gpa); struct kvm_memory_slot *slot; + struct page *page; kvm_pfn_t pfn; slot = gfn_to_memslot(vcpu->kvm, gfn); @@ -3859,7 +3860,7 @@ static int __sev_snp_update_protected_guest_state(struct kvm_vcpu *vcpu) * The new VMSA will be private memory guest memory, so * retrieve the PFN from the gmem backend. */ - if (kvm_gmem_get_pfn(vcpu->kvm, slot, gfn, &pfn, NULL)) + if (kvm_gmem_get_pfn(vcpu->kvm, slot, gfn, &pfn, &page, NULL)) return -EINVAL; /* @@ -3888,7 +3889,7 @@ static int __sev_snp_update_protected_guest_state(struct kvm_vcpu *vcpu) * changes then care should be taken to ensure * svm->sev_es.vmsa is pinned through some other means. */ - kvm_release_pfn_clean(pfn); + kvm_release_page_clean(page); } /* @@ -4688,6 +4689,7 @@ void sev_handle_rmp_fault(struct kvm_vcpu *vcpu, gpa_t gpa, u64 error_code) struct kvm_memory_slot *slot; struct kvm *kvm = vcpu->kvm; int order, rmp_level, ret; + struct page *page; bool assigned; kvm_pfn_t pfn; gfn_t gfn; @@ -4714,7 +4716,7 @@ void sev_handle_rmp_fault(struct kvm_vcpu *vcpu, gpa_t gpa, u64 error_code) return; } - ret = kvm_gmem_get_pfn(kvm, slot, gfn, &pfn, &order); + ret = kvm_gmem_get_pfn(kvm, slot, gfn, &pfn, &page, &order); if (ret) { pr_warn_ratelimited("SEV: Unexpected RMP fault, no backing page for private GPA 0x%llx\n", gpa); @@ -4772,7 +4774,7 @@ void sev_handle_rmp_fault(struct kvm_vcpu *vcpu, gpa_t gpa, u64 error_code) out: trace_kvm_rmp_fault(vcpu, gpa, pfn, error_code, rmp_level, ret); out_no_trace: - put_page(pfn_to_page(pfn)); + kvm_release_page_unused(page); } static bool is_pfn_range_shared(kvm_pfn_t start, kvm_pfn_t end) diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 3b9afb40e935..504483d35197 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -2490,11 +2490,13 @@ static inline bool kvm_mem_is_private(struct kvm *kvm, gfn_t gfn) #ifdef CONFIG_KVM_PRIVATE_MEM int kvm_gmem_get_pfn(struct kvm *kvm, struct kvm_memory_slot *slot, - gfn_t gfn, kvm_pfn_t *pfn, int *max_order); + gfn_t gfn, kvm_pfn_t *pfn, struct page **page, + int *max_order); #else static inline int kvm_gmem_get_pfn(struct kvm *kvm, struct kvm_memory_slot *slot, gfn_t gfn, - kvm_pfn_t *pfn, int *max_order) + kvm_pfn_t *pfn, struct page **page, + int *max_order) { KVM_BUG_ON(1, kvm); return -EIO; diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c index 8a878e57c5d4..47a9f68f7b24 100644 --- a/virt/kvm/guest_memfd.c +++ b/virt/kvm/guest_memfd.c @@ -594,7 +594,8 @@ static struct folio *__kvm_gmem_get_pfn(struct file *file, } int kvm_gmem_get_pfn(struct kvm *kvm, struct kvm_memory_slot *slot, - gfn_t gfn, kvm_pfn_t *pfn, int *max_order) + gfn_t gfn, kvm_pfn_t *pfn, struct page **page, + int *max_order) { pgoff_t index = kvm_gmem_get_index(slot, gfn); struct file *file = kvm_gmem_get_file(slot); @@ -615,7 +616,10 @@ int kvm_gmem_get_pfn(struct kvm *kvm, struct kvm_memory_slot *slot, r = kvm_gmem_prepare_folio(kvm, slot, gfn, folio); folio_unlock(folio); - if (r < 0) + + if (!r) + *page = folio_file_page(folio, index); + else folio_put(folio); out: From patchwork Thu Oct 10 18:23:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831065 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id DD8F7D24442 for ; Thu, 10 Oct 2024 20:15:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:Content-Type:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=l4aL9ye2ZxrCnqTpLqQlinT65BUul16gfmDzid4NN5Y=; b=hanppG4DX6sTAw XvCJUWWPDlP1xBw4xNL39LFtBVn4JbcnxDp7mR9iY/3AWGcYnQgSOxhNUHryGeI5AefyqychkCNlv N3UdhsMFRBtD+8qJefH/8M303x+erCEQh40qvb44zN+4UHOkAld75qB54gonUBpmhTby8KrLMjB3b TlkX8cKAxo7lSNMTjTzPf6Ar90GoHvZ48Ql4hszbVsja3svDpD6nu8RMajvF4Gyeus+hPHMiAuh7+ g7MNQmg1wJrmnRv08P3ZYjQQjgmQrOD4spq4IwtnYU1qTfcYx6ESrw0HrsrNPj5xeswSK+TElP58k K4jTomYXc2UtzwYt4ZcQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syzZV-0000000EFNL-3cyP; Thu, 10 Oct 2024 20:15:41 +0000 Received: from mail-pf1-x449.google.com ([2607:f8b0:4864:20::449]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxrs-0000000DquX-3stc for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:26:34 +0000 Received: by mail-pf1-x449.google.com with SMTP id d2e1a72fcca58-71dff575924so1543448b3a.1 for ; Thu, 10 Oct 2024 11:26:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584792; x=1729189592; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=l4aL9ye2ZxrCnqTpLqQlinT65BUul16gfmDzid4NN5Y=; b=4NJxd+23djSQD5IrKrOM7+QBFAb+zUEzLAelhcpuCsN/jNgk8WGR2zJ21WF08csT9V yhMLGsZmfSz963iR9mrnBU4cwLKvwl5BPrO7GoCUIghqK86NVrPcw9z4snYKKkabblTO z/jBEzoAkqypTekJV+lLgO+n/EPna95q8QOQmqUodoUob08Ymw8w1UizJ3J5xwCC3ymz wTklvrSW+SKTZSOpfYN6lllfuRcZhijDV7ltjf7exYNEDFDq4/QVOcZQfFIaQqokMloc 9m29lmfaoY7ZsvkPtfUGCgdPTiTz7Ji2mxd/cOFOBuo50KHj5VvP2JR6hGeFnLweMZll 9c8w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584792; x=1729189592; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=l4aL9ye2ZxrCnqTpLqQlinT65BUul16gfmDzid4NN5Y=; b=jsThcBy2TDF+ZaxI3REM7ukrI76CH01Yqe4wLrt1E2TktGwC5N7oI9cAfdXOZsyqlm 3k1nbu1r3oSJ2Vdt0fUnkOGS8CN+tzeAYeUfznwZuBklMkOUUj83o6F/m7GBlMfUag6I rQrATVTd0fpZpm8AvJJfyhFM45S85ewA4tpeC96cyF8j3yL480wnikTdJy2d6U8bMvOI nyT+2Kuhqx0bmQ1WmE8u6Jr3HayH8+Uv//9rkPc10wKLhdprQuqWeBhdp8cU1Nv3n3d/ PhKQ5IC56ueyVhtBoAyIx/l0Di34i2SEyo6MyGSg28AO+eFTSrmza0Au7L9o8Z/H+oAN wDNw== X-Forwarded-Encrypted: i=1; AJvYcCWKefpZjrethw6tzbwjJENhVk1Htw/MA+ssdaVcDiI4OLMiCtlSkgiy9nXLLnxSdmYQdNtZinyLcmsK6jvYy+3T@lists.infradead.org X-Gm-Message-State: AOJu0YyJd5gngOOKZz41LLGpoJ1Nwqwc87z2iJhUExLehDMhBv+/Lzm9 Zp98zw9mIIbmxPL5r+NGSIcSVfDIU/SmJ5byzQtVhrspQkPYjyOi63jZW+FveIyp6fleDIBv02w +Ng== X-Google-Smtp-Source: AGHT+IFunsAHL0GYDEyxrwNnSExkBiXOv88aJkEgjVxw5ZDXOBsEr4Eha3ByEK4QIBQ4extfr7SnAz+Fl/s= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:aa7:9184:0:b0:71d:f7c9:8cb3 with SMTP id d2e1a72fcca58-71e1dbf1ab5mr7799b3a.5.1728584790633; Thu, 10 Oct 2024 11:26:30 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:49 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-48-seanjc@google.com> Subject: [PATCH v13 47/85] KVM: x86/mmu: Put refcounted pages instead of blindly releasing pfns From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112633_110419_1EAF8A4F X-CRM114-Status: GOOD ( 12.29 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Now that all x86 page fault paths precisely track refcounted pages, use Use kvm_page_fault.refcounted_page to put references to struct page memory when finishing page faults. This is a baby step towards eliminating kvm_pfn_to_refcounted_page(). Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/mmu.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index a038cde74f0d..f9b7e3a7370f 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4373,6 +4373,9 @@ static void kvm_mmu_finish_page_fault(struct kvm_vcpu *vcpu, lockdep_assert_once(lockdep_is_held(&vcpu->kvm->mmu_lock) || r == RET_PF_RETRY); + if (!fault->refcounted_page) + return; + /* * If the page that KVM got from the *primary MMU* is writable, and KVM * installed or reused a SPTE, mark the page/folio dirty. Note, this @@ -4384,9 +4387,9 @@ static void kvm_mmu_finish_page_fault(struct kvm_vcpu *vcpu, * folio dirty if KVM could locklessly make the SPTE writable. */ if (!fault->map_writable || r == RET_PF_RETRY) - kvm_release_pfn_clean(fault->pfn); + kvm_release_page_clean(fault->refcounted_page); else - kvm_release_pfn_dirty(fault->pfn); + kvm_release_page_dirty(fault->refcounted_page); } static int kvm_mmu_faultin_pfn_private(struct kvm_vcpu *vcpu, From patchwork Thu Oct 10 18:23:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831066 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 988B1D24442 for ; Thu, 10 Oct 2024 20:17:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:Content-Type:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=BN+hsHae4h8wzhzshUJRGKM30XxzO0F311sC9z4rUGc=; b=VCAri3Dw4Q+Fmj 2HCULb5HQ4eMmPvkZN4sBGBJsL32WP0YF6gYsoMrH/HR9sh1BKYq482oZsT4T4LM1c5YRg0VSPonc V5Swps/xHUySxEb8A5Vvwr4hstXwgqDwpyUSREMY34GZDBDnd1KKD3dQ5GP9ACfnp1NtY3gPOZizB 7VDRJCRb6y2p+2lGlP8niQWZe1Q8TEgV2btrp8RZvkiMp+MA0C/2BKd8yT2+WpckV8Ji4g80gsW3H VgHxf8Timo0YImRF31kQgxXHp21QkcWI0lqzYmx7l+MGhiu+S5WWBuKOUJjcx85PE1va7oPK2ICoe lJsNwTWAhxCJy4Vw/3ag==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syzao-0000000EFZB-41Y0; Thu, 10 Oct 2024 20:17:02 +0000 Received: from mail-yw1-x1149.google.com ([2607:f8b0:4864:20::1149]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxrv-0000000DqwK-2NcZ for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:26:38 +0000 Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-6e2e4874925so21251407b3.1 for ; Thu, 10 Oct 2024 11:26:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584794; x=1729189594; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=BN+hsHae4h8wzhzshUJRGKM30XxzO0F311sC9z4rUGc=; b=mFEQ1pk0GSblJlbvVxjSomWrM6IE+btPApHqV5NrilXRXoFJJ2QtCA8C3DRzZU638K fGwnvg2LFw4lj/o2cvU/G8+4i1rkmzmCYJ+K8kAtgcwfRhJdswGO4ZjYFmc0fOpnGAaO qM8T1NUyZqa09HXc5T1Ol762RIpMtB9T2BxUQSneDHsTS4aPWVdavwKvZRgVyE0BRFlK J2q6XJotNBexnIBUs1cGiNgPYYBQ8CryEKe7aIWrWBYO5N0YgHcBVCFW6MVxRz/ypuMy GI2HAbh6rArHdBxq4xWdbR4NHv2mRM+UDfic7XIfgYAex/UlvCHyl6arPi0UFMRFcmya 8iOg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584794; x=1729189594; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=BN+hsHae4h8wzhzshUJRGKM30XxzO0F311sC9z4rUGc=; b=NrCrwiTjGTPlkKBW/TbS0yOQoNapgj1lqfNwIvP7Li2+oCCdOHl/pVNssncKyNEa8a gwZSkNGp5rEDq28DYsoPWC7jpV0Keyt1JKaqZRyyf3pBjxet10lCN6TtNvh5X8aCNMzU Q8qDHMXfqkqJzPg+MHWiZc8pVwZ/WiZUyvFysO5HLmNulHOd3JdITtkjTd9lHfPUfOnr 3qaBvypaW7iEdTrIuOj2bjfIxJx/9tbKOssgiofl+cuVpaRTZ1EjJbuxiyxQlKAn+RjO xTG4hBunR0zEb/t9jJR3phEo2xqlq3A+39mhaBqhr0Nu/7AZMv/apVB6m7e4nhWWa1hs NhQQ== X-Forwarded-Encrypted: i=1; AJvYcCWqlR3xAPP2gSAq7hvUZX6nHfzeVr3zvhTEKMXS9au+9EaYrvbim1wB07O/+2tWJlC6/+0I66LcWBg80O6YnKeU@lists.infradead.org X-Gm-Message-State: AOJu0YwD3Cj6wJtaoMzzO5TkdheVEit8Bs2tv8HxZPThYJ5ZNuzrLE6D OaR3n7lVLYileCLwugMewrBqedj9boldUfDpxC+TnYUC7uMrGs2gJCT3upo9QX46k+rWkbQGfbM rRQ== X-Google-Smtp-Source: AGHT+IEjw5s9ln5Mmco0bHKHDYNNIWwRc71SIREcZZABOz/fcRYPRAU5+BeNEqASF/uZrX6+QTap8C7eoHE= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:690c:2c01:b0:6e2:3e17:1838 with SMTP id 00721157ae682-6e3221407b0mr440167b3.1.1728584793806; Thu, 10 Oct 2024 11:26:33 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:50 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-49-seanjc@google.com> Subject: [PATCH v13 48/85] KVM: x86/mmu: Don't mark unused faultin pages as accessed From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112635_659550_730172D0 X-CRM114-Status: GOOD ( 13.44 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org When finishing guest page faults, don't mark pages as accessed if KVM is resuming the guest _without_ installing a mapping, i.e. if the page isn't being used. While it's possible that marking the page accessed could avoid minor thrashing due to reclaiming a page that the guest is about to access, it's far more likely that the gfn=>pfn mapping was was invalidated, e.g. due a memslot change, or because the corresponding VMA is being modified. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/mmu.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index f9b7e3a7370f..e14b84d2f55b 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4386,7 +4386,9 @@ static void kvm_mmu_finish_page_fault(struct kvm_vcpu *vcpu, * fault handler, and so KVM must (somewhat) speculatively mark the * folio dirty if KVM could locklessly make the SPTE writable. */ - if (!fault->map_writable || r == RET_PF_RETRY) + if (r == RET_PF_RETRY) + kvm_release_page_unused(fault->refcounted_page); + else if (!fault->map_writable) kvm_release_page_clean(fault->refcounted_page); else kvm_release_page_dirty(fault->refcounted_page); From patchwork Thu Oct 10 18:23:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831186 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 36B87D24454 for ; Thu, 10 Oct 2024 21:31:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:Content-Type:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=Ndpfmsd07O3XHkbROvy9SWySHpmtrF0LUOG5EHCQjAE=; b=NyVsPtgCSi1+s+ X3iNHqKv1CxvLCqk6i/2+BrRNexzdoHZR4o/YZiZeLl5Wyd0ySO/pbPUo2sSvKnVwnlNf56NvOOwu P08MQm2/JRsNNsMr88eYO+lc+ErYkC2psPIUsU6MO9ClPRT6r6CJQrOiQYHEpHhX4sXgxfjQK/2/+ KX4zMdHppbIoEVB+A8dls1kchXm0vNA2FJhDzng1JQwzHIT9oPLSLskxB7O94wu5zboebdQZYCiWB 4DhSdOuJUYsEI8cBV9u+V4jXbCMAomwTMDB/jhHj+HdhBwhngbOSF4IeKQ8QsX9HbXIA9QdBCVVhi 22Bsb3cplF8wkXpK6aLw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1sz0kn-0000000EPpl-150I; Thu, 10 Oct 2024 21:31:25 +0000 Received: from mail-pg1-x549.google.com ([2607:f8b0:4864:20::549]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxrx-0000000DqyL-1mGr for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:26:39 +0000 Received: by mail-pg1-x549.google.com with SMTP id 41be03b00d2f7-7c6a9c1a9b8so928314a12.0 for ; Thu, 10 Oct 2024 11:26:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584796; x=1729189596; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=Ndpfmsd07O3XHkbROvy9SWySHpmtrF0LUOG5EHCQjAE=; b=4HGW8f7tjownBfYqPr3g4q5rap4pEjc4NDH2XoQtWUJmNugApsKx93exzBPNkaqmo7 22HQMvd+Q4rL+NUcId2bgBlZd7JTUfOuTG+AaGGp5/MieBVOiymMRBah3+emzZqc2o47 EaCg48mEguoYCSN3ZQZnwHaOSKoILqP3AFoA8aYxTJ9yw2yQTyuvbig92YQxp/aOz8iq L7zSmYRy7UmWkkcy15/lY4+BC2929WmoIEeZ+DilgVBJARphsvEY2zJXni0FF0XUYOSc OlRUp7qqzox27rKuo+WQQnNLhRVDeqO+GFbPztsLp2jp1XWndoR4wUm2paLbo9oPpwYg FXkQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584796; x=1729189596; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=Ndpfmsd07O3XHkbROvy9SWySHpmtrF0LUOG5EHCQjAE=; b=BqBX6FbrJ2/Kr2HGVHAbNi9Q1mqT+H9t82XbqU/ZurOlgabU8V+Swfmcf+BOfTxV9U C/C/zll/8cQAUN7ydVq0c2zXvTKLnrAMdsJi2pqmdmQU53p+gn4gs8gwzuAtGu0873ig 9DcxVcT7izrkkAQZr6R9dCoZVqWoc6yJeAv9APlTY07tdjyRj6S7NPKa6xthuooSD+Eq Ht1E532flwv6Ys8H0LBGTEfC47IAJtLqHgY3HDO5hNmXN2MPAEIF1NFXkQx+PIW/l/dQ GA28acg5Jb96pXl6nLHydZWCWiVu5pJ/OTMYMnaS2bbJqC/RSwIVxiI4VTyAN3j2ABLU fovw== X-Forwarded-Encrypted: i=1; AJvYcCW3wOMWOUTgI9upxStNRNfo3O56UhviC3OvCqn1iYcXGPJ+YW+zHGpngEF1gjETclNXOVRfbk0xsLZB85KAPmZT@lists.infradead.org X-Gm-Message-State: AOJu0YzIbW8sjDET/sONP7bVRwklwGYmdfJZ4k2vUHjNUA6EwiAKCXmS 7pxKYiMgM6E7HCZlJLQly2oDyQ3bYoh9CBfDrjryubvgbQwzz2REP+hu1q5XnFIZqb0OE1R5Ia3 hDA== X-Google-Smtp-Source: AGHT+IGJc8hgBZq6lOovV4Dgly5ctIXEs7jDnJfEupYxVqninaLorgqTg5WNW0BPJZYnJnHdEtp0yWaf7UM= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a65:450a:0:b0:717:a912:c302 with SMTP id 41be03b00d2f7-7ea53525e73mr49a12.1.1728584796119; Thu, 10 Oct 2024 11:26:36 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:51 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-50-seanjc@google.com> Subject: [PATCH v13 49/85] KVM: Move x86's API to release a faultin page to common KVM From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112637_566118_2A3183EE X-CRM114-Status: GOOD ( 16.37 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Move KVM x86's helper that "finishes" the faultin process to common KVM so that the logic can be shared across all architectures. Note, not all architectures implement a fast page fault path, but the gist of the comment applies to all architectures. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/mmu.c | 24 ++---------------------- include/linux/kvm_host.h | 26 ++++++++++++++++++++++++++ 2 files changed, 28 insertions(+), 22 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index e14b84d2f55b..5acdaf3b1007 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4370,28 +4370,8 @@ static u8 kvm_max_private_mapping_level(struct kvm *kvm, kvm_pfn_t pfn, static void kvm_mmu_finish_page_fault(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault, int r) { - lockdep_assert_once(lockdep_is_held(&vcpu->kvm->mmu_lock) || - r == RET_PF_RETRY); - - if (!fault->refcounted_page) - return; - - /* - * If the page that KVM got from the *primary MMU* is writable, and KVM - * installed or reused a SPTE, mark the page/folio dirty. Note, this - * may mark a folio dirty even if KVM created a read-only SPTE, e.g. if - * the GFN is write-protected. Folios can't be safely marked dirty - * outside of mmu_lock as doing so could race with writeback on the - * folio. As a result, KVM can't mark folios dirty in the fast page - * fault handler, and so KVM must (somewhat) speculatively mark the - * folio dirty if KVM could locklessly make the SPTE writable. - */ - if (r == RET_PF_RETRY) - kvm_release_page_unused(fault->refcounted_page); - else if (!fault->map_writable) - kvm_release_page_clean(fault->refcounted_page); - else - kvm_release_page_dirty(fault->refcounted_page); + kvm_release_faultin_page(vcpu->kvm, fault->refcounted_page, + r == RET_PF_RETRY, fault->map_writable); } static int kvm_mmu_faultin_pfn_private(struct kvm_vcpu *vcpu, diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 504483d35197..9f7682ece4a1 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -1231,6 +1231,32 @@ static inline void kvm_release_page_unused(struct page *page) void kvm_release_page_clean(struct page *page); void kvm_release_page_dirty(struct page *page); +static inline void kvm_release_faultin_page(struct kvm *kvm, struct page *page, + bool unused, bool dirty) +{ + lockdep_assert_once(lockdep_is_held(&kvm->mmu_lock) || unused); + + if (!page) + return; + + /* + * If the page that KVM got from the *primary MMU* is writable, and KVM + * installed or reused a SPTE, mark the page/folio dirty. Note, this + * may mark a folio dirty even if KVM created a read-only SPTE, e.g. if + * the GFN is write-protected. Folios can't be safely marked dirty + * outside of mmu_lock as doing so could race with writeback on the + * folio. As a result, KVM can't mark folios dirty in the fast page + * fault handler, and so KVM must (somewhat) speculatively mark the + * folio dirty if KVM could locklessly make the SPTE writable. + */ + if (unused) + kvm_release_page_unused(page); + else if (dirty) + kvm_release_page_dirty(page); + else + kvm_release_page_clean(page); +} + kvm_pfn_t __kvm_faultin_pfn(const struct kvm_memory_slot *slot, gfn_t gfn, unsigned int foll, bool *writable, struct page **refcounted_page); From patchwork Thu Oct 10 18:23:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831307 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 962E1D24459 for ; Thu, 10 Oct 2024 23:31:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:Content-Type:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=iay4RI/uPKTQE0By5qqA//dP+sG5w7++36T/RHxbCNM=; b=VQO+brxt2fEkGH VRNmrNp9yrZ5sEUDrOipz08oP9DWKFegKlCbK/3UJcIu8ZMlYbVSQskpidH3UqJ8z/o9tIEp5e3wj ZsDedLwQJkosC+37oVP2uAJ3LC4YWoAK0cu7YyEOMGeJoBuA8Y903LmBd7PuwrTo7SQQyYUeHMCoD bBrgLpQIqfzwxGPAqqrC43gwI6TgLy2tyVvjDIw/cw/L/E8TTcSOnAm8hwVUb/3O1VtZkFTnZ8CpI dDbrTEYIhEJ6GYqbhgAags9VZrAtA3F+QHhE2rAMDcmyKyKi4uYDI6Hu/ts+KnFq4/V4yo8VLDBJO aqyd9TOgIhJ2kNSjBDYg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1sz2d0-0000000Ef5R-3Krz; Thu, 10 Oct 2024 23:31:30 +0000 Received: from mail-pj1-x104a.google.com ([2607:f8b0:4864:20::104a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxs1-0000000Dr14-1PXT for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:26:42 +0000 Received: by mail-pj1-x104a.google.com with SMTP id 98e67ed59e1d1-2e2de6496easo621778a91.1 for ; Thu, 10 Oct 2024 11:26:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584800; x=1729189600; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=iay4RI/uPKTQE0By5qqA//dP+sG5w7++36T/RHxbCNM=; b=elSopkBGV/LG/XHB/FpT5FQ1XT7+s8gQjY51jYXfFfnx0SB95Fq9HirqWw0fsoteYW Bffj+ulNayw15y81/0D2n6Yv0IDMpZGIOP92+zOPG2I+hWzmnSL6j82Db+Nnuc+If0F+ jRQFR5zcK+C9bg3YuwgbQ6dacmJfuU9Yzf5hjvl0BdTAcz0oc/hsnwzTI6VTwzB1N359 yYxZs5N9ir8CnrpandHwL87esEKL7WNWT1uDtaC+el7gvgzBd+WfI6Ag1K5EEQ/8XTqN hmhMLcy6QLvFtNVMU8MuYmQBYmGdxbPvlTqSq9ZDA868gww6BrArZnM7UTRqLXqpeVdC 26wQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584800; x=1729189600; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=iay4RI/uPKTQE0By5qqA//dP+sG5w7++36T/RHxbCNM=; b=MeufxJSAFDhMnG5nnwhc8Z3avPDJy7p8A+OiF7FcBDlmuwIpZEdaTzxIAWgpWmfUzE fa5kDtTuXIAGwbgfmsb37i1BfL0E0JWNgbe0+boTDW0yRH1KyAImybiILK0W1ZB/sSCM v0GHsuLg3+qiGRLrivEo2UjvAKNOASKmaihVeke780eH00Lt4J0uJdvL6mEDgtuQ8AJ0 CBm+qGJ4w9Uj0haMXhHrtXw0sRKxDkTEGQQ8AZP60B8bXcJzQW18/+nHeiiDjKngz1HK dj4CAzO9YQvfSRuMx4t5o7NUwR1BKgBI2KJNfBI5bDw38lxLUG8wRgveTwpvlfPkMGtm 6PKg== X-Forwarded-Encrypted: i=1; AJvYcCWrC/JR3jqrfa6CuDus4i73PmkiJu7VUzkemNyjV7KvkH9TKNUE6P7s5Xr6QE/kfzIFRoGWW1FYeZhNFxd4rBb7@lists.infradead.org X-Gm-Message-State: AOJu0YxoUCfCqaUzAC8DLo4TKOYpJzWi/+nR9A07Emcs2OObuZdfN9G4 Rfz1b/NRlvI0FQVPTXD1d1JSe8v+fyrZIHn/3+0YrIbuuaiuGFNgB5QI2UrRi1yapK9ialchfUK Tvg== X-Google-Smtp-Source: AGHT+IEd4uxjZrUyEQQM0MU+jkiJqh3LQboji5/ZXtrCgOTVgrRv0yMO5ln1NKE0y5PwnoqkXxjnsp8xFDc= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:90a:fe86:b0:2da:872e:9ea4 with SMTP id 98e67ed59e1d1-2e2f0d7e9d7mr54a91.3.1728584798284; Thu, 10 Oct 2024 11:26:38 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:52 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-51-seanjc@google.com> Subject: [PATCH v13 50/85] KVM: VMX: Hold mmu_lock until page is released when updating APIC access page From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112641_414347_13A7488B X-CRM114-Status: GOOD ( 14.25 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Hold mmu_lock across kvm_release_pfn_clean() when refreshing the APIC access page address to ensure that KVM doesn't mark a page/folio as accessed after it has been unmapped. Practically speaking marking a folio accesses is benign in this scenario, as KVM does hold a reference (it's really just marking folios dirty that is problematic), but there's no reason not to be paranoid (moving the APIC access page isn't a hot path), and no reason to be different from other mmu_notifier-protected flows in KVM. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/x86/kvm/vmx/vmx.c | 21 +++++++++------------ 1 file changed, 9 insertions(+), 12 deletions(-) diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 1a4438358c5e..851be0820e04 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -6832,25 +6832,22 @@ void vmx_set_apic_access_page_addr(struct kvm_vcpu *vcpu) return; read_lock(&vcpu->kvm->mmu_lock); - if (mmu_invalidate_retry_gfn(kvm, mmu_seq, gfn)) { + if (mmu_invalidate_retry_gfn(kvm, mmu_seq, gfn)) kvm_make_request(KVM_REQ_APIC_PAGE_RELOAD, vcpu); - read_unlock(&vcpu->kvm->mmu_lock); - goto out; - } + else + vmcs_write64(APIC_ACCESS_ADDR, pfn_to_hpa(pfn)); - vmcs_write64(APIC_ACCESS_ADDR, pfn_to_hpa(pfn)); - read_unlock(&vcpu->kvm->mmu_lock); - - /* - * No need for a manual TLB flush at this point, KVM has already done a - * flush if there were SPTEs pointing at the previous page. - */ -out: /* * Do not pin apic access page in memory, the MMU notifier * will call us again if it is migrated or swapped out. */ kvm_release_pfn_clean(pfn); + + /* + * No need for a manual TLB flush at this point, KVM has already done a + * flush if there were SPTEs pointing at the previous page. + */ + read_unlock(&vcpu->kvm->mmu_lock); } void vmx_hwapic_isr_update(int max_isr) From patchwork Thu Oct 10 18:23:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831067 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 2C45DD2444D for ; Thu, 10 Oct 2024 20:21:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:Content-Type:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=W6Zzl9AmyqedpEIwNpLXiwPcmrnHw6ZPa/9pyFpLcZ0=; b=WCb8PZ6JfatFxB LmB6flgCWBJ/X1jZtABmMov4IuihAeVVonHT7sU9vMF2s1QnBClzBOmz8s0e7BQFEeDfgoHvfv9gz T0OI6l+rG+nE2hlVQLsCaTF07jKcEKQaAueVwY3bn+E2Xpg8dItOcn69WaOV6CT5ltQe0v9aOYOxp MebChTgfHIohKdDYoIZxD7Pm04SIdD8cWX3RcwE9+lgmY6LccCPGzCeUw4QTB9PvsODpMxKkbFkej 7FKZpACWkbbZoekC0KjZrrMneQUmHsofg5Y8V/cNZm/xtvwuUd9h3K5ZjA7wNjhVBHJaUvwqwkdFP WQ9MT+K3+QBeSRHIj2Wg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syzej-0000000EFmb-1pw8; Thu, 10 Oct 2024 20:21:05 +0000 Received: from mail-yw1-x114a.google.com ([2607:f8b0:4864:20::114a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxs2-0000000Dr2S-2lb9 for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:26:44 +0000 Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-6e3245ed6b8so23856787b3.0 for ; Thu, 10 Oct 2024 11:26:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584801; x=1729189601; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=W6Zzl9AmyqedpEIwNpLXiwPcmrnHw6ZPa/9pyFpLcZ0=; b=OrgrVnTtrz+qiD9I0S55GdwguFxumnKv3WwSP9lTvNcek28RXofi/P4YgBMLPPYMhb zCOiCCo9fd6Nk8bAoy3uGkg5aB+LFL9zzvsIoGmtZJWHk7S3XH6jGl4UgTdmy0jdzTRe uigdTw+2nDTBogSg6xzYhiat0dWmlOkE4NK4fdBCXO6eLIjWbfaMMyYAgIx+4I9JXs+q YYlcvGdNTOyx4Od36oRcYng2RADTfSTd5NrCF3iOCvOKsNAyx+6m8RHLPIhQGep99KBK 95kPdxMBUQVU3/eq0UM2UXAuETUAQK8pz5gNiTU0JXjKfe390eNJWy/Ucn6yEkU139JE isgA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584801; x=1729189601; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=W6Zzl9AmyqedpEIwNpLXiwPcmrnHw6ZPa/9pyFpLcZ0=; b=np7HjsuGFgoDa7Z4gAqoqVgI/LEmBik/Kqn7pI7nyOuUUWVjPyRyJ6GuvENR6up/I2 EacAAnGM1mOwrQDCexQM+wECYufc6KPOj0qDmXDFBNkVSboASYlRY0s5lnz2u19ChYEZ WAVQkkL0h4C7hWwce4wUOeHQdBxWn5GN4OvK+r/d4Tvm7D5pZU65sJW9Lg3myuTUyOuA cjnkFlO6WTn2n4XD5+XUVdAycGJf8CA69O6vCjW/Ra0U2z6c8SpddCz/YP36P17lwsHG ISgJxIYRp6y9+CXwk0xZfZkaUXCjMqnT6RWl2czM0Wwpiw3znJwMbPmaGQESIslO9EEG LRTA== X-Forwarded-Encrypted: i=1; AJvYcCXqMow+H7lRwq/1uTKfktNsnlpKihsD+Zcpy6jtxgM8YuR3J92scznxCRThVPJYDx1F+qRTfXUYLYnfTTe3B2dB@lists.infradead.org X-Gm-Message-State: AOJu0YyOwE0D8Wq2IZfDuK4n9pNo6SrLX1wK6XuldLV0GOvS/HcnRBkp E5zsCgMWM/T/Pn9asL0XbOULUf8p4s+kdO9TmcSCBAJo/UqzKPrt7XI73dtjNShOovfnXLSuUfx qOQ== X-Google-Smtp-Source: AGHT+IGFUTZueI+Si3awRWj8jA+YGlhXMEzUcm/2nm7iA7dHF8070ZlvI1MSF0nUJUhFBhXPPz9v/uVg5Fg= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:690c:2f08:b0:6e2:2c72:3abb with SMTP id 00721157ae682-6e3224db667mr816567b3.7.1728584801239; Thu, 10 Oct 2024 11:26:41 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:53 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-52-seanjc@google.com> Subject: [PATCH v13 51/85] KVM: VMX: Use __kvm_faultin_page() to get APIC access page/pfn From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112642_735781_7E781906 X-CRM114-Status: GOOD ( 15.80 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Use __kvm_faultin_page() get the APIC access page so that KVM can precisely release the refcounted page, i.e. to remove yet another user of kvm_pfn_to_refcounted_page(). While the path isn't handling a guest page fault, the semantics are effectively the same; KVM just happens to be mapping the pfn into a VMCS field instead of a secondary MMU. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/x86/kvm/vmx/vmx.c | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 851be0820e04..44cc25dfebba 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -6790,8 +6790,10 @@ void vmx_set_apic_access_page_addr(struct kvm_vcpu *vcpu) struct kvm *kvm = vcpu->kvm; struct kvm_memslots *slots = kvm_memslots(kvm); struct kvm_memory_slot *slot; + struct page *refcounted_page; unsigned long mmu_seq; kvm_pfn_t pfn; + bool writable; /* Defer reload until vmcs01 is the current VMCS. */ if (is_guest_mode(vcpu)) { @@ -6827,7 +6829,7 @@ void vmx_set_apic_access_page_addr(struct kvm_vcpu *vcpu) * controls the APIC-access page memslot, and only deletes the memslot * if APICv is permanently inhibited, i.e. the memslot won't reappear. */ - pfn = gfn_to_pfn_memslot(slot, gfn); + pfn = __kvm_faultin_pfn(slot, gfn, FOLL_WRITE, &writable, &refcounted_page); if (is_error_noslot_pfn(pfn)) return; @@ -6838,10 +6840,13 @@ void vmx_set_apic_access_page_addr(struct kvm_vcpu *vcpu) vmcs_write64(APIC_ACCESS_ADDR, pfn_to_hpa(pfn)); /* - * Do not pin apic access page in memory, the MMU notifier - * will call us again if it is migrated or swapped out. + * Do not pin the APIC access page in memory so that it can be freely + * migrated, the MMU notifier will call us again if it is migrated or + * swapped out. KVM backs the memslot with anonymous memory, the pfn + * should always point at a refcounted page (if the pfn is valid). */ - kvm_release_pfn_clean(pfn); + if (!WARN_ON_ONCE(!refcounted_page)) + kvm_release_page_clean(refcounted_page); /* * No need for a manual TLB flush at this point, KVM has already done a From patchwork Thu Oct 10 18:23:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831068 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 635EBD2444D for ; Thu, 10 Oct 2024 20:22:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Type:Cc:To: From:Subject:Message-ID:References:Mime-Version:In-Reply-To:Date: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=/fC7L3Fchvvuu9GyGKk9VEzuiyLOIeBtO2SWEaXDNWE=; b=BLMVEeR2AEZi1VQn0nW1YLKjB0 dSAU3TmSAXZPPxWjeKclgDrEls6QY7lWx+470FtSMs8NUeWcjrEWlQL09SsxYtNHJ7KTdoPYF9mA4 p/kXjIuRmfP8hZSpcICSaaz1uBW1pJ9I/QyurRFzRSpLIeBrHcCT+5JFmMHdk70tTuZgilNxxUg2X mdhO5cSRoQFOZdEngsJf7le/5UkIMI2w7nPrTvSdaFm1NXGUsTUXATykHwpTs66RCbWbRybMrLvY7 YD+a20KR/iBFGTDuTwUIUIKH9soMDn8zGzA1IEHIasVzMjq3p19l///9DtrPWtovBnnGOL5GBKNUJ +ABrzicg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syzg1-0000000EFvQ-4Bxj; Thu, 10 Oct 2024 20:22:26 +0000 Received: from mail-yw1-x114a.google.com ([2607:f8b0:4864:20::114a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxs4-0000000Dr4B-3eCa for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:26:46 +0000 Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-6e32f43c797so11877127b3.1 for ; Thu, 10 Oct 2024 11:26:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584803; x=1729189603; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=/fC7L3Fchvvuu9GyGKk9VEzuiyLOIeBtO2SWEaXDNWE=; b=BnLmoeZUgDZeTfvLJFFaz58gmWaTmR+ISKXM1NM9+8Tz6u4K97yuHNDt/+ETUMaj+3 fbCJPJ11scsCf1SP+PIrpg0K/mVzzcYJSLqADVfzCIHdWlRu3FqR50x4ezGvzQsafiwI V/afjWOZ/n1EQinWq7bWXdQ4rQ6WU70i1MUJGIwMRhIC/nDA9OKg2xkV95XZg4jVhnIq MCaeu9xDMI7eJlCThHlmOwEzeN/sh1eYMUjy5l3EXfzuPM2unnM2PEBaD7dRrjT1aG9O 8Z7SFpLghppE2P8lmROc1IplRfOtIfNZcuJoAatjBuBbu51DS2IopbOgls6I3ZHBiyPh LPFQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584803; x=1729189603; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=/fC7L3Fchvvuu9GyGKk9VEzuiyLOIeBtO2SWEaXDNWE=; b=ho0FUsjfPnCuKdxOMAPary3qD0FyrMPA07E2/ckOdv/yFJiKqLGKkeMuThjGNxv9IB kyfpvEOjeRR++IeiDgF9hygAuFM7rlbyDOP+HoapqMQYHLpFa602Ff3SHMl8+9IZfdWl kAXGncvi/boSZ8fYdcoiuLMg3u1kEJ/nTVvJ7Z6qAkB0HZtZ+sDQ+4cBMJiB784b3ZBf XU2e/gxVDnUWv5VKw2xSgG5YaiIDvb7X9tZMWHHXXr14Ewh24JXTQTOh7Jg0QxiX/RC7 K7eoX0QLiqQBcRkxd3hANkzYRPalTf44addbFVFg1kRYKZcWlIrf9CKsC1uXJc7eyuYq OSPA== X-Forwarded-Encrypted: i=1; AJvYcCWzgQaTcEtb8mksDD/tod8Ni7W7G7mosRazvuT/6H4XDJVk4Qen4pu19dPUonklOFOhTxxKagAvmJlre9NewkXr@lists.infradead.org X-Gm-Message-State: AOJu0YxQ3/Z7RhwBlyATWt/IqXVqvKlJHHmXFgO1ES0H6/+ubz5pQ3LN kZv2g5pQbVD3gFtaTkn8TLtGT6fGnfLz1wHPJ4ASWQufeavTSJ9D8A9IK+JaBprLfmuL7oc6N/Z LNQ== X-Google-Smtp-Source: AGHT+IF0bQSeBV+k7XDKFBXYzNvC7N99ueMTlOEEH5KNk6JZHzu/00W1ndtLnT51eyBFoV94Bmz3yNyB+P4= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:690c:dd1:b0:6db:e464:addc with SMTP id 00721157ae682-6e344ce9308mr1337b3.4.1728584803335; Thu, 10 Oct 2024 11:26:43 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:54 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-53-seanjc@google.com> Subject: [PATCH v13 52/85] KVM: PPC: e500: Mark "struct page" dirty in kvmppc_e500_shadow_map() From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112645_046479_CA0C8B69 X-CRM114-Status: GOOD ( 13.28 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Mark the underlying page as dirty in kvmppc_e500_ref_setup()'s sole caller, kvmppc_e500_shadow_map(), which will allow converting e500 to __kvm_faultin_pfn() + kvm_release_faultin_page() without having to do a weird dance between ref_setup() and shadow_map(). Opportunistically drop the redundant kvm_set_pfn_accessed(), as shadow_map() puts the page via kvm_release_pfn_clean(). Signed-off-by: Sean Christopherson --- arch/powerpc/kvm/e500_mmu_host.c | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-) diff --git a/arch/powerpc/kvm/e500_mmu_host.c b/arch/powerpc/kvm/e500_mmu_host.c index c664fdec75b1..5c2adfd19e12 100644 --- a/arch/powerpc/kvm/e500_mmu_host.c +++ b/arch/powerpc/kvm/e500_mmu_host.c @@ -242,7 +242,7 @@ static inline int tlbe_is_writable(struct kvm_book3e_206_tlb_entry *tlbe) return tlbe->mas7_3 & (MAS3_SW|MAS3_UW); } -static inline void kvmppc_e500_ref_setup(struct tlbe_ref *ref, +static inline bool kvmppc_e500_ref_setup(struct tlbe_ref *ref, struct kvm_book3e_206_tlb_entry *gtlbe, kvm_pfn_t pfn, unsigned int wimg) { @@ -252,11 +252,7 @@ static inline void kvmppc_e500_ref_setup(struct tlbe_ref *ref, /* Use guest supplied MAS2_G and MAS2_E */ ref->flags |= (gtlbe->mas2 & MAS2_ATTRIB_MASK) | wimg; - /* Mark the page accessed */ - kvm_set_pfn_accessed(pfn); - - if (tlbe_is_writable(gtlbe)) - kvm_set_pfn_dirty(pfn); + return tlbe_is_writable(gtlbe); } static inline void kvmppc_e500_ref_release(struct tlbe_ref *ref) @@ -337,6 +333,7 @@ static inline int kvmppc_e500_shadow_map(struct kvmppc_vcpu_e500 *vcpu_e500, unsigned int wimg = 0; pgd_t *pgdir; unsigned long flags; + bool writable = false; /* used to check for invalidations in progress */ mmu_seq = kvm->mmu_invalidate_seq; @@ -490,7 +487,9 @@ static inline int kvmppc_e500_shadow_map(struct kvmppc_vcpu_e500 *vcpu_e500, goto out; } } - kvmppc_e500_ref_setup(ref, gtlbe, pfn, wimg); + writable = kvmppc_e500_ref_setup(ref, gtlbe, pfn, wimg); + if (writable) + kvm_set_pfn_dirty(pfn); kvmppc_e500_setup_stlbe(&vcpu_e500->vcpu, gtlbe, tsize, ref, gvaddr, stlbe); From patchwork Thu Oct 10 18:23:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831200 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 2FF7DD24456 for ; Thu, 10 Oct 2024 21:46:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Type:Cc:To: From:Subject:Message-ID:References:Mime-Version:In-Reply-To:Date: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=RAEBOjiR86B8MKio5svj+gL/GJb0VtWq7ImNxArS0H8=; b=Kwwz4Dr5i//fBDM6BHBX3zNzFk nN7ysdvrDmQ3RknEbLQxXlRIDNEArtYwQO7+8oQTIG5Cd7to2LHBSngvZRfN0OhKQvYTosvJZlUbw K30CFOFpwxnYMonluf4k93hk7j0hhWLOLAypRh2iIS1DGpJgH2WZtT1poMKBmxYMlJWtPNoTRlqoV phYo9/UVHvn0jM+box9Lj/6w51PBX/AVw9tFEh/7WQ3SXYLaLN11QtxdTPa674RUHxzAHc3s9OF6o rcePe4h8mKwQuoDnpIeuZRbjJK2I1apIBfVe2F7/jMqIpyMeSqEp91trFDoSgHHQeQ0b8XJzBtfcI BrtrZ1/g==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1sz0zK-0000000ESSd-0Brt; Thu, 10 Oct 2024 21:46:26 +0000 Received: from mail-pj1-x1049.google.com ([2607:f8b0:4864:20::1049]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxs6-0000000Dr5x-4B0R for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:26:48 +0000 Received: by mail-pj1-x1049.google.com with SMTP id 98e67ed59e1d1-2e18b6cd304so1251732a91.0 for ; Thu, 10 Oct 2024 11:26:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584805; x=1729189605; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=RAEBOjiR86B8MKio5svj+gL/GJb0VtWq7ImNxArS0H8=; b=M+42XwPeNm2oSLyAg3Vl9WyPEJno5UFR5cRhnp+qZDz+0cqpTMt6YrirrABxaPv4FV X5sXr42pqnv6G1sQWDVLbO442GXAXQG2xyM3M6hcjelKEbRIy6LZO34fqlWHgsRi7Dn5 pe3x97wUmaoCQM1ZXS/3Vl1pXCclzZ79Oao4ekCGwyg+ntTW5kORcLutbnMz2yqINPCW e6N8R1w5Gma7o0kQmT4SxmZY4nm22nwQ7NlqJ5SdPUEv2jT5Vm9fPfwj7eK0gmVTDy6p /bUqBoiqtFPjelGUsNSKXHemm2iWSIzBddpPfZJioxZ+Ky5Ja03rjPcof5VsFg8VXqyr vjOg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584805; x=1729189605; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=RAEBOjiR86B8MKio5svj+gL/GJb0VtWq7ImNxArS0H8=; b=BVSAxtCRm7XWWOIRRNhzGok+6PeDkNNhODQVPK5ie2qx7flAXVpj34mZpHhGUqtLRB OAJATPwa1f4ni6bvy5WEMuFU+w26O4wI6kZc8yjYLb94HPz/oq/g3hHWeE38GILBubW2 F1nt3YQKhFPyY9C9+n7usCrYkwUQ9uaU8jl9rwFFxq5rHRoG6X8CLph0ON5j2RAlBbYX Xj5A7Z+g+gptu4Ume/ADI331dPZJbHNaqyB6KgXi4hlrPXMYSafoewDnP+G3UmjmgemK MpLbAd6ADAQH5EkC8lNmHmoqjucYzEFHP9gm7cYNBr1DjWX5goUInu9YzMT6HbXMmX9d CxHw== X-Forwarded-Encrypted: i=1; AJvYcCVqyGH9GTcafiOaJMaVUuHdP2zYGEViQo7EpAdLfkJ9ivuSw2RVR6mre2C31WzwSgJvwHkFQH9w9faJ/r3Sa5kj@lists.infradead.org X-Gm-Message-State: AOJu0Yy/b0smcwm9XC8/Y7ZNanBqQMxTacK94+f+oPYhaJNYIiLvs/BE rQnINowjFuvOypchPXD/iSRmDoSQETuFs6xs/WNtuzdmXPCd53ZgU4GDwnVq216T8GOBsxe0sSi T4Q== X-Google-Smtp-Source: AGHT+IH3g+or1uGKI8zaqA7g6VjAId1uToioip8sNGZo/KyXggeZoODfUOK1q5z450FXHERnWYgwR7EwX2I= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:90a:5107:b0:2e2:ca3e:10fe with SMTP id 98e67ed59e1d1-2e2f0f88a54mr24a91.8.1728584805241; Thu, 10 Oct 2024 11:26:45 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:55 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-54-seanjc@google.com> Subject: [PATCH v13 53/85] KVM: PPC: e500: Mark "struct page" pfn accessed before dropping mmu_lock From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112647_145524_4C77BAF4 X-CRM114-Status: UNSURE ( 9.80 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Mark pages accessed before dropping mmu_lock when faulting in guest memory so that shadow_map() can convert to kvm_release_faultin_page() without tripping its lockdep assertion on mmu_lock being held. Marking pages accessed outside of mmu_lock is ok (not great, but safe), but marking pages _dirty_ outside of mmu_lock can make filesystems unhappy. Signed-off-by: Sean Christopherson --- arch/powerpc/kvm/e500_mmu_host.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/arch/powerpc/kvm/e500_mmu_host.c b/arch/powerpc/kvm/e500_mmu_host.c index 5c2adfd19e12..334dd96f8081 100644 --- a/arch/powerpc/kvm/e500_mmu_host.c +++ b/arch/powerpc/kvm/e500_mmu_host.c @@ -498,11 +498,9 @@ static inline int kvmppc_e500_shadow_map(struct kvmppc_vcpu_e500 *vcpu_e500, kvmppc_mmu_flush_icache(pfn); out: - spin_unlock(&kvm->mmu_lock); - /* Drop refcount on page, so that mmu notifiers can clear it */ kvm_release_pfn_clean(pfn); - + spin_unlock(&kvm->mmu_lock); return ret; } From patchwork Thu Oct 10 18:23:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831199 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B761AD24454 for ; Thu, 10 Oct 2024 21:46:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Type:Cc:To: From:Subject:Message-ID:References:Mime-Version:In-Reply-To:Date: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=nHP38EtPz8K1+ebGaPqvgAKvgaQP36cKGSrn8JpxRdw=; b=3WKf74C9tlyhkqlIrVMr9pBw4Q bEGvKx43YYX7F4VAjVOdlVBjiWcXf15AKrYSKWDX8BZKRsi7G9u6jn97mA6KP+4vsYpr9lW27MKEm dQ8NLYpL0Y/Hzt/548hJBKhvWA+gVRuojVpqaOik1fdl3k1uP96//pgv4B38BirxmQ2wXR+D77T9o XzbDNJKkmny3tkPydaXtQKSKw393FyxwYreXPvrEXslQkNrdKX+c1s3fs0GP0JLeZghpFV9uNiGo2 hrVm5z63aVkWS0BOc8MFbVeGUJ7JO1Vl4H7VZoNf0cRQuCZu08+2LB2PyyGH40C3Du24EpIgJjbtW cagf71VA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1sz0zK-0000000ESSo-2SlJ; Thu, 10 Oct 2024 21:46:26 +0000 Received: from mail-yb1-xb49.google.com ([2607:f8b0:4864:20::b49]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxs8-0000000Dr7J-3AaR for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:26:50 +0000 Received: by mail-yb1-xb49.google.com with SMTP id 3f1490d57ef6-e290a9a294fso1855086276.2 for ; Thu, 10 Oct 2024 11:26:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584807; x=1729189607; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=nHP38EtPz8K1+ebGaPqvgAKvgaQP36cKGSrn8JpxRdw=; b=uPMYi0jlcTO7wI+Xm2S76hfStVln/w8HUwai+nZMA8xWjcjtyTkYO5BLvyQK9HVnVf IoC4lxHelaOpDAbOwDoB+oedr7QlY1s0rbnaO+yPdv/IKAl3sf4qGFiPJRU3CPMHE1Ts bmeAWdy0d19eYiKgbHnbIWmxF0u1ugZLdKpLZO2jEwuBE0mZanlwFxa/r776lqNmdndR 7aH8b9KOMLsBIOlzOWkLUDwWW1S1geoFO5SEFRbl3u07z8W40g/J3NQFWAUGK5Zf9guL qrxNClMmqikhJpVN2wcLFUeTcVHJZSytmG4budlvJ+haM4VLpLTbPtFg47vTPcWmRRDU xtzw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584807; x=1729189607; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=nHP38EtPz8K1+ebGaPqvgAKvgaQP36cKGSrn8JpxRdw=; b=YT4/mrzaf+KgkCbFdlFJs9g3QQo3eQslPIKKWeJ2mQRd6UznEw74fqhHG2hutIEyNM y9+PbLCgsIBqG9xliXDhLTCxgwkQZNzI02fQ3dE3h9dNaGgDA09suBgVmEob4r3yUva4 ff0h//Ok/A05KfPaIa9pjBrz/iKdcxz759ZfTFL4fwvgLyJpSlfqFeP9gs5cdNu2WZ+D 4+MlUvCz9tv2bolYmiteefL9f7IUAfxtqia1cy2Gt6TU55BKQRnwcpKbuTJCJEyGHpKR gUGQAPzSmivwdPkT91X07AlHvmAiIBinceUXnlNoT6SKi5cX33VBQeavLPIksi96Ow8d BthA== X-Forwarded-Encrypted: i=1; AJvYcCXf0F4uJuiRA24ETE8MOi7iNebHXYuVqJgCo7Rcj8M/VSdq5UtjlG0P7PJKaPJmxnT6q99+x2vUAl+Vp0AcbUWi@lists.infradead.org X-Gm-Message-State: AOJu0Yz7+BPyBntEbonlDZa0mrIq2AsZI9zXYSbM7A0ZnsBvoZgxh5Z2 i4zy7TpcenH56L7HecO5s6N9ySr4ez4yEBS0URD2nACZN1AuMd870SnTFtJECzyWOA7fZBDZnbH X0Q== X-Google-Smtp-Source: AGHT+IHQvsj0FcPHwm2v4Ks73fb0XY1uO3QmFFCKoUyfp7OKRYByAKtc8uXQ4anZ8QaSR/IH5oavOTtruEI= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a5b:b86:0:b0:e0e:8b26:484e with SMTP id 3f1490d57ef6-e28fe516b5amr5188276.8.1728584807403; Thu, 10 Oct 2024 11:26:47 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:56 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-55-seanjc@google.com> Subject: [PATCH v13 54/85] KVM: PPC: e500: Use __kvm_faultin_pfn() to handle page faults From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112648_869843_DC7F57BD X-CRM114-Status: GOOD ( 13.35 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Convert PPC e500 to use __kvm_faultin_pfn()+kvm_release_faultin_page(), and continue the inexorable march towards the demise of kvm_pfn_to_refcounted_page(). Signed-off-by: Sean Christopherson --- arch/powerpc/kvm/e500_mmu_host.c | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/arch/powerpc/kvm/e500_mmu_host.c b/arch/powerpc/kvm/e500_mmu_host.c index 334dd96f8081..e5a145b578a4 100644 --- a/arch/powerpc/kvm/e500_mmu_host.c +++ b/arch/powerpc/kvm/e500_mmu_host.c @@ -322,6 +322,7 @@ static inline int kvmppc_e500_shadow_map(struct kvmppc_vcpu_e500 *vcpu_e500, { struct kvm_memory_slot *slot; unsigned long pfn = 0; /* silence GCC warning */ + struct page *page = NULL; unsigned long hva; int pfnmap = 0; int tsize = BOOK3E_PAGESZ_4K; @@ -443,7 +444,7 @@ static inline int kvmppc_e500_shadow_map(struct kvmppc_vcpu_e500 *vcpu_e500, if (likely(!pfnmap)) { tsize_pages = 1UL << (tsize + 10 - PAGE_SHIFT); - pfn = gfn_to_pfn_memslot(slot, gfn); + pfn = __kvm_faultin_pfn(slot, gfn, FOLL_WRITE, NULL, &page); if (is_error_noslot_pfn(pfn)) { if (printk_ratelimit()) pr_err("%s: real page not found for gfn %lx\n", @@ -488,8 +489,6 @@ static inline int kvmppc_e500_shadow_map(struct kvmppc_vcpu_e500 *vcpu_e500, } } writable = kvmppc_e500_ref_setup(ref, gtlbe, pfn, wimg); - if (writable) - kvm_set_pfn_dirty(pfn); kvmppc_e500_setup_stlbe(&vcpu_e500->vcpu, gtlbe, tsize, ref, gvaddr, stlbe); @@ -498,8 +497,7 @@ static inline int kvmppc_e500_shadow_map(struct kvmppc_vcpu_e500 *vcpu_e500, kvmppc_mmu_flush_icache(pfn); out: - /* Drop refcount on page, so that mmu notifiers can clear it */ - kvm_release_pfn_clean(pfn); + kvm_release_faultin_page(kvm, page, !!ret, writable); spin_unlock(&kvm->mmu_lock); return ret; } From patchwork Thu Oct 10 18:23:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831201 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 907C7D24455 for ; Thu, 10 Oct 2024 21:46:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Type:Cc:To: From:Subject:Message-ID:References:Mime-Version:In-Reply-To:Date: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=K6YUqJX41UvLftiTjNafBx3/10X+6KBTIzQjXduU2xw=; b=csMwehN31RHuOxaRI7QWXa0TZz A8XJDO5PJHeNSr1UVFUsW2CxQGxH6NE8xK9CSzZsyyS5OcYrIyFBtBQHS2IU+nkTMdkGUiyoXtK9Q lsE1/cdF8QcsKPj30WqjWVphvKSL7arhMuNXuX4PV70SN64mAG6Kux3a7BdRZgsKmFoLiob4qSg21 bUo1CPxU0zi+Uj63YQoQBVE9+Gt2OlNmpRrxELDWIWWzEV1mbeRz62elSA7IyrU+Amr4asvifLGPC BQRYzJLsKPK97YxjjKjtSuDVfq/YrzN3rDP7HTJ5ugSc5KIKKE0pcEPg1aEIEGcsMENSsmkJnvckt qHrsBXIw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1sz0zN-0000000ESUR-0Xq4; Thu, 10 Oct 2024 21:46:29 +0000 Received: from desiato.infradead.org ([2001:8b0:10b:1:d65d:64ff:fe57:4e05]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxsH-0000000DrEt-0NTG for linux-arm-kernel@bombadil.infradead.org; Thu, 10 Oct 2024 18:26:57 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Content-Type:Cc:To:From:Subject: Message-ID:References:Mime-Version:In-Reply-To:Date:Reply-To:Sender: Content-Transfer-Encoding:Content-ID:Content-Description; bh=K6YUqJX41UvLftiTjNafBx3/10X+6KBTIzQjXduU2xw=; b=dvfE6nwf754F42CDB2xgtEtI3r HVCHC/Xef8oy8zv+SDugB+zWqbLC7B+g5DnSMjcfeGoZzv/xcesZqPyE5Xk+b624o5I5ByydyllJW TZIYO4FGHSvqxuFFibyXXszXvxqXmE75jE9zoFKdT5KtQQ9muSP8q6RGO5kf6UknwNaJfJnMMRznY nwuKDrchtEHtAvc3xBmyFyWXZYkQlnzS9LPYILGZWM0pa+9WM7IhObWpUHTTbCvbcd2GSg2dGLzab OtMRmw1voNGTWjzb6rzGQNMXGFlIMJowcK47cX/3WAJkMHYo1WsC4ZaimP+03WGaA5Wau9Rm/wMJk 7simCZww==; Received: from mail-pf1-x449.google.com ([2607:f8b0:4864:20::449]) by desiato.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxsD-00000005UX0-1Y2o for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:26:55 +0000 Received: by mail-pf1-x449.google.com with SMTP id d2e1a72fcca58-71e0228d71dso1225691b3a.0 for ; Thu, 10 Oct 2024 11:26:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584810; x=1729189610; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=K6YUqJX41UvLftiTjNafBx3/10X+6KBTIzQjXduU2xw=; b=0NCiVq29rFGnhx5H88IoL3ATW2C2rUWOjXZSdIu+X2IO47lSMoJniHN5TNSeqAXiCE vdYQvKFrx3oDn1gRyQMyhpHTz+6quCptZkRRo3o7mi+i280e9CsMWrw12MOxLkLf/z0d n9S3M2ueKS5c0w86AOpvJSx6WYvwwYWNDLTNWHkWQHmHyk73I3u+hhvSuESEc9jxPtu9 I8RiTJ2ti60nNLRKX10QJc1F/7mmAIhzBpW5L7gTAasZ32ELbTbyr66jwUWqpuYSli8q TzZYiRXhNgW10vDG12czBFzP40DQAV1W9mSZKLrtG3DwUE5f8tQuEUoHd9oyl6g+1FqZ QTyw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584810; x=1729189610; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=K6YUqJX41UvLftiTjNafBx3/10X+6KBTIzQjXduU2xw=; b=TTaO/WRbU6FE1ZMqY0drjxc6eu25wPXLMW3hX6aeLGGxM0r6fegahvqXiAORMcU0Sz BGiHkrfkgNnKlyfvS8Vqp+czvpP+8Zor3TBBgHfijMYY0OQ90ANfAFLIJ/SsXwzwIb/Y MGcO8k1MXZTOnw+vzcd+ZZ4cPrJR1S9IvThTcLqI0l1RVCgWCgqewtu7xxSfQ+4Jn6j+ c9qE0lQGY+JZXn6LP2GdFnljRB6cHhKRocPm5jcBh0MrlrMbG0l3SmEyc0thxk3hdutU iaC8O2TE8HUebKvyIRhauiI+rAkqOV+UTJuLjFFwOS2mQMd2XEAVXQi4l29nKRYSQXKy SLBA== X-Forwarded-Encrypted: i=1; AJvYcCUFsh2ojUVw4+D0Pg7YZ0K+ecb7GnB26rIgl/qcIiXib4qKW2e8rMbAKhG6eudkqoIMyq4X2wxGeFa0mNX1JzWE@lists.infradead.org X-Gm-Message-State: AOJu0Yyh5a5ExZY9JIwiqO5vvZOUnI2yzAvJMzPd+m7SGj9mcc0yyqis tYG2bDV/t5pbaErQtL7+qFsTPgBQaUorCZotUt3gwE36eH7y3ZLsH83yhaRgb73QwXekUKXeHsB jxw== X-Google-Smtp-Source: AGHT+IEO1ECBC98fGvB+GRqYHAQaBdxJcqvY9DmP3QI4gZm0KdOyij1fLmFl0bPXgfvra8JJdQe8JNFxBHM= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:6a00:9199:b0:71e:1e8:e337 with SMTP id d2e1a72fcca58-71e1dbe467fmr8496b3a.4.1728584809106; Thu, 10 Oct 2024 11:26:49 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:57 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-56-seanjc@google.com> Subject: [PATCH v13 55/85] KVM: arm64: Mark "struct page" pfns accessed/dirty before dropping mmu_lock From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_192653_511275_4AEFD359 X-CRM114-Status: GOOD ( 11.89 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Mark pages/folios accessed+dirty prior to dropping mmu_lock, as marking a page/folio dirty after it has been written back can make some filesystems unhappy (backing KVM guests will such filesystem files is uncommon, and the race is minuscule, hence the lack of complaints). While scary sounding, practically speaking the worst case scenario is that KVM would trigger this WARN in filemap_unaccount_folio(): /* * At this point folio must be either written or cleaned by * truncate. Dirty folio here signals a bug and loss of * unwritten data - on ordinary filesystems. * * But it's harmless on in-memory filesystems like tmpfs; and can * occur when a driver which did get_user_pages() sets page dirty * before putting it, while the inode is being finally evicted. * * Below fixes dirty accounting after removing the folio entirely * but leaves the dirty flag set: it has no effect for truncated * folio and anyway will be cleared before returning folio to * buddy allocator. */ if (WARN_ON_ONCE(folio_test_dirty(folio) && mapping_can_writeback(mapping))) folio_account_cleaned(folio, inode_to_wb(mapping->host)); KVM won't actually write memory because the stage-2 mappings are protected by the mmu_notifier, i.e. there is no risk of loss of data, even if the VM were backed by memory that needs writeback. See the link below for additional details. This will also allow converting arm64 to kvm_release_faultin_page(), which requires that mmu_lock be held (for the aforementioned reason). Link: https://lore.kernel.org/all/cover.1683044162.git.lstoakes@gmail.com Signed-off-by: Sean Christopherson --- arch/arm64/kvm/mmu.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c index dd221587fcca..ecc6c2b56c43 100644 --- a/arch/arm64/kvm/mmu.c +++ b/arch/arm64/kvm/mmu.c @@ -1692,15 +1692,17 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, } out_unlock: + if (writable && !ret) + kvm_release_pfn_dirty(pfn); + else + kvm_release_pfn_clean(pfn); + read_unlock(&kvm->mmu_lock); /* Mark the page dirty only if the fault is handled successfully */ - if (writable && !ret) { - kvm_set_pfn_dirty(pfn); + if (writable && !ret) mark_page_dirty_in_slot(kvm, memslot, gfn); - } - kvm_release_pfn_clean(pfn); return ret != -EAGAIN ? ret : 0; } From patchwork Thu Oct 10 18:23:58 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831311 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 91496D2444F for ; Thu, 10 Oct 2024 23:31:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:Content-Type:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=L6S7ecEpHeP2pNYK74CcyfNjVxKIOVPI1u0giMU3AdU=; b=OgeLa3wg0KM9JK VhmovM0esCrHdu3DXkDPsq8c7ibgB3rEfW4dWkyZAkWuJslXzJmoqEeg4fF7GcLoVVj1f1xhnRyz6 MVt17QQqJAz6TVjuB10O1iuzu/Yzuggebqd1TmuReOYnsQVLyzrintkwLF6CF6c262TSGUie7iY3J org+8FG0JmVjw1jmK8xOfzFLdMK5qNqvi0PKPfILKZce0U0Qj5l3L4bcIQp2+vA/FSR9J3Fz9jfvW UtmMxQku1QBw2PSMHPZNb+Jnn46JRwDFxTh1moSVvi/lqI/hmJAOwh2au05WsGHajwpnZSALjr3/6 4FC4iAx/6j0eo2u8nYIQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1sz2d1-0000000Ef5X-1TKh; Thu, 10 Oct 2024 23:31:31 +0000 Received: from mail-pj1-x1049.google.com ([2607:f8b0:4864:20::1049]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxsD-0000000DrC2-47GE for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:26:56 +0000 Received: by mail-pj1-x1049.google.com with SMTP id 98e67ed59e1d1-2e2e146c77cso672191a91.3 for ; Thu, 10 Oct 2024 11:26:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584812; x=1729189612; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=L6S7ecEpHeP2pNYK74CcyfNjVxKIOVPI1u0giMU3AdU=; b=Tuq8jkdyupHP826P5bLjfoLYxHZeD/rX8rWQA7Q8ZEMCeDRAxuIwjVqa40D00ztFy+ D2mzL2FwoGdy9Y/Zmc9BOowAx/frGlZMdf9np48O+Abjk+HSQ8te8uSbkGaO+iXzsKte 5kwzt6k+5tFWim+CNFlXXNaVBlBDWjqVnLWP+hMDO9SIN+dnSIgvcfkYQUAyfOB42RJ+ gnVrEAThIhugh/52odDRt13phaIvxB1EuFKBb2MgPPMXJDRyW7wjwfkJnPK/O46Cls14 kti4++RxaaPlgzg9f/fpqZUZf4UJe/oCupqUIDVWRpDygUJokX5Sf1I+q/Rq2nmvNSE+ rlfQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584812; x=1729189612; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=L6S7ecEpHeP2pNYK74CcyfNjVxKIOVPI1u0giMU3AdU=; b=JugU28e4gMHOT3wDI2eSlTiUcC8Axm++9sMWtgqqgBbuQfZV3QNSdGWQ6fyUDno/eE es58pBykNR3RsN385sifzGCG2lCKwnsGKvNADIebtoV8RYlNB2y4Kz2j7KMBN9LFFQxz PK/tGX037lGpBG8yJMk3WYlXkSDxsH3f5lo9RuIFbvjSzq20SOVJYRTquGBogEu0OlUB 6pO4MTzZobR7abD+46rL/9GjtIxNjwLDGTO15Gft3GJrH6rt8i/XcfXXWDs1o14yh3x2 xCdXbdHFJUUNqX9i9f7CLFQhYqvY0lYJcV831V5R8YcY0T7HXxObrDRXBuYnAlif97Rc fUOQ== X-Forwarded-Encrypted: i=1; AJvYcCXqPctA7FY6cfWc0h0cfJBqrGXhZgkgtn+zmwfd7sa2V9MDW5ErjUpkp6+npNNx6v1SHf9JCcYfiYv+aIDORWct@lists.infradead.org X-Gm-Message-State: AOJu0Yz5ymk71MoNdZzhznH1cbWlaiYxM21NSzh/We6V2fk98LhAN56B NFDXmynBVoDKNah+LT3hF6Rev5aQQepfXsOXUyq97WrDC/M0hsQOAOFseAYARD1r6bFC+b9fk9k eHA== X-Google-Smtp-Source: AGHT+IFNqFi+zU38KeaNCnXXdZA3xW66mqPsBP1LkkibZO//JxtU2HWKMaWXgt9tHH1pSDOFY+ZWY6Gk4GE= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:90b:4f4b:b0:2d8:bf47:947c with SMTP id 98e67ed59e1d1-2e2f0c456f8mr35a91.3.1728584812033; Thu, 10 Oct 2024 11:26:52 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:58 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-57-seanjc@google.com> Subject: [PATCH v13 56/85] KVM: arm64: Use __kvm_faultin_pfn() to handle memory aborts From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112654_113387_11702C97 X-CRM114-Status: GOOD ( 12.62 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Convert arm64 to use __kvm_faultin_pfn()+kvm_release_faultin_page(). Three down, six to go. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/arm64/kvm/mmu.c | 15 ++++++--------- 1 file changed, 6 insertions(+), 9 deletions(-) diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c index ecc6c2b56c43..4054356c9712 100644 --- a/arch/arm64/kvm/mmu.c +++ b/arch/arm64/kvm/mmu.c @@ -1439,6 +1439,7 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, long vma_pagesize, fault_granule; enum kvm_pgtable_prot prot = KVM_PGTABLE_PROT_R; struct kvm_pgtable *pgt; + struct page *page; if (fault_is_perm) fault_granule = kvm_vcpu_trap_get_perm_fault_granule(vcpu); @@ -1560,7 +1561,7 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, /* * Read mmu_invalidate_seq so that KVM can detect if the results of - * vma_lookup() or __gfn_to_pfn_memslot() become stale prior to + * vma_lookup() or __kvm_faultin_pfn() become stale prior to * acquiring kvm->mmu_lock. * * Rely on mmap_read_unlock() for an implicit smp_rmb(), which pairs @@ -1569,8 +1570,8 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, mmu_seq = vcpu->kvm->mmu_invalidate_seq; mmap_read_unlock(current->mm); - pfn = __gfn_to_pfn_memslot(memslot, gfn, false, NULL, - write_fault, &writable); + pfn = __kvm_faultin_pfn(memslot, gfn, write_fault ? FOLL_WRITE : 0, + &writable, &page); if (pfn == KVM_PFN_ERR_HWPOISON) { kvm_send_hwpoison_signal(hva, vma_shift); return 0; @@ -1583,7 +1584,7 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, * If the page was identified as device early by looking at * the VMA flags, vma_pagesize is already representing the * largest quantity we can map. If instead it was mapped - * via gfn_to_pfn_prot(), vma_pagesize is set to PAGE_SIZE + * via __kvm_faultin_pfn(), vma_pagesize is set to PAGE_SIZE * and must not be upgraded. * * In both cases, we don't let transparent_hugepage_adjust() @@ -1692,11 +1693,7 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, } out_unlock: - if (writable && !ret) - kvm_release_pfn_dirty(pfn); - else - kvm_release_pfn_clean(pfn); - + kvm_release_faultin_page(kvm, page, !!ret, writable); read_unlock(&kvm->mmu_lock); /* Mark the page dirty only if the fault is handled successfully */ From patchwork Thu Oct 10 18:23:59 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830927 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9C4D7CFC5FE for ; Thu, 10 Oct 2024 19:28:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Type:Cc:To: From:Subject:Message-ID:References:Mime-Version:In-Reply-To:Date: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=9PLNR7J7/StxhsadOj2FvapRbIqIfrDfQ6nFmhEUvk8=; b=PttS/3pYo5/p+36rC10DaRBBqx ESnF39wE6BDWuWX+5RoZMxNoNzpMYsMqWpbta4uVK9ILaGoKitKnL1WxKNmvgbPxr/6R8iusq+gNv jraJnQE99oLgBTJBk9jmULlicQYrFX83OgvqHUXMeS98edLAvsP3d203R0JP/3+XYdcy9MlVMx+dp acai9vijyzLtuLrxKJ+BS5IlGSHnmmnbjn0hDQ6cTAXd55Kh8UBTf8iBsbDqlyEnSzKoQMv8Pwc8V huLkOEQfEye84suUxd4LlQ00e91SUAV9ftXxOUyLH1sQ68WXZhHo+6jxesFF6jtSQpFljEUxwsB3k tQtVvppA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syypz-0000000E6kX-3Ttq; Thu, 10 Oct 2024 19:28:39 +0000 Received: from mail-yw1-x1149.google.com ([2607:f8b0:4864:20::1149]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxsF-0000000DrDY-3LSa for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:26:58 +0000 Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-6e2e4874925so21257117b3.1 for ; Thu, 10 Oct 2024 11:26:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584814; x=1729189614; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=9PLNR7J7/StxhsadOj2FvapRbIqIfrDfQ6nFmhEUvk8=; b=oxXZstpB7bZForNibrdtUdgMDhKCKO5NoyWz/FZhmXi1O/5zBMkdNGoOBmZUnuHPL5 G5xj1yNE8SP9DIK/5R+FSLX1z3xFvSK7JT9CV/kxO/TU+0fFWjpeJqmMwLlYFO0Q+Tv4 I858qQIY6Wc0qtONrNlC8gbObQgUKRGiuFgn7O1Kkb3dzs2Qp7EmMuvNXtmlilk77dG6 00Dp5QMLKTvno0LnGfKXj09dJixXH9wsYC90KdPrw/8w/aMV7XGhbAVdXGuP5cSdmKEJ /2emmDAUPw3bLmVyf1eaRDVyfGHUrBZvZno+DFT7U8iIof4e8MNFX2e2pMznO8WrlsKE bLEQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584814; x=1729189614; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=9PLNR7J7/StxhsadOj2FvapRbIqIfrDfQ6nFmhEUvk8=; b=xRNs+Aw4qNshcxWU5/RTmjvs8AD0sgoL1GXOyPrT3ORtfp2kDOuTknRo5NURriSt82 lcHc5m7N3C7kb2YWW8DPVQuBzROZq3tLhBKrZdPNJuHF+5bztfHeJd8cW7EVNQzdqnFg edh/1JvO371brqwgIKv+w3l2E/0UlbZNcEs370kPvw3/LW7gFZ4zR3T27qroGhWzbKp4 kiLMoZ3rVoCjA8Iz19cD35SCePjAlFJcPqEZOTqfELFGtue0jPbpcoXdtfdk0UC68+h2 K205oVzFcLeHLwkp+FxclAjT0u928mVUvWuEyPhJtFv5jL9lLYl2/xgMgy560s8QS2cB ctmQ== X-Forwarded-Encrypted: i=1; AJvYcCUHGdLcBGI7YCINb1GXgzM9eMT0M2Ho3CqsBhLMpoLll0WtWirkJfRZaRmqAUjC+12Nsp+7DJ9SI3KEdJCEYR38@lists.infradead.org X-Gm-Message-State: AOJu0YyIEdCSgfdUUQk1HXQDAxoU2LwTYqiVxsKPuEa94HhjSXN5iflL vgV+PvCOvch5Q5Hr1uOdZ8KaXgTfT9EUm5gsAWHYXZ9O506XKkpTvuG0e6CW0aDYTtJ3WlMp2FY N/g== X-Google-Smtp-Source: AGHT+IFTz5/NgoL1lG2XSkf5pquU6CJl4M8l3x57mh4U7FbzzS84rW+oHIk+OXb87gKu92fZB4cgNqwOitE= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:690c:46c3:b0:6e3:21cf:a67f with SMTP id 00721157ae682-6e32242fb5emr1006557b3.7.1728584814138; Thu, 10 Oct 2024 11:26:54 -0700 (PDT) Date: Thu, 10 Oct 2024 11:23:59 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-58-seanjc@google.com> Subject: [PATCH v13 57/85] KVM: RISC-V: Mark "struct page" pfns dirty iff a stage-2 PTE is installed From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112655_909064_6B8BA079 X-CRM114-Status: GOOD ( 11.90 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Don't mark pages dirty if KVM bails from the page fault handler without installing a stage-2 mapping, i.e. if the page is guaranteed to not be written by the guest. In addition to being a (very) minor fix, this paves the way for converting RISC-V to use kvm_release_faultin_page(). Reviewed-by: Andrew Jones Acked-by: Anup Patel Signed-off-by: Sean Christopherson --- arch/riscv/kvm/mmu.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/arch/riscv/kvm/mmu.c b/arch/riscv/kvm/mmu.c index b63650f9b966..06aa5a0d056d 100644 --- a/arch/riscv/kvm/mmu.c +++ b/arch/riscv/kvm/mmu.c @@ -669,7 +669,6 @@ int kvm_riscv_gstage_map(struct kvm_vcpu *vcpu, goto out_unlock; if (writable) { - kvm_set_pfn_dirty(hfn); mark_page_dirty(kvm, gfn); ret = gstage_map_page(kvm, pcache, gpa, hfn << PAGE_SHIFT, vma_pagesize, false, true); @@ -682,6 +681,9 @@ int kvm_riscv_gstage_map(struct kvm_vcpu *vcpu, kvm_err("Failed to map in G-stage\n"); out_unlock: + if ((!ret || ret == -EEXIST) && writable) + kvm_set_pfn_dirty(hfn); + spin_unlock(&kvm->mmu_lock); kvm_set_pfn_accessed(hfn); kvm_release_pfn_clean(hfn); From patchwork Thu Oct 10 18:24:00 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830929 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9F409CFC5FF for ; Thu, 10 Oct 2024 19:30:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Type:Cc:To: From:Subject:Message-ID:References:Mime-Version:In-Reply-To:Date: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=h7pIaRprT8i6iodquNsVL5vPZSNT9KsZLdIQsfz/EL8=; b=u1It9FbLyvXVxf+P41QCI8p0HJ MQbpIK9EP2A/NpzmCsjz9onn8aRrG/K08fsquvGRtQ5STBp7FWymffDSKVJQ3lg50YuGKEB6vdhEd KN96HSriRvOeCRutuSDPaMYfvOxpRvhmoS5racMFHcmAGg0uPRQLxcFaMt1Ic187L/b6KjEHUe8bz mKgME1V0Q8IJ+EYwGnpIH5YJFmjJiTgQNsJAJcz++uvk9IRQQ86RkVUfkNndkSmIzpixc0INO3R1L bAoIR/ewAaZf61Lvf8DGPfX2E0WnwzRz3D3fPUzavrvxJjfLhC4/sM1p0ZB4IdwWKGWQ+bDeRgQrR ZOlhHvnw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syyrK-0000000E6zS-0en5; Thu, 10 Oct 2024 19:30:02 +0000 Received: from mail-yb1-xb49.google.com ([2607:f8b0:4864:20::b49]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxsH-0000000DrFe-2CUq for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:27:00 +0000 Received: by mail-yb1-xb49.google.com with SMTP id 3f1490d57ef6-e02b5792baaso1676937276.2 for ; Thu, 10 Oct 2024 11:26:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584816; x=1729189616; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=h7pIaRprT8i6iodquNsVL5vPZSNT9KsZLdIQsfz/EL8=; b=SlOntVtK7MfXmxpZt9qlDfPDhXBRuKbOOqW7IqQ0P93vwPs1FWZo8d/fPJW4iNnsl9 WaFEojLUv/xsVJmZwP672jadiAn28zabsdyG4luGg8vBEBqdfYs+GmhJTKS7MdDeavEu IIS/1rUscUKoFp5AiL5O+Hix5GdZV6jM1pdKwvb8MlIpTYXGrXcWMrPmxpoPEpHJVb9n zpWK/On5Nqt1OS/n+NtmVJUQ9bqeF9a8ZoOJNBl3NwrmW1rSb5CDX6JCZIt8XI8kpm9c 5W0lnYFPhUi4nG7xC4RgfzO65Hjb/3a1HK1I7N7Vy96v150QVnYn24O+JGrFNRZCzfq8 7meQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584816; x=1729189616; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=h7pIaRprT8i6iodquNsVL5vPZSNT9KsZLdIQsfz/EL8=; b=JnlSU8tCNOiANxfcBnaauoSi2I/pK9JNk6yYjlZGXAC8CWxoZxtlc2c4PKYEDD9oTP q/bw5MdwYsuOrsz+1SNJ2V7dAYaiI9lo9w5W/ViqzXb5pSv2yqAfo9eI1FDmSnSPPL2U vF3DK4CM9mJmEFYGYtMCLYzMWt+8qLzSY2zQQZt3LFrztvqLUa89laD7vEB0rKM6X7tp XfzOWenH42Akm1H4gKmfAhkutCFfAvVKOzQcO0xtoLoiD4MPKXRLiVKAtQGTmJEX4oaL rToBpQPm8X3gd2AzI62fuSd7MRvJHw5QqFIZgLM8CxjHlPiR+cucsGTyRFXyl/ltXI8Q wGfA== X-Forwarded-Encrypted: i=1; AJvYcCVR+RULSVtE3oRl0xxWldxfRCBB6l/skttIR3KJYwFytkdhq/22lWTUOOz+hzGrE9vXKg8itAwrYm5u5jOZPRLc@lists.infradead.org X-Gm-Message-State: AOJu0YzNZVls3kFxhDBtjA9tojT2K8b/YynSSrYG0cd4CeBVvmQCJBYs UVKjLxLs79flPGnyivJ98rHkAILi19H5rOTwo6cJmNm3lf76hyUT0RbhUV6Lq7U4tae1maw9fim Hew== X-Google-Smtp-Source: AGHT+IER2ThMFqRrPlrH2YqE2Mm6yZZ3YtFM0VnOojG5BgkcM2HHrRnMjpl0ZvMfhTqewlD6nF2BR0rNKwA= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a5b:c45:0:b0:e0b:f69b:da30 with SMTP id 3f1490d57ef6-e28fe41d0e6mr89144276.9.1728584816073; Thu, 10 Oct 2024 11:26:56 -0700 (PDT) Date: Thu, 10 Oct 2024 11:24:00 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-59-seanjc@google.com> Subject: [PATCH v13 58/85] KVM: RISC-V: Mark "struct page" pfns accessed before dropping mmu_lock From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112657_665069_D28300FA X-CRM114-Status: GOOD ( 11.20 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Mark pages accessed before dropping mmu_lock when faulting in guest memory so that RISC-V can convert to kvm_release_faultin_page() without tripping its lockdep assertion on mmu_lock being held. Marking pages accessed outside of mmu_lock is ok (not great, but safe), but marking pages _dirty_ outside of mmu_lock can make filesystems unhappy (see the link below). Do both under mmu_lock to minimize the chances of doing the wrong thing in the future. Link: https://lore.kernel.org/all/cover.1683044162.git.lstoakes@gmail.com Reviewed-by: Andrew Jones Acked-by: Anup Patel Signed-off-by: Sean Christopherson --- arch/riscv/kvm/mmu.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/arch/riscv/kvm/mmu.c b/arch/riscv/kvm/mmu.c index 06aa5a0d056d..2e9aee518142 100644 --- a/arch/riscv/kvm/mmu.c +++ b/arch/riscv/kvm/mmu.c @@ -682,11 +682,11 @@ int kvm_riscv_gstage_map(struct kvm_vcpu *vcpu, out_unlock: if ((!ret || ret == -EEXIST) && writable) - kvm_set_pfn_dirty(hfn); + kvm_release_pfn_dirty(hfn); + else + kvm_release_pfn_clean(hfn); spin_unlock(&kvm->mmu_lock); - kvm_set_pfn_accessed(hfn); - kvm_release_pfn_clean(hfn); return ret; } From patchwork Thu Oct 10 18:24:01 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830930 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 796C0CFC5FE for ; Thu, 10 Oct 2024 19:31:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Type:Cc:To: From:Subject:Message-ID:References:Mime-Version:In-Reply-To:Date: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=OPPvMTYiUGb6pAXW02Q14XiUF2+KvArosM2sRK6VjAA=; b=Z2iIqyCRLvEfG5E+OYp39IG3Xn bxfOLEIRAFYcITw/FoC/yrcbT6k1LV0TjFaNG26Zm26gjd7Tul9ulxe0dFokBH3A8VFd4LOB8jRU8 4OBoik7PCqK5zK9nKwldWWp2ESzT31mh+kDsE1guMQQjYJBiT9hkr0UDw1ObZHcaH8BKaFnhjeYJA DVEc0iSdjGddwLdRohcHoDG/CARIrEwfq2uB56gkTWKBI2w52wwxmoG7ms0T3MCCXmn3kSS6ZS1YS WWQq8l9lFWsAB8OFzEuRFfPmWgRidiw5gZsL/H508AqVHkzmGxZfu4WrX6iE09IcNvlAy0P/zIXit msn0tDFw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syyse-0000000E7Dk-0y47; Thu, 10 Oct 2024 19:31:24 +0000 Received: from mail-pg1-x549.google.com ([2607:f8b0:4864:20::549]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxsJ-0000000DrHJ-17c0 for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:27:02 +0000 Received: by mail-pg1-x549.google.com with SMTP id 41be03b00d2f7-7e6c40a5795so1041173a12.3 for ; Thu, 10 Oct 2024 11:26:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584818; x=1729189618; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=OPPvMTYiUGb6pAXW02Q14XiUF2+KvArosM2sRK6VjAA=; b=jHZE+5uWuKTW/XclrN7CxgU01R867qap0OT1d1SQAxMutGogOns2m7h4OczE0MMOAh 1bOC4OAreKrsxWwPn6Lq4yUn6ww5VGN/P/6dCQu7Wz1aJXS6RA5MlDKU8blhQdAK3+oJ w7Hui36VpO0Ua+GpakXkDzGrrZtAOQ4i+bpjS2LZB15ep1oIi954CEVd8VSCUEpuTEmH uMVRVR+qn2o1f5I/7taWb50bvQjkE/GagwpScYZSMNiGtslKvTiZXEYVCdh7kpLhPoVU OeNf3Txm2Xav5Rvn23+I0GhmOXaelr06WRlnGWvEKKd7xtGeetms/WSDotPLATORh2a1 urMQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584818; x=1729189618; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=OPPvMTYiUGb6pAXW02Q14XiUF2+KvArosM2sRK6VjAA=; b=VFI+42+49rzon9N9oyctzW9FjM+uU8ljcPUjy+oc514DVKjup51T0MrYj3Bt3QbXvD EeGlvjzduyb3zmA0LQW0RIt1ziOFUAImyTRbOR1so1ckraKLfQ7ifCOcXXL/31VAH+rx jWZnsDKulSuw/ymgvkxG8VVXPp1vznqdunsTUXBno/vr8JwBlT/aPwDWLRaguaZEvRs+ tEJTwm9iP/FH0eidQJXfCSnRBvuAEIqmGy+fnOaownBSjRIU53wNs53VkKdHgw9W9dEP Lwd3r0ihGzUHT3nvp4iOLgWRbQtFWf7cdVQUX6bG2IXhXDixUqEp3b1mGq6Fiqe+ZzXi UWOA== X-Forwarded-Encrypted: i=1; AJvYcCUopsP68u+eLV0O5aAyYwTTzYy3DirlyUeOQ6eC/Dl9SAMN7bQU20r4H4i+MmCr8u2R4cyn+7pe82Ajd0EgI3Au@lists.infradead.org X-Gm-Message-State: AOJu0YwvrJHb99IDrOjVyDnoZ8Kb9pxQsxrz4/tYPMT96phD66HUJ+O0 TMGfltR0S00h8Mz5SmZu9W3+MuNkjlFq9Mc2Dxuv7tqOb5Q3ST/xNx+fqYgXEEffqfL0c9cOzQl M7g== X-Google-Smtp-Source: AGHT+IHVZkXcNH8eDb3GIQubykYaXKWHdrpGe/zHTLQHIjPMwQpAN0ra/QncFunLfkL/hgYZuPxk0qgLkH8= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:903:2343:b0:20c:747d:ec15 with SMTP id d9443c01a7336-20c747df618mr290175ad.1.1728584817934; Thu, 10 Oct 2024 11:26:57 -0700 (PDT) Date: Thu, 10 Oct 2024 11:24:01 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-60-seanjc@google.com> Subject: [PATCH v13 59/85] KVM: RISC-V: Use kvm_faultin_pfn() when mapping pfns into the guest From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112659_428112_68420971 X-CRM114-Status: GOOD ( 11.55 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Convert RISC-V to __kvm_faultin_pfn()+kvm_release_faultin_page(), which are new APIs to consolidate arch code and provide consistent behavior across all KVM architectures. Opportunisticaly fix a s/priort/prior typo in the related comment. Reviewed-by: Andrew Jones Acked-by: Anup Patel Signed-off-by: Sean Christopherson --- arch/riscv/kvm/mmu.c | 11 ++++------- 1 file changed, 4 insertions(+), 7 deletions(-) diff --git a/arch/riscv/kvm/mmu.c b/arch/riscv/kvm/mmu.c index 2e9aee518142..e11ad1b616f3 100644 --- a/arch/riscv/kvm/mmu.c +++ b/arch/riscv/kvm/mmu.c @@ -601,6 +601,7 @@ int kvm_riscv_gstage_map(struct kvm_vcpu *vcpu, bool logging = (memslot->dirty_bitmap && !(memslot->flags & KVM_MEM_READONLY)) ? true : false; unsigned long vma_pagesize, mmu_seq; + struct page *page; /* We need minimum second+third level pages */ ret = kvm_mmu_topup_memory_cache(pcache, gstage_pgd_levels); @@ -631,7 +632,7 @@ int kvm_riscv_gstage_map(struct kvm_vcpu *vcpu, /* * Read mmu_invalidate_seq so that KVM can detect if the results of - * vma_lookup() or gfn_to_pfn_prot() become stale priort to acquiring + * vma_lookup() or __kvm_faultin_pfn() become stale prior to acquiring * kvm->mmu_lock. * * Rely on mmap_read_unlock() for an implicit smp_rmb(), which pairs @@ -647,7 +648,7 @@ int kvm_riscv_gstage_map(struct kvm_vcpu *vcpu, return -EFAULT; } - hfn = gfn_to_pfn_prot(kvm, gfn, is_write, &writable); + hfn = kvm_faultin_pfn(vcpu, gfn, is_write, &writable, &page); if (hfn == KVM_PFN_ERR_HWPOISON) { send_sig_mceerr(BUS_MCEERR_AR, (void __user *)hva, vma_pageshift, current); @@ -681,11 +682,7 @@ int kvm_riscv_gstage_map(struct kvm_vcpu *vcpu, kvm_err("Failed to map in G-stage\n"); out_unlock: - if ((!ret || ret == -EEXIST) && writable) - kvm_release_pfn_dirty(hfn); - else - kvm_release_pfn_clean(hfn); - + kvm_release_faultin_page(kvm, page, ret && ret != -EEXIST, writable); spin_unlock(&kvm->mmu_lock); return ret; } From patchwork Thu Oct 10 18:24:02 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830950 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 98CFECFC60D for ; Thu, 10 Oct 2024 19:35:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Type:Cc:To: From:Subject:Message-ID:References:Mime-Version:In-Reply-To:Date: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=SD6mqWUZeYJtl6GnT7U0LdjoV3gPIs/AdF7iNpavyUo=; b=2sCeeXcxbnN6+TdONR+vD6yXIf lZpcHYcQIurj04wgdbQF4LdJBjBa3TLEfTrF+WhsVOFN4xTg1030kxbsAeHXlJbrblvjIN1qw99nq YVQUKO2eVyyVbdP8NtZHVqwpjXGAX146bNXD8u0GM6Cepx2Bj2cUaPc0GiEh0vc/beQpRkLtEh1lf /6xwMVjtARjfLOBjlzVHyLSP8urIwqaP8QJHLtyRRv/7e04AYuMB+rdJ+VZm44S0jxrXP7zr0rV1w y1/4A3yWptbJB+W2tOm/OthrXQ0RG2DXcG45xQDxAOFkX6hpJ3PoohpoHWY6l8TpnE2ukL3ZWsOVn C/OT2EGw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syywd-0000000E7rp-1WCi; Thu, 10 Oct 2024 19:35:31 +0000 Received: from desiato.infradead.org ([2001:8b0:10b:1:d65d:64ff:fe57:4e05]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxsR-0000000DrNl-347Y for linux-arm-kernel@bombadil.infradead.org; Thu, 10 Oct 2024 18:27:07 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Content-Type:Cc:To:From:Subject: Message-ID:References:Mime-Version:In-Reply-To:Date:Reply-To:Sender: Content-Transfer-Encoding:Content-ID:Content-Description; bh=SD6mqWUZeYJtl6GnT7U0LdjoV3gPIs/AdF7iNpavyUo=; b=DwzYVeNZ/acQo4ULb5dSveePRL eDM6PNN6Skkxps2sQ5O+U9eoZ41zciyPZD8NaS/bo+9xTCuWFpTEvxjIa14wzt8vODwLvTfdJgRY9 8bCvJ9wKXDepebIcjb8SvqmLr95TTtN9VI8eg7roGfGoqNfsHZNCCS2n7q15kZ0gwbdNkSU8hQrPF oX3lF2GFRc/d/CAX5/JVYgBqJs9m+PfWKxpmh8H+cf7aAEF4Z9fAbteABYeahWPYiZYy6DoCwrjFk JoK9xaBQ8x1gUZNPGeUFd1pmGvGx47CuR5yJ8NRx4wjsUwr48NfH5qI9CFlcI0vJj6XAHtmSJpVe2 Wv/XZZ4Q==; Received: from mail-yb1-xb4a.google.com ([2607:f8b0:4864:20::b4a]) by desiato.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxsO-00000005UXd-0gjz for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:27:06 +0000 Received: by mail-yb1-xb4a.google.com with SMTP id 3f1490d57ef6-e2904d0cad0so2052465276.1 for ; Thu, 10 Oct 2024 11:27:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584820; x=1729189620; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=SD6mqWUZeYJtl6GnT7U0LdjoV3gPIs/AdF7iNpavyUo=; b=aBDkusJdX7g5BtG4H5fZwYumgK8uIRaVDx6oulzlDnAE3QYd/mM7atYX1oex5UM4kp 36HYJ8PnDzc4AfK9p/87+lpLgfHVh0ndquokKS44VTbMF2Oucv/TNfFgaSF5koQ84Xx+ eB1oBqtktOPbKyCDITEKMRpMt/zheIQaROymC4XkoClTL+P0IlaWL4HGnExqhsEfvMNX Grcu4zqEHuWTwdwnkzgTXcXDWS+EhrV5yrg6rh0PIUpcmQja2DfSNNDcjEgoXjjL+DOb 4f1UgnylDz6N1QWZCTJX94ziWXvHKu5BciXd0kjBuyWLffYTsruMTc2z38kb5Ir5V3zW suxg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584820; x=1729189620; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=SD6mqWUZeYJtl6GnT7U0LdjoV3gPIs/AdF7iNpavyUo=; b=dBTTjmCBMTWfJ2ILYKwyNRaSPZHNavRjVkopUn+aK+BvxCRm4qhKDmt03YPBD2O80P g8F/FSURsHIrtcFluSdFU5OEHnEy5PvusZg0ZR50oj0kzfhqvyg7w0xsdue8NSTANzH7 m29PPuNbUxLSbi+Z8ChGQ9UTVl6tyFS+8jsafglGynAonZm3Hf8YIA/hlIw15WJCpeIN jjNvXYz0YdS0E6JRuHHorF5MoP42dSPFX0gxJYOG85WAprDxa4N/KBChqePil48ybhem PZk5IFEfNRG+Gls+hRYP0WTZ3tA01yCeQdDP4yO04WpqSkhdUTV1504aRfVspq/IIUMC OSig== X-Forwarded-Encrypted: i=1; AJvYcCX3HZoU76ETojqQlGcgTt2fZGLOjLyFzaRQ30zZCKeZdn+H7spsKdr5wdgouD/Qid+ZUBTW9E/ug0OW7mM2H+gw@lists.infradead.org X-Gm-Message-State: AOJu0YwHwA0q/vKYp3WoBDMRJm1lUeE8SjYKjioLLKhD8eeMuznJLSUS JbsmDW5zKt6U7Aagkfmrh6kit6+o797AZYh999rvt9sD8iKGuLmCjAQW5avK/+QLt3DoEd9UCOu m+Q== X-Google-Smtp-Source: AGHT+IGRQvxFZI4ySDHPZ8aV1dJi8vCYkQEt4dLiT8ndc3AvxflLIddldJBCv79vdNFr7/DaBU5iasgDdOk= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a25:ce11:0:b0:e28:fb96:d0f4 with SMTP id 3f1490d57ef6-e28fe33505emr40781276.2.1728584819828; Thu, 10 Oct 2024 11:26:59 -0700 (PDT) Date: Thu, 10 Oct 2024 11:24:02 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-61-seanjc@google.com> Subject: [PATCH v13 60/85] KVM: PPC: Use __kvm_faultin_pfn() to handle page faults on Book3s HV From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_192704_393566_6CE8178B X-CRM114-Status: GOOD ( 13.94 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Replace Book3s HV's homebrewed fault-in logic with __kvm_faultin_pfn(), which functionally does pretty much the exact same thing. Note, when the code was written, KVM indeed didn't do fast GUP without "!atomic && !async", but that has long since changed (KVM tries fast GUP for all writable mappings). Signed-off-by: Sean Christopherson --- arch/powerpc/kvm/book3s_64_mmu_hv.c | 25 ++++--------------------- 1 file changed, 4 insertions(+), 21 deletions(-) diff --git a/arch/powerpc/kvm/book3s_64_mmu_hv.c b/arch/powerpc/kvm/book3s_64_mmu_hv.c index 2f1d58984b41..f305395cf26e 100644 --- a/arch/powerpc/kvm/book3s_64_mmu_hv.c +++ b/arch/powerpc/kvm/book3s_64_mmu_hv.c @@ -603,27 +603,10 @@ int kvmppc_book3s_hv_page_fault(struct kvm_vcpu *vcpu, write_ok = writing; hva = gfn_to_hva_memslot(memslot, gfn); - /* - * Do a fast check first, since __gfn_to_pfn_memslot doesn't - * do it with !atomic && !async, which is how we call it. - * We always ask for write permission since the common case - * is that the page is writable. - */ - if (get_user_page_fast_only(hva, FOLL_WRITE, &page)) { - write_ok = true; - } else { - /* Call KVM generic code to do the slow-path check */ - pfn = __gfn_to_pfn_memslot(memslot, gfn, false, NULL, - writing, &write_ok); - if (is_error_noslot_pfn(pfn)) - return -EFAULT; - page = NULL; - if (pfn_valid(pfn)) { - page = pfn_to_page(pfn); - if (PageReserved(page)) - page = NULL; - } - } + pfn = __kvm_faultin_pfn(memslot, gfn, writing ? FOLL_WRITE : 0, + &write_ok, &page); + if (is_error_noslot_pfn(pfn)) + return -EFAULT; /* * Read the PTE from the process' radix tree and use that From patchwork Thu Oct 10 18:24:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831202 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5C880D24454 for ; Thu, 10 Oct 2024 21:46:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Type:Cc:To: From:Subject:Message-ID:References:Mime-Version:In-Reply-To:Date: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=YGpJ+GsXH4oG8p6vIrNjfshF8svSSAFECmaA34wp4no=; b=nnpXAIH+IrvayLFxONCJ+OTMW9 XY6+/5qMKzaUFe0GLF9G8kLU5bsbBcbRrhxntlZhSg/lkSUUo0qdyLWyhL04PCbS0lUbmnC8xF/Cx DoiV77dNmI7loJ52XhY84k6HKYkMQgFiOje0sJWK1kg+wiO3TK7lX7adhPPE6ZIArugCU505TEfXq nuuRMELOhH26k7TtjFSOD3AEE/rcuBv4a+ptijUD/iSeVyerUu0Q92yo5JJxRy4avNOd7OpkSeI4T yyDfpe+pBFXbH7cYNHDqusMgdFN/LT5sgEVSXKE47FariHmRhJB6gGZn0dKmeE8ZDWx2R5iDxFOA3 9XGLz3RA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1sz0zP-0000000ESWn-0Z59; Thu, 10 Oct 2024 21:46:31 +0000 Received: from mail-yb1-xb4a.google.com ([2607:f8b0:4864:20::b4a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxsN-0000000DrKL-0mQc for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:27:04 +0000 Received: by mail-yb1-xb4a.google.com with SMTP id 3f1490d57ef6-e290b8b69f8so2139785276.2 for ; Thu, 10 Oct 2024 11:27:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584822; x=1729189622; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=YGpJ+GsXH4oG8p6vIrNjfshF8svSSAFECmaA34wp4no=; b=AFXrQf91aXR1KTzaINYKc11bKcpAL/FBJRU5jqINr+Nhzi0EHvlki0oC9dCnjEGnWx 0zVm9mSBj2AcgW4LR34T//6/HY2HcASslxa2iNw7wwXCbTgZ8oxdMI635Z5mVbucf5i8 v2DE+6sqEAB9v5KGBcqRVR6xzDXFBq5pkHblKYkHjr3+BtsbZCDKCmyrCZD1LW0IdcUM 1pD9rayn4EnfsTjku1tz91kdoDjX74SAWAwF9d2bm028TG3RB/P064MIKeodw8ez2JK+ zBjIC/kVsK1IkoUFRaArozf068vnlLw74GKO4rSiTlL8f/D9lFwS307IYBCFbQOTHiVU JxKQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584822; x=1729189622; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=YGpJ+GsXH4oG8p6vIrNjfshF8svSSAFECmaA34wp4no=; b=YP4VugjGqHtmVFjHqnNg/j4/qLigv4PmtgkYQ57LIZvWQHf9LSFONWD09hBpW29Z1J JLANIuymR6Bsq4pMFXsLWU4LiM4wAOizu+9dPZO1z082CWFWz7yRwvFZvIi8dFCM0nJB JGkzhtEEQ2oaxuKbhDAJEv1p0QvqL7vSX7CXi9yU5eE6ahoOgX/ZtwF6w3fw7zFJJ9T7 J/YcE1MmuiSlkeiBY31YzTmg3Ti+eO/Dx3IWsUy4YEwtISGDxb9y6vj3fdIMIuCuqWsD F1jkJIwX2qOKp79R/SCHu0A3DnxhnTcWfoZqeUeVfXOwj+lz7iz/qh+rcNvMF4u854/8 nFFA== X-Forwarded-Encrypted: i=1; AJvYcCVK2paEqcncDClXTVgrp7T1BNuSatImlm/duR/7C1tk4lRFJHBYlsKWKAxeLdRhp/u3ozxfra+gPRasvJ34vvT2@lists.infradead.org X-Gm-Message-State: AOJu0Yxw7rxmBeDMNlQO3EC+xZtga2cNK1xosfo4w1ztEx+ty9CXjwSd GcNS+LYQnUkTBE3IkdIeImHeB/q1Eig5LhUszFzhdsCrcHQLGytja+pzDXSPTdeznsekt3Vdgu/ k5Q== X-Google-Smtp-Source: AGHT+IFyzgxXPfejJoY0SZCQ3T+Vo8hzb1HJlB5l94FFDfZID39IeT9MHN/ZgR4beqIvxQ/gO8qC3c4Qafk= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:6902:4ac:b0:e28:fdfc:b788 with SMTP id 3f1490d57ef6-e28fe4426b8mr4979276.9.1728584821558; Thu, 10 Oct 2024 11:27:01 -0700 (PDT) Date: Thu, 10 Oct 2024 11:24:03 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-62-seanjc@google.com> Subject: [PATCH v13 61/85] KVM: PPC: Use __kvm_faultin_pfn() to handle page faults on Book3s Radix From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112703_362588_949BCFE3 X-CRM114-Status: GOOD ( 14.54 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Replace Book3s Radix's homebrewed (read: copy+pasted) fault-in logic with __kvm_faultin_pfn(), which functionally does pretty much the exact same thing. Note, when the code was written, KVM indeed didn't do fast GUP without "!atomic && !async", but that has long since changed (KVM tries fast GUP for all writable mappings). Signed-off-by: Sean Christopherson --- arch/powerpc/kvm/book3s_64_mmu_radix.c | 29 +++++--------------------- 1 file changed, 5 insertions(+), 24 deletions(-) diff --git a/arch/powerpc/kvm/book3s_64_mmu_radix.c b/arch/powerpc/kvm/book3s_64_mmu_radix.c index 8304b6f8fe45..14891d0a3b73 100644 --- a/arch/powerpc/kvm/book3s_64_mmu_radix.c +++ b/arch/powerpc/kvm/book3s_64_mmu_radix.c @@ -829,40 +829,21 @@ int kvmppc_book3s_instantiate_page(struct kvm_vcpu *vcpu, unsigned long mmu_seq; unsigned long hva, gfn = gpa >> PAGE_SHIFT; bool upgrade_write = false; - bool *upgrade_p = &upgrade_write; pte_t pte, *ptep; unsigned int shift, level; int ret; bool large_enable; + kvm_pfn_t pfn; /* used to check for invalidations in progress */ mmu_seq = kvm->mmu_invalidate_seq; smp_rmb(); - /* - * Do a fast check first, since __gfn_to_pfn_memslot doesn't - * do it with !atomic && !async, which is how we call it. - * We always ask for write permission since the common case - * is that the page is writable. - */ hva = gfn_to_hva_memslot(memslot, gfn); - if (!kvm_ro && get_user_page_fast_only(hva, FOLL_WRITE, &page)) { - upgrade_write = true; - } else { - unsigned long pfn; - - /* Call KVM generic code to do the slow-path check */ - pfn = __gfn_to_pfn_memslot(memslot, gfn, false, NULL, - writing, upgrade_p); - if (is_error_noslot_pfn(pfn)) - return -EFAULT; - page = NULL; - if (pfn_valid(pfn)) { - page = pfn_to_page(pfn); - if (PageReserved(page)) - page = NULL; - } - } + pfn = __kvm_faultin_pfn(memslot, gfn, writing ? FOLL_WRITE : 0, + &upgrade_write, &page); + if (is_error_noslot_pfn(pfn)) + return -EFAULT; /* * Read the PTE from the process' radix tree and use that From patchwork Thu Oct 10 18:24:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830931 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B536BCFC5FE for ; Thu, 10 Oct 2024 19:34:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Type:Cc:To: From:Subject:Message-ID:References:Mime-Version:In-Reply-To:Date: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=8JV5lSZeo1By1ou6uwDGqoH8kmaYenoeIxSHz9El/d0=; b=NmCYy0MlYcl/GxFqDe9rk7Sk2k HDfv3gGvTBh2YNt6KW8bXtCg4QKa4Vb8On8VIM7n7Jx2qGUDD9iTVwJV1yK8WDkxpGpi/qZvbkggw WgT/lH2yQz1z1kUSMgDm02oAmKqYGglXq4jZlTh+WD64ZIkT1KUE1gXFRiyZrEz/5jWg1XUxZzQ13 9ZdA3UUrCbSqeukA5ECAaccYadt75oWhpp8vX6GUUkivFWIbc1yWfuctN0G5LhNMFl/Zni4UH55VM kolvNoSgjZBHV9/Vrcc5aCQHqvSclzR8y1FbRjYQxK10eWQuKMacFtqhcAKB2onfNTKtK9O/JqZMa xJ36c9Bw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syyvK-0000000E7ha-3Seq; Thu, 10 Oct 2024 19:34:10 +0000 Received: from mail-pf1-x449.google.com ([2607:f8b0:4864:20::449]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxsO-0000000DrMJ-3vro for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:27:07 +0000 Received: by mail-pf1-x449.google.com with SMTP id d2e1a72fcca58-71e1e989aa2so1656074b3a.3 for ; Thu, 10 Oct 2024 11:27:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584824; x=1729189624; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=8JV5lSZeo1By1ou6uwDGqoH8kmaYenoeIxSHz9El/d0=; b=tfVvytWhY08fj6I/CIwHTgEx2WklGedcKVKQGHSPBJOoelJUycx/EZ/LKn+/zj318X +kIiEPwDv/ASCm+gLQ84XPKhqTSK5qjUYu2ciwEDod/zaHs8mN1gvJCvKbI5NpfvZypx qOwChr4uhvzgYifTcfVHYeIKAb8EfGqtCxyKhKoDtd+i42wQEeJvNFUrcS+w2JiMkh07 w1qh6lYrF02WLFCBkElXAmzCsdAdd/czW6ZFQBo27MM6UJCiVnyAhbX+hCaS0deCq7BJ +zOPhFnwvwq357J5qFvUs1Y2+q1JPAAp++JCFyYOWMGcoALF2Atj2zMDguXp+4l7N4RV RY+w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584824; x=1729189624; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=8JV5lSZeo1By1ou6uwDGqoH8kmaYenoeIxSHz9El/d0=; b=KLJAnPzKhsOl4AzwtvuzKmQTE9PGNOm8a92Qh60hf1naMS4oFHAMhr3HlzbOyQjrs1 Bg0Yly8+0aCnRMevM0XNwhxDWM8Ktl/zUB66Fz7rDNBw4gqQzf5wKu+a8vd8kJKgjimi KYFA5oWR3Yoz2dNDpxQpyXLAXTq6nPqdbgsAj49e6LBhWamshrKJmUR7YNuhMjO8J2Ki PRRZKZEcF+OzxtdamVU4twHOcYXl4hkOiT4YT+IKnRlPH2cb29jN2hf0IuIRAjU7tUj3 nPlqlN1nhTLfYbNuzEXuP+kCH3PHazWII+mXQylElr3WjFC8MBrWMxu4zg3pT0qwcqeL R/iw== X-Forwarded-Encrypted: i=1; AJvYcCWa1az9HY6HfBIxlHnbS9TuGusMqE20a+vBNfTc9unbs6PfXvso8QTDzmh6kj1quVFVPOG3hzpnucCcApsz1EfW@lists.infradead.org X-Gm-Message-State: AOJu0YyY3FYwCYxTdmNRbj7sDZxixByX2EjyVAHu2mIKgkQiVBQocGUJ +VpqYks8Z5AxUjFvYk4ft2q682VpydfCDmsnMVFfvkvS2XTfpgAqfQB02B06CRneHwA4aakwq3U mHw== X-Google-Smtp-Source: AGHT+IFev0i08WnhWXOHYVCiDa8hDSrBgilu3q+XI+BXP7ns5qqyHbYBrSKz+aJVjoCmfpAuiTEXvCWN4KQ= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:6a00:2daa:b0:71e:401:6580 with SMTP id d2e1a72fcca58-71e1dbf2587mr27202b3a.6.1728584823552; Thu, 10 Oct 2024 11:27:03 -0700 (PDT) Date: Thu, 10 Oct 2024 11:24:04 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-63-seanjc@google.com> Subject: [PATCH v13 62/85] KVM: PPC: Drop unused @kvm_ro param from kvmppc_book3s_instantiate_page() From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112705_100928_C84DB633 X-CRM114-Status: GOOD ( 14.50 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Drop @kvm_ro from kvmppc_book3s_instantiate_page() as it is now only written, and never read. No functional change intended. Signed-off-by: Sean Christopherson --- arch/powerpc/include/asm/kvm_book3s.h | 2 +- arch/powerpc/kvm/book3s_64_mmu_radix.c | 6 ++---- arch/powerpc/kvm/book3s_hv_nested.c | 4 +--- 3 files changed, 4 insertions(+), 8 deletions(-) diff --git a/arch/powerpc/include/asm/kvm_book3s.h b/arch/powerpc/include/asm/kvm_book3s.h index 10618622d7ef..3d289dbe3982 100644 --- a/arch/powerpc/include/asm/kvm_book3s.h +++ b/arch/powerpc/include/asm/kvm_book3s.h @@ -203,7 +203,7 @@ extern bool kvmppc_hv_handle_set_rc(struct kvm *kvm, bool nested, extern int kvmppc_book3s_instantiate_page(struct kvm_vcpu *vcpu, unsigned long gpa, struct kvm_memory_slot *memslot, - bool writing, bool kvm_ro, + bool writing, pte_t *inserted_pte, unsigned int *levelp); extern int kvmppc_init_vm_radix(struct kvm *kvm); extern void kvmppc_free_radix(struct kvm *kvm); diff --git a/arch/powerpc/kvm/book3s_64_mmu_radix.c b/arch/powerpc/kvm/book3s_64_mmu_radix.c index 14891d0a3b73..b3e6e73d6a08 100644 --- a/arch/powerpc/kvm/book3s_64_mmu_radix.c +++ b/arch/powerpc/kvm/book3s_64_mmu_radix.c @@ -821,7 +821,7 @@ bool kvmppc_hv_handle_set_rc(struct kvm *kvm, bool nested, bool writing, int kvmppc_book3s_instantiate_page(struct kvm_vcpu *vcpu, unsigned long gpa, struct kvm_memory_slot *memslot, - bool writing, bool kvm_ro, + bool writing, pte_t *inserted_pte, unsigned int *levelp) { struct kvm *kvm = vcpu->kvm; @@ -931,7 +931,6 @@ int kvmppc_book3s_radix_page_fault(struct kvm_vcpu *vcpu, struct kvm_memory_slot *memslot; long ret; bool writing = !!(dsisr & DSISR_ISSTORE); - bool kvm_ro = false; /* Check for unusual errors */ if (dsisr & DSISR_UNSUPP_MMU) { @@ -984,7 +983,6 @@ int kvmppc_book3s_radix_page_fault(struct kvm_vcpu *vcpu, ea, DSISR_ISSTORE | DSISR_PROTFAULT); return RESUME_GUEST; } - kvm_ro = true; } /* Failed to set the reference/change bits */ @@ -1002,7 +1000,7 @@ int kvmppc_book3s_radix_page_fault(struct kvm_vcpu *vcpu, /* Try to insert a pte */ ret = kvmppc_book3s_instantiate_page(vcpu, gpa, memslot, writing, - kvm_ro, NULL, NULL); + NULL, NULL); if (ret == 0 || ret == -EAGAIN) ret = RESUME_GUEST; diff --git a/arch/powerpc/kvm/book3s_hv_nested.c b/arch/powerpc/kvm/book3s_hv_nested.c index 05f5220960c6..771173509617 100644 --- a/arch/powerpc/kvm/book3s_hv_nested.c +++ b/arch/powerpc/kvm/book3s_hv_nested.c @@ -1527,7 +1527,6 @@ static long int __kvmhv_nested_page_fault(struct kvm_vcpu *vcpu, unsigned long n_gpa, gpa, gfn, perm = 0UL; unsigned int shift, l1_shift, level; bool writing = !!(dsisr & DSISR_ISSTORE); - bool kvm_ro = false; long int ret; if (!gp->l1_gr_to_hr) { @@ -1607,7 +1606,6 @@ static long int __kvmhv_nested_page_fault(struct kvm_vcpu *vcpu, ea, DSISR_ISSTORE | DSISR_PROTFAULT); return RESUME_GUEST; } - kvm_ro = true; } /* 2. Find the host pte for this L1 guest real address */ @@ -1629,7 +1627,7 @@ static long int __kvmhv_nested_page_fault(struct kvm_vcpu *vcpu, if (!pte_present(pte) || (writing && !(pte_val(pte) & _PAGE_WRITE))) { /* No suitable pte found -> try to insert a mapping */ ret = kvmppc_book3s_instantiate_page(vcpu, gpa, memslot, - writing, kvm_ro, &pte, &level); + writing, &pte, &level); if (ret == -EAGAIN) return RESUME_GUEST; else if (ret) From patchwork Thu Oct 10 18:24:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830992 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A9978D2444C for ; Thu, 10 Oct 2024 19:37:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Type:Cc:To: From:Subject:Message-ID:References:Mime-Version:In-Reply-To:Date: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=I+2gp5vLlKRJWMtv7YF+IvP9h9YjJXEpCIK4AMigCis=; b=199gMs0QeQx26bNVn4Gd0R8X19 jm0axYqydqUFiitc82euStbzNGEJqHLE3QFwCQEQe69B1pMNwOCiJXdX0MoHEOLbM458uUi7b+q2O +sGvRhsLnY5+j9ok63tenexKseP2VeMyuAcuoeDlE+L0kHCKPsgx9M5mzEbmjVbfXVTyy54RvLpAT Wwb/lW1Bu4QNXuKdbScVnfHvy1NTuFWD8naUJFLh3VUBSWw2esYlUNnftsA+uIqtbmc+lTO3YOQAe 6A42l031JPM1OYpvINYck4AXvLnv46cScgbF74a2HBAG8Z7OWX6podGmeFDKum+M8bLKaPiSCzyTy lyyfxFYA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syyxy-0000000E8CC-2v8Y; Thu, 10 Oct 2024 19:36:54 +0000 Received: from mail-pg1-x54a.google.com ([2607:f8b0:4864:20::54a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxsR-0000000DrNf-1Bd6 for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:27:09 +0000 Received: by mail-pg1-x54a.google.com with SMTP id 41be03b00d2f7-7e9f9a927a5so1044401a12.2 for ; Thu, 10 Oct 2024 11:27:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584826; x=1729189626; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=I+2gp5vLlKRJWMtv7YF+IvP9h9YjJXEpCIK4AMigCis=; b=ovwcCNiTe5u04NlAVbvO4xK8yzOXNx3Ter+QhNXl3VOJCQEzI/gEvnC3H1V7Pqxq5A sOkzivM5rCRz91Y8yeZwvNqvIZLyWI/UTPOKy44Uv53XB221DIga2phGSlSM+PBCmaOl bgVg0eim/0V6+Ct3L/8ml5n/a7aAynz40Pzgot6a8ms+YupfuzIQ2Mh4wpNXQzF1nicw 60opQHHdJC1LDVjxEqhBrglkeQMOhacGkNexNdqMNHeRff1T8NraOFVlEAgUebACjRAr 6UmULf0Wwv971DDjdvY+42yiNd3AMS3OUiyNgkJ4jpH03Q1sbY6akeGf+wKLHMkEHkZ9 eKlw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584826; x=1729189626; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=I+2gp5vLlKRJWMtv7YF+IvP9h9YjJXEpCIK4AMigCis=; b=GVcozwt0mkv4mUqmUB8p2j9F988lzouL1Pu+Zc5z2RK7LY39rnq+E7zUbrLMQ5E4np bEt0cM6A3hP9CL05oCqwvtRoa0UGMHMKECkCwh78693vpAjVKdjsvb4+6M4n1YqtT5qU MuWwurhhWwgzl5TPmWFLS3T7trlzCP069mov4xyaWPCrXSpH7CPUqSco5jJZy7Ip2lkw WFpvVYzRkrSaUpbzolIOUg4Ei5bbHlfhT3gJS6CVb2snMq9zCsTWOpeCt3iQ5/JTOmt+ +7aUvGXfsadr8A+kjPXwtALNpRZelU9pwFq6IkxqVoQHukbUZyYPsnRXlUPH1f7SgQ1b I2kw== X-Forwarded-Encrypted: i=1; AJvYcCXw70VsMRgRjAt8Tql3zF6D64/1TUPpYI+lcDn66n8BRQ9Noj4cUqLq5p8/jaWdoMp2dzxkP4oIkDcwfFKLHh2C@lists.infradead.org X-Gm-Message-State: AOJu0YwNgk2+Riv4rufnkhGDdYEerNZ/8S5wp8VbUNBoH7n3DUkt8jqL Uz95YHte/78VhkByQLQ9Wk0H4h7SDR6O3isMTWBerLFKcbxVoamnp7jw4qTYvg1PZh2pKV87EQl E7Q== X-Google-Smtp-Source: AGHT+IEBzxfSdV5URAWtipQSxApJSdWqJAcjcFibIBQs32g4G2eVPMYjVJgo5zFwBPuTEySHq4oumcdrCOE= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a63:e546:0:b0:684:6543:719 with SMTP id 41be03b00d2f7-7ea535307afmr40a12.4.1728584825522; Thu, 10 Oct 2024 11:27:05 -0700 (PDT) Date: Thu, 10 Oct 2024 11:24:05 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-64-seanjc@google.com> Subject: [PATCH v13 63/85] KVM: PPC: Book3S: Mark "struct page" pfns dirty/accessed after installing PTE From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112707_450108_3E5C8326 X-CRM114-Status: GOOD ( 11.27 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Mark pages/folios dirty/accessed after installing a PTE, and more specifically after acquiring mmu_lock and checking for an mmu_notifier invalidation. Marking a page/folio dirty after it has been written back can make some filesystems unhappy (backing KVM guests will such filesystem files is uncommon, and the race is minuscule, hence the lack of complaints). See the link below for details. This will also allow converting Book3S to kvm_release_faultin_page(), which requires that mmu_lock be held (for the aforementioned reason). Link: https://lore.kernel.org/all/cover.1683044162.git.lstoakes@gmail.com Signed-off-by: Sean Christopherson --- arch/powerpc/kvm/book3s_64_mmu_host.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/arch/powerpc/kvm/book3s_64_mmu_host.c b/arch/powerpc/kvm/book3s_64_mmu_host.c index bc6a381b5346..d0e4f7bbdc3d 100644 --- a/arch/powerpc/kvm/book3s_64_mmu_host.c +++ b/arch/powerpc/kvm/book3s_64_mmu_host.c @@ -121,13 +121,10 @@ int kvmppc_mmu_map_page(struct kvm_vcpu *vcpu, struct kvmppc_pte *orig_pte, vpn = hpt_vpn(orig_pte->eaddr, map->host_vsid, MMU_SEGSIZE_256M); - kvm_set_pfn_accessed(pfn); if (!orig_pte->may_write || !writable) rflags |= PP_RXRX; - else { + else mark_page_dirty(vcpu->kvm, gfn); - kvm_set_pfn_dirty(pfn); - } if (!orig_pte->may_execute) rflags |= HPTE_R_N; @@ -202,8 +199,11 @@ int kvmppc_mmu_map_page(struct kvm_vcpu *vcpu, struct kvmppc_pte *orig_pte, } out_unlock: + if (!orig_pte->may_write || !writable) + kvm_release_pfn_clean(pfn); + else + kvm_release_pfn_dirty(pfn); spin_unlock(&kvm->mmu_lock); - kvm_release_pfn_clean(pfn); if (cpte) kvmppc_mmu_hpte_cache_free(cpte); From patchwork Thu Oct 10 18:24:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831203 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 800ECD24454 for ; Thu, 10 Oct 2024 21:46:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Type:Cc:To: From:Subject:Message-ID:References:Mime-Version:In-Reply-To:Date: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=dJnHBDyCwUFZKgDYTVV2s9CsyA2Ll45yxWv/Qp8Ijpw=; b=wSs774zhdSMOBVeH4ySd7Tsh2g 3fgstoeC9itBzhkXTufx1yFVqq1203iftbx7lhrRFkC7iQu3SgwkYg8zgdljuMDr2M6VrYJHy+d42 wsKRr9OTme7FetguFaZmWJW+Zp0f6+61AIa70jULqQYymk+Mvd3LkAXTgU9z5OuHKu/7YwrK9F2l0 wnCXBOqQu0DtS7pk2IwyHD45f3ROqk1RlcPZ0PRh4Pw4wsPVBybS8mDVyqL3Pug0KZxLM21ChsjIa CEzMAwpj0RhMzdAnNdBfF4O3k0uJJo9u3oiPgPscfx9vtWknLWi6EfdlDktU7wouc/EZVkXPGSZwm vAKHiVvg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1sz0zR-0000000ESZY-3sGi; Thu, 10 Oct 2024 21:46:33 +0000 Received: from mail-yw1-x1149.google.com ([2607:f8b0:4864:20::1149]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxsT-0000000DrOx-0y0a for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:27:10 +0000 Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-6d9e31e66eeso26625147b3.1 for ; Thu, 10 Oct 2024 11:27:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584828; x=1729189628; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=dJnHBDyCwUFZKgDYTVV2s9CsyA2Ll45yxWv/Qp8Ijpw=; b=n4KGo6iNfJSLuN1FKHV+WJOm5A+uI/ubGTtJ7sHKInJL5Xz8lmeIq55am41KvLA2ao 32uC3ioQczNVzq2DxLqf3xFXC9No9LpQFKnH9af6e2SKUOnEbEhDKXiM2qWFf0+Q+SDY VvKzXsdj5AGASllnhLZaZvtgBvU2ST8INbiun/Eg7+zqd2dc9v3qGMZBbRvRBQ1twMpQ 9iW7KPnnLn9R81KDtQBmPHNifMhaLFh9czAYZ3/RwTs6MHRWTKCL2TUaPKVCZrGm5K6H 83KMhNqw1PBzG4354BXYdTvM5XlctmgLuKWl1gCBzusUOde3O5gGUoew6t6QNAVUy5Mb Uc3w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584828; x=1729189628; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=dJnHBDyCwUFZKgDYTVV2s9CsyA2Ll45yxWv/Qp8Ijpw=; b=h3/DbsvtotDgFL9Yn1jr45+l2c+zcqHEA5kUBwiaBYIsAbQU7IeJo1Ple1dl3TRu59 3Hr0Tto5BhNRztWvJ2HAbIYVmcBlTp1EmmczezHSpZgJJ9UpQSNKwAoksf8Ic4z4pAru 4D7o8EEJ542X555E4zCfjHo1f0c3Dus1xa+6WfNWCk098/w+m75MlMZ8aShLZ6/DZHCn mU8247LiRAITeaVMwKhWYqcvjcRqPL0jyX0to6FWfZoPWGEImxnKRBnmyV6IkJZ/vRpd qwTZ/DXPf5sQdlYXLExFjrWv8mfb8dgeTO3Z1W6JrDnGRVeMbYVcaC9uIeJr+ccqfbFy vPJg== X-Forwarded-Encrypted: i=1; AJvYcCWNifZUZQfYOzOTlFkZVPKSFpRAD0CYslmEyHasOxYjoSqM+jGyoaUBncmCFMAMoiENR5HCZqQeTkFBlSBMsHtd@lists.infradead.org X-Gm-Message-State: AOJu0YyXRQolslm5K9rR19ljFtnmDMSBZMIxnqO4TDXZBBQgWveYjmQ2 mqq64yuWzjs5WCx3Qf7Juf8xSrH43EL2nfpIIW+1ac20z9eG4MIbs5o0OfNInDCNcte1AiBi4GH q9Q== X-Google-Smtp-Source: AGHT+IE3wQG+AmAWHFVOEkWtTHcZkQX/AbMzk25RszCXWWBD0mfEZbS2aMxfU1imj+f8RQ2cZBD1mpHhlQw= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a5b:a07:0:b0:e28:ef8f:7423 with SMTP id 3f1490d57ef6-e28fe355de0mr48920276.4.1728584827639; Thu, 10 Oct 2024 11:27:07 -0700 (PDT) Date: Thu, 10 Oct 2024 11:24:06 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-65-seanjc@google.com> Subject: [PATCH v13 64/85] KVM: PPC: Use kvm_faultin_pfn() to handle page faults on Book3s PR From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112709_335153_C1AEDAEC X-CRM114-Status: GOOD ( 14.69 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Convert Book3S PR to __kvm_faultin_pfn()+kvm_release_faultin_page(), which are new APIs to consolidate arch code and provide consistent behavior across all KVM architectures. Signed-off-by: Sean Christopherson --- arch/powerpc/include/asm/kvm_book3s.h | 2 +- arch/powerpc/kvm/book3s.c | 7 ++++--- arch/powerpc/kvm/book3s_32_mmu_host.c | 7 ++++--- arch/powerpc/kvm/book3s_64_mmu_host.c | 10 +++++----- 4 files changed, 14 insertions(+), 12 deletions(-) diff --git a/arch/powerpc/include/asm/kvm_book3s.h b/arch/powerpc/include/asm/kvm_book3s.h index 3d289dbe3982..e1ff291ba891 100644 --- a/arch/powerpc/include/asm/kvm_book3s.h +++ b/arch/powerpc/include/asm/kvm_book3s.h @@ -235,7 +235,7 @@ extern void kvmppc_set_bat(struct kvm_vcpu *vcpu, struct kvmppc_bat *bat, extern void kvmppc_giveup_ext(struct kvm_vcpu *vcpu, ulong msr); extern int kvmppc_emulate_paired_single(struct kvm_vcpu *vcpu); extern kvm_pfn_t kvmppc_gpa_to_pfn(struct kvm_vcpu *vcpu, gpa_t gpa, - bool writing, bool *writable); + bool writing, bool *writable, struct page **page); extern void kvmppc_add_revmap_chain(struct kvm *kvm, struct revmap_entry *rev, unsigned long *rmap, long pte_index, int realmode); extern void kvmppc_update_dirty_map(const struct kvm_memory_slot *memslot, diff --git a/arch/powerpc/kvm/book3s.c b/arch/powerpc/kvm/book3s.c index ff6c38373957..d79c5d1098c0 100644 --- a/arch/powerpc/kvm/book3s.c +++ b/arch/powerpc/kvm/book3s.c @@ -422,7 +422,7 @@ int kvmppc_core_prepare_to_enter(struct kvm_vcpu *vcpu) EXPORT_SYMBOL_GPL(kvmppc_core_prepare_to_enter); kvm_pfn_t kvmppc_gpa_to_pfn(struct kvm_vcpu *vcpu, gpa_t gpa, bool writing, - bool *writable) + bool *writable, struct page **page) { ulong mp_pa = vcpu->arch.magic_page_pa & KVM_PAM; gfn_t gfn = gpa >> PAGE_SHIFT; @@ -437,13 +437,14 @@ kvm_pfn_t kvmppc_gpa_to_pfn(struct kvm_vcpu *vcpu, gpa_t gpa, bool writing, kvm_pfn_t pfn; pfn = (kvm_pfn_t)virt_to_phys((void*)shared_page) >> PAGE_SHIFT; - get_page(pfn_to_page(pfn)); + *page = pfn_to_page(pfn); + get_page(*page); if (writable) *writable = true; return pfn; } - return gfn_to_pfn_prot(vcpu->kvm, gfn, writing, writable); + return kvm_faultin_pfn(vcpu, gfn, writing, writable, page); } EXPORT_SYMBOL_GPL(kvmppc_gpa_to_pfn); diff --git a/arch/powerpc/kvm/book3s_32_mmu_host.c b/arch/powerpc/kvm/book3s_32_mmu_host.c index 4b3a8d80cfa3..5b7212edbb13 100644 --- a/arch/powerpc/kvm/book3s_32_mmu_host.c +++ b/arch/powerpc/kvm/book3s_32_mmu_host.c @@ -130,6 +130,7 @@ extern char etext[]; int kvmppc_mmu_map_page(struct kvm_vcpu *vcpu, struct kvmppc_pte *orig_pte, bool iswrite) { + struct page *page; kvm_pfn_t hpaddr; u64 vpn; u64 vsid; @@ -145,7 +146,7 @@ int kvmppc_mmu_map_page(struct kvm_vcpu *vcpu, struct kvmppc_pte *orig_pte, bool writable; /* Get host physical address for gpa */ - hpaddr = kvmppc_gpa_to_pfn(vcpu, orig_pte->raddr, iswrite, &writable); + hpaddr = kvmppc_gpa_to_pfn(vcpu, orig_pte->raddr, iswrite, &writable, &page); if (is_error_noslot_pfn(hpaddr)) { printk(KERN_INFO "Couldn't get guest page for gpa %lx!\n", orig_pte->raddr); @@ -232,7 +233,7 @@ int kvmppc_mmu_map_page(struct kvm_vcpu *vcpu, struct kvmppc_pte *orig_pte, pte = kvmppc_mmu_hpte_cache_next(vcpu); if (!pte) { - kvm_release_pfn_clean(hpaddr >> PAGE_SHIFT); + kvm_release_page_unused(page); r = -EAGAIN; goto out; } @@ -250,7 +251,7 @@ int kvmppc_mmu_map_page(struct kvm_vcpu *vcpu, struct kvmppc_pte *orig_pte, kvmppc_mmu_hpte_cache_map(vcpu, pte); - kvm_release_pfn_clean(hpaddr >> PAGE_SHIFT); + kvm_release_page_clean(page); out: return r; } diff --git a/arch/powerpc/kvm/book3s_64_mmu_host.c b/arch/powerpc/kvm/book3s_64_mmu_host.c index d0e4f7bbdc3d..be20aee6fd7d 100644 --- a/arch/powerpc/kvm/book3s_64_mmu_host.c +++ b/arch/powerpc/kvm/book3s_64_mmu_host.c @@ -88,13 +88,14 @@ int kvmppc_mmu_map_page(struct kvm_vcpu *vcpu, struct kvmppc_pte *orig_pte, struct hpte_cache *cpte; unsigned long gfn = orig_pte->raddr >> PAGE_SHIFT; unsigned long pfn; + struct page *page; /* used to check for invalidations in progress */ mmu_seq = kvm->mmu_invalidate_seq; smp_rmb(); /* Get host physical address for gpa */ - pfn = kvmppc_gpa_to_pfn(vcpu, orig_pte->raddr, iswrite, &writable); + pfn = kvmppc_gpa_to_pfn(vcpu, orig_pte->raddr, iswrite, &writable, &page); if (is_error_noslot_pfn(pfn)) { printk(KERN_INFO "Couldn't get guest page for gpa %lx!\n", orig_pte->raddr); @@ -199,10 +200,9 @@ int kvmppc_mmu_map_page(struct kvm_vcpu *vcpu, struct kvmppc_pte *orig_pte, } out_unlock: - if (!orig_pte->may_write || !writable) - kvm_release_pfn_clean(pfn); - else - kvm_release_pfn_dirty(pfn); + /* FIXME: Don't unconditionally pass unused=false. */ + kvm_release_faultin_page(kvm, page, false, + orig_pte->may_write && writable); spin_unlock(&kvm->mmu_lock); if (cpte) kvmppc_mmu_hpte_cache_free(cpte); From patchwork Thu Oct 10 18:24:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831204 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 429FED24456 for ; Thu, 10 Oct 2024 21:46:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Type:Cc:To: From:Subject:Message-ID:References:Mime-Version:In-Reply-To:Date: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=0kIT1cNFK/xt59MDB/Io88ATgyMybLemEwY+DfOgUM8=; b=1b4ZW8L7adH41/Gj+VMBO7RWPR S7ASJHfW760o6GsrH/bbseOOp0EgAXd9Bts651zLz+NPevjKTUNSdSz4MjFVJpPAeycpNzQCMcfdo yGApkXAwWvRqwPohzhNky/Q+PBbu2GwTKu5cWO7+oIqY/sxWw5nlmvY53HJB8AcW1XnGcXcqBEdoo dCLPH6rltRD+b4lLMVwO5ey1c2f0x/fyC7bQ7+GddHp/gGCdv5YIZxPqmVvyuCP8J8O2DCyZ01nmp t0fVTN+YB7dlykCucqOC6jce3lDHTFX3tXzgliBJcUQp/AhN20mf8qG4/tds0cccSmi8vLQpVCoPt wrMV8Itg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1sz0zW-0000000ESdC-05Ot; Thu, 10 Oct 2024 21:46:38 +0000 Received: from mail-pl1-x64a.google.com ([2607:f8b0:4864:20::64a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxsU-0000000DrQL-3Loz for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:27:13 +0000 Received: by mail-pl1-x64a.google.com with SMTP id d9443c01a7336-207510f3242so16008525ad.0 for ; Thu, 10 Oct 2024 11:27:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584829; x=1729189629; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=0kIT1cNFK/xt59MDB/Io88ATgyMybLemEwY+DfOgUM8=; b=ncGV/c+AluGiB5Wub5u8UvSkBS0krkBKkswJl6TMY+O1XuP40I467d8ozAcdtO7hSX bztAb+6FlXzcvwIsAQ6TTFHDKDi7VhRVvgGcklZgclHPJPmQIJUmDxuuKMxJyiWV4Us9 Ml94yw8Oe73armjC74hfiiSHJHgPJIO7CSlAJKylZc6klQTjK+b7jKsxlkI+sKtaU9qd cifr07vNF2dIWwlBAwdWJ4VSg4uRAO8nMuyrn3L4ExIpZkQBXLiKPMlqGK55FpGn/Tqj k6bkx9fc/VgXfATlBMbhoK+qvMiheT5pGQJ8Qkty2qp7noqUc5YtxHoCd3kwzYuiRXs3 y4BQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584829; x=1729189629; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=0kIT1cNFK/xt59MDB/Io88ATgyMybLemEwY+DfOgUM8=; b=ZhZbXbQ+jHbfUKd0AU1nXQpvZcRb93fJZt8TevN9h43j+rG4igGIgxIDdap+IQssaw 2wyUg8SfyEbKyYs05/qhPxw+grR1oLjowgIBgadR3RDxCa27faw1xFZdfDBZF22O3130 sHgbwKYJM7HlUH9eTR2qqO0r7ZakB9G+Nf4ZY5U3ywVP/pnX4/uNNGSZO+0x81TJI7zi 6pj7H5I7G/vR/WYNLTh5ZMV+LwlS7M6CDr1TYYepbmhWGrUxoNFMIbfH8kbfIF6FeTY/ sr0nBHBW9pRc8FyxqEQNewu/OyC8CGKXlTTADKB/2xjD4ZDE5UofH+tnMHP1IlWffLa0 +0aA== X-Forwarded-Encrypted: i=1; AJvYcCWhEVfCktHyQJyeGUWy2oS1tIqGBMQzevrcwAr6XguqDIsuHsd1e0JzletfQrqOOA8k8uxgQvtpd+9hm5hevATt@lists.infradead.org X-Gm-Message-State: AOJu0Yy7j0omKGXONmHC4K3Ozjb5BzCNv4LQtgZn7Qh4SUcqjyUeMKY9 /JH9aA8oF0fLfJhWOqSmSfQaQ37qez2PT7KY7t3zfFKw6q2+EbqT7Wn2BSRZ1JFF1ZOvmTMCZ4b d9w== X-Google-Smtp-Source: AGHT+IEVjHGN/ZoxQo+Z413816spxD1Vig97FmVkGQ57WYZx6k/7CY2I8TFVs2SPVm+Oq4UkGx7TgXf3+Fw= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:903:192:b0:20b:bd8d:427a with SMTP id d9443c01a7336-20c6377ab7bmr1062205ad.5.1728584829203; Thu, 10 Oct 2024 11:27:09 -0700 (PDT) Date: Thu, 10 Oct 2024 11:24:07 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-66-seanjc@google.com> Subject: [PATCH v13 65/85] KVM: LoongArch: Mark "struct page" pfns dirty only in "slow" page fault path From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112711_022484_1B4FFC21 X-CRM114-Status: GOOD ( 11.29 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Mark pages/folios dirty only the slow page fault path, i.e. only when mmu_lock is held and the operation is mmu_notifier-protected, as marking a page/folio dirty after it has been written back can make some filesystems unhappy (backing KVM guests will such filesystem files is uncommon, and the race is minuscule, hence the lack of complaints). See the link below for details. Link: https://lore.kernel.org/all/cover.1683044162.git.lstoakes@gmail.com Reviewed-by: Bibo Mao Signed-off-by: Sean Christopherson --- arch/loongarch/kvm/mmu.c | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-) diff --git a/arch/loongarch/kvm/mmu.c b/arch/loongarch/kvm/mmu.c index 28681dfb4b85..cc2a5f289b14 100644 --- a/arch/loongarch/kvm/mmu.c +++ b/arch/loongarch/kvm/mmu.c @@ -608,13 +608,13 @@ static int kvm_map_page_fast(struct kvm_vcpu *vcpu, unsigned long gpa, bool writ if (kvm_pte_young(changed)) kvm_set_pfn_accessed(pfn); - if (kvm_pte_dirty(changed)) { - mark_page_dirty(kvm, gfn); - kvm_set_pfn_dirty(pfn); - } if (page) put_page(page); } + + if (kvm_pte_dirty(changed)) + mark_page_dirty(kvm, gfn); + return ret; out: spin_unlock(&kvm->mmu_lock); @@ -915,12 +915,14 @@ static int kvm_map_page(struct kvm_vcpu *vcpu, unsigned long gpa, bool write) else ++kvm->stat.pages; kvm_set_pte(ptep, new_pte); - spin_unlock(&kvm->mmu_lock); - if (prot_bits & _PAGE_DIRTY) { - mark_page_dirty_in_slot(kvm, memslot, gfn); + if (writeable) kvm_set_pfn_dirty(pfn); - } + + spin_unlock(&kvm->mmu_lock); + + if (prot_bits & _PAGE_DIRTY) + mark_page_dirty_in_slot(kvm, memslot, gfn); kvm_release_pfn_clean(pfn); out: From patchwork Thu Oct 10 18:24:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13830993 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 404C1D24442 for ; Thu, 10 Oct 2024 19:41:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Type:Cc:To: From:Subject:Message-ID:References:Mime-Version:In-Reply-To:Date: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=2XctyNuj5rmp2glWM4MaDBrVp36o24kp0nhTjUz4HGM=; b=VdkDwVdfUzehPU5/Rv5ZkLf6xJ p8s3uGUq47q52z6Cmyf5q8NULJNnJdWM3uPBe5BVz083j3qw0ih86w/nYdJZg9c0aL4kBtidWBpCZ y7BD7OcfY2fgbxoW0VTyENJO6tiywXF9ayQc3X2lxpjaZOv/4NtyANj4ZKiRnWbQgltLYZBidCGB6 761Zmj4q1NYx2NLfOvB5SqULjAg3IhdPRd1jYhvRrUv+tBlFJ4wz2exESsfNypVxtaaQ/PIJq6nLU afeLLVPSTK/tAfengSnWQLzWE55JAMZHOPCGr2bCDOO8PKQvislAexOTRCib+8aoUP5hYPd+e+nyk tnU27DaA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syz1v-0000000E8tP-0lF6; Thu, 10 Oct 2024 19:40:59 +0000 Received: from mail-yb1-xb4a.google.com ([2607:f8b0:4864:20::b4a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxsW-0000000DrSL-2dKB for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:27:15 +0000 Received: by mail-yb1-xb4a.google.com with SMTP id 3f1490d57ef6-e28edea9af6so1639540276.3 for ; Thu, 10 Oct 2024 11:27:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584831; x=1729189631; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=2XctyNuj5rmp2glWM4MaDBrVp36o24kp0nhTjUz4HGM=; b=YAw0UCrfTR8yZSyO2C++/akkY9F8ImVdU+QHywnyEzDNoziQZcTnFyWLoTIQpJhvbV Y8Pdn3YxYXd0G6JVJDHuHfdxdQ9m+Qk7S6GzmJ9qM85akpQuGxzBDKtJPWDI2IUxznCD vq6DmPEgcnDV0SF23KJrvt4oUa9M6w69GM7oSp9H743VLbGqcPfYLAjnqaTw6lH5kL5I AeSEU2al1IUe8APYV1qkZosGuvEU/HUkQXE+1awQrmJRkpjT8XqM9EaKYUWt37X5qHY/ OQdNCE4r4uavOa2sXQyoineMlK5JoEa5HC2eFTf5e4O7iVUYru4c1B0gd0vQezhVxbT8 DApg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584831; x=1729189631; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=2XctyNuj5rmp2glWM4MaDBrVp36o24kp0nhTjUz4HGM=; b=wt8CoIfBnC2pE481KomsivOpvbJDSsgau5Q+tQ4Iye8cgIoPrJKTSWObyzSltVROLo C2+AM6xRoe3zkdpV3/8AR2kSHv8uwX2WpJRc9hD8dxoZWyvFhzKue69xP0zh7Km1yPYp 5uwOkiMWFhA8z1vzr8vaFiMyIX5DL9gJx/66NNndsQwCJ77a7FWdvpZpISjGO8sm7659 bZuimgbkm9CxfyBYAhxy6586Z3zY+SnIit7nHVZVTVkWGc+oxeFO6NisWOmpol5Y4vr2 YBvab/GChDtyNvTSGqEFu/EPVjBRtjhj5WI+pH08uiGCTQXiS8Ro9vMwlgpr3uBtKPMC oYXQ== X-Forwarded-Encrypted: i=1; AJvYcCXyqjCfq4TxjxUYa3lhtlAQZe3hqPiI0Xuu3CXQs1qc+2CyDuNUf/H8K0efQK+/QmH1j2pSSuIPs/pAZeej7GpP@lists.infradead.org X-Gm-Message-State: AOJu0YxgJuxzgQPV52N9YsoCv/GgKlgArOZM4Y74YbaHYnuDUUPK/00w IQn1ILgd3rb+UCKefUL/vxLecqK04C43ozHYhd3iwpker5zeuGmWBzDR1/ZO+OUJ96XcFvVpivj Zrg== X-Google-Smtp-Source: AGHT+IHdtHq1a5g5QmW1TlBMuTKUOZ7R4d6aOVSlzi7r2JK/pfzCqaKRixPsJw6iRhPElP6sWQ73LfqmPps= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a25:d353:0:b0:e28:e74f:4cb with SMTP id 3f1490d57ef6-e28fe0df614mr96115276.0.1728584831124; Thu, 10 Oct 2024 11:27:11 -0700 (PDT) Date: Thu, 10 Oct 2024 11:24:08 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-67-seanjc@google.com> Subject: [PATCH v13 66/85] KVM: LoongArch: Mark "struct page" pfns accessed only in "slow" page fault path From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112712_874527_2C55D90A X-CRM114-Status: GOOD ( 13.45 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Mark pages accessed only in the slow path, before dropping mmu_lock when faulting in guest memory so that LoongArch can convert to kvm_release_faultin_page() without tripping its lockdep assertion on mmu_lock being held. Reviewed-by: Bibo Mao Signed-off-by: Sean Christopherson --- arch/loongarch/kvm/mmu.c | 20 ++------------------ 1 file changed, 2 insertions(+), 18 deletions(-) diff --git a/arch/loongarch/kvm/mmu.c b/arch/loongarch/kvm/mmu.c index cc2a5f289b14..ed43504c5c7e 100644 --- a/arch/loongarch/kvm/mmu.c +++ b/arch/loongarch/kvm/mmu.c @@ -552,12 +552,10 @@ bool kvm_test_age_gfn(struct kvm *kvm, struct kvm_gfn_range *range) static int kvm_map_page_fast(struct kvm_vcpu *vcpu, unsigned long gpa, bool write) { int ret = 0; - kvm_pfn_t pfn = 0; kvm_pte_t *ptep, changed, new; gfn_t gfn = gpa >> PAGE_SHIFT; struct kvm *kvm = vcpu->kvm; struct kvm_memory_slot *slot; - struct page *page; spin_lock(&kvm->mmu_lock); @@ -570,8 +568,6 @@ static int kvm_map_page_fast(struct kvm_vcpu *vcpu, unsigned long gpa, bool writ /* Track access to pages marked old */ new = kvm_pte_mkyoung(*ptep); - /* call kvm_set_pfn_accessed() after unlock */ - if (write && !kvm_pte_dirty(new)) { if (!kvm_pte_write(new)) { ret = -EFAULT; @@ -595,23 +591,11 @@ static int kvm_map_page_fast(struct kvm_vcpu *vcpu, unsigned long gpa, bool writ } changed = new ^ (*ptep); - if (changed) { + if (changed) kvm_set_pte(ptep, new); - pfn = kvm_pte_pfn(new); - page = kvm_pfn_to_refcounted_page(pfn); - if (page) - get_page(page); - } + spin_unlock(&kvm->mmu_lock); - if (changed) { - if (kvm_pte_young(changed)) - kvm_set_pfn_accessed(pfn); - - if (page) - put_page(page); - } - if (kvm_pte_dirty(changed)) mark_page_dirty(kvm, gfn); From patchwork Thu Oct 10 18:24:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831205 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D12D6D24454 for ; Thu, 10 Oct 2024 21:47:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Type:Cc:To: From:Subject:Message-ID:References:Mime-Version:In-Reply-To:Date: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=3FFUjrr9Bcuid6sLVD5OEsWVTb8VG25P9bGjkNYbAzk=; b=Xbw80HhaumOfhaHAKuA1UOhDOQ Umk+yHMYysSBIhtKRVcwqNxdMz0feFFg9mKmK1qEX8pjwuoxZWbQsXRIJjw44lRikNnYBndkyLzlQ aeWOhyjHfxkwTmwoSqjwfPhCxL2jIe4hWNnpXCk2P42HIuGmkQXbOnnVsceogA+O4nWN16okX3SPv h7M20oNf1Rriv929UVjvppM8hBvjYx3uyMrAFCIa4KJQrcXwx7oYN7n/vnSL2mOqAvx88lpV+c7/8 cLnDtTX67U6howBnb0ekAjzNbYx1Zi9dlFN4dSsqzvuM9wHO0CcNV2K/0Zdk4z2QtbhbdUCdfkkgl ROO8ZhEA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1sz0zh-0000000ESn9-0vqK; Thu, 10 Oct 2024 21:46:49 +0000 Received: from mail-pj1-x1049.google.com ([2607:f8b0:4864:20::1049]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxsY-0000000DrTr-2kJD for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:27:18 +0000 Received: by mail-pj1-x1049.google.com with SMTP id 98e67ed59e1d1-2e2ca4fb175so1249365a91.3 for ; Thu, 10 Oct 2024 11:27:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584833; x=1729189633; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=3FFUjrr9Bcuid6sLVD5OEsWVTb8VG25P9bGjkNYbAzk=; b=jRyJBHlowWWe5rLfr7phIk02+Dz1smiC9B+8+l8fjItytQHfaPqfwXS9kwAda+9AmE shexSqSJbO2Rx5qii1tALAryS4pGnWvtVZCfeOEjmFWSO4HKZPxS8nkGfwDKwQymv5/V wfkt5jyod4tGS84UNkBF5H1yJop/76JLfNyM1euymRTGgEs/dQq38usgpogljBsjQeGI Dm9WN/3n4aZGT70TBOP644TkbWftIVyZDtlLOtZt6iA+3KaFCeUyJfaSP6mtPhXMMbZl celjnP4Ov3sMYoEohXSA+AP8wA6JsEeZNgXH6S6QNub5U4W5uofyo/ebOoZSoyqeVbK2 vrZQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584833; x=1729189633; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=3FFUjrr9Bcuid6sLVD5OEsWVTb8VG25P9bGjkNYbAzk=; b=QyD1WWQXxyY57LczOW8t3m/guqPNI0o82wX27nsMGE56OtTmchhYE3EFCGfaaF2x95 HsLgeoBjsimd8oYQM63N/Tzv7K6MV2iM4kEptkCaz3rsEJCmMQBAkqF7BG+hb1Ij08OV lloVtkGoVgM/0CxVaZVZWVJKnuWN7cKyONEcZC7ZafbPSxTE0/j9dhyFk2BdLM0DF/ih IBzt/1jF96id5QbDfrhD6mBtSOoy5Xk7o9S1VV3XocOfOEdEPrfP9uVIPv4q/8y9nXk9 OyDHPP68QoxITJSgnAVUrp+4pq+/HiJDM8daMOv/W6f/sADinmXoCwygLxXgTbI2YsLm iQBw== X-Forwarded-Encrypted: i=1; AJvYcCWBazJ/DXqo44Kd9CY0oMjCFUi2ixcIff/+qejt4k6Me/IfNG8NjRLOj8SLAZy4nic6ZU3e7bZfoVr/+XjbhKoh@lists.infradead.org X-Gm-Message-State: AOJu0Yzwz4JLcX+yS9K/FxUScTXVMPIgQYdiXcZXqbu7JpaxTtv9qt6O 9EDYfc2teI9nDPi2BLveYnS1h0avgrrMlsAJVo+SUmsi75xNLagBsYs3P960MhqK2lWTBalKbzD e/A== X-Google-Smtp-Source: AGHT+IFpwqxWKAWfrUvKF1EatvnBv5g5CrzcpgT/ZTuiTODJnzoCoYK053S1MFwF0WyL9wI3LVhDYDAiv74= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:90b:46d4:b0:2e2:e148:3d37 with SMTP id 98e67ed59e1d1-2e2f09f1a56mr94a91.2.1728584833102; Thu, 10 Oct 2024 11:27:13 -0700 (PDT) Date: Thu, 10 Oct 2024 11:24:09 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-68-seanjc@google.com> Subject: [PATCH v13 67/85] KVM: LoongArch: Mark "struct page" pfn accessed before dropping mmu_lock From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112714_817511_A226DCEA X-CRM114-Status: GOOD ( 11.35 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Mark pages accessed before dropping mmu_lock when faulting in guest memory so that LoongArch can convert to kvm_release_faultin_page() without tripping its lockdep assertion on mmu_lock being held. Reviewed-by: Bibo Mao Signed-off-by: Sean Christopherson --- arch/loongarch/kvm/mmu.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/loongarch/kvm/mmu.c b/arch/loongarch/kvm/mmu.c index ed43504c5c7e..7066cafcce64 100644 --- a/arch/loongarch/kvm/mmu.c +++ b/arch/loongarch/kvm/mmu.c @@ -902,13 +902,13 @@ static int kvm_map_page(struct kvm_vcpu *vcpu, unsigned long gpa, bool write) if (writeable) kvm_set_pfn_dirty(pfn); + kvm_release_pfn_clean(pfn); spin_unlock(&kvm->mmu_lock); if (prot_bits & _PAGE_DIRTY) mark_page_dirty_in_slot(kvm, memslot, gfn); - kvm_release_pfn_clean(pfn); out: srcu_read_unlock(&kvm->srcu, srcu_idx); return err; From patchwork Thu Oct 10 18:24:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831069 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 85B71D2444D for ; Thu, 10 Oct 2024 20:25:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Type:Cc:To: From:Subject:Message-ID:References:Mime-Version:In-Reply-To:Date: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=9LTMkDJZxM2S5Qh4t0DfHllWvhtFq1rJ9EqXHTYHy/A=; b=vvwkJjh+MZUA/ErIVF9vyppXgj loWjyiv62YEf6PiW9sdAMfSJrOJ4mLkx2ob787L2fLuhp+4JgyY415eLA5rk0lUWIoi+4sfGDTmd+ rdZU7WtAjwJw3N5/6Tg5J6RW93ZWJItvTCS5nKEFyxlRQk+DnYqDE562dswH/0Xfogmyz1IoRZbVV kUQYdlu/zg326cjkvsPhkE0ZvvcqEAndbZFL1Q67Efs363CZ8t14VYfM5NelSwN+wvWd+2TLgvIm2 J0LNYsrt/J+qdnqSeDTqAqTbFz7WveJFvNwZ0c69AVwNRX0dc+azMC0qyWWKkZ8qOI2zchnb28dzZ FuggUWrQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syzie-0000000EG7J-1n3Z; Thu, 10 Oct 2024 20:25:08 +0000 Received: from mail-pf1-x44a.google.com ([2607:f8b0:4864:20::44a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxsc-0000000DrVQ-1G7o for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:27:20 +0000 Received: by mail-pf1-x44a.google.com with SMTP id d2e1a72fcca58-71e019ab268so1589832b3a.2 for ; Thu, 10 Oct 2024 11:27:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584835; x=1729189635; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=9LTMkDJZxM2S5Qh4t0DfHllWvhtFq1rJ9EqXHTYHy/A=; b=Ezg7/CMWQDMLN8YEdrgAzjhCIggY01A7M2zQRTVFeivnUP9xNYMnDgSn08tTNAQswa kqk7p2xGoJBvlWCJY2F295EpfrTh6Gs382v094DWZ6fCATf7c3raAMlaHVsWLbMuZX8q lhxCBrETU5/+2nt/FLnhY0WsswBeH/2OGGYCpGMOpF9BEDb2/zwz1gDzgmFNd4XSF4Jc yE5+qnvk/nYcYh7E+904AvUOhyHPUho7ahqpJEfAu8ClaWnh4qMEVzY/715tDYQ7YiPf TRld3kwCAO1N9Z5t2zjmuJFVNJrxILimSxqerNKmR1ypmopxODnWp9w7ztgiyY+zhKy6 q3iQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584835; x=1729189635; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=9LTMkDJZxM2S5Qh4t0DfHllWvhtFq1rJ9EqXHTYHy/A=; b=No/wFE+CQQ+ULM819+smAgGeTiwnW4dAKFTw0TXY3kJ/C2eWNS4tjm8eT7g6dfpchB WiOWyU1LsTIUDvaFYvIkLfPZRvQCfbY2vcWYD2Fe6f4mYXCI+EVEVfAlsMPHCpdoppX8 LdftG0GdpXN1syuF/OrMShUzpkgprPO9JLqtnkDV/zBJJEF70WfwxCWQyfLZGrJSFAfB mOcJvKDNMzetm7V/VaznBtiScQkF4yMY3JcnpoYOe5W7Gh9cxjYXH8GM3H3MUGVxdHz5 FxSwaIlZsCaYh0jxGvCZYUvPDIl4YiOzrjePW8e+FBvbohoWyTLmfp7VUakUj0crU+BL Fmlw== X-Forwarded-Encrypted: i=1; AJvYcCXB6XANJR2Zem644DBn1oIkfku2l/ddJFXO9ZpQcKo/pD1oanLIOQ6945A+8BH1uI9W4e2S0p5c0NF+Nm5HEAsI@lists.infradead.org X-Gm-Message-State: AOJu0YzRESSTlfBKHTjrB8to0ppbWVx0Gy6fbSfElZumZsZaHiKVU08l twC4gykj3M0HGMIaRq1/XiPbMLnMwj/yWSeEFMzpIAc43lzBX63wt3GnJPLIZYRkWRso4OyWRdY dIw== X-Google-Smtp-Source: AGHT+IExt7obORK9hFSNzpsBdJKygVAVx/dPvrUuCl2bsyTFt9B1GPfq0Q1aHG/lNRl/yht5gsZcWFEXy1g= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:6a00:8614:b0:70d:fba:c51c with SMTP id d2e1a72fcca58-71e1dbd032amr6898b3a.3.1728584835380; Thu, 10 Oct 2024 11:27:15 -0700 (PDT) Date: Thu, 10 Oct 2024 11:24:10 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-69-seanjc@google.com> Subject: [PATCH v13 68/85] KVM: LoongArch: Use kvm_faultin_pfn() to map pfns into the guest From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112718_485056_BFB42ADA X-CRM114-Status: GOOD ( 14.20 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Convert LoongArch to kvm_faultin_pfn()+kvm_release_faultin_page(), which are new APIs to consolidate arch code and provide consistent behavior across all KVM architectures. Signed-off-by: Sean Christopherson --- arch/loongarch/kvm/mmu.c | 14 ++++++-------- 1 file changed, 6 insertions(+), 8 deletions(-) diff --git a/arch/loongarch/kvm/mmu.c b/arch/loongarch/kvm/mmu.c index 7066cafcce64..4d203294767c 100644 --- a/arch/loongarch/kvm/mmu.c +++ b/arch/loongarch/kvm/mmu.c @@ -780,6 +780,7 @@ static int kvm_map_page(struct kvm_vcpu *vcpu, unsigned long gpa, bool write) struct kvm *kvm = vcpu->kvm; struct kvm_memory_slot *memslot; struct kvm_mmu_memory_cache *memcache = &vcpu->arch.mmu_page_cache; + struct page *page; /* Try the fast path to handle old / clean pages */ srcu_idx = srcu_read_lock(&kvm->srcu); @@ -807,7 +808,7 @@ static int kvm_map_page(struct kvm_vcpu *vcpu, unsigned long gpa, bool write) mmu_seq = kvm->mmu_invalidate_seq; /* * Ensure the read of mmu_invalidate_seq isn't reordered with PTE reads in - * gfn_to_pfn_prot() (which calls get_user_pages()), so that we don't + * kvm_faultin_pfn() (which calls get_user_pages()), so that we don't * risk the page we get a reference to getting unmapped before we have a * chance to grab the mmu_lock without mmu_invalidate_retry() noticing. * @@ -819,7 +820,7 @@ static int kvm_map_page(struct kvm_vcpu *vcpu, unsigned long gpa, bool write) smp_rmb(); /* Slow path - ask KVM core whether we can access this GPA */ - pfn = gfn_to_pfn_prot(kvm, gfn, write, &writeable); + pfn = kvm_faultin_pfn(vcpu, gfn, write, &writeable, &page); if (is_error_noslot_pfn(pfn)) { err = -EFAULT; goto out; @@ -831,10 +832,10 @@ static int kvm_map_page(struct kvm_vcpu *vcpu, unsigned long gpa, bool write) /* * This can happen when mappings are changed asynchronously, but * also synchronously if a COW is triggered by - * gfn_to_pfn_prot(). + * kvm_faultin_pfn(). */ spin_unlock(&kvm->mmu_lock); - kvm_release_pfn_clean(pfn); + kvm_release_page_unused(page); if (retry_no > 100) { retry_no = 0; schedule(); @@ -900,10 +901,7 @@ static int kvm_map_page(struct kvm_vcpu *vcpu, unsigned long gpa, bool write) ++kvm->stat.pages; kvm_set_pte(ptep, new_pte); - if (writeable) - kvm_set_pfn_dirty(pfn); - kvm_release_pfn_clean(pfn); - + kvm_release_faultin_page(kvm, page, false, writeable); spin_unlock(&kvm->mmu_lock); if (prot_bits & _PAGE_DIRTY) From patchwork Thu Oct 10 18:24:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831070 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 034F9D2444D for ; Thu, 10 Oct 2024 20:26:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Type:Cc:To: From:Subject:Message-ID:References:Mime-Version:In-Reply-To:Date: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=Sag0fj8OwucY2qRogRu5ELvjWktNcy71jZ/+qgEdHaE=; b=pFRGpIzA+G4CNK/sxEZSHOXDzR +foAdENe2slcy1+iR+3j1EEiViZ4VmZUo/fiqOZD9iGnSa1NKpSzG6QtOrosTVJOx7mbxSlzZeeiW Ro3cXzNransgMZnATknP9eaSacJYRBTLEaq1YqUih/hLTUp4rprJRP37CzoZKw8iyLUv39W5bsNg8 HXNy/IEck6NM/3OWnGmvXF2YQxj3eYNzhlXRMKlHHYegzIhTIaB+JKJnBht1ykSNSLeR3PWMsPcaf nvSw7T3pudKrfSRHN7xBmnOUAkSD9tG2uS5KvVUJudZoy/mEMmaTrXvHqddFoY/RJVBw5wGQstJqN 8Hq3XzGw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syzjx-0000000EGGN-3eEs; Thu, 10 Oct 2024 20:26:29 +0000 Received: from mail-yw1-x114a.google.com ([2607:f8b0:4864:20::114a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxsc-0000000DrWB-3n4M for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:27:21 +0000 Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-6e284982a31so22465637b3.3 for ; Thu, 10 Oct 2024 11:27:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584837; x=1729189637; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=Sag0fj8OwucY2qRogRu5ELvjWktNcy71jZ/+qgEdHaE=; b=E2zHzaRZqpx7PmmOp3LUhoHU9Dr1MkBl5G0WTprILSNA1mq0tHxEkTsU10v77+QvpN 0zFwtgeL7W3gLusd7cDl6EFytNV+h4xDtKwQ84UL7e0cR2cdNgj3Dnms8wEJu/GQeCw2 rt7R7dMOceITVKPIgq5JWL/qwK3SfMqFQ2pgXkciOlQiuO16Q7TJPnLvr3j8HJGzk/g/ PNbNvTkYqJyNtEyeLClT7pwiry+CiUZP34/omc1IrYUyQZyp2bxaEbPY4j91F8t3KOyK XOI9XrCQ/PvTIRVwt9Rf9AqfV0T7By8bPGHJYXiy5nTIL84TTZYDMKUFvEYpQPHXGEzJ 9snQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584837; x=1729189637; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=Sag0fj8OwucY2qRogRu5ELvjWktNcy71jZ/+qgEdHaE=; b=qhCqtIQW2JNGLQq7o2IsaPsEc5n7XSIMMBGw7pY0Oz3reuv4bxD3OULbXYnJBVbtlk vQiQyCdMPmvHLrUXv7M/CmPsCq6m2Ha2LZ3E8L9IM+13Juq+noLdfjlhTJr2MQMXFbzT ou3+5RwkA36/BEnxSdqusRW9p1zIDFQ7lcSpl4FPEjARtoB3NsMsPNnnAsEl8iQfoGyn SnPQ/ChNbqz762FE5I16SZoW0GihDma6dpdMpmFVwUSbu5gQTSDoFPnZGjxwxRqqWCtL my/Qw4lZ/3wiC2ggAL7yyo1+orPkrDsM+JckJKjDA0D4PGeMDUQRZMEwT/t5w8UepO3p QJBg== X-Forwarded-Encrypted: i=1; AJvYcCVX0HaFWC/hGYM7zmfE51z5e5OsXr2i5+XuL2RKDPAow8TaM1y6lMfmRigFQhs2ku90rkd6LeDtMi9HiNWR+PMO@lists.infradead.org X-Gm-Message-State: AOJu0Yyc1TorAv2qQOkfTA0OWj3iGo+FJNkG3vXQ05/sYCR7uY4FRZm9 8VJP59bdtsTEPQ2QkqgSbeWDqCag2Z3gBL0eXffP3DvnGKfiIZuiQgQkUuk7MswbpPl8U05p7MI nsA== X-Google-Smtp-Source: AGHT+IEjJriLMGICq9yty21Fx3Z3FQyTG0XCg9lS5qhc+MvTVoRASG8drmw0OxSoWZyAtrQmM3pJDo8jQYk= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a5b:a4c:0:b0:e28:f454:7de5 with SMTP id 3f1490d57ef6-e28fe52740fmr75826276.6.1728584837458; Thu, 10 Oct 2024 11:27:17 -0700 (PDT) Date: Thu, 10 Oct 2024 11:24:11 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-70-seanjc@google.com> Subject: [PATCH v13 69/85] KVM: MIPS: Mark "struct page" pfns dirty only in "slow" page fault path From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112719_142882_3A9A7A43 X-CRM114-Status: GOOD ( 11.15 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Mark pages/folios dirty only the slow page fault path, i.e. only when mmu_lock is held and the operation is mmu_notifier-protected, as marking a page/folio dirty after it has been written back can make some filesystems unhappy (backing KVM guests will such filesystem files is uncommon, and the race is minuscule, hence the lack of complaints). See the link below for details. Link: https://lore.kernel.org/all/cover.1683044162.git.lstoakes@gmail.com Signed-off-by: Sean Christopherson --- arch/mips/kvm/mmu.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/arch/mips/kvm/mmu.c b/arch/mips/kvm/mmu.c index c17157e700c0..4da9ce4eb54d 100644 --- a/arch/mips/kvm/mmu.c +++ b/arch/mips/kvm/mmu.c @@ -514,7 +514,6 @@ static int _kvm_mips_map_page_fast(struct kvm_vcpu *vcpu, unsigned long gpa, set_pte(ptep, pte_mkdirty(*ptep)); pfn = pte_pfn(*ptep); mark_page_dirty(kvm, gfn); - kvm_set_pfn_dirty(pfn); } if (out_entry) @@ -628,7 +627,6 @@ static int kvm_mips_map_page(struct kvm_vcpu *vcpu, unsigned long gpa, if (write_fault) { prot_bits |= __WRITEABLE; mark_page_dirty(kvm, gfn); - kvm_set_pfn_dirty(pfn); } } entry = pfn_pte(pfn, __pgprot(prot_bits)); @@ -642,6 +640,9 @@ static int kvm_mips_map_page(struct kvm_vcpu *vcpu, unsigned long gpa, if (out_buddy) *out_buddy = *ptep_buddy(ptep); + if (writeable) + kvm_set_pfn_dirty(pfn); + spin_unlock(&kvm->mmu_lock); kvm_release_pfn_clean(pfn); kvm_set_pfn_accessed(pfn); From patchwork Thu Oct 10 18:24:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831071 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6C477D2444F for ; Thu, 10 Oct 2024 20:28:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Type:Cc:To: From:Subject:Message-ID:References:Mime-Version:In-Reply-To:Date: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=KbunjaDKwfDzESX7bCyZnVXI4Jj60WAsli6cgjlK44o=; b=AEog2PvZjtwXIjtl/59HiurQO7 vd7kNarZwNQMTyHExssjAUGWCG5UwPyn08vBHQvft10yNmDet3vRnPbI/M5wooLbrF+OnowDnUTDD tnzr1DlHI/rdCnTuv+IzeUNWJBuym0HJ71fWEPK+fjUrualIZQmKIdXurVKp9mYUxMpUi9dg7MOC8 JuPf7JZiP+6t4pkQZ9aD2qzeXAmLR0IJuQgS440z6DAEG/r2FFNJpwRbPKvwnkYITCXDM34Vesuw5 rl2V6YBDFl9hbQGJq40GQhQC6IlbdD2nD6k7eQGYht4aBjtv5jFRDxm8m4XNoBCKy8q9PaeuQ2hUf sEU53jCQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syzlG-0000000EGOX-2Gsb; Thu, 10 Oct 2024 20:27:50 +0000 Received: from mail-yw1-x114a.google.com ([2607:f8b0:4864:20::114a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxsg-0000000DrYz-2BT6 for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:27:24 +0000 Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-6e22f8dc491so24503007b3.1 for ; Thu, 10 Oct 2024 11:27:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584841; x=1729189641; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=KbunjaDKwfDzESX7bCyZnVXI4Jj60WAsli6cgjlK44o=; b=d36EgFFD42cTaH04fi/y4xFv+qp2ifFMxUrP1SQnXseIpwdmnbEzTQ8DSdY5otquib Fe4VayuL6/R0aasi9i7aCCPV4Gv1zlkR0JHNaWNpB/Uu253T6upc+xiprDa9B/mJybKL 77xr20zI26cg8Mq8sN8ZrtHXoNuERM9pAq2+ZDWuy6zeA9+GuTt7xnLiNbaJwQKdPD2U 8iRnxJv76mYNSesvHPOFSlccB6NnI7KLu7OUcwUJa63Q2yECwrEM4nizOHw+DX60vWLd /6jJWIBZt18OZ6zN0L0nZtncrUpAsn3nxdPUEj281wfWOhAwbAmyb2k5lD1F9+UTfMxZ W5xg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584841; x=1729189641; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=KbunjaDKwfDzESX7bCyZnVXI4Jj60WAsli6cgjlK44o=; b=hR+0NArmRX0LG4+cpz1+qmEhPqGF9RT/q+FBWETpVxwFzB3vEEd8R6G9sxjq7UHHj+ gPS9M5j/mXeNOsF0Xt4RVnpsntzRD5LxzPHYcZh8yHULa73sbBzYj+idiKUETU3m3Yhi QFNV1SFieC64Zx3nWI1VT2HU9A1HKDRLbwH7sEBX3wIIyunQ5GDdb692XcJwjX9EGzMO 5Bk++7HqFsi8VGZs3ikK9jC6HUqV4+r7Tn1t6Z6Il26A8ryHVBe5LGypgZ42IGJI8TKj Tn+dQ1uCERVb3+hQQ92YQchTK2rTR7XTiC+yVEmLCI+yrXaJror9ZqpEHmB/Sp05IzL8 Trog== X-Forwarded-Encrypted: i=1; AJvYcCVKkKiQd+bT2vYm8CBTLVjwt7Ql87cLotFerTTFEPHt8KyuM9m+RusEfmNg68J3CZjXlVtDtJW40eAhsc00Ui6p@lists.infradead.org X-Gm-Message-State: AOJu0Yy8eCJmqtLyFkyi73FFMJIJ0WTemFFzhDoIdusN7c/bMbq1HUn/ u7Z6VpsYe8il6h6Ud+DNNdRLrh5UsxIHPiiWRnvQKEcPReX5In2kH3JdA4l1QDNee1P7V8jiNYr LXQ== X-Google-Smtp-Source: AGHT+IEbamZE/KJDJ4kw63fZFZN0PThnG54xKd82AQ09AwFkx3KIodEPvEqsDBHZyT6DCHkY/GuTD4px7aQ= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a25:6c05:0:b0:e24:c3eb:ad03 with SMTP id 3f1490d57ef6-e28fe540170mr1289276.10.1728584839383; Thu, 10 Oct 2024 11:27:19 -0700 (PDT) Date: Thu, 10 Oct 2024 11:24:12 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-71-seanjc@google.com> Subject: [PATCH v13 70/85] KVM: MIPS: Mark "struct page" pfns accessed only in "slow" page fault path From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112722_648024_9E66034C X-CRM114-Status: GOOD ( 13.72 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Mark pages accessed only in the slow page fault path in order to remove an unnecessary user of kvm_pfn_to_refcounted_page(). Marking pages accessed in the primary MMU during KVM page fault handling isn't harmful, but it's largely pointless and likely a waste of a cycles since the primary MMU will call into KVM via mmu_notifiers when aging pages. I.e. KVM participates in a "pull" model, so there's no need to also "push" updates. Signed-off-by: Sean Christopherson --- arch/mips/kvm/mmu.c | 12 ++---------- 1 file changed, 2 insertions(+), 10 deletions(-) diff --git a/arch/mips/kvm/mmu.c b/arch/mips/kvm/mmu.c index 4da9ce4eb54d..f1e4b618ec6d 100644 --- a/arch/mips/kvm/mmu.c +++ b/arch/mips/kvm/mmu.c @@ -484,8 +484,6 @@ static int _kvm_mips_map_page_fast(struct kvm_vcpu *vcpu, unsigned long gpa, struct kvm *kvm = vcpu->kvm; gfn_t gfn = gpa >> PAGE_SHIFT; pte_t *ptep; - kvm_pfn_t pfn = 0; /* silence bogus GCC warning */ - bool pfn_valid = false; int ret = 0; spin_lock(&kvm->mmu_lock); @@ -498,12 +496,9 @@ static int _kvm_mips_map_page_fast(struct kvm_vcpu *vcpu, unsigned long gpa, } /* Track access to pages marked old */ - if (!pte_young(*ptep)) { + if (!pte_young(*ptep)) set_pte(ptep, pte_mkyoung(*ptep)); - pfn = pte_pfn(*ptep); - pfn_valid = true; - /* call kvm_set_pfn_accessed() after unlock */ - } + if (write_fault && !pte_dirty(*ptep)) { if (!pte_write(*ptep)) { ret = -EFAULT; @@ -512,7 +507,6 @@ static int _kvm_mips_map_page_fast(struct kvm_vcpu *vcpu, unsigned long gpa, /* Track dirtying of writeable pages */ set_pte(ptep, pte_mkdirty(*ptep)); - pfn = pte_pfn(*ptep); mark_page_dirty(kvm, gfn); } @@ -523,8 +517,6 @@ static int _kvm_mips_map_page_fast(struct kvm_vcpu *vcpu, unsigned long gpa, out: spin_unlock(&kvm->mmu_lock); - if (pfn_valid) - kvm_set_pfn_accessed(pfn); return ret; } From patchwork Thu Oct 10 18:24:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831072 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id CD087D2444D for ; Thu, 10 Oct 2024 20:29:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Type:Cc:To: From:Subject:Message-ID:References:Mime-Version:In-Reply-To:Date: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=OZ/c8QPJoSxQXGCWT5aWp1nMy1t7EEzE7+N8fqq+BFY=; b=MRRytUybmQHJt5D7SZ26+jaVNp 3WRhsjohqEkZPon8QhpdnhwBV7AI1ac9MqpGI0PGO1o3SXiQDYtKQPQIrt1bNV6Uh5gg4LeW9LiqP joWYcI/1EXkNrOwV8y9O0uBRrVfgWJ4hhDGQaJuhArFAIteeP1yqUVbPLBdSxYWKwBuwOK/utCukI kO4hvg0sx6KDUiaIRI6NOU0Alt4m5HDQPFfFvLwnmbs4GIZmC9cCkwAvXHphHA1Fbg80VCjngZIL7 zLoIamyesuRLzadIJZw7BiUw1MH6yyuJXlqgbmD8a9WroQ+8Z7OWMS2tFCBt+iWVamfPHuwfEznFM /EJVcU9Q==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syzmZ-0000000EGXc-2UbU; Thu, 10 Oct 2024 20:29:13 +0000 Received: from mail-yb1-xb49.google.com ([2607:f8b0:4864:20::b49]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxsh-0000000DraG-2ywl for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:27:25 +0000 Received: by mail-yb1-xb49.google.com with SMTP id 3f1490d57ef6-e0b8fa94718so1790493276.0 for ; Thu, 10 Oct 2024 11:27:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584842; x=1729189642; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=OZ/c8QPJoSxQXGCWT5aWp1nMy1t7EEzE7+N8fqq+BFY=; b=v6KPJlqYJR4mGmcmL/AsMF84scs4V0oyq2QlyNfixKKzIATDSNUV+0XlnkYG0wXdfG 1TGlx8Q3aEM7vHjZJ7j9Aff5MlUcPnC5NpHxe2emoEMK6I3xeIS+wrEnMFX0gGMrKMd6 3o+8Tcuf41G1Rd4uU3ujqDhOcby0NH+Z3wn6chCUf85Q1DZyOLbb/v5kRcBVgBP2tXFX wAs/sI95wXa968+EifVDV5rU72Gj02WH3yR+CxwZG+eE0WG8Dd7khjpUpzuT79kEnRBO BRi/5p+4SSJNd/ZT54cjZaN3WBY1yijvzhZ2QmbtJmCAtVIifH/3gMdwG9OHl/0vDIL0 j49A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584842; x=1729189642; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=OZ/c8QPJoSxQXGCWT5aWp1nMy1t7EEzE7+N8fqq+BFY=; b=nojopnNWwuQjmnsE6IWaAwVr1Ytft55m6feEXOloEhEy+j2LZYVN/Uqu6GGNihFCvl 7S7xZs76qMdQkXUMn57FK5IeB/eVGxp7HmCyTGDaaGoTOoPsQQfV3JNtxUV0KoZUBEHS iMVb+40w+aKCSMgKkbhmk02IV358x/010ajLum99kPPZ87pIgV27TBrig7rhtefGYQVO a92sJOKjwyYET1ttmsFhZJa8yJyQ6GdzDQqKDaQPodkL8hnMIaWM+U5V4/2x4fwitYBH ssETvyrzmkzBAKsIEQJssMHoI4dBocc02RUtil3NJKph2YmkHWwNCS2Kiz/0d44yWAaS OC/w== X-Forwarded-Encrypted: i=1; AJvYcCVjmkJ82BU/Y1XPiU6L+D4nAE/lIZcQ5WPlATEOfQPX3ZgaIjLbxrVDRhYm4HgimH51EQ2rXsfUTe8zH/0NUNWh@lists.infradead.org X-Gm-Message-State: AOJu0YyvU888aIp14ZXXV6VqimvPJAhKgtQmL9CZx6fjj0tABmp1ZNH4 v9Cm1wHSFJaT+lQDe6c/46yTK2ss2HofXok+1WFEGKQVTOqHsaUZXryvOdpEQEV9pJWrknTy9hj CZg== X-Google-Smtp-Source: AGHT+IHhD+Vk6TwFIS5v8fA24SNPRWU4vCcQM+1g+zVw6L+6pfrMpIdoNti+PehtHCeQDGmk8Jps/9lQ5Ek= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a25:d30a:0:b0:e28:fe07:9cc1 with SMTP id 3f1490d57ef6-e28fe4a3a81mr68085276.3.1728584842321; Thu, 10 Oct 2024 11:27:22 -0700 (PDT) Date: Thu, 10 Oct 2024 11:24:13 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-72-seanjc@google.com> Subject: [PATCH v13 71/85] KVM: MIPS: Mark "struct page" pfns accessed prior to dropping mmu_lock From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112723_910008_413D9E68 X-CRM114-Status: GOOD ( 10.18 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Mark pages accessed before dropping mmu_lock when faulting in guest memory so that MIPS can convert to kvm_release_faultin_page() without tripping its lockdep assertion on mmu_lock being held. Signed-off-by: Sean Christopherson --- arch/mips/kvm/mmu.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/arch/mips/kvm/mmu.c b/arch/mips/kvm/mmu.c index f1e4b618ec6d..69463ab24d97 100644 --- a/arch/mips/kvm/mmu.c +++ b/arch/mips/kvm/mmu.c @@ -634,10 +634,9 @@ static int kvm_mips_map_page(struct kvm_vcpu *vcpu, unsigned long gpa, if (writeable) kvm_set_pfn_dirty(pfn); - - spin_unlock(&kvm->mmu_lock); kvm_release_pfn_clean(pfn); - kvm_set_pfn_accessed(pfn); + + spin_unlock(&kvm->mmu_lock); out: srcu_read_unlock(&kvm->srcu, srcu_idx); return err; From patchwork Thu Oct 10 18:24:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831308 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 46FE5D2444F for ; Thu, 10 Oct 2024 23:31:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Type:Cc:To: From:Subject:Message-ID:References:Mime-Version:In-Reply-To:Date: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=ojyPU0tzKIRRbswoqL6JjpEgqRgu1FFXpokE5rqa8U0=; b=3Us4cB+c4fgy5QhxGvIeIkhWQU QmSSoK4FGDUhEuzSKhPAchBeczYkf81Ss00F0afhOQXxKhaRb7m+hpku5iQ5sPICoJk4j8qTAZJhv di3+PYIC3GHAbB7OLEpyKtem+TC9k7wdsMvFMw8o9TUGlCqjUngvCguZQZF4ozMCCv2nx5HfzZ7H1 cWrdZj/04Vo7NLK2TVHWMXoRHFbBOSLbv9mc7lm4UYKjwkclbv5NRUzngDXr0sN2OVAdRcI9ptJVN C/lRxUwLWv7EqM10ABV9DcPmo1qbLu8fEh5oW3mRPCBqUwoLtRlVtzVvu7RZKxaSr36H6Bv24vg+r PrkZGgsg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1sz2d1-0000000Ef5c-3po9; Thu, 10 Oct 2024 23:31:31 +0000 Received: from mail-yb1-xb4a.google.com ([2607:f8b0:4864:20::b4a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxsj-0000000DrcG-3sIp for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:27:27 +0000 Received: by mail-yb1-xb4a.google.com with SMTP id 3f1490d57ef6-e2913beeddaso635561276.0 for ; Thu, 10 Oct 2024 11:27:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584844; x=1729189644; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=ojyPU0tzKIRRbswoqL6JjpEgqRgu1FFXpokE5rqa8U0=; b=OAZSNh4NZXQ/Zipvt47P/0efWGRR16annLUzUsTQgPtiN5G+fdWQUGTrVxyFBmqnhB 5AwMcO0uoBvpg7YhIua7lfEp5t/Wdpj3/l3GdjJ7VGRuLVwzna5mFAu5gcMnBfIGaUJv J6ZtjS5V8iY78UnacO6paUXR+XjxiokqAg6xpYBPOjRZbJJ2/OlFErl6lNLT3KX2S1eP I8AQ8GkmS6CRQMk3isu+HpdKOIFg7PcvPLchk2swDOijcWMRHLjAwION6+UAevFGARbl XMgILnZ8FZ3qaVR2BJLppR7eXDvA4spUXJRz/lQJetwc+AO7yz1h7BktBMonH7m3/0Ta 6SNA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584844; x=1729189644; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=ojyPU0tzKIRRbswoqL6JjpEgqRgu1FFXpokE5rqa8U0=; b=cGeUflukaaSSdXrZWbGXWCQPOct3oDX+XAvAjQD3PF8EIcGKSLiL6jo6hj5rwD8bbX Nkil84yBOf0BWv8NdGclrERWxBE9dfIqiYaa8kPBQA1PXUTvIUtDBD+Ay8JyBLMdNntl 3sXxMe6MKJVVrQfNOEIg3qhaQSp5lvfq3nxPKLQ0d3Hv709BsRxZK801mRZ9ODeMFKB7 OQYAj3v2i13j1JvY8PWRTiW5bYDIpM3Uzip3oSWZX8m9qczozvtlWwRnESVLIWOFNn6u gkAc4wpDKvVN++IKnEDy352fiOwVHZiW1tpoQkmAcpuDZleyf22UXkG9kJEjpXkHdRk/ 9hsg== X-Forwarded-Encrypted: i=1; AJvYcCWv/8j2z3gCiySEiHuHuJKlbEvB+ZJL5CGAjWzzQ+PBfsuKAMw2Ae4KxZkQLhzqU6L+ppxisii0MAeP2kYxlnXs@lists.infradead.org X-Gm-Message-State: AOJu0YxOrEkNa6IqNPrCIwAffFTRFtdVytnQGCUP55OSSZCE+zlfdojW Np5s75A6Z+W2bDDdQMbHM0HVNRT3/XfgRhEBV0IcOi8py29/etb/UkJ4Cp/3KcnrU6GKq7yh72t FSQ== X-Google-Smtp-Source: AGHT+IHEyKrdGSoNbMWcRNiKhgC/jPgVf/TjauaPCrq7yqhpXyuJQagleroMC0w40O7lZsOojHLr7EwrZqI= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a25:6805:0:b0:e20:cfc2:a326 with SMTP id 3f1490d57ef6-e29184333b9mr26276.6.1728584844184; Thu, 10 Oct 2024 11:27:24 -0700 (PDT) Date: Thu, 10 Oct 2024 11:24:14 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-73-seanjc@google.com> Subject: [PATCH v13 72/85] KVM: MIPS: Use kvm_faultin_pfn() to map pfns into the guest From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112726_141513_6EFC3FEF X-CRM114-Status: GOOD ( 13.62 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Convert MIPS to kvm_faultin_pfn()+kvm_release_faultin_page(), which are new APIs to consolidate arch code and provide consistent behavior across all KVM architectures. Signed-off-by: Sean Christopherson --- arch/mips/kvm/mmu.c | 14 ++++++-------- 1 file changed, 6 insertions(+), 8 deletions(-) diff --git a/arch/mips/kvm/mmu.c b/arch/mips/kvm/mmu.c index 69463ab24d97..d2c3b6b41f18 100644 --- a/arch/mips/kvm/mmu.c +++ b/arch/mips/kvm/mmu.c @@ -557,6 +557,7 @@ static int kvm_mips_map_page(struct kvm_vcpu *vcpu, unsigned long gpa, bool writeable; unsigned long prot_bits; unsigned long mmu_seq; + struct page *page; /* Try the fast path to handle old / clean pages */ srcu_idx = srcu_read_lock(&kvm->srcu); @@ -578,7 +579,7 @@ static int kvm_mips_map_page(struct kvm_vcpu *vcpu, unsigned long gpa, mmu_seq = kvm->mmu_invalidate_seq; /* * Ensure the read of mmu_invalidate_seq isn't reordered with PTE reads - * in gfn_to_pfn_prot() (which calls get_user_pages()), so that we don't + * in kvm_faultin_pfn() (which calls get_user_pages()), so that we don't * risk the page we get a reference to getting unmapped before we have a * chance to grab the mmu_lock without mmu_invalidate_retry() noticing. * @@ -590,7 +591,7 @@ static int kvm_mips_map_page(struct kvm_vcpu *vcpu, unsigned long gpa, smp_rmb(); /* Slow path - ask KVM core whether we can access this GPA */ - pfn = gfn_to_pfn_prot(kvm, gfn, write_fault, &writeable); + pfn = kvm_faultin_pfn(vcpu, gfn, write_fault, &writeable, &page); if (is_error_noslot_pfn(pfn)) { err = -EFAULT; goto out; @@ -602,10 +603,10 @@ static int kvm_mips_map_page(struct kvm_vcpu *vcpu, unsigned long gpa, /* * This can happen when mappings are changed asynchronously, but * also synchronously if a COW is triggered by - * gfn_to_pfn_prot(). + * kvm_faultin_pfn(). */ spin_unlock(&kvm->mmu_lock); - kvm_release_pfn_clean(pfn); + kvm_release_page_unused(page); goto retry; } @@ -632,10 +633,7 @@ static int kvm_mips_map_page(struct kvm_vcpu *vcpu, unsigned long gpa, if (out_buddy) *out_buddy = *ptep_buddy(ptep); - if (writeable) - kvm_set_pfn_dirty(pfn); - kvm_release_pfn_clean(pfn); - + kvm_release_faultin_page(kvm, page, false, writeable); spin_unlock(&kvm->mmu_lock); out: srcu_read_unlock(&kvm->srcu, srcu_idx); From patchwork Thu Oct 10 18:24:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831206 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 672E7D24454 for ; Thu, 10 Oct 2024 21:47:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Type:Cc:To: From:Subject:Message-ID:References:Mime-Version:In-Reply-To:Date: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=t4DEWxyHdS8FH8bmPlxV35CavnLWtw4MDWneo+GIfaw=; b=I6JnTaeP75QW4aDlgrqWWZ5Bke Eyt3tJRE+foqYhHtApoiesMB/fU6loXwXv7WoVP1M79E4LgudCQUKAtkgCgoojUZpVPRYCs1bpa0V LXuTL0v0od2iqBb1C1ArZIqXAIMwj6HbZKqfLBDTusNZxui+cvvCa7kV/095TDx6C4oLuvgK/p0fC Yo9W1yW5nj3gr9hlk8mjR7Jrcee/Noy/0THXg4M+yv1QxSkHrsFQD0ioZWKB7H175BI4evx54KHgI JiP2GOE/Jcc7j5vnvRIggHepCuU+eJYPvOnEqrAljwa+qFOkqsty8ctYMFfPuuFMopzbmU0T+1TXC gucq+dFQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1sz100-0000000ET41-0dac; Thu, 10 Oct 2024 21:47:08 +0000 Received: from mail-pg1-x54a.google.com ([2607:f8b0:4864:20::54a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxsn-0000000Drep-22Ob for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:27:31 +0000 Received: by mail-pg1-x54a.google.com with SMTP id 41be03b00d2f7-7ea0bf14523so1033826a12.1 for ; Thu, 10 Oct 2024 11:27:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584848; x=1729189648; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=t4DEWxyHdS8FH8bmPlxV35CavnLWtw4MDWneo+GIfaw=; b=3qI8Gw0xZEjRCMSlShvB9CxPYVE+sQUqWY+2lQ3x/hsrogq9AlmlMY0W+xIiiyB2+G ivumflW0xs/76E0l8I7fd7bbDdyPLteemqZYYQ+ZKK/YPblG/xVoZuzgyJoRX97tQfwU YIGnX9P9pbtCRUkjnKrNiuDGenR/Sjm4DPbwNWZAbMadx+iMNdYcHOHirra/2+DNgg7J SV89gZ2MqD3S2NzVbA5KhNlU7zbQKrI9AUvsp4uNOzuM55xfH+GZz5OMQp029NBhBRbT 1kBG/UEPxC0x7ucQl28rHGPkGDnEsOsIC7omLOZgZouSoX8ld20Qe3dCRqoJaXRUVo6j ZO5w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584848; x=1729189648; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=t4DEWxyHdS8FH8bmPlxV35CavnLWtw4MDWneo+GIfaw=; b=ISAI/hB67sOfuQ2yUPvYJAIFaPouXPsnz4erHXabM481MD8IlFD5KbDzZYg0ZsfKxF 6wy5s5diKvQZpq50TYG+ffMu75DjY1ba0oEFAnjm0yiuzhAKTfp5k0p8auB616+rXx6U bNu2FKIO2K/QCR7Lwrt/thbODDQ+HVp8Gpcpq8esaOaQOZK6hDb6gHiHtEGWuQBUcQ5r gvyb7FjoupLkLjeT9CnL8dOj/iN6P401KrgV4XUnShuMEhRVT/IgPvsYhLBsQxhJnuSA jJXWtj2eXeW5XDD4rxFzK7mI73HRxRCge/lYSf740B6lT2GDT4WBKIKachMU5w/xjRY4 /pmA== X-Forwarded-Encrypted: i=1; AJvYcCU+KJYaw2OT5TZUsLvoASWjh0exmLQE0Ke45MVY0qkH0Vn+rkX2PYnOan8fOovQzlfQZr3gjtVK0tSr1RA/bULs@lists.infradead.org X-Gm-Message-State: AOJu0YzXorGyiEuz1UYQ7wnj8A2ae2NP+jiwL51rYYZwZEJq0xJkCbF/ 2/owx/+2qlIs/w8uK/599tB/OPz5TcdjGR818Io3KnrT3Y1NNyxrUgAN8GZUPFsUDsie4X7/2YQ n0A== X-Google-Smtp-Source: AGHT+IHU1NaHt28BJAfE9+Eihx21gIRaHWD3wJUzNDiwOfeaS4l1n7K6hdxyKTp49hAFCJqVbhS/fDDFwRQ= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a63:2545:0:b0:7e6:b4dd:fc0d with SMTP id 41be03b00d2f7-7ea535b0baemr20a12.7.1728584846518; Thu, 10 Oct 2024 11:27:26 -0700 (PDT) Date: Thu, 10 Oct 2024 11:24:15 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-74-seanjc@google.com> Subject: [PATCH v13 73/85] KVM: PPC: Remove extra get_page() to fix page refcount leak From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112729_718982_4DE36885 X-CRM114-Status: GOOD ( 10.29 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Don't manually do get_page() when patching dcbz, as gfn_to_page() gifts the caller a reference. I.e. doing get_page() will leak the page due to not putting all references. Signed-off-by: Sean Christopherson --- arch/powerpc/kvm/book3s_pr.c | 1 - 1 file changed, 1 deletion(-) diff --git a/arch/powerpc/kvm/book3s_pr.c b/arch/powerpc/kvm/book3s_pr.c index d7721297b9b6..cd7ab6d85090 100644 --- a/arch/powerpc/kvm/book3s_pr.c +++ b/arch/powerpc/kvm/book3s_pr.c @@ -652,7 +652,6 @@ static void kvmppc_patch_dcbz(struct kvm_vcpu *vcpu, struct kvmppc_pte *pte) hpage_offset &= ~0xFFFULL; hpage_offset /= 4; - get_page(hpage); page = kmap_atomic(hpage); /* patch dcbz into reserved instruction, so we trap */ From patchwork Thu Oct 10 18:24:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831004 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0CF91D24442 for ; Thu, 10 Oct 2024 19:46:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Type:Cc:To: From:Subject:Message-ID:References:Mime-Version:In-Reply-To:Date: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=DOlbrv8SiXT4cHz79KGvl4RMUSE/a2Kzzqa4ZWuWNYk=; b=yqITPkMwrbBio30PZoPveQhWUI 16nHx21ilAyUdzVvLVQOD6Kco6qHY8mXTl4W0kKVQwPKbi3DqAheNDU3Q/0ZxikQnHAP+tgTn6mF+ Pei6BpyNVtogk60bDT88iP1WmaDzQkcAAhai3fm1ADTUAo4CUIpWBHB8Cp79RqWbvqsdveh8nu03u bthnvf2qdoaC0pQiLsINrINE8Q03X0zV375oA4cTCC9bau9kVvwmFDlA4eVY/DiHvfTdeiswcTw+W 1mO93M63wth8OFUBTw5NwV/ebcSu0zPJiJgHHDd2vsqQ5UMI/PKSkNzvZMt6ajdTeqQQsNWsi7olK Ap6RByLw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syz6u-0000000EAeK-16Cd; Thu, 10 Oct 2024 19:46:08 +0000 Received: from mail-yw1-x114a.google.com ([2607:f8b0:4864:20::114a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxsp-0000000DrgS-1NgQ for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:27:32 +0000 Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-6e2270a147aso21797327b3.0 for ; Thu, 10 Oct 2024 11:27:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584850; x=1729189650; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=DOlbrv8SiXT4cHz79KGvl4RMUSE/a2Kzzqa4ZWuWNYk=; b=WG3oPKOSidsrkm4ZH+Q4EkkXjyWwVnO477YpGgUaQXbbCDGKf93WFRJzL3OwJpofqf p8bdC1jeQCqxkvH9UEPZCI4TUBUrl11FjPcF0Jg+56xJraONLFpIh4p8QB77QEpzyASg WVsvq1N9adyKBs7THN8PQgi2x2Rkj4Fo63J7UR0df8h9znJyqu5xPmc1cvtbMBbD4Xom up1kKmglzcsU9NWDYJmdhKbHEZKKESlhu0NvtbrxieUxie5gb9cciF1QAZfHOGbJGhuE kLWFwIdmEyWnDLNui2wg1ranKldu2obulk+W9zlMSWf7qQsfQ4M5Jphmr1wnhyoiFjiT ia4g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584850; x=1729189650; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=DOlbrv8SiXT4cHz79KGvl4RMUSE/a2Kzzqa4ZWuWNYk=; b=w3EhioOAyAFNPoZhYTfGzP7SwFpIdFzR5lxFS/kCdofCgy4f/qc0u4lRn/GT5b03RV zwUXzKrv69RtKWBktZUo+Z+uDilPSaCoGpfmlFa9s7W9tumxNy9s4JXruMVRUFTMBy+g g94z+7/cpiseIMzknxWQbPxnno0o6zVZoQgFgPHchWkFq+baOmH2WgiS0BPBebpZsh6/ GxAicq5QdnyaSwmIqMf8v8Pun8ae6oAobjcqYXQ4Bk6YFNzaVbCWLEqewb9Uiru1FMSd TXyw6aYCBVrj7SB328oiPMaW82oHqv7qN7A6oxQBSRsITX4jl3RFwkTiD/6lSW9lVHyc lGpg== X-Forwarded-Encrypted: i=1; AJvYcCVVFX8YTnjSYfWkvJ7cxj5/ZBUPvkeLXkb/Zt5vRlCQQVKYvRSShWPTuzHtMffdkHgskclG3ovncCn1WCNnmf0o@lists.infradead.org X-Gm-Message-State: AOJu0YypV73yKT2Ji4YFe5V6sWv8iXAJZKR7pvfuFUTGYIiJx7RG9eoF 7N4L5gNsRVq9LqC2fJmscCi1reZGucklzfpbmMNC3SqwWWkN6QIIeFshpdaT630JomACXmskTHl aJg== X-Google-Smtp-Source: AGHT+IFYivPtvbNeJBK5QJ/MFgPKbd4XQ4GDAEMeTnfD7MuZnOh5ZwxKPEpX2RNQ0xKTZSwXcPzTQuagswA= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a0d:f043:0:b0:6db:c6eb:bae9 with SMTP id 00721157ae682-6e3220dbc31mr798327b3.2.1728584849896; Thu, 10 Oct 2024 11:27:29 -0700 (PDT) Date: Thu, 10 Oct 2024 11:24:16 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-75-seanjc@google.com> Subject: [PATCH v13 74/85] KVM: PPC: Use kvm_vcpu_map() to map guest memory to patch dcbz instructions From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112731_419387_88939AFA X-CRM114-Status: GOOD ( 13.27 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Use kvm_vcpu_map() when patching dcbz in guest memory, as a regular GUP isn't technically sufficient when writing to data in the target pages. As per Documentation/core-api/pin_user_pages.rst: Correct (uses FOLL_PIN calls): pin_user_pages() write to the data within the pages unpin_user_pages() INCORRECT (uses FOLL_GET calls): get_user_pages() write to the data within the pages put_page() As a happy bonus, using kvm_vcpu_{,un}map() takes care of creating a mapping and marking the page dirty. Signed-off-by: Sean Christopherson --- arch/powerpc/kvm/book3s_pr.c | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-) diff --git a/arch/powerpc/kvm/book3s_pr.c b/arch/powerpc/kvm/book3s_pr.c index cd7ab6d85090..83bcdc80ce51 100644 --- a/arch/powerpc/kvm/book3s_pr.c +++ b/arch/powerpc/kvm/book3s_pr.c @@ -639,28 +639,27 @@ static void kvmppc_set_pvr_pr(struct kvm_vcpu *vcpu, u32 pvr) */ static void kvmppc_patch_dcbz(struct kvm_vcpu *vcpu, struct kvmppc_pte *pte) { - struct page *hpage; + struct kvm_host_map map; u64 hpage_offset; u32 *page; - int i; + int i, r; - hpage = gfn_to_page(vcpu->kvm, pte->raddr >> PAGE_SHIFT); - if (!hpage) + r = kvm_vcpu_map(vcpu, pte->raddr >> PAGE_SHIFT, &map); + if (r) return; hpage_offset = pte->raddr & ~PAGE_MASK; hpage_offset &= ~0xFFFULL; hpage_offset /= 4; - page = kmap_atomic(hpage); + page = map.hva; /* patch dcbz into reserved instruction, so we trap */ for (i=hpage_offset; i < hpage_offset + (HW_PAGE_SIZE / 4); i++) if ((be32_to_cpu(page[i]) & 0xff0007ff) == INS_DCBZ) page[i] &= cpu_to_be32(0xfffffff7); - kunmap_atomic(page); - put_page(hpage); + kvm_vcpu_unmap(vcpu, &map); } static bool kvmppc_visible_gpa(struct kvm_vcpu *vcpu, gpa_t gpa) From patchwork Thu Oct 10 18:24:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831236 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D34A6D24456 for ; Thu, 10 Oct 2024 22:16:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:Content-Type:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=fZpnWf51XalE5JTpUzeqj9xBy85lPuBTlHLH/3crpzM=; b=J4DmC2fahQoKa6 8pu3FnFmEAM62xJXtFdkkv4LJAly8JiFl6sCZUzHTmNmdcdESdA0YdfU++kNz7jCYgjsUb/Os+9nC Pk/gPwRVnDBsaDDSk9XUaMP3ldJUqjAr5wwKw3+cf98TTTvWMv6/TXFjLYQ/WMcNIPTsUbGf0ApwR uRsNKqoFJJwWXyo6zZbia6Qh6cds5xIkOHi3JC0MyL95PmpfsFuykBJ71bJGyyCxJ9jw0HloxDL/X iBiLQE3ZEMs2rRmkIcDpG51o/T4X62oRYOoEEtPgQvJq+3HeATtpL40bd1PuQDtPaEu8ZZUoOtSP0 teuInsZ2TnLwPTUWYZow==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1sz1SQ-0000000EVvO-48Ez; Thu, 10 Oct 2024 22:16:30 +0000 Received: from mail-pf1-x449.google.com ([2607:f8b0:4864:20::449]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxsr-0000000Dri9-0EWW for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:27:35 +0000 Received: by mail-pf1-x449.google.com with SMTP id d2e1a72fcca58-71e123d3d22so1429801b3a.3 for ; Thu, 10 Oct 2024 11:27:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584852; x=1729189652; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=fZpnWf51XalE5JTpUzeqj9xBy85lPuBTlHLH/3crpzM=; b=cLJk3d4tBFRZOiFs+6bQQUgL9+5XLv36ZxgTPI3j7N9jPbVYQn5UYEHXG30q5cZAtp iLoVd863iI6QslfhR4bSkprz1688JE5LF2FO2gSOhcM+IGJX0QpjdBnRDdMQ62ouC3oJ bplSy/qMe21l7Xy1xG1IlCZN2yro1FWFNRNmv6sS9NMfZSUFAnJgg0EuPch5URNPcMG1 EWbkSmQF0LU/UoEnI63VnExMYr/1qn9aDn18jIArJhfw+WuPYh6KtbBICRCY7K4lBcNN jSLMr/sBO077LKQZMCJI/BZ+oDjzP6wnkJ4048tQ5WDcVuMAxNcfuoW2taX/zjS5bjPT jaEQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584852; x=1729189652; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=fZpnWf51XalE5JTpUzeqj9xBy85lPuBTlHLH/3crpzM=; b=PmpLXWdl0dgWJsLNmmqQqVy/o3gOJQVv36tyosg7a2CE6bmzqcSZppuUHaf42s7aUA 5kgrS87pKnUYHzfGyG65jjFOTFMehkkmRPJ2sgsyNYHNoK0WVOnfOtsxykqwWZpEP7mv tVPxlPk7oIy6I0bW9LLCzIh9dtJdfeIPz4kWt607NgWGmX0+vmG0d056jh0pticqZei8 pEmtKivW+7dX6PVxg2+pp2eXF5q+OspJSI+PrLAer2IqzndoNTcJNjBtOhaCJnFUUZRJ AoU5YSYBBxm3d99ZSpObxIMCLwNj8rcTXD4yB8CTnSOPB/TRHHhJnXRh81kZBLTdB+Yd E/Sw== X-Forwarded-Encrypted: i=1; AJvYcCXorcikM98aOmI5T9i/E1zI/XlUq6Ub7c8nJCeM4IbsRA8fcl69/PiVeHtWe8ZHcUlAzdYdKhAgliEdSHLMQ5sT@lists.infradead.org X-Gm-Message-State: AOJu0YxjpD9ZBykfyreJu4g7ENlbo8jsgwcPF//LkhQKwrn2QiSJICnO qe7lSgRdKoB3P56HmLtX3Q01V1Wk1cwu3mLqg0vsfuDZewrJfJJdoaxqcsLBCIAH8PQwM7wVayJ 64Q== X-Google-Smtp-Source: AGHT+IHVUYgAS8Gi1OfHaumBjOVs/68U//Xc3NGlK2wEye9l2w1SuCdDQ/WQfsMbBzqgGqsCESzcy3AUxv4= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:aa7:93b4:0:b0:71d:f64d:712c with SMTP id d2e1a72fcca58-71e1db65a16mr7365b3a.1.1728584851459; Thu, 10 Oct 2024 11:27:31 -0700 (PDT) Date: Thu, 10 Oct 2024 11:24:17 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-76-seanjc@google.com> Subject: [PATCH v13 75/85] KVM: Convert gfn_to_page() to use kvm_follow_pfn() From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112733_259812_B03F50DB X-CRM114-Status: GOOD ( 10.00 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Convert gfn_to_page() to the new kvm_follow_pfn() internal API, which will eventually allow removing gfn_to_pfn() and kvm_pfn_to_refcounted_page(). Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- virt/kvm/kvm_main.c | 16 +++++++++------- 1 file changed, 9 insertions(+), 7 deletions(-) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 696d5e429b3e..1782242a4800 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -3145,14 +3145,16 @@ EXPORT_SYMBOL_GPL(kvm_prefetch_pages); */ struct page *gfn_to_page(struct kvm *kvm, gfn_t gfn) { - kvm_pfn_t pfn; + struct page *refcounted_page = NULL; + struct kvm_follow_pfn kfp = { + .slot = gfn_to_memslot(kvm, gfn), + .gfn = gfn, + .flags = FOLL_WRITE, + .refcounted_page = &refcounted_page, + }; - pfn = gfn_to_pfn(kvm, gfn); - - if (is_error_noslot_pfn(pfn)) - return NULL; - - return kvm_pfn_to_refcounted_page(pfn); + (void)kvm_follow_pfn(&kfp); + return refcounted_page; } EXPORT_SYMBOL_GPL(gfn_to_page); From patchwork Thu Oct 10 18:24:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831238 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 51A28D24459 for ; Thu, 10 Oct 2024 22:16:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:Content-Type:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=fQZoDf0NIFdWK0Ory3IxzF5D2dmoMJWboRGGHV/nWyI=; b=zFnd4r9dRT/Me0 4W0hxvTOOLBSuE8g4SpVzODH3ARyHnSl9cEiD1dBcYlJQAEwnZS80wEoaLkISkAnDZieEvoRx1OEc /7oJrgCqarQkOHxAMkvqBtg6IQWu7ISAdfOnUU4M+M8Eb0GV333D443Q0yH6IOpwgPLy0hRcdd0IX 0l7/m8MwK2juGhqbrbhHoPnzWw7cfyb54HA6+AnKB8Oqs0W4hzIHACR2RLpUNRV9c9OvnirU3HgUU YIZdugyuq74GzSLIgwlMAHfPqKiwlX1KP5LMwUZjRSOr5sgsrvzgbj/HCiZ0beVbwnku3fMz5KdFU hl2Yi/k9oa1ASHkTwAqQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1sz1ST-0000000EVxR-3GUE; Thu, 10 Oct 2024 22:16:33 +0000 Received: from mail-yw1-x114a.google.com ([2607:f8b0:4864:20::114a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxss-0000000Drk9-3wPL for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:27:37 +0000 Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-6e21dcc7044so22862087b3.1 for ; Thu, 10 Oct 2024 11:27:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584854; x=1729189654; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=fQZoDf0NIFdWK0Ory3IxzF5D2dmoMJWboRGGHV/nWyI=; b=aLG1ERGsB5NvGh5NeKe3MCWim49sfOkO/B4LSwLQ0B6zMt1Z4zcHaGKAQHVba9OrFU QUTVJ4caJJgKcEln7aMvhzkYAClxH4agKnVmAQUlEtIcIbWcMEx6nPQMfPsnIsjywymV H6URPqrZmddDnbO3fB+DaERHSqNHQzuWZ0yDyezoaY8+rRHdLQ4Ga8EmDXO1zEkwK9s3 g+YEQ2EL99QEKhplKwek9Qwbsoh5C+rUXoK63HH2nyy1uU/moUakBBPGG28zmfGGFcjz h/DZDDZHkaZcqSHfipSgnfEpXCOx/kjd+vWOiVgJh82gxS0bYE1WVYdtDM+oPtbZzxJT x46Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584854; x=1729189654; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=fQZoDf0NIFdWK0Ory3IxzF5D2dmoMJWboRGGHV/nWyI=; b=l4Gey2FqWN7Kj/xXRiTI+3MFLqH4/MPmNOs210qjN21m43vCdOXmQv8VVXbNgb+nfq K+p6h04DBf+KbHCppHh0SULwSdnPTDt0oMxF7+TvshZi032XM2qH7z2/4OyFXSLDd1nv esBQ5DEAKckQuNCGMspgSLb2QwB94mhCIWxZhEOsvABh5frWlWh2nZ061aZfmK6Xhe9z Hnev/S7Y+QLsVnxdStrt27zsrCVuq7ovZUXkfv0tjH2hfgHcXAJ6/fCYxfZ2cfBlgHFw qQEzUJHMWw0HThKytqO6QeAD5ZE+X0jpSqE8eIGKSYoDK071B7ahG+LGz2qkhBT7/Az3 3MMA== X-Forwarded-Encrypted: i=1; AJvYcCXt2ZlUBewl+3Oo6CEG+8bBRDd9p7TpsJqcUOo5fevDBMFdmJJhqHFQE29lJoELpq+eB+BQk3p3ws0vCOGWqf3K@lists.infradead.org X-Gm-Message-State: AOJu0YzYvuVNxzWwqoSedXYOa3MqZH63ZMqiLXM6KgKhm/jog4NcH5tK v2EwVPuJD16IjTPfibbbG7g3f2WvZ3KcxVPI5m0XE4MrpiDvNK3s4LD7FPBAf1lYL6IsKJ5f+nR WUg== X-Google-Smtp-Source: AGHT+IGkMjnJcd2oWHVLgC5xfQJb4mHLQjnUvb6XgW9ZV8VH9lzrMuhcRfl3e0TBTVqXdJBO9V/Z2ttoh2Y= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:690c:4091:b0:64a:e220:bfb5 with SMTP id 00721157ae682-6e32213fbacmr192077b3.1.1728584853563; Thu, 10 Oct 2024 11:27:33 -0700 (PDT) Date: Thu, 10 Oct 2024 11:24:18 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-77-seanjc@google.com> Subject: [PATCH v13 76/85] KVM: Add support for read-only usage of gfn_to_page() From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112735_076881_D71333D9 X-CRM114-Status: GOOD ( 15.59 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Rework gfn_to_page() to support read-only accesses so that it can be used by arm64 to get MTE tags out of guest memory. Opportunistically rewrite the comment to be even more stern about using gfn_to_page(), as there are very few scenarios where requiring a struct page is actually the right thing to do (though there are such scenarios). Add a FIXME to call out that KVM probably should be pinning pages, not just getting pages. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- include/linux/kvm_host.h | 7 ++++++- virt/kvm/kvm_main.c | 15 ++++++++------- 2 files changed, 14 insertions(+), 8 deletions(-) diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 9f7682ece4a1..af928b59b2ab 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -1213,7 +1213,12 @@ void kvm_arch_flush_shadow_memslot(struct kvm *kvm, int kvm_prefetch_pages(struct kvm_memory_slot *slot, gfn_t gfn, struct page **pages, int nr_pages); -struct page *gfn_to_page(struct kvm *kvm, gfn_t gfn); +struct page *__gfn_to_page(struct kvm *kvm, gfn_t gfn, bool write); +static inline struct page *gfn_to_page(struct kvm *kvm, gfn_t gfn) +{ + return __gfn_to_page(kvm, gfn, true); +} + unsigned long gfn_to_hva(struct kvm *kvm, gfn_t gfn); unsigned long gfn_to_hva_prot(struct kvm *kvm, gfn_t gfn, bool *writable); unsigned long gfn_to_hva_memslot(struct kvm_memory_slot *slot, gfn_t gfn); diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 1782242a4800..8f8b2cd01189 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -3138,25 +3138,26 @@ int kvm_prefetch_pages(struct kvm_memory_slot *slot, gfn_t gfn, EXPORT_SYMBOL_GPL(kvm_prefetch_pages); /* - * Do not use this helper unless you are absolutely certain the gfn _must_ be - * backed by 'struct page'. A valid example is if the backing memslot is - * controlled by KVM. Note, if the returned page is valid, it's refcount has - * been elevated by gfn_to_pfn(). + * Don't use this API unless you are absolutely, positively certain that KVM + * needs to get a struct page, e.g. to pin the page for firmware DMA. + * + * FIXME: Users of this API likely need to FOLL_PIN the page, not just elevate + * its refcount. */ -struct page *gfn_to_page(struct kvm *kvm, gfn_t gfn) +struct page *__gfn_to_page(struct kvm *kvm, gfn_t gfn, bool write) { struct page *refcounted_page = NULL; struct kvm_follow_pfn kfp = { .slot = gfn_to_memslot(kvm, gfn), .gfn = gfn, - .flags = FOLL_WRITE, + .flags = write ? FOLL_WRITE : 0, .refcounted_page = &refcounted_page, }; (void)kvm_follow_pfn(&kfp); return refcounted_page; } -EXPORT_SYMBOL_GPL(gfn_to_page); +EXPORT_SYMBOL_GPL(__gfn_to_page); int __kvm_vcpu_map(struct kvm_vcpu *vcpu, gfn_t gfn, struct kvm_host_map *map, bool writable) From patchwork Thu Oct 10 18:24:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831239 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3CAA8D2444F for ; Thu, 10 Oct 2024 22:16:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:Content-Type:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=04eHUk6AClz9cRMQcjlWjE46ZR1uXceY2ogFB00LGDM=; b=E4N0q5yXGgwOia AWxfkV57swwgPKWKSlGmQ5T32DUw9NhQWmezIDxLZZ38BzYnSSWaCxBkZE8G7bdDBxAlMb7Wgsx+k 1TJIsgGl0WPKbpAf63ykF7+H/M3n1L8VW5ZmPOXvgyHqUGhMfIih6uD3Az5E3YP/dcU1oH7ay2R+V 0ujwe6fixrdFmoVKVU+uc14uoRbIQXCmBbq3vrONhF5S0VPqijdOrKxEzYGjLmDhKjOKWdf79GK0W Uzu62cPRoniinOV0x8Ap4n35aTRGQRU71T+U4aQxy3qxn7fyQu9pH8gvfon/ngM84wAGdsvzNOyHa ithX3tB2iaxTxv/2iVbw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1sz1SW-0000000EVzV-1ZBE; Thu, 10 Oct 2024 22:16:36 +0000 Received: from mail-pg1-x54a.google.com ([2607:f8b0:4864:20::54a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxsv-0000000Drly-1LhO for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:27:39 +0000 Received: by mail-pg1-x54a.google.com with SMTP id 41be03b00d2f7-7e9b2d75d6dso895814a12.1 for ; Thu, 10 Oct 2024 11:27:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584856; x=1729189656; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=04eHUk6AClz9cRMQcjlWjE46ZR1uXceY2ogFB00LGDM=; b=alq1Gi8IKHXoRenVXYXecNt3KC/lj7CRLd4Ivpw77cRV2hhaqmorMgi3dwBXJZi2Z3 +oSIHdym3Kp6z1HFNs3xH8wtQWXWauU8ROxoDYovdX651koTj7r/xxKLyCYmTJRKZ2Q7 SZ6WMuDGYO+bEyQTNVfOCYgEzHS+3cAAeAuzf6XXFt3BezmXoMSk1NB74uRtHh2g67pb m2KGWKyP5u4SgLIVf5Unq6dz72HfCgCIZ6QIcRV/fMvo9IuyY/oxw9iWwhx0/1tvNyg4 NQoXsHYuYADad7JGxE3yi6t0dOyKX2LqcGYeBRRgvvWIXq3TkwikALhUOeoR75a/tGED uyZA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584856; x=1729189656; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=04eHUk6AClz9cRMQcjlWjE46ZR1uXceY2ogFB00LGDM=; b=Pc7ZoNbwnSOSdJljohhd7m80QuVwpt0Ih3Z6BtL77W2/c76eKnwnjSalBE/hnMIHr8 awEgBlJfLqUNbPAI4VTS1QV+icKfrQQ3WU3NikbUCYLxaw+DMieUQ8F2LN9JFAkvwncn CVJiGO10716XXpjqujjjnONVe78R/tItcZIrgAkwQhwXiPXhiAjAZZdBggYXCEdh9JmY h6k6CqJS/WVcH+I8rEAeeHTc6Snap8fx8ihOa2vNfjegFSeLaA55nscd16E43zpcpNKK D7S9vh/VhZzckWG3WP/V0svmpyEj2z7t+IMiAYO2KK7fm9hH6xCdf35wGH2IDQ2e3Swc XcjQ== X-Forwarded-Encrypted: i=1; AJvYcCWQjt8P0C56xzeou1bM+Krxvh9w+IcHKk8JGDbXDUjGO2+4vjcK/K34wmBf/NPeV/PIL7Y/VceRQRknG1M7S+Ok@lists.infradead.org X-Gm-Message-State: AOJu0YxoZt+fROHO6CPoZnh00HukfjqYQSR15Y4phok6b1SQILqEl51A CO+AM+oyY54XIhYWgYNlmG/BMzSoA8xMnAlIoJtMnjANdrIaddWFdaoA2jQ5AYe3je2QJ7FoIl1 eaA== X-Google-Smtp-Source: AGHT+IHGiwNd3UbnZUzPre1tk98dNYoM10q7gFGG4lq0PldsJuDm1rxMHI+g8Yc3UsVnmkzWnOurUMFmqIQ= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:903:41c4:b0:20c:8f78:67c9 with SMTP id d9443c01a7336-20c8f786946mr26385ad.5.1728584855618; Thu, 10 Oct 2024 11:27:35 -0700 (PDT) Date: Thu, 10 Oct 2024 11:24:19 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-78-seanjc@google.com> Subject: [PATCH v13 77/85] KVM: arm64: Use __gfn_to_page() when copying MTE tags to/from userspace From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112737_427911_894E1629 X-CRM114-Status: GOOD ( 12.12 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Use __gfn_to_page() instead when copying MTE tags between guest and userspace. This will eventually allow removing gfn_to_pfn_prot(), gfn_to_pfn(), kvm_pfn_to_refcounted_page(), and related APIs. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/arm64/kvm/guest.c | 21 +++++++++------------ 1 file changed, 9 insertions(+), 12 deletions(-) diff --git a/arch/arm64/kvm/guest.c b/arch/arm64/kvm/guest.c index 962f985977c2..4cd7ffa76794 100644 --- a/arch/arm64/kvm/guest.c +++ b/arch/arm64/kvm/guest.c @@ -1051,20 +1051,18 @@ int kvm_vm_ioctl_mte_copy_tags(struct kvm *kvm, } while (length > 0) { - kvm_pfn_t pfn = gfn_to_pfn_prot(kvm, gfn, write, NULL); + struct page *page = __gfn_to_page(kvm, gfn, write); void *maddr; unsigned long num_tags; - struct page *page; - if (is_error_noslot_pfn(pfn)) { - ret = -EFAULT; - goto out; - } - - page = pfn_to_online_page(pfn); if (!page) { + ret = -EFAULT; + goto out; + } + + if (!pfn_to_online_page(page_to_pfn(page))) { /* Reject ZONE_DEVICE memory */ - kvm_release_pfn_clean(pfn); + kvm_release_page_unused(page); ret = -EFAULT; goto out; } @@ -1078,7 +1076,7 @@ int kvm_vm_ioctl_mte_copy_tags(struct kvm *kvm, /* No tags in memory, so write zeros */ num_tags = MTE_GRANULES_PER_PAGE - clear_user(tags, MTE_GRANULES_PER_PAGE); - kvm_release_pfn_clean(pfn); + kvm_release_page_clean(page); } else { /* * Only locking to serialise with a concurrent @@ -1093,8 +1091,7 @@ int kvm_vm_ioctl_mte_copy_tags(struct kvm *kvm, if (num_tags != MTE_GRANULES_PER_PAGE) mte_clear_page_tags(maddr); set_page_mte_tagged(page); - - kvm_release_pfn_dirty(pfn); + kvm_release_page_dirty(page); } if (num_tags != MTE_GRANULES_PER_PAGE) { From patchwork Thu Oct 10 18:24:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831309 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0819DD2445D for ; Thu, 10 Oct 2024 23:31:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Type:Cc:To: From:Subject:Message-ID:References:Mime-Version:In-Reply-To:Date: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=43Dd+lWRPu/QQcFLGDLhCyyiH7uAqu5vbmltTcj5okg=; b=EqYMyPmRmciRzIvh4SAgWsLHWo B2dLj+dL+oQE5TEu1k2tbjdy/g85cUhrIQY/N/ylYzAi2FxZRTiQOgRA89sJSuDmcoLFILXvxlVHr UZ99H9jAYEzBBlwtweIaGAgxloGQzll7WJ0Bz8G0wCYtfUX71FSMzWKt0JmYlAiurHDImu72NZZe/ dxzoMFRVLEgVljypQLPJk+KUZ3+FtJorwB4mbUpOzSQ+tIAjUz3VkZc2rRSDD4zgFRD3F+X1l81gn f4u4lN9ZN7Oj1HyrkklzxP7bHmLOeN6vxVMvnQq7nQ1CnB+y6Slpiyp42ydcugXs7zZUQGJytGtqP mrd/yTXw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1sz2d2-0000000Ef6U-3bJc; Thu, 10 Oct 2024 23:31:32 +0000 Received: from mail-pg1-x549.google.com ([2607:f8b0:4864:20::549]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxsx-0000000Drnv-1uva for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:27:41 +0000 Received: by mail-pg1-x549.google.com with SMTP id 41be03b00d2f7-778702b9f8fso1040809a12.1 for ; Thu, 10 Oct 2024 11:27:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584858; x=1729189658; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=43Dd+lWRPu/QQcFLGDLhCyyiH7uAqu5vbmltTcj5okg=; b=ge/KVFe5o7m0CrYItpSkaRrqEs6HCvjuf4dtPzfv98KYlQGh/QLeT5Z5fSRGccyiyM L5Z9adg+js6Zw4rVdQ2gjoah1thpDjUSIdwA5BEg90iA0A65+u/GFFXrzuRt0RMsF8nc FjEi9xlv7xPIBxHYIHGj0TGsieacYRgZvJkfGXimSosPbUmJj+3ajkP7+MzdBEKWOnvp SHbcEfX0hrL5Pr7biZ1dmkKwIXNL+Zbo7SPItkvjjlFS+KCoBI062lzohEowRpVeDVTa 7NPcGhOmHlAyjnfSrXiHN7Vs52TomwHW0GC81PbeGkG4HVA6M8JwCgLDg1dR+4rNISfj iZjQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584858; x=1729189658; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=43Dd+lWRPu/QQcFLGDLhCyyiH7uAqu5vbmltTcj5okg=; b=iPm89Ff5xBbE3irK1NsQ9/ZmQMLQ7jw81wbLPMwUzeotXToCkLKMcACPg2wNclvwpQ UvbFGwzZuhp5GNxvS1OWDRhPyEHR33EdJKl4Dws8cIPkzeolR1BFIOCreLgja2SCLP0M zuUZpgel+fRop0XQijoV1KRJlRpzDQhnXBy3GjSX6UEUiGIj15BHUOxEa3SbdUYPVqSr iC6oKJT27Tc5UXAtd4ESPvST2gDhgGZgeqbi7SaimeDw4a6WMqRM+c6yO3smFinMQd6o nl6mNJuLh6YBdRAsv4M7VE4gJ/ts6mXa35N0YMZ4cPYgVk+enuOWAWd2+aI8/g7cZ2NZ +1WQ== X-Forwarded-Encrypted: i=1; AJvYcCU0f+iQIlLRn+0ZB0yET5mJ6USf9kth59aJOBYcyskNqWdi6uxTzoErHqhCTjfW+zvkIu8sGrDrbsMzbadxlY1Q@lists.infradead.org X-Gm-Message-State: AOJu0YzGszGK9dRb0hIBcXOJgrceUrRWQOAt47g6fw86udGW8X8KPyrG R9CGouEN5Rd/S1dJQHx92+9R/ws7yHOo/Kal0PsHVPoGg8mmRRLZY4u4vsrwc7DmV8+hIKOoa92 cdw== X-Google-Smtp-Source: AGHT+IGjs5y0lK7jvR8/zbNWJNDHfDWRUWSPa98vwOnU2bZ6sNQJgoH/6j7HDiYxyC+lHu033MhghQMujcY= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a63:b208:0:b0:7e6:c3f2:24c2 with SMTP id 41be03b00d2f7-7ea535dce6cmr30a12.6.1728584857741; Thu, 10 Oct 2024 11:27:37 -0700 (PDT) Date: Thu, 10 Oct 2024 11:24:20 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-79-seanjc@google.com> Subject: [PATCH v13 78/85] KVM: PPC: Explicitly require struct page memory for Ultravisor sharing From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112739_716215_360A56C7 X-CRM114-Status: GOOD ( 11.88 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Explicitly require "struct page" memory when sharing memory between guest and host via an Ultravisor. Given the number of pfn_to_page() calls in the code, it's safe to assume that KVM already requires that the pfn returned by gfn_to_pfn() is backed by struct page, i.e. this is likely a bug fix, not a reduction in KVM capabilities. Switching to gfn_to_page() will eventually allow removing gfn_to_pfn() and kvm_pfn_to_refcounted_page(). Signed-off-by: Sean Christopherson --- arch/powerpc/kvm/book3s_hv_uvmem.c | 25 ++++++++++++------------- 1 file changed, 12 insertions(+), 13 deletions(-) diff --git a/arch/powerpc/kvm/book3s_hv_uvmem.c b/arch/powerpc/kvm/book3s_hv_uvmem.c index 92f33115144b..3a6592a31a10 100644 --- a/arch/powerpc/kvm/book3s_hv_uvmem.c +++ b/arch/powerpc/kvm/book3s_hv_uvmem.c @@ -879,9 +879,8 @@ static unsigned long kvmppc_share_page(struct kvm *kvm, unsigned long gpa, { int ret = H_PARAMETER; - struct page *uvmem_page; + struct page *page, *uvmem_page; struct kvmppc_uvmem_page_pvt *pvt; - unsigned long pfn; unsigned long gfn = gpa >> page_shift; int srcu_idx; unsigned long uvmem_pfn; @@ -901,8 +900,8 @@ static unsigned long kvmppc_share_page(struct kvm *kvm, unsigned long gpa, retry: mutex_unlock(&kvm->arch.uvmem_lock); - pfn = gfn_to_pfn(kvm, gfn); - if (is_error_noslot_pfn(pfn)) + page = gfn_to_page(kvm, gfn); + if (!page) goto out; mutex_lock(&kvm->arch.uvmem_lock); @@ -911,16 +910,16 @@ static unsigned long kvmppc_share_page(struct kvm *kvm, unsigned long gpa, pvt = uvmem_page->zone_device_data; pvt->skip_page_out = true; pvt->remove_gfn = false; /* it continues to be a valid GFN */ - kvm_release_pfn_clean(pfn); + kvm_release_page_unused(page); goto retry; } - if (!uv_page_in(kvm->arch.lpid, pfn << page_shift, gpa, 0, + if (!uv_page_in(kvm->arch.lpid, page_to_pfn(page) << page_shift, gpa, 0, page_shift)) { kvmppc_gfn_shared(gfn, kvm); ret = H_SUCCESS; } - kvm_release_pfn_clean(pfn); + kvm_release_page_clean(page); mutex_unlock(&kvm->arch.uvmem_lock); out: srcu_read_unlock(&kvm->srcu, srcu_idx); @@ -1083,21 +1082,21 @@ kvmppc_h_svm_page_out(struct kvm *kvm, unsigned long gpa, int kvmppc_send_page_to_uv(struct kvm *kvm, unsigned long gfn) { - unsigned long pfn; + struct page *page; int ret = U_SUCCESS; - pfn = gfn_to_pfn(kvm, gfn); - if (is_error_noslot_pfn(pfn)) + page = gfn_to_page(kvm, gfn); + if (!page) return -EFAULT; mutex_lock(&kvm->arch.uvmem_lock); if (kvmppc_gfn_is_uvmem_pfn(gfn, kvm, NULL)) goto out; - ret = uv_page_in(kvm->arch.lpid, pfn << PAGE_SHIFT, gfn << PAGE_SHIFT, - 0, PAGE_SHIFT); + ret = uv_page_in(kvm->arch.lpid, page_to_pfn(page) << PAGE_SHIFT, + gfn << PAGE_SHIFT, 0, PAGE_SHIFT); out: - kvm_release_pfn_clean(pfn); + kvm_release_page_clean(page); mutex_unlock(&kvm->arch.uvmem_lock); return (ret == U_SUCCESS) ? RESUME_GUEST : -EFAULT; } From patchwork Thu Oct 10 18:24:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831077 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 8A8FFD24450 for ; Thu, 10 Oct 2024 20:32:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:Content-Type:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=Q1MnEbvSXNK1DBYHKiiPeGTlNBdM3PqCgrnaMRmX38Y=; b=AB7tMIGoyoI5Q7 Mu/csRy2EYi5zrwuGmXfx5kB8dd3QuQ5RZT9O7vhkWJL5htsEFp2p9INF1ogebfM/uamKUG90+cTs U3FuUem9DdAbsz88xrNMYMtWosLG2bHehgaUiZjow8TsGHBrZfSJn+Z/VUyT4ja3kyzW6szOCOMyu TGhZslFwtZfscTY1ZlGOYzqlSl9GjHDUOtpn1Mdzzz3OccDv+LRve1VlvLXmp/uKfEiOX0JqRTq4f kGCaI2eBM6sfQPmByi0hWBK6Z2L+odFOww8E3u6ZWN14NptrICJlds0DnUBPMmztZe8tQuXRUViPU bCPc9yNztt65p5XxWoLQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syzpc-0000000EHO5-1Cnf; Thu, 10 Oct 2024 20:32:20 +0000 Received: from mail-yb1-xb49.google.com ([2607:f8b0:4864:20::b49]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxsz-0000000Drpn-3QGl for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:27:43 +0000 Received: by mail-yb1-xb49.google.com with SMTP id 3f1490d57ef6-e165fc5d94fso1879185276.2 for ; Thu, 10 Oct 2024 11:27:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584860; x=1729189660; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=Q1MnEbvSXNK1DBYHKiiPeGTlNBdM3PqCgrnaMRmX38Y=; b=i5UNoI6Kh250+uksVmaSRP/cHhMpWg5vufAjEq5qaf4KW2tpEbQBEnUAxe8WtIPzVD RJtBDpF/jSRcHx7V05sRWdvewtg7NDONoenoJgCZkES3b9pZIwJplK/ea4WgKsJKtwN5 vMqotF/eVBLeHPF6N8JBMZWeQP/i8pY0rcsG2bjOsA3Ak+wU161wXH9k9YYRw5nCULcN Q2W/gCFxPgQdHaPAphdCHemVDWrBZIyWZIh98QKAKROgiWi1Q9z6KLI7U/yyFI2iRCjS 6yq/rY9hUeyZBGeU+ENe+RD5wX0ilZR4YZoGuleOjUADwYxAX1LH7/JJjIHIfRbA8QTS JeXw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584860; x=1729189660; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=Q1MnEbvSXNK1DBYHKiiPeGTlNBdM3PqCgrnaMRmX38Y=; b=mKfWbPQcjlLesuhMKI7ACfolZG03VBowkCnWL6AVumkYo3ppnfRiw8r02xeQHy6b3X 8Sr6Pg8gLETyooFPXnCFm8+0mzy+O/h/yW/dN4FPNFKNA+e9d3GgItAn2PzYyRDSIFdR e1ZD2U2At7f8eo0aY5L3hiAPmFiGrBvpO/hfPa8KEmUR21lKMBp9ato38FNJxbxeCL/q D6yjFscyVOKS03/KCzPlZ5XZwAozBcIEfwIyVKYDWxugYUCPytgZiQUV4SU1B1dD4Qdt 1LLfusFNI0a+bg4RHV2pLPar8I2tiPLkbDzoZJJhzcy7wO46wFCH+NJkJrktCosH87xB /ZLQ== X-Forwarded-Encrypted: i=1; AJvYcCUwX7U2+aHUhp/ki2cFR4u60MuTMvx2OLVjdVSapwW4xPhCCQ6SD2Fdxf4+03KhAYUf39bS/CmYHdIa7yfVZ4AP@lists.infradead.org X-Gm-Message-State: AOJu0YxE10DPu6ekq/pQytPa0mbt23t/GzmksrpCf2nKGMkC/zrirvFO BJiYtglQY2/z54pD7h7VwIqV2uW/Nt3QteBEXRWa0jzU98mTPBlWzngegNS3uDIvYvMXJ4YkQNU dGg== X-Google-Smtp-Source: AGHT+IGvHgo281GUU728l2kpqfRNdoQ8M4AtWxtMdwK+Sbd/Nmo7aGxeBXrCmxWS+HWzNnhFIgaVqhxkocM= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a25:f20e:0:b0:e11:5da7:337 with SMTP id 3f1490d57ef6-e28fe3505c2mr129032276.3.1728584859805; Thu, 10 Oct 2024 11:27:39 -0700 (PDT) Date: Thu, 10 Oct 2024 11:24:21 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-80-seanjc@google.com> Subject: [PATCH v13 79/85] KVM: Drop gfn_to_pfn() APIs now that all users are gone From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112741_973898_C91D30ED X-CRM114-Status: UNSURE ( 9.24 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Drop gfn_to_pfn() and all its variants now that all users are gone. No functional change intended. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- include/linux/kvm_host.h | 11 -------- virt/kvm/kvm_main.c | 59 ---------------------------------------- 2 files changed, 70 deletions(-) diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index af928b59b2ab..4a1eaa40a215 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -1274,14 +1274,6 @@ static inline kvm_pfn_t kvm_faultin_pfn(struct kvm_vcpu *vcpu, gfn_t gfn, write ? FOLL_WRITE : 0, writable, refcounted_page); } -kvm_pfn_t gfn_to_pfn(struct kvm *kvm, gfn_t gfn); -kvm_pfn_t gfn_to_pfn_prot(struct kvm *kvm, gfn_t gfn, bool write_fault, - bool *writable); -kvm_pfn_t gfn_to_pfn_memslot(const struct kvm_memory_slot *slot, gfn_t gfn); -kvm_pfn_t __gfn_to_pfn_memslot(const struct kvm_memory_slot *slot, gfn_t gfn, - bool interruptible, bool no_wait, - bool write_fault, bool *writable); - void kvm_release_pfn_clean(kvm_pfn_t pfn); void kvm_release_pfn_dirty(kvm_pfn_t pfn); void kvm_set_pfn_dirty(kvm_pfn_t pfn); @@ -1356,9 +1348,6 @@ unsigned long kvm_host_page_size(struct kvm_vcpu *vcpu, gfn_t gfn); void mark_page_dirty_in_slot(struct kvm *kvm, const struct kvm_memory_slot *memslot, gfn_t gfn); void mark_page_dirty(struct kvm *kvm, gfn_t gfn); - -kvm_pfn_t kvm_vcpu_gfn_to_pfn(struct kvm_vcpu *vcpu, gfn_t gfn); - int __kvm_vcpu_map(struct kvm_vcpu *vcpu, gpa_t gpa, struct kvm_host_map *map, bool writable); void kvm_vcpu_unmap(struct kvm_vcpu *vcpu, struct kvm_host_map *map); diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 8f8b2cd01189..b2c8d429442d 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -3039,65 +3039,6 @@ static kvm_pfn_t kvm_follow_pfn(struct kvm_follow_pfn *kfp) return hva_to_pfn(kfp); } -kvm_pfn_t __gfn_to_pfn_memslot(const struct kvm_memory_slot *slot, gfn_t gfn, - bool interruptible, bool no_wait, - bool write_fault, bool *writable) -{ - struct kvm_follow_pfn kfp = { - .slot = slot, - .gfn = gfn, - .map_writable = writable, - }; - - if (write_fault) - kfp.flags |= FOLL_WRITE; - if (no_wait) - kfp.flags |= FOLL_NOWAIT; - if (interruptible) - kfp.flags |= FOLL_INTERRUPTIBLE; - - return kvm_follow_pfn(&kfp); -} -EXPORT_SYMBOL_GPL(__gfn_to_pfn_memslot); - -kvm_pfn_t gfn_to_pfn_prot(struct kvm *kvm, gfn_t gfn, bool write_fault, - bool *writable) -{ - struct kvm_follow_pfn kfp = { - .slot = gfn_to_memslot(kvm, gfn), - .gfn = gfn, - .flags = write_fault ? FOLL_WRITE : 0, - .map_writable = writable, - }; - - return kvm_follow_pfn(&kfp); -} -EXPORT_SYMBOL_GPL(gfn_to_pfn_prot); - -kvm_pfn_t gfn_to_pfn_memslot(const struct kvm_memory_slot *slot, gfn_t gfn) -{ - struct kvm_follow_pfn kfp = { - .slot = slot, - .gfn = gfn, - .flags = FOLL_WRITE, - }; - - return kvm_follow_pfn(&kfp); -} -EXPORT_SYMBOL_GPL(gfn_to_pfn_memslot); - -kvm_pfn_t gfn_to_pfn(struct kvm *kvm, gfn_t gfn) -{ - return gfn_to_pfn_memslot(gfn_to_memslot(kvm, gfn), gfn); -} -EXPORT_SYMBOL_GPL(gfn_to_pfn); - -kvm_pfn_t kvm_vcpu_gfn_to_pfn(struct kvm_vcpu *vcpu, gfn_t gfn) -{ - return gfn_to_pfn_memslot(kvm_vcpu_gfn_to_memslot(vcpu, gfn), gfn); -} -EXPORT_SYMBOL_GPL(kvm_vcpu_gfn_to_pfn); - kvm_pfn_t __kvm_faultin_pfn(const struct kvm_memory_slot *slot, gfn_t gfn, unsigned int foll, bool *writable, struct page **refcounted_page) From patchwork Thu Oct 10 18:24:22 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831274 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4A36BD2445E for ; Thu, 10 Oct 2024 23:01:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Type:Cc:To: From:Subject:Message-ID:References:Mime-Version:In-Reply-To:Date: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=A6YyPrl/aVt4zvDZFXUjzF/Zy9evtaTkH7tjbHPMI5E=; b=rhU0wWoavTm9+Yp3we/wu6swXn VWW2ok5Q5RLtyj7Ej3bh9kvLtae0ChYBHrY89p2aE+TGRRKWFoxy6ZcMMn0wtMpyczkUdcvXqFx8Z U3tIomQa/OHJVhFs54CaRD0vF2o8ptt2pCBUXbkSwKaVBxoHlKvkgKqoMZPGYHEVfyq4Ai1PB6xMu xQsw0hN2/+c/EkGGMsvsELD9+ThbMil8lCN3vtVl1utYYHkPBWIyDu9PaZNfX77LXckaD5l6IP1uw OpEdxwZlRcTAK2fZxaCiF8tDVJdRBi0g1sOVW8EXafJ6si13HErkRi3JeuQo2mv4xSMet1EwLhQX2 I/xgjuXw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1sz29y-0000000Ec7e-1pAo; Thu, 10 Oct 2024 23:01:30 +0000 Received: from mail-pf1-x449.google.com ([2607:f8b0:4864:20::449]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxt1-0000000Drr1-2O4J for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:27:45 +0000 Received: by mail-pf1-x449.google.com with SMTP id d2e1a72fcca58-71e1e989aa2so1656605b3a.3 for ; Thu, 10 Oct 2024 11:27:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584862; x=1729189662; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=A6YyPrl/aVt4zvDZFXUjzF/Zy9evtaTkH7tjbHPMI5E=; b=ls6EkiR4KuhBz6/sYUk1bFoSMivDvkbu3Ml1zJw7Cvt4EQSBWmD1o8EtK4HJvzfTGw hC730xjyAX2jlRhTdUDbZ4yQM03Vvp+wRCa3voXldfXjYz/y3KRfiPSfPQvoEMkhWV9M jMqxVQrr9LdJzhtKSrtfqCOm56lhRocISlwa60nFFZzbQAAoHnHDKtjqmghany/Exgs/ pP+fhdk2VmvD7o+2ZQYsc8z/6o5klCeCgp8pjFTTL1t3EhZkhvKN0EWKDORfwXvvtHYi wt2sjPeF1qKNEIryfkK05p9i2SDvhfBiXnxLot1wvYGKZsMKwYYZHhhjIXzBzWpa4BmA 1MmA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584862; x=1729189662; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=A6YyPrl/aVt4zvDZFXUjzF/Zy9evtaTkH7tjbHPMI5E=; b=N+Lr5dSjYIiFa6o5Xx1mjerNwqcrpIMPeV2RJ4yJabpLlCXpQb8U4KoIA+v4RmqK/E ak18qz1VQCq4it93uxXbMdew0M1iB2t4Ma4zeoZld2HdX+junZ6AV20xT0R/NUBRAfDC UrWwZ1mwbZOdOtO4XNur+oe1fZQkoEvYmsJOMq/b7G51H8v8wBk/J4+1nrMd4sNUwbWO sAC7HtLxmVOxq4q2gbE04utI0rl4nz18MCI324pE58tQoPDmUKhUM6JfRp14A4mRWGeZ slhJYipi37ZWrunqHw5vAQxuu3sh2Glh9rNvTKf9sY3tkbOTztp7uoYaFBIcXHDCfaMA donw== X-Forwarded-Encrypted: i=1; AJvYcCX8quhSXocgpz1gK9DZgjhCLlwq9Tsexe8flZCJwekWEM3aCcrtazTfyLUeDf5HMHzn/XcVkUru4a1mdsPjtRZW@lists.infradead.org X-Gm-Message-State: AOJu0Yy4xRurfAasB7V0RZbtFRHgU+DB5WbFlc4YOIzJ8As0ujPMBFru u+naHixIoilMKbK1JgI4kAnb/S5Cd/TmyhXaa1wQS8fu8f8FkvR9vVMyXthGdkFWOts/5JrH+Qi LRg== X-Google-Smtp-Source: AGHT+IFRVsKTyvnlBUrQtNNg8zvYT0OU1qw5Av8a7Gq/P6vdrCxit4uLl8sEcLrSKXXTWA/vla3myJTtPpY= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:6a00:9443:b0:71d:ff10:7c4 with SMTP id d2e1a72fcca58-71e1dbd1d0fmr15196b3a.4.1728584861919; Thu, 10 Oct 2024 11:27:41 -0700 (PDT) Date: Thu, 10 Oct 2024 11:24:22 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-81-seanjc@google.com> Subject: [PATCH v13 80/85] KVM: s390: Use kvm_release_page_dirty() to unpin "struct page" memory From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112743_679320_3D4587E9 X-CRM114-Status: GOOD ( 12.80 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Use kvm_release_page_dirty() when unpinning guest pages, as the pfn was retrieved via pin_guest_page(), i.e. is guaranteed to be backed by struct page memory. This will allow dropping kvm_release_pfn_dirty() and friends. Signed-off-by: Sean Christopherson --- arch/s390/kvm/vsie.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/s390/kvm/vsie.c b/arch/s390/kvm/vsie.c index 763a070f5955..e1fdf83879cf 100644 --- a/arch/s390/kvm/vsie.c +++ b/arch/s390/kvm/vsie.c @@ -670,7 +670,7 @@ static int pin_guest_page(struct kvm *kvm, gpa_t gpa, hpa_t *hpa) /* Unpins a page previously pinned via pin_guest_page, marking it as dirty. */ static void unpin_guest_page(struct kvm *kvm, gpa_t gpa, hpa_t hpa) { - kvm_release_pfn_dirty(hpa >> PAGE_SHIFT); + kvm_release_page_dirty(pfn_to_page(hpa >> PAGE_SHIFT)); /* mark the page always as dirty for migration */ mark_page_dirty(kvm, gpa_to_gfn(gpa)); } From patchwork Thu Oct 10 18:24:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831088 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 274CCD2444D for ; Thu, 10 Oct 2024 20:35:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:Content-Type:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=7IfrRBwnhP/ARkGggvkJ//bNWbTHtvwsCnjZUdYSoz0=; b=V4tombvI4akwVR JFVQ6hp9bOvOPhhx5PpPzO0oWcWOlP1Vy3x08QSuiChQv/WnR6zDgsPaNcWd1zYg5Mqkf/AViawdV W6ZkVyPDMpFJhz2XfiyADnIHXh9/8EyBFEr3MVV4Cgi8jRuQEVYjy4ENSn/kOBmtGCKbyT6Rteo39 nfHhL0RoYAHVbdKAxEY7Bh34JCNQYv2+WN3BMUkjQ9fwuTy3r5+wyvg0dnI+ScfgqMEaTclua+PDo qyyCybZqcVZATCQdBoT1SDIeiW7s/QBBxw78b8ijPzWuNi4pKvgOpCD++56ChXF80UZrqCLmfg56X c4TBvB3LZTTRIt2+Fgwg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syzsK-0000000EHp1-44yG; Thu, 10 Oct 2024 20:35:09 +0000 Received: from mail-yb1-xb4a.google.com ([2607:f8b0:4864:20::b4a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxt3-0000000DrsU-2X8Z for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:27:47 +0000 Received: by mail-yb1-xb4a.google.com with SMTP id 3f1490d57ef6-e28eea1074dso1985523276.0 for ; Thu, 10 Oct 2024 11:27:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584864; x=1729189664; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=7IfrRBwnhP/ARkGggvkJ//bNWbTHtvwsCnjZUdYSoz0=; b=J8taJzN9t4s6HfK9u69vl1xjw/eLAz6KM1QsCV7wHhxGzLMHMqrB4B7KQn2Pb8VPjz X3Gysx6a10sADIsjtWJMpLpSugHV+E/l0HLp7kiFs/dOcFljnIA5sZLidASSdLrQfZX9 9X5eNMsraY6t+GseYThGg9Rn6iAOYfdqVDtIGerSU6Y30fee7uv030SPw6meaPF0Uc25 sTllS9cUGwDgPbQWSiu7DoEP//VL9LoKzRHmp/xUnHXTWsskxPUFohgwPbENzH6G18dh kT0KCIrvOHsjeeiF5DVSyAG2QYB7RrwDv/Ob+GGkK35pg1Tfd+LtAI8McZJFCLPRBXTk NZ6Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584864; x=1729189664; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=7IfrRBwnhP/ARkGggvkJ//bNWbTHtvwsCnjZUdYSoz0=; b=GKkqkWxQnDVho/gwMWPzgcaws9ZNujUKVUoc4UpXtlFsdIGolJp51U1BqYFLLhAte9 b5IupIkZ6+oAE1zIo0N0bOvsp7aKZ2QVXBLWaRLJ8xhIIRcFGAPHhVKG5nlR2KGSnaKA k7xEj0hVgSv0rdqWFkYTcRC9IlVATqpiQ2jd3SqCXI5iLOnRT05SlNHS7ymViol6CQDg 7QSNmJCSzrNLe6E2LWZqa17wPKb786EJGMxRD3hIpHbcl5nCKrEFr9wBQ3DvmKYk1Xnf i6oGlSuqbez6vlMB21k8MLlFFClscSxnL6gJwG6eB+S7PqZwf1beEw2JFP1onDXj/UAM Zr0g== X-Forwarded-Encrypted: i=1; AJvYcCUO7UAnDwCaikCaF+jdDStL2QqxYzTdj2J4lu1P+Y6UfOvr49hiuZRSkRDIgieg2lvtHWJ66VcSA/AlZFadNOXA@lists.infradead.org X-Gm-Message-State: AOJu0Yyc65cNh9p9h2e9+O9P4dNGGlpz57l30wqeCUycYHRtlsb61HHL u1fYvmt/cj1bldgLCN7Rrk/4N2boTOet6Uk7qJzPljEmy6dFbVlAAGsT8T7wWNkluTes5ASI7kx xGQ== X-Google-Smtp-Source: AGHT+IEHmTL+CF0lwT1L0/srW8AumAgUZ3z+xXpT6s6F1mY7sAC4x3PdYh5uuuYsI/CKxaNz7ipzx99kqDg= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a25:ed0b:0:b0:e17:8e73:866c with SMTP id 3f1490d57ef6-e28fe421ddfmr136730276.10.1728584863963; Thu, 10 Oct 2024 11:27:43 -0700 (PDT) Date: Thu, 10 Oct 2024 11:24:23 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-82-seanjc@google.com> Subject: [PATCH v13 81/85] KVM: Make kvm_follow_pfn.refcounted_page a required field From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112745_744120_492E772B X-CRM114-Status: GOOD ( 11.46 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Now that the legacy gfn_to_pfn() APIs are gone, and all callers of hva_to_pfn() pass in a refcounted_page pointer, make it a required field to ensure all future usage in KVM plays nice. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- virt/kvm/kvm_main.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index b2c8d429442d..a483da96f4be 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -2834,8 +2834,7 @@ static kvm_pfn_t kvm_resolve_pfn(struct kvm_follow_pfn *kfp, struct page *page, pfn = page_to_pfn(page); } - if (kfp->refcounted_page) - *kfp->refcounted_page = page; + *kfp->refcounted_page = page; return pfn; } @@ -2986,6 +2985,9 @@ kvm_pfn_t hva_to_pfn(struct kvm_follow_pfn *kfp) might_sleep(); + if (WARN_ON_ONCE(!kfp->refcounted_page)) + return KVM_PFN_ERR_FAULT; + if (hva_to_pfn_fast(kfp, &pfn)) return pfn; From patchwork Thu Oct 10 18:24:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831138 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 576EFD24451 for ; Thu, 10 Oct 2024 21:01:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:Content-Type:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=l0R/snR1lBsefu+Gd9v26W9gBjpskR/lINZXMlcZUSs=; b=PRR1p6U7+d//vu +A0XC2bAbvoe1c0RUDTfH/QLFPgxTS+H/gcK2E/HTXWgCwX9SF1UBfoNvb2aU9kNVuaPNY797/Ccq DGm3drtwfPDeLia4xs53BWPMSKu2TROR88c6kuSpJmuRb7/cbH1BbvLtaqow85qCn2I8WElL4+KK+ Ivu1b99UlgThDPVtIKKbvw4kctSLIeIVxhD/kVef3CbwYHlHq9AOqSethmtmjv5AHbKd7NoDeTZRe kDRRclgFRzeZCxegTEDoPJS3Gm3FcqCWZNQs92BqiJGQKpe1domcch3pHBCJFFhJjC7ynoh4G5+G8 QJ7nkx+VNq7b9UO8gMUg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1sz0Hr-0000000ELoY-1h1N; Thu, 10 Oct 2024 21:01:31 +0000 Received: from mail-yw1-x114a.google.com ([2607:f8b0:4864:20::114a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxt6-0000000Drua-3E2p for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:27:51 +0000 Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-6e2d1860a62so23490337b3.0 for ; Thu, 10 Oct 2024 11:27:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584866; x=1729189666; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=l0R/snR1lBsefu+Gd9v26W9gBjpskR/lINZXMlcZUSs=; b=oW/VXzAGHJr65LOPS7f+dxkMLtFUFWnBWUdwJwkoJynjNOMQA6FyT/YEbpKkDQB8Y1 XeIzl5K+t84icDZE9u4ugYhCQHyhHKiuRnoiR2Lm7C1CCatKcksv+5I0FiTm4qTRbBb0 cvFh3Pt4jkcYCbXegCrQTtJpKleedb6mm9BKTQq01qUz3KYjfrjunAPBGh+J9kARPELz TBCYbi38ooUbc2ZMOdoZYQZumPdBuR5OB6mAmUFQ49qCz1YIqoFGVX+FsfjgnF+fQ1da U7+/aWlz9istv976pUh+e8rfrPsTAHulSo8C7s5jKqn07pvvsJtIBFi3eS1ZDtNdKBDa b4bQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584866; x=1729189666; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=l0R/snR1lBsefu+Gd9v26W9gBjpskR/lINZXMlcZUSs=; b=RRenopegb0HFqRigeePSgAyAB6sLrsbWRhaR6vZWYxgDwhTohCfHIjS673UWz2kCVT Co7mJuFCPgI22963q0lrF7VCKh18aL7zdjKvkfsTV+nlx+aKQc1LkMcniYQShtRLF8AR 2XtXcnz3DSPHEtVtzrp8DCXzxuNUBrKLs0RXWp7LokawrPtwZ+xiks+CjokhuDP0R3Tg 8UhGlwF04HoL8SbbIBDw6n3a203KrsAW0GmT/G/C0m1y141P59TSj+Ey7TfW9qq+q7ul px6liKR0F5x1WbaEfgWtsnERbIYPmM1o92fGKIRj31nGJrMwLspJjuEkUhw7+4YgouZ2 vLmA== X-Forwarded-Encrypted: i=1; AJvYcCXDvlTNV/5LuZ91Exy9TwGqm2WxJsQvTdUp+jvptJaRMH7L+ZfcTJM7y5ynt0ifVDxgpCYGAJ6GHU55hn8Q5esl@lists.infradead.org X-Gm-Message-State: AOJu0YzVkPKf0lBz/lWTXKm9t+5bjxmTefcA5WfShMzRaEAGUcHqEajC hu5LxqbVzmDHl0oIMnpTRBZzXSgfqt6SWK3oWbP4GNhEa6z7vXEAnQH69Gj5Lm6HDL863nlM/LJ sbg== X-Google-Smtp-Source: AGHT+IE49fRlqLzz+ddXUJ4fW719WQNIEaF+exc+KxAZEodOOT6YKCh59uZdX+xzyPwnVHJ15Mb6QwFZ+jU= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:690c:470f:b0:6e3:39e5:f0e8 with SMTP id 00721157ae682-6e339e5f461mr119227b3.6.1728584866570; Thu, 10 Oct 2024 11:27:46 -0700 (PDT) Date: Thu, 10 Oct 2024 11:24:24 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-83-seanjc@google.com> Subject: [PATCH v13 82/85] KVM: x86/mmu: Don't mark "struct page" accessed when zapping SPTEs From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112748_886832_0D5268F8 X-CRM114-Status: GOOD ( 14.92 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Don't mark pages/folios as accessed in the primary MMU when zapping SPTEs, as doing so relies on kvm_pfn_to_refcounted_page(), and generally speaking is unnecessary and wasteful. KVM participates in page aging via mmu_notifiers, so there's no need to push "accessed" updates to the primary MMU. And if KVM zaps a SPTe in response to an mmu_notifier, marking it accessed _after_ the primary MMU has decided to zap the page is likely to go unnoticed, i.e. odds are good that, if the page is being zapped for reclaim, the page will be swapped out regardless of whether or not KVM marks the page accessed. Dropping x86's use of kvm_set_pfn_accessed() also paves the way for removing kvm_pfn_to_refcounted_page() and all its users. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/x86/kvm/mmu/mmu.c | 17 ----------------- arch/x86/kvm/mmu/tdp_mmu.c | 3 --- 2 files changed, 20 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 5acdaf3b1007..55eeca931e23 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -559,10 +559,8 @@ static bool mmu_spte_update(u64 *sptep, u64 new_spte) */ static u64 mmu_spte_clear_track_bits(struct kvm *kvm, u64 *sptep) { - kvm_pfn_t pfn; u64 old_spte = *sptep; int level = sptep_to_sp(sptep)->role.level; - struct page *page; if (!is_shadow_present_pte(old_spte) || !spte_has_volatile_bits(old_spte)) @@ -574,21 +572,6 @@ static u64 mmu_spte_clear_track_bits(struct kvm *kvm, u64 *sptep) return old_spte; kvm_update_page_stats(kvm, level, -1); - - pfn = spte_to_pfn(old_spte); - - /* - * KVM doesn't hold a reference to any pages mapped into the guest, and - * instead uses the mmu_notifier to ensure that KVM unmaps any pages - * before they are reclaimed. Sanity check that, if the pfn is backed - * by a refcounted page, the refcount is elevated. - */ - page = kvm_pfn_to_refcounted_page(pfn); - WARN_ON_ONCE(page && !page_count(page)); - - if (is_accessed_spte(old_spte)) - kvm_set_pfn_accessed(pfn); - return old_spte; } diff --git a/arch/x86/kvm/mmu/tdp_mmu.c b/arch/x86/kvm/mmu/tdp_mmu.c index 8aa0d7a7602b..91caa73a905b 100644 --- a/arch/x86/kvm/mmu/tdp_mmu.c +++ b/arch/x86/kvm/mmu/tdp_mmu.c @@ -861,9 +861,6 @@ static bool tdp_mmu_zap_leafs(struct kvm *kvm, struct kvm_mmu_page *root, tdp_mmu_iter_set_spte(kvm, &iter, SHADOW_NONPRESENT_VALUE); - if (is_accessed_spte(iter.old_spte)) - kvm_set_pfn_accessed(spte_to_pfn(iter.old_spte)); - /* * Zappings SPTEs in invalid roots doesn't require a TLB flush, * see kvm_tdp_mmu_zap_invalidated_roots() for details. From patchwork Thu Oct 10 18:24:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831017 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9C146D24447 for ; Thu, 10 Oct 2024 19:55:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:Content-Type:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=qYRk8t9w2U8aQ/ID4AWjouVrA1tdB4hWvcNmB/sB3HQ=; b=kUWsRGEy7u85lh ct08HyHwMpV2Bw+tds6LQr1i66y9PZzgMjBrISLb4Gv7lb8yv+ZUnR9ATwFqeBD06KkyLsE/S0+1q r2R3Fk+qMN5uVpV/PtWaj+/cgdOPf9tXhD/ImdVSsF/GXs+nRnm5AEuQaowMdfzaCqtMk2QDmY1hX OOuQSvfqa1bj5ZmLGxq/qjjMPVxBuNfiWoTMcwShZRqc97XeIayURfMThTUbz35SWyY7X75yQ4Nr/ yI99JU4yfTfRyiaC5hkbyzhGlE3qhTnT4XS4PTdi68bFDq6m8a/j2BmJa6vgOBfvMJDlBSfaL89Xz MNPjzUgproSiUfXo9lmA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syzFo-0000000ECiS-2VCV; Thu, 10 Oct 2024 19:55:20 +0000 Received: from mail-pl1-x64a.google.com ([2607:f8b0:4864:20::64a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxt7-0000000Drv1-1LFM for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:27:51 +0000 Received: by mail-pl1-x64a.google.com with SMTP id d9443c01a7336-204e310e050so14844465ad.0 for ; Thu, 10 Oct 2024 11:27:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584868; x=1729189668; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=qYRk8t9w2U8aQ/ID4AWjouVrA1tdB4hWvcNmB/sB3HQ=; b=MUIvCWd3OKL3bG2LnW491oxhTPr2o7/fCOnjUOkW6ycXy+5uHrA31BINqdPJuJ3C3/ 5DXZ4T8bTuWTd5pncs9YLc3/n2uR8x54Vs/bYASESllzopfA6VCT9SAWgifi3sSY6Qkl RJ0aG8HQgu0RB6tNsUB6isyfkRPB4kYBT4bcNCJEtYN4QGGG6Gb5tjcu36I2dXx7LaSe eG9yeEcdUT1bz6iz6MXsGzWuvXb823sD9bk5OtzpOhbQRpZvKKKqxXMfUSPh3lkWrLvQ 63KzXC9l1Jx6YZ1qfhtiZQ9PLgmDhUFzpW+u6EFMfvHpAsOXJ5IxILDg0Et7pSJqVR/l vVbQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584868; x=1729189668; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=qYRk8t9w2U8aQ/ID4AWjouVrA1tdB4hWvcNmB/sB3HQ=; b=WqzInM3KjpJzQVRHAK130Hc6m9ORPOM1NDvl9ovLsbCBskxyQts2Lw7YN0AjihETxI kAkOo5K+YwxTqmVCyTXQ2wJQppeezWEG7iaix6FJpL3tV/CCSnHnkSG35Gm5G6JCsfft 7h86iMZHAISS8v+69H54cLtuGtkQmEk0FniL0DHuKP/TxK62Za4BiMOB8bgDAf1SWS20 OmvDRGEBIu3dxTb/IQqkayY5hBSTWHZ8OFldBg1cPXJC+D2XS0U2TN4A0eNpZAWgsFUl LatMQENlfP/M92r7rDINb+iK7b0Umfj4J+yTY1Oxvh0kbnScULAWcgX9R09e9QALmlqE JHvg== X-Forwarded-Encrypted: i=1; AJvYcCV9SE1lKyUamE81p6ep0NrVxF+2OiqwF/dSoWg/qv+E4wXsKgp0zT+KACxXZhqnxuQFhi7pfI+1FKDOCjt2LqkW@lists.infradead.org X-Gm-Message-State: AOJu0YxEUqwnuAsTX3BOF/J7u5CIaefv3cjuqQTR9NHnTiau93Xo3YA3 8XTO3fjhoo+3ybFcRNyJgJbsMX6sMI8ZwTAlMYoQGN6iDQBXisRtXNCo9/qLmgTcBZWJUlgZbol DpQ== X-Google-Smtp-Source: AGHT+IHfwq/leOSDpGNmElHsEevN2NLhktzxL5WWYsNV7s51yqV4fIBatX1NQsHzbv29AfLNcoyCuLVeClM= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a17:902:e745:b0:20c:857b:5dcb with SMTP id d9443c01a7336-20c857b62ffmr639275ad.4.1728584868183; Thu, 10 Oct 2024 11:27:48 -0700 (PDT) Date: Thu, 10 Oct 2024 11:24:25 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-84-seanjc@google.com> Subject: [PATCH v13 83/85] KVM: arm64: Don't mark "struct page" accessed when making SPTE young From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112749_405244_01336A6D X-CRM114-Status: GOOD ( 13.68 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Don't mark pages/folios as accessed in the primary MMU when making a SPTE young in KVM's secondary MMU, as doing so relies on kvm_pfn_to_refcounted_page(), and generally speaking is unnecessary and wasteful. KVM participates in page aging via mmu_notifiers, so there's no need to push "accessed" updates to the primary MMU. Dropping use of kvm_set_pfn_accessed() also paves the way for removing kvm_pfn_to_refcounted_page() and all its users. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- arch/arm64/include/asm/kvm_pgtable.h | 4 +--- arch/arm64/kvm/hyp/pgtable.c | 7 ++----- arch/arm64/kvm/mmu.c | 6 +----- 3 files changed, 4 insertions(+), 13 deletions(-) diff --git a/arch/arm64/include/asm/kvm_pgtable.h b/arch/arm64/include/asm/kvm_pgtable.h index 03f4c3d7839c..aab04097b505 100644 --- a/arch/arm64/include/asm/kvm_pgtable.h +++ b/arch/arm64/include/asm/kvm_pgtable.h @@ -674,10 +674,8 @@ int kvm_pgtable_stage2_wrprotect(struct kvm_pgtable *pgt, u64 addr, u64 size); * * If there is a valid, leaf page-table entry used to translate @addr, then * set the access flag in that entry. - * - * Return: The old page-table entry prior to setting the flag, 0 on failure. */ -kvm_pte_t kvm_pgtable_stage2_mkyoung(struct kvm_pgtable *pgt, u64 addr); +void kvm_pgtable_stage2_mkyoung(struct kvm_pgtable *pgt, u64 addr); /** * kvm_pgtable_stage2_test_clear_young() - Test and optionally clear the access diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c index b11bcebac908..40bd55966540 100644 --- a/arch/arm64/kvm/hyp/pgtable.c +++ b/arch/arm64/kvm/hyp/pgtable.c @@ -1245,19 +1245,16 @@ int kvm_pgtable_stage2_wrprotect(struct kvm_pgtable *pgt, u64 addr, u64 size) NULL, NULL, 0); } -kvm_pte_t kvm_pgtable_stage2_mkyoung(struct kvm_pgtable *pgt, u64 addr) +void kvm_pgtable_stage2_mkyoung(struct kvm_pgtable *pgt, u64 addr) { - kvm_pte_t pte = 0; int ret; ret = stage2_update_leaf_attrs(pgt, addr, 1, KVM_PTE_LEAF_ATTR_LO_S2_AF, 0, - &pte, NULL, + NULL, NULL, KVM_PGTABLE_WALK_HANDLE_FAULT | KVM_PGTABLE_WALK_SHARED); if (!ret) dsb(ishst); - - return pte; } struct stage2_age_data { diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c index 4054356c9712..e2ae9005e333 100644 --- a/arch/arm64/kvm/mmu.c +++ b/arch/arm64/kvm/mmu.c @@ -1706,18 +1706,14 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, /* Resolve the access fault by making the page young again. */ static void handle_access_fault(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa) { - kvm_pte_t pte; struct kvm_s2_mmu *mmu; trace_kvm_access_fault(fault_ipa); read_lock(&vcpu->kvm->mmu_lock); mmu = vcpu->arch.hw_mmu; - pte = kvm_pgtable_stage2_mkyoung(mmu->pgt, fault_ipa); + kvm_pgtable_stage2_mkyoung(mmu->pgt, fault_ipa); read_unlock(&vcpu->kvm->mmu_lock); - - if (kvm_pte_valid(pte)) - kvm_set_pfn_accessed(kvm_pte_to_pfn(pte)); } /** From patchwork Thu Oct 10 18:24:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831310 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9CC77D2444F for ; Thu, 10 Oct 2024 23:31:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:Content-Type:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=BeNSr3VxopvO4QfIOfKviYSPGRm8vVX1K3YR0sLhejo=; b=KI4X3r/jt8tTfx WcTHVA22NsCCk/HB+9e7oE2zjwkIzPhlNboXWsSVrBM2VpWwIV1/htMZ0jeV22uZ7DzYJ6151+9oW hgN72GGbob9uJj98u2DCTkRAU0zBOAf1Y25DiCQz63gxQCGkeM7armCg8wCHYSM+nb7uMHwrTQcHB oOVBs+dh6QEPP70JlxihLbPAFj3HCEIw/vJI85+7HLxvAj48babKrg64rYnYpANirdeAelAENMu6N 49QgTkbxMYersH6z6OJb08Q9i9aTNCgHr4pFUorRAlQnx9Y/N3OKe4CDdx/DPXAwI4wYkwFQEStrt QK6Poi7l9kr35fFhc88Q==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1sz2d5-0000000Ef8S-1zSm; Thu, 10 Oct 2024 23:31:35 +0000 Received: from mail-yw1-x1149.google.com ([2607:f8b0:4864:20::1149]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxt9-0000000Drwo-1p0Y for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:27:54 +0000 Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-6e1fbe2a6b1so23313717b3.2 for ; Thu, 10 Oct 2024 11:27:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584870; x=1729189670; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=BeNSr3VxopvO4QfIOfKviYSPGRm8vVX1K3YR0sLhejo=; b=QrJKmIjvSEAhTIlwVFJOmQ5I7I1VQhdnUVXCZMq3P4JJusmgXnFbWJC6rS25qcKrc9 PN5Xs1EwDCBnS6Jfegln//omwt2zD+qAgL/jptfn/AmjxhsbXA9WzAJnKDAbX142AR6h RNYj2DIR2JEFC5wyhzG4jhM0r+T/Z/aReqH/PvQPn9B40wk+edj62VBTntkLj/5FBB25 QYk85cbXR2hELgL/DYbNsXGMlrv7GaJaMXWXi7F9kTmNYdUQHst26FT47gHsPnKmbLt7 75NSePZV/v2zlVeBCcHeWFDP2qwe6gTBBEdY9I3NyXl+0CYfK1SourHBzLUeeikafN8J vQPQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584870; x=1729189670; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=BeNSr3VxopvO4QfIOfKviYSPGRm8vVX1K3YR0sLhejo=; b=aQGXMn0WJ0G80Gw5RDWt5SEtUCHBMbCJRgeXXBg+OwRvKP9Ofey8QU2QSAizqhFCEu XidS4no8pPRP4M+xvQnmEp6Y2Bjq44HjvlszMWiMQYDSkU0xrJOtSHT177Ct0GQNR96M 0qyG/jHUpDk6r40Mh5hiLbjt+5tS1p9Vb+UbsWE4RlW3UFVllbkd2jv2Lr2od1TMYBXB iPYWpTUAmbqaF0ky4XYxP+fKd8uMMxqD3gSNTM4ykSJZbMy74C//0vaU8OY2e2x89yui 3PER/iIOks7gpVvgyNngeB4N75+YWzDG/+SyNO51ARj4nc4c8BkCfLmmmUHauRIxj27O R9Rw== X-Forwarded-Encrypted: i=1; AJvYcCVLvIiB33V6rVGno8Yw3qV5LyKT2msykVHcbPJCIw/+z5hN269SEJR1GS/IsjXKdxX2dYsWp4cHaqHPe9xA4UQL@lists.infradead.org X-Gm-Message-State: AOJu0YxJB0E0Mx7DaWQVmRdWbXRu39Pv2HjuSaEE1P81WuWNuTfS6/Mp +6Cgo7YIk1OVYyk1ZqBwbxqnY625INnC0hlMaCiyNUrPQGc/le8gTHUpZNnZvaXK1fKmC9qwRYw lDw== X-Google-Smtp-Source: AGHT+IHhmEpfdjhYs/g96WpbaMl3fao5EtfU192MAf3yyGbJNoRiDL4cTPbNYR1fq37SA+x5liddjIzSKiU= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a05:690c:2e86:b0:6dd:bcce:7cd4 with SMTP id 00721157ae682-6e3221404ddmr516647b3.2.1728584870148; Thu, 10 Oct 2024 11:27:50 -0700 (PDT) Date: Thu, 10 Oct 2024 11:24:26 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-85-seanjc@google.com> Subject: [PATCH v13 84/85] KVM: Drop APIs that manipulate "struct page" via pfns From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112751_633890_F1BE3366 X-CRM114-Status: GOOD ( 10.94 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Remove all kvm_{release,set}_pfn_*() APIs now that all users are gone. No functional change intended. Reviewed-by: Alex Bennée Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- include/linux/kvm_host.h | 5 ---- virt/kvm/kvm_main.c | 55 ---------------------------------------- 2 files changed, 60 deletions(-) diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 4a1eaa40a215..d045f8310a48 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -1274,11 +1274,6 @@ static inline kvm_pfn_t kvm_faultin_pfn(struct kvm_vcpu *vcpu, gfn_t gfn, write ? FOLL_WRITE : 0, writable, refcounted_page); } -void kvm_release_pfn_clean(kvm_pfn_t pfn); -void kvm_release_pfn_dirty(kvm_pfn_t pfn); -void kvm_set_pfn_dirty(kvm_pfn_t pfn); -void kvm_set_pfn_accessed(kvm_pfn_t pfn); - int kvm_read_guest_page(struct kvm *kvm, gfn_t gfn, void *data, int offset, int len); int kvm_read_guest(struct kvm *kvm, gpa_t gpa, void *data, unsigned long len); diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index a483da96f4be..396ca14f18f3 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -3164,61 +3164,6 @@ void kvm_vcpu_unmap(struct kvm_vcpu *vcpu, struct kvm_host_map *map) } EXPORT_SYMBOL_GPL(kvm_vcpu_unmap); -void kvm_release_pfn_clean(kvm_pfn_t pfn) -{ - struct page *page; - - if (is_error_noslot_pfn(pfn)) - return; - - page = kvm_pfn_to_refcounted_page(pfn); - if (!page) - return; - - kvm_release_page_clean(page); -} -EXPORT_SYMBOL_GPL(kvm_release_pfn_clean); - -void kvm_release_pfn_dirty(kvm_pfn_t pfn) -{ - struct page *page; - - if (is_error_noslot_pfn(pfn)) - return; - - page = kvm_pfn_to_refcounted_page(pfn); - if (!page) - return; - - kvm_release_page_dirty(page); -} -EXPORT_SYMBOL_GPL(kvm_release_pfn_dirty); - -/* - * Note, checking for an error/noslot pfn is the caller's responsibility when - * directly marking a page dirty/accessed. Unlike the "release" helpers, the - * "set" helpers are not to be used when the pfn might point at garbage. - */ -void kvm_set_pfn_dirty(kvm_pfn_t pfn) -{ - if (WARN_ON(is_error_noslot_pfn(pfn))) - return; - - if (pfn_valid(pfn)) - kvm_set_page_dirty(pfn_to_page(pfn)); -} -EXPORT_SYMBOL_GPL(kvm_set_pfn_dirty); - -void kvm_set_pfn_accessed(kvm_pfn_t pfn) -{ - if (WARN_ON(is_error_noslot_pfn(pfn))) - return; - - if (pfn_valid(pfn)) - kvm_set_page_accessed(pfn_to_page(pfn)); -} -EXPORT_SYMBOL_GPL(kvm_set_pfn_accessed); - static int next_segment(unsigned long len, int offset) { if (len > PAGE_SIZE - offset) From patchwork Thu Oct 10 18:24:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 13831018 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id F0025D24447 for ; Thu, 10 Oct 2024 19:58:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Reply-To:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:Content-Type:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=nwXbeHTlA0ocP8s6EMi2R3V8WkxZqKgY3fo4QQ/WBXk=; b=cKyiU0C4OFf0mL Y/pE/FswYwWdYFRDeF0ERsC0qaEhYU81gBBhxOB5ZZtPJ7WwPhavp0vYm6g8732FrCRZq7lRrn+f4 gDdV/8xgvLA8W6SZy5E8cfVqtOdZ3JL70c0LEz78sVWF5udDWxIlROxIDZOqPbwFOBt5gy4yyMRf5 GbISBPuYwSYvI1ZqXX0wvH5DTJuJN7uqRTupbSZzydHOohxvrdiVJPSyAvaiOYlU4TIh6WtqzsqFz FDYYFHyNluwOy4DGX9h8Kwn7ZL2wPE6cEDUQzAUQIjXCbgbFW44ix9ftC6tm0dS0cmyQgN/Ya6/JE AMCDIpBNPQvOjkIU1/Dg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syzIR-0000000ED75-4BUi; Thu, 10 Oct 2024 19:58:04 +0000 Received: from mail-pg1-x549.google.com ([2607:f8b0:4864:20::549]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syxtC-0000000Drz5-210O for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 18:27:56 +0000 Received: by mail-pg1-x549.google.com with SMTP id 41be03b00d2f7-7b696999c65so864045a12.3 for ; Thu, 10 Oct 2024 11:27:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728584873; x=1729189673; darn=lists.infradead.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:from:to:cc:subject:date :message-id:reply-to; bh=nwXbeHTlA0ocP8s6EMi2R3V8WkxZqKgY3fo4QQ/WBXk=; b=C91LwOthEGXkxiPQx1j9UVtQEFjUoPUaMYYqnTNP0TCswa9+hhVSEARBcovA5nnV2c 32Zbt1YX9VZVNqA/MIhC1E9BJoIzGjcTF5OAV0U6hy0FCdm9R8+piH4EntBUT5Flzs0g 7XRT8akLUDK2TQ7w5LYG7bObbSDKy6EuyXBiGjndcjvGJU4XUeHMGrzHCRWi3yzoHhuS sjBWzkfEmw+kj5JGm+gKX2+ScM8kNPIL2wd94uiV/uBTA9RB+YhiZeF2TIqolgG7sF1b +fe9fpJERCCgWOqYOz0sVLDF1YIWhDfhuugU+/HTW/vPaD/KWNLvNxRYp2hahT2HA9dL ndpQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728584873; x=1729189673; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:reply-to:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=nwXbeHTlA0ocP8s6EMi2R3V8WkxZqKgY3fo4QQ/WBXk=; b=VzATI7w0VN9p32Iu93qW4WcmN0Vj7zw5Yn+VB6EzawYRpQSocomEaVQYwJ4hU+BSz5 4jdNlz2thg4P/W/qpKdzS5JqS3NY9vYjdPpSSaU+/ZScZ/nbOGcH8LML9zSzXcz/sRuC m6M0dYFEUiTALIsH85L5SB5ksqRonfuOUPv51QizuqSx76vHL0O1g7948/ifmnrDAFQf JoLkl0r+65yaIOLrMugAyfnX9PMH3iuZTAEz3ZfZBXDKtsx7ss1j6b8hdX69aMDxf31Q d6yXi17nM9PmxGhTJTFyruUMDql5bTtYJMpHLriBYXJ722Hsg6FXc5h7m6VsHahrvJiI /+eg== X-Forwarded-Encrypted: i=1; AJvYcCVI1Y4rxihgtMJniHffjThHQBIEKjBC1AbR/wijwa8xlZpR7ZrRqJdyZ7KihdrdJy8K1BO7M7sk+FQ2JWE8EHX5@lists.infradead.org X-Gm-Message-State: AOJu0Yx37YxgxWCr3/KOSszi17Bf2JAL6PnWVXfB4HHgJQxGpRu4goPf rJtQCFPnDBcMPHkK/YSWLOMKkEj92+pDKODN8Hy9EYsjXdG6K9ToJDq0GJWvKZgEesAv7peYLY8 rvQ== X-Google-Smtp-Source: AGHT+IHZp/jcJ6VF/U4r8Ds8cBqCm7oOG+WyWj8b167iLxGpwwfpynwEnbVm0Ou2T4AqluEw0PW/YyS/yKE= X-Received: from zagreus.c.googlers.com ([fda3:e722:ac3:cc00:9d:3983:ac13:c240]) (user=seanjc job=sendgmr) by 2002:a63:e715:0:b0:7d5:d088:57d4 with SMTP id 41be03b00d2f7-7ea535b6b4fmr26a12.9.1728584872016; Thu, 10 Oct 2024 11:27:52 -0700 (PDT) Date: Thu, 10 Oct 2024 11:24:27 -0700 In-Reply-To: <20241010182427.1434605-1-seanjc@google.com> Mime-Version: 1.0 References: <20241010182427.1434605-1-seanjc@google.com> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog Message-ID: <20241010182427.1434605-86-seanjc@google.com> Subject: [PATCH v13 85/85] KVM: Don't grab reference on VM_MIXEDMAP pfns that have a "struct page" From: Sean Christopherson To: Paolo Bonzini , Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Christian Borntraeger , Janosch Frank , Claudio Imbrenda , Sean Christopherson Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, loongarch@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, " =?utf-8?q?Alex_Benn=C3=A9e?= " , Yan Zhao , David Matlack , David Stevens , Andrew Jones X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_112754_615256_517A0A6F X-CRM114-Status: GOOD ( 21.82 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Sean Christopherson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Now that KVM no longer relies on an ugly heuristic to find its struct page references, i.e. now that KVM can't get false positives on VM_MIXEDMAP pfns, remove KVM's hack to elevate the refcount for pfns that happen to have a valid struct page. In addition to removing a long-standing wart in KVM, this allows KVM to map non-refcounted struct page memory into the guest, e.g. for exposing GPU TTM buffers to KVM guests. Tested-by: Alex Bennée Signed-off-by: Sean Christopherson --- include/linux/kvm_host.h | 3 -- virt/kvm/kvm_main.c | 75 ++-------------------------------------- 2 files changed, 2 insertions(+), 76 deletions(-) diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index d045f8310a48..02f0206fd2dc 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -1730,9 +1730,6 @@ void kvm_arch_sync_events(struct kvm *kvm); int kvm_cpu_has_pending_timer(struct kvm_vcpu *vcpu); -struct page *kvm_pfn_to_refcounted_page(kvm_pfn_t pfn); -bool kvm_is_zone_device_page(struct page *page); - struct kvm_irq_ack_notifier { struct hlist_node link; unsigned gsi; diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 396ca14f18f3..b1b10dc408a0 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -160,52 +160,6 @@ __weak void kvm_arch_guest_memory_reclaimed(struct kvm *kvm) { } -bool kvm_is_zone_device_page(struct page *page) -{ - /* - * The metadata used by is_zone_device_page() to determine whether or - * not a page is ZONE_DEVICE is guaranteed to be valid if and only if - * the device has been pinned, e.g. by get_user_pages(). WARN if the - * page_count() is zero to help detect bad usage of this helper. - */ - if (WARN_ON_ONCE(!page_count(page))) - return false; - - return is_zone_device_page(page); -} - -/* - * Returns a 'struct page' if the pfn is "valid" and backed by a refcounted - * page, NULL otherwise. Note, the list of refcounted PG_reserved page types - * is likely incomplete, it has been compiled purely through people wanting to - * back guest with a certain type of memory and encountering issues. - */ -struct page *kvm_pfn_to_refcounted_page(kvm_pfn_t pfn) -{ - struct page *page; - - if (!pfn_valid(pfn)) - return NULL; - - page = pfn_to_page(pfn); - if (!PageReserved(page)) - return page; - - /* The ZERO_PAGE(s) is marked PG_reserved, but is refcounted. */ - if (is_zero_pfn(pfn)) - return page; - - /* - * ZONE_DEVICE pages currently set PG_reserved, but from a refcounting - * perspective they are "normal" pages, albeit with slightly different - * usage rules. - */ - if (kvm_is_zone_device_page(page)) - return page; - - return NULL; -} - /* * Switches to specified vcpu, until a matching vcpu_put() */ @@ -2804,35 +2758,10 @@ static kvm_pfn_t kvm_resolve_pfn(struct kvm_follow_pfn *kfp, struct page *page, if (kfp->map_writable) *kfp->map_writable = writable; - /* - * FIXME: Remove this once KVM no longer blindly calls put_page() on - * every pfn that points at a struct page. - * - * Get a reference for follow_pte() pfns if they happen to point at a - * struct page, as KVM will ultimately call kvm_release_pfn_clean() on - * the returned pfn, i.e. KVM expects to have a reference. - * - * Certain IO or PFNMAP mappings can be backed with valid struct pages, - * but be allocated without refcounting, e.g. tail pages of - * non-compound higher order allocations. Grabbing and putting a - * reference to such pages would cause KVM to prematurely free a page - * it doesn't own (KVM gets and puts the one and only reference). - * Don't allow those pages until the FIXME is resolved. - * - * Don't grab a reference for pins, callers that pin pages are required - * to check refcounted_page, i.e. must not blindly release the pfn. - */ - if (map) { + if (map) pfn = map->pfn; - - if (!kfp->pin) { - page = kvm_pfn_to_refcounted_page(pfn); - if (page && !get_page_unless_zero(page)) - return KVM_PFN_ERR_FAULT; - } - } else { + else pfn = page_to_pfn(page); - } *kfp->refcounted_page = page;