From patchwork Thu Oct 10 08:59:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fuad Tabba X-Patchwork-Id: 13829776 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 071FBCF07D5 for ; Thu, 10 Oct 2024 08:59:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 14A266B0096; Thu, 10 Oct 2024 04:59:49 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0CF9A6B0099; Thu, 10 Oct 2024 04:59:48 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E1A006B009A; Thu, 10 Oct 2024 04:59:48 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id BCF546B0096 for ; Thu, 10 Oct 2024 04:59:48 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 0D904C0B2A for ; Thu, 10 Oct 2024 08:59:45 +0000 (UTC) X-FDA: 82657094856.04.D79E63E Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) by imf06.hostedemail.com (Postfix) with ESMTP id C40F2180012 for ; Thu, 10 Oct 2024 08:59:45 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=XO5AfexE; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf06.hostedemail.com: domain of 3gZcHZwUKCA47oppou22uzs.q20zw18B-00y9oqy.25u@flex--tabba.bounces.google.com designates 209.85.128.201 as permitted sender) smtp.mailfrom=3gZcHZwUKCA47oppou22uzs.q20zw18B-00y9oqy.25u@flex--tabba.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1728550685; a=rsa-sha256; cv=none; b=tHVMUgWUMNwDRRXrqEnxqeXef3u9RHS8mVZtBYyzSa6U9t8jcDUPyv/4UcScW0WaXLFYZq yop1b/bG9gCczCRucaqZcDuAbIeURi7JvgFpkbZlVrnMNL1rWqHZ7Se1Zg85ixaRvw6+AK /rkiXsqySvtkELDWilc8ejFhRhpnu2g= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=XO5AfexE; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf06.hostedemail.com: domain of 3gZcHZwUKCA47oppou22uzs.q20zw18B-00y9oqy.25u@flex--tabba.bounces.google.com designates 209.85.128.201 as permitted sender) smtp.mailfrom=3gZcHZwUKCA47oppou22uzs.q20zw18B-00y9oqy.25u@flex--tabba.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1728550685; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=8uqb2Hq7+/kapW0F00vPZqAJ1bU0uvb9gYwuhhY1Sds=; b=O0rIFmZrE5ohjoG4Xq0lKscSefUON7qVAGu+K3GFDp3j+DAD8GadxNKyTesL/PxWbTJGFT i7xcHVb/xdEoIV/7ONQJGKN2OiLdLzEZmPg51G9gwvYHEmx51g/aQGHWhw+FQ8b+StgUIw W/LTW8zif2RvoP5VAyiwQbfFeUNU2zo= Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-6e21dcc7044so12980037b3.1 for ; Thu, 10 Oct 2024 01:59:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1728550786; x=1729155586; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=8uqb2Hq7+/kapW0F00vPZqAJ1bU0uvb9gYwuhhY1Sds=; b=XO5AfexE+0DQl49ceULnc8iBFiNWb1sbUerZNmgMDeKYd3SKw/+vd+8lxDX3Zd+lbu XUtmoUvuVLYn04UdclgpCM5xXY3gYj+HXsQWimX+fCkkT9HnREIG1laFrTEIvXP/B6DO NhoSkKNIJ9Rtz8nvsQX2m4f33TfEdZv77ZMpkgAgLOsQmWA6uiaLNAISK+zvDIKiQ5QA OwkvFn3XJ0UjafIlR0tPFtP1gbPrs9zlg4cLA6+Y2hzp/1gno01CRPXTz9KRYudK5Vf8 LXwTQtHVDYiwk3RYw9xms9g1aZVm2CEatR9Qd2jo2QEGM/Yqwla3AAkT4iimDEf0ZPHt J2wQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728550786; x=1729155586; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=8uqb2Hq7+/kapW0F00vPZqAJ1bU0uvb9gYwuhhY1Sds=; b=vlJdJPTv7LraeEtlePxepZr8Iyjc8JzBSdqhO94P/+ut+cnSI74teq71XrxepspLW5 KSF+CTtjvhMNCesGisJWjMTJ68P7XUV44IWvUE4Jdsq/rrxeA0RZj/e+Eqi/kE/xK7le pM9oas/HWvcd5A/Fm4Vdz783wJGmdu5RQsX126cnDsElxfRJ+NTaobiQ0EDeCLRiVjZp Nyx7XdMYaFUr9cy018pZ2T/QfJvtduPa07W7AQ3SMHNGPgv1cte3gQzJhAApvCS4lnDh YoFaqBSLmD7iTRH+U4BHhDDOhpK5GlFhIvh1IruFBhityPfmkmHsq6jzYXyng4axv2Ah 4OtA== X-Forwarded-Encrypted: i=1; AJvYcCX/JUEpnJCXowyamOfCqqUYKO979yEy0POJXbZYQmfgVT0SNnvB31LbrboYcv/jC6RFunR0PJf4VA==@kvack.org X-Gm-Message-State: AOJu0Yy6wR02fuwBvR039ZVSqCzqWAZ6F3tkVpEcqQKQqzyRWQBny5Y6 A3o8MNcx2In4Gmv6xY3M6+YWo37Ae8UDILs3itlYpCnA1oon1Hj1JJkiNo2YHb9DMnVbzLo9lQ= = X-Google-Smtp-Source: AGHT+IG7SQRbRuMfYH/Rui+CBChrEOHfesWWRZDd/p/7xcfBTKh/y+cfP2FSghBgXkgmmW6X7QZVv1DCpQ== X-Received: from fuad.c.googlers.com ([fda3:e722:ac3:cc00:28:9cb1:c0a8:1613]) (user=tabba job=sendgmr) by 2002:a81:b545:0:b0:6e2:19f3:ff75 with SMTP id 00721157ae682-6e3221a89dbmr187767b3.6.1728550785757; Thu, 10 Oct 2024 01:59:45 -0700 (PDT) Date: Thu, 10 Oct 2024 09:59:24 +0100 In-Reply-To: <20241010085930.1546800-1-tabba@google.com> Mime-Version: 1.0 References: <20241010085930.1546800-1-tabba@google.com> X-Mailer: git-send-email 2.47.0.rc0.187.ge670bccf7e-goog Message-ID: <20241010085930.1546800-6-tabba@google.com> Subject: [PATCH v3 05/11] KVM: guest_memfd: Add guest_memfd support to kvm_(read|/write)_guest_page() From: Fuad Tabba To: kvm@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-mm@kvack.org Cc: pbonzini@redhat.com, chenhuacai@kernel.org, mpe@ellerman.id.au, anup@brainfault.org, paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, seanjc@google.com, viro@zeniv.linux.org.uk, brauner@kernel.org, willy@infradead.org, akpm@linux-foundation.org, xiaoyao.li@intel.com, yilun.xu@intel.com, chao.p.peng@linux.intel.com, jarkko@kernel.org, amoorthy@google.com, dmatlack@google.com, yu.c.zhang@linux.intel.com, isaku.yamahata@intel.com, mic@digikod.net, vbabka@suse.cz, vannapurve@google.com, ackerleytng@google.com, mail@maciej.szmigiero.name, david@redhat.com, michael.roth@amd.com, wei.w.wang@intel.com, liam.merwick@oracle.com, isaku.yamahata@gmail.com, kirill.shutemov@linux.intel.com, suzuki.poulose@arm.com, steven.price@arm.com, quic_eberman@quicinc.com, quic_mnalajal@quicinc.com, quic_tsoni@quicinc.com, quic_svaddagi@quicinc.com, quic_cvanscha@quicinc.com, quic_pderrin@quicinc.com, quic_pheragu@quicinc.com, catalin.marinas@arm.com, james.morse@arm.com, yuzenghui@huawei.com, oliver.upton@linux.dev, maz@kernel.org, will@kernel.org, qperret@google.com, keirf@google.com, roypat@amazon.co.uk, shuah@kernel.org, hch@infradead.org, jgg@nvidia.com, rientjes@google.com, jhubbard@nvidia.com, fvdl@google.com, hughd@google.com, jthoughton@google.com, tabba@google.com X-Rspamd-Queue-Id: C40F2180012 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: rc193uw9poh5u9nui4g16dkqd8kbexf9 X-HE-Tag: 1728550785-514522 X-HE-Meta: U2FsdGVkX1+BS0lqkZyYHQVzJJXTd2Ia0LmM0051XW1YIPupkV46O/MoUJ2Ue9Jktbt6tRlbbEG5n8ha1f/DNb3dkGiAa6yA5RuppmBY/GGdF7jCX7l/7OwrIdSXYsSuN/1lEMEjRC0QHXVjOKqtFegIdwI051nEV1WnfqDhAsV1gfqHXhoGxNrCJX44YqBAftFZPFAhjs/HkW7Z0zJrPlJ9Ax2KNQfct4LQ7FVMfYhPL9aHWJm0Wh+V11+NEDZ0RpqaBORAwcJa2hFsczTosRMuPREaclBelO1h+Fee7IDHFsz7e5dOAuTbHIt1JrxusMj6OJK9/Kcc/y1hqZG8mmPxsvAcha3rssjrb0uQTotO8k0/F3tBFEXbR2/yVOkEnzXOvbdmc5kPINbd9GhAcStSNj8uYitKVT0yMx0aM9YQUBSU517lugZfx0q4OOnt7AkLb4RmlcdNPfWl/PFaweIzNf3uymCHysG9ytji/fb2NdcYNcdpEGGvo6sdOli4aJNZv9aMnE4Q/fHeZiFl0Pm/a0c5UEXje53FZT1gGFDOf40fjzQmSBHeP/4zueyr5sGhqQ2xRfbbML9YK0DgUr0w7rvbLpMVAe8wzQ0uprQAWVhxB48SUCzX/nlgeEeQAQcFCyrQr3sKXR2fdiBUCG1Z5AHPAHTHyQpcQhYBFpSnJeqicDiJeLoUBx+QIInzJRYwS7SNOAYinWgoumrF79l4TrddlfyMmOop8u5pWUA+mTUlgBaQ4OhFqDh1AF6f2R1GktHFX/N4e7Qx4Kyz7npNFgxf2RYHuNHs0xuXoXa0wzRj5iOcG6D+XldMfIx9cMfO0ebcv+I78j9WZPDl+ZlFWw2ZVnCHbIFLotr+l6AvlxATlfuIjBhkouRaYLcpHh0hRbL312SDxakfl0eH01P79guD8u4MWrdkDwB+gxQbElz1a2JZoOdYFjOWqkGLLv1cW3l5xCROA+qVR7U 34K9tCvN /MEi4rjc76umGzbIGC/dFKdb4enXtedPmGnWrKoqpeqqEbjncOnTqL6D9nnoMvXhJV5XqM1ulPwdUWPQPrirhDcg6ZfcxNWLf4SQOUcZSi6T98vAasDIrKNpOmhHHGVfLixf5IDHTA77PaK+GGcvBcz2d3tSV6zfacmymQ5u8m0I+P+ooJPFTAVmn3J+OKL45N9lMAjR6PfbL+RxVOnc/G/KZ1Kw+sK3dB1rLtYUeX1fpHdeKkKAmH2cFTewNU8yYBDKMpux7jXugLByMUfQOBv5vvGNUjH2CMw2jLrJtErOMm1zPvpWDDiWv182JD1STsQTrH31VSAHv8zSCZdFPwcyAAmeFwdspW2OYMvNWkBWnoI4WlnkROLivHAmFgmIUokk8EvzXrWlb/5wwPjc94zq9EawmOpdfl8Wgf3lH4mbBTnhJXHvrg1iDBWa+8Ap8UXzS/msTZVLDr3kM6rmBYZ67kaGNp0njfd7JHrRWMOE1vSfsz5v8AxOkotEL/74HLYNfZexFU2EfQj8iBnfZnnoR4kXiwAsUCXFvw6yiJe69yVLNCHUlgtMAGNP+AkleDEBiYW2FDGfmn77ymeK+zoMryJr6gA9x/TAnUaPhsxu/HQjctQMWltyYzmaTArAV4O64LY0GOBrucU/9UM6rKC9R9g== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000012, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Make kvm_(read|/write)_guest_page() capable of accessing guest memory for slots that don't have a userspace address, but only if the memory is mappable, which also indicates that it is accessible by the host. Signed-off-by: Fuad Tabba --- virt/kvm/kvm_main.c | 137 ++++++++++++++++++++++++++++++++++++++------ 1 file changed, 118 insertions(+), 19 deletions(-) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index aed9cf2f1685..77e6412034b9 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -3399,23 +3399,114 @@ int kvm_gmem_clear_mappable(struct kvm *kvm, gfn_t start, gfn_t end) return kvm_gmem_toggle_mappable(kvm, start, end, false); } +static int __kvm_read_guest_memfd_page(struct kvm *kvm, + struct kvm_memory_slot *slot, + gfn_t gfn, void *data, int offset, + int len) +{ + struct page *page; + u64 pfn; + int r; + + /* + * Holds the folio lock until after checking whether it can be faulted + * in, to avoid races with paths that change a folio's mappability. + */ + r = kvm_gmem_get_pfn_locked(kvm, slot, gfn, &pfn, NULL); + if (r) + return r; + + page = pfn_to_page(pfn); + + if (!kvm_gmem_is_mappable(kvm, gfn, gfn + 1)) { + r = -EPERM; + goto unlock; + } + memcpy(data, page_address(page) + offset, len); +unlock: + if (r) + put_page(page); + else + kvm_release_pfn_clean(pfn); + unlock_page(page); + + return r; +} + +static int __kvm_write_guest_memfd_page(struct kvm *kvm, + struct kvm_memory_slot *slot, + gfn_t gfn, const void *data, + int offset, int len) +{ + struct page *page; + u64 pfn; + int r; + + /* + * Holds the folio lock until after checking whether it can be faulted + * in, to avoid races with paths that change a folio's mappability. + */ + r = kvm_gmem_get_pfn_locked(kvm, slot, gfn, &pfn, NULL); + if (r) + return r; + + page = pfn_to_page(pfn); + + if (!kvm_gmem_is_mappable(kvm, gfn, gfn + 1)) { + r = -EPERM; + goto unlock; + } + memcpy(page_address(page) + offset, data, len); +unlock: + if (r) + put_page(page); + else + kvm_release_pfn_dirty(pfn); + unlock_page(page); + + return r; +} +#else +static int __kvm_read_guest_memfd_page(struct kvm *kvm, + struct kvm_memory_slot *slot, + gfn_t gfn, void *data, int offset, + int len) +{ + WARN_ON_ONCE(1); + return -EIO; +} + +static int __kvm_write_guest_memfd_page(struct kvm *kvm, + struct kvm_memory_slot *slot, + gfn_t gfn, const void *data, + int offset, int len) +{ + WARN_ON_ONCE(1); + return -EIO; +} #endif /* CONFIG_KVM_GMEM_MAPPABLE */ /* Copy @len bytes from guest memory at '(@gfn * PAGE_SIZE) + @offset' to @data */ -static int __kvm_read_guest_page(struct kvm_memory_slot *slot, gfn_t gfn, - void *data, int offset, int len) + +static int __kvm_read_guest_page(struct kvm *kvm, struct kvm_memory_slot *slot, + gfn_t gfn, void *data, int offset, int len) { - int r; unsigned long addr; if (WARN_ON_ONCE(offset + len > PAGE_SIZE)) return -EFAULT; + if (IS_ENABLED(CONFIG_KVM_GMEM_MAPPABLE) && + kvm_slot_can_be_private(slot) && + !slot->userspace_addr) { + return __kvm_read_guest_memfd_page(kvm, slot, gfn, data, + offset, len); + } + addr = gfn_to_hva_memslot_prot(slot, gfn, NULL); if (kvm_is_error_hva(addr)) return -EFAULT; - r = __copy_from_user(data, (void __user *)addr + offset, len); - if (r) + if (__copy_from_user(data, (void __user *)addr + offset, len)) return -EFAULT; return 0; } @@ -3425,7 +3516,7 @@ int kvm_read_guest_page(struct kvm *kvm, gfn_t gfn, void *data, int offset, { struct kvm_memory_slot *slot = gfn_to_memslot(kvm, gfn); - return __kvm_read_guest_page(slot, gfn, data, offset, len); + return __kvm_read_guest_page(kvm, slot, gfn, data, offset, len); } EXPORT_SYMBOL_GPL(kvm_read_guest_page); @@ -3434,7 +3525,7 @@ int kvm_vcpu_read_guest_page(struct kvm_vcpu *vcpu, gfn_t gfn, void *data, { struct kvm_memory_slot *slot = kvm_vcpu_gfn_to_memslot(vcpu, gfn); - return __kvm_read_guest_page(slot, gfn, data, offset, len); + return __kvm_read_guest_page(vcpu->kvm, slot, gfn, data, offset, len); } EXPORT_SYMBOL_GPL(kvm_vcpu_read_guest_page); @@ -3511,22 +3602,30 @@ EXPORT_SYMBOL_GPL(kvm_vcpu_read_guest_atomic); /* Copy @len bytes from @data into guest memory at '(@gfn * PAGE_SIZE) + @offset' */ static int __kvm_write_guest_page(struct kvm *kvm, - struct kvm_memory_slot *memslot, gfn_t gfn, - const void *data, int offset, int len) + struct kvm_memory_slot *slot, gfn_t gfn, + const void *data, int offset, int len) { - int r; - unsigned long addr; - if (WARN_ON_ONCE(offset + len > PAGE_SIZE)) return -EFAULT; - addr = gfn_to_hva_memslot(memslot, gfn); - if (kvm_is_error_hva(addr)) - return -EFAULT; - r = __copy_to_user((void __user *)addr + offset, data, len); - if (r) - return -EFAULT; - mark_page_dirty_in_slot(kvm, memslot, gfn); + if (IS_ENABLED(CONFIG_KVM_GMEM_MAPPABLE) && + kvm_slot_can_be_private(slot) && + !slot->userspace_addr) { + int r = __kvm_write_guest_memfd_page(kvm, slot, gfn, data, + offset, len); + + if (r) + return r; + } else { + unsigned long addr = gfn_to_hva_memslot(slot, gfn); + + if (kvm_is_error_hva(addr)) + return -EFAULT; + if (__copy_to_user((void __user *)addr + offset, data, len)) + return -EFAULT; + } + + mark_page_dirty_in_slot(kvm, slot, gfn); return 0; }