From patchwork Fri Mar 28 15:31:31 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fuad Tabba X-Patchwork-Id: 14032184 Received: from mail-wm1-f74.google.com (mail-wm1-f74.google.com [209.85.128.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E04691D88AC for ; Fri, 28 Mar 2025 15:31:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743175907; cv=none; b=dFtJq6SDHy59qrt2xbnYoGLLe+OvnnfeeaXYc+VlsooDja3ELj65rQnFIGH9KaPh8ZpFLgeuykKfHOrouf2boB9tRhr2mUhcEiZA/uREDuZpvODtSK/O7koTsOnlrrIawBKTyST2bVwLgpC1Es96XiQTPeina7d6raoByUEqqcE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743175907; c=relaxed/simple; bh=FNZ5rbF/GwQ04BpQvSx2eGZdOReGNlh9r02eIimMdjs=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=azpGcLXTw3qvJtewtksi3iMH/4nKGx9/n8J/Ji8rLs4QxsblVDieCdSgUrEoksb+JWgMV4aHcGvxhFg9McGdwOqkkBCsjMNmYnPEM5BPu6q3+/H+sN8bUqQG+Qhv/EeFIn8Y1H4KqeMyZp5F4Yrr+WFbI/G7TGu5k3D/iCYYdRo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--tabba.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=2aYYOI0K; arc=none smtp.client-ip=209.85.128.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--tabba.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="2aYYOI0K" Received: by mail-wm1-f74.google.com with SMTP id 5b1f17b1804b1-4394040fea1so12110695e9.0 for ; Fri, 28 Mar 2025 08:31:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1743175904; x=1743780704; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=jBMUpGqTWD4UpO2EN6fhtj6WpgPcfU+elgx/+TqNl3U=; b=2aYYOI0KPANBxjlszDqd/xXP/bSZ6k0VyLFVb2g3Hjwfjip072qB5SQ47pKumhA/wi /NKDhMQ8odpO2gBmkDm91ir0EEk/pKon2pTIJioYRZaZDBmT6uOm2YAiNh2/6b+8h59m 1SGvh8At4SnqpqFin24IehveOR4TFHYji+OKdlCdqhOFb6QxmwogndJ01crbb9UGlqIt Xwrv3BwBPc8ozKG29f/GeGyrFPUWm/2i9SSc3h/Q0EmsMlo3S3/tiPS78BXynrMIYXqO 9u98XKkOGXEq29yLrMHLP04DgyuyugmcQoDQ015OKmnirrkejeTAkie7K5JTFgTaSGGs VSmw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1743175904; x=1743780704; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=jBMUpGqTWD4UpO2EN6fhtj6WpgPcfU+elgx/+TqNl3U=; b=PMC+Rc493MunSiWBMmXzarLgHv6Q/kPInDdbg1MHQkWO0q15D7Y65hXIJmXS4fWP0c M7EIS2Z0Pgg4bSuTcqTfwp+UeXzxaU2h0N7pNpGezLjyFU81OYuNU+uXuhsOykVFXDjL uCTSMz6shqL61Tcv853gZ6iau5/woKKBYpheUmWEUKPZyW+mWdyBCbFmDZUJbQiyhPrA 1JhtcwHabO+tTrDUQ5OJLeMBv63CFIH7/wMk8//GffZWQLnNsSKGQV/sIR6fCXewUeaR lP4OeAanGQAee6+7DL26FtiI4F9mzl0ZH0c2eLOePi6BR+IIjqtU+uRgmF7s5n4X3UFv WWoA== X-Forwarded-Encrypted: i=1; AJvYcCXCGID1w80aaNeCNcc7jCOKdsApJH3oJNQsxDbcJw987t9JaN4/WAYPjPius1SLpXWeD5nu8TUMnsYZZbGx@vger.kernel.org X-Gm-Message-State: AOJu0YxgQiVMSvQ9yVRS7tzKYHT9rgqrZHHlvPlZBazjXClLsWzB6BJA 09VDvWfGSk9rAKt4uJdloUmyM+wYKm2a8uiJ3qvxZCtLHmf3ELb14aCL5LB+J99d1WMWsXkJqg= = X-Google-Smtp-Source: AGHT+IGsVvvboBRAyzbhN+VadIQHCOH8PGYVPxU5z0Lyav96vZClKVh4KR5eyXtOx0aT0txiHrEbB6kiHg== X-Received: from wmrn12.prod.google.com ([2002:a05:600c:500c:b0:43d:41a2:b768]) (user=tabba job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:6dca:b0:43c:f509:2bbf with SMTP id 5b1f17b1804b1-43d911902efmr30762195e9.15.1743175904477; Fri, 28 Mar 2025 08:31:44 -0700 (PDT) Date: Fri, 28 Mar 2025 15:31:31 +0000 In-Reply-To: <20250328153133.3504118-1-tabba@google.com> Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250328153133.3504118-1-tabba@google.com> X-Mailer: git-send-email 2.49.0.472.ge94155a9ec-goog Message-ID: <20250328153133.3504118-6-tabba@google.com> Subject: [PATCH v7 5/7] KVM: guest_memfd: Restore folio state after final folio_put() From: Fuad Tabba To: kvm@vger.kernel.org, linux-arm-msm@vger.kernel.org, linux-mm@kvack.org Cc: pbonzini@redhat.com, chenhuacai@kernel.org, mpe@ellerman.id.au, anup@brainfault.org, paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, seanjc@google.com, viro@zeniv.linux.org.uk, brauner@kernel.org, willy@infradead.org, akpm@linux-foundation.org, xiaoyao.li@intel.com, yilun.xu@intel.com, chao.p.peng@linux.intel.com, jarkko@kernel.org, amoorthy@google.com, dmatlack@google.com, isaku.yamahata@intel.com, mic@digikod.net, vbabka@suse.cz, vannapurve@google.com, ackerleytng@google.com, mail@maciej.szmigiero.name, david@redhat.com, michael.roth@amd.com, wei.w.wang@intel.com, liam.merwick@oracle.com, isaku.yamahata@gmail.com, kirill.shutemov@linux.intel.com, suzuki.poulose@arm.com, steven.price@arm.com, quic_eberman@quicinc.com, quic_mnalajal@quicinc.com, quic_tsoni@quicinc.com, quic_svaddagi@quicinc.com, quic_cvanscha@quicinc.com, quic_pderrin@quicinc.com, quic_pheragu@quicinc.com, catalin.marinas@arm.com, james.morse@arm.com, yuzenghui@huawei.com, oliver.upton@linux.dev, maz@kernel.org, will@kernel.org, qperret@google.com, keirf@google.com, roypat@amazon.co.uk, shuah@kernel.org, hch@infradead.org, jgg@nvidia.com, rientjes@google.com, jhubbard@nvidia.com, fvdl@google.com, hughd@google.com, jthoughton@google.com, peterx@redhat.com, pankaj.gupta@amd.com, tabba@google.com Before transitioning a guest_memfd folio to unshared, thereby disallowing access by the host and allowing the hypervisor to transition its view of the guest page as private, we need to ensure that the host doesn't have any references to the folio. This patch uses the guest_memfd folio type to register a callback that informs the guest_memfd subsystem when the last reference is dropped, therefore knowing that the host doesn't have any remaining references. Signed-off-by: Fuad Tabba --- The function kvm_slot_gmem_register_callback() isn't used in this series. It will be used later in code that performs unsharing of memory. I have tested it with pKVM, based on downstream code [*]. It's included in this RFC since it demonstrates the plan to handle unsharing of private folios. [*] https://android-kvm.googlesource.com/linux/+/refs/heads/tabba/guestmem-6.13-v7-pkvm --- include/linux/kvm_host.h | 6 ++ virt/kvm/guest_memfd.c | 143 ++++++++++++++++++++++++++++++++++++++- 2 files changed, 148 insertions(+), 1 deletion(-) diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index bf82faf16c53..d9d9d72d8beb 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -2607,6 +2607,7 @@ int kvm_gmem_slot_set_shared(struct kvm_memory_slot *slot, gfn_t start, int kvm_gmem_slot_clear_shared(struct kvm_memory_slot *slot, gfn_t start, gfn_t end); bool kvm_gmem_slot_is_guest_shared(struct kvm_memory_slot *slot, gfn_t gfn); +int kvm_gmem_slot_register_callback(struct kvm_memory_slot *slot, gfn_t gfn); void kvm_gmem_handle_folio_put(struct folio *folio); #else static inline int kvm_gmem_set_shared(struct kvm *kvm, gfn_t start, gfn_t end) @@ -2638,6 +2639,11 @@ static inline bool kvm_gmem_slot_is_guest_shared(struct kvm_memory_slot *slot, WARN_ON_ONCE(1); return false; } +static inline int kvm_gmem_slot_register_callback(struct kvm_memory_slot *slot, gfn_t gfn) +{ + WARN_ON_ONCE(1); + return -EINVAL; +} #endif /* CONFIG_KVM_GMEM_SHARED_MEM */ #endif diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c index 3b4d724084a8..ce19bd6c2031 100644 --- a/virt/kvm/guest_memfd.c +++ b/virt/kvm/guest_memfd.c @@ -392,6 +392,27 @@ enum folio_shareability { KVM_GMEM_NONE_SHARED = 0b11, /* Not shared, transient state. */ }; +/* + * Unregisters the __folio_put() callback from the folio. + * + * Restores a folio's refcount after all pending references have been released, + * and removes the folio type, thereby removing the callback. Now the folio can + * be freed normaly once all actual references have been dropped. + * + * Must be called with the folio locked and the offsets_lock write lock held. + */ +static void kvm_gmem_restore_pending_folio(struct folio *folio, struct inode *inode) +{ + rwlock_t *offsets_lock = &kvm_gmem_private(inode)->offsets_lock; + + lockdep_assert_held_write(offsets_lock); + WARN_ON_ONCE(!folio_test_locked(folio)); + WARN_ON_ONCE(!folio_test_guestmem(folio)); + + __folio_clear_guestmem(folio); + folio_ref_add(folio, folio_nr_pages(folio)); +} + static int kvm_gmem_offset_set_shared(struct inode *inode, pgoff_t index) { struct xarray *shared_offsets = &kvm_gmem_private(inode)->shared_offsets; @@ -400,6 +421,24 @@ static int kvm_gmem_offset_set_shared(struct inode *inode, pgoff_t index) lockdep_assert_held_write(offsets_lock); + /* + * If the folio is NONE_SHARED, it indicates that it is transitioning to + * private (GUEST_SHARED). Transition it to shared (ALL_SHARED) + * immediately, and remove the callback. + */ + if (xa_to_value(xa_load(shared_offsets, index)) == KVM_GMEM_NONE_SHARED) { + struct folio *folio = filemap_lock_folio(inode->i_mapping, index); + + if (WARN_ON_ONCE(IS_ERR(folio))) + return PTR_ERR(folio); + + if (folio_test_guestmem(folio)) + kvm_gmem_restore_pending_folio(folio, inode); + + folio_unlock(folio); + folio_put(folio); + } + return xa_err(xa_store(shared_offsets, index, xval, GFP_KERNEL)); } @@ -503,9 +542,111 @@ static int kvm_gmem_offset_range_clear_shared(struct inode *inode, return r; } +/* + * Registers a callback to __folio_put(), so that gmem knows that the host does + * not have any references to the folio. The callback itself is registered by + * setting the folio type to guestmem. + * + * Returns 0 if a callback was registered or already has been registered, or + * -EAGAIN if the host has references, indicating a callback wasn't registered. + * + * Must be called with the folio locked and the offsets_lock write lock held. + */ +static int kvm_gmem_register_callback(struct folio *folio, struct inode *inode, pgoff_t index) +{ + struct xarray *shared_offsets = &kvm_gmem_private(inode)->shared_offsets; + rwlock_t *offsets_lock = &kvm_gmem_private(inode)->offsets_lock; + void *xval_guest = xa_mk_value(KVM_GMEM_GUEST_SHARED); + int refcount; + int r = 0; + + lockdep_assert_held_write(offsets_lock); + WARN_ON_ONCE(!folio_test_locked(folio)); + + if (folio_test_guestmem(folio)) + return 0; + + if (folio_mapped(folio)) + return -EAGAIN; + + refcount = folio_ref_count(folio); + if (!folio_ref_freeze(folio, refcount)) + return -EAGAIN; + + /* + * Register callback by setting the folio type and subtracting gmem's + * references for it to trigger once outstanding references are dropped. + */ + if (refcount > 1) { + __folio_set_guestmem(folio); + refcount -= folio_nr_pages(folio); + } else { + /* No outstanding references, transition it to guest shared. */ + r = WARN_ON_ONCE(xa_err(xa_store(shared_offsets, index, xval_guest, GFP_KERNEL))); + } + + folio_ref_unfreeze(folio, refcount); + return r; +} + +int kvm_gmem_slot_register_callback(struct kvm_memory_slot *slot, gfn_t gfn) +{ + unsigned long pgoff = slot->gmem.pgoff + gfn - slot->base_gfn; + struct inode *inode = file_inode(READ_ONCE(slot->gmem.file)); + rwlock_t *offsets_lock = &kvm_gmem_private(inode)->offsets_lock; + struct folio *folio; + int r; + + write_lock(offsets_lock); + + folio = filemap_lock_folio(inode->i_mapping, pgoff); + if (WARN_ON_ONCE(IS_ERR(folio))) { + write_unlock(offsets_lock); + return PTR_ERR(folio); + } + + r = kvm_gmem_register_callback(folio, inode, pgoff); + + folio_unlock(folio); + folio_put(folio); + write_unlock(offsets_lock); + + return r; +} +EXPORT_SYMBOL_GPL(kvm_gmem_slot_register_callback); + +/* + * Callback function for __folio_put(), i.e., called once all references by the + * host to the folio have been dropped. This allows gmem to transition the state + * of the folio to shared with the guest, and allows the hypervisor to continue + * transitioning its state to private, since the host cannot attempt to access + * it anymore. + */ void kvm_gmem_handle_folio_put(struct folio *folio) { - WARN_ONCE(1, "A placeholder that shouldn't trigger. Work in progress."); + struct address_space *mapping; + struct xarray *shared_offsets; + rwlock_t *offsets_lock; + struct inode *inode; + pgoff_t index; + void *xval; + + mapping = folio->mapping; + if (WARN_ON_ONCE(!mapping)) + return; + + inode = mapping->host; + index = folio->index; + shared_offsets = &kvm_gmem_private(inode)->shared_offsets; + offsets_lock = &kvm_gmem_private(inode)->offsets_lock; + xval = xa_mk_value(KVM_GMEM_GUEST_SHARED); + + write_lock(offsets_lock); + folio_lock(folio); + kvm_gmem_restore_pending_folio(folio, inode); + folio_unlock(folio); + WARN_ON_ONCE(xa_err(xa_store(shared_offsets, index, xval, GFP_KERNEL))); + write_unlock(offsets_lock); } EXPORT_SYMBOL_GPL(kvm_gmem_handle_folio_put);