From patchwork Wed Jul 24 01:10:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Houghton X-Patchwork-Id: 13740503 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1F11AC3DA49 for ; Wed, 24 Jul 2024 01:11:35 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D21EA6B0092; Tue, 23 Jul 2024 21:11:22 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CAAE26B0095; Tue, 23 Jul 2024 21:11:22 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AFB7A6B0096; Tue, 23 Jul 2024 21:11:22 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 8B1E56B0092 for ; Tue, 23 Jul 2024 21:11:22 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 50A7F1A057E for ; Wed, 24 Jul 2024 01:11:22 +0000 (UTC) X-FDA: 82372868004.30.831FDFA Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) by imf24.hostedemail.com (Postfix) with ESMTP id 849CF180003 for ; Wed, 24 Jul 2024 01:11:20 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=jCGW+We7; spf=pass (imf24.hostedemail.com: domain of 3t1SgZgoKCB4DNBIOABNIHAIIAF8.6IGFCHOR-GGEP46E.ILA@flex--jthoughton.bounces.google.com designates 209.85.128.202 as permitted sender) smtp.mailfrom=3t1SgZgoKCB4DNBIOABNIHAIIAF8.6IGFCHOR-GGEP46E.ILA@flex--jthoughton.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1721783456; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=xYFUlEH9+669o/RlL44au3jQWbYqLeFR1nQhpVmQt2k=; b=gRllc61YO/wA1/91PWYRUfuhW2QFxGyKtE37cqOvhmOq6tIy9eggxe3Huue+RUNEETuAL1 TO66Ro0p7lXsaIY09+fqnVRa7hjo5hI6XMGlqfkM3IjsJjPGIkbZtwJjbx+b84d7IGCY5I dBKt1+ylK2+CXvCYznRctbUdm3ow8bg= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=jCGW+We7; spf=pass (imf24.hostedemail.com: domain of 3t1SgZgoKCB4DNBIOABNIHAIIAF8.6IGFCHOR-GGEP46E.ILA@flex--jthoughton.bounces.google.com designates 209.85.128.202 as permitted sender) smtp.mailfrom=3t1SgZgoKCB4DNBIOABNIHAIIAF8.6IGFCHOR-GGEP46E.ILA@flex--jthoughton.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1721783456; a=rsa-sha256; cv=none; b=6sLsbrC9exksTvRe+/h/NKNLG6yKnd7CZ7ye+NHQhOyKA6JKe6vYx/fXBtEbpatmxUcWhN rtlL1Xyq5oJmI9pAD8XJx0mFxLD2p0mOKR1q3E7e34kIJzSnC5Eq+EAFgDlxAQkxmlgd6E NOBVNArSAJuGWzSHD4/LtUZjW6pJueY= Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-664b7a67ad4so176397207b3.2 for ; Tue, 23 Jul 2024 18:11:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1721783479; x=1722388279; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=xYFUlEH9+669o/RlL44au3jQWbYqLeFR1nQhpVmQt2k=; b=jCGW+We71dbOf0PyjAdvG6vdyiJfT444ffBuGsZcduPKCA1+yokaXitVuGiKrWRcfX 8g9rm0A4RXtk9iX3fVYeDxtstCHt4/bWBYQBpQE1l9vCDtPHMu5ENHR6fxXEdVBvsFb5 SiWTxCc9lCw6K35wFkykA7M5FzRYA2/IUXpJlUnMlTJPSIBScXMnvrvX+gx658BNqwOl 4aXIilCkdN3CADkPqH8TyWHNwZuGGDnwdTjGsqT2FQnupNL6y5lx4qExbtBG2TsaeGWc DWE0LEwHT4+oNDqDVrkrz9sloo8BeRNRi1r/h/Sg4i6MZPyR8dj9OM4Iwgr847UeHrJB SIvg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1721783479; x=1722388279; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=xYFUlEH9+669o/RlL44au3jQWbYqLeFR1nQhpVmQt2k=; b=sGgYzVUH93OpOJSr9IkDyXTxR0mZ8rKC6hR+tXpuZWRq25SE7b34p83pTS/m8b2YOR UC1rZ/fY2p0QYQjpCFW/S0VMAGX82YRyrUzaLaU44qVX037yKwQKYckbOI8yOKiP3nRd wm80Q9PnbXlyQ5LiGGIVWEKZFupzn8PvnQSpe2u/9iVPW7ZpPK08imWU8KTqWnWphsNf NisS1xQn5ELxAEmKcibEndjgeYw3aY1QQqMYaheUd4PY8L/rJ06lIRvD8YaqfER2l+p3 F8lZpwiXTwmChw2uLzvSjo4WBoTlSmneMMan/aOZpdjG8+tj2iA6V1Vt8vPPiI80vHzb bd9w== X-Forwarded-Encrypted: i=1; AJvYcCVR9wtJM2F3unI2S1cXDgsWjBtnoxqHg420zBfkYbV9lVrDN9n7u8b/tnu7meIUYLBmeOinARC7JHTKXrP6lN9NUik= X-Gm-Message-State: AOJu0YxWr8KWPHHrwLNQmSRkrE5ecFaWvC7MyApsgo6RsC8a+erwo+K6 ZQN0M1dWxIi60w06yH7A02tDEbcjZPTK9Ce18yhWONC7iuSdW5qCizPVpIRSdNIL/Wr+dT3JkUH c/N7IdhP4Mtk8AFv8Bw== X-Google-Smtp-Source: AGHT+IEN/Y2KcEHN/92i673drFtDbgqlQb3JUSQlrIWx5MUp4ovVZHssNawTpvQt6yIfvMN/stHEpv668SrbKdDV X-Received: from jthoughton.c.googlers.com ([fda3:e722:ac3:cc00:14:4d90:c0a8:2a4f]) (user=jthoughton job=sendgmr) by 2002:a05:690c:289:b0:630:28e3:2568 with SMTP id 00721157ae682-671f1095e3fmr479257b3.3.1721783479490; Tue, 23 Jul 2024 18:11:19 -0700 (PDT) Date: Wed, 24 Jul 2024 01:10:32 +0000 In-Reply-To: <20240724011037.3671523-1-jthoughton@google.com> Mime-Version: 1.0 References: <20240724011037.3671523-1-jthoughton@google.com> X-Mailer: git-send-email 2.46.0.rc1.232.g9752f9e123-goog Message-ID: <20240724011037.3671523-8-jthoughton@google.com> Subject: [PATCH v6 07/11] KVM: Pass fast_only to kvm_{test_,}age_gfn From: James Houghton To: Andrew Morton , Paolo Bonzini Cc: Ankit Agrawal , Axel Rasmussen , Catalin Marinas , David Matlack , David Rientjes , James Houghton , James Morse , Jason Gunthorpe , Jonathan Corbet , Marc Zyngier , Oliver Upton , Raghavendra Rao Ananta , Ryan Roberts , Sean Christopherson , Shaoqin Huang , Suzuki K Poulose , Wei Xu , Will Deacon , Yu Zhao , Zenghui Yu , kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: 849CF180003 X-Stat-Signature: 186ii6ab8yreiecf6yddowroo4bz8w4x X-Rspam-User: X-HE-Tag: 1721783480-582757 X-HE-Meta: U2FsdGVkX199KAku0TdirhEp3BhKJp9N2Vgx5YyBN9JwS9fuetuIXY9/zmRcyYU6Knbdro8cfr2yTNAt7KevQs0TJPHGHyXvD4bhIOzZ/NDFUcvv5Lkzr3SX9OXosmUTE6l7ZxOKHFxAcX079+eq0eyfakW/CudTWgBz2re0LU5ktaftdkqYv+C15gYnqcJ9np3F+NGFdOqRbHNLAZAxzHzKPq4s3uiLPpicChXLqp3w9eX0fwvHt2c3HEsQKUhPi7KEz+SKQM0+JZtWa6ms/B9+/EI5IW12L5NkxRjot3Ca+iEUcJPv0FdvdHI5lSTsoh3L1hDXGO9lsXfxe6InJ0piTg/LkZqOIpdQOOUsho710OnOMgOrxvwqWQUl5j2WgN0PtJZ6O1fOnX8a0Iv17Z8qMC8pbzpecaDYz5htsVMwKqHVE97z2MS0C7OlWq5471omelixUoEZH0YW1VesVz1q21ymJjW9JQFINvGWdvef/fVWWOntYKEa0pGs3P8xa5CPvpwIZe1pZynnnnJDJLcMRlZZWChxDR51DBWKLXxbGVTxDQK2+vug3elCPhACg41DqpGXM+3yl9SGvFjTwHClWWSgMoO0QJgAtuA3/mf5Laus4lbZ5IQY9HxSI8NQs2PsjILyJMTH1PzlvE7twjH+FfYaiqDnmQFEjhZg6Vle5+SVC5NSE8zC3STgj3ar72WY7CtEbOinL7MUOTd0E41hE3BLXPQJZFoUxiDdR+ri5zcnEP7mYqNtbT9xz6ywtUO9f1VHniIDMdL3befvDoeo9zInGpE1THXn+kfoWV0elJeVi0j9OUQcLraFXlMvUx/uAwIJ1taqcqnADMxiUKMUZU9+Bbi0asdBxd4MINL1dB6Amrfdhy9nneSeXvSliAI9Ul7X5nG+TrDmhQgwGMoBCViGV77U4yqAWjMh1gQhZTahpzXGf1/NW7dGFT+KpCO7NGPonsbYDZNchjk Z/WWbFVt mtHcGnNP5Q8oc2u6ZgmbytaPt/Vi4N5UWW5Ac8B3/y+Yu1b9mXhqge5xOPK0GYE/hDd+uxwFKKpSRqCgq6vv6nPawJKt7b3sjHjuO0gqc6X3O2c/xAlcG13rKWdJ8Bno5wpYkxGXcLH5K8xA7cF/T43EYiqL+dXBK/Q2cJnGvmB7fwbJFOXF8XMO4YhTR5pE9VLDhL5Jom/ZDxgAPNyygBLIR47Fn2XrI108pSs4L9wjidxE9VVZ3B0xdzZJVYdCAYy7reS1hFkenki+/91Uo7LQeQQkfGUaA14b1rKZ/GBNrAwCGapZgnxszgkSPSanRIu0nAwPUmHrIxRKIVpPJ1G5ejITJI9UXeTy4GKdKkZJrA7HpcN9y2+UtMVVtfN7/UWiuso8k73hMB0VH4slbjcOmfONfrMsnS+kE3knQTH/Z+afL1jUKSo7tXarSieRmz0aKYChNojsU4KkTuDdSwROiWwRoQy+0ezLCWjhAYKC8ZSYYkQYmsvRhTXz7SbydBchPKlwUDZqa3y3ggoWNRFokXY6//LbhFUPOo0QstOrGAsYHxrvENJq4ggoA27Bg4ZVhQU3NY7REG7D3jXP9aj3y+w== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Provide the basics for architectures to implement a fast-only version of kvm_{test_,}age_gfn. Add CONFIG_HAVE_KVM_MMU_NOTIFIER_YOUNG_FAST_ONLY that architectures will set if they non-trivially implement test_young() and clear_young() when called with fast_only. Signed-off-by: James Houghton --- include/linux/kvm_host.h | 1 + virt/kvm/Kconfig | 4 ++++ virt/kvm/kvm_main.c | 37 +++++++++++++++++++++---------------- 3 files changed, 26 insertions(+), 16 deletions(-) diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 8cd80f969cff..944c5fba2344 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -258,6 +258,7 @@ int kvm_async_pf_wakeup_all(struct kvm_vcpu *vcpu); #ifdef CONFIG_KVM_GENERIC_MMU_NOTIFIER union kvm_mmu_notifier_arg { unsigned long attributes; + bool fast_only; }; struct kvm_gfn_range { diff --git a/virt/kvm/Kconfig b/virt/kvm/Kconfig index 632334861001..cb4d5384c2f2 100644 --- a/virt/kvm/Kconfig +++ b/virt/kvm/Kconfig @@ -103,6 +103,10 @@ config KVM_GENERIC_MMU_NOTIFIER config KVM_MMU_NOTIFIER_YOUNG_LOCKLESS bool +config HAVE_KVM_MMU_NOTIFIER_YOUNG_FAST_ONLY + select KVM_GENERIC_MMU_NOTIFIER + bool + config KVM_GENERIC_MEMORY_ATTRIBUTES depends on KVM_GENERIC_MMU_NOTIFIER bool diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 959b6d5d8ce4..86fb2b560d98 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -697,18 +697,20 @@ static __always_inline int kvm_handle_hva_range(struct mmu_notifier *mn, static __always_inline int kvm_handle_hva_range_no_flush(struct mmu_notifier *mn, unsigned long start, unsigned long end, - gfn_handler_t handler) + gfn_handler_t handler, + bool fast_only) { struct kvm *kvm = mmu_notifier_to_kvm(mn); const struct kvm_mmu_notifier_range range = { - .start = start, - .end = end, - .handler = handler, - .on_lock = (void *)kvm_null_fn, - .flush_on_ret = false, - .may_block = false, - .lockless = + .start = start, + .end = end, + .handler = handler, + .on_lock = (void *)kvm_null_fn, + .flush_on_ret = false, + .may_block = false, + .lockless = IS_ENABLED(CONFIG_KVM_MMU_NOTIFIER_YOUNG_LOCKLESS), + .arg.fast_only = fast_only, }; return __kvm_handle_hva_range(kvm, &range).ret; @@ -900,7 +902,8 @@ static int kvm_mmu_notifier_clear_young(struct mmu_notifier *mn, * cadence. If we find this inaccurate, we might come up with a * more sophisticated heuristic later. */ - return kvm_handle_hva_range_no_flush(mn, start, end, kvm_age_gfn); + return kvm_handle_hva_range_no_flush(mn, start, end, kvm_age_gfn, + fast_only); } static int kvm_mmu_notifier_test_young(struct mmu_notifier *mn, @@ -911,7 +914,7 @@ static int kvm_mmu_notifier_test_young(struct mmu_notifier *mn, trace_kvm_test_age_hva(address, fast_only); return kvm_handle_hva_range_no_flush(mn, address, address + 1, - kvm_test_age_gfn); + kvm_test_age_gfn, fast_only); } static void kvm_mmu_notifier_release(struct mmu_notifier *mn, @@ -926,17 +929,19 @@ static void kvm_mmu_notifier_release(struct mmu_notifier *mn, } static const struct mmu_notifier_ops kvm_mmu_notifier_ops = { - .invalidate_range_start = kvm_mmu_notifier_invalidate_range_start, - .invalidate_range_end = kvm_mmu_notifier_invalidate_range_end, - .clear_flush_young = kvm_mmu_notifier_clear_flush_young, - .clear_young = kvm_mmu_notifier_clear_young, - .test_young = kvm_mmu_notifier_test_young, - .release = kvm_mmu_notifier_release, + .invalidate_range_start = kvm_mmu_notifier_invalidate_range_start, + .invalidate_range_end = kvm_mmu_notifier_invalidate_range_end, + .clear_flush_young = kvm_mmu_notifier_clear_flush_young, + .clear_young = kvm_mmu_notifier_clear_young, + .test_young = kvm_mmu_notifier_test_young, + .release = kvm_mmu_notifier_release, }; static int kvm_init_mmu_notifier(struct kvm *kvm) { kvm->mmu_notifier.ops = &kvm_mmu_notifier_ops; + kvm->mmu_notifier.has_fast_aging = + IS_ENABLED(CONFIG_HAVE_KVM_MMU_NOTIFIER_YOUNG_FAST_ONLY); return mmu_notifier_register(&kvm->mmu_notifier, current->mm); }