From patchwork Wed Jul 24 01:10:33 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Houghton X-Patchwork-Id: 13740504 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1D6B1C3DA49 for ; Wed, 24 Jul 2024 01:11:38 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9C3FE6B0095; Tue, 23 Jul 2024 21:11:23 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 925F86B0096; Tue, 23 Jul 2024 21:11:23 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 79F116B0098; Tue, 23 Jul 2024 21:11:23 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 57D046B0095 for ; Tue, 23 Jul 2024 21:11:23 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 0D870160520 for ; Wed, 24 Jul 2024 01:11:23 +0000 (UTC) X-FDA: 82372868046.12.882D300 Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) by imf01.hostedemail.com (Postfix) with ESMTP id 48BE540004 for ; Wed, 24 Jul 2024 01:11:21 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b="dtEZ+oq/"; spf=pass (imf01.hostedemail.com: domain of 3uFSgZgoKCB8EOCJPBCOJIBJJBG9.7JHGDIPS-HHFQ57F.JMB@flex--jthoughton.bounces.google.com designates 209.85.128.201 as permitted sender) smtp.mailfrom=3uFSgZgoKCB8EOCJPBCOJIBJJBG9.7JHGDIPS-HHFQ57F.JMB@flex--jthoughton.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1721783458; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=JW+jUUoqcXqadXhonZGbNsXZ7+Zma3rlwNzyePdiaMw=; b=oS7lLSfEIp7thwBSAVW4R2spAxh/O/22o4kGjZsehS2CV3Zpwg6JbxUveD9Rk0O97fODiI nyN7lW2HdimEFvnRXfawMIiVHyvxGOaklNb5N6xXACtUZ+dANnO5ql5qRzMN0zdJ2MvO2v t+upalSk4kyKV9jOWg8EEjR4wUY6R7A= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b="dtEZ+oq/"; spf=pass (imf01.hostedemail.com: domain of 3uFSgZgoKCB8EOCJPBCOJIBJJBG9.7JHGDIPS-HHFQ57F.JMB@flex--jthoughton.bounces.google.com designates 209.85.128.201 as permitted sender) smtp.mailfrom=3uFSgZgoKCB8EOCJPBCOJIBJJBG9.7JHGDIPS-HHFQ57F.JMB@flex--jthoughton.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1721783458; a=rsa-sha256; cv=none; b=tEtFLft8ZtdMX23QZa7KR+sjYoU2OZLZH33yyV5BdDHqXWJ/T7zGWvjXdHiIMwzCvO4T/V 8qtnCS1GyApXmZbjv3GNNRKafkNcd1TthcG7lxkeNtwBPkhwOvV/4IghL8xWfmlN8dbmLl TZFeVXs2vBLSBfKpSPtWsukcXN1x72k= Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-672bea19dd3so3556577b3.1 for ; Tue, 23 Jul 2024 18:11:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1721783480; x=1722388280; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=JW+jUUoqcXqadXhonZGbNsXZ7+Zma3rlwNzyePdiaMw=; b=dtEZ+oq/BJWU7h68CoxJ36a3ilCU2JNr6si+aIcVhHSx14AIeCQ8vIB0YQYilv8bFN UdF5C8oIpkUaUBSS6KPYA5IiDMYkKzuqei9yMftbkEQLje3un7EuCIJK/6SY7UrA3fwX Ucqnv5E2pX8uHKpmUD1rEni9iEIW0L19KsYuurWsJaCmc7GgGm8ALzIfCope1MGHZkFq lCzpWVF4qE7f9nhvTcdZs753qLBLNpLpLWO+gEqYGYJ7UipB8ESbwqF8sB7v+jqCnXL+ M8FWp63mYtETXVCKfHv0kZDqBl21/uIBfVbWU9J/cOb9mDd/f32LjbMopvVyIzbAERGl fZ4g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1721783480; x=1722388280; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=JW+jUUoqcXqadXhonZGbNsXZ7+Zma3rlwNzyePdiaMw=; b=KQRVj+SdY8odMpkjc+gsJEQ9kfFUqeuRf4bxA5OlVVQa5FRRHoxTQWAllEyNrmtYsZ SZ+cPZeho1f8gkUNmNoYOztLADQpdQYd7XYgw3Hu0KJefapJp6Gv/MoctkYDcrhXLvyf B5wpNFUgib1Nlcqxw9oEnBhJ0EnQJVKRancwGE5tUk8Y5v33pMOpndhTi8uzi1IEKZYb tQSzSGXifKwJLbvhl9ZeYS4cG42LgMAXMbIjhXrnkPyvi4gw1NrLbuRC8XixvNCktCqo iYkBeJXF1BNwTfDGtC0gz7nFuQPCRvvSlcduIXD2gsO/Rr/JGnE8EOQIeZdVGTLVHams ugyg== X-Forwarded-Encrypted: i=1; AJvYcCXiUjLaYK7fidIz3d54fb0m2t6KWeNRN1vPMFHcOqID0ZC0s9JUsXmqhoER98mobqX0Q5+Q68HyKVSsmwjMSLue+r0= X-Gm-Message-State: AOJu0YxpPzbI25hfS5wnNVmBCxjaJEi0KaCkXkPAZ081DqlO6MEFkJMO GHhaz96vOivaY6Ujtk5Ge9awO2PlDBpGSRDZf1bClMQfV4wXmcJiYvUeJsp3PNRF5nqm1ZSCch7 nGKZZwH99JXhULUd7oQ== X-Google-Smtp-Source: AGHT+IHToUjK35hNR0etsZGQeAjUCPxr50Jqta5Si1cnS8NseXl88g/0Ew0GUmmAUp0HML6hFdfWagbQP5rsIFm/ X-Received: from jthoughton.c.googlers.com ([fda3:e722:ac3:cc00:14:4d90:c0a8:2a4f]) (user=jthoughton job=sendgmr) by 2002:a05:690c:102:b0:66b:fb2f:1c2 with SMTP id 00721157ae682-671f4e0a6damr821967b3.7.1721783480393; Tue, 23 Jul 2024 18:11:20 -0700 (PDT) Date: Wed, 24 Jul 2024 01:10:33 +0000 In-Reply-To: <20240724011037.3671523-1-jthoughton@google.com> Mime-Version: 1.0 References: <20240724011037.3671523-1-jthoughton@google.com> X-Mailer: git-send-email 2.46.0.rc1.232.g9752f9e123-goog Message-ID: <20240724011037.3671523-9-jthoughton@google.com> Subject: [PATCH v6 08/11] KVM: x86: Optimize kvm_{test_,}age_gfn a little bit From: James Houghton To: Andrew Morton , Paolo Bonzini Cc: Ankit Agrawal , Axel Rasmussen , Catalin Marinas , David Matlack , David Rientjes , James Houghton , James Morse , Jason Gunthorpe , Jonathan Corbet , Marc Zyngier , Oliver Upton , Raghavendra Rao Ananta , Ryan Roberts , Sean Christopherson , Shaoqin Huang , Suzuki K Poulose , Wei Xu , Will Deacon , Yu Zhao , Zenghui Yu , kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org X-Rspamd-Server: rspam03 X-Rspam-User: X-Rspamd-Queue-Id: 48BE540004 X-Stat-Signature: xbwn6m8yfugcz5qzuysoa5cos5iwn714 X-HE-Tag: 1721783481-705413 X-HE-Meta: U2FsdGVkX19oAortkafQmY8irdU79EgVf2Mx0mm1QTFEFXMunRobkogekwk3QMN6c1nS8qNu71stJKfPzKqeYjtiryZxlX1U3DZuDGY76k/Dtugum2WKGN0knCbwft8Ys0NlDQA969dj5wUEYDtVvsq7y9H5jn2XquRC1saS01m8ZVnXFRbuu0ijVXZ9DrVwpfvUepSYPRlJvEpKItECEbGupWwkrtV8kqmJFvGj+xpi+wwOVvZx2abLW+/bVtD3DoLiWt8qpfq6tQctN9YchWzhTwZ0GESYLKc8DtnLdHaA7MxvQRyRWFrm4gDFWxEGQpOdtQ86yyYZSlIcDxS90sf+R/nF1CC8PSFG4l3jZR23uwV/CZlv7uhaM/fwU1aDPY3gLI22Fwk3UHAO2B6Dr/A6jBvWx8msKUemOOiKkC2KUP3aPCAn7+e4rBMrAbDsvtsmSigVHV0pYKBsqrpf3F7pwKKX4341eplPT1faICtFV3DTmdE0a6+p4oDR3KYgFvSRwefnWFd53B/HYuDMPCOzxL6cybth32/xr9CS6w1gFdfwem2vSHtAsoUzt8pxSPJ3oB8jojdc+TmvmU00ZtzbRXSMiQmyCpIvAIPLxvhY/JPY1HBN6g6AqCh81u7lCu6gGBdFWdGlpDrV9Wn+N6gO+NG2X8aDzOXuHI+lyJ8n9thv7gnEUv3efKDnnZqq/SYoz2Ua+Bfw+JHKJIGPed6Au63hfpMeYCX50V0AceAuvooEQ6z1i/tcMauQ+peVA8Om3YRnJxJOMSNF/uAtVf5cAv0IterfdBOcRd5oPbrdIiKXQTxfeuNbrjXIM6sY9nIQjAnDef9Iw9shysuoRAk48bchovrCRPlsNhDLsdmtVGkux+KnpBqFpx7C3Zuwl+n8AB6PZrxLkd5qKJgKqkuXpz6PoB8+k20vgUfODph/07sncCOEzxhn/g5JwXPhzz6sZkO0K7AfXpjj3kN tEpxz/rt LRaxqvbpaSAIsYxgkb136nIf/lwTBI7jUStK7HSk4kQkNsb5m4tQuGekzzQkDieAZ8Vz3H69U+pABpO+hxBZHLcxl+057lkYNy/PCmwIO0GZs/NE5VSmUFzVbcZzhcO4eq84ntTuF1syvakZjRND3Lp2cvEAhj8eZBMFWxnhfkiToh5uAWpnG/86KB17KGHS9j+pmcFRznfUQvj/eRPXzEfyDWe44KbMbRIEccPir+cyn660lyTcYJKcHMen4MSuLJZ3Egjda2WvLIdR4dQQ+qK4GwNsJm8fCQGGumby2uUicmWxhVDfyeru0uQ9RA8w7l42XVShR08dTT7QxPanUAQnC5suYs+2jez3QAfmOkRRmAmXb+QaU/rExdZ1luED3PkN0D+sG+Ps1zm5rWV1VY7UpB/N6U59JSNYmEB22iRZBvAOjwYUiw5lNNj8uU802uIqm2kdQxFy1pnGCrABzYxhyHJJTgHh7mx0JGTRMCvSckNMb4o+C5blr3N+xxL7Aru9UY3nTmu8PVuXHqx0RfL2F8AP47Ce2J/A6SFhtID7jEavRffpy/L9kSrEZK9IrpTowkkcam454pgoVavbsiaI/7A== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Optimize both kvm_age_gfn and kvm_test_age_gfn's interaction with the shadow MMU by, rather than checking if our memslot has rmaps, check if there are any indirect_shadow_pages at all. Also, for kvm_test_age_gfn, reorder the TDP MMU check to be first. If we find that the range is young, we do not need to check the shadow MMU. Signed-off-by: James Houghton --- arch/x86/kvm/mmu/mmu.c | 21 +++++++++++++-------- 1 file changed, 13 insertions(+), 8 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 7b93ce8f0680..919d59385f89 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -1629,19 +1629,24 @@ static void rmap_add(struct kvm_vcpu *vcpu, const struct kvm_memory_slot *slot, __rmap_add(vcpu->kvm, cache, slot, spte, gfn, access); } +static bool kvm_has_shadow_mmu_sptes(struct kvm *kvm) +{ + return !tdp_mmu_enabled || READ_ONCE(kvm->arch.indirect_shadow_pages); +} + bool kvm_age_gfn(struct kvm *kvm, struct kvm_gfn_range *range) { bool young = false; - if (kvm_memslots_have_rmaps(kvm)) { + if (tdp_mmu_enabled) + young |= kvm_tdp_mmu_age_gfn_range(kvm, range); + + if (kvm_has_shadow_mmu_sptes(kvm)) { write_lock(&kvm->mmu_lock); young = kvm_handle_gfn_range(kvm, range, kvm_age_rmap); write_unlock(&kvm->mmu_lock); } - if (tdp_mmu_enabled) - young |= kvm_tdp_mmu_age_gfn_range(kvm, range); - return young; } @@ -1649,15 +1654,15 @@ bool kvm_test_age_gfn(struct kvm *kvm, struct kvm_gfn_range *range) { bool young = false; - if (kvm_memslots_have_rmaps(kvm)) { + if (tdp_mmu_enabled) + young |= kvm_tdp_mmu_test_age_gfn(kvm, range); + + if (!young && kvm_has_shadow_mmu_sptes(kvm)) { write_lock(&kvm->mmu_lock); young = kvm_handle_gfn_range(kvm, range, kvm_test_age_rmap); write_unlock(&kvm->mmu_lock); } - if (tdp_mmu_enabled) - young |= kvm_tdp_mmu_test_age_gfn(kvm, range); - return young; }