From patchwork Mon Apr 4 23:41:51 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yosry Ahmed X-Patchwork-Id: 12800942 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id AA22DC433F5 for ; Mon, 4 Apr 2022 23:43:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B3A006B0074; Mon, 4 Apr 2022 19:42:16 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id AE9126B0075; Mon, 4 Apr 2022 19:42:16 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 962666B0078; Mon, 4 Apr 2022 19:42:16 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (relay.a.hostedemail.com [64.99.140.24]) by kanga.kvack.org (Postfix) with ESMTP id 899B06B0074 for ; Mon, 4 Apr 2022 19:42:16 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 6300F20571 for ; Mon, 4 Apr 2022 23:42:06 +0000 (UTC) X-FDA: 79320822252.03.8A79AD2 Received: from mail-pf1-f201.google.com (mail-pf1-f201.google.com [209.85.210.201]) by imf09.hostedemail.com (Postfix) with ESMTP id F2BA8140033 for ; Mon, 4 Apr 2022 23:42:05 +0000 (UTC) Received: by mail-pf1-f201.google.com with SMTP id f84-20020a623857000000b004fe3d6455ccso395555pfa.0 for ; Mon, 04 Apr 2022 16:42:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=zGPia6Y75PegYdtijlGH8Rirg8fMk3EtyUzE3+aq8gA=; b=oxrndx4rUGCxdYTcZSFyDdbsgSWZ+IkShvkWvLxweXOH/Ob2dyRylsH1lB4NocUiFn Ga3YaLjAzq/XYcDDRp/LAbFX0Bq3Py2YIszyVtaU65PLg8YY3y1iRhqzHzA7pSa1L8wN zzkWhWnl4shoUlPtJ0P5N04UDJZ9SJMrJFzvzbOXNaFuGQGh0UooJMEpodaPKh3POKNu cSktEFN3eQRN1BMcgpKy+u74uXUsXA+ApOG6dvll62pFH/J+Tnswch0wxoTS0vs+pPwX gUsV2HmYBM0X0ssLZZA+729NPsfL4HVgO2kiynY66hWh/lV/B7V2jGjnJzj9HIz1py08 MrUw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=zGPia6Y75PegYdtijlGH8Rirg8fMk3EtyUzE3+aq8gA=; b=BtAN6ON5YZA7hZx6rs2ck+PKKiWFallUIEDSIZU6+qW0XF1gleVjICQf6MMEF//9zq R+MV0xkfDX9uhCOsB1NX619naJ/oFzRLosyMIi2zVz/2rH+0AtJSEBJu00Dg9nOkpMVN eMWIvk4u0c2oQjReQLZo7fOatqbypiqKOpdIZxABGc+JKOP7hZH2FCyVXMHup+6GzusZ KJy3TmErrPXb0FbFRWk6e/4vLVJWsWGn7pvipKCLbX6VCHsIrdCTwPHDqYTwTd5GOtDG oDoQqOKZ9ZZh4OMPA5/6uBb6VN2wrxqWPmzyMS+ywuBW+GWh8JhzhPJlrYDwlRusBDux J6/Q== X-Gm-Message-State: AOAM5339gyzydm6k7Mw0DFK7qSk01m0DlSyZ4EzRVLqDVZ+u78jhocHF AqO3Bq8fpuUoNTyUKSbqNpfiQhMSenuqLxd7 X-Google-Smtp-Source: ABdhPJzD1f0Jx7sGhlZtl856UUxIwCksK4t+nrYVjP8gP5u+t88ub1/JzUx63f9lr8FQjeB9VSzhzIkW2iQctlIT X-Received: from yosry.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:2327]) (user=yosryahmed job=sendgmr) by 2002:a17:903:2488:b0:156:1e8d:a82 with SMTP id p8-20020a170903248800b001561e8d0a82mr506452plw.51.1649115724990; Mon, 04 Apr 2022 16:42:04 -0700 (PDT) Date: Mon, 4 Apr 2022 23:41:51 +0000 In-Reply-To: <20220404234154.1251388-1-yosryahmed@google.com> Message-Id: <20220404234154.1251388-3-yosryahmed@google.com> Mime-Version: 1.0 References: <20220404234154.1251388-1-yosryahmed@google.com> X-Mailer: git-send-email 2.35.1.1094.g7c7d902a7c-goog Subject: [PATCH v2 2/5] KVM: x86: mm: count KVM page table pages in pagetable stats From: Yosry Ahmed To: Huacai Chen , Aleksandar Markovic , Anup Patel , Atish Patra , Paolo Bonzini , Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel Cc: mizhang@google.com, David Matlack , kvm@vger.kernel.org, kvm-riscv@lists.infradead.org, linux-mips@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Shakeel Butt , Andrew Morton , Yosry Ahmed X-Stat-Signature: 8kqymazowoao3mo8g91jx1umc65p5dzq Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=oxrndx4r; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf09.hostedemail.com: domain of 3TIJLYgoKCJcPFJIP18D547FF7C5.3FDC9ELO-DDBM13B.FI7@flex--yosryahmed.bounces.google.com designates 209.85.210.201 as permitted sender) smtp.mailfrom=3TIJLYgoKCJcPFJIP18D547FF7C5.3FDC9ELO-DDBM13B.FI7@flex--yosryahmed.bounces.google.com X-Rspam-User: X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: F2BA8140033 X-HE-Tag: 1649115725-543563 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Count the pages used by KVM in x86 for page tables in pagetable stats. For legacy code, accounting pagetable stats is combined KVM's existing for mmu pages in newly introduced kvm_[un]account_mmu_page() helpers. For tdp mmu, introduce new tdp_[un]account_mmu_page() helpers. That combines accounting pagetable stats with the tdp_mmu_pages counter accounting. tdp_mmu_pages counter introduced in this series [1]. This patch was rebased on top of the first two patches in that series. [1]https://lore.kernel.org/lkml/20220401063636.2414200-1-mizhang@google.com/ Signed-off-by: Yosry Ahmed --- arch/x86/kvm/mmu/mmu.c | 16 ++++++++++++++-- arch/x86/kvm/mmu/tdp_mmu.c | 16 ++++++++++++++-- 2 files changed, 28 insertions(+), 4 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index f4020837fb48..28579b96a483 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -1671,6 +1671,18 @@ static inline void kvm_mod_used_mmu_pages(struct kvm *kvm, long nr) percpu_counter_add(&kvm_total_used_mmu_pages, nr); } +static void kvm_account_mmu_page(struct kvm *kvm, struct kvm_mmu_page *sp) +{ + kvm_mod_used_mmu_pages(kvm, +1); + kvm_account_pgtable_pages((void *)sp->spt, +1); +} + +static void kvm_unaccount_mmu_page(struct kvm *kvm, struct kvm_mmu_page *sp) +{ + kvm_mod_used_mmu_pages(kvm, -1); + kvm_account_pgtable_pages((void *)sp->spt, -1); +} + static void kvm_mmu_free_page(struct kvm_mmu_page *sp) { MMU_WARN_ON(!is_empty_shadow_page(sp->spt)); @@ -1726,7 +1738,7 @@ static struct kvm_mmu_page *kvm_mmu_alloc_page(struct kvm_vcpu *vcpu, int direct */ sp->mmu_valid_gen = vcpu->kvm->arch.mmu_valid_gen; list_add(&sp->link, &vcpu->kvm->arch.active_mmu_pages); - kvm_mod_used_mmu_pages(vcpu->kvm, +1); + kvm_account_mmu_page(vcpu->kvm, sp); return sp; } @@ -2342,7 +2354,7 @@ static bool __kvm_mmu_prepare_zap_page(struct kvm *kvm, list_add(&sp->link, invalid_list); else list_move(&sp->link, invalid_list); - kvm_mod_used_mmu_pages(kvm, -1); + kvm_unaccount_mmu_page(kvm, sp); } else { /* * Remove the active root from the active page list, the root diff --git a/arch/x86/kvm/mmu/tdp_mmu.c b/arch/x86/kvm/mmu/tdp_mmu.c index ed34f3f75f18..12bfcfc610c5 100644 --- a/arch/x86/kvm/mmu/tdp_mmu.c +++ b/arch/x86/kvm/mmu/tdp_mmu.c @@ -371,6 +371,18 @@ static void handle_changed_spte_dirty_log(struct kvm *kvm, int as_id, gfn_t gfn, } } +static void tdp_account_mmu_page(struct kvm *kvm, struct kvm_mmu_page *sp) +{ + atomic64_inc(&kvm->arch.tdp_mmu_pages); + kvm_account_pgtable_pages((void *)sp->spt, +1); +} + +static void tdp_unaccount_mmu_page(struct kvm *kvm, struct kvm_mmu_page *sp) +{ + atomic64_dec(&kvm->arch.tdp_mmu_pages); + kvm_account_pgtable_pages((void *)sp->spt, -1); +} + /** * tdp_mmu_unlink_sp() - Remove a shadow page from the list of used pages * @@ -383,7 +395,7 @@ static void handle_changed_spte_dirty_log(struct kvm *kvm, int as_id, gfn_t gfn, static void tdp_mmu_unlink_sp(struct kvm *kvm, struct kvm_mmu_page *sp, bool shared) { - atomic64_dec(&kvm->arch.tdp_mmu_pages); + tdp_unaccount_mmu_page(kvm, sp); if (!sp->lpage_disallowed) return; @@ -1121,7 +1133,7 @@ static int tdp_mmu_link_sp(struct kvm *kvm, struct tdp_iter *iter, tdp_mmu_set_spte(kvm, iter, spte); } - atomic64_inc(&kvm->arch.tdp_mmu_pages); + tdp_account_mmu_page(kvm, sp); return 0; }