From patchwork Tue Apr 26 05:38:59 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yosry Ahmed X-Patchwork-Id: 12826480 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 91836C433F5 for ; Tue, 26 Apr 2022 05:40:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=EFvEvURR40Tp9apTKW7tyFk3W3LnWVymhgoZSEQ/gt0=; b=U/tzDs5++c4Mpb3NYbFkjPsci2 V6wHGUPQYw4KdfwB+44tP+rCuIt+DfTJgik0p9gaYjqFHyr1DGtPTstSQzQKPw6nXK2+ZeQ3ds/vu 7mWeOT/8gEkHN3fqTUuN+AYa9nmMMZpK1fyCJufoo26RXLxuZbgrq4aAeO5J8wj3Eg3sfsLeh70Or /sGMCKTldE2i7MyHpm/2Vn9YGXowaG/zZkDAzg4kHGLZwQlPFjxwEjhFRMe2DmMTCLc0JZxFKaGB4 Sa2TktQzxvWsoGPRYBRcDQBIGGleeFa9N9QaDiCejgTlOteamGlMfaipU9YLUVD2dOcmhwGyArTC9 x1VUuU4A==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1njDvE-00Cbo2-3B; Tue, 26 Apr 2022 05:39:36 +0000 Received: from mail-pl1-x649.google.com ([2607:f8b0:4864:20::649]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1njDuz-00Cbb0-TR for linux-arm-kernel@lists.infradead.org; Tue, 26 Apr 2022 05:39:23 +0000 Received: by mail-pl1-x649.google.com with SMTP id u8-20020a170903124800b0015195a5826cso10721593plh.4 for ; Mon, 25 Apr 2022 22:39:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=pqJIcFO1YbM2kMD/7P/nIKLZQYZcxdFTC0Vu8kenpz8=; b=hMgaPC2CMNA38BUc/4Z6knwfUpImRGhDmly5NSUZyOtnP8B4CuknojlkCLYqYS4Xx1 Mhi5lC5QrZ7YRgbk+KhF1jxO4da//wil1kv0DtVbRdO8F+TyX8sogYFIfc6A4c6xm3ms lYBbLBA0PvfaZECq+R+oZTMXzoWCVObiN+831v/ZE7dPqbNMSBmVDHDznmgQjIaZcF80 DbgNwnHh+LuA3EHgAweI6pmmir7P1NHgymNtVNBQQWlypnpCZ27UdD5h4YOFmIpgGjKJ bshFBhtMcvXmKyOL1UgI8bURuYaux5ZQU9gUxnGIys2/Uzfi1iE84Wa9YQO783bR8pr0 4w0A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=pqJIcFO1YbM2kMD/7P/nIKLZQYZcxdFTC0Vu8kenpz8=; b=VTTFQq7jnWg31E1JaBfmhJdlfpVTDi54uYpoykmCCFL4RFbfVrGZ1JWKkNMSFj4qvC Kdtp3HYQ2G0K/92fAuFfETBOlEkhn3C6SPUiggfIt/Hy3qbcjUAGDOGnFVhnCu/yzvbv Q4SfnBFKfKKczA5Tlp3auAKM4Pa8O6TKakLB/4V/UkHl6Z01s3UDnbWmbS5qFMNzqT2B BAIrsVXSX6ekqhYfX1RSFy+avGTQV58+Jm/0pxI1jyCFJkJj8dqYmedJIm4mXx3zRCbl 2NdofEtMhR0wJ4ZrykQRsQQvEZB3n0uM+GPrApOLe3s5UHpro83Hs9lj89JRwPsPrw8G sfOw== X-Gm-Message-State: AOAM533NKojnoqkH9yMcGW0yiQeLtjCYJo6qSr5mPU7OtAYRaTaFBPNA M6AxrzFHO/32G7fMN1r/xj6CQqPpccDix/W8 X-Google-Smtp-Source: ABdhPJxejvTv5A7Rpc8BPLWrIBFEPFhn9b9ix6mcFYNGcgcKtqjBfcMW7mNudjijKMPTfwkilfD5EMyQa5cknhMO X-Received: from yosry.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:2327]) (user=yosryahmed job=sendgmr) by 2002:a63:82c2:0:b0:3ab:5747:8837 with SMTP id w185-20020a6382c2000000b003ab57478837mr6091278pgd.297.1650951559185; Mon, 25 Apr 2022 22:39:19 -0700 (PDT) Date: Tue, 26 Apr 2022 05:38:59 +0000 In-Reply-To: <20220426053904.3684293-1-yosryahmed@google.com> Message-Id: <20220426053904.3684293-2-yosryahmed@google.com> Mime-Version: 1.0 References: <20220426053904.3684293-1-yosryahmed@google.com> X-Mailer: git-send-email 2.36.0.rc2.479.g8af0fa9b8e-goog Subject: [PATCH v3 1/6] mm: add NR_SECONDARY_PAGETABLE stat From: Yosry Ahmed To: Sean Christopherson , Huacai Chen , Aleksandar Markovic , Anup Patel , Atish Patra , Paolo Bonzini , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , Andrew Morton , Johannes Weiner , Michal Hocko , Roman Gushchin , Shakeel Butt , James Morse , Catalin Marinas , Shameer Kolothum , Marc Zyngier , Alexandru Elisei , Suzuki K Poulose Cc: linux-mips@vger.kernel.org, kvm@vger.kernel.org, kvm-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, Yosry Ahmed X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220425_223921_982643_7C00BDA0 X-CRM114-Status: GOOD ( 13.93 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Add NR_SECONDARY_PAGETABLE stat to count secondary page table uses, e.g. KVM shadow page tables. This provides more insights on the kernel memory used by a workload. This stat will be used by subsequent patches to count KVM pagetable pages usage. Signed-off-by: Yosry Ahmed --- drivers/base/node.c | 2 ++ fs/proc/meminfo.c | 2 ++ include/linux/mmzone.h | 1 + mm/memcontrol.c | 1 + mm/page_alloc.c | 6 +++++- mm/vmstat.c | 1 + 6 files changed, 12 insertions(+), 1 deletion(-) diff --git a/drivers/base/node.c b/drivers/base/node.c index ec8bb24a5a22..9fe716832546 100644 --- a/drivers/base/node.c +++ b/drivers/base/node.c @@ -433,6 +433,7 @@ static ssize_t node_read_meminfo(struct device *dev, "Node %d ShadowCallStack:%8lu kB\n" #endif "Node %d PageTables: %8lu kB\n" + "Node %d SecPageTables: %8lu kB\n" "Node %d NFS_Unstable: %8lu kB\n" "Node %d Bounce: %8lu kB\n" "Node %d WritebackTmp: %8lu kB\n" @@ -459,6 +460,7 @@ static ssize_t node_read_meminfo(struct device *dev, nid, node_page_state(pgdat, NR_KERNEL_SCS_KB), #endif nid, K(node_page_state(pgdat, NR_PAGETABLE)), + nid, K(node_page_state(pgdat, NR_SECONDARY_PAGETABLE)), nid, 0UL, nid, K(sum_zone_node_page_state(nid, NR_BOUNCE)), nid, K(node_page_state(pgdat, NR_WRITEBACK_TEMP)), diff --git a/fs/proc/meminfo.c b/fs/proc/meminfo.c index 6fa761c9cc78..fad29024eb2e 100644 --- a/fs/proc/meminfo.c +++ b/fs/proc/meminfo.c @@ -108,6 +108,8 @@ static int meminfo_proc_show(struct seq_file *m, void *v) #endif show_val_kb(m, "PageTables: ", global_node_page_state(NR_PAGETABLE)); + show_val_kb(m, "SecPageTables: ", + global_node_page_state(NR_SECONDARY_PAGETABLE)); show_val_kb(m, "NFS_Unstable: ", 0); show_val_kb(m, "Bounce: ", diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index 962b14d403e8..35f57f2578c0 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -219,6 +219,7 @@ enum node_stat_item { NR_KERNEL_SCS_KB, /* measured in KiB */ #endif NR_PAGETABLE, /* used for pagetables */ + NR_SECONDARY_PAGETABLE, /* secondary pagetables, e.g. kvm shadow pagetables */ #ifdef CONFIG_SWAP NR_SWAPCACHE, #endif diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 725f76723220..89fbd1793960 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -1388,6 +1388,7 @@ static const struct memory_stat memory_stats[] = { { "kernel", MEMCG_KMEM }, { "kernel_stack", NR_KERNEL_STACK_KB }, { "pagetables", NR_PAGETABLE }, + { "secondary_pagetables", NR_SECONDARY_PAGETABLE }, { "percpu", MEMCG_PERCPU_B }, { "sock", MEMCG_SOCK }, { "vmalloc", MEMCG_VMALLOC }, diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 2db95780e003..96d00ae9d5c1 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -5932,7 +5932,8 @@ void show_free_areas(unsigned int filter, nodemask_t *nodemask) " active_file:%lu inactive_file:%lu isolated_file:%lu\n" " unevictable:%lu dirty:%lu writeback:%lu\n" " slab_reclaimable:%lu slab_unreclaimable:%lu\n" - " mapped:%lu shmem:%lu pagetables:%lu bounce:%lu\n" + " mapped:%lu shmem:%lu pagetables:%lu\n" + " secondary_pagetables:%lu bounce:%lu\n" " kernel_misc_reclaimable:%lu\n" " free:%lu free_pcp:%lu free_cma:%lu\n", global_node_page_state(NR_ACTIVE_ANON), @@ -5949,6 +5950,7 @@ void show_free_areas(unsigned int filter, nodemask_t *nodemask) global_node_page_state(NR_FILE_MAPPED), global_node_page_state(NR_SHMEM), global_node_page_state(NR_PAGETABLE), + global_node_page_state(NR_SECONDARY_PAGETABLE), global_zone_page_state(NR_BOUNCE), global_node_page_state(NR_KERNEL_MISC_RECLAIMABLE), global_zone_page_state(NR_FREE_PAGES), @@ -5982,6 +5984,7 @@ void show_free_areas(unsigned int filter, nodemask_t *nodemask) " shadow_call_stack:%lukB" #endif " pagetables:%lukB" + " secondary_pagetables:%lukB" " all_unreclaimable? %s" "\n", pgdat->node_id, @@ -6007,6 +6010,7 @@ void show_free_areas(unsigned int filter, nodemask_t *nodemask) node_page_state(pgdat, NR_KERNEL_SCS_KB), #endif K(node_page_state(pgdat, NR_PAGETABLE)), + K(node_page_state(pgdat, NR_SECONDARY_PAGETABLE)), pgdat->kswapd_failures >= MAX_RECLAIM_RETRIES ? "yes" : "no"); } diff --git a/mm/vmstat.c b/mm/vmstat.c index b75b1a64b54c..50bbec73809b 100644 --- a/mm/vmstat.c +++ b/mm/vmstat.c @@ -1240,6 +1240,7 @@ const char * const vmstat_text[] = { "nr_shadow_call_stack", #endif "nr_page_table_pages", + "nr_secondary_page_table_pages", #ifdef CONFIG_SWAP "nr_swapcached", #endif From patchwork Tue Apr 26 05:39:00 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yosry Ahmed X-Patchwork-Id: 12826481 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 90E13C433EF for ; Tue, 26 Apr 2022 05:40:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=0VAuXnI++npRCpZMHwJLPaSAFay3HF61YAsaSYdoLdc=; b=fEK6Z2oOwOYT9vv+wSUo+LC89/ HNaDFqlDTgtTw1NsYQCi0YD1WrGgkU3i8PggW4nkQAX6LD5pGcdnKqXA3acpHrAJngs35LEidckET dYmTeAx6TrNRn+N6rub8FaJ9jdfTJEbtfIIPpMSO2Q8VUDtmHS5nQC7ybLh5ajYsPHXd5g9O/eMr8 wtBtB9QCOdk6UOOeLztAwYWA0JdbYJn3uXVL2QnWxN2H1SprVofis45AJ60hTShZYWVa8Ir3+nXF2 tyTrXLSqP7UusRSb0MaAsnYKdZiQBROxoJiyvOyB/SIobWlhZ8EX/Tp872T7DTBGY00w/it0qtVMV gEy0XgxA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1njDvN-00Cbsi-8X; Tue, 26 Apr 2022 05:39:45 +0000 Received: from mail-pg1-x549.google.com ([2607:f8b0:4864:20::549]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1njDv1-00Cbbk-DY for linux-arm-kernel@lists.infradead.org; Tue, 26 Apr 2022 05:39:24 +0000 Received: by mail-pg1-x549.google.com with SMTP id q13-20020a638c4d000000b003821725ad66so10357074pgn.23 for ; Mon, 25 Apr 2022 22:39:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=75ck2E1viXT4HNKGmhZGUqApOq35yGNlgD3kbgrzKrw=; b=j/6W1Jv8f36ZodhL68vNcySdndUH2oV1c5OzKpRXYolTLJgwBi7OaIg/S/yKMnJp+b lqqaZUm1JBr3oQixRNwaNTgB81/saUAppnPAjz68yxAbmgh4igyoOQ92n9+HzD+TEgqv ajW2TFBTyLuEXXVz0pJuHgyUGpGRZnhkjtQoBS+4am6LkE8kl60VYw6Q/6q6/jo9xQNu GX4i53uLVaKJIRtdwjxSSMxM+nKgqH6qhEODIoJjq5XuOby3Ip1qDLu+YxucaAIrUMzY Psb6biLd1G6xVSJaPjnJffEREeUS4oL482CYqW+UsriFD7QsTwFn/nDOVqJKFywwwAfj 8XNg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=75ck2E1viXT4HNKGmhZGUqApOq35yGNlgD3kbgrzKrw=; b=LqxYoneXSXzDIw28dUPmBmVEVGZIW/azmao8nj59cD0hFpq9HAomCTyAX4b4JiOaNN yQUU5NrGuWZ1PhObSZ0UoEK2QMi3pv53CPcDA+/0QSQIXTF+8i5pbOowLHOowhx30Y/r hJ+G3IrWa/mcG2IixPQGd22UMmMVDLLxFJP/9d2VVf868ZRit+gZ2onG7wzvqBx1HxQf 77mim/qmMabiicGuOVm0zAVhx9uzw7N4c2eoV/kPi+XdHSxH3lJ3ZK9n0I/bmQPPloCl h87T6jbH83nGqOC9gph9gi4lA53aCWdh9NAGdnunO/cm62KaGsoFTFjQ5PGcRZI1E9pi 211g== X-Gm-Message-State: AOAM5331ToiZTUIJzXCkYib3gzA1y9l3LGLmWNy1HPchhkg8GeqKS5ic WFaGSu0uWbaIEW2jL78IjfrJm49tDLIPv/Da X-Google-Smtp-Source: ABdhPJz+02KDDV9V68suFOiQPkifF2E9mLN9n/bWhK+6QOZ7ZJ3fdHfj/Lkck5hrYAftknNAaw0TdY1+WqYFvIkS X-Received: from yosry.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:2327]) (user=yosryahmed job=sendgmr) by 2002:a17:902:9881:b0:158:f258:c3c3 with SMTP id s1-20020a170902988100b00158f258c3c3mr21548367plp.7.1650951560612; Mon, 25 Apr 2022 22:39:20 -0700 (PDT) Date: Tue, 26 Apr 2022 05:39:00 +0000 In-Reply-To: <20220426053904.3684293-1-yosryahmed@google.com> Message-Id: <20220426053904.3684293-3-yosryahmed@google.com> Mime-Version: 1.0 References: <20220426053904.3684293-1-yosryahmed@google.com> X-Mailer: git-send-email 2.36.0.rc2.479.g8af0fa9b8e-goog Subject: [PATCH v3 2/6] KVM: mmu: add a helper to account page table pages used by KVM. From: Yosry Ahmed To: Sean Christopherson , Huacai Chen , Aleksandar Markovic , Anup Patel , Atish Patra , Paolo Bonzini , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , Andrew Morton , Johannes Weiner , Michal Hocko , Roman Gushchin , Shakeel Butt , James Morse , Catalin Marinas , Shameer Kolothum , Marc Zyngier , Alexandru Elisei , Suzuki K Poulose Cc: linux-mips@vger.kernel.org, kvm@vger.kernel.org, kvm-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, Yosry Ahmed X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220425_223923_512240_363B3504 X-CRM114-Status: GOOD ( 10.67 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Add a helper to account pages used by KVM for page tables as pagetable stats. This function will be used by subsequent patches in different archs. Signed-off-by: Yosry Ahmed --- include/linux/kvm_host.h | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 252ee4a61b58..54cc4634053c 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -2221,6 +2221,15 @@ static inline void kvm_handle_signal_exit(struct kvm_vcpu *vcpu) } #endif /* CONFIG_KVM_XFER_TO_GUEST_WORK */ +/* + * If nr > 1, we assume virt is the address of the first page of a block of + * pages that were allocated together (i.e accounted together). + */ +static inline void kvm_account_pgtable_pages(void *virt, int nr) +{ + mod_lruvec_page_state(virt_to_page(virt), NR_SECONDARY_PAGETABLE, nr); +} + /* * This defines how many reserved entries we want to keep before we * kick the vcpu to the userspace to avoid dirty ring full. This From patchwork Tue Apr 26 05:39:01 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yosry Ahmed X-Patchwork-Id: 12826482 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 83F18C433EF for ; Tue, 26 Apr 2022 05:40:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=SwpZJGiAyxElpjyGNCEG2g9oAAeFNrvkA2EngHyh5Wg=; b=IPxy7VEcPKIyH9qONVO1I391x/ OZFfHlyZsLtQUJixEH8ZnWJqBuX2mEDloC1oCLSTUSRnr7jMb5+QHsLT+2DVS2fvf+IDCZezfwdMR sAT/Yugb7PZ0jwaSSIy/0T4mG+rbKhvyG47/FGJ2Fs25qTb0mq1DHu6xT60MdmGLiisLHpUlYLVoh C6NdCepsED62cZ61/y3fkbGgOT3h2kIw4Ug8S7x3Uil5ELoMpJCjV1OeuPfy4LkFA2XMV6L629dd4 30UeSUTAIEj9ocTeLnJdsiND0NMj0NoSt1Hlk8V5b6ZYrNvSQen5P/dwB7GQbS2NABy6HNIzbpNey 19wGV18w==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1njDvW-00Cbvt-CV; Tue, 26 Apr 2022 05:39:54 +0000 Received: from mail-pj1-x104a.google.com ([2607:f8b0:4864:20::104a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1njDv3-00CbdK-Pf for linux-arm-kernel@lists.infradead.org; Tue, 26 Apr 2022 05:39:27 +0000 Received: by mail-pj1-x104a.google.com with SMTP id s5-20020a17090aa10500b001d9a8e99e3aso647755pjp.2 for ; Mon, 25 Apr 2022 22:39:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=hEHBnHYwkzv2bOtPfk7H6AvzPnNcWPKpvOGkhdPVPjo=; b=DLc3lupOo8rx1wWbuV0lLgrWbKtbqj3WcXXLDzE4k2Y8czyusZqACX6ZnOE6B8ztaC am7MpQeXB4aItWkOAQ2oHlKhWHUdect3mQCIVLzhsaMmYiLv4m8FDjT5rGgSMVj6xFYe n/2zHp73ToEvYp7vTtGZ/ZOGc9fhJjKlDa7mNMowqbkak7jlAdBqGFu6TyUdSzX7j+tW 1GvB7gFqlyLilqCV94FzR63CxUYfiYooVrXSE9xnDdRgW//UD6tV4gP3rptKWSzN29Uy sjZA6fTvWQAJfVcB244P9F1XJ1tGayKZbE7/uQxhcWiS26Y/VIBnPxv5p5L8sde0AXSC zWQA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=hEHBnHYwkzv2bOtPfk7H6AvzPnNcWPKpvOGkhdPVPjo=; b=gIws5RJ6Ci0dNCImbyQr2kNX30hqUUyqSTD6BdiqROMdM3f+Mt3OSvHzw/kTqGaHfD 4m1kysENns/kZ5UGlTTasXB5O+PkrRFvoyyOHBp2AHeT4n8ZRrih9f6OX74oA4MGEusi 2S2Hak870eY7RWyeWLZPNxvcuw2PszhQFP2X1Z01UDIjMs99+mamkGrN/PIVIvZJcd9E VH/f+/+jx1Kp87w9tG5vQWmYt+rA71Nn+jIW9/3NtPArB0KGpavF1vS4fqc1+kVn7u5d D4eCrv8I/k64lzKlDfmV5LUToqhvdVe/Y+K50WXQKCv1LfQyuieHkvuKDJAST8sGTjqK +D+g== X-Gm-Message-State: AOAM531hCg4Kb3Vk9QbvEPXX2+Xt9tXhkUjzmRyYT/pbsWpdhQyW//1s rENxtPDJMkAOS/tYbZvDjot8EhexLRDWA/DQ X-Google-Smtp-Source: ABdhPJw1/Xuct4B2pJQt4L0Xw8CSz2dpJiVWmWgaHUUCpj09w2ZMYvrNGeSsqW4nHEy3AMUsWi84mvmV71p47dv/ X-Received: from yosry.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:2327]) (user=yosryahmed job=sendgmr) by 2002:a05:6a00:1955:b0:505:7902:36d3 with SMTP id s21-20020a056a00195500b00505790236d3mr22411201pfk.77.1650951562123; Mon, 25 Apr 2022 22:39:22 -0700 (PDT) Date: Tue, 26 Apr 2022 05:39:01 +0000 In-Reply-To: <20220426053904.3684293-1-yosryahmed@google.com> Message-Id: <20220426053904.3684293-4-yosryahmed@google.com> Mime-Version: 1.0 References: <20220426053904.3684293-1-yosryahmed@google.com> X-Mailer: git-send-email 2.36.0.rc2.479.g8af0fa9b8e-goog Subject: [PATCH v3 3/6] KVM: x86/mmu: count KVM page table pages in pagetable stats From: Yosry Ahmed To: Sean Christopherson , Huacai Chen , Aleksandar Markovic , Anup Patel , Atish Patra , Paolo Bonzini , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , Andrew Morton , Johannes Weiner , Michal Hocko , Roman Gushchin , Shakeel Butt , James Morse , Catalin Marinas , Shameer Kolothum , Marc Zyngier , Alexandru Elisei , Suzuki K Poulose Cc: linux-mips@vger.kernel.org, kvm@vger.kernel.org, kvm-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, Yosry Ahmed X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220425_223925_859784_E268804B X-CRM114-Status: GOOD ( 14.11 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Count the pages used by KVM in x86 for page tables in pagetable stats. For legacy code, accounting pagetable stats is combined KVM's existing for mmu pages in newly introduced kvm_[un]account_mmu_page() helpers. For tdp mmu, introduce new tdp_[un]account_mmu_page() helpers. That combines accounting pagetable stats with the tdp_mmu_pages counter accounting. tdp_mmu_pages counter introduced in this series [1]. This patch was rebased on top of the first two patches in that series. [1]https://lore.kernel.org/lkml/20220401063636.2414200-1-mizhang@google.com/ Signed-off-by: Yosry Ahmed --- arch/x86/kvm/mmu/mmu.c | 16 ++++++++++++++-- arch/x86/kvm/mmu/tdp_mmu.c | 16 ++++++++++++++-- 2 files changed, 28 insertions(+), 4 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 78d8e1d8fb99..e5b0e826445d 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -1679,6 +1679,18 @@ static inline void kvm_mod_used_mmu_pages(struct kvm *kvm, long nr) percpu_counter_add(&kvm_total_used_mmu_pages, nr); } +static void kvm_account_mmu_page(struct kvm *kvm, struct kvm_mmu_page *sp) +{ + kvm_mod_used_mmu_pages(kvm, +1); + kvm_account_pgtable_pages((void *)sp->spt, +1); +} + +static void kvm_unaccount_mmu_page(struct kvm *kvm, struct kvm_mmu_page *sp) +{ + kvm_mod_used_mmu_pages(kvm, -1); + kvm_account_pgtable_pages((void *)sp->spt, -1); +} + static void kvm_mmu_free_page(struct kvm_mmu_page *sp) { MMU_WARN_ON(!is_empty_shadow_page(sp->spt)); @@ -1734,7 +1746,7 @@ static struct kvm_mmu_page *kvm_mmu_alloc_page(struct kvm_vcpu *vcpu, int direct */ sp->mmu_valid_gen = vcpu->kvm->arch.mmu_valid_gen; list_add(&sp->link, &vcpu->kvm->arch.active_mmu_pages); - kvm_mod_used_mmu_pages(vcpu->kvm, +1); + kvm_account_mmu_page(vcpu->kvm, sp); return sp; } @@ -2363,7 +2375,7 @@ static bool __kvm_mmu_prepare_zap_page(struct kvm *kvm, list_add(&sp->link, invalid_list); else list_move(&sp->link, invalid_list); - kvm_mod_used_mmu_pages(kvm, -1); + kvm_unaccount_mmu_page(kvm, sp); } else { /* * Remove the active root from the active page list, the root diff --git a/arch/x86/kvm/mmu/tdp_mmu.c b/arch/x86/kvm/mmu/tdp_mmu.c index 3456277ade18..6295c4da5dee 100644 --- a/arch/x86/kvm/mmu/tdp_mmu.c +++ b/arch/x86/kvm/mmu/tdp_mmu.c @@ -371,6 +371,18 @@ static void handle_changed_spte_dirty_log(struct kvm *kvm, int as_id, gfn_t gfn, } } +static void tdp_account_mmu_page(struct kvm *kvm, struct kvm_mmu_page *sp) +{ + atomic64_inc(&kvm->arch.tdp_mmu_pages); + kvm_account_pgtable_pages((void *)sp->spt, +1); +} + +static void tdp_unaccount_mmu_page(struct kvm *kvm, struct kvm_mmu_page *sp) +{ + atomic64_dec(&kvm->arch.tdp_mmu_pages); + kvm_account_pgtable_pages((void *)sp->spt, -1); +} + /** * tdp_mmu_unlink_sp() - Remove a shadow page from the list of used pages * @@ -383,7 +395,7 @@ static void handle_changed_spte_dirty_log(struct kvm *kvm, int as_id, gfn_t gfn, static void tdp_mmu_unlink_sp(struct kvm *kvm, struct kvm_mmu_page *sp, bool shared) { - atomic64_dec(&kvm->arch.tdp_mmu_pages); + tdp_unaccount_mmu_page(kvm, sp); if (!sp->lpage_disallowed) return; @@ -1121,7 +1133,7 @@ static int tdp_mmu_link_sp(struct kvm *kvm, struct tdp_iter *iter, tdp_mmu_set_spte(kvm, iter, spte); } - atomic64_inc(&kvm->arch.tdp_mmu_pages); + tdp_account_mmu_page(kvm, sp); return 0; } From patchwork Tue Apr 26 05:39:02 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yosry Ahmed X-Patchwork-Id: 12826483 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B662DC433F5 for ; Tue, 26 Apr 2022 05:41:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=otxuD9571pwUe0n3hQXDej2SdlZXl4fVbAZj7hVzlpE=; b=mciSb7DXCuQCowGzaH35xiEF5d MfKE4gEi85kbUiY+TvX1KVbge/2ogO+cQsZkyBiGCwx9ExRUZa5sl7/xdt6DZdD/whvIpIw5B5M1B FXZWtCvUP5nhDBaeJj+1ms8Gp+3ETUk7gzXTRpoAWjS6hsaJudCWyORG3d+KGs7VWfGUR8smv9BUV N6ZzJlSqsvJv2FoA/tHMUojS5bQg0kruCotutkkombE6dxc33osJUvrLfoTKfGSklzM+wGeCai6cS OFUciCErKAwPqdVKcUcSs45ORwJ68m01fe0fCqVrFkU86Rt9o+Mm3d7jTMYYh0p40P4ap8zcpoCFF cbNoaF0g==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1njDvj-00Cc3U-Cc; Tue, 26 Apr 2022 05:40:08 +0000 Received: from mail-yb1-xb4a.google.com ([2607:f8b0:4864:20::b4a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1njDv4-00CbeS-T0 for linux-arm-kernel@lists.infradead.org; Tue, 26 Apr 2022 05:39:30 +0000 Received: by mail-yb1-xb4a.google.com with SMTP id i5-20020a258b05000000b006347131d40bso14940104ybl.17 for ; Mon, 25 Apr 2022 22:39:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=rkVIMEOGh9mkvJwwveKReDVW9pdj0MI38yR9aww0L8Y=; b=VaxZ9QDojjxQlUm2YdMrIs+Np0z42EAwX+5FwdFsGmfirABaWBLGu4+O/+OnZVMnB2 yV65SCNvcLlifKvPOwLyoSN2raRpMJixf7XBJbvt2IgBnKAMgmy9EoI2V5qAYUNEpNBj 0DkcosfssQhmfPgbfFCqPzbvAl505xwfkuwgUY1jUAmgROsb/sCH5Ta6e8jkGULAR2ow BeFv0T7Gmt1OSWvZTaVYZh1317pjpDLxCxlGz6k2PjNYyi23rjONu2KGEk+wljw4z3b7 izkoeFykixRYthOusX49NBX8Jn+iVpAOay4WfgPdTSfqzbBgNYavCJX02+HflylcLe+b RI4w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=rkVIMEOGh9mkvJwwveKReDVW9pdj0MI38yR9aww0L8Y=; b=JG436hihOgeGvYZt6lv4BfELAaFTer/8KmxK4bYgYr1/Eczcjtlzba4BO93k4dmMKj LwLSXzS/zXsFqVeQZ+ZJtqKtfOwmqgW2NHnfIg8gccCI9wx+souEvi2O0IFRBzxwaCem MgWJ+ia0j0M6wUsgB1b2GI7j5PlFMiw9/8ucn2+ZcHvdQgymqFfvVrR7qE7tuoLa2PkT detp92eJlPF0NtWbhwMDouHj2imUHiBtysqozExgpIc/+TXdtFmJ6cMGQaEy01QjQnLQ u1u4LESjNyGqej3YFlbqFdbwjxsi1s242X3FrofaAxl5KiBT6gk5QUYybxlOcoBIV73X DL0g== X-Gm-Message-State: AOAM532KYIChXb0pZqZ9POaBhWUWPt07cMFV8hNKZXQWeCAkA2xMGR+V bOvhIanjMu2k56M4xuF0v312/4idtm+rUuVx X-Google-Smtp-Source: ABdhPJxmQBQiztCDbvRqDZKtPeTInmWcTF3Beh6DXCAkOb/yzMJI4LJT6RtDgsJfCqVNqUFCNRVJFLpOB1qAQ38k X-Received: from yosry.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:2327]) (user=yosryahmed job=sendgmr) by 2002:a81:1c0a:0:b0:2f8:70d:ccb6 with SMTP id c10-20020a811c0a000000b002f8070dccb6mr4728213ywc.365.1650951563874; Mon, 25 Apr 2022 22:39:23 -0700 (PDT) Date: Tue, 26 Apr 2022 05:39:02 +0000 In-Reply-To: <20220426053904.3684293-1-yosryahmed@google.com> Message-Id: <20220426053904.3684293-5-yosryahmed@google.com> Mime-Version: 1.0 References: <20220426053904.3684293-1-yosryahmed@google.com> X-Mailer: git-send-email 2.36.0.rc2.479.g8af0fa9b8e-goog Subject: [PATCH v3 4/6] KVM: arm64/mmu: count KVM page table pages in pagetable stats From: Yosry Ahmed To: Sean Christopherson , Huacai Chen , Aleksandar Markovic , Anup Patel , Atish Patra , Paolo Bonzini , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , Andrew Morton , Johannes Weiner , Michal Hocko , Roman Gushchin , Shakeel Butt , James Morse , Catalin Marinas , Shameer Kolothum , Marc Zyngier , Alexandru Elisei , Suzuki K Poulose Cc: linux-mips@vger.kernel.org, kvm@vger.kernel.org, kvm-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, Yosry Ahmed X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220425_223927_082043_FC493413 X-CRM114-Status: GOOD ( 15.96 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Count the pages used by KVM in arm64 for page tables in pagetable stats. Account pages allocated for PTEs in pgtable init functions and kvm_set_table_pte(). Since most page table pages are freed using put_page(), add a helper function put_pte_page() that checks if this is the last ref for a pte page before putting it, and unaccounts stats accordingly. Signed-off-by: Yosry Ahmed --- arch/arm64/kernel/image-vars.h | 3 ++ arch/arm64/kvm/hyp/pgtable.c | 50 +++++++++++++++++++++------------- 2 files changed, 34 insertions(+), 19 deletions(-) diff --git a/arch/arm64/kernel/image-vars.h b/arch/arm64/kernel/image-vars.h index 241c86b67d01..25bf058714f6 100644 --- a/arch/arm64/kernel/image-vars.h +++ b/arch/arm64/kernel/image-vars.h @@ -143,6 +143,9 @@ KVM_NVHE_ALIAS(__hyp_rodata_end); /* pKVM static key */ KVM_NVHE_ALIAS(kvm_protected_mode_initialized); +/* Called by kvm_account_pgtable_pages() to update pagetable stats */ +KVM_NVHE_ALIAS(__mod_lruvec_page_state); + #endif /* CONFIG_KVM */ #endif /* __ARM64_KERNEL_IMAGE_VARS_H */ diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c index 2cb3867eb7c2..53e13c3313e9 100644 --- a/arch/arm64/kvm/hyp/pgtable.c +++ b/arch/arm64/kvm/hyp/pgtable.c @@ -152,6 +152,7 @@ static void kvm_set_table_pte(kvm_pte_t *ptep, kvm_pte_t *childp, WARN_ON(kvm_pte_valid(old)); smp_store_release(ptep, pte); + kvm_account_pgtable_pages((void *)childp, +1); } static kvm_pte_t kvm_init_valid_leaf_pte(u64 pa, kvm_pte_t attr, u32 level) @@ -326,6 +327,14 @@ int kvm_pgtable_get_leaf(struct kvm_pgtable *pgt, u64 addr, return ret; } +static void put_pte_page(kvm_pte_t *ptep, struct kvm_pgtable_mm_ops *mm_ops) +{ + /* If this is the last page ref, decrement pagetable stats first. */ + if (!mm_ops->page_count || mm_ops->page_count(ptep) == 1) + kvm_account_pgtable_pages((void *)ptep, -1); + mm_ops->put_page(ptep); +} + struct hyp_map_data { u64 phys; kvm_pte_t attr; @@ -488,10 +497,10 @@ static int hyp_unmap_walker(u64 addr, u64 end, u32 level, kvm_pte_t *ptep, dsb(ish); isb(); - mm_ops->put_page(ptep); + put_pte_page(ptep, mm_ops); if (childp) - mm_ops->put_page(childp); + put_pte_page(childp, mm_ops); return 0; } @@ -522,6 +531,7 @@ int kvm_pgtable_hyp_init(struct kvm_pgtable *pgt, u32 va_bits, pgt->pgd = (kvm_pte_t *)mm_ops->zalloc_page(NULL); if (!pgt->pgd) return -ENOMEM; + kvm_account_pgtable_pages((void *)pgt->pgd, +1); pgt->ia_bits = va_bits; pgt->start_level = KVM_PGTABLE_MAX_LEVELS - levels; @@ -541,10 +551,10 @@ static int hyp_free_walker(u64 addr, u64 end, u32 level, kvm_pte_t *ptep, if (!kvm_pte_valid(pte)) return 0; - mm_ops->put_page(ptep); + put_pte_page(ptep, mm_ops); if (kvm_pte_table(pte, level)) - mm_ops->put_page(kvm_pte_follow(pte, mm_ops)); + put_pte_page(kvm_pte_follow(pte, mm_ops), mm_ops); return 0; } @@ -558,7 +568,7 @@ void kvm_pgtable_hyp_destroy(struct kvm_pgtable *pgt) }; WARN_ON(kvm_pgtable_walk(pgt, 0, BIT(pgt->ia_bits), &walker)); - pgt->mm_ops->put_page(pgt->pgd); + put_pte_page(pgt->pgd, pgt->mm_ops); pgt->pgd = NULL; } @@ -694,7 +704,7 @@ static void stage2_put_pte(kvm_pte_t *ptep, struct kvm_s2_mmu *mmu, u64 addr, kvm_call_hyp(__kvm_tlb_flush_vmid_ipa, mmu, addr, level); } - mm_ops->put_page(ptep); + put_pte_page(ptep, mm_ops); } static bool stage2_pte_cacheable(struct kvm_pgtable *pgt, kvm_pte_t pte) @@ -795,7 +805,7 @@ static int stage2_map_walk_leaf(u64 addr, u64 end, u32 level, kvm_pte_t *ptep, if (data->anchor) { if (stage2_pte_is_counted(pte)) - mm_ops->put_page(ptep); + put_pte_page(ptep, mm_ops); return 0; } @@ -848,8 +858,8 @@ static int stage2_map_walk_table_post(u64 addr, u64 end, u32 level, childp = kvm_pte_follow(*ptep, mm_ops); } - mm_ops->put_page(childp); - mm_ops->put_page(ptep); + put_pte_page(childp, mm_ops); + put_pte_page(ptep, mm_ops); return ret; } @@ -962,7 +972,7 @@ static int stage2_unmap_walker(u64 addr, u64 end, u32 level, kvm_pte_t *ptep, if (!kvm_pte_valid(pte)) { if (stage2_pte_is_counted(pte)) { kvm_clear_pte(ptep); - mm_ops->put_page(ptep); + put_pte_page(ptep, mm_ops); } return 0; } @@ -988,7 +998,7 @@ static int stage2_unmap_walker(u64 addr, u64 end, u32 level, kvm_pte_t *ptep, kvm_granule_size(level)); if (childp) - mm_ops->put_page(childp); + put_pte_page(childp, mm_ops); return 0; } @@ -1177,16 +1187,17 @@ int __kvm_pgtable_stage2_init(struct kvm_pgtable *pgt, struct kvm_s2_mmu *mmu, enum kvm_pgtable_stage2_flags flags, kvm_pgtable_force_pte_cb_t force_pte_cb) { - size_t pgd_sz; + u32 pgd_num; u64 vtcr = mmu->arch->vtcr; u32 ia_bits = VTCR_EL2_IPA(vtcr); u32 sl0 = FIELD_GET(VTCR_EL2_SL0_MASK, vtcr); u32 start_level = VTCR_EL2_TGRAN_SL0_BASE - sl0; - pgd_sz = kvm_pgd_pages(ia_bits, start_level) * PAGE_SIZE; - pgt->pgd = mm_ops->zalloc_pages_exact(pgd_sz); + pgd_num = kvm_pgd_pages(ia_bits, start_level); + pgt->pgd = mm_ops->zalloc_pages_exact(pgd_num * PAGE_SIZE); if (!pgt->pgd) return -ENOMEM; + kvm_account_pgtable_pages((void *)pgt->pgd, +pgd_num); pgt->ia_bits = ia_bits; pgt->start_level = start_level; @@ -1210,17 +1221,17 @@ static int stage2_free_walker(u64 addr, u64 end, u32 level, kvm_pte_t *ptep, if (!stage2_pte_is_counted(pte)) return 0; - mm_ops->put_page(ptep); + put_pte_page(ptep, mm_ops); if (kvm_pte_table(pte, level)) - mm_ops->put_page(kvm_pte_follow(pte, mm_ops)); + put_pte_page(kvm_pte_follow(pte, mm_ops), mm_ops); return 0; } void kvm_pgtable_stage2_destroy(struct kvm_pgtable *pgt) { - size_t pgd_sz; + u32 pgd_num; struct kvm_pgtable_walker walker = { .cb = stage2_free_walker, .flags = KVM_PGTABLE_WALK_LEAF | @@ -1229,7 +1240,8 @@ void kvm_pgtable_stage2_destroy(struct kvm_pgtable *pgt) }; WARN_ON(kvm_pgtable_walk(pgt, 0, BIT(pgt->ia_bits), &walker)); - pgd_sz = kvm_pgd_pages(pgt->ia_bits, pgt->start_level) * PAGE_SIZE; - pgt->mm_ops->free_pages_exact(pgt->pgd, pgd_sz); + pgd_num = kvm_pgd_pages(pgt->ia_bits, pgt->start_level); + kvm_account_pgtable_pages((void *)pgt->pgd, -pgd_num); + pgt->mm_ops->free_pages_exact(pgt->pgd, pgd_num * PAGE_SIZE); pgt->pgd = NULL; } From patchwork Tue Apr 26 05:39:03 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yosry Ahmed X-Patchwork-Id: 12826484 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5A7DAC433EF for ; Tue, 26 Apr 2022 05:41:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=+zkXZzr+3sUAsyy3BDJgpFkHAbPawS1+FfcJt1oajqQ=; b=CkHLoAungRKI4clv8R4Umlnz40 RGFJEqDeg2G957Vgwv8kIJ7w8VwwnU6yt6F5yqsoMLpkMlMme99pkdZ5t/6tkGH3BITF3J+cug27I ag46Qxmff/rR+ZYcAFBQLHyXYfW0ejS8eyGmedBfzLKJPy9bBENUttI/JAONIlsVpHvPhM5zqWwS3 KBKlYlQal9FQ0+XQ86HD3mlbDPsSLnK4zs5OGlyTO5bKuFMsH/TGWaqwak4RlDLOqgkU2BLVrm410 6QYU2+SghVToXvkCsSaMWDb7D9Usz8zVwM+VnlUPQClQH0nXKYgTseIBU02sFbJdRD5Zy//M1rjcJ HbKRKM6w==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1njDw5-00CcFz-Pw; Tue, 26 Apr 2022 05:40:30 +0000 Received: from mail-pf1-x449.google.com ([2607:f8b0:4864:20::449]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1njDv6-00CbfS-OK for linux-arm-kernel@lists.infradead.org; Tue, 26 Apr 2022 05:39:31 +0000 Received: by mail-pf1-x449.google.com with SMTP id f136-20020a62388e000000b0050d26161b6fso4983765pfa.21 for ; Mon, 25 Apr 2022 22:39:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=a9VzBJlMLigws4pRoWaDUwEnDDiqJQ0nJjM8cxStkWo=; b=p8mJqCwHacTbFhvYHu8ybGXK2P+PghFMqaxHHJuqdpta07t/m8JQTkdFg82eEvuWYd OdSYMSKPBvO7ptjR046sAX/bHvFVjhg/pzuwVzXmGfoIsID/2GXXQHWYzh25sp8Zhqzf htWeN1iMzxl7/9Pdv1zinxHd6q+dx76sncvVGGjZOTWwwfxxB2qSbB8gn6IY5+O3yc7/ BeF9G0WjghrY1fZ5jCmiJkVNg5fp3DJ1BPeCzpmLCtKqzsEWqGLssoJmyaPfwKCC+CiD 5TQ/VweVfCM1sVLfYaViWaZEyQtlCxE0YpkORn8opIIBD11EyDSBFcvRbtm+lxTTiRjt 7Vwg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=a9VzBJlMLigws4pRoWaDUwEnDDiqJQ0nJjM8cxStkWo=; b=KTX+2cOUxxbqVif/W4luhyI9drXGQTNIlnkugjrfCKV+XHqlUxJ2ndfuh3Kk/Ydnbr VwasajeZvO8cctV7cfInGj22UK/wU+bwgAGYUnlnQk8BLru0sdtWFWf0d4Fw65lFcys9 7tYwtfYJiv/Ipvv9EfvCGa/coU1xWEvTCOas+GaqYn3N82InQeEWN0mm7b48GrJ20keI G3wBsrxXOa6ptQ7+qgYzFPrQDcBXebwbbY26RVGJjf3f53H7c7agbRySHrQNM5chfjKf rwt3+hS5SOe7D09KO/g9mOj+e1+O3VBWQ4sp5GFDdUAglQAWDKt+FhObk8FgXB/U63yJ shEQ== X-Gm-Message-State: AOAM532XrcIkBlS67MjL+lGID5U6SJRmb5vhU33Qz7gAKFFd9rYSGAtP RCRuPGBb1AIAQ/aPX35vY+qvAQJEDVfhn1AC X-Google-Smtp-Source: ABdhPJxYCaRW6ujjy+jD12c3CK0Xph6XJaTEbXPAfsnpbu06l91YAgqtjLNqwlREDx07xf+SlrnmNlHPGW+NrZOR X-Received: from yosry.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:2327]) (user=yosryahmed job=sendgmr) by 2002:a63:de12:0:b0:3ab:7c33:2894 with SMTP id f18-20020a63de12000000b003ab7c332894mr3565024pgg.187.1650951565540; Mon, 25 Apr 2022 22:39:25 -0700 (PDT) Date: Tue, 26 Apr 2022 05:39:03 +0000 In-Reply-To: <20220426053904.3684293-1-yosryahmed@google.com> Message-Id: <20220426053904.3684293-6-yosryahmed@google.com> Mime-Version: 1.0 References: <20220426053904.3684293-1-yosryahmed@google.com> X-Mailer: git-send-email 2.36.0.rc2.479.g8af0fa9b8e-goog Subject: [PATCH v3 5/6] KVM: riscv/mmu: count KVM page table pages in pagetable stats From: Yosry Ahmed To: Sean Christopherson , Huacai Chen , Aleksandar Markovic , Anup Patel , Atish Patra , Paolo Bonzini , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , Andrew Morton , Johannes Weiner , Michal Hocko , Roman Gushchin , Shakeel Butt , James Morse , Catalin Marinas , Shameer Kolothum , Marc Zyngier , Alexandru Elisei , Suzuki K Poulose Cc: linux-mips@vger.kernel.org, kvm@vger.kernel.org, kvm-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, Yosry Ahmed X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220425_223928_855730_E469EDCC X-CRM114-Status: GOOD ( 14.71 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Count the pages used by KVM in riscv for page tables in pagetable stats. Signed-off-by: Yosry Ahmed --- arch/riscv/kvm/mmu.c | 26 +++++++++++++++++++------- 1 file changed, 19 insertions(+), 7 deletions(-) diff --git a/arch/riscv/kvm/mmu.c b/arch/riscv/kvm/mmu.c index f80a34fbf102..fcfb75713750 100644 --- a/arch/riscv/kvm/mmu.c +++ b/arch/riscv/kvm/mmu.c @@ -152,6 +152,7 @@ static int stage2_set_pte(struct kvm *kvm, u32 level, next_ptep = kvm_mmu_memory_cache_alloc(pcache); if (!next_ptep) return -ENOMEM; + kvm_account_pgtable_pages((void *)next_ptep, +1); *ptep = pfn_pte(PFN_DOWN(__pa(next_ptep)), __pgprot(_PAGE_TABLE)); } else { @@ -229,6 +230,7 @@ static void stage2_op_pte(struct kvm *kvm, gpa_t addr, pte_t *next_ptep; u32 next_ptep_level; unsigned long next_page_size, page_size; + struct page *p; ret = stage2_level_to_page_size(ptep_level, &page_size); if (ret) @@ -252,8 +254,13 @@ static void stage2_op_pte(struct kvm *kvm, gpa_t addr, for (i = 0; i < PTRS_PER_PTE; i++) stage2_op_pte(kvm, addr + i * next_page_size, &next_ptep[i], next_ptep_level, op); - if (op == STAGE2_OP_CLEAR) - put_page(virt_to_page(next_ptep)); + if (op == STAGE2_OP_CLEAR) { + p = virt_to_page(next_ptep); + if (page_count(p) == 1) + kvm_account_pgtable_pages((void *)next_ptep, + -1); + put_page(p); + } } else { if (op == STAGE2_OP_CLEAR) set_pte(ptep, __pte(0)); @@ -700,25 +707,27 @@ int kvm_riscv_stage2_map(struct kvm_vcpu *vcpu, int kvm_riscv_stage2_alloc_pgd(struct kvm *kvm) { struct page *pgd_page; + int order; if (kvm->arch.pgd != NULL) { kvm_err("kvm_arch already initialized?\n"); return -EINVAL; } - pgd_page = alloc_pages(GFP_KERNEL | __GFP_ZERO, - get_order(stage2_pgd_size)); + order = get_order(stage2_pgd_size); + pgd_page = alloc_pages(GFP_KERNEL | __GFP_ZERO, order); if (!pgd_page) return -ENOMEM; kvm->arch.pgd = page_to_virt(pgd_page); kvm->arch.pgd_phys = page_to_phys(pgd_page); - + kvm_account_pgtable_pages((void *)kvm->arch.pgd, +(1UL << order)); return 0; } void kvm_riscv_stage2_free_pgd(struct kvm *kvm) { void *pgd = NULL; + int order; spin_lock(&kvm->mmu_lock); if (kvm->arch.pgd) { @@ -729,8 +738,11 @@ void kvm_riscv_stage2_free_pgd(struct kvm *kvm) } spin_unlock(&kvm->mmu_lock); - if (pgd) - free_pages((unsigned long)pgd, get_order(stage2_pgd_size)); + if (pgd) { + order = get_order(stage2_pgd_size); + kvm_account_pgtable_pages((void *)pgd, -(1UL << order)); + free_pages((unsigned long)pgd, order); + } } void kvm_riscv_stage2_update_hgatp(struct kvm_vcpu *vcpu) From patchwork Tue Apr 26 05:39:04 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yosry Ahmed X-Patchwork-Id: 12826485 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id EF0E8C433FE for ; Tue, 26 Apr 2022 05:41:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=gE/aBBJziDCHY/gAsavTWwuing/kaYcovNhProc37NY=; b=qQGlitaLAi4nyczvbop4+fnu7C XzXk4/edoHGGj5CXwhRfN05iMSQOZsDB116gqzOEMk84ihrAkbAkRapLkhotxf0PvVKKZgB86gXCF NO7KQ/ZPTpogTOk6PXL3GvPSsSu5WYj5ProNSuPpxG/WE+rFjiylBYaKKfPzCuEYv3Fmk05rgAGmz sUxg2MiuXxo9T/LOfcERpmpe0rKT1l2N4FGz8lAkbpcAo5YKbWeCq2H19Tu5Db02ae299ZFIvGkGI ++pfKDQmV6sZlt5lXJFkf2yPMWKXfRuDiC5fozRApmSNxCAVaSMpaH/o3k1cIMHHlG+K+r1rVm+m/ MA5UQtkA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1njDwN-00CcPL-0t; Tue, 26 Apr 2022 05:40:47 +0000 Received: from mail-pg1-x549.google.com ([2607:f8b0:4864:20::549]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1njDv6-00Cbh2-Rr for linux-arm-kernel@lists.infradead.org; Tue, 26 Apr 2022 05:39:32 +0000 Received: by mail-pg1-x549.google.com with SMTP id u3-20020a637903000000b003ab040c4807so2793213pgc.13 for ; Mon, 25 Apr 2022 22:39:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=UwrHlKcfzo3tcKJryiZhc3lnWD2SSBQRLRI0NxtXWrY=; b=lNtBON0+mp7QtqbJUF/yxX4uqPIpQ5Y7hbeXAfVLxM4ZET7uCEK3knJCpkoyZMJa9s i7nzVR4VlmZxg+avleS7/Kn/bJZBjnZBXIxlYf/uArgxa5lJaYe/WDTMeZAb28KsbwWU ocTkXwvTjA1uESTwFj+I7wr5Rl5sM2mE2cN5TIARIpTfKY2dD2+gpHj1coFpygAy+fc7 zaCTEQdiWA+0CcKFMaPUEAafXSOqS1yBx10IaODNf6+xQNBSZhNIV/u+JCOUySMFWRfr Ms9jv8dEpQuxx8npEiVjqTZIl7JhMtAENDSGpOK7TpWZOGwxxlN4+Ot896M99UFlnkfM gIIQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=UwrHlKcfzo3tcKJryiZhc3lnWD2SSBQRLRI0NxtXWrY=; b=QEGumo2ze7zwJ7e/L2MgLK8hjwjuxPtFHiIiH1M2IcztkxfFFYWP8+UYKI1YZrKrXn V3c6Z6I5AIWV6GBbuBzfcmegkgrfPnwbhh3m5xtn5vdSCOet1A+qpGSX5OCGcfTbUFyl B9WyD+oOM7orvrljM+AK3bRBByu7LXDOoNvvulSQya/BJVxQe6nZYAvoOnMhb55ezKo3 ycWQToOdL7MGncuYju2twDb1hgZMwX72ZoPJm//6t9tnp6UuhyBKK8vIECkwuS9R/k8u Ry8mAacX3uaYfwLn+KrjYu9b6mtb0QzeYPxUcSEm37Zm1s5+jll+c2UOCNKA4HiWBco2 Yjrw== X-Gm-Message-State: AOAM530IWkavvugRLyWr3NhfqNzkGEDy/ccwNv016A9VZzXeYEwieq1z w6U8VRpKPn02HzZy+Innpds888FHqVJs4pWo X-Google-Smtp-Source: ABdhPJx5ME5q1MQ2ad+XyRFl1/rgChFdKOrjKjC1cyKbrSm7xzgtrgQj+f7lrmteCRcN2yNApoWsTRuKdbMVeiBw X-Received: from yosry.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:2327]) (user=yosryahmed job=sendgmr) by 2002:a05:6a00:1acd:b0:50c:f6ca:6e6f with SMTP id f13-20020a056a001acd00b0050cf6ca6e6fmr18897423pfv.75.1650951567275; Mon, 25 Apr 2022 22:39:27 -0700 (PDT) Date: Tue, 26 Apr 2022 05:39:04 +0000 In-Reply-To: <20220426053904.3684293-1-yosryahmed@google.com> Message-Id: <20220426053904.3684293-7-yosryahmed@google.com> Mime-Version: 1.0 References: <20220426053904.3684293-1-yosryahmed@google.com> X-Mailer: git-send-email 2.36.0.rc2.479.g8af0fa9b8e-goog Subject: [PATCH v3 6/6] KVM: mips/mmu: count KVM page table pages in pagetable stats From: Yosry Ahmed To: Sean Christopherson , Huacai Chen , Aleksandar Markovic , Anup Patel , Atish Patra , Paolo Bonzini , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , Andrew Morton , Johannes Weiner , Michal Hocko , Roman Gushchin , Shakeel Butt , James Morse , Catalin Marinas , Shameer Kolothum , Marc Zyngier , Alexandru Elisei , Suzuki K Poulose Cc: linux-mips@vger.kernel.org, kvm@vger.kernel.org, kvm-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, Yosry Ahmed X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220425_223928_952876_06043F0A X-CRM114-Status: GOOD ( 12.56 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Count the pages used by KVM in mips for page tables in pagetable stats. Signed-off-by: Yosry Ahmed --- arch/mips/kvm/mips.c | 1 + arch/mips/kvm/mmu.c | 9 ++++++++- 2 files changed, 9 insertions(+), 1 deletion(-) diff --git a/arch/mips/kvm/mips.c b/arch/mips/kvm/mips.c index a25e0b73ee70..e60c1920a408 100644 --- a/arch/mips/kvm/mips.c +++ b/arch/mips/kvm/mips.c @@ -175,6 +175,7 @@ static void kvm_mips_free_gpa_pt(struct kvm *kvm) { /* It should always be safe to remove after flushing the whole range */ WARN_ON(!kvm_mips_flush_gpa_pt(kvm, 0, ~0)); + kvm_account_pgtable_pages((void *)kvm->arch.gpa_mm.pgd, -1); pgd_free(NULL, kvm->arch.gpa_mm.pgd); } diff --git a/arch/mips/kvm/mmu.c b/arch/mips/kvm/mmu.c index 1bfd1b501d82..18da2ac2ded7 100644 --- a/arch/mips/kvm/mmu.c +++ b/arch/mips/kvm/mmu.c @@ -81,8 +81,10 @@ pgd_t *kvm_pgd_alloc(void) pgd_t *ret; ret = (pgd_t *)__get_free_pages(GFP_KERNEL, PGD_ORDER); - if (ret) + if (ret) { kvm_pgd_init(ret); + kvm_account_pgtable_pages((void *)ret, +1); + } return ret; } @@ -125,6 +127,7 @@ static pte_t *kvm_mips_walk_pgd(pgd_t *pgd, struct kvm_mmu_memory_cache *cache, pmd_init((unsigned long)new_pmd, (unsigned long)invalid_pte_table); pud_populate(NULL, pud, new_pmd); + kvm_account_pgtable_pages((void *)new_pmd, +1); } pmd = pmd_offset(pud, addr); if (pmd_none(*pmd)) { @@ -135,6 +138,7 @@ static pte_t *kvm_mips_walk_pgd(pgd_t *pgd, struct kvm_mmu_memory_cache *cache, new_pte = kvm_mmu_memory_cache_alloc(cache); clear_page(new_pte); pmd_populate_kernel(NULL, pmd, new_pte); + kvm_account_pgtable_pages((void *)new_pte, +1); } return pte_offset_kernel(pmd, addr); } @@ -189,6 +193,7 @@ static bool kvm_mips_flush_gpa_pmd(pmd_t *pmd, unsigned long start_gpa, if (kvm_mips_flush_gpa_pte(pte, start_gpa, end)) { pmd_clear(pmd + i); + kvm_account_pgtable_pages((void *)pte, -1); pte_free_kernel(NULL, pte); } else { safe_to_remove = false; @@ -217,6 +222,7 @@ static bool kvm_mips_flush_gpa_pud(pud_t *pud, unsigned long start_gpa, if (kvm_mips_flush_gpa_pmd(pmd, start_gpa, end)) { pud_clear(pud + i); + kvm_account_pgtable_pages((void *)pmd, -1); pmd_free(NULL, pmd); } else { safe_to_remove = false; @@ -247,6 +253,7 @@ static bool kvm_mips_flush_gpa_pgd(pgd_t *pgd, unsigned long start_gpa, if (kvm_mips_flush_gpa_pud(pud, start_gpa, end)) { pgd_clear(pgd + i); + kvm_account_pgtable_pages((void *)pud, -1); pud_free(NULL, pud); } else { safe_to_remove = false;