From patchwork Fri Apr 29 20:11:28 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yosry Ahmed X-Patchwork-Id: 12832702 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id BF61EC433F5 for ; Fri, 29 Apr 2022 20:13:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=KQTYQXJ5ITkg3gi6g94YHU0lud5a9w82bG5TXcTuqEI=; b=h1KCC2BkHvi8+FoBin8KpCSBc+ l+y/EcRgmZt2nuAArs522bfzRi59XHuQ82PA1qI7bK1xFu33N2YWJZ5Y+1DOg/pBc5NIUkBMCpRnF HRD89DRgss/MYp1cUSEvwEwOhz0JpNPOB1o55AzSz8u65Ysko17cQdwpBxNZTbYmdAzOKCCeXNlSd 0LXgFANuJ13kj368D7h89XRtheibzuRRCb44gcJkVMsAabLs8dAMCPFfOv1/VjYBTXW1HEAz4LX0k lFSGV3R7o/uKLHxoEAwVGsZS0bbx30Vms0mnMtFQSc67S41yzFXepICqayVgtXGMXsbTPt+8jyTy7 SRslAEyg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nkWyA-00CQPW-S0; Fri, 29 Apr 2022 20:12:02 +0000 Received: from mail-pf1-x449.google.com ([2607:f8b0:4864:20::449]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1nkWxw-00CQIE-8v for linux-arm-kernel@lists.infradead.org; Fri, 29 Apr 2022 20:11:50 +0000 Received: by mail-pf1-x449.google.com with SMTP id b184-20020a62cfc1000000b0050d209cb8dcso4659765pfg.3 for ; Fri, 29 Apr 2022 13:11:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=XWmvv+rOOJ4TRVNVgMsu676xrbizP5MVw15ikvl+YNA=; b=HfyI+I9KFfzEY57e1psyAcoEEK5S5FLD37YEwxCaI6DxL7RJnIvpQx5Q/7BdxAVEkU eYhxQ7BpkX9PuBZbZXdm2r8LsiQFhO29Vp1bPrFbgpxw1v+4JyexJU5CZ6256Jy6mjXq 4AE+iGeFG5FbXNEWcjqIcljDZjSvD98jqCiGHSAPnS0Bb5iP1tfqk2oJZ4TEfzSkbzvz DMomOJ4iqkg7e02QkJvx+0VmEHEMHzLy2KYBVSfODpmoru87S3XM9d/lUmrLB403zYM7 MJXD5qa+NJqKJVjWF1YBFJE02T+OxD5+ClPsk9eymj1Ca5zO/O2QLzOSXboWxZEDa0V8 os+g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=XWmvv+rOOJ4TRVNVgMsu676xrbizP5MVw15ikvl+YNA=; b=sUxQXBh6W0J3H92OOT0R/G4TZT6A5elLOMW/l5zdumExiZkjASj3GpBckKP+Q4fdou 3BnXQJAQjbeWYB1v05nMxfEh5Tlz756Vea8d7SSPn+LFGYMw8oKQfjYiNqTiXM9T8a6c R5kWAFrAPRQIlWEMPPkj4mtZhg5D1giJ7DitANjBa2iMeUg8GOFQvqBw9sd3xOxuQnJR /tPt/nfHPQyep2k7eOEMt/nMthsh8RTf4x/oQR3+/zDTYolFRDU0WOKtIL+abeSBWLMP ir/ky3JewXhlKnnF65KQRY+jLlWIHweg9PHlc2MVgDmbwq2VlIa5lnHAwwKyZGtNr956 YReQ== X-Gm-Message-State: AOAM531mDAZqa7sz3l7uRnjGh7iAX1L3TnKsv7ImgMdArSn9KYnrsthF 2Ewtp04ulyMFOdP/iOimoOEXh6k5WZMmHuEh X-Google-Smtp-Source: ABdhPJy9SD0sNZwqjvHjM9AufQRXPR3twuAdcCEyG93tOd2nvXNc9PoZExEZOIxlcmnY0YHIDyX4NMuf6jEIFQl6 X-Received: from yosry.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:2327]) (user=yosryahmed job=sendgmr) by 2002:a63:e245:0:b0:3a7:dce1:64b1 with SMTP id y5-20020a63e245000000b003a7dce164b1mr771201pgj.67.1651263106339; Fri, 29 Apr 2022 13:11:46 -0700 (PDT) Date: Fri, 29 Apr 2022 20:11:28 +0000 In-Reply-To: <20220429201131.3397875-1-yosryahmed@google.com> Message-Id: <20220429201131.3397875-2-yosryahmed@google.com> Mime-Version: 1.0 References: <20220429201131.3397875-1-yosryahmed@google.com> X-Mailer: git-send-email 2.36.0.464.gb9c8b46e94-goog Subject: [PATCH v4 1/4] mm: add NR_SECONDARY_PAGETABLE to count secondary page table uses. From: Yosry Ahmed To: Tejun Heo , Johannes Weiner , Zefan Li , Marc Zyngier , James Morse , Alexandru Elisei , Suzuki K Poulose , Paolo Bonzini , Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , Andrew Morton , Michal Hocko , Roman Gushchin , Shakeel Butt , Oliver Upton Cc: cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, kvm@vger.kernel.org, linux-mm@kvack.org, Yosry Ahmed X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220429_131148_424718_F7E580D7 X-CRM114-Status: GOOD ( 15.84 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Add NR_SECONDARY_PAGETABLE stat to count secondary page table uses, e.g. KVM mmu. This provides more insights on the kernel memory used by a workload. This stat will be used by subsequent patches to count KVM mmu memory usage. Signed-off-by: Yosry Ahmed --- Documentation/admin-guide/cgroup-v2.rst | 5 +++++ Documentation/filesystems/proc.rst | 4 ++++ drivers/base/node.c | 2 ++ fs/proc/meminfo.c | 2 ++ include/linux/mmzone.h | 1 + mm/memcontrol.c | 1 + mm/page_alloc.c | 6 +++++- mm/vmstat.c | 1 + 8 files changed, 21 insertions(+), 1 deletion(-) diff --git a/Documentation/admin-guide/cgroup-v2.rst b/Documentation/admin-guide/cgroup-v2.rst index 69d7a6983f78..828cb6b6f918 100644 --- a/Documentation/admin-guide/cgroup-v2.rst +++ b/Documentation/admin-guide/cgroup-v2.rst @@ -1312,6 +1312,11 @@ PAGE_SIZE multiple when read back. pagetables Amount of memory allocated for page tables. + secondary_pagetables + Amount of memory allocated for secondary page tables, + this currently includes KVM mmu allocations on x86 + and arm64. + percpu (npn) Amount of memory used for storing per-cpu kernel data structures. diff --git a/Documentation/filesystems/proc.rst b/Documentation/filesystems/proc.rst index 061744c436d9..894d6317f3bd 100644 --- a/Documentation/filesystems/proc.rst +++ b/Documentation/filesystems/proc.rst @@ -973,6 +973,7 @@ You may not have all of these fields. SReclaimable: 159856 kB SUnreclaim: 124508 kB PageTables: 24448 kB + SecPageTables: 0 kB NFS_Unstable: 0 kB Bounce: 0 kB WritebackTmp: 0 kB @@ -1067,6 +1068,9 @@ SUnreclaim PageTables amount of memory dedicated to the lowest level of page tables. +SecPageTables + amount of memory dedicated to secondary page tables, this + currently includes KVM mmu allocations on x86 and arm64. NFS_Unstable Always zero. Previous counted pages which had been written to the server, but has not been committed to stable storage. diff --git a/drivers/base/node.c b/drivers/base/node.c index ec8bb24a5a22..9fe716832546 100644 --- a/drivers/base/node.c +++ b/drivers/base/node.c @@ -433,6 +433,7 @@ static ssize_t node_read_meminfo(struct device *dev, "Node %d ShadowCallStack:%8lu kB\n" #endif "Node %d PageTables: %8lu kB\n" + "Node %d SecPageTables: %8lu kB\n" "Node %d NFS_Unstable: %8lu kB\n" "Node %d Bounce: %8lu kB\n" "Node %d WritebackTmp: %8lu kB\n" @@ -459,6 +460,7 @@ static ssize_t node_read_meminfo(struct device *dev, nid, node_page_state(pgdat, NR_KERNEL_SCS_KB), #endif nid, K(node_page_state(pgdat, NR_PAGETABLE)), + nid, K(node_page_state(pgdat, NR_SECONDARY_PAGETABLE)), nid, 0UL, nid, K(sum_zone_node_page_state(nid, NR_BOUNCE)), nid, K(node_page_state(pgdat, NR_WRITEBACK_TEMP)), diff --git a/fs/proc/meminfo.c b/fs/proc/meminfo.c index 6fa761c9cc78..fad29024eb2e 100644 --- a/fs/proc/meminfo.c +++ b/fs/proc/meminfo.c @@ -108,6 +108,8 @@ static int meminfo_proc_show(struct seq_file *m, void *v) #endif show_val_kb(m, "PageTables: ", global_node_page_state(NR_PAGETABLE)); + show_val_kb(m, "SecPageTables: ", + global_node_page_state(NR_SECONDARY_PAGETABLE)); show_val_kb(m, "NFS_Unstable: ", 0); show_val_kb(m, "Bounce: ", diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index 962b14d403e8..35f57f2578c0 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -219,6 +219,7 @@ enum node_stat_item { NR_KERNEL_SCS_KB, /* measured in KiB */ #endif NR_PAGETABLE, /* used for pagetables */ + NR_SECONDARY_PAGETABLE, /* secondary pagetables, e.g. kvm shadow pagetables */ #ifdef CONFIG_SWAP NR_SWAPCACHE, #endif diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 725f76723220..89fbd1793960 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -1388,6 +1388,7 @@ static const struct memory_stat memory_stats[] = { { "kernel", MEMCG_KMEM }, { "kernel_stack", NR_KERNEL_STACK_KB }, { "pagetables", NR_PAGETABLE }, + { "secondary_pagetables", NR_SECONDARY_PAGETABLE }, { "percpu", MEMCG_PERCPU_B }, { "sock", MEMCG_SOCK }, { "vmalloc", MEMCG_VMALLOC }, diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 2db95780e003..96d00ae9d5c1 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -5932,7 +5932,8 @@ void show_free_areas(unsigned int filter, nodemask_t *nodemask) " active_file:%lu inactive_file:%lu isolated_file:%lu\n" " unevictable:%lu dirty:%lu writeback:%lu\n" " slab_reclaimable:%lu slab_unreclaimable:%lu\n" - " mapped:%lu shmem:%lu pagetables:%lu bounce:%lu\n" + " mapped:%lu shmem:%lu pagetables:%lu\n" + " secondary_pagetables:%lu bounce:%lu\n" " kernel_misc_reclaimable:%lu\n" " free:%lu free_pcp:%lu free_cma:%lu\n", global_node_page_state(NR_ACTIVE_ANON), @@ -5949,6 +5950,7 @@ void show_free_areas(unsigned int filter, nodemask_t *nodemask) global_node_page_state(NR_FILE_MAPPED), global_node_page_state(NR_SHMEM), global_node_page_state(NR_PAGETABLE), + global_node_page_state(NR_SECONDARY_PAGETABLE), global_zone_page_state(NR_BOUNCE), global_node_page_state(NR_KERNEL_MISC_RECLAIMABLE), global_zone_page_state(NR_FREE_PAGES), @@ -5982,6 +5984,7 @@ void show_free_areas(unsigned int filter, nodemask_t *nodemask) " shadow_call_stack:%lukB" #endif " pagetables:%lukB" + " secondary_pagetables:%lukB" " all_unreclaimable? %s" "\n", pgdat->node_id, @@ -6007,6 +6010,7 @@ void show_free_areas(unsigned int filter, nodemask_t *nodemask) node_page_state(pgdat, NR_KERNEL_SCS_KB), #endif K(node_page_state(pgdat, NR_PAGETABLE)), + K(node_page_state(pgdat, NR_SECONDARY_PAGETABLE)), pgdat->kswapd_failures >= MAX_RECLAIM_RETRIES ? "yes" : "no"); } diff --git a/mm/vmstat.c b/mm/vmstat.c index b75b1a64b54c..50bbec73809b 100644 --- a/mm/vmstat.c +++ b/mm/vmstat.c @@ -1240,6 +1240,7 @@ const char * const vmstat_text[] = { "nr_shadow_call_stack", #endif "nr_page_table_pages", + "nr_secondary_page_table_pages", #ifdef CONFIG_SWAP "nr_swapcached", #endif From patchwork Fri Apr 29 20:11:29 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yosry Ahmed X-Patchwork-Id: 12832703 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 394DFC433F5 for ; Fri, 29 Apr 2022 20:13:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=n8h+rCYele8EdLZvizyNFi88wKBst6iSOyay7cH9CXY=; b=XDg4AiwkJq6UzuwYaImkONTZoG scSKiWDbYvZhfXTbR3mu6JPqydnbogBDjRNQZFdPMSLZpMCHnaJqY0d0HjgfGGG9A1ZtMrfSbPSdt sUtJrh7aA/SVUWSho4sNF9BOEJXmmAzgaxh28EtnG8vozPzeik+9gHvSCGVaILCQS/pUFJ2OPuPLO NWthUV1hBWjuywPbjpIb5LlRhapKOgvs9syXlUXzETRSjdJC+/Ow1xEe6uuyLEUFKCLZAFitdmQ4a rvdi+20InyKdZLnE3q1Gw5CtPUytBlV1KzX4Y6HqQqIgWG9wYnwwCxFrFGxmdfS/xRJVSOwAYow7L x3MmLBGw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nkWyL-00CQUa-OT; Fri, 29 Apr 2022 20:12:14 +0000 Received: from mail-pl1-x64a.google.com ([2607:f8b0:4864:20::64a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1nkWxx-00CQIb-RK for linux-arm-kernel@lists.infradead.org; Fri, 29 Apr 2022 20:11:52 +0000 Received: by mail-pl1-x64a.google.com with SMTP id i16-20020a170902cf1000b001540b6a09e3so4668906plg.0 for ; Fri, 29 Apr 2022 13:11:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=HQRALrLqPLOKdX3/Qrvlj2304W4cY881g6B2XTfjhjE=; b=e1Gj1cFIx7qA9DjuG4CERdhHtwjjaMyqc0jE42K6DIa/HcAF7kLYbOh7A4zE3CfdpC MXLc5TLYF/85xAw4JqYk5Z+4VaQG+kEY5rcUmOxE6tic9A7AB/r3soW+wIKvxsx6s6sb dwf/GudGOtBTMTSquKo6sp/pI7w4IG3EJ9Ed+c2JR8p72budxuezZoM2UOclpII5lS61 jLjhhkg61TxTY5Ps6VReS8PPdn/lF7OSnXOocjCyRcBRNrRTP7mhwtpcGmbPstkKa1Z3 UjsKnboQEBlCrWAyY27GIJsYPsHmlwb5DzS1Loav7ERt/ViNzQEWA6OAJ7ojRdb9jzbP ysnA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=HQRALrLqPLOKdX3/Qrvlj2304W4cY881g6B2XTfjhjE=; b=6liMmdVjcoqv6G9jbBxe5UdUQqC3ANy/QoyH5EX+GHhaPfWm+tE6tuLqcUVDDIYUbA Sgq6EoB3lidpulEIw4rCGEj3phLOKbGLNdkWgFoOQHtNhS1f2dWOMvB5LC1EB2opwG2S XOW/s2Td+FvjApxMNPkvnWBHrKhnEgRs5fUxyB92nzXjCnKz2GInkOolJPqEMLtB0zJQ KqDoqdVPtnzQAODj/UjqtRxj7DMBZsNG3u9nRW1k/133iraEJ5bxJJ0WJGJs0y3Isvtv lpAKGXOL0AyD1okgMvut/W1br6Bjb6YHnznDxwYCIUoQjm36vGlnlJ8Dpu85N3lazYm4 fqCQ== X-Gm-Message-State: AOAM533Z9aFFRzeb7AEgJK4u2lhIOea4cVjOFDXPsu2UJQ37CUVga+D1 3bBoMUbtbBFHsouqV1BwFYzSvro89BT+fo0V X-Google-Smtp-Source: ABdhPJyFiTvIOFIZYo+38It2ytGBtkvfsnlq74KMff/yRu6trFEODoofBTp8U5qgZN1B+vJQKJMiROSbmndcmE1m X-Received: from yosry.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:2327]) (user=yosryahmed job=sendgmr) by 2002:a62:87cd:0:b0:50d:bf61:3de9 with SMTP id i196-20020a6287cd000000b0050dbf613de9mr906362pfe.16.1651263107958; Fri, 29 Apr 2022 13:11:47 -0700 (PDT) Date: Fri, 29 Apr 2022 20:11:29 +0000 In-Reply-To: <20220429201131.3397875-1-yosryahmed@google.com> Message-Id: <20220429201131.3397875-3-yosryahmed@google.com> Mime-Version: 1.0 References: <20220429201131.3397875-1-yosryahmed@google.com> X-Mailer: git-send-email 2.36.0.464.gb9c8b46e94-goog Subject: [PATCH v4 2/4] KVM: mmu: add a helper to account memory used by KVM mmu. From: Yosry Ahmed To: Tejun Heo , Johannes Weiner , Zefan Li , Marc Zyngier , James Morse , Alexandru Elisei , Suzuki K Poulose , Paolo Bonzini , Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , Andrew Morton , Michal Hocko , Roman Gushchin , Shakeel Butt , Oliver Upton Cc: cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, kvm@vger.kernel.org, linux-mm@kvack.org, Yosry Ahmed X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220429_131149_933297_E38FAD01 X-CRM114-Status: GOOD ( 10.15 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Add a helper to account pages used by KVM for page tables in secondary pagetable stats. This function will be used by subsequent patches in different archs. Signed-off-by: Yosry Ahmed --- include/linux/kvm_host.h | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 252ee4a61b58..54cc4634053c 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -2221,6 +2221,15 @@ static inline void kvm_handle_signal_exit(struct kvm_vcpu *vcpu) } #endif /* CONFIG_KVM_XFER_TO_GUEST_WORK */ +/* + * If nr > 1, we assume virt is the address of the first page of a block of + * pages that were allocated together (i.e accounted together). + */ +static inline void kvm_account_pgtable_pages(void *virt, int nr) +{ + mod_lruvec_page_state(virt_to_page(virt), NR_SECONDARY_PAGETABLE, nr); +} + /* * This defines how many reserved entries we want to keep before we * kick the vcpu to the userspace to avoid dirty ring full. This From patchwork Fri Apr 29 20:11:30 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yosry Ahmed X-Patchwork-Id: 12832704 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 2A917C433EF for ; Fri, 29 Apr 2022 20:13:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=R5zqlNnZQYZiCfY5gPdZ66BXZUIHsA56X3m3voZIhxQ=; b=WQxkhDJSEkAUPqNRbnTbwPU6xy YO1WeGAXCYK62xf/mqpwq+WscfbJU1uqGD2Z17O9/sf5inU6af+0EJAq1TlqnkDHo3PXOlhYfCxd1 VQzm60MKvE8epLpaK/16YfOf/y0ANi/GGPaqsAfZDMBBgSprsI5DLhRR8baCeWKwBlGF7hEw9YCoP JHlnPOQ6ZUtozEuOe6bW0B8cKGjp10QmZNdfYEp1gIAzrDjjWEOXAHAfXwtRnZUMz8nas7ySV07Cz TrLCPTCLb80VOK24u5wzO3kBOTcvtvXlQUuHCcHGERKOsVGGbAUz3zdyNXN7kl1rKzPJE0XWHJUts IwMgDWjw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nkWyZ-00CQaI-M0; Fri, 29 Apr 2022 20:12:27 +0000 Received: from mail-pg1-x549.google.com ([2607:f8b0:4864:20::549]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1nkWxz-00CQJg-5j for linux-arm-kernel@lists.infradead.org; Fri, 29 Apr 2022 20:11:53 +0000 Received: by mail-pg1-x549.google.com with SMTP id o5-20020a639205000000b003ab492e038dso4211200pgd.12 for ; Fri, 29 Apr 2022 13:11:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=WJSDpsc2WvoBFv9mTA61a5Umprsh/icO78ayN0cr9Bs=; b=XK8m1lEPYyGBc9UNev+UIJgN/P7IC1BvuvVMJTbn/r6py0F3Lw6Fv3q8ILFyBYMVLZ MnvcTtwLPs9qLTOzDrfbzBbzcFsKgFcIpkP1iDnH3nigu5jcxcjT3SQG6HmSeBeeWNXh nWqMIpSxa67WhLX5qrBEPIdWZClzZXZCQmTuCwlOQ52XSMIiZurYcyKlAE100i/H4G+X p0yV6mz7huTMa+l9U35w3XcRl/r9Xa8/Qmqq531nkOdpeqQgz1iv9qqZRPytgVAUNefO TLvub9at+LkqfyoK37ZiP+2KwOj6XQdMrsV1bz+J2e5sV3w4mKrIKmrOwHFLFZvOJ7UN qxsg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=WJSDpsc2WvoBFv9mTA61a5Umprsh/icO78ayN0cr9Bs=; b=2GIhWzt+0inlmgmdLWCoPzbu1KmX60Q18QhDc95KTiXdjQTWNsFFJr+hSlv44pmJkP mH/VmqPHnwzuqMrkE/J8u3KvjQVuK2324T1oIZKoPnl0IHD/Ad2mox3WMIC3IMN5R/ap TXJrm73RNDiUCclaKDS1cSeSCVO7jsZXwzroNNYgKFwRbb/ddRScCbcACSUQwvo0b0dg rWvvvcbKnz1uRRA36XKKlbbJpBm26OGXlPsexywz8OfKI6b6dK88TSI8eiUl/VrwNCFH Swr9PATiHkI3laLbHjw6w3abXXLywZivZWdMJXu0cKhqZVDdP+t7S27+gqF/WGR6FrzP w67Q== X-Gm-Message-State: AOAM533JsGAaeIkA+ZNvTTenJL7HW5z+ayXKY08DWHcugQenp7L6BrIk fOK/kQLd6N+n+iXuk1cyEbuk4yoUGO7nXIf6 X-Google-Smtp-Source: ABdhPJy8M95IzubrsL7x95NvP2WiBYnG038db0b/2iw0dRRfI3FAVtvV34XGoZ0NsslM2ywf/tGMkw5/l/vSYvKs X-Received: from yosry.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:2327]) (user=yosryahmed job=sendgmr) by 2002:a63:5154:0:b0:39d:5e18:9d98 with SMTP id r20-20020a635154000000b0039d5e189d98mr761530pgl.145.1651263109293; Fri, 29 Apr 2022 13:11:49 -0700 (PDT) Date: Fri, 29 Apr 2022 20:11:30 +0000 In-Reply-To: <20220429201131.3397875-1-yosryahmed@google.com> Message-Id: <20220429201131.3397875-4-yosryahmed@google.com> Mime-Version: 1.0 References: <20220429201131.3397875-1-yosryahmed@google.com> X-Mailer: git-send-email 2.36.0.464.gb9c8b46e94-goog Subject: [PATCH v4 3/4] KVM: x86/mmu: count KVM mmu usage in secondary pagetable stats. From: Yosry Ahmed To: Tejun Heo , Johannes Weiner , Zefan Li , Marc Zyngier , James Morse , Alexandru Elisei , Suzuki K Poulose , Paolo Bonzini , Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , Andrew Morton , Michal Hocko , Roman Gushchin , Shakeel Butt , Oliver Upton Cc: cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, kvm@vger.kernel.org, linux-mm@kvack.org, Yosry Ahmed X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220429_131151_262487_068C9C05 X-CRM114-Status: GOOD ( 14.08 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Count the pages used by KVM mmu on x86 for in secondary pagetable stats. For the legacy mmu, accounting pagetable stats is combined KVM's existing for mmu pages in newly introduced kvm_[un]account_mmu_page() helpers. For tdp mmu, introduce new tdp_[un]account_mmu_page() helpers. That combines accounting pagetable stats with the tdp_mmu_pages counter accounting. tdp_mmu_pages counter introduced in this series [1]. This patch was rebased on top of the first two patches in that series. [1]https://lore.kernel.org/lkml/20220401063636.2414200-1-mizhang@google.com/ Signed-off-by: Yosry Ahmed --- arch/x86/kvm/mmu/mmu.c | 16 ++++++++++++++-- arch/x86/kvm/mmu/tdp_mmu.c | 16 ++++++++++++++-- 2 files changed, 28 insertions(+), 4 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 78d8e1d8fb99..e5b0e826445d 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -1679,6 +1679,18 @@ static inline void kvm_mod_used_mmu_pages(struct kvm *kvm, long nr) percpu_counter_add(&kvm_total_used_mmu_pages, nr); } +static void kvm_account_mmu_page(struct kvm *kvm, struct kvm_mmu_page *sp) +{ + kvm_mod_used_mmu_pages(kvm, +1); + kvm_account_pgtable_pages((void *)sp->spt, +1); +} + +static void kvm_unaccount_mmu_page(struct kvm *kvm, struct kvm_mmu_page *sp) +{ + kvm_mod_used_mmu_pages(kvm, -1); + kvm_account_pgtable_pages((void *)sp->spt, -1); +} + static void kvm_mmu_free_page(struct kvm_mmu_page *sp) { MMU_WARN_ON(!is_empty_shadow_page(sp->spt)); @@ -1734,7 +1746,7 @@ static struct kvm_mmu_page *kvm_mmu_alloc_page(struct kvm_vcpu *vcpu, int direct */ sp->mmu_valid_gen = vcpu->kvm->arch.mmu_valid_gen; list_add(&sp->link, &vcpu->kvm->arch.active_mmu_pages); - kvm_mod_used_mmu_pages(vcpu->kvm, +1); + kvm_account_mmu_page(vcpu->kvm, sp); return sp; } @@ -2363,7 +2375,7 @@ static bool __kvm_mmu_prepare_zap_page(struct kvm *kvm, list_add(&sp->link, invalid_list); else list_move(&sp->link, invalid_list); - kvm_mod_used_mmu_pages(kvm, -1); + kvm_unaccount_mmu_page(kvm, sp); } else { /* * Remove the active root from the active page list, the root diff --git a/arch/x86/kvm/mmu/tdp_mmu.c b/arch/x86/kvm/mmu/tdp_mmu.c index 3456277ade18..6295c4da5dee 100644 --- a/arch/x86/kvm/mmu/tdp_mmu.c +++ b/arch/x86/kvm/mmu/tdp_mmu.c @@ -371,6 +371,18 @@ static void handle_changed_spte_dirty_log(struct kvm *kvm, int as_id, gfn_t gfn, } } +static void tdp_account_mmu_page(struct kvm *kvm, struct kvm_mmu_page *sp) +{ + atomic64_inc(&kvm->arch.tdp_mmu_pages); + kvm_account_pgtable_pages((void *)sp->spt, +1); +} + +static void tdp_unaccount_mmu_page(struct kvm *kvm, struct kvm_mmu_page *sp) +{ + atomic64_dec(&kvm->arch.tdp_mmu_pages); + kvm_account_pgtable_pages((void *)sp->spt, -1); +} + /** * tdp_mmu_unlink_sp() - Remove a shadow page from the list of used pages * @@ -383,7 +395,7 @@ static void handle_changed_spte_dirty_log(struct kvm *kvm, int as_id, gfn_t gfn, static void tdp_mmu_unlink_sp(struct kvm *kvm, struct kvm_mmu_page *sp, bool shared) { - atomic64_dec(&kvm->arch.tdp_mmu_pages); + tdp_unaccount_mmu_page(kvm, sp); if (!sp->lpage_disallowed) return; @@ -1121,7 +1133,7 @@ static int tdp_mmu_link_sp(struct kvm *kvm, struct tdp_iter *iter, tdp_mmu_set_spte(kvm, iter, spte); } - atomic64_inc(&kvm->arch.tdp_mmu_pages); + tdp_account_mmu_page(kvm, sp); return 0; } From patchwork Fri Apr 29 20:11:31 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yosry Ahmed X-Patchwork-Id: 12832705 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E51B0C433F5 for ; Fri, 29 Apr 2022 20:13:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=+sv9eGvYyhTr4jKYXz+lbuJNtoJW5n5vU9KnKGDgChQ=; b=2YqQk8yZWuZAAA/zEbBcxUABQ2 BIXbcolFWxs88dG9sqHY1tooUvgB9hEEyxDK6do0v07i00tCCJhIcxLVvdQP/Tgp2E568NTWW9FhY dytT+k+Qbr5vRA/2A1qBqdDXw2I+Q2qVpDACi7aLOnsJfEwAAeTZyK3vYYIjvZJKgR8cWT7bazPk2 sucw/+hwBc0qTcEKM1DqkA02g2Ih2SBfMZo2Z3/Nl0gLBS075zbkaWLWlbdanZh9UvqZEZ1qjsQEt auH2dKzBN/QycCm8TFVsx2hEJcEXQaRCSi5cCKA74HB4bD8OmUoh98sPpTTHsRVtFbXiSXGbaFLBJ MqewKf9A==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nkWys-00CQiL-5d; Fri, 29 Apr 2022 20:12:46 +0000 Received: from mail-pl1-x649.google.com ([2607:f8b0:4864:20::649]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1nkWyJ-00CQKb-LA for linux-arm-kernel@lists.infradead.org; Fri, 29 Apr 2022 20:12:13 +0000 Received: by mail-pl1-x649.google.com with SMTP id i10-20020a1709026aca00b00158f14b4f2fso4662362plt.2 for ; Fri, 29 Apr 2022 13:11:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=S3BBsZ5bvk4uzoCkIECYzRGc2TYJKfjLopd3GTSDAKA=; b=jyU87f8rJUN/0hbg4f3kdULDqmUiyc07Xb66s8cBmC8eSW5afxPVORIXgO/XcbFugu /GO6FtfJ4ZPnIibnkDPCd2vWdazZzlWxhMF1+MfgeVHE4EB8soDI4WiD7pxUHc0XysjZ dz0NPsguoIx0yCM0GzW+6AJFD3eMtBn5IljqCyl24S9cni2H7/qF97uyykmSQ8zzR3qj 0ctQn1mlUvVg2EZJl93vg7f3fTcTbPn8ws29rj4RPyM04s2wTJC2/9P5OL5vwTLWBxg5 QDDNee7lmxpZqa2y/Fn8/pPEocI38Oecy4dHL012AY61PD0PXJ/gQDtUls02A2N8uheX E/zA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=S3BBsZ5bvk4uzoCkIECYzRGc2TYJKfjLopd3GTSDAKA=; b=1LFyGl2Bf/qy0gR9DYm3r7u0L78gzHY7+2iMRlX2sNNHvpHL4seleDeALlvZtnOcbC Zco8q7VRbl4Rae68+Nej21z2rhj+O6+XgshPcEj3670HCKgx5eXrwOGlmcPYhVROc1Ry rVgfvuYBpHuYr9tK3jhTyF4xCPU22IGpBSkf/rCQQnbVjhYdxOjGIG1VfjcWHawd5ONI fQHbfHnZuo32fgdfYobjk+So8t0y4y4iL5HBSi/v4NtFHUmymxJTYvFKUYWG0Jqnzteb RHzgRBZBay/FPqb4FgKJ4YMewv5AlV2jJPwPwUx7tnBchJ8H2tUvEYeyKgDV6oSJomVN Qxdg== X-Gm-Message-State: AOAM530rGfbdy7j2hE53O+9qwDmee13mAQK2DPWErhWfSkQ8LM9sVM/1 VrEWpmKYCynXuEJdLyh9Zhsa6+4yoRilKpQW X-Google-Smtp-Source: ABdhPJxOmVuJGvLk2Hjh+EQ9JUFzd7qmz4ZeddVRcuaWheP9plljXX7Xz37I8NLuX36SAI5J9foHUODTgkzaf7D4 X-Received: from yosry.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:2327]) (user=yosryahmed job=sendgmr) by 2002:a05:6a00:1411:b0:4fd:e594:fac0 with SMTP id l17-20020a056a00141100b004fde594fac0mr831484pfu.79.1651263110818; Fri, 29 Apr 2022 13:11:50 -0700 (PDT) Date: Fri, 29 Apr 2022 20:11:31 +0000 In-Reply-To: <20220429201131.3397875-1-yosryahmed@google.com> Message-Id: <20220429201131.3397875-5-yosryahmed@google.com> Mime-Version: 1.0 References: <20220429201131.3397875-1-yosryahmed@google.com> X-Mailer: git-send-email 2.36.0.464.gb9c8b46e94-goog Subject: [PATCH v4 4/4] KVM: arm64/mmu: count KVM s2 mmu usage in secondary pagetable stats From: Yosry Ahmed To: Tejun Heo , Johannes Weiner , Zefan Li , Marc Zyngier , James Morse , Alexandru Elisei , Suzuki K Poulose , Paolo Bonzini , Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , Andrew Morton , Michal Hocko , Roman Gushchin , Shakeel Butt , Oliver Upton Cc: cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, kvm@vger.kernel.org, linux-mm@kvack.org, Yosry Ahmed X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220429_131211_716522_04E023D6 X-CRM114-Status: GOOD ( 12.71 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Count the pages used by KVM in arm64 for stage2 mmu in secondary pagetable stats. Signed-off-by: Yosry Ahmed --- arch/arm64/kvm/mmu.c | 35 +++++++++++++++++++++++++++++++---- 1 file changed, 31 insertions(+), 4 deletions(-) diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c index 53ae2c0640bc..fc5030307cce 100644 --- a/arch/arm64/kvm/mmu.c +++ b/arch/arm64/kvm/mmu.c @@ -92,9 +92,13 @@ static bool kvm_is_device_pfn(unsigned long pfn) static void *stage2_memcache_zalloc_page(void *arg) { struct kvm_mmu_memory_cache *mc = arg; + void *virt; /* Allocated with __GFP_ZERO, so no need to zero */ - return kvm_mmu_memory_cache_alloc(mc); + virt = kvm_mmu_memory_cache_alloc(mc); + if (virt) + kvm_account_pgtable_pages(virt, +1); + return virt; } static void *kvm_host_zalloc_pages_exact(size_t size) @@ -102,6 +106,20 @@ static void *kvm_host_zalloc_pages_exact(size_t size) return alloc_pages_exact(size, GFP_KERNEL_ACCOUNT | __GFP_ZERO); } +static void *kvm_s2_zalloc_pages_exact(size_t size) +{ + void *virt = kvm_host_zalloc_pages_exact(size); + if (virt) + kvm_account_pgtable_pages(virt, +(size >> PAGE_SHIFT)); + return virt; +} + +static void kvm_s2_free_pages_exact(void *virt, size_t size) +{ + kvm_account_pgtable_pages(virt, -(size >> PAGE_SHIFT)); + free_pages_exact(virt, size); +} + static void kvm_host_get_page(void *addr) { get_page(virt_to_page(addr)); @@ -112,6 +130,15 @@ static void kvm_host_put_page(void *addr) put_page(virt_to_page(addr)); } +static void kvm_s2_put_page(void *addr) +{ + struct page *p = virt_to_page(addr); + /* Dropping last refcount, the page will be freed */ + if (page_count(p) == 1) + kvm_account_pgtable_pages(addr, -1); + put_page(p); +} + static int kvm_host_page_count(void *addr) { return page_count(virt_to_page(addr)); @@ -603,10 +630,10 @@ static int get_user_mapping_size(struct kvm *kvm, u64 addr) static struct kvm_pgtable_mm_ops kvm_s2_mm_ops = { .zalloc_page = stage2_memcache_zalloc_page, - .zalloc_pages_exact = kvm_host_zalloc_pages_exact, - .free_pages_exact = free_pages_exact, + .zalloc_pages_exact = kvm_s2_zalloc_pages_exact, + .free_pages_exact = kvm_s2_free_pages_exact, .get_page = kvm_host_get_page, - .put_page = kvm_host_put_page, + .put_page = kvm_s2_put_page, .page_count = kvm_host_page_count, .phys_to_virt = kvm_host_va, .virt_to_phys = kvm_host_pa,