From patchwork Sun Dec 6 10:14:40 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 11953857 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 88901C1B0D8 for ; Sun, 6 Dec 2020 10:18:00 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6C9C5230FA for ; Sun, 6 Dec 2020 10:18:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726642AbgLFKRw (ORCPT ); Sun, 6 Dec 2020 05:17:52 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34538 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726873AbgLFKRk (ORCPT ); Sun, 6 Dec 2020 05:17:40 -0500 Received: from mail-pf1-x442.google.com (mail-pf1-x442.google.com [IPv6:2607:f8b0:4864:20::442]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 439D1C061A52 for ; Sun, 6 Dec 2020 02:17:00 -0800 (PST) Received: by mail-pf1-x442.google.com with SMTP id c79so7019663pfc.2 for ; Sun, 06 Dec 2020 02:17:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=JLGZC7sv5nAUpuhCShqzZAhxLbvYoPKLszNPrb+RhvY=; b=sDQL9F0PUFel7IRE81/3eSpUKyMdwDgTZi3UBN+42b+bU2PNE/t3ymmlZSkl6G0zUE NSCG35LdEVF++dP/WfDe2t2uqtwSFpsGuaN57CFdrOLHb4WjfZ3jys2v2GYv1slCoxpV lHGI0MzqMCRhrpIXGMhucScdPb77XwS1gVQ8DA3u+zB/gPDofVk7OR0MNcSQkNxlPfaJ 1RyynIXs88y3N+8Tuq6M1bZWNQoh5e5Y9f2FauDg2E2dRqtshDAZubBtA0H7xFytfcsr bP6zs0AWlZzZkTE8tZ0EnxPaZLjkWuzOMlhnw7qO6H+KSceypFyZdtq4GtFb8oTd85Yc RNxQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=JLGZC7sv5nAUpuhCShqzZAhxLbvYoPKLszNPrb+RhvY=; b=F6EXCb1Pk5xeCGkzNKHy+V3jNN1OHRviU/6mX5lKYdKJo8dgeN/kwVDyL25xUcoKiA xXYTMbuHZLF6M+ivbKL7+B9TlHJOHr/Tjc78zMIWMPiHYTPEUJi5zjBIQh6kryeUyfG9 lISrMt7QMdF2N+zcJWRAU7mLZobCXYzt+OOiYnE2bW1Urca5WpIfuZ3TRIs2JO0OBAkl ahsSKDCwjXxGv/MwVW/+Yeig2M2IsdEjtfM6/+bJXrDvMTpqvyDf7ev5hFgs+23gldR/ Wn13UrqT7ctd01/cd1k1oxkCP5HrTF6mZwuIL/H4n4joHB5DKvtVhaosLZt7IU8z3Mz5 JXAw== X-Gm-Message-State: AOAM532SUNZ/k6Y6Y/1NjNHJPOr9+LR74m/z+0d9KzAaq/Qj68wiB/gZ StUHijD43QyQKJtNS0jZtin2eA== X-Google-Smtp-Source: ABdhPJwrEym39f3aZ6PyYuujbHaEjIbhRNOiqu9p7TRph3J6zvXaoYG5vwozGaIuyksCOLbrQRARnQ== X-Received: by 2002:a62:88c3:0:b029:18c:3203:efb7 with SMTP id l186-20020a6288c30000b029018c3203efb7mr11447624pfd.33.1607249819828; Sun, 06 Dec 2020 02:16:59 -0800 (PST) Received: from localhost.localdomain ([103.136.221.70]) by smtp.gmail.com with ESMTPSA id g16sm10337657pfb.201.2020.12.06.02.16.51 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 06 Dec 2020 02:16:59 -0800 (PST) From: Muchun Song To: gregkh@linuxfoundation.org, rafael@kernel.org, adobriyan@gmail.com, akpm@linux-foundation.org, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, hughd@google.com, will@kernel.org, guro@fb.com, rppt@kernel.org, tglx@linutronix.de, esyr@redhat.com, peterx@redhat.com, krisman@collabora.com, surenb@google.com, avagin@openvz.org, elver@google.com, rdunlap@infradead.org, iamjoonsoo.kim@lge.com Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, Muchun Song Subject: [RESEND PATCH v2 01/12] mm: memcontrol: fix NR_ANON_THPS account Date: Sun, 6 Dec 2020 18:14:40 +0800 Message-Id: <20201206101451.14706-2-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20201206101451.14706-1-songmuchun@bytedance.com> References: <20201206101451.14706-1-songmuchun@bytedance.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org The unit of NR_ANON_THPS is HPAGE_PMD_NR already. So it should inc/dec by one rather than nr_pages. Fixes: 468c398233da ("mm: memcontrol: switch to native NR_ANON_THPS counter") Signed-off-by: Muchun Song Acked-by: Michal Hocko Acked-by: Johannes Weiner --- mm/memcontrol.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 22d9bd688d6d..695dedf8687a 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -5634,10 +5634,8 @@ static int mem_cgroup_move_account(struct page *page, __mod_lruvec_state(from_vec, NR_ANON_MAPPED, -nr_pages); __mod_lruvec_state(to_vec, NR_ANON_MAPPED, nr_pages); if (PageTransHuge(page)) { - __mod_lruvec_state(from_vec, NR_ANON_THPS, - -nr_pages); - __mod_lruvec_state(to_vec, NR_ANON_THPS, - nr_pages); + __dec_lruvec_state(from_vec, NR_ANON_THPS); + __inc_lruvec_state(to_vec, NR_ANON_THPS); } } From patchwork Sun Dec 6 10:14:41 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 11953859 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BAE1BC433FE for ; Sun, 6 Dec 2020 10:18:37 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8A0A6230FA for ; Sun, 6 Dec 2020 10:18:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726351AbgLFKSS (ORCPT ); Sun, 6 Dec 2020 05:18:18 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34636 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726506AbgLFKSR (ORCPT ); Sun, 6 Dec 2020 05:18:17 -0500 Received: from mail-pj1-x1043.google.com (mail-pj1-x1043.google.com [IPv6:2607:f8b0:4864:20::1043]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 931F8C061A55 for ; Sun, 6 Dec 2020 02:17:08 -0800 (PST) Received: by mail-pj1-x1043.google.com with SMTP id z12so5667996pjn.1 for ; Sun, 06 Dec 2020 02:17:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=tb1F00/nSlox2NKAPVic2CylQOjgm2ke+JSF79slOJI=; b=uAhtAkvDbhKkFqddHAKBOhza/mhkmYbQ1gO6u++TYaJcsXVMyP3DxJmUjnpMeUG3SW XaMHSAVvWel0CQp+XAMCU1fy5kVW4g/nQk1Ljm3WTrKzYFBut8uMBDb529RBGkfk+A2v M/rf2Vw6BzysfA+PgBb0lHWSxh7zZIVcUHPgsevcH+FXhSayuBVPKaI2/sRJUjVwZ3sh mDVTvJagBELE8ENh8vCH8AgbeDky1F6PQf5oWry/i1XGz24pO3nc2eO7pWTPkntDwSlm P1bSiP4lHuYOVRc4yTGX5e4058PUKvlIZS8LbLvOAgCUcIs6oHfH+7dSCl4EgNqeLuj7 eYVQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=tb1F00/nSlox2NKAPVic2CylQOjgm2ke+JSF79slOJI=; b=O8VH2daJZpGYgEK+a9kPUeb0KaOXJuif1heT/X3yVm4YWnI1zRrMZeuSRWaU3ENlI/ 5IN3GOURgjjGU/Trsmn/a/cBbhZ2BZFNXzBh380LMMrQfrse4RtvnTfi0k1S8a91D750 uyuK0kAcIjlndgr1c0sMdGAkk1ST4YjEqo2tv8UOcIit4ZorcEYLGl1dIP7wJ4kE/CjU nLQpyLQoR+KckJo6s95Ij9h4Bmiqq8tbsPmqG23ljTIZ5sg2go85fkVss4RDrQbJini8 Onk7pN9LipZKHCi4NmHyX7jYRxzeBGgsecDVC++8TThWdCxdfNFHAKAX/dMWTQOHh45p 9VHg== X-Gm-Message-State: AOAM533O+Cyn5aWGoDRSlNUfo8/bV/3YHqKLha4fYF82gIZ47xdokisF UNmfuApSyaFqPkWpIjkGxjgvSw== X-Google-Smtp-Source: ABdhPJzEBo93k11SoyRC/DlZb7e56oJgos7Jvh1hQZakYwMm4rGixAiJhZdQfEqnglOZ1ymwlJS7CA== X-Received: by 2002:a17:90a:bd0b:: with SMTP id y11mr8078590pjr.25.1607249828160; Sun, 06 Dec 2020 02:17:08 -0800 (PST) Received: from localhost.localdomain ([103.136.221.70]) by smtp.gmail.com with ESMTPSA id g16sm10337657pfb.201.2020.12.06.02.17.00 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 06 Dec 2020 02:17:07 -0800 (PST) From: Muchun Song To: gregkh@linuxfoundation.org, rafael@kernel.org, adobriyan@gmail.com, akpm@linux-foundation.org, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, hughd@google.com, will@kernel.org, guro@fb.com, rppt@kernel.org, tglx@linutronix.de, esyr@redhat.com, peterx@redhat.com, krisman@collabora.com, surenb@google.com, avagin@openvz.org, elver@google.com, rdunlap@infradead.org, iamjoonsoo.kim@lge.com Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, Muchun Song Subject: [RESEND PATCH v2 02/12] mm: memcontrol: convert NR_ANON_THPS account to pages Date: Sun, 6 Dec 2020 18:14:41 +0800 Message-Id: <20201206101451.14706-3-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20201206101451.14706-1-songmuchun@bytedance.com> References: <20201206101451.14706-1-songmuchun@bytedance.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Convert the NR_ANON_THPS account to pages. Signed-off-by: Muchun Song --- drivers/base/node.c | 3 +-- fs/proc/meminfo.c | 2 +- mm/huge_memory.c | 3 ++- mm/memcontrol.c | 20 ++++++-------------- mm/page_alloc.c | 2 +- mm/rmap.c | 7 ++++--- 6 files changed, 15 insertions(+), 22 deletions(-) diff --git a/drivers/base/node.c b/drivers/base/node.c index 6ffa470e2984..7ebe4c2f64d1 100644 --- a/drivers/base/node.c +++ b/drivers/base/node.c @@ -461,8 +461,7 @@ static ssize_t node_read_meminfo(struct device *dev, nid, K(sunreclaimable) #ifdef CONFIG_TRANSPARENT_HUGEPAGE , - nid, K(node_page_state(pgdat, NR_ANON_THPS) * - HPAGE_PMD_NR), + nid, K(node_page_state(pgdat, NR_ANON_THPS)), nid, K(node_page_state(pgdat, NR_SHMEM_THPS) * HPAGE_PMD_NR), nid, K(node_page_state(pgdat, NR_SHMEM_PMDMAPPED) * diff --git a/fs/proc/meminfo.c b/fs/proc/meminfo.c index 887a5532e449..1f7e1945c313 100644 --- a/fs/proc/meminfo.c +++ b/fs/proc/meminfo.c @@ -129,7 +129,7 @@ static int meminfo_proc_show(struct seq_file *m, void *v) #ifdef CONFIG_TRANSPARENT_HUGEPAGE show_val_kb(m, "AnonHugePages: ", - global_node_page_state(NR_ANON_THPS) * HPAGE_PMD_NR); + global_node_page_state(NR_ANON_THPS)); show_val_kb(m, "ShmemHugePages: ", global_node_page_state(NR_SHMEM_THPS) * HPAGE_PMD_NR); show_val_kb(m, "ShmemPmdMapped: ", diff --git a/mm/huge_memory.c b/mm/huge_memory.c index c94dfc0cc1a3..1a13e1dab3ec 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -2178,7 +2178,8 @@ static void __split_huge_pmd_locked(struct vm_area_struct *vma, pmd_t *pmd, lock_page_memcg(page); if (atomic_add_negative(-1, compound_mapcount_ptr(page))) { /* Last compound_mapcount is gone. */ - __dec_lruvec_page_state(page, NR_ANON_THPS); + __mod_lruvec_page_state(page, NR_ANON_THPS, + -HPAGE_PMD_NR); if (TestClearPageDoubleMap(page)) { /* No need in mapcount reference anymore */ for (i = 0; i < HPAGE_PMD_NR; i++) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 695dedf8687a..2700c1db5a1a 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -1513,7 +1513,7 @@ static struct memory_stat memory_stats[] = { * on some architectures, the macro of HPAGE_PMD_SIZE is not * constant(e.g. powerpc). */ - { "anon_thp", 0, NR_ANON_THPS }, + { "anon_thp", PAGE_SIZE, NR_ANON_THPS }, { "file_thp", 0, NR_FILE_THPS }, { "shmem_thp", 0, NR_SHMEM_THPS }, #endif @@ -1546,8 +1546,7 @@ static int __init memory_stats_init(void) for (i = 0; i < ARRAY_SIZE(memory_stats); i++) { #ifdef CONFIG_TRANSPARENT_HUGEPAGE - if (memory_stats[i].idx == NR_ANON_THPS || - memory_stats[i].idx == NR_FILE_THPS || + if (memory_stats[i].idx == NR_FILE_THPS || memory_stats[i].idx == NR_SHMEM_THPS) memory_stats[i].ratio = HPAGE_PMD_SIZE; #endif @@ -4069,10 +4068,6 @@ static int memcg_stat_show(struct seq_file *m, void *v) if (memcg1_stats[i] == MEMCG_SWAP && !do_memsw_account()) continue; nr = memcg_page_state_local(memcg, memcg1_stats[i]); -#ifdef CONFIG_TRANSPARENT_HUGEPAGE - if (memcg1_stats[i] == NR_ANON_THPS) - nr *= HPAGE_PMD_NR; -#endif seq_printf(m, "%s %lu\n", memcg1_stat_names[i], nr * PAGE_SIZE); } @@ -4103,10 +4098,6 @@ static int memcg_stat_show(struct seq_file *m, void *v) if (memcg1_stats[i] == MEMCG_SWAP && !do_memsw_account()) continue; nr = memcg_page_state(memcg, memcg1_stats[i]); -#ifdef CONFIG_TRANSPARENT_HUGEPAGE - if (memcg1_stats[i] == NR_ANON_THPS) - nr *= HPAGE_PMD_NR; -#endif seq_printf(m, "total_%s %llu\n", memcg1_stat_names[i], (u64)nr * PAGE_SIZE); } @@ -5634,10 +5625,11 @@ static int mem_cgroup_move_account(struct page *page, __mod_lruvec_state(from_vec, NR_ANON_MAPPED, -nr_pages); __mod_lruvec_state(to_vec, NR_ANON_MAPPED, nr_pages); if (PageTransHuge(page)) { - __dec_lruvec_state(from_vec, NR_ANON_THPS); - __inc_lruvec_state(to_vec, NR_ANON_THPS); + __mod_lruvec_state(from_vec, NR_ANON_THPS, + -nr_pages); + __mod_lruvec_state(to_vec, NR_ANON_THPS, + nr_pages); } - } } else { __mod_lruvec_state(from_vec, NR_FILE_PAGES, -nr_pages); diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 56e603eea1dd..f97ca98d361f 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -5570,7 +5570,7 @@ void show_free_areas(unsigned int filter, nodemask_t *nodemask) K(node_page_state(pgdat, NR_SHMEM_THPS) * HPAGE_PMD_NR), K(node_page_state(pgdat, NR_SHMEM_PMDMAPPED) * HPAGE_PMD_NR), - K(node_page_state(pgdat, NR_ANON_THPS) * HPAGE_PMD_NR), + K(node_page_state(pgdat, NR_ANON_THPS)), #endif K(node_page_state(pgdat, NR_WRITEBACK_TEMP)), node_page_state(pgdat, NR_KERNEL_STACK_KB), diff --git a/mm/rmap.c b/mm/rmap.c index 08c56aaf72eb..f59e92e26b61 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -1144,7 +1144,8 @@ void do_page_add_anon_rmap(struct page *page, * disabled. */ if (compound) - __inc_lruvec_page_state(page, NR_ANON_THPS); + __mod_lruvec_page_state(page, NR_ANON_THPS, + HPAGE_PMD_NR); __mod_lruvec_page_state(page, NR_ANON_MAPPED, nr); } @@ -1186,7 +1187,7 @@ void page_add_new_anon_rmap(struct page *page, if (hpage_pincount_available(page)) atomic_set(compound_pincount_ptr(page), 0); - __inc_lruvec_page_state(page, NR_ANON_THPS); + __mod_lruvec_page_state(page, NR_ANON_THPS, HPAGE_PMD_NR); } else { /* Anon THP always mapped first with PMD */ VM_BUG_ON_PAGE(PageTransCompound(page), page); @@ -1292,7 +1293,7 @@ static void page_remove_anon_compound_rmap(struct page *page) if (!IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE)) return; - __dec_lruvec_page_state(page, NR_ANON_THPS); + __mod_lruvec_page_state(page, NR_ANON_THPS, -HPAGE_PMD_NR); if (TestClearPageDoubleMap(page)) { /* From patchwork Sun Dec 6 10:14:42 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 11953865 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0AF3FC19425 for ; Sun, 6 Dec 2020 10:18:38 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id EADEF230FB for ; Sun, 6 Dec 2020 10:18:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727103AbgLFKSY (ORCPT ); Sun, 6 Dec 2020 05:18:24 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34642 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727058AbgLFKSX (ORCPT ); Sun, 6 Dec 2020 05:18:23 -0500 Received: from mail-pg1-x536.google.com (mail-pg1-x536.google.com [IPv6:2607:f8b0:4864:20::536]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 05FDAC08C5F2 for ; Sun, 6 Dec 2020 02:17:17 -0800 (PST) Received: by mail-pg1-x536.google.com with SMTP id q3so6467733pgr.3 for ; Sun, 06 Dec 2020 02:17:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=i2WmQI9Q0soqzGFU1vD0NNfzSFXtCmp1bcFtyNd4sgI=; b=Ms1jKhSa+qgJDEXhLIQ4samWqeYhB2vhUcYFQ9kIBfeKOFDxvVzFk87TrixWrRD6Rr +vaXPRp3mQ8tdRqpuDAFtslhGGYCDSz9lAeIhR+7d3Au95mDEpciTi58EJCUbOV74boF irHf7UgHfqeYxqPOZMX694NrAsulQCs6XWfG4XayIwUmc6E7MAPpDhiT8R6HB7wMt/sK SNrLW1p5/PnRMUlJ+0fav+HcOTb1EAz0rIYHqUEVW3GpZpT1zdHfzbuDuyQrNuUcrO3L MmsEJi3uDyzEQa2sWlVjoK6VcSbmc9uIkM5UCU+YCzAwjoR5Wi2Lk1tuSgR+J6AkErEc KAhQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=i2WmQI9Q0soqzGFU1vD0NNfzSFXtCmp1bcFtyNd4sgI=; b=dCCY536UorofQDUORJzW8iuUSfHGsvMsELtsyLfD4PTnwCQO6hqvCXjFgl0LXdtkdC r8vHXZttJgKry6ykKNekPILMnXXdttS0EXF2kaglhmor/xuiKNqrmyvPunXqfYpIGsSc eFw6bP5NGX1KQL24TR25ddnR+1IbVX1KrPPMgsZxNEND1DGEYHxHlM+IXUhzXro4IUdC FNRT8uEi7wXk27Ty4qNI8tyyDLDtRnE/wOSyuxbiEJ9zpE/oq2wbguuARd/4fCMLEB+X cPBOetYEdYiKDdr0gpfZSzAonPup0QmwU3HuaFQ0wUrCBUkkLS9mdTBoYBDT4KoaSZTg OSPA== X-Gm-Message-State: AOAM532IZ0LX2Rkdb13vLAkiq/YtzNr4M50E2pLYmKTNVS601+2QQOCe ANvPLQNRuzJ+/Bza8N8F7NIJrg== X-Google-Smtp-Source: ABdhPJxX4VV3xHN6Lrb6jwWVudCM8xvy7cP+M5Gcg95AHBIo8KyIJoYWtLsufEfRR9NNE3e7JelnGw== X-Received: by 2002:a62:8f03:0:b029:18c:968a:4cf9 with SMTP id n3-20020a628f030000b029018c968a4cf9mr11487716pfd.59.1607249836594; Sun, 06 Dec 2020 02:17:16 -0800 (PST) Received: from localhost.localdomain ([103.136.221.70]) by smtp.gmail.com with ESMTPSA id g16sm10337657pfb.201.2020.12.06.02.17.08 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 06 Dec 2020 02:17:16 -0800 (PST) From: Muchun Song To: gregkh@linuxfoundation.org, rafael@kernel.org, adobriyan@gmail.com, akpm@linux-foundation.org, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, hughd@google.com, will@kernel.org, guro@fb.com, rppt@kernel.org, tglx@linutronix.de, esyr@redhat.com, peterx@redhat.com, krisman@collabora.com, surenb@google.com, avagin@openvz.org, elver@google.com, rdunlap@infradead.org, iamjoonsoo.kim@lge.com Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, Muchun Song Subject: [RESEND PATCH v2 03/12] mm: memcontrol: convert NR_FILE_THPS account to pages Date: Sun, 6 Dec 2020 18:14:42 +0800 Message-Id: <20201206101451.14706-4-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20201206101451.14706-1-songmuchun@bytedance.com> References: <20201206101451.14706-1-songmuchun@bytedance.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Converrt NR_FILE_THPS account to pages. Signed-off-by: Muchun Song --- drivers/base/node.c | 3 +-- fs/proc/meminfo.c | 2 +- mm/filemap.c | 2 +- mm/huge_memory.c | 3 ++- mm/khugepaged.c | 2 +- mm/memcontrol.c | 5 ++--- 6 files changed, 8 insertions(+), 9 deletions(-) diff --git a/drivers/base/node.c b/drivers/base/node.c index 7ebe4c2f64d1..2db28acdaa4f 100644 --- a/drivers/base/node.c +++ b/drivers/base/node.c @@ -466,8 +466,7 @@ static ssize_t node_read_meminfo(struct device *dev, HPAGE_PMD_NR), nid, K(node_page_state(pgdat, NR_SHMEM_PMDMAPPED) * HPAGE_PMD_NR), - nid, K(node_page_state(pgdat, NR_FILE_THPS) * - HPAGE_PMD_NR), + nid, K(node_page_state(pgdat, NR_FILE_THPS)), nid, K(node_page_state(pgdat, NR_FILE_PMDMAPPED) * HPAGE_PMD_NR) #endif diff --git a/fs/proc/meminfo.c b/fs/proc/meminfo.c index 1f7e1945c313..f4157f26cbf5 100644 --- a/fs/proc/meminfo.c +++ b/fs/proc/meminfo.c @@ -135,7 +135,7 @@ static int meminfo_proc_show(struct seq_file *m, void *v) show_val_kb(m, "ShmemPmdMapped: ", global_node_page_state(NR_SHMEM_PMDMAPPED) * HPAGE_PMD_NR); show_val_kb(m, "FileHugePages: ", - global_node_page_state(NR_FILE_THPS) * HPAGE_PMD_NR); + global_node_page_state(NR_FILE_THPS)); show_val_kb(m, "FilePmdMapped: ", global_node_page_state(NR_FILE_PMDMAPPED) * HPAGE_PMD_NR); #endif diff --git a/mm/filemap.c b/mm/filemap.c index 28e7309c29c8..c4dcb1144883 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -206,7 +206,7 @@ static void unaccount_page_cache_page(struct address_space *mapping, if (PageTransHuge(page)) __dec_lruvec_page_state(page, NR_SHMEM_THPS); } else if (PageTransHuge(page)) { - __dec_lruvec_page_state(page, NR_FILE_THPS); + __mod_lruvec_page_state(page, NR_FILE_THPS, -HPAGE_PMD_NR); filemap_nr_thps_dec(mapping); } diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 1a13e1dab3ec..37840bdeaad0 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -2748,7 +2748,8 @@ int split_huge_page_to_list(struct page *page, struct list_head *list) if (PageSwapBacked(head)) __dec_lruvec_page_state(head, NR_SHMEM_THPS); else - __dec_lruvec_page_state(head, NR_FILE_THPS); + __mod_lruvec_page_state(head, NR_FILE_THPS, + -HPAGE_PMD_NR); } __split_huge_page(page, list, end); diff --git a/mm/khugepaged.c b/mm/khugepaged.c index ad316d2e1fee..1e1ced2208d0 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1859,7 +1859,7 @@ static void collapse_file(struct mm_struct *mm, if (is_shmem) __inc_lruvec_page_state(new_page, NR_SHMEM_THPS); else { - __inc_lruvec_page_state(new_page, NR_FILE_THPS); + __mod_lruvec_page_state(new_page, NR_FILE_THPS, HPAGE_PMD_NR); filemap_nr_thps_inc(mapping); } diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 2700c1db5a1a..c4557de2b211 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -1514,7 +1514,7 @@ static struct memory_stat memory_stats[] = { * constant(e.g. powerpc). */ { "anon_thp", PAGE_SIZE, NR_ANON_THPS }, - { "file_thp", 0, NR_FILE_THPS }, + { "file_thp", PAGE_SIZE, NR_FILE_THPS }, { "shmem_thp", 0, NR_SHMEM_THPS }, #endif { "inactive_anon", PAGE_SIZE, NR_INACTIVE_ANON }, @@ -1546,8 +1546,7 @@ static int __init memory_stats_init(void) for (i = 0; i < ARRAY_SIZE(memory_stats); i++) { #ifdef CONFIG_TRANSPARENT_HUGEPAGE - if (memory_stats[i].idx == NR_FILE_THPS || - memory_stats[i].idx == NR_SHMEM_THPS) + if (memory_stats[i].idx == NR_SHMEM_THPS) memory_stats[i].ratio = HPAGE_PMD_SIZE; #endif VM_BUG_ON(!memory_stats[i].ratio); From patchwork Sun Dec 6 10:14:43 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 11953861 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.9 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,UNWANTED_LANGUAGE_BODY, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D9612C4361B for ; Sun, 6 Dec 2020 10:18:37 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A4AA3230FB for ; Sun, 6 Dec 2020 10:18:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726648AbgLFKSV (ORCPT ); Sun, 6 Dec 2020 05:18:21 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34656 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726506AbgLFKSU (ORCPT ); Sun, 6 Dec 2020 05:18:20 -0500 Received: from mail-pl1-x62e.google.com (mail-pl1-x62e.google.com [IPv6:2607:f8b0:4864:20::62e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AB667C08E861 for ; Sun, 6 Dec 2020 02:17:25 -0800 (PST) Received: by mail-pl1-x62e.google.com with SMTP id 4so5589996plk.5 for ; Sun, 06 Dec 2020 02:17:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=jqJ1l3kUFANKBttrjy8eqm/Bfbx0CB9v1pkRqyyS8to=; b=KmA+ORfQc43rX3WMdlp11TQtIwpSJKpCxx5KoM4F25X7T30VLyaba67ZPkmP5cSM7c PKM6k/+aB9OVyO/8dQ+UJ86/av3LnTXnc1p/BRBzA0G00SzeU8PcqZ9Urmcf9zAXuAoZ xXzf2hK7TGTUPRpZVFDts8Ltd9Zsw1mZc60SLFI1uD+HV5B26N+YKp2clMDnKZYVQEBU AL7xPd2zVYSTvthcCID8/4MrAWZvFilD4o1eUb1vzOhLiYzYXGP8SjCSKOeJEG3OUwKx hyLQXCGOesN4uzI5XVZPYMZuoUNXP9kiHZSDYi9/0RQPtXc/uXCNVFSZxXt75fOvG6Ln Zo+g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=jqJ1l3kUFANKBttrjy8eqm/Bfbx0CB9v1pkRqyyS8to=; b=KOTkpLVZ55zPsKnHCChtVAEkVxIiQ+7DA0DCpSrIBTEVh/65DVbC69yMbuJ0tjz1OT lrHjws7G7N6VrftBBznhVdKpTW7r+yYKvWtCg7BYllJYldni1qy0gxEK71XvsaOEvBH5 4WMPseBX0XfSRyDE0LvQnvABl2qMvK8lItCbc9lnrzrbRSwPE2Kp62vGwGS9plyS0irc QOu9NL4fX56fAgf6x20prIQmMDSTjlhRPXznrsmO1nFY6Hu3pMX6Ux6sQttu+KN4tBqh 1lO0FlIy0b/7utbhG1+B3xLFwxJ9q9MudoVWJpcTZLNo7agvVRLU5JtNjiu5lrtH87bz DP2w== X-Gm-Message-State: AOAM532I89PbvGBQkk3HWeI6K95lEXtRzZfIVW9Fp1f4f2cwYBaafNj8 HV7QMv9zQgCKrS920AF1bV8JGw== X-Google-Smtp-Source: ABdhPJyATYkKaw9y7vPdixMLM0rw9slgfozfwgOvQwnqmmhL7taHvYOu1U+qJ+3OrLbTIC6GDGWDUg== X-Received: by 2002:a17:902:7b97:b029:d8:ec6e:5c28 with SMTP id w23-20020a1709027b97b02900d8ec6e5c28mr11410388pll.40.1607249845181; Sun, 06 Dec 2020 02:17:25 -0800 (PST) Received: from localhost.localdomain ([103.136.221.70]) by smtp.gmail.com with ESMTPSA id g16sm10337657pfb.201.2020.12.06.02.17.17 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 06 Dec 2020 02:17:24 -0800 (PST) From: Muchun Song To: gregkh@linuxfoundation.org, rafael@kernel.org, adobriyan@gmail.com, akpm@linux-foundation.org, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, hughd@google.com, will@kernel.org, guro@fb.com, rppt@kernel.org, tglx@linutronix.de, esyr@redhat.com, peterx@redhat.com, krisman@collabora.com, surenb@google.com, avagin@openvz.org, elver@google.com, rdunlap@infradead.org, iamjoonsoo.kim@lge.com Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, Muchun Song Subject: [RESEND PATCH v2 04/12] mm: memcontrol: convert NR_SHMEM_THPS account to pages Date: Sun, 6 Dec 2020 18:14:43 +0800 Message-Id: <20201206101451.14706-5-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20201206101451.14706-1-songmuchun@bytedance.com> References: <20201206101451.14706-1-songmuchun@bytedance.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Convert NR_SHMEM_THPS account to pages Signed-off-by: Muchun Song --- drivers/base/node.c | 3 +-- fs/proc/meminfo.c | 2 +- mm/filemap.c | 2 +- mm/huge_memory.c | 3 ++- mm/khugepaged.c | 2 +- mm/memcontrol.c | 26 ++------------------------ mm/page_alloc.c | 2 +- mm/shmem.c | 3 ++- 8 files changed, 11 insertions(+), 32 deletions(-) diff --git a/drivers/base/node.c b/drivers/base/node.c index 2db28acdaa4f..3e1094717e40 100644 --- a/drivers/base/node.c +++ b/drivers/base/node.c @@ -462,8 +462,7 @@ static ssize_t node_read_meminfo(struct device *dev, #ifdef CONFIG_TRANSPARENT_HUGEPAGE , nid, K(node_page_state(pgdat, NR_ANON_THPS)), - nid, K(node_page_state(pgdat, NR_SHMEM_THPS) * - HPAGE_PMD_NR), + nid, K(node_page_state(pgdat, NR_SHMEM_THPS)), nid, K(node_page_state(pgdat, NR_SHMEM_PMDMAPPED) * HPAGE_PMD_NR), nid, K(node_page_state(pgdat, NR_FILE_THPS)), diff --git a/fs/proc/meminfo.c b/fs/proc/meminfo.c index f4157f26cbf5..b4d8a6ee822d 100644 --- a/fs/proc/meminfo.c +++ b/fs/proc/meminfo.c @@ -131,7 +131,7 @@ static int meminfo_proc_show(struct seq_file *m, void *v) show_val_kb(m, "AnonHugePages: ", global_node_page_state(NR_ANON_THPS)); show_val_kb(m, "ShmemHugePages: ", - global_node_page_state(NR_SHMEM_THPS) * HPAGE_PMD_NR); + global_node_page_state(NR_SHMEM_THPS)); show_val_kb(m, "ShmemPmdMapped: ", global_node_page_state(NR_SHMEM_PMDMAPPED) * HPAGE_PMD_NR); show_val_kb(m, "FileHugePages: ", diff --git a/mm/filemap.c b/mm/filemap.c index c4dcb1144883..5fdefbbc1bc2 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -204,7 +204,7 @@ static void unaccount_page_cache_page(struct address_space *mapping, if (PageSwapBacked(page)) { __mod_lruvec_page_state(page, NR_SHMEM, -nr); if (PageTransHuge(page)) - __dec_lruvec_page_state(page, NR_SHMEM_THPS); + __mod_lruvec_page_state(page, NR_SHMEM_THPS, -HPAGE_PMD_NR); } else if (PageTransHuge(page)) { __mod_lruvec_page_state(page, NR_FILE_THPS, -HPAGE_PMD_NR); filemap_nr_thps_dec(mapping); diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 37840bdeaad0..0e8541bd9f50 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -2746,7 +2746,8 @@ int split_huge_page_to_list(struct page *page, struct list_head *list) spin_unlock(&ds_queue->split_queue_lock); if (mapping) { if (PageSwapBacked(head)) - __dec_lruvec_page_state(head, NR_SHMEM_THPS); + __mod_lruvec_page_state(head, NR_SHMEM_THPS, + -HPAGE_PMD_NR); else __mod_lruvec_page_state(head, NR_FILE_THPS, -HPAGE_PMD_NR); diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 1e1ced2208d0..4fe79ccfc312 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1857,7 +1857,7 @@ static void collapse_file(struct mm_struct *mm, } if (is_shmem) - __inc_lruvec_page_state(new_page, NR_SHMEM_THPS); + __mod_lruvec_page_state(new_page, NR_SHMEM_THPS, HPAGE_PMD_NR); else { __mod_lruvec_page_state(new_page, NR_FILE_THPS, HPAGE_PMD_NR); filemap_nr_thps_inc(mapping); diff --git a/mm/memcontrol.c b/mm/memcontrol.c index c4557de2b211..6d4365d2fd1c 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -1497,7 +1497,7 @@ struct memory_stat { unsigned int idx; }; -static struct memory_stat memory_stats[] = { +static const struct memory_stat memory_stats[] = { { "anon", PAGE_SIZE, NR_ANON_MAPPED }, { "file", PAGE_SIZE, NR_FILE_PAGES }, { "kernel_stack", 1024, NR_KERNEL_STACK_KB }, @@ -1508,14 +1508,9 @@ static struct memory_stat memory_stats[] = { { "file_dirty", PAGE_SIZE, NR_FILE_DIRTY }, { "file_writeback", PAGE_SIZE, NR_WRITEBACK }, #ifdef CONFIG_TRANSPARENT_HUGEPAGE - /* - * The ratio will be initialized in memory_stats_init(). Because - * on some architectures, the macro of HPAGE_PMD_SIZE is not - * constant(e.g. powerpc). - */ { "anon_thp", PAGE_SIZE, NR_ANON_THPS }, { "file_thp", PAGE_SIZE, NR_FILE_THPS }, - { "shmem_thp", 0, NR_SHMEM_THPS }, + { "shmem_thp", PAGE_SIZE, NR_SHMEM_THPS }, #endif { "inactive_anon", PAGE_SIZE, NR_INACTIVE_ANON }, { "active_anon", PAGE_SIZE, NR_ACTIVE_ANON }, @@ -1540,23 +1535,6 @@ static struct memory_stat memory_stats[] = { { "workingset_nodereclaim", 1, WORKINGSET_NODERECLAIM }, }; -static int __init memory_stats_init(void) -{ - int i; - - for (i = 0; i < ARRAY_SIZE(memory_stats); i++) { -#ifdef CONFIG_TRANSPARENT_HUGEPAGE - if (memory_stats[i].idx == NR_SHMEM_THPS) - memory_stats[i].ratio = HPAGE_PMD_SIZE; -#endif - VM_BUG_ON(!memory_stats[i].ratio); - VM_BUG_ON(memory_stats[i].idx >= MEMCG_NR_STAT); - } - - return 0; -} -pure_initcall(memory_stats_init); - static char *memory_stat_format(struct mem_cgroup *memcg) { struct seq_buf s; diff --git a/mm/page_alloc.c b/mm/page_alloc.c index f97ca98d361f..b6a79196e870 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -5567,7 +5567,7 @@ void show_free_areas(unsigned int filter, nodemask_t *nodemask) K(node_page_state(pgdat, NR_WRITEBACK)), K(node_page_state(pgdat, NR_SHMEM)), #ifdef CONFIG_TRANSPARENT_HUGEPAGE - K(node_page_state(pgdat, NR_SHMEM_THPS) * HPAGE_PMD_NR), + K(node_page_state(pgdat, NR_SHMEM_THPS)), K(node_page_state(pgdat, NR_SHMEM_PMDMAPPED) * HPAGE_PMD_NR), K(node_page_state(pgdat, NR_ANON_THPS)), diff --git a/mm/shmem.c b/mm/shmem.c index 5da4f1a3e663..ea5d8c9ccb5b 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -713,7 +713,8 @@ static int shmem_add_to_page_cache(struct page *page, } if (PageTransHuge(page)) { count_vm_event(THP_FILE_ALLOC); - __inc_lruvec_page_state(page, NR_SHMEM_THPS); + __mod_lruvec_page_state(page, NR_SHMEM_THPS, + HPAGE_PMD_NR); } mapping->nrpages += nr; __mod_lruvec_page_state(page, NR_FILE_PAGES, nr); From patchwork Sun Dec 6 10:14:44 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 11953863 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EB29FC4167B for ; Sun, 6 Dec 2020 10:18:37 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B863823108 for ; Sun, 6 Dec 2020 10:18:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726780AbgLFKSW (ORCPT ); Sun, 6 Dec 2020 05:18:22 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34662 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726506AbgLFKSV (ORCPT ); Sun, 6 Dec 2020 05:18:21 -0500 Received: from mail-pj1-x102a.google.com (mail-pj1-x102a.google.com [IPv6:2607:f8b0:4864:20::102a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 648BEC08E863 for ; Sun, 6 Dec 2020 02:17:34 -0800 (PST) Received: by mail-pj1-x102a.google.com with SMTP id b5so1862445pjl.0 for ; Sun, 06 Dec 2020 02:17:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=dMdFQuYVxvcZn8v/8mwYCe8GX7PJ+nj7kDaLzG21jFc=; b=IK7YsvdANv45P9bPp/+tcViV/5ZRgaIqVfZiGOFYl4AT+LDlornq1SJTCzpg34ziSS alDGdrMAfAtUerS0JIvbwSFBVnTCSwpGj1vioyYSEm0l1ZNifTM47TmHD1Wm7jShBBsS JR8Yb3XlKjGUnBPCc2cDSgs3KJPPtBJ+g/cmzdAYqsZpBys3F3GY6iVjK/h2prjwkc3o qfMpVKOzzEXgZSpClDBkBsNQ7/43uK7PqHOx07K20kjKHeCvkA8i/49owagWlouNNsBT Sf5aTqmaU2G2M6httDGpi0ZLfknJoyCbIUHrh42l5rIAL+p0BHsDdrasQSp7AzO/kHoA E8mA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=dMdFQuYVxvcZn8v/8mwYCe8GX7PJ+nj7kDaLzG21jFc=; b=ReUmipBEagzfYx+HuvYhSXhe4pnYLUxxxBSZ5v1nfr4WdPcdjGRV/+ZvjqGRMijsCh pVR9qX9ct6N/7WhA8SJnV7XQsDh28jKh9arueQn6C9NWiuH+3yU3AYyb752/1FmjH/0W eCtdiLF0JMziwp0HeYJToY542YMOQa2STSeZOsvbf5O+aW9sepPBNnwfzB8Lm0GY2DeH 3IV6NTrkvwq209NdvEqwm4Ir1xmI7aTOZ0MEMJqa4S8teici0yZKER1VQHan27XXm1gj k3F97d7clLXbis61iEX/+hgXBNrLWvvoyIw/hgl+Kyr5UXOWWNZK9y3oFb9C4uta2ARY VMRw== X-Gm-Message-State: AOAM532J0NA7GzAM9CPMxK3zewtDc6DN6hbn+3sxH9qx+X7IMe+Dqu+9 md6MN0WSBjXEZo4ASi+wAngchQ== X-Google-Smtp-Source: ABdhPJzYhUQEoJu2OmvjmllpIGf6IO82ScoCt1uZsQ3igOUoPB/moId0BftRZgOTLUmgdDW/V8B93A== X-Received: by 2002:a17:90b:1987:: with SMTP id mv7mr11947217pjb.66.1607249854013; Sun, 06 Dec 2020 02:17:34 -0800 (PST) Received: from localhost.localdomain ([103.136.221.70]) by smtp.gmail.com with ESMTPSA id g16sm10337657pfb.201.2020.12.06.02.17.25 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 06 Dec 2020 02:17:33 -0800 (PST) From: Muchun Song To: gregkh@linuxfoundation.org, rafael@kernel.org, adobriyan@gmail.com, akpm@linux-foundation.org, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, hughd@google.com, will@kernel.org, guro@fb.com, rppt@kernel.org, tglx@linutronix.de, esyr@redhat.com, peterx@redhat.com, krisman@collabora.com, surenb@google.com, avagin@openvz.org, elver@google.com, rdunlap@infradead.org, iamjoonsoo.kim@lge.com Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, Muchun Song Subject: [RESEND PATCH v2 05/12] mm: memcontrol: convert NR_SHMEM_PMDMAPPED account to pages Date: Sun, 6 Dec 2020 18:14:44 +0800 Message-Id: <20201206101451.14706-6-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20201206101451.14706-1-songmuchun@bytedance.com> References: <20201206101451.14706-1-songmuchun@bytedance.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Convert NR_SHMEM_PMDMAPPED account to pages. Signed-off-by: Muchun Song --- drivers/base/node.c | 3 +-- fs/proc/meminfo.c | 2 +- mm/page_alloc.c | 3 +-- mm/rmap.c | 6 ++++-- 4 files changed, 7 insertions(+), 7 deletions(-) diff --git a/drivers/base/node.c b/drivers/base/node.c index 3e1094717e40..e5abc6144dab 100644 --- a/drivers/base/node.c +++ b/drivers/base/node.c @@ -463,8 +463,7 @@ static ssize_t node_read_meminfo(struct device *dev, , nid, K(node_page_state(pgdat, NR_ANON_THPS)), nid, K(node_page_state(pgdat, NR_SHMEM_THPS)), - nid, K(node_page_state(pgdat, NR_SHMEM_PMDMAPPED) * - HPAGE_PMD_NR), + nid, K(node_page_state(pgdat, NR_SHMEM_PMDMAPPED)), nid, K(node_page_state(pgdat, NR_FILE_THPS)), nid, K(node_page_state(pgdat, NR_FILE_PMDMAPPED) * HPAGE_PMD_NR) diff --git a/fs/proc/meminfo.c b/fs/proc/meminfo.c index b4d8a6ee822d..84886b2cc2f7 100644 --- a/fs/proc/meminfo.c +++ b/fs/proc/meminfo.c @@ -133,7 +133,7 @@ static int meminfo_proc_show(struct seq_file *m, void *v) show_val_kb(m, "ShmemHugePages: ", global_node_page_state(NR_SHMEM_THPS)); show_val_kb(m, "ShmemPmdMapped: ", - global_node_page_state(NR_SHMEM_PMDMAPPED) * HPAGE_PMD_NR); + global_node_page_state(NR_SHMEM_PMDMAPPED)); show_val_kb(m, "FileHugePages: ", global_node_page_state(NR_FILE_THPS)); show_val_kb(m, "FilePmdMapped: ", diff --git a/mm/page_alloc.c b/mm/page_alloc.c index b6a79196e870..d103513b3e4f 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -5568,8 +5568,7 @@ void show_free_areas(unsigned int filter, nodemask_t *nodemask) K(node_page_state(pgdat, NR_SHMEM)), #ifdef CONFIG_TRANSPARENT_HUGEPAGE K(node_page_state(pgdat, NR_SHMEM_THPS)), - K(node_page_state(pgdat, NR_SHMEM_PMDMAPPED) - * HPAGE_PMD_NR), + K(node_page_state(pgdat, NR_SHMEM_PMDMAPPED)), K(node_page_state(pgdat, NR_ANON_THPS)), #endif K(node_page_state(pgdat, NR_WRITEBACK_TEMP)), diff --git a/mm/rmap.c b/mm/rmap.c index f59e92e26b61..3089ad6bf468 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -1219,7 +1219,8 @@ void page_add_file_rmap(struct page *page, bool compound) if (!atomic_inc_and_test(compound_mapcount_ptr(page))) goto out; if (PageSwapBacked(page)) - __inc_node_page_state(page, NR_SHMEM_PMDMAPPED); + __mod_lruvec_page_state(page, NR_SHMEM_PMDMAPPED, + HPAGE_PMD_NR); else __inc_node_page_state(page, NR_FILE_PMDMAPPED); } else { @@ -1260,7 +1261,8 @@ static void page_remove_file_rmap(struct page *page, bool compound) if (!atomic_add_negative(-1, compound_mapcount_ptr(page))) return; if (PageSwapBacked(page)) - __dec_node_page_state(page, NR_SHMEM_PMDMAPPED); + __mod_lruvec_page_state(page, NR_SHMEM_PMDMAPPED, + -HPAGE_PMD_NR); else __dec_node_page_state(page, NR_FILE_PMDMAPPED); } else { From patchwork Sun Dec 6 10:14:45 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 11953867 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 684F3C2BB3F for ; Sun, 6 Dec 2020 10:18:38 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 53AF123108 for ; Sun, 6 Dec 2020 10:18:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727315AbgLFKSf (ORCPT ); Sun, 6 Dec 2020 05:18:35 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34710 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727240AbgLFKSf (ORCPT ); Sun, 6 Dec 2020 05:18:35 -0500 Received: from mail-pg1-x530.google.com (mail-pg1-x530.google.com [IPv6:2607:f8b0:4864:20::530]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 46364C0613D4 for ; Sun, 6 Dec 2020 02:17:43 -0800 (PST) Received: by mail-pg1-x530.google.com with SMTP id t37so6452344pga.7 for ; Sun, 06 Dec 2020 02:17:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=47URTm8Dh9aVQTOnIPzCjSnQ3r0uw/hpFPQHZdAjXJk=; b=ZFrm0RAsDKjEspMJB3v1V7ooekc7E7o5kl6qta47nxPfgFfA0L3kv9YJmSyWr+/B31 JHbHQopIMAu9OZDsv9OzAyQyEvdtpOUYOU4uChMVGIF7Rha4Kuyjtf6rvPmWRFq5r0Hb YpLTSeIofdez0J4pdps2gKdUCVEdB8v5HEpGQlyPdg9S6BnGVAvnNjCt9SvzcI6vyvP0 AnBBUWKm3mxAbbRDBv+EBHvtTprB++jbhIeX1HnMXf4B51kCpzqpciDO6VLGjCSqEfqj l7aae0EI4r2sW1m8S0lkLyNyXIeCmU8aRNz4NeRgVcg0KuTQ+ZB92ejjuYcxaPQNk/6O FZjQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=47URTm8Dh9aVQTOnIPzCjSnQ3r0uw/hpFPQHZdAjXJk=; b=E6ncL6rekR6sUjnHvNaVPeUeKZ8FBsGzQPRsSwcXfrqJu744bqznvLnz00NpDmssVr gTxLbhvOe3PHv1URorLNkV1okRL1gWeJrDY/tvMO/h/zftobgH3yOADE7imxLgx546aN lEjI2hj9LfWnmAfYosDXGpTtojE8EIF1CepDY/+rEknc6/yQ7uZjkxVCdWhEa03wIuzD g3klDGZx/TvCK766+/0dOLUkqfSCqDbl53zQ7DQ2JXCOSrxTayAEY2Nz5Sg1bkOl/qWS QSws3j0oQ9SP6ItsZVViDgp5dnAhGPO+VStnvcmYqOVZoHqsvlYHgKosdNAeCRiReG/A iAcQ== X-Gm-Message-State: AOAM533/WlQUDuf5P0Vb+r1dLFmAOD0Zf6kQpjam8OL3m50/X1LY40PG AH7ohVwzMqaapRWBMfU4cHCh0A== X-Google-Smtp-Source: ABdhPJyRYGohzoTg4dhzau/id+aOZcLwu77zYAllJcJ4h4jgoseBbrsSpJiXHpTE4u8OWvmslmKRBw== X-Received: by 2002:a62:6346:0:b029:198:3d33:feb with SMTP id x67-20020a6263460000b02901983d330febmr11740529pfb.8.1607249862830; Sun, 06 Dec 2020 02:17:42 -0800 (PST) Received: from localhost.localdomain ([103.136.221.70]) by smtp.gmail.com with ESMTPSA id g16sm10337657pfb.201.2020.12.06.02.17.34 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 06 Dec 2020 02:17:42 -0800 (PST) From: Muchun Song To: gregkh@linuxfoundation.org, rafael@kernel.org, adobriyan@gmail.com, akpm@linux-foundation.org, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, hughd@google.com, will@kernel.org, guro@fb.com, rppt@kernel.org, tglx@linutronix.de, esyr@redhat.com, peterx@redhat.com, krisman@collabora.com, surenb@google.com, avagin@openvz.org, elver@google.com, rdunlap@infradead.org, iamjoonsoo.kim@lge.com Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, Muchun Song Subject: [RESEND PATCH v2 06/12] mm: memcontrol: convert NR_FILE_PMDMAPPED account to pages Date: Sun, 6 Dec 2020 18:14:45 +0800 Message-Id: <20201206101451.14706-7-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20201206101451.14706-1-songmuchun@bytedance.com> References: <20201206101451.14706-1-songmuchun@bytedance.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Convert NR_FILE_PMDMAPPED account to pages Signed-off-by: Muchun Song --- drivers/base/node.c | 3 +-- fs/proc/meminfo.c | 2 +- mm/rmap.c | 6 ++++-- 3 files changed, 6 insertions(+), 5 deletions(-) diff --git a/drivers/base/node.c b/drivers/base/node.c index e5abc6144dab..f77652e6339f 100644 --- a/drivers/base/node.c +++ b/drivers/base/node.c @@ -465,8 +465,7 @@ static ssize_t node_read_meminfo(struct device *dev, nid, K(node_page_state(pgdat, NR_SHMEM_THPS)), nid, K(node_page_state(pgdat, NR_SHMEM_PMDMAPPED)), nid, K(node_page_state(pgdat, NR_FILE_THPS)), - nid, K(node_page_state(pgdat, NR_FILE_PMDMAPPED) * - HPAGE_PMD_NR) + nid, K(node_page_state(pgdat, NR_FILE_PMDMAPPED)) #endif ); len += hugetlb_report_node_meminfo(buf, len, nid); diff --git a/fs/proc/meminfo.c b/fs/proc/meminfo.c index 84886b2cc2f7..5a83012d8b72 100644 --- a/fs/proc/meminfo.c +++ b/fs/proc/meminfo.c @@ -137,7 +137,7 @@ static int meminfo_proc_show(struct seq_file *m, void *v) show_val_kb(m, "FileHugePages: ", global_node_page_state(NR_FILE_THPS)); show_val_kb(m, "FilePmdMapped: ", - global_node_page_state(NR_FILE_PMDMAPPED) * HPAGE_PMD_NR); + global_node_page_state(NR_FILE_PMDMAPPED)); #endif #ifdef CONFIG_CMA diff --git a/mm/rmap.c b/mm/rmap.c index 3089ad6bf468..e383c5619501 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -1222,7 +1222,8 @@ void page_add_file_rmap(struct page *page, bool compound) __mod_lruvec_page_state(page, NR_SHMEM_PMDMAPPED, HPAGE_PMD_NR); else - __inc_node_page_state(page, NR_FILE_PMDMAPPED); + __mod_lruvec_page_state(page, NR_FILE_PMDMAPPED, + HPAGE_PMD_NR); } else { if (PageTransCompound(page) && page_mapping(page)) { VM_WARN_ON_ONCE(!PageLocked(page)); @@ -1264,7 +1265,8 @@ static void page_remove_file_rmap(struct page *page, bool compound) __mod_lruvec_page_state(page, NR_SHMEM_PMDMAPPED, -HPAGE_PMD_NR); else - __dec_node_page_state(page, NR_FILE_PMDMAPPED); + __mod_lruvec_page_state(page, NR_FILE_PMDMAPPED, + -HPAGE_PMD_NR); } else { if (!atomic_add_negative(-1, &page->_mapcount)) return; From patchwork Sun Dec 6 10:14:46 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 11953871 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9AECFC2BB40 for ; Sun, 6 Dec 2020 10:18:38 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6725A230FA for ; Sun, 6 Dec 2020 10:18:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727260AbgLFKSe (ORCPT ); Sun, 6 Dec 2020 05:18:34 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34634 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727162AbgLFKSd (ORCPT ); Sun, 6 Dec 2020 05:18:33 -0500 Received: from mail-pj1-x1042.google.com (mail-pj1-x1042.google.com [IPv6:2607:f8b0:4864:20::1042]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0C8B8C061A4F for ; Sun, 6 Dec 2020 02:17:52 -0800 (PST) Received: by mail-pj1-x1042.google.com with SMTP id m5so5744018pjv.5 for ; Sun, 06 Dec 2020 02:17:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=OffO4J0XNbKZKsjRiAzQ4jXTCwqv9VY0gL30Fq4+1Jk=; b=H94p/twke3EGgzTfnzLG5zYHpHqM5U9I4h/8S6xaN8YSvCZ9DO+xCRNJwC6Z/juFsl M6Tsvms6S5YZ3AfjawkS+pa4BYclyYqZ7ErIOUHfuViHuPEQJKE7YUPDUVxKXrSe9f+W /RfGFNjF/dCQOKuXuj5rOX6YluRTAYjaWtfsjUt0UVOlFiCQyJQdkNVKiKC+nmLF4x2S t+ClAU7xY+b1CeVAoG4FMpAXmde0vMVwPKAXitdvLV6I6kKeLmjwwcVcjKO7y8jbA/++ 6pgQQpVzBIBijbMxqL/JI/EWPnG4lZP2c+/SrwDWsGQ9CTdpscRKJYHtqtnwm4wK7gPA DPPg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=OffO4J0XNbKZKsjRiAzQ4jXTCwqv9VY0gL30Fq4+1Jk=; b=NkWrJMVHUeIMcAX5hihWUpLoFMn6kzBhMk+o+4OjEeiRL4ODFTrCe+UHyPEo54Z6xa PfWRWxCl/AB2c6l6aMi82F/CLhtaeRlyFTw/PWxHrwKjwxPz0DYxrI5J9KWKPv+BMTYa o12AFpIGdMUo0iBYoyqfdPV2DoUJHSHMPD1AhIb6zYIAGuR9nCQECYV+yUoWAWZANBCg 1NTjX3lFCzUdl/7kyhpZKFopO5cgDv68Sjgl/+9ceMtm/IkFb3ccaTGCAQ/nUgOmhkXT AeZaVUnaHdG7OMvCVXCAB5IT5bnd4u6WDXEpgUUvBbT+FtglI/kgBhwcOiSbPoDy3H2K tlGg== X-Gm-Message-State: AOAM530qOO2sziGnOCpVewmxaGGCvAkbGucjwiazxRxJwga1gEgOzc8O C7zjW4KA4a6OXU9WsBlY6mrQpQ== X-Google-Smtp-Source: ABdhPJyaSoVwqS1hV86Jy9HYtaaTN3KW8cLPtOEHhySz3KTEmRALmhiqeeK5DCu0P0C+H1csGtcR0Q== X-Received: by 2002:a17:90a:e386:: with SMTP id b6mr11649470pjz.134.1607249871646; Sun, 06 Dec 2020 02:17:51 -0800 (PST) Received: from localhost.localdomain ([103.136.221.70]) by smtp.gmail.com with ESMTPSA id g16sm10337657pfb.201.2020.12.06.02.17.43 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 06 Dec 2020 02:17:51 -0800 (PST) From: Muchun Song To: gregkh@linuxfoundation.org, rafael@kernel.org, adobriyan@gmail.com, akpm@linux-foundation.org, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, hughd@google.com, will@kernel.org, guro@fb.com, rppt@kernel.org, tglx@linutronix.de, esyr@redhat.com, peterx@redhat.com, krisman@collabora.com, surenb@google.com, avagin@openvz.org, elver@google.com, rdunlap@infradead.org, iamjoonsoo.kim@lge.com Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, Muchun Song Subject: [RESEND PATCH v2 07/12] mm: memcontrol: convert kernel stack account to bytes Date: Sun, 6 Dec 2020 18:14:46 +0800 Message-Id: <20201206101451.14706-8-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20201206101451.14706-1-songmuchun@bytedance.com> References: <20201206101451.14706-1-songmuchun@bytedance.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org The kernel stack account is the one that counts in KiB. This patch convert it from KiB to byte. Signed-off-by: Muchun Song --- drivers/base/node.c | 2 +- fs/proc/meminfo.c | 2 +- include/linux/mmzone.h | 2 +- kernel/fork.c | 8 ++++---- mm/memcontrol.c | 2 +- mm/page_alloc.c | 2 +- 6 files changed, 9 insertions(+), 9 deletions(-) diff --git a/drivers/base/node.c b/drivers/base/node.c index f77652e6339f..92a75bad35c9 100644 --- a/drivers/base/node.c +++ b/drivers/base/node.c @@ -446,7 +446,7 @@ static ssize_t node_read_meminfo(struct device *dev, nid, K(node_page_state(pgdat, NR_FILE_MAPPED)), nid, K(node_page_state(pgdat, NR_ANON_MAPPED)), nid, K(i.sharedram), - nid, node_page_state(pgdat, NR_KERNEL_STACK_KB), + nid, node_page_state(pgdat, NR_KERNEL_STACK_B) / SZ_1K, #ifdef CONFIG_SHADOW_CALL_STACK nid, node_page_state(pgdat, NR_KERNEL_SCS_KB), #endif diff --git a/fs/proc/meminfo.c b/fs/proc/meminfo.c index 5a83012d8b72..799a537d4218 100644 --- a/fs/proc/meminfo.c +++ b/fs/proc/meminfo.c @@ -101,7 +101,7 @@ static int meminfo_proc_show(struct seq_file *m, void *v) show_val_kb(m, "SReclaimable: ", sreclaimable); show_val_kb(m, "SUnreclaim: ", sunreclaim); seq_printf(m, "KernelStack: %8lu kB\n", - global_node_page_state(NR_KERNEL_STACK_KB)); + global_node_page_state(NR_KERNEL_STACK_B) / SZ_1K); #ifdef CONFIG_SHADOW_CALL_STACK seq_printf(m, "ShadowCallStack:%8lu kB\n", global_node_page_state(NR_KERNEL_SCS_KB)); diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index 15132adaa233..bd34416293ec 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -202,7 +202,7 @@ enum node_stat_item { NR_KERNEL_MISC_RECLAIMABLE, /* reclaimable non-slab kernel pages */ NR_FOLL_PIN_ACQUIRED, /* via: pin_user_page(), gup flag: FOLL_PIN */ NR_FOLL_PIN_RELEASED, /* pages returned via unpin_user_page() */ - NR_KERNEL_STACK_KB, /* measured in KiB */ + NR_KERNEL_STACK_B, /* measured in byte */ #if IS_ENABLED(CONFIG_SHADOW_CALL_STACK) NR_KERNEL_SCS_KB, /* measured in KiB */ #endif diff --git a/kernel/fork.c b/kernel/fork.c index 345f378e104d..2913d7c43dcb 100644 --- a/kernel/fork.c +++ b/kernel/fork.c @@ -382,11 +382,11 @@ static void account_kernel_stack(struct task_struct *tsk, int account) /* All stack pages are in the same node. */ if (vm) - mod_lruvec_page_state(vm->pages[0], NR_KERNEL_STACK_KB, - account * (THREAD_SIZE / 1024)); + mod_lruvec_page_state(vm->pages[0], NR_KERNEL_STACK_B, + account * THREAD_SIZE); else - mod_lruvec_kmem_state(stack, NR_KERNEL_STACK_KB, - account * (THREAD_SIZE / 1024)); + mod_lruvec_kmem_state(stack, NR_KERNEL_STACK_B, + account * THREAD_SIZE); } static int memcg_charge_kernel_stack(struct task_struct *tsk) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 6d4365d2fd1c..48d70c1ad301 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -1500,7 +1500,7 @@ struct memory_stat { static const struct memory_stat memory_stats[] = { { "anon", PAGE_SIZE, NR_ANON_MAPPED }, { "file", PAGE_SIZE, NR_FILE_PAGES }, - { "kernel_stack", 1024, NR_KERNEL_STACK_KB }, + { "kernel_stack", 1, NR_KERNEL_STACK_B }, { "percpu", 1, MEMCG_PERCPU_B }, { "sock", PAGE_SIZE, MEMCG_SOCK }, { "shmem", PAGE_SIZE, NR_SHMEM }, diff --git a/mm/page_alloc.c b/mm/page_alloc.c index d103513b3e4f..d2821ba7f682 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -5572,7 +5572,7 @@ void show_free_areas(unsigned int filter, nodemask_t *nodemask) K(node_page_state(pgdat, NR_ANON_THPS)), #endif K(node_page_state(pgdat, NR_WRITEBACK_TEMP)), - node_page_state(pgdat, NR_KERNEL_STACK_KB), + node_page_state(pgdat, NR_KERNEL_STACK_B) / SZ_1K, #ifdef CONFIG_SHADOW_CALL_STACK node_page_state(pgdat, NR_KERNEL_SCS_KB), #endif From patchwork Sun Dec 6 10:14:47 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 11953875 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B1F54C4167B for ; Sun, 6 Dec 2020 10:19:12 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 703092313B for ; Sun, 6 Dec 2020 10:19:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727387AbgLFKSm (ORCPT ); Sun, 6 Dec 2020 05:18:42 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34656 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727240AbgLFKSl (ORCPT ); Sun, 6 Dec 2020 05:18:41 -0500 Received: from mail-pf1-x443.google.com (mail-pf1-x443.google.com [IPv6:2607:f8b0:4864:20::443]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DDADCC08E9AA for ; Sun, 6 Dec 2020 02:18:03 -0800 (PST) Received: by mail-pf1-x443.google.com with SMTP id w6so7024349pfu.1 for ; Sun, 06 Dec 2020 02:18:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=IzVbUHzHNutFlJQXX/d4rCoABm7bks/8Sv9HD05GfEo=; b=poeTINc8GQTJYdjLzUjOwVcyWKpDqMJuexxM4Jb6DPltaseMinhqh/fI4za09P7SpJ ueTfpbdcaNT8FRgCzXBnvchsMG19r9qFb6dmWplY0+dV4prCug2v2aNQDXQeD4f4ITSk EoVkV9OIP/YW0/Sa7OyTso7ZG60JcGENiXRqAvbK4UuODW674qsOoXE/pNVvUrmgshUS 85409VfqkcMfK8Xr4ByeyYmoKcONo5Ce9w22Krdh3gyADCxxiAkKBnxjcsgF8uV5S52E vCvZZh3sftwph8jcw8YfxrYT88oU5Tt46j5Q212IwnFvnbLVED2uTA1ydilqWk7ajFo5 i5mQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=IzVbUHzHNutFlJQXX/d4rCoABm7bks/8Sv9HD05GfEo=; b=fsYCWIE20dUCi/InaQQfH+T1hw/B1RcDqYmMr4nkyc3yO6bpOIhKfK7TfhU07SAMMF wa0YhkynlyJe+UuzlXBFW3fOs6SXX44oLdaYok1QIL572AQxN7d+lJ1rvOjiRIqVh/e6 QjFP9WJOVlRJmLV42hRSLhYL893LjjGp1o+7SyoieZfxgCuFfq7nbWGGHlg1ExB96Tt8 IPsPclnpqxq/upk7JjWwFHI00xm0xTnUqpid9806uHlHvN+G7gM4ztXW4NOkaXotlwWB ypS86WcakTzmKI8506d3zvM7e7jxZcW4cpfccrUizp/wP2XfIk3f8Su1A5egWFSan/z8 SmUg== X-Gm-Message-State: AOAM533s3ZQciI3PwUdrWKEPkEZuzqxKxNdPniHPhY7M9+nAckVoGfeg KB7MPE1UeWitqvDXjrUQi691Xw== X-Google-Smtp-Source: ABdhPJwOE+mifEeZdUynoRMflnzW+cfbd7QXeq8UPaKhUO7NDUpn7fwwqZ66YSE0/Dk/xryAVB2Udg== X-Received: by 2002:aa7:9501:0:b029:155:3b11:d5c4 with SMTP id b1-20020aa795010000b02901553b11d5c4mr11232823pfp.76.1607249883472; Sun, 06 Dec 2020 02:18:03 -0800 (PST) Received: from localhost.localdomain ([103.136.221.70]) by smtp.gmail.com with ESMTPSA id g16sm10337657pfb.201.2020.12.06.02.17.51 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 06 Dec 2020 02:18:02 -0800 (PST) From: Muchun Song To: gregkh@linuxfoundation.org, rafael@kernel.org, adobriyan@gmail.com, akpm@linux-foundation.org, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, hughd@google.com, will@kernel.org, guro@fb.com, rppt@kernel.org, tglx@linutronix.de, esyr@redhat.com, peterx@redhat.com, krisman@collabora.com, surenb@google.com, avagin@openvz.org, elver@google.com, rdunlap@infradead.org, iamjoonsoo.kim@lge.com Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, Muchun Song Subject: [RESEND PATCH v2 08/12] mm: memcontrol: convert NR_KERNEL_SCS_KB account to bytes Date: Sun, 6 Dec 2020 18:14:47 +0800 Message-Id: <20201206101451.14706-9-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20201206101451.14706-1-songmuchun@bytedance.com> References: <20201206101451.14706-1-songmuchun@bytedance.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Convert NR_KERNEL_SCS_KB account to bytes Signed-off-by: Muchun Song --- drivers/base/node.c | 2 +- fs/proc/meminfo.c | 2 +- include/linux/mmzone.h | 2 +- kernel/scs.c | 4 ++-- mm/page_alloc.c | 2 +- 5 files changed, 6 insertions(+), 6 deletions(-) diff --git a/drivers/base/node.c b/drivers/base/node.c index 92a75bad35c9..bc01ce0b2fcd 100644 --- a/drivers/base/node.c +++ b/drivers/base/node.c @@ -448,7 +448,7 @@ static ssize_t node_read_meminfo(struct device *dev, nid, K(i.sharedram), nid, node_page_state(pgdat, NR_KERNEL_STACK_B) / SZ_1K, #ifdef CONFIG_SHADOW_CALL_STACK - nid, node_page_state(pgdat, NR_KERNEL_SCS_KB), + nid, node_page_state(pgdat, NR_KERNEL_SCS_B) / SZ_1K, #endif nid, K(sum_zone_node_page_state(nid, NR_PAGETABLE)), nid, 0UL, diff --git a/fs/proc/meminfo.c b/fs/proc/meminfo.c index 799a537d4218..69895e83d4fc 100644 --- a/fs/proc/meminfo.c +++ b/fs/proc/meminfo.c @@ -104,7 +104,7 @@ static int meminfo_proc_show(struct seq_file *m, void *v) global_node_page_state(NR_KERNEL_STACK_B) / SZ_1K); #ifdef CONFIG_SHADOW_CALL_STACK seq_printf(m, "ShadowCallStack:%8lu kB\n", - global_node_page_state(NR_KERNEL_SCS_KB)); + global_node_page_state(NR_KERNEL_SCS_B) / SZ_1K); #endif show_val_kb(m, "PageTables: ", global_zone_page_state(NR_PAGETABLE)); diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index bd34416293ec..1f9c83778629 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -204,7 +204,7 @@ enum node_stat_item { NR_FOLL_PIN_RELEASED, /* pages returned via unpin_user_page() */ NR_KERNEL_STACK_B, /* measured in byte */ #if IS_ENABLED(CONFIG_SHADOW_CALL_STACK) - NR_KERNEL_SCS_KB, /* measured in KiB */ + NR_KERNEL_SCS_B, /* measured in byte */ #endif NR_VM_NODE_STAT_ITEMS }; diff --git a/kernel/scs.c b/kernel/scs.c index 4ff4a7ba0094..8db89c932ddc 100644 --- a/kernel/scs.c +++ b/kernel/scs.c @@ -17,8 +17,8 @@ static void __scs_account(void *s, int account) { struct page *scs_page = virt_to_page(s); - mod_node_page_state(page_pgdat(scs_page), NR_KERNEL_SCS_KB, - account * (SCS_SIZE / SZ_1K)); + mod_node_page_state(page_pgdat(scs_page), NR_KERNEL_SCS_B, + account * SCS_SIZE); } static void *scs_alloc(int node) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index d2821ba7f682..58916b3afdab 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -5574,7 +5574,7 @@ void show_free_areas(unsigned int filter, nodemask_t *nodemask) K(node_page_state(pgdat, NR_WRITEBACK_TEMP)), node_page_state(pgdat, NR_KERNEL_STACK_B) / SZ_1K, #ifdef CONFIG_SHADOW_CALL_STACK - node_page_state(pgdat, NR_KERNEL_SCS_KB), + node_page_state(pgdat, NR_KERNEL_SCS_B) / SZ_1K, #endif pgdat->kswapd_failures >= MAX_RECLAIM_RETRIES ? "yes" : "no"); From patchwork Sun Dec 6 10:14:48 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 11953869 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id ADFFEC2BB48 for ; Sun, 6 Dec 2020 10:18:38 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 88A62230FB for ; Sun, 6 Dec 2020 10:18:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727333AbgLFKSh (ORCPT ); Sun, 6 Dec 2020 05:18:37 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34662 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727240AbgLFKSg (ORCPT ); Sun, 6 Dec 2020 05:18:36 -0500 Received: from mail-pj1-x1043.google.com (mail-pj1-x1043.google.com [IPv6:2607:f8b0:4864:20::1043]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4C23DC094240 for ; Sun, 6 Dec 2020 02:18:14 -0800 (PST) Received: by mail-pj1-x1043.google.com with SMTP id b5so1862879pjl.0 for ; Sun, 06 Dec 2020 02:18:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=jxjzCofAcULr7bnQMbpOvZCW6tNXMo722rzO5X3X+g0=; b=l0t429mzGcnZFwxy/27EHeyds85DHU2/SvU/cTTFIMKpTZQ13vGzScVjnNJc1Qfgdl tq3J8LH2Po4yAdE4Qo1uNZ9GfP2dCXBM7HpnjMHacvON/LTEfdfapf3sJCtF6f1t+m// Rdasl44TWp0U1bs3j6e3qWDr9Qn4cvq1JzncWvKY7tQL+4hmLSOUSIIajJobIPEmCai1 ok61QC0poafIrVv2iaxO+AbBLSIJ7BNrJjU/r+EtWVpKKiZe8qX3AGUSS983k1LeN9AL G+VQSyqTH4X1j0tI7iZy9yvMHaPD1FXpVWoKPjzR3GdmseCxF/0+PH5flDA0QOufjbFW Swlw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=jxjzCofAcULr7bnQMbpOvZCW6tNXMo722rzO5X3X+g0=; b=pGtXd7t2xCLSkKPbmhmTttPfbpCyoRmhzlBorNbYuZIYDihzO47Luw35tCpNgz8E5Q NUSPDSeSgKJFQzh2zR4rSFrr1CfHwQk2ODv86KOMBkb5Grh0pHyD4da99ijt2/Ixkgeh 5NNfsJ39Roirg4LMPV2MNb+OCiGJvf4sNdquyo6dh+ow2sZxAZFtUnuLBynYOYvGg+Ez 48bGMBRnKHXpsTvTwGd1JBSeQg5Sd8OcZrbBRl+d2d0oVZLCaB69BdWxEYnZ+9Zjfm+g 9H0K3P4M9tAXVecAOdknHsXMAgsbMAM+rQNgkoeAsThO2pyKJ1L7bOExJqXbQuf51clc kZsQ== X-Gm-Message-State: AOAM533Vm4kVXqMj1MKdSvYNFvuvlY/5Gz9aU2bST8UmmuYO75s5qU/+ Nwhq/tKqWO2ax5KvfZC7bdhHAQ== X-Google-Smtp-Source: ABdhPJwhfw1qSbAuBw+1RM7qEp36SUEwopFNg3Q2irPAgag3zrJxkL8XTXGjZkVkY1IB6k2hYZijDg== X-Received: by 2002:a17:90b:390b:: with SMTP id ob11mr11955031pjb.66.1607249893914; Sun, 06 Dec 2020 02:18:13 -0800 (PST) Received: from localhost.localdomain ([103.136.221.70]) by smtp.gmail.com with ESMTPSA id g16sm10337657pfb.201.2020.12.06.02.18.03 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 06 Dec 2020 02:18:13 -0800 (PST) From: Muchun Song To: gregkh@linuxfoundation.org, rafael@kernel.org, adobriyan@gmail.com, akpm@linux-foundation.org, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, hughd@google.com, will@kernel.org, guro@fb.com, rppt@kernel.org, tglx@linutronix.de, esyr@redhat.com, peterx@redhat.com, krisman@collabora.com, surenb@google.com, avagin@openvz.org, elver@google.com, rdunlap@infradead.org, iamjoonsoo.kim@lge.com Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, Muchun Song Subject: [RESEND PATCH v2 09/12] mm: memcontrol: convert vmstat slab counters to bytes Date: Sun, 6 Dec 2020 18:14:48 +0800 Message-Id: <20201206101451.14706-10-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20201206101451.14706-1-songmuchun@bytedance.com> References: <20201206101451.14706-1-songmuchun@bytedance.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org the global and per-node counters are stored in pages, however memcg and lruvec counters are stored in bytes. This scheme looks weird. So convert all vmstat slab counters to bytes. Signed-off-by: Muchun Song --- include/linux/vmstat.h | 17 ++++++++++------- mm/vmstat.c | 21 ++++++++++----------- 2 files changed, 20 insertions(+), 18 deletions(-) diff --git a/include/linux/vmstat.h b/include/linux/vmstat.h index 322dcbfcc933..fd1a3d5d4926 100644 --- a/include/linux/vmstat.h +++ b/include/linux/vmstat.h @@ -197,18 +197,26 @@ static inline unsigned long global_node_page_state_pages(enum node_stat_item item) { long x = atomic_long_read(&vm_node_stat[item]); + #ifdef CONFIG_SMP if (x < 0) x = 0; #endif + if (vmstat_item_in_bytes(item)) + x >>= PAGE_SHIFT; return x; } static inline unsigned long global_node_page_state(enum node_stat_item item) { - VM_WARN_ON_ONCE(vmstat_item_in_bytes(item)); + long x = atomic_long_read(&vm_node_stat[item]); - return global_node_page_state_pages(item); + VM_WARN_ON_ONCE(vmstat_item_in_bytes(item)); +#ifdef CONFIG_SMP + if (x < 0) + x = 0; +#endif + return x; } static inline unsigned long zone_page_state(struct zone *zone, @@ -312,11 +320,6 @@ static inline void __mod_zone_page_state(struct zone *zone, static inline void __mod_node_page_state(struct pglist_data *pgdat, enum node_stat_item item, int delta) { - if (vmstat_item_in_bytes(item)) { - VM_WARN_ON_ONCE(delta & (PAGE_SIZE - 1)); - delta >>= PAGE_SHIFT; - } - node_page_state_add(delta, pgdat, item); } diff --git a/mm/vmstat.c b/mm/vmstat.c index 8d77ee426e22..7fb0c7cb9516 100644 --- a/mm/vmstat.c +++ b/mm/vmstat.c @@ -345,11 +345,6 @@ void __mod_node_page_state(struct pglist_data *pgdat, enum node_stat_item item, long x; long t; - if (vmstat_item_in_bytes(item)) { - VM_WARN_ON_ONCE(delta & (PAGE_SIZE - 1)); - delta >>= PAGE_SHIFT; - } - x = delta + __this_cpu_read(*p); t = __this_cpu_read(pcp->stat_threshold); @@ -554,11 +549,6 @@ static inline void mod_node_state(struct pglist_data *pgdat, s8 __percpu *p = pcp->vm_node_stat_diff + item; long o, n, t, z; - if (vmstat_item_in_bytes(item)) { - VM_WARN_ON_ONCE(delta & (PAGE_SIZE - 1)); - delta >>= PAGE_SHIFT; - } - do { z = 0; /* overflow to node counters */ @@ -1012,19 +1002,28 @@ unsigned long node_page_state_pages(struct pglist_data *pgdat, enum node_stat_item item) { long x = atomic_long_read(&pgdat->vm_stat[item]); + #ifdef CONFIG_SMP if (x < 0) x = 0; #endif + if (vmstat_item_in_bytes(item)) + x >>= PAGE_SHIFT; return x; } unsigned long node_page_state(struct pglist_data *pgdat, enum node_stat_item item) { + long x = atomic_long_read(&pgdat->vm_stat[item]); + VM_WARN_ON_ONCE(vmstat_item_in_bytes(item)); - return node_page_state_pages(pgdat, item); +#ifdef CONFIG_SMP + if (x < 0) + x = 0; +#endif + return x; } #endif From patchwork Sun Dec 6 10:14:49 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 11953873 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7C088C4361B for ; Sun, 6 Dec 2020 10:19:12 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 424E32312D for ; Sun, 6 Dec 2020 10:19:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727345AbgLFKSk (ORCPT ); Sun, 6 Dec 2020 05:18:40 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34644 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727240AbgLFKSj (ORCPT ); Sun, 6 Dec 2020 05:18:39 -0500 Received: from mail-pl1-x642.google.com (mail-pl1-x642.google.com [IPv6:2607:f8b0:4864:20::642]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 42B14C08E861 for ; Sun, 6 Dec 2020 02:18:23 -0800 (PST) Received: by mail-pl1-x642.google.com with SMTP id x15so5596201pll.2 for ; Sun, 06 Dec 2020 02:18:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=cL9BVbVQLgAGW6GpXAhL1X3CA8w9xPKU91tirmQZrTs=; b=0MtFDJWkKOQQMDcBAWzMx6gdC5YkqHBAIYhp+BQWqaLCRwKSjpNgXt0q+GE5YDnhq7 zZlx16g/g5Hty2Kbo00woChd1R2HTbXvLREFGr+dgsJNg5wqsoVf732VjIl8t0tx/guG xKJjwyDtJ23mJ3u2LiGrZt8158SzBOEHqu1EONODMauwN44TsNCh+5mOLcjf6KWCRj+Q PpDhsHo4azjOMj55FDu9gE2qP5mrbjysGxlYeo4D7bHzSkfdcbJN0sB1YfNpHorDM+rk KTXv8MboLM+aP9XEdWHdigIwbkC+YMT4rLQKIz5VS8XacdkjzRDyPoEiVJiUVpjvTk70 B/IQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=cL9BVbVQLgAGW6GpXAhL1X3CA8w9xPKU91tirmQZrTs=; b=Z+3DuF0sffXEEN6gd+pDSy1MU3nnQL0/XeByYFV2mSf8IAa24wmWdIA2LAo6MiJyW9 cVQ05FRslhJIFqKXLm9CPKZP/4IoNddpYu4/4bxRu5dCFNZeYClFcmwGz4GBoPTRbn4Z vGKcRA4QKszBQT1Pd242cyNkTyHbn76C11hmc2wzB+CWRSVzHUAVRDb+/NoxAKohzjLX K9YOxmYk3GHr+c+zV5HGE+Z2vaSxDN//DURdWyq0gCyRjHljpHkBWDbMyB6CEPj32kiz qis+zmxUNJG3W+CRps6LOHBPLZrU3i1dsKE+tCxyH/VQjaoKjb9hswfurEYVFOdni6xf e7iw== X-Gm-Message-State: AOAM532/WwVtMOir/41vq3J2P4OzTcrLsJbfyKXmpkpn1Y3ptYoqWuHw 7ikXnSAynpN8zh+cWAXA4wrMTA== X-Google-Smtp-Source: ABdhPJz5DNYCSJVhDue++Ah/epNUxyRIwmJ9vSLViSgMOWI7q8z87Dx9FtBFYEBh1GWmEzjkBxvqzQ== X-Received: by 2002:a17:902:b582:b029:d6:6008:264d with SMTP id a2-20020a170902b582b02900d66008264dmr11050550pls.80.1607249902812; Sun, 06 Dec 2020 02:18:22 -0800 (PST) Received: from localhost.localdomain ([103.136.221.70]) by smtp.gmail.com with ESMTPSA id g16sm10337657pfb.201.2020.12.06.02.18.14 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 06 Dec 2020 02:18:22 -0800 (PST) From: Muchun Song To: gregkh@linuxfoundation.org, rafael@kernel.org, adobriyan@gmail.com, akpm@linux-foundation.org, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, hughd@google.com, will@kernel.org, guro@fb.com, rppt@kernel.org, tglx@linutronix.de, esyr@redhat.com, peterx@redhat.com, krisman@collabora.com, surenb@google.com, avagin@openvz.org, elver@google.com, rdunlap@infradead.org, iamjoonsoo.kim@lge.com Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, Muchun Song Subject: [RESEND PATCH v2 10/12] mm: memcontrol: scale stat_threshold for byted-sized vmstat Date: Sun, 6 Dec 2020 18:14:49 +0800 Message-Id: <20201206101451.14706-11-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20201206101451.14706-1-songmuchun@bytedance.com> References: <20201206101451.14706-1-songmuchun@bytedance.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Some vmstat counters are being accounted in bytes not pages, so the stat_threshold should also scale to bytes. The vmstat counters are already long type for memcg (can reference to struct lruvec_stat). For the global per-node vmstat counters also can scale to long. But the maximum vmstat threshold is 125, so the type of s32 is enough. Signed-off-by: Muchun Song --- include/linux/mmzone.h | 17 ++++++----------- include/linux/vmstat.h | 1 - mm/vmstat.c | 24 +++++++++++++----------- 3 files changed, 19 insertions(+), 23 deletions(-) diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index 1f9c83778629..d53328551225 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -216,17 +216,12 @@ enum node_stat_item { */ static __always_inline bool vmstat_item_in_bytes(int idx) { - /* - * Global and per-node slab counters track slab pages. - * It's expected that changes are multiples of PAGE_SIZE. - * Internally values are stored in pages. - * - * Per-memcg and per-lruvec counters track memory, consumed - * by individual slab objects. These counters are actually - * byte-precise. - */ return (idx == NR_SLAB_RECLAIMABLE_B || - idx == NR_SLAB_UNRECLAIMABLE_B); + idx == NR_SLAB_UNRECLAIMABLE_B || +#if IS_ENABLED(CONFIG_SHADOW_CALL_STACK) + idx == NR_KERNEL_SCS_B || +#endif + idx == NR_KERNEL_STACK_B); } /* @@ -340,7 +335,7 @@ struct per_cpu_pageset { struct per_cpu_nodestat { s8 stat_threshold; - s8 vm_node_stat_diff[NR_VM_NODE_STAT_ITEMS]; + s32 vm_node_stat_diff[NR_VM_NODE_STAT_ITEMS]; }; #endif /* !__GENERATING_BOUNDS.H */ diff --git a/include/linux/vmstat.h b/include/linux/vmstat.h index fd1a3d5d4926..afd84dc2398c 100644 --- a/include/linux/vmstat.h +++ b/include/linux/vmstat.h @@ -211,7 +211,6 @@ static inline unsigned long global_node_page_state(enum node_stat_item item) { long x = atomic_long_read(&vm_node_stat[item]); - VM_WARN_ON_ONCE(vmstat_item_in_bytes(item)); #ifdef CONFIG_SMP if (x < 0) x = 0; diff --git a/mm/vmstat.c b/mm/vmstat.c index 7fb0c7cb9516..25751b1d8e2e 100644 --- a/mm/vmstat.c +++ b/mm/vmstat.c @@ -341,13 +341,15 @@ void __mod_node_page_state(struct pglist_data *pgdat, enum node_stat_item item, long delta) { struct per_cpu_nodestat __percpu *pcp = pgdat->per_cpu_nodestats; - s8 __percpu *p = pcp->vm_node_stat_diff + item; + s32 __percpu *p = pcp->vm_node_stat_diff + item; long x; long t; x = delta + __this_cpu_read(*p); t = __this_cpu_read(pcp->stat_threshold); + if (vmstat_item_in_bytes(item)) + t <<= PAGE_SHIFT; if (unlikely(abs(x) > t)) { node_page_state_add(x, pgdat, item); @@ -399,15 +401,15 @@ void __inc_zone_state(struct zone *zone, enum zone_stat_item item) void __inc_node_state(struct pglist_data *pgdat, enum node_stat_item item) { struct per_cpu_nodestat __percpu *pcp = pgdat->per_cpu_nodestats; - s8 __percpu *p = pcp->vm_node_stat_diff + item; - s8 v, t; + s32 __percpu *p = pcp->vm_node_stat_diff + item; + s32 v, t; VM_WARN_ON_ONCE(vmstat_item_in_bytes(item)); v = __this_cpu_inc_return(*p); t = __this_cpu_read(pcp->stat_threshold); if (unlikely(v > t)) { - s8 overstep = t >> 1; + s32 overstep = t >> 1; node_page_state_add(v + overstep, pgdat, item); __this_cpu_write(*p, -overstep); @@ -445,8 +447,8 @@ void __dec_zone_state(struct zone *zone, enum zone_stat_item item) void __dec_node_state(struct pglist_data *pgdat, enum node_stat_item item) { struct per_cpu_nodestat __percpu *pcp = pgdat->per_cpu_nodestats; - s8 __percpu *p = pcp->vm_node_stat_diff + item; - s8 v, t; + s32 __percpu *p = pcp->vm_node_stat_diff + item; + s32 v, t; VM_WARN_ON_ONCE(vmstat_item_in_bytes(item)); @@ -546,7 +548,7 @@ static inline void mod_node_state(struct pglist_data *pgdat, enum node_stat_item item, int delta, int overstep_mode) { struct per_cpu_nodestat __percpu *pcp = pgdat->per_cpu_nodestats; - s8 __percpu *p = pcp->vm_node_stat_diff + item; + s32 __percpu *p = pcp->vm_node_stat_diff + item; long o, n, t, z; do { @@ -563,6 +565,8 @@ static inline void mod_node_state(struct pglist_data *pgdat, * for all cpus in a node. */ t = this_cpu_read(pcp->stat_threshold); + if (vmstat_item_in_bytes(item)) + t <<= PAGE_SHIFT; o = this_cpu_read(*p); n = delta + o; @@ -829,7 +833,7 @@ static int refresh_cpu_vm_stats(bool do_pagesets) struct per_cpu_nodestat __percpu *p = pgdat->per_cpu_nodestats; for (i = 0; i < NR_VM_NODE_STAT_ITEMS; i++) { - int v; + s32 v; v = this_cpu_xchg(p->vm_node_stat_diff[i], 0); if (v) { @@ -899,7 +903,7 @@ void cpu_vm_stats_fold(int cpu) for (i = 0; i < NR_VM_NODE_STAT_ITEMS; i++) if (p->vm_node_stat_diff[i]) { - int v; + s32 v; v = p->vm_node_stat_diff[i]; p->vm_node_stat_diff[i] = 0; @@ -1017,8 +1021,6 @@ unsigned long node_page_state(struct pglist_data *pgdat, { long x = atomic_long_read(&pgdat->vm_stat[item]); - VM_WARN_ON_ONCE(vmstat_item_in_bytes(item)); - #ifdef CONFIG_SMP if (x < 0) x = 0; From patchwork Sun Dec 6 10:14:50 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 11953879 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 61C65C433FE for ; Sun, 6 Dec 2020 10:19:36 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2B0192313F for ; Sun, 6 Dec 2020 10:19:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727556AbgLFKTM (ORCPT ); Sun, 6 Dec 2020 05:19:12 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34818 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726597AbgLFKTM (ORCPT ); Sun, 6 Dec 2020 05:19:12 -0500 Received: from mail-pj1-x1042.google.com (mail-pj1-x1042.google.com [IPv6:2607:f8b0:4864:20::1042]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EA4E4C061A55 for ; Sun, 6 Dec 2020 02:18:31 -0800 (PST) Received: by mail-pj1-x1042.google.com with SMTP id o7so5744474pjj.2 for ; Sun, 06 Dec 2020 02:18:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=KwRaMRI+Sr6xQo3P1J4+HoZOIYaqIcm7QaRM3zeaHfM=; b=f/VNxHgtOj2/ijqqrMgBgEUMKgTq1TDJOBO6yN20rM9HpH8TxNGjJYzwKTkp3RG5sD USGASwcjMY+URzaWD2oF4Pv1KZGFh4zftUJ6WMVXbHKNpARcJxdsDopTTjMW+4Bx1CMk f2c5UACjKkSxlREuzyENkckmbtZVOQTFTlnlEPmMSdM0RyQlwfpdOGuZmJ9MMihGnp35 vxeZEdaUuGsIF96EnguTyhCMMADUD3KzIYz3IyE0jVXLFmkeoxxKPbhM1wOXunzhe9Jw 9p3eJ/ZZXPy00wlkw4joCyNiL5Fs2TV2Gpkjx954WMxsVeup4myYhUKVdDRCrwp027Q2 Ndsw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=KwRaMRI+Sr6xQo3P1J4+HoZOIYaqIcm7QaRM3zeaHfM=; b=S/FkM9jQRGn0Pdjm+Lge/0hxbT1NnfQWjLpxZRVlDKClGhb2WvGwPXqVBdMwBwap9t Ugi76GMMAW2IGHFQCKZZN7zHc9Rp7xeNEwVbhvaFwzUjN1tSQzLcsHcWdKsIMGxQuG3i 7pdjkI823W9Yk4qhQM1DCzJJ0qbOr4Wej6TZll5lN/YMca2zyU45UiI6iAVsrKX+yrdf kjVGzkOYRsFy/JohjPsslIHVQnLrYYfJpKGjit1s2aQMjw5cn6DeewLTP+KKe3ZqAldN fVXnTk4F1QlcYl/6zlT0hUVqLrS37x1K06O6ZCFIwuzNelwTdtk1hI9ibJoAGI1rA/ph 32iA== X-Gm-Message-State: AOAM530JAlJLY5UERb93G12X1gVa+eEGjKRiDiGR8Oa/lS8Ngjq10iLu 03B0LM5p+4/tAY/ZNxF2xVKhHA== X-Google-Smtp-Source: ABdhPJxLE9y1Wu89jVTjYxdBJC2dpD5Vw0WGeqkY8mH2oUPiW1ZCgwzB4TvQGoJ/pALX8b1qIIx9ng== X-Received: by 2002:a17:90a:bf88:: with SMTP id d8mr12070599pjs.102.1607249911546; Sun, 06 Dec 2020 02:18:31 -0800 (PST) Received: from localhost.localdomain ([103.136.221.70]) by smtp.gmail.com with ESMTPSA id g16sm10337657pfb.201.2020.12.06.02.18.23 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 06 Dec 2020 02:18:30 -0800 (PST) From: Muchun Song To: gregkh@linuxfoundation.org, rafael@kernel.org, adobriyan@gmail.com, akpm@linux-foundation.org, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, hughd@google.com, will@kernel.org, guro@fb.com, rppt@kernel.org, tglx@linutronix.de, esyr@redhat.com, peterx@redhat.com, krisman@collabora.com, surenb@google.com, avagin@openvz.org, elver@google.com, rdunlap@infradead.org, iamjoonsoo.kim@lge.com Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, Muchun Song Subject: [RESEND PATCH v2 11/12] mm: memcontrol: make the slab calculation consistent Date: Sun, 6 Dec 2020 18:14:50 +0800 Message-Id: <20201206101451.14706-12-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20201206101451.14706-1-songmuchun@bytedance.com> References: <20201206101451.14706-1-songmuchun@bytedance.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Although the ratio of the slab is one, we also should read the ratio from the related memory_stats instead of hard-coding. And the local variable of size is already the value of slab_unreclaimable. So we do not need to read again. The unit of the vmstat counters are either pages or bytes now. So we can drop the ratio in struct memory_stat. This can make the code clean and simple. And get rid of the awkward mix of static and runtime initialization of the memory_stats table. Signed-off-by: Muchun Song --- mm/memcontrol.c | 108 ++++++++++++++++++++++++++++++++++++-------------------- 1 file changed, 70 insertions(+), 38 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 48d70c1ad301..49fbcf003bf5 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -1493,48 +1493,71 @@ static bool mem_cgroup_wait_acct_move(struct mem_cgroup *memcg) struct memory_stat { const char *name; - unsigned int ratio; unsigned int idx; }; static const struct memory_stat memory_stats[] = { - { "anon", PAGE_SIZE, NR_ANON_MAPPED }, - { "file", PAGE_SIZE, NR_FILE_PAGES }, - { "kernel_stack", 1, NR_KERNEL_STACK_B }, - { "percpu", 1, MEMCG_PERCPU_B }, - { "sock", PAGE_SIZE, MEMCG_SOCK }, - { "shmem", PAGE_SIZE, NR_SHMEM }, - { "file_mapped", PAGE_SIZE, NR_FILE_MAPPED }, - { "file_dirty", PAGE_SIZE, NR_FILE_DIRTY }, - { "file_writeback", PAGE_SIZE, NR_WRITEBACK }, + { "anon", NR_ANON_MAPPED }, + { "file", NR_FILE_PAGES }, + { "kernel_stack", NR_KERNEL_STACK_B }, + { "percpu", MEMCG_PERCPU_B }, + { "sock", MEMCG_SOCK }, + { "shmem", NR_SHMEM }, + { "file_mapped", NR_FILE_MAPPED }, + { "file_dirty", NR_FILE_DIRTY }, + { "file_writeback", NR_WRITEBACK }, #ifdef CONFIG_TRANSPARENT_HUGEPAGE - { "anon_thp", PAGE_SIZE, NR_ANON_THPS }, - { "file_thp", PAGE_SIZE, NR_FILE_THPS }, - { "shmem_thp", PAGE_SIZE, NR_SHMEM_THPS }, + { "anon_thp", NR_ANON_THPS }, + { "file_thp", NR_FILE_THPS }, + { "shmem_thp", NR_SHMEM_THPS }, #endif - { "inactive_anon", PAGE_SIZE, NR_INACTIVE_ANON }, - { "active_anon", PAGE_SIZE, NR_ACTIVE_ANON }, - { "inactive_file", PAGE_SIZE, NR_INACTIVE_FILE }, - { "active_file", PAGE_SIZE, NR_ACTIVE_FILE }, - { "unevictable", PAGE_SIZE, NR_UNEVICTABLE }, - - /* - * Note: The slab_reclaimable and slab_unreclaimable must be - * together and slab_reclaimable must be in front. - */ - { "slab_reclaimable", 1, NR_SLAB_RECLAIMABLE_B }, - { "slab_unreclaimable", 1, NR_SLAB_UNRECLAIMABLE_B }, + { "inactive_anon", NR_INACTIVE_ANON }, + { "active_anon", NR_ACTIVE_ANON }, + { "inactive_file", NR_INACTIVE_FILE }, + { "active_file", NR_ACTIVE_FILE }, + { "unevictable", NR_UNEVICTABLE }, + { "slab_reclaimable", NR_SLAB_RECLAIMABLE_B }, + { "slab_unreclaimable", NR_SLAB_UNRECLAIMABLE_B }, /* The memory events */ - { "workingset_refault_anon", 1, WORKINGSET_REFAULT_ANON }, - { "workingset_refault_file", 1, WORKINGSET_REFAULT_FILE }, - { "workingset_activate_anon", 1, WORKINGSET_ACTIVATE_ANON }, - { "workingset_activate_file", 1, WORKINGSET_ACTIVATE_FILE }, - { "workingset_restore_anon", 1, WORKINGSET_RESTORE_ANON }, - { "workingset_restore_file", 1, WORKINGSET_RESTORE_FILE }, - { "workingset_nodereclaim", 1, WORKINGSET_NODERECLAIM }, + { "workingset_refault_anon", WORKINGSET_REFAULT_ANON }, + { "workingset_refault_file", WORKINGSET_REFAULT_FILE }, + { "workingset_activate_anon", WORKINGSET_ACTIVATE_ANON }, + { "workingset_activate_file", WORKINGSET_ACTIVATE_FILE }, + { "workingset_restore_anon", WORKINGSET_RESTORE_ANON }, + { "workingset_restore_file", WORKINGSET_RESTORE_FILE }, + { "workingset_nodereclaim", WORKINGSET_NODERECLAIM }, }; +/* Translate stat items to the correct unit for memory.stat output */ +static int memcg_page_state_unit(int item) +{ + int unit; + + switch (item) { + case WORKINGSET_REFAULT_ANON: + case WORKINGSET_REFAULT_FILE: + case WORKINGSET_ACTIVATE_ANON: + case WORKINGSET_ACTIVATE_FILE: + case WORKINGSET_RESTORE_ANON: + case WORKINGSET_RESTORE_FILE: + case WORKINGSET_NODERECLAIM: + unit = 1; + break; + default: + unit = memcg_stat_item_in_bytes(item) ? 1 : PAGE_SIZE; + break; + } + + return unit; +} + +static inline unsigned long memcg_page_state_output(struct mem_cgroup *memcg, + int item) +{ + return memcg_page_state(memcg, item) * memcg_page_state_unit(item); +} + static char *memory_stat_format(struct mem_cgroup *memcg) { struct seq_buf s; @@ -1558,13 +1581,16 @@ static char *memory_stat_format(struct mem_cgroup *memcg) for (i = 0; i < ARRAY_SIZE(memory_stats); i++) { u64 size; - size = memcg_page_state(memcg, memory_stats[i].idx); - size *= memory_stats[i].ratio; + size = memcg_page_state_output(memcg, memory_stats[i].idx); seq_buf_printf(&s, "%s %llu\n", memory_stats[i].name, size); + /* + * We are printing reclaimable, unreclaimable of the slab + * and the sum of both. + */ if (unlikely(memory_stats[i].idx == NR_SLAB_UNRECLAIMABLE_B)) { - size = memcg_page_state(memcg, NR_SLAB_RECLAIMABLE_B) + - memcg_page_state(memcg, NR_SLAB_UNRECLAIMABLE_B); + size += memcg_page_state_output(memcg, + NR_SLAB_RECLAIMABLE_B); seq_buf_printf(&s, "slab %llu\n", size); } } @@ -6358,6 +6384,12 @@ static int memory_stat_show(struct seq_file *m, void *v) } #ifdef CONFIG_NUMA +static inline unsigned long lruvec_page_state_output(struct lruvec *lruvec, + int item) +{ + return lruvec_page_state(lruvec, item) * memcg_page_state_unit(item); +} + static int memory_numa_stat_show(struct seq_file *m, void *v) { int i; @@ -6375,8 +6407,8 @@ static int memory_numa_stat_show(struct seq_file *m, void *v) struct lruvec *lruvec; lruvec = mem_cgroup_lruvec(memcg, NODE_DATA(nid)); - size = lruvec_page_state(lruvec, memory_stats[i].idx); - size *= memory_stats[i].ratio; + size = lruvec_page_state_output(lruvec, + memory_stats[i].idx); seq_printf(m, " N%d=%llu", nid, size); } seq_putc(m, '\n'); From patchwork Sun Dec 6 10:14:51 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 11953877 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F0420C19425 for ; Sun, 6 Dec 2020 10:19:12 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C7BCB2313B for ; Sun, 6 Dec 2020 10:19:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727395AbgLFKTC (ORCPT ); Sun, 6 Dec 2020 05:19:02 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34644 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726356AbgLFKTB (ORCPT ); Sun, 6 Dec 2020 05:19:01 -0500 Received: from mail-pf1-x444.google.com (mail-pf1-x444.google.com [IPv6:2607:f8b0:4864:20::444]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6D94EC0613D4 for ; Sun, 6 Dec 2020 02:18:40 -0800 (PST) Received: by mail-pf1-x444.google.com with SMTP id p4so1132692pfg.0 for ; Sun, 06 Dec 2020 02:18:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=lP6NOXY9P1Y1tdnLUE8WJ9oCZQOprYuVn3yHjkNbfog=; b=ZcB+MrFfITKTQD6yHNcSXXGhcuAfJVfq9Vc0JYQrQO8x+y4QfNb3GbTp1Bw/q0Xh1C SRPp2uOrMxEWh2MTi3HsJw/K2uDSI7BPt7W9VaUyVAlrworEQCgS3w0i4ICgyZ0NW2Zx rpIPH2IBLP/NecuXoWwQwHU8HJfMMqjmeilA0qISzpe3Ua5VcpfB1uHFE8rwuzE/+BBu dU61wVmkRUSnIPY6q3zmuFpZhfYSc1xyARC+ZWy/++zR4K0ULoqC3Ec/hYdfb+kc7cKa 6FL91dJW9nMFXkchdG699Ol55L2kOxnfCBh+PsPo6Ib+GHG2WmGfHHoPHmJdZUhbgFZi R05g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=lP6NOXY9P1Y1tdnLUE8WJ9oCZQOprYuVn3yHjkNbfog=; b=UYjS03Idbnu6RLPMjqnGq9I+gzkGCr7MtCB4v6h4M3VK8df9HWDMe50zErNRMHKrbA tAgJR++Wi4FJ6DuTfI0XhmdN04jLbfIu0v2hcThwRmyds+MvH3lKVQIFyC4MGdLNgZ4e zAMxa4dhMOCXoDCF0dw9hZjuPuWjiplxy8Cs4/UDP7gpCPOPY3nYEIKkRqA3JGScpYxv Sn3hXy9zyemDDa9/yvF+VPr7z7fCX0oRJqKsCqfDpRlC7Tnzs0u2Dst4F8G1wyEFncOf 2wok1HuUsMMJUbhKgP3B8v/RfEcj9kjxONDKkyXV8ZNkzGHQu+eo3Dr39t+z4qwJd7qW LsNA== X-Gm-Message-State: AOAM533U6oxIwhgKXjwTrKU/Hlj5ggVmCYloJWOl7wr68c3a5iUoaTL6 PORMi1ISIve/dhtsiaV5Rqnedw== X-Google-Smtp-Source: ABdhPJyTR5IEC8wjJwZanqPdp0TYqYuwBV1k98LMJDgi/aO+lbtgHvOt9jKwuJZe7B/Wxcytd89bHw== X-Received: by 2002:aa7:991c:0:b029:19d:b276:1ecb with SMTP id z28-20020aa7991c0000b029019db2761ecbmr11518358pff.73.1607249919971; Sun, 06 Dec 2020 02:18:39 -0800 (PST) Received: from localhost.localdomain ([103.136.221.70]) by smtp.gmail.com with ESMTPSA id g16sm10337657pfb.201.2020.12.06.02.18.31 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 06 Dec 2020 02:18:39 -0800 (PST) From: Muchun Song To: gregkh@linuxfoundation.org, rafael@kernel.org, adobriyan@gmail.com, akpm@linux-foundation.org, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, hughd@google.com, will@kernel.org, guro@fb.com, rppt@kernel.org, tglx@linutronix.de, esyr@redhat.com, peterx@redhat.com, krisman@collabora.com, surenb@google.com, avagin@openvz.org, elver@google.com, rdunlap@infradead.org, iamjoonsoo.kim@lge.com Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, Muchun Song Subject: [RESEND PATCH v2 12/12] mm: memcontrol: remove {global_}node_page_state_pages Date: Sun, 6 Dec 2020 18:14:51 +0800 Message-Id: <20201206101451.14706-13-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20201206101451.14706-1-songmuchun@bytedance.com> References: <20201206101451.14706-1-songmuchun@bytedance.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Now the unit of the vmstat counters are either pages or bytes. So we can adjust the node_page_state to always returns values in pages and remove the node_page_state_pages. Signed-off-by: Muchun Song --- drivers/base/node.c | 10 +++++----- fs/proc/meminfo.c | 12 ++++++------ include/linux/vmstat.h | 17 +---------------- kernel/power/snapshot.c | 2 +- mm/oom_kill.c | 2 +- mm/page_alloc.c | 10 +++++----- mm/vmscan.c | 2 +- mm/vmstat.c | 23 ++++++----------------- 8 files changed, 26 insertions(+), 52 deletions(-) diff --git a/drivers/base/node.c b/drivers/base/node.c index bc01ce0b2fcd..42298e3552e5 100644 --- a/drivers/base/node.c +++ b/drivers/base/node.c @@ -374,8 +374,8 @@ static ssize_t node_read_meminfo(struct device *dev, unsigned long sreclaimable, sunreclaimable; si_meminfo_node(&i, nid); - sreclaimable = node_page_state_pages(pgdat, NR_SLAB_RECLAIMABLE_B); - sunreclaimable = node_page_state_pages(pgdat, NR_SLAB_UNRECLAIMABLE_B); + sreclaimable = node_page_state(pgdat, NR_SLAB_RECLAIMABLE_B); + sunreclaimable = node_page_state(pgdat, NR_SLAB_UNRECLAIMABLE_B); len = sysfs_emit_at(buf, len, "Node %d MemTotal: %8lu kB\n" "Node %d MemFree: %8lu kB\n" @@ -446,9 +446,9 @@ static ssize_t node_read_meminfo(struct device *dev, nid, K(node_page_state(pgdat, NR_FILE_MAPPED)), nid, K(node_page_state(pgdat, NR_ANON_MAPPED)), nid, K(i.sharedram), - nid, node_page_state(pgdat, NR_KERNEL_STACK_B) / SZ_1K, + nid, K(node_page_state(pgdat, NR_KERNEL_STACK_B)), #ifdef CONFIG_SHADOW_CALL_STACK - nid, node_page_state(pgdat, NR_KERNEL_SCS_B) / SZ_1K, + nid, K(node_page_state(pgdat, NR_KERNEL_SCS_B)), #endif nid, K(sum_zone_node_page_state(nid, NR_PAGETABLE)), nid, 0UL, @@ -517,7 +517,7 @@ static ssize_t node_read_vmstat(struct device *dev, for (i = 0; i < NR_VM_NODE_STAT_ITEMS; i++) len += sysfs_emit_at(buf, len, "%s %lu\n", node_stat_name(i), - node_page_state_pages(pgdat, i)); + node_page_state(pgdat, i)); return len; } diff --git a/fs/proc/meminfo.c b/fs/proc/meminfo.c index 69895e83d4fc..95ea5f062161 100644 --- a/fs/proc/meminfo.c +++ b/fs/proc/meminfo.c @@ -52,8 +52,8 @@ static int meminfo_proc_show(struct seq_file *m, void *v) pages[lru] = global_node_page_state(NR_LRU_BASE + lru); available = si_mem_available(); - sreclaimable = global_node_page_state_pages(NR_SLAB_RECLAIMABLE_B); - sunreclaim = global_node_page_state_pages(NR_SLAB_UNRECLAIMABLE_B); + sreclaimable = global_node_page_state(NR_SLAB_RECLAIMABLE_B); + sunreclaim = global_node_page_state(NR_SLAB_UNRECLAIMABLE_B); show_val_kb(m, "MemTotal: ", i.totalram); show_val_kb(m, "MemFree: ", i.freeram); @@ -100,11 +100,11 @@ static int meminfo_proc_show(struct seq_file *m, void *v) show_val_kb(m, "Slab: ", sreclaimable + sunreclaim); show_val_kb(m, "SReclaimable: ", sreclaimable); show_val_kb(m, "SUnreclaim: ", sunreclaim); - seq_printf(m, "KernelStack: %8lu kB\n", - global_node_page_state(NR_KERNEL_STACK_B) / SZ_1K); + show_val_kb(m, "KernelStack: ", + global_node_page_state(NR_KERNEL_STACK_B)); #ifdef CONFIG_SHADOW_CALL_STACK - seq_printf(m, "ShadowCallStack:%8lu kB\n", - global_node_page_state(NR_KERNEL_SCS_B) / SZ_1K); + show_val_kb(m, "ShadowCallStack:", + global_node_page_state(NR_KERNEL_SCS_B)); #endif show_val_kb(m, "PageTables: ", global_zone_page_state(NR_PAGETABLE)); diff --git a/include/linux/vmstat.h b/include/linux/vmstat.h index afd84dc2398c..ae821e016fdd 100644 --- a/include/linux/vmstat.h +++ b/include/linux/vmstat.h @@ -193,8 +193,7 @@ static inline unsigned long global_zone_page_state(enum zone_stat_item item) return x; } -static inline -unsigned long global_node_page_state_pages(enum node_stat_item item) +static inline unsigned long global_node_page_state(enum node_stat_item item) { long x = atomic_long_read(&vm_node_stat[item]); @@ -207,17 +206,6 @@ unsigned long global_node_page_state_pages(enum node_stat_item item) return x; } -static inline unsigned long global_node_page_state(enum node_stat_item item) -{ - long x = atomic_long_read(&vm_node_stat[item]); - -#ifdef CONFIG_SMP - if (x < 0) - x = 0; -#endif - return x; -} - static inline unsigned long zone_page_state(struct zone *zone, enum zone_stat_item item) { @@ -258,12 +246,9 @@ extern unsigned long sum_zone_node_page_state(int node, extern unsigned long sum_zone_numa_state(int node, enum numa_stat_item item); extern unsigned long node_page_state(struct pglist_data *pgdat, enum node_stat_item item); -extern unsigned long node_page_state_pages(struct pglist_data *pgdat, - enum node_stat_item item); #else #define sum_zone_node_page_state(node, item) global_zone_page_state(item) #define node_page_state(node, item) global_node_page_state(item) -#define node_page_state_pages(node, item) global_node_page_state_pages(item) #endif /* CONFIG_NUMA */ #ifdef CONFIG_SMP diff --git a/kernel/power/snapshot.c b/kernel/power/snapshot.c index d63560e1cf87..664520bdaa20 100644 --- a/kernel/power/snapshot.c +++ b/kernel/power/snapshot.c @@ -1705,7 +1705,7 @@ static unsigned long minimum_image_size(unsigned long saveable) { unsigned long size; - size = global_node_page_state_pages(NR_SLAB_RECLAIMABLE_B) + size = global_node_page_state(NR_SLAB_RECLAIMABLE_B) + global_node_page_state(NR_ACTIVE_ANON) + global_node_page_state(NR_INACTIVE_ANON) + global_node_page_state(NR_ACTIVE_FILE) diff --git a/mm/oom_kill.c b/mm/oom_kill.c index 04b19b7b5435..73861473c7d4 100644 --- a/mm/oom_kill.c +++ b/mm/oom_kill.c @@ -188,7 +188,7 @@ static bool should_dump_unreclaim_slab(void) global_node_page_state(NR_ISOLATED_FILE) + global_node_page_state(NR_UNEVICTABLE); - return (global_node_page_state_pages(NR_SLAB_UNRECLAIMABLE_B) > nr_lru); + return (global_node_page_state(NR_SLAB_UNRECLAIMABLE_B) > nr_lru); } /** diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 58916b3afdab..d16c9388c0b8 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -5372,7 +5372,7 @@ long si_mem_available(void) * items that are in use, and cannot be freed. Cap this estimate at the * low watermark. */ - reclaimable = global_node_page_state_pages(NR_SLAB_RECLAIMABLE_B) + + reclaimable = global_node_page_state(NR_SLAB_RECLAIMABLE_B) + global_node_page_state(NR_KERNEL_MISC_RECLAIMABLE); available += reclaimable - min(reclaimable / 2, wmark_low); @@ -5516,8 +5516,8 @@ void show_free_areas(unsigned int filter, nodemask_t *nodemask) global_node_page_state(NR_UNEVICTABLE), global_node_page_state(NR_FILE_DIRTY), global_node_page_state(NR_WRITEBACK), - global_node_page_state_pages(NR_SLAB_RECLAIMABLE_B), - global_node_page_state_pages(NR_SLAB_UNRECLAIMABLE_B), + global_node_page_state(NR_SLAB_RECLAIMABLE_B), + global_node_page_state(NR_SLAB_UNRECLAIMABLE_B), global_node_page_state(NR_FILE_MAPPED), global_node_page_state(NR_SHMEM), global_zone_page_state(NR_PAGETABLE), @@ -5572,9 +5572,9 @@ void show_free_areas(unsigned int filter, nodemask_t *nodemask) K(node_page_state(pgdat, NR_ANON_THPS)), #endif K(node_page_state(pgdat, NR_WRITEBACK_TEMP)), - node_page_state(pgdat, NR_KERNEL_STACK_B) / SZ_1K, + K(node_page_state(pgdat, NR_KERNEL_STACK_B)), #ifdef CONFIG_SHADOW_CALL_STACK - node_page_state(pgdat, NR_KERNEL_SCS_B) / SZ_1K, + K(node_page_state(pgdat, NR_KERNEL_SCS_B)), #endif pgdat->kswapd_failures >= MAX_RECLAIM_RETRIES ? "yes" : "no"); diff --git a/mm/vmscan.c b/mm/vmscan.c index 469016222cdb..5d3c8fa68979 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -4220,7 +4220,7 @@ int node_reclaim(struct pglist_data *pgdat, gfp_t gfp_mask, unsigned int order) * unmapped file backed pages. */ if (node_pagecache_reclaimable(pgdat) <= pgdat->min_unmapped_pages && - node_page_state_pages(pgdat, NR_SLAB_RECLAIMABLE_B) <= + node_page_state(pgdat, NR_SLAB_RECLAIMABLE_B) <= pgdat->min_slab_pages) return NODE_RECLAIM_FULL; diff --git a/mm/vmstat.c b/mm/vmstat.c index 25751b1d8e2e..b7cdef585efd 100644 --- a/mm/vmstat.c +++ b/mm/vmstat.c @@ -1000,22 +1000,9 @@ unsigned long sum_zone_numa_state(int node, } /* - * Determine the per node value of a stat item. + * Determine the per node value of a stat item. This always returns + * values in pages. */ -unsigned long node_page_state_pages(struct pglist_data *pgdat, - enum node_stat_item item) -{ - long x = atomic_long_read(&pgdat->vm_stat[item]); - -#ifdef CONFIG_SMP - if (x < 0) - x = 0; -#endif - if (vmstat_item_in_bytes(item)) - x >>= PAGE_SHIFT; - return x; -} - unsigned long node_page_state(struct pglist_data *pgdat, enum node_stat_item item) { @@ -1025,6 +1012,8 @@ unsigned long node_page_state(struct pglist_data *pgdat, if (x < 0) x = 0; #endif + if (vmstat_item_in_bytes(item)) + x >>= PAGE_SHIFT; return x; } #endif @@ -1626,7 +1615,7 @@ static void zoneinfo_show_print(struct seq_file *m, pg_data_t *pgdat, seq_printf(m, "\n per-node stats"); for (i = 0; i < NR_VM_NODE_STAT_ITEMS; i++) { seq_printf(m, "\n %-12s %lu", node_stat_name(i), - node_page_state_pages(pgdat, i)); + node_page_state(pgdat, i)); } } seq_printf(m, @@ -1747,7 +1736,7 @@ static void *vmstat_start(struct seq_file *m, loff_t *pos) #endif for (i = 0; i < NR_VM_NODE_STAT_ITEMS; i++) - v[i] = global_node_page_state_pages(i); + v[i] = global_node_page_state(i); v += NR_VM_NODE_STAT_ITEMS; global_dirty_limits(v + NR_DIRTY_BG_THRESHOLD,