From patchwork Sun Dec 6 08:23:07 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 11953807 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B0492C433FE for ; Sun, 6 Dec 2020 08:26:53 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 5D4C423108 for ; Sun, 6 Dec 2020 08:26:53 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5D4C423108 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 001306B006E; Sun, 6 Dec 2020 03:26:53 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id F18B38D0003; Sun, 6 Dec 2020 03:26:52 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E2E558D0002; Sun, 6 Dec 2020 03:26:52 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0205.hostedemail.com [216.40.44.205]) by kanga.kvack.org (Postfix) with ESMTP id CD0C16B006E for ; Sun, 6 Dec 2020 03:26:52 -0500 (EST) Received: from smtpin23.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 95DF31EF1 for ; Sun, 6 Dec 2020 08:26:52 +0000 (UTC) X-FDA: 77562176664.23.bee46_110f176273d4 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin23.hostedemail.com (Postfix) with ESMTP id 7098137604 for ; Sun, 6 Dec 2020 08:26:52 +0000 (UTC) X-HE-Tag: bee46_110f176273d4 X-Filterd-Recvd-Size: 6534 Received: from mail-pl1-f179.google.com (mail-pl1-f179.google.com [209.85.214.179]) by imf32.hostedemail.com (Postfix) with ESMTP for ; Sun, 6 Dec 2020 08:26:51 +0000 (UTC) Received: by mail-pl1-f179.google.com with SMTP id v3so5498840plz.13 for ; Sun, 06 Dec 2020 00:26:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=dMdFQuYVxvcZn8v/8mwYCe8GX7PJ+nj7kDaLzG21jFc=; b=bkG0WB3RwFLFzPbPgpLleJ0rGpSRW4ZQxqqCN4SZCVMNhOE+6vRa7Ver9MYRZVo4t3 rPtQfxUPapsWjqVM6Y7U7hpwUw5niO3DrO+s3axG5coXtERIg4yIWb0ik49349FczzWS I8+FgDwL5u1kifwFAttp/XswWwZBO3znNOiZybI/11JT/W0twTy0eFX+G2ysNTVBpToZ K+FoosB92qJt4nqZBt3/MxGQdvjIVF51JIu9/ajN1Crj+QkaYNmJagv79NoCPzpK9L4H RTKTSSI4zHK69vBOJfXY8y3AsfqrNC9c5LiWa7tacsnG9FTsiwr55gWMmVNUAqdfaK9f weHA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=dMdFQuYVxvcZn8v/8mwYCe8GX7PJ+nj7kDaLzG21jFc=; b=FBvFHaTz251FRBUQvN2A6+mVuIY7ml9bZVMzIJ25wSn/iqSR21OTpccRKhWk4ZUyBs g/KvYHl5+FaLix3rCIviGgiP8P765PLKgL9rN+K9VJr/ByKuYxeoJthR2ed7sqbPIv6e srDDP/RJ4DthvrfDXgIa+fWaddPND85544RobaxD2Ebymh3+DGXGsQA/5MzFlSRVQa9f Ijp8ECJT0b7U69vaufX09F/l79Pr9z1oe2nTV2hMMYELZhRXLip+qzWCDPEKWVuZCr/Q oUanRO8df6/wTezdHuLpFgZ3swVq3NA3wmMdf7Jlbkg2ifC/ebH0uIcPk2r8gp030gGq 5HiA== X-Gm-Message-State: AOAM530tOhbjgJtKX+QvI120+qDw2DT9KxHW8inYzx1IWJjBM8xwQX1D x14JuOZxEbRcopm1LecOtbXscQ== X-Google-Smtp-Source: ABdhPJwcjHAfTtSUxS7urJ6wJNdAgpbJNYdhbRgdmKaXU+rhJtTE9zSDtELoP3aX0EHa2HVHeel0fQ== X-Received: by 2002:a17:902:7481:b029:da:c15d:8646 with SMTP id h1-20020a1709027481b02900dac15d8646mr11221261pll.54.1607243210975; Sun, 06 Dec 2020 00:26:50 -0800 (PST) Received: from localhost.localdomain ([103.136.221.70]) by smtp.gmail.com with ESMTPSA id iq3sm6884104pjb.57.2020.12.06.00.26.43 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 06 Dec 2020 00:26:50 -0800 (PST) From: Muchun Song To: gregkh@linuxfoundation.org, rafael@kernel.org, adobriyan@gmail.com, akpm@linux-foundation.org, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, hughd@google.com, will@kernel.org, guro@fb.com, rppt@kernel.org, tglx@linutronix.de, esyr@redhat.com, peterx@redhat.com, krisman@collabora.com, surenb@google.com, avagin@openvz.org, elver@google.com, rdunlap@infradead.org, iamjoonsoo.kim@lge.com Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, Muchun Song Subject: [PATCH v2 05/12] mm: memcontrol: convert NR_SHMEM_PMDMAPPED account to pages Date: Sun, 6 Dec 2020 16:23:07 +0800 Message-Id: <20201206082318.11532-11-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20201206082318.11532-1-songmuchun@bytedance.com> References: <20201206082318.11532-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Convert NR_SHMEM_PMDMAPPED account to pages. Signed-off-by: Muchun Song --- drivers/base/node.c | 3 +-- fs/proc/meminfo.c | 2 +- mm/page_alloc.c | 3 +-- mm/rmap.c | 6 ++++-- 4 files changed, 7 insertions(+), 7 deletions(-) diff --git a/drivers/base/node.c b/drivers/base/node.c index 3e1094717e40..e5abc6144dab 100644 --- a/drivers/base/node.c +++ b/drivers/base/node.c @@ -463,8 +463,7 @@ static ssize_t node_read_meminfo(struct device *dev, , nid, K(node_page_state(pgdat, NR_ANON_THPS)), nid, K(node_page_state(pgdat, NR_SHMEM_THPS)), - nid, K(node_page_state(pgdat, NR_SHMEM_PMDMAPPED) * - HPAGE_PMD_NR), + nid, K(node_page_state(pgdat, NR_SHMEM_PMDMAPPED)), nid, K(node_page_state(pgdat, NR_FILE_THPS)), nid, K(node_page_state(pgdat, NR_FILE_PMDMAPPED) * HPAGE_PMD_NR) diff --git a/fs/proc/meminfo.c b/fs/proc/meminfo.c index b4d8a6ee822d..84886b2cc2f7 100644 --- a/fs/proc/meminfo.c +++ b/fs/proc/meminfo.c @@ -133,7 +133,7 @@ static int meminfo_proc_show(struct seq_file *m, void *v) show_val_kb(m, "ShmemHugePages: ", global_node_page_state(NR_SHMEM_THPS)); show_val_kb(m, "ShmemPmdMapped: ", - global_node_page_state(NR_SHMEM_PMDMAPPED) * HPAGE_PMD_NR); + global_node_page_state(NR_SHMEM_PMDMAPPED)); show_val_kb(m, "FileHugePages: ", global_node_page_state(NR_FILE_THPS)); show_val_kb(m, "FilePmdMapped: ", diff --git a/mm/page_alloc.c b/mm/page_alloc.c index b6a79196e870..d103513b3e4f 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -5568,8 +5568,7 @@ void show_free_areas(unsigned int filter, nodemask_t *nodemask) K(node_page_state(pgdat, NR_SHMEM)), #ifdef CONFIG_TRANSPARENT_HUGEPAGE K(node_page_state(pgdat, NR_SHMEM_THPS)), - K(node_page_state(pgdat, NR_SHMEM_PMDMAPPED) - * HPAGE_PMD_NR), + K(node_page_state(pgdat, NR_SHMEM_PMDMAPPED)), K(node_page_state(pgdat, NR_ANON_THPS)), #endif K(node_page_state(pgdat, NR_WRITEBACK_TEMP)), diff --git a/mm/rmap.c b/mm/rmap.c index f59e92e26b61..3089ad6bf468 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -1219,7 +1219,8 @@ void page_add_file_rmap(struct page *page, bool compound) if (!atomic_inc_and_test(compound_mapcount_ptr(page))) goto out; if (PageSwapBacked(page)) - __inc_node_page_state(page, NR_SHMEM_PMDMAPPED); + __mod_lruvec_page_state(page, NR_SHMEM_PMDMAPPED, + HPAGE_PMD_NR); else __inc_node_page_state(page, NR_FILE_PMDMAPPED); } else { @@ -1260,7 +1261,8 @@ static void page_remove_file_rmap(struct page *page, bool compound) if (!atomic_add_negative(-1, compound_mapcount_ptr(page))) return; if (PageSwapBacked(page)) - __dec_node_page_state(page, NR_SHMEM_PMDMAPPED); + __mod_lruvec_page_state(page, NR_SHMEM_PMDMAPPED, + -HPAGE_PMD_NR); else __dec_node_page_state(page, NR_FILE_PMDMAPPED); } else { From patchwork Sun Dec 6 08:29:43 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 11953815 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BF2FFC433FE for ; Sun, 6 Dec 2020 08:31:24 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 641BA23122 for ; Sun, 6 Dec 2020 08:31:24 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 641BA23122 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id E9E2F8D0001; Sun, 6 Dec 2020 03:31:23 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id E4E786B006E; Sun, 6 Dec 2020 03:31:23 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D3C838D0001; Sun, 6 Dec 2020 03:31:23 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0004.hostedemail.com [216.40.44.4]) by kanga.kvack.org (Postfix) with ESMTP id BACB06B006C for ; Sun, 6 Dec 2020 03:31:23 -0500 (EST) Received: from smtpin30.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 7557A1EE6 for ; Sun, 6 Dec 2020 08:31:23 +0000 (UTC) X-FDA: 77562188046.30.rain33_4713998273d4 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin30.hostedemail.com (Postfix) with ESMTP id 4D6A2180B3C83 for ; Sun, 6 Dec 2020 08:31:23 +0000 (UTC) X-HE-Tag: rain33_4713998273d4 X-Filterd-Recvd-Size: 7316 Received: from mail-pj1-f65.google.com (mail-pj1-f65.google.com [209.85.216.65]) by imf31.hostedemail.com (Postfix) with ESMTP for ; Sun, 6 Dec 2020 08:31:22 +0000 (UTC) Received: by mail-pj1-f65.google.com with SMTP id hk16so5668290pjb.4 for ; Sun, 06 Dec 2020 00:31:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=OffO4J0XNbKZKsjRiAzQ4jXTCwqv9VY0gL30Fq4+1Jk=; b=TT7+K+1nRTgpAuR6NuDugZ2v05TPgDWKZHs2Li/yPUAEEUo8Cht5NFg9H+LJ+3hKGj zMenSaZF7+z5va1V11ZGUx7xrCq3qns92bZp3V30+fwNSrARjKZYFzpvf49RgHrsYbCh DxrEIrOcuhLYmVd5re4RS0UK+p7GU/ZUK03iUhCo9VY1E95WJTL4SXgNzo/GvVrbodPg kqYlZLzqO/pxrE055y1DHdwoavQ2grMfe2Dt1W/H9xZTilQQqjTLSqMnRSSi4dV8cGsd ULW6LOt+wgmHyN3AMigBmdW9g7BgLWCUX0Y8VDaTYcMS1kdaOlfX7OrApEQTQyM4QbzZ 1wuA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=OffO4J0XNbKZKsjRiAzQ4jXTCwqv9VY0gL30Fq4+1Jk=; b=bUHQEBNDB/xClT1gQZ+hcDKYKHuVNj2oG4J9Is0WAnf6BVnL7B/LoIOsreOqbr7x6e NVXyQAa70KBSByO9RkUw2Rba/GTdDZgYNgkzd/oPIF0lbYkqtmL4i+fjcCXpXnQ8HrIz trHAnLGHumSEYH9K96CweLqx9rzG0g2d2XO9C8vQ700aPJ74avZi1C79aEWsb5UjDR9B Eq7uAIW7mJMyJl0MyZb7ffDXQo0ildqLdX4MAie4S4rljm6nKAHMoEG8Jpmm6YhVKEG8 S/SCMp6d0lh2c2u2N1wFXsudhft372DGSx+50HLYy6lEWGx95u1BrIx9Xk5/hwOCbEBI pFoA== X-Gm-Message-State: AOAM531WzqvYu5KxutONw3XxQl/w6knzU6aJ08id/bQVwLbCW13ACnHm A+/w7jVyLadLvgZdak5vGl4flA== X-Google-Smtp-Source: ABdhPJxJ0PIrT/1KnctdSa8KJHvOTFN6Sq6NrbQoXJzEPP2nt5Nr+b+oKu79Z4xx013lCL/hx6OPvQ== X-Received: by 2002:a17:90a:c505:: with SMTP id k5mr11879203pjt.53.1607243481862; Sun, 06 Dec 2020 00:31:21 -0800 (PST) Received: from localhost.localdomain ([103.136.221.70]) by smtp.gmail.com with ESMTPSA id c2sm10229107pfa.59.2020.12.06.00.31.13 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 06 Dec 2020 00:31:21 -0800 (PST) From: Muchun Song To: gregkh@linuxfoundation.org, rafael@kernel.org, adobriyan@gmail.com, akpm@linux-foundation.org, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, hughd@google.com, will@kernel.org, guro@fb.com, rppt@kernel.org, tglx@linutronix.de, esyr@redhat.com, peterx@redhat.com, krisman@collabora.com, surenb@google.com, avagin@openvz.org, elver@google.com, rdunlap@infradead.org, iamjoonsoo.kim@lge.com Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, Muchun Song Subject: [PATCH v2 07/12] mm: memcontrol: convert kernel stack account to bytes Date: Sun, 6 Dec 2020 16:29:43 +0800 Message-Id: <20201206082948.11812-1-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The kernel stack account is the one that counts in KiB. This patch convert it from KiB to byte. Signed-off-by: Muchun Song --- drivers/base/node.c | 2 +- fs/proc/meminfo.c | 2 +- include/linux/mmzone.h | 2 +- kernel/fork.c | 8 ++++---- mm/memcontrol.c | 2 +- mm/page_alloc.c | 2 +- 6 files changed, 9 insertions(+), 9 deletions(-) diff --git a/drivers/base/node.c b/drivers/base/node.c index f77652e6339f..92a75bad35c9 100644 --- a/drivers/base/node.c +++ b/drivers/base/node.c @@ -446,7 +446,7 @@ static ssize_t node_read_meminfo(struct device *dev, nid, K(node_page_state(pgdat, NR_FILE_MAPPED)), nid, K(node_page_state(pgdat, NR_ANON_MAPPED)), nid, K(i.sharedram), - nid, node_page_state(pgdat, NR_KERNEL_STACK_KB), + nid, node_page_state(pgdat, NR_KERNEL_STACK_B) / SZ_1K, #ifdef CONFIG_SHADOW_CALL_STACK nid, node_page_state(pgdat, NR_KERNEL_SCS_KB), #endif diff --git a/fs/proc/meminfo.c b/fs/proc/meminfo.c index 5a83012d8b72..799a537d4218 100644 --- a/fs/proc/meminfo.c +++ b/fs/proc/meminfo.c @@ -101,7 +101,7 @@ static int meminfo_proc_show(struct seq_file *m, void *v) show_val_kb(m, "SReclaimable: ", sreclaimable); show_val_kb(m, "SUnreclaim: ", sunreclaim); seq_printf(m, "KernelStack: %8lu kB\n", - global_node_page_state(NR_KERNEL_STACK_KB)); + global_node_page_state(NR_KERNEL_STACK_B) / SZ_1K); #ifdef CONFIG_SHADOW_CALL_STACK seq_printf(m, "ShadowCallStack:%8lu kB\n", global_node_page_state(NR_KERNEL_SCS_KB)); diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index 15132adaa233..bd34416293ec 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -202,7 +202,7 @@ enum node_stat_item { NR_KERNEL_MISC_RECLAIMABLE, /* reclaimable non-slab kernel pages */ NR_FOLL_PIN_ACQUIRED, /* via: pin_user_page(), gup flag: FOLL_PIN */ NR_FOLL_PIN_RELEASED, /* pages returned via unpin_user_page() */ - NR_KERNEL_STACK_KB, /* measured in KiB */ + NR_KERNEL_STACK_B, /* measured in byte */ #if IS_ENABLED(CONFIG_SHADOW_CALL_STACK) NR_KERNEL_SCS_KB, /* measured in KiB */ #endif diff --git a/kernel/fork.c b/kernel/fork.c index 345f378e104d..2913d7c43dcb 100644 --- a/kernel/fork.c +++ b/kernel/fork.c @@ -382,11 +382,11 @@ static void account_kernel_stack(struct task_struct *tsk, int account) /* All stack pages are in the same node. */ if (vm) - mod_lruvec_page_state(vm->pages[0], NR_KERNEL_STACK_KB, - account * (THREAD_SIZE / 1024)); + mod_lruvec_page_state(vm->pages[0], NR_KERNEL_STACK_B, + account * THREAD_SIZE); else - mod_lruvec_kmem_state(stack, NR_KERNEL_STACK_KB, - account * (THREAD_SIZE / 1024)); + mod_lruvec_kmem_state(stack, NR_KERNEL_STACK_B, + account * THREAD_SIZE); } static int memcg_charge_kernel_stack(struct task_struct *tsk) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 6d4365d2fd1c..48d70c1ad301 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -1500,7 +1500,7 @@ struct memory_stat { static const struct memory_stat memory_stats[] = { { "anon", PAGE_SIZE, NR_ANON_MAPPED }, { "file", PAGE_SIZE, NR_FILE_PAGES }, - { "kernel_stack", 1024, NR_KERNEL_STACK_KB }, + { "kernel_stack", 1, NR_KERNEL_STACK_B }, { "percpu", 1, MEMCG_PERCPU_B }, { "sock", PAGE_SIZE, MEMCG_SOCK }, { "shmem", PAGE_SIZE, NR_SHMEM }, diff --git a/mm/page_alloc.c b/mm/page_alloc.c index d103513b3e4f..d2821ba7f682 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -5572,7 +5572,7 @@ void show_free_areas(unsigned int filter, nodemask_t *nodemask) K(node_page_state(pgdat, NR_ANON_THPS)), #endif K(node_page_state(pgdat, NR_WRITEBACK_TEMP)), - node_page_state(pgdat, NR_KERNEL_STACK_KB), + node_page_state(pgdat, NR_KERNEL_STACK_B) / SZ_1K, #ifdef CONFIG_SHADOW_CALL_STACK node_page_state(pgdat, NR_KERNEL_SCS_KB), #endif From patchwork Sun Dec 6 08:29:44 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 11953829 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D7238C433FE for ; Sun, 6 Dec 2020 08:31:32 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 6EF1A2311B for ; Sun, 6 Dec 2020 08:31:32 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6EF1A2311B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 10A0C6B006C; Sun, 6 Dec 2020 03:31:32 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 0BB478D0003; Sun, 6 Dec 2020 03:31:32 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EC3B48D0002; Sun, 6 Dec 2020 03:31:31 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0042.hostedemail.com [216.40.44.42]) by kanga.kvack.org (Postfix) with ESMTP id D76766B006C for ; Sun, 6 Dec 2020 03:31:31 -0500 (EST) Received: from smtpin01.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 99F5F1EE6 for ; Sun, 6 Dec 2020 08:31:31 +0000 (UTC) X-FDA: 77562188382.01.comb00_3c0fbf4273d4 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin01.hostedemail.com (Postfix) with ESMTP id 6855D100480A0 for ; Sun, 6 Dec 2020 08:31:31 +0000 (UTC) X-HE-Tag: comb00_3c0fbf4273d4 X-Filterd-Recvd-Size: 6495 Received: from mail-pl1-f193.google.com (mail-pl1-f193.google.com [209.85.214.193]) by imf49.hostedemail.com (Postfix) with ESMTP for ; Sun, 6 Dec 2020 08:31:30 +0000 (UTC) Received: by mail-pl1-f193.google.com with SMTP id t18so5531040plo.0 for ; Sun, 06 Dec 2020 00:31:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=IzVbUHzHNutFlJQXX/d4rCoABm7bks/8Sv9HD05GfEo=; b=XreRyw7hI2GhJlkkEvcGA2V6QiV/heX0OqkEPDJNiVRh13DkcCk2CnH8x5TOVy3I+s y02dVE1gUP3ZD3/imp3klJjjP+4ilXqHVYYUm7/LusipLgC3A7YPIWeCGylkxbl1GKXc bztOgUedenKv35sfCpKz81oU1YqoCh3PJmBLbhMW1zdX/wqEQQ4whVobfMBxoHctz4jI VomjFpsE1D/Mvz+q4IZvgsTAQmw9d8bvdRlG1exX9LrOzSPmLBy5AblhcxgFI2uh26+6 rBWdMDiekPPNL+d04iEj9lketLc4Aex6bkD5FSH2c6a7TRQzehD3C1UViO1ys7fBoe5Q +jdw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=IzVbUHzHNutFlJQXX/d4rCoABm7bks/8Sv9HD05GfEo=; b=HbxcGZGKZfRHwpI2WOEEldUg/N3tDY9efdQ4FIIHLUMAa30AzeiD1TI1/HuzoCht57 rw9ieinQYs9/9pnFyBcPslKKF+Tsfx9TwgQGcEWi+e7yHAzcFO/AokA5txvpF8fW3ywg 6idUuE3DTEBBKeGOAFm7GcjzZcN/bXafOFIBi8jsum7Eio2HeIJJMU4DZZEYFXFPKRI3 Y5BXuSU+bUj9BN/j1dnXH37AM6GPrNyVpKz4a71huQAI/E+v2YwssdHIZTL1KkTe1a4m 2U0lZNhNdgS6a94nkR7u9hhJH24/gBXERJs27HG8A4m/QUFwnAtWPOVm3yVt2vDOigJ2 xSdA== X-Gm-Message-State: AOAM533DqfArdmo+IzUcPisYAWJL59qLt4DoFUY/UkP81W4AzbRbm0sa OazFc9cjIyRwYWFwOn4cpJ08YQ== X-Google-Smtp-Source: ABdhPJynwpxOsYQ/ww5Wmc/2+KLF/FJHt01PK4yorEkHsqzIB9RYPLFGWtFzIdRxoJlkCVCYPoX9Pw== X-Received: by 2002:a17:902:bd84:b029:da:17d0:e754 with SMTP id q4-20020a170902bd84b02900da17d0e754mr10655163pls.68.1607243490117; Sun, 06 Dec 2020 00:31:30 -0800 (PST) Received: from localhost.localdomain ([103.136.221.70]) by smtp.gmail.com with ESMTPSA id c2sm10229107pfa.59.2020.12.06.00.31.22 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 06 Dec 2020 00:31:29 -0800 (PST) From: Muchun Song To: gregkh@linuxfoundation.org, rafael@kernel.org, adobriyan@gmail.com, akpm@linux-foundation.org, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, hughd@google.com, will@kernel.org, guro@fb.com, rppt@kernel.org, tglx@linutronix.de, esyr@redhat.com, peterx@redhat.com, krisman@collabora.com, surenb@google.com, avagin@openvz.org, elver@google.com, rdunlap@infradead.org, iamjoonsoo.kim@lge.com Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, Muchun Song Subject: [PATCH v2 08/12] mm: memcontrol: convert NR_KERNEL_SCS_KB account to bytes Date: Sun, 6 Dec 2020 16:29:44 +0800 Message-Id: <20201206082948.11812-2-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20201206082948.11812-1-songmuchun@bytedance.com> References: <20201206082948.11812-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Convert NR_KERNEL_SCS_KB account to bytes Signed-off-by: Muchun Song --- drivers/base/node.c | 2 +- fs/proc/meminfo.c | 2 +- include/linux/mmzone.h | 2 +- kernel/scs.c | 4 ++-- mm/page_alloc.c | 2 +- 5 files changed, 6 insertions(+), 6 deletions(-) diff --git a/drivers/base/node.c b/drivers/base/node.c index 92a75bad35c9..bc01ce0b2fcd 100644 --- a/drivers/base/node.c +++ b/drivers/base/node.c @@ -448,7 +448,7 @@ static ssize_t node_read_meminfo(struct device *dev, nid, K(i.sharedram), nid, node_page_state(pgdat, NR_KERNEL_STACK_B) / SZ_1K, #ifdef CONFIG_SHADOW_CALL_STACK - nid, node_page_state(pgdat, NR_KERNEL_SCS_KB), + nid, node_page_state(pgdat, NR_KERNEL_SCS_B) / SZ_1K, #endif nid, K(sum_zone_node_page_state(nid, NR_PAGETABLE)), nid, 0UL, diff --git a/fs/proc/meminfo.c b/fs/proc/meminfo.c index 799a537d4218..69895e83d4fc 100644 --- a/fs/proc/meminfo.c +++ b/fs/proc/meminfo.c @@ -104,7 +104,7 @@ static int meminfo_proc_show(struct seq_file *m, void *v) global_node_page_state(NR_KERNEL_STACK_B) / SZ_1K); #ifdef CONFIG_SHADOW_CALL_STACK seq_printf(m, "ShadowCallStack:%8lu kB\n", - global_node_page_state(NR_KERNEL_SCS_KB)); + global_node_page_state(NR_KERNEL_SCS_B) / SZ_1K); #endif show_val_kb(m, "PageTables: ", global_zone_page_state(NR_PAGETABLE)); diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index bd34416293ec..1f9c83778629 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -204,7 +204,7 @@ enum node_stat_item { NR_FOLL_PIN_RELEASED, /* pages returned via unpin_user_page() */ NR_KERNEL_STACK_B, /* measured in byte */ #if IS_ENABLED(CONFIG_SHADOW_CALL_STACK) - NR_KERNEL_SCS_KB, /* measured in KiB */ + NR_KERNEL_SCS_B, /* measured in byte */ #endif NR_VM_NODE_STAT_ITEMS }; diff --git a/kernel/scs.c b/kernel/scs.c index 4ff4a7ba0094..8db89c932ddc 100644 --- a/kernel/scs.c +++ b/kernel/scs.c @@ -17,8 +17,8 @@ static void __scs_account(void *s, int account) { struct page *scs_page = virt_to_page(s); - mod_node_page_state(page_pgdat(scs_page), NR_KERNEL_SCS_KB, - account * (SCS_SIZE / SZ_1K)); + mod_node_page_state(page_pgdat(scs_page), NR_KERNEL_SCS_B, + account * SCS_SIZE); } static void *scs_alloc(int node) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index d2821ba7f682..58916b3afdab 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -5574,7 +5574,7 @@ void show_free_areas(unsigned int filter, nodemask_t *nodemask) K(node_page_state(pgdat, NR_WRITEBACK_TEMP)), node_page_state(pgdat, NR_KERNEL_STACK_B) / SZ_1K, #ifdef CONFIG_SHADOW_CALL_STACK - node_page_state(pgdat, NR_KERNEL_SCS_KB), + node_page_state(pgdat, NR_KERNEL_SCS_B) / SZ_1K, #endif pgdat->kswapd_failures >= MAX_RECLAIM_RETRIES ? "yes" : "no"); From patchwork Sun Dec 6 08:29:45 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 11953831 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 54E2CC433FE for ; Sun, 6 Dec 2020 08:31:41 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 042A82311B for ; Sun, 6 Dec 2020 08:31:40 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 042A82311B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 9E4798D0002; Sun, 6 Dec 2020 03:31:40 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 9BDC16B0071; Sun, 6 Dec 2020 03:31:40 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8CF728D0002; Sun, 6 Dec 2020 03:31:40 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0168.hostedemail.com [216.40.44.168]) by kanga.kvack.org (Postfix) with ESMTP id 77ECD6B006E for ; Sun, 6 Dec 2020 03:31:40 -0500 (EST) Received: from smtpin20.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 40ED11EE6 for ; Sun, 6 Dec 2020 08:31:40 +0000 (UTC) X-FDA: 77562188760.20.anger77_0e16b6b273d4 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin20.hostedemail.com (Postfix) with ESMTP id 1F39D180C0609 for ; Sun, 6 Dec 2020 08:31:40 +0000 (UTC) X-HE-Tag: anger77_0e16b6b273d4 X-Filterd-Recvd-Size: 6599 Received: from mail-pg1-f195.google.com (mail-pg1-f195.google.com [209.85.215.195]) by imf42.hostedemail.com (Postfix) with ESMTP for ; Sun, 6 Dec 2020 08:31:39 +0000 (UTC) Received: by mail-pg1-f195.google.com with SMTP id t37so6366699pga.7 for ; Sun, 06 Dec 2020 00:31:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=jxjzCofAcULr7bnQMbpOvZCW6tNXMo722rzO5X3X+g0=; b=wNv4qDSehcayO/ihogNTwYkGmNfh5RiiwJJPy5Ul8/X5rRuoAcaO9IWiT/zhh7c3zE DkQdTnRVChsF0HNvUG0ZpVzWqoGDxZHC6UdxNFd1esXfxLRYG5JPaZt/ehJmxRKGxRcy j2K7ImVTdAGGUwQzHwmDDriZDLsK3oJ2kzyams7iLzW6JmsFMSnV6U0Q5tVhmTk88NLQ P1mxFKPCxWivQAZSOj3l5kb7OdCK33TsD2BMo7RzFcUXbMtYUb38SlvmlQtvx8PkvIV3 EV0/A75iolf7+KiwyhLHgs4vUFCGQa7dSkHBYsNwoQvuzRX5BJsEWtgU5PNGExnjgrB6 0zag== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=jxjzCofAcULr7bnQMbpOvZCW6tNXMo722rzO5X3X+g0=; b=WGgpbt+YQmDAlxj00PtToD6vIBalaDE+AiqDlkcL69iDSf51dy/YoPn43+p41Hdfs5 mE3JmXzc4DoEyDpBFLT8ZuBi4ReO0AN19ZO7xDLxA58JKwrsxS1+xF2c0b3LdCQ/hQR4 MCek3iIng8MOjPIWsvxwdWE+qxcv90570Q3o75muYHr4dZAOfhFi+XRJL0U2oFrCTIUE CSDlPx0J+iWxdsuQW32qZl6jKk1pAFkE/ALxXjnSYlyrduRe9EkMz3TDR5GS2nkGQ7vB 203UDFTRlSTFrhogJ5ssIdUbG9hndjuFz0FdGccv1StND9WRlDzlIwzJPUbudghP9W4R TRNA== X-Gm-Message-State: AOAM531jxxQPExDNu1cmAkoXeKLsPh6NQcY6VvVHEEJpu8diT9yHhLz7 qimysh4a50S1uzcG613e2aiPvA== X-Google-Smtp-Source: ABdhPJwo9cB+Tm2M3Iwyh9yt/rFYa5lNsd3ZIr0YFpCtVd3XQ5n1KNw6GWOLG8mW/TLUg0oyOGP4rA== X-Received: by 2002:a62:ab0f:0:b029:197:f771:fe8e with SMTP id p15-20020a62ab0f0000b0290197f771fe8emr11234671pff.65.1607243498804; Sun, 06 Dec 2020 00:31:38 -0800 (PST) Received: from localhost.localdomain ([103.136.221.70]) by smtp.gmail.com with ESMTPSA id c2sm10229107pfa.59.2020.12.06.00.31.30 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 06 Dec 2020 00:31:38 -0800 (PST) From: Muchun Song To: gregkh@linuxfoundation.org, rafael@kernel.org, adobriyan@gmail.com, akpm@linux-foundation.org, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, hughd@google.com, will@kernel.org, guro@fb.com, rppt@kernel.org, tglx@linutronix.de, esyr@redhat.com, peterx@redhat.com, krisman@collabora.com, surenb@google.com, avagin@openvz.org, elver@google.com, rdunlap@infradead.org, iamjoonsoo.kim@lge.com Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, Muchun Song Subject: [PATCH v2 09/12] mm: memcontrol: convert vmstat slab counters to bytes Date: Sun, 6 Dec 2020 16:29:45 +0800 Message-Id: <20201206082948.11812-3-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20201206082948.11812-1-songmuchun@bytedance.com> References: <20201206082948.11812-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: the global and per-node counters are stored in pages, however memcg and lruvec counters are stored in bytes. This scheme looks weird. So convert all vmstat slab counters to bytes. Signed-off-by: Muchun Song --- include/linux/vmstat.h | 17 ++++++++++------- mm/vmstat.c | 21 ++++++++++----------- 2 files changed, 20 insertions(+), 18 deletions(-) diff --git a/include/linux/vmstat.h b/include/linux/vmstat.h index 322dcbfcc933..fd1a3d5d4926 100644 --- a/include/linux/vmstat.h +++ b/include/linux/vmstat.h @@ -197,18 +197,26 @@ static inline unsigned long global_node_page_state_pages(enum node_stat_item item) { long x = atomic_long_read(&vm_node_stat[item]); + #ifdef CONFIG_SMP if (x < 0) x = 0; #endif + if (vmstat_item_in_bytes(item)) + x >>= PAGE_SHIFT; return x; } static inline unsigned long global_node_page_state(enum node_stat_item item) { - VM_WARN_ON_ONCE(vmstat_item_in_bytes(item)); + long x = atomic_long_read(&vm_node_stat[item]); - return global_node_page_state_pages(item); + VM_WARN_ON_ONCE(vmstat_item_in_bytes(item)); +#ifdef CONFIG_SMP + if (x < 0) + x = 0; +#endif + return x; } static inline unsigned long zone_page_state(struct zone *zone, @@ -312,11 +320,6 @@ static inline void __mod_zone_page_state(struct zone *zone, static inline void __mod_node_page_state(struct pglist_data *pgdat, enum node_stat_item item, int delta) { - if (vmstat_item_in_bytes(item)) { - VM_WARN_ON_ONCE(delta & (PAGE_SIZE - 1)); - delta >>= PAGE_SHIFT; - } - node_page_state_add(delta, pgdat, item); } diff --git a/mm/vmstat.c b/mm/vmstat.c index 8d77ee426e22..7fb0c7cb9516 100644 --- a/mm/vmstat.c +++ b/mm/vmstat.c @@ -345,11 +345,6 @@ void __mod_node_page_state(struct pglist_data *pgdat, enum node_stat_item item, long x; long t; - if (vmstat_item_in_bytes(item)) { - VM_WARN_ON_ONCE(delta & (PAGE_SIZE - 1)); - delta >>= PAGE_SHIFT; - } - x = delta + __this_cpu_read(*p); t = __this_cpu_read(pcp->stat_threshold); @@ -554,11 +549,6 @@ static inline void mod_node_state(struct pglist_data *pgdat, s8 __percpu *p = pcp->vm_node_stat_diff + item; long o, n, t, z; - if (vmstat_item_in_bytes(item)) { - VM_WARN_ON_ONCE(delta & (PAGE_SIZE - 1)); - delta >>= PAGE_SHIFT; - } - do { z = 0; /* overflow to node counters */ @@ -1012,19 +1002,28 @@ unsigned long node_page_state_pages(struct pglist_data *pgdat, enum node_stat_item item) { long x = atomic_long_read(&pgdat->vm_stat[item]); + #ifdef CONFIG_SMP if (x < 0) x = 0; #endif + if (vmstat_item_in_bytes(item)) + x >>= PAGE_SHIFT; return x; } unsigned long node_page_state(struct pglist_data *pgdat, enum node_stat_item item) { + long x = atomic_long_read(&pgdat->vm_stat[item]); + VM_WARN_ON_ONCE(vmstat_item_in_bytes(item)); - return node_page_state_pages(pgdat, item); +#ifdef CONFIG_SMP + if (x < 0) + x = 0; +#endif + return x; } #endif From patchwork Sun Dec 6 08:29:46 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 11953833 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CF900C433FE for ; Sun, 6 Dec 2020 08:31:49 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 6F56423108 for ; Sun, 6 Dec 2020 08:31:49 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6F56423108 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 06DBA8D0003; Sun, 6 Dec 2020 03:31:49 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 0448B6B0071; Sun, 6 Dec 2020 03:31:48 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E772A8D0003; Sun, 6 Dec 2020 03:31:48 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0110.hostedemail.com [216.40.44.110]) by kanga.kvack.org (Postfix) with ESMTP id D121A6B006E for ; Sun, 6 Dec 2020 03:31:48 -0500 (EST) Received: from smtpin07.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 93486181AEF10 for ; Sun, 6 Dec 2020 08:31:48 +0000 (UTC) X-FDA: 77562189096.07.bean13_0e002ca273d4 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin07.hostedemail.com (Postfix) with ESMTP id 6F81F1803F9AA for ; Sun, 6 Dec 2020 08:31:48 +0000 (UTC) X-HE-Tag: bean13_0e002ca273d4 X-Filterd-Recvd-Size: 8766 Received: from mail-pj1-f65.google.com (mail-pj1-f65.google.com [209.85.216.65]) by imf05.hostedemail.com (Postfix) with ESMTP for ; Sun, 6 Dec 2020 08:31:47 +0000 (UTC) Received: by mail-pj1-f65.google.com with SMTP id m5so5671196pjv.5 for ; Sun, 06 Dec 2020 00:31:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=cL9BVbVQLgAGW6GpXAhL1X3CA8w9xPKU91tirmQZrTs=; b=x8fvuLarCFgaQTRPR4ffz/JJqgV+fhAwSGezh2EEVl+aDi7Dz1rixLAZS3bee4/Cjk yTGJNzm19fwdVNSUefGAt/1Wpp4o+VzhKWewiGyppNqXwi0CgEQsI/hZ1WPDO5K5ydIQ eRGdBfLsH4lswDedLsHI/qspqiRz4LmeZfVhabRi4uvmhk4Dq7H60Zlrvh1Rt7dIufz3 J1FWj8pGKuya9oJVeHptAgoqD1GoWwEJChbD0OAh6qXkKyV5GZqTOeqfmnt/JzlOVNcR rYXgLQfzO6U/LyVw7BGLIdIbkmEnYVEFW95PERZbdV2rsZqnmmBlkC2DdbYTpQk4MRQ9 H+4w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=cL9BVbVQLgAGW6GpXAhL1X3CA8w9xPKU91tirmQZrTs=; b=iX4k5fqx3EepMk7MltSU6hJGNDfDS/ft0CJS2yncPK9yLfBvckrEkeWeIEPufQfKVx QpqNDIIKrBo5R29UaK3Acgs1vv1xqfjtD48FOHZOkMYQpnW7ehLK1jMtuOo0sZJVDiTk FUdn4nxV2XyGT0Me+i/2jwP33cFtf+0/FMInsu+QIcgD3gy6xEktPvslXrD+DNGYfyyG I+5oZJFZgVpRuA9wo4PJMBSGbJ/HnDUSoN7cDW1wNejYZBk+pHKAhSzqSSUJKIm+xMiq jbcc/FtHyLoU3wRVCdTUw/g19AZK35z3z4/t42rf6g/ewhHggyEzUAPHrN2oaamrl/8X l1Pg== X-Gm-Message-State: AOAM533MbfilCbkNriQYI70vyafs6a75MDAfIAPnSnKm9VyPPtaYPwNm vypW3oGuruKNBPRlJ2EEOgWbAg== X-Google-Smtp-Source: ABdhPJzglciP6WEF9pyhvIJXUJQdKJTvf5/6EhbcvbjvVcthuUO6f4w5qWd4LOe+Qy42RJZ84jcY1Q== X-Received: by 2002:a17:902:be11:b029:da:ba30:5791 with SMTP id r17-20020a170902be11b02900daba305791mr11119985pls.13.1607243507099; Sun, 06 Dec 2020 00:31:47 -0800 (PST) Received: from localhost.localdomain ([103.136.221.70]) by smtp.gmail.com with ESMTPSA id c2sm10229107pfa.59.2020.12.06.00.31.39 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 06 Dec 2020 00:31:46 -0800 (PST) From: Muchun Song To: gregkh@linuxfoundation.org, rafael@kernel.org, adobriyan@gmail.com, akpm@linux-foundation.org, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, hughd@google.com, will@kernel.org, guro@fb.com, rppt@kernel.org, tglx@linutronix.de, esyr@redhat.com, peterx@redhat.com, krisman@collabora.com, surenb@google.com, avagin@openvz.org, elver@google.com, rdunlap@infradead.org, iamjoonsoo.kim@lge.com Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, Muchun Song Subject: [PATCH v2 10/12] mm: memcontrol: scale stat_threshold for byted-sized vmstat Date: Sun, 6 Dec 2020 16:29:46 +0800 Message-Id: <20201206082948.11812-4-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20201206082948.11812-1-songmuchun@bytedance.com> References: <20201206082948.11812-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Some vmstat counters are being accounted in bytes not pages, so the stat_threshold should also scale to bytes. The vmstat counters are already long type for memcg (can reference to struct lruvec_stat). For the global per-node vmstat counters also can scale to long. But the maximum vmstat threshold is 125, so the type of s32 is enough. Signed-off-by: Muchun Song --- include/linux/mmzone.h | 17 ++++++----------- include/linux/vmstat.h | 1 - mm/vmstat.c | 24 +++++++++++++----------- 3 files changed, 19 insertions(+), 23 deletions(-) diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index 1f9c83778629..d53328551225 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -216,17 +216,12 @@ enum node_stat_item { */ static __always_inline bool vmstat_item_in_bytes(int idx) { - /* - * Global and per-node slab counters track slab pages. - * It's expected that changes are multiples of PAGE_SIZE. - * Internally values are stored in pages. - * - * Per-memcg and per-lruvec counters track memory, consumed - * by individual slab objects. These counters are actually - * byte-precise. - */ return (idx == NR_SLAB_RECLAIMABLE_B || - idx == NR_SLAB_UNRECLAIMABLE_B); + idx == NR_SLAB_UNRECLAIMABLE_B || +#if IS_ENABLED(CONFIG_SHADOW_CALL_STACK) + idx == NR_KERNEL_SCS_B || +#endif + idx == NR_KERNEL_STACK_B); } /* @@ -340,7 +335,7 @@ struct per_cpu_pageset { struct per_cpu_nodestat { s8 stat_threshold; - s8 vm_node_stat_diff[NR_VM_NODE_STAT_ITEMS]; + s32 vm_node_stat_diff[NR_VM_NODE_STAT_ITEMS]; }; #endif /* !__GENERATING_BOUNDS.H */ diff --git a/include/linux/vmstat.h b/include/linux/vmstat.h index fd1a3d5d4926..afd84dc2398c 100644 --- a/include/linux/vmstat.h +++ b/include/linux/vmstat.h @@ -211,7 +211,6 @@ static inline unsigned long global_node_page_state(enum node_stat_item item) { long x = atomic_long_read(&vm_node_stat[item]); - VM_WARN_ON_ONCE(vmstat_item_in_bytes(item)); #ifdef CONFIG_SMP if (x < 0) x = 0; diff --git a/mm/vmstat.c b/mm/vmstat.c index 7fb0c7cb9516..25751b1d8e2e 100644 --- a/mm/vmstat.c +++ b/mm/vmstat.c @@ -341,13 +341,15 @@ void __mod_node_page_state(struct pglist_data *pgdat, enum node_stat_item item, long delta) { struct per_cpu_nodestat __percpu *pcp = pgdat->per_cpu_nodestats; - s8 __percpu *p = pcp->vm_node_stat_diff + item; + s32 __percpu *p = pcp->vm_node_stat_diff + item; long x; long t; x = delta + __this_cpu_read(*p); t = __this_cpu_read(pcp->stat_threshold); + if (vmstat_item_in_bytes(item)) + t <<= PAGE_SHIFT; if (unlikely(abs(x) > t)) { node_page_state_add(x, pgdat, item); @@ -399,15 +401,15 @@ void __inc_zone_state(struct zone *zone, enum zone_stat_item item) void __inc_node_state(struct pglist_data *pgdat, enum node_stat_item item) { struct per_cpu_nodestat __percpu *pcp = pgdat->per_cpu_nodestats; - s8 __percpu *p = pcp->vm_node_stat_diff + item; - s8 v, t; + s32 __percpu *p = pcp->vm_node_stat_diff + item; + s32 v, t; VM_WARN_ON_ONCE(vmstat_item_in_bytes(item)); v = __this_cpu_inc_return(*p); t = __this_cpu_read(pcp->stat_threshold); if (unlikely(v > t)) { - s8 overstep = t >> 1; + s32 overstep = t >> 1; node_page_state_add(v + overstep, pgdat, item); __this_cpu_write(*p, -overstep); @@ -445,8 +447,8 @@ void __dec_zone_state(struct zone *zone, enum zone_stat_item item) void __dec_node_state(struct pglist_data *pgdat, enum node_stat_item item) { struct per_cpu_nodestat __percpu *pcp = pgdat->per_cpu_nodestats; - s8 __percpu *p = pcp->vm_node_stat_diff + item; - s8 v, t; + s32 __percpu *p = pcp->vm_node_stat_diff + item; + s32 v, t; VM_WARN_ON_ONCE(vmstat_item_in_bytes(item)); @@ -546,7 +548,7 @@ static inline void mod_node_state(struct pglist_data *pgdat, enum node_stat_item item, int delta, int overstep_mode) { struct per_cpu_nodestat __percpu *pcp = pgdat->per_cpu_nodestats; - s8 __percpu *p = pcp->vm_node_stat_diff + item; + s32 __percpu *p = pcp->vm_node_stat_diff + item; long o, n, t, z; do { @@ -563,6 +565,8 @@ static inline void mod_node_state(struct pglist_data *pgdat, * for all cpus in a node. */ t = this_cpu_read(pcp->stat_threshold); + if (vmstat_item_in_bytes(item)) + t <<= PAGE_SHIFT; o = this_cpu_read(*p); n = delta + o; @@ -829,7 +833,7 @@ static int refresh_cpu_vm_stats(bool do_pagesets) struct per_cpu_nodestat __percpu *p = pgdat->per_cpu_nodestats; for (i = 0; i < NR_VM_NODE_STAT_ITEMS; i++) { - int v; + s32 v; v = this_cpu_xchg(p->vm_node_stat_diff[i], 0); if (v) { @@ -899,7 +903,7 @@ void cpu_vm_stats_fold(int cpu) for (i = 0; i < NR_VM_NODE_STAT_ITEMS; i++) if (p->vm_node_stat_diff[i]) { - int v; + s32 v; v = p->vm_node_stat_diff[i]; p->vm_node_stat_diff[i] = 0; @@ -1017,8 +1021,6 @@ unsigned long node_page_state(struct pglist_data *pgdat, { long x = atomic_long_read(&pgdat->vm_stat[item]); - VM_WARN_ON_ONCE(vmstat_item_in_bytes(item)); - #ifdef CONFIG_SMP if (x < 0) x = 0; From patchwork Sun Dec 6 08:29:47 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 11953835 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 961EEC4361B for ; Sun, 6 Dec 2020 08:31:59 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 43AF623108 for ; Sun, 6 Dec 2020 08:31:59 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 43AF623108 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id DD8E98D0005; Sun, 6 Dec 2020 03:31:58 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D64836B0071; Sun, 6 Dec 2020 03:31:58 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C7C538D0005; Sun, 6 Dec 2020 03:31:58 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0125.hostedemail.com [216.40.44.125]) by kanga.kvack.org (Postfix) with ESMTP id AC9D56B006E for ; Sun, 6 Dec 2020 03:31:58 -0500 (EST) Received: from smtpin09.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 5DB23181AEF10 for ; Sun, 6 Dec 2020 08:31:58 +0000 (UTC) X-FDA: 77562189516.09.jump49_48134bb273d4 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin09.hostedemail.com (Postfix) with ESMTP id 444B8180AD806 for ; Sun, 6 Dec 2020 08:31:58 +0000 (UTC) X-HE-Tag: jump49_48134bb273d4 X-Filterd-Recvd-Size: 9699 Received: from mail-pf1-f196.google.com (mail-pf1-f196.google.com [209.85.210.196]) by imf09.hostedemail.com (Postfix) with ESMTP for ; Sun, 6 Dec 2020 08:31:57 +0000 (UTC) Received: by mail-pf1-f196.google.com with SMTP id f9so6322322pfc.11 for ; Sun, 06 Dec 2020 00:31:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=KwRaMRI+Sr6xQo3P1J4+HoZOIYaqIcm7QaRM3zeaHfM=; b=PRd0vzD/2Sfkm55VZCIWYO1SYAuzM56m2d7biop5RolLR2ogMnVEGiPJhLhDPDObMW NgaZSB9qXSMiN+B7CM1wNTUxoGLnKFhBTm1vJI+vyEnU8CnO4gDi1iY7f36ef/ReRVDo tYJfaEHnhV61ky7Ql+2u3VfR7XAaCOxb0ltJsDBQhaoZr8r98oJbn3uqUcoiyquGen98 Y/U+6ZzOwPmy0jIW2h0lzdvQQn9+63sqBSV650Sq1ENh3kBNyvHct57dyqPXdGB50ruD OVVW9URrc5fGzg3xl32gVd1mPix6CgFbHzuJstcUCxP7fSs6IiomiZ+BKJ1qr0Acgvg9 iTYA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=KwRaMRI+Sr6xQo3P1J4+HoZOIYaqIcm7QaRM3zeaHfM=; b=nfHfV4KaUOOVy10ajXcJASoh72Z0A/YjhybFRwdhm9t5BV24FFZappmisTXiPmwQ+y DMr9EEO0W0c6Vh/76en+IaX47TW4RFiWYf5/OGmvfnqqwvwzm+85U3BOPslYBmOyTRBa XzaeBlnp3aEMkgjWVqCttVnRqVn5BbRC/dcr1kT636GP1mAXyuZJh2Tc4n4kIUgun7Na CjhJaJoU7mItR4/Mig7ko0afSNGnb6A9/+XZT9KSrSHJOljmDuJi178hoGdKuITmVOcJ zPELTXVrzK2ArC+c5TyxMxHhc8ArzTnpfUIDLA8JR5C+lM4fglrO52unuNAItUq+geP2 75LQ== X-Gm-Message-State: AOAM531chLxu9RSBiZvPPjPQoqurUhZHY6OAsfCUu7MAcQCP2BFFNJVT 80NYszgZUbn4ORtwDsDyz5+p8g== X-Google-Smtp-Source: ABdhPJxdUC6z9YY2WIIE315X8TQ7+1z6todWeE3J1HIe1vWBnZ5aoN34w2v6gsnRsL4eio6wszhukw== X-Received: by 2002:a62:1749:0:b029:19d:960c:1bb8 with SMTP id 70-20020a6217490000b029019d960c1bb8mr11645979pfx.63.1607243516981; Sun, 06 Dec 2020 00:31:56 -0800 (PST) Received: from localhost.localdomain ([103.136.221.70]) by smtp.gmail.com with ESMTPSA id c2sm10229107pfa.59.2020.12.06.00.31.47 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 06 Dec 2020 00:31:56 -0800 (PST) From: Muchun Song To: gregkh@linuxfoundation.org, rafael@kernel.org, adobriyan@gmail.com, akpm@linux-foundation.org, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, hughd@google.com, will@kernel.org, guro@fb.com, rppt@kernel.org, tglx@linutronix.de, esyr@redhat.com, peterx@redhat.com, krisman@collabora.com, surenb@google.com, avagin@openvz.org, elver@google.com, rdunlap@infradead.org, iamjoonsoo.kim@lge.com Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, Muchun Song Subject: [PATCH v2 11/12] mm: memcontrol: make the slab calculation consistent Date: Sun, 6 Dec 2020 16:29:47 +0800 Message-Id: <20201206082948.11812-5-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20201206082948.11812-1-songmuchun@bytedance.com> References: <20201206082948.11812-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Although the ratio of the slab is one, we also should read the ratio from the related memory_stats instead of hard-coding. And the local variable of size is already the value of slab_unreclaimable. So we do not need to read again. The unit of the vmstat counters are either pages or bytes now. So we can drop the ratio in struct memory_stat. This can make the code clean and simple. And get rid of the awkward mix of static and runtime initialization of the memory_stats table. Signed-off-by: Muchun Song --- mm/memcontrol.c | 108 ++++++++++++++++++++++++++++++++++++-------------------- 1 file changed, 70 insertions(+), 38 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 48d70c1ad301..49fbcf003bf5 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -1493,48 +1493,71 @@ static bool mem_cgroup_wait_acct_move(struct mem_cgroup *memcg) struct memory_stat { const char *name; - unsigned int ratio; unsigned int idx; }; static const struct memory_stat memory_stats[] = { - { "anon", PAGE_SIZE, NR_ANON_MAPPED }, - { "file", PAGE_SIZE, NR_FILE_PAGES }, - { "kernel_stack", 1, NR_KERNEL_STACK_B }, - { "percpu", 1, MEMCG_PERCPU_B }, - { "sock", PAGE_SIZE, MEMCG_SOCK }, - { "shmem", PAGE_SIZE, NR_SHMEM }, - { "file_mapped", PAGE_SIZE, NR_FILE_MAPPED }, - { "file_dirty", PAGE_SIZE, NR_FILE_DIRTY }, - { "file_writeback", PAGE_SIZE, NR_WRITEBACK }, + { "anon", NR_ANON_MAPPED }, + { "file", NR_FILE_PAGES }, + { "kernel_stack", NR_KERNEL_STACK_B }, + { "percpu", MEMCG_PERCPU_B }, + { "sock", MEMCG_SOCK }, + { "shmem", NR_SHMEM }, + { "file_mapped", NR_FILE_MAPPED }, + { "file_dirty", NR_FILE_DIRTY }, + { "file_writeback", NR_WRITEBACK }, #ifdef CONFIG_TRANSPARENT_HUGEPAGE - { "anon_thp", PAGE_SIZE, NR_ANON_THPS }, - { "file_thp", PAGE_SIZE, NR_FILE_THPS }, - { "shmem_thp", PAGE_SIZE, NR_SHMEM_THPS }, + { "anon_thp", NR_ANON_THPS }, + { "file_thp", NR_FILE_THPS }, + { "shmem_thp", NR_SHMEM_THPS }, #endif - { "inactive_anon", PAGE_SIZE, NR_INACTIVE_ANON }, - { "active_anon", PAGE_SIZE, NR_ACTIVE_ANON }, - { "inactive_file", PAGE_SIZE, NR_INACTIVE_FILE }, - { "active_file", PAGE_SIZE, NR_ACTIVE_FILE }, - { "unevictable", PAGE_SIZE, NR_UNEVICTABLE }, - - /* - * Note: The slab_reclaimable and slab_unreclaimable must be - * together and slab_reclaimable must be in front. - */ - { "slab_reclaimable", 1, NR_SLAB_RECLAIMABLE_B }, - { "slab_unreclaimable", 1, NR_SLAB_UNRECLAIMABLE_B }, + { "inactive_anon", NR_INACTIVE_ANON }, + { "active_anon", NR_ACTIVE_ANON }, + { "inactive_file", NR_INACTIVE_FILE }, + { "active_file", NR_ACTIVE_FILE }, + { "unevictable", NR_UNEVICTABLE }, + { "slab_reclaimable", NR_SLAB_RECLAIMABLE_B }, + { "slab_unreclaimable", NR_SLAB_UNRECLAIMABLE_B }, /* The memory events */ - { "workingset_refault_anon", 1, WORKINGSET_REFAULT_ANON }, - { "workingset_refault_file", 1, WORKINGSET_REFAULT_FILE }, - { "workingset_activate_anon", 1, WORKINGSET_ACTIVATE_ANON }, - { "workingset_activate_file", 1, WORKINGSET_ACTIVATE_FILE }, - { "workingset_restore_anon", 1, WORKINGSET_RESTORE_ANON }, - { "workingset_restore_file", 1, WORKINGSET_RESTORE_FILE }, - { "workingset_nodereclaim", 1, WORKINGSET_NODERECLAIM }, + { "workingset_refault_anon", WORKINGSET_REFAULT_ANON }, + { "workingset_refault_file", WORKINGSET_REFAULT_FILE }, + { "workingset_activate_anon", WORKINGSET_ACTIVATE_ANON }, + { "workingset_activate_file", WORKINGSET_ACTIVATE_FILE }, + { "workingset_restore_anon", WORKINGSET_RESTORE_ANON }, + { "workingset_restore_file", WORKINGSET_RESTORE_FILE }, + { "workingset_nodereclaim", WORKINGSET_NODERECLAIM }, }; +/* Translate stat items to the correct unit for memory.stat output */ +static int memcg_page_state_unit(int item) +{ + int unit; + + switch (item) { + case WORKINGSET_REFAULT_ANON: + case WORKINGSET_REFAULT_FILE: + case WORKINGSET_ACTIVATE_ANON: + case WORKINGSET_ACTIVATE_FILE: + case WORKINGSET_RESTORE_ANON: + case WORKINGSET_RESTORE_FILE: + case WORKINGSET_NODERECLAIM: + unit = 1; + break; + default: + unit = memcg_stat_item_in_bytes(item) ? 1 : PAGE_SIZE; + break; + } + + return unit; +} + +static inline unsigned long memcg_page_state_output(struct mem_cgroup *memcg, + int item) +{ + return memcg_page_state(memcg, item) * memcg_page_state_unit(item); +} + static char *memory_stat_format(struct mem_cgroup *memcg) { struct seq_buf s; @@ -1558,13 +1581,16 @@ static char *memory_stat_format(struct mem_cgroup *memcg) for (i = 0; i < ARRAY_SIZE(memory_stats); i++) { u64 size; - size = memcg_page_state(memcg, memory_stats[i].idx); - size *= memory_stats[i].ratio; + size = memcg_page_state_output(memcg, memory_stats[i].idx); seq_buf_printf(&s, "%s %llu\n", memory_stats[i].name, size); + /* + * We are printing reclaimable, unreclaimable of the slab + * and the sum of both. + */ if (unlikely(memory_stats[i].idx == NR_SLAB_UNRECLAIMABLE_B)) { - size = memcg_page_state(memcg, NR_SLAB_RECLAIMABLE_B) + - memcg_page_state(memcg, NR_SLAB_UNRECLAIMABLE_B); + size += memcg_page_state_output(memcg, + NR_SLAB_RECLAIMABLE_B); seq_buf_printf(&s, "slab %llu\n", size); } } @@ -6358,6 +6384,12 @@ static int memory_stat_show(struct seq_file *m, void *v) } #ifdef CONFIG_NUMA +static inline unsigned long lruvec_page_state_output(struct lruvec *lruvec, + int item) +{ + return lruvec_page_state(lruvec, item) * memcg_page_state_unit(item); +} + static int memory_numa_stat_show(struct seq_file *m, void *v) { int i; @@ -6375,8 +6407,8 @@ static int memory_numa_stat_show(struct seq_file *m, void *v) struct lruvec *lruvec; lruvec = mem_cgroup_lruvec(memcg, NODE_DATA(nid)); - size = lruvec_page_state(lruvec, memory_stats[i].idx); - size *= memory_stats[i].ratio; + size = lruvec_page_state_output(lruvec, + memory_stats[i].idx); seq_printf(m, " N%d=%llu", nid, size); } seq_putc(m, '\n'); From patchwork Sun Dec 6 08:29:48 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 11953837 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 39748C4361B for ; Sun, 6 Dec 2020 08:32:08 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id A8E4A23110 for ; Sun, 6 Dec 2020 08:32:07 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A8E4A23110 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 4D9408D0006; Sun, 6 Dec 2020 03:32:07 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 48A686B0071; Sun, 6 Dec 2020 03:32:07 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 304118D0006; Sun, 6 Dec 2020 03:32:07 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0057.hostedemail.com [216.40.44.57]) by kanga.kvack.org (Postfix) with ESMTP id 1914A6B006E for ; Sun, 6 Dec 2020 03:32:07 -0500 (EST) Received: from smtpin10.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id D0232181AEF10 for ; Sun, 6 Dec 2020 08:32:06 +0000 (UTC) X-FDA: 77562189852.10.goose28_5502e1a273d4 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin10.hostedemail.com (Postfix) with ESMTP id AD0C716A4AA for ; Sun, 6 Dec 2020 08:32:06 +0000 (UTC) X-HE-Tag: goose28_5502e1a273d4 X-Filterd-Recvd-Size: 13523 Received: from mail-pl1-f195.google.com (mail-pl1-f195.google.com [209.85.214.195]) by imf20.hostedemail.com (Postfix) with ESMTP for ; Sun, 6 Dec 2020 08:32:06 +0000 (UTC) Received: by mail-pl1-f195.google.com with SMTP id bj5so5525297plb.4 for ; Sun, 06 Dec 2020 00:32:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=lP6NOXY9P1Y1tdnLUE8WJ9oCZQOprYuVn3yHjkNbfog=; b=EXcZ7sDghEHanmw7oN1QvmiFqdNgVMPn4nxp5QrGV7ENEoDYwofyu11Px+/NibCKAf fmyDb/uqNjxbGizqa2LrxwD13xo7YmNycDZvWPARPTKLARWA/9ZWeJQJtRccj0B5wy+S h6zBOQpoOfi6JblccslS0e8TdwLvzMpIv8ZO+uQ7RJxaRXZmyQoUD5vmhOeGU/VuGl1N XttH3IKOla/ZAbSTwpsejWyqi86ar9zjgUH56qfCAp6qtLvH4E5rqoVgYJJW6lq1i6Mh Ftzp9nx4AFEkD2MKP2r0mJMHYeu2nmvkFpu64metQYT24FytfEiSGhDt/V2LgBObhBlQ jb+g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=lP6NOXY9P1Y1tdnLUE8WJ9oCZQOprYuVn3yHjkNbfog=; b=IeZkY2gFb31PL1LnZAbAgSxPdNFdmkTDCwIn9TAJ2oPF92/AtjFMxCIaPZElZ6gIjF wQzDuAtFzCcVQy6t9ySoztE8D7APgT8nsrchXBJKolYulASjox4Zo0bB1bJRLYv68Uyo SRr9BOsVlG+rpAlPrNs3B+m3Sjy42hEhlbppGxoLgOfNC0cUyE8N1YM6RuhwYvZmsA2w w0hYKBzUtsCzLZdM82hPB6NkysOXLvvSyf0ABGXVkP92CsHJIehvohEhe+tG4Mo/4Ttl /6gynl49MPiI8RjWU04Eo5J3xIY099EnQAdDrzfKoh4IirWTbkI/7EtimlKHRxIpITDR YugQ== X-Gm-Message-State: AOAM532bxPxgORSvglojA2Sjlo+mt+17NLPD6yHo+/eWv/nggaSI26H1 p3koCk+lNle5kwoDn9p0KRnMrw== X-Google-Smtp-Source: ABdhPJwfN8K8E4tC4ljKBXcesQmCoi2n20gNTIjFvrvBhaPpV6baXBdLdTDzByNenjoMaa1RhWAeiQ== X-Received: by 2002:a17:902:8bcb:b029:d9:d765:d7f3 with SMTP id r11-20020a1709028bcbb02900d9d765d7f3mr10785319plo.69.1607243525364; Sun, 06 Dec 2020 00:32:05 -0800 (PST) Received: from localhost.localdomain ([103.136.221.70]) by smtp.gmail.com with ESMTPSA id c2sm10229107pfa.59.2020.12.06.00.31.57 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 06 Dec 2020 00:32:04 -0800 (PST) From: Muchun Song To: gregkh@linuxfoundation.org, rafael@kernel.org, adobriyan@gmail.com, akpm@linux-foundation.org, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, hughd@google.com, will@kernel.org, guro@fb.com, rppt@kernel.org, tglx@linutronix.de, esyr@redhat.com, peterx@redhat.com, krisman@collabora.com, surenb@google.com, avagin@openvz.org, elver@google.com, rdunlap@infradead.org, iamjoonsoo.kim@lge.com Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, Muchun Song Subject: [PATCH v2 12/12] mm: memcontrol: remove {global_}node_page_state_pages Date: Sun, 6 Dec 2020 16:29:48 +0800 Message-Id: <20201206082948.11812-6-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20201206082948.11812-1-songmuchun@bytedance.com> References: <20201206082948.11812-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Now the unit of the vmstat counters are either pages or bytes. So we can adjust the node_page_state to always returns values in pages and remove the node_page_state_pages. Signed-off-by: Muchun Song --- drivers/base/node.c | 10 +++++----- fs/proc/meminfo.c | 12 ++++++------ include/linux/vmstat.h | 17 +---------------- kernel/power/snapshot.c | 2 +- mm/oom_kill.c | 2 +- mm/page_alloc.c | 10 +++++----- mm/vmscan.c | 2 +- mm/vmstat.c | 23 ++++++----------------- 8 files changed, 26 insertions(+), 52 deletions(-) diff --git a/drivers/base/node.c b/drivers/base/node.c index bc01ce0b2fcd..42298e3552e5 100644 --- a/drivers/base/node.c +++ b/drivers/base/node.c @@ -374,8 +374,8 @@ static ssize_t node_read_meminfo(struct device *dev, unsigned long sreclaimable, sunreclaimable; si_meminfo_node(&i, nid); - sreclaimable = node_page_state_pages(pgdat, NR_SLAB_RECLAIMABLE_B); - sunreclaimable = node_page_state_pages(pgdat, NR_SLAB_UNRECLAIMABLE_B); + sreclaimable = node_page_state(pgdat, NR_SLAB_RECLAIMABLE_B); + sunreclaimable = node_page_state(pgdat, NR_SLAB_UNRECLAIMABLE_B); len = sysfs_emit_at(buf, len, "Node %d MemTotal: %8lu kB\n" "Node %d MemFree: %8lu kB\n" @@ -446,9 +446,9 @@ static ssize_t node_read_meminfo(struct device *dev, nid, K(node_page_state(pgdat, NR_FILE_MAPPED)), nid, K(node_page_state(pgdat, NR_ANON_MAPPED)), nid, K(i.sharedram), - nid, node_page_state(pgdat, NR_KERNEL_STACK_B) / SZ_1K, + nid, K(node_page_state(pgdat, NR_KERNEL_STACK_B)), #ifdef CONFIG_SHADOW_CALL_STACK - nid, node_page_state(pgdat, NR_KERNEL_SCS_B) / SZ_1K, + nid, K(node_page_state(pgdat, NR_KERNEL_SCS_B)), #endif nid, K(sum_zone_node_page_state(nid, NR_PAGETABLE)), nid, 0UL, @@ -517,7 +517,7 @@ static ssize_t node_read_vmstat(struct device *dev, for (i = 0; i < NR_VM_NODE_STAT_ITEMS; i++) len += sysfs_emit_at(buf, len, "%s %lu\n", node_stat_name(i), - node_page_state_pages(pgdat, i)); + node_page_state(pgdat, i)); return len; } diff --git a/fs/proc/meminfo.c b/fs/proc/meminfo.c index 69895e83d4fc..95ea5f062161 100644 --- a/fs/proc/meminfo.c +++ b/fs/proc/meminfo.c @@ -52,8 +52,8 @@ static int meminfo_proc_show(struct seq_file *m, void *v) pages[lru] = global_node_page_state(NR_LRU_BASE + lru); available = si_mem_available(); - sreclaimable = global_node_page_state_pages(NR_SLAB_RECLAIMABLE_B); - sunreclaim = global_node_page_state_pages(NR_SLAB_UNRECLAIMABLE_B); + sreclaimable = global_node_page_state(NR_SLAB_RECLAIMABLE_B); + sunreclaim = global_node_page_state(NR_SLAB_UNRECLAIMABLE_B); show_val_kb(m, "MemTotal: ", i.totalram); show_val_kb(m, "MemFree: ", i.freeram); @@ -100,11 +100,11 @@ static int meminfo_proc_show(struct seq_file *m, void *v) show_val_kb(m, "Slab: ", sreclaimable + sunreclaim); show_val_kb(m, "SReclaimable: ", sreclaimable); show_val_kb(m, "SUnreclaim: ", sunreclaim); - seq_printf(m, "KernelStack: %8lu kB\n", - global_node_page_state(NR_KERNEL_STACK_B) / SZ_1K); + show_val_kb(m, "KernelStack: ", + global_node_page_state(NR_KERNEL_STACK_B)); #ifdef CONFIG_SHADOW_CALL_STACK - seq_printf(m, "ShadowCallStack:%8lu kB\n", - global_node_page_state(NR_KERNEL_SCS_B) / SZ_1K); + show_val_kb(m, "ShadowCallStack:", + global_node_page_state(NR_KERNEL_SCS_B)); #endif show_val_kb(m, "PageTables: ", global_zone_page_state(NR_PAGETABLE)); diff --git a/include/linux/vmstat.h b/include/linux/vmstat.h index afd84dc2398c..ae821e016fdd 100644 --- a/include/linux/vmstat.h +++ b/include/linux/vmstat.h @@ -193,8 +193,7 @@ static inline unsigned long global_zone_page_state(enum zone_stat_item item) return x; } -static inline -unsigned long global_node_page_state_pages(enum node_stat_item item) +static inline unsigned long global_node_page_state(enum node_stat_item item) { long x = atomic_long_read(&vm_node_stat[item]); @@ -207,17 +206,6 @@ unsigned long global_node_page_state_pages(enum node_stat_item item) return x; } -static inline unsigned long global_node_page_state(enum node_stat_item item) -{ - long x = atomic_long_read(&vm_node_stat[item]); - -#ifdef CONFIG_SMP - if (x < 0) - x = 0; -#endif - return x; -} - static inline unsigned long zone_page_state(struct zone *zone, enum zone_stat_item item) { @@ -258,12 +246,9 @@ extern unsigned long sum_zone_node_page_state(int node, extern unsigned long sum_zone_numa_state(int node, enum numa_stat_item item); extern unsigned long node_page_state(struct pglist_data *pgdat, enum node_stat_item item); -extern unsigned long node_page_state_pages(struct pglist_data *pgdat, - enum node_stat_item item); #else #define sum_zone_node_page_state(node, item) global_zone_page_state(item) #define node_page_state(node, item) global_node_page_state(item) -#define node_page_state_pages(node, item) global_node_page_state_pages(item) #endif /* CONFIG_NUMA */ #ifdef CONFIG_SMP diff --git a/kernel/power/snapshot.c b/kernel/power/snapshot.c index d63560e1cf87..664520bdaa20 100644 --- a/kernel/power/snapshot.c +++ b/kernel/power/snapshot.c @@ -1705,7 +1705,7 @@ static unsigned long minimum_image_size(unsigned long saveable) { unsigned long size; - size = global_node_page_state_pages(NR_SLAB_RECLAIMABLE_B) + size = global_node_page_state(NR_SLAB_RECLAIMABLE_B) + global_node_page_state(NR_ACTIVE_ANON) + global_node_page_state(NR_INACTIVE_ANON) + global_node_page_state(NR_ACTIVE_FILE) diff --git a/mm/oom_kill.c b/mm/oom_kill.c index 04b19b7b5435..73861473c7d4 100644 --- a/mm/oom_kill.c +++ b/mm/oom_kill.c @@ -188,7 +188,7 @@ static bool should_dump_unreclaim_slab(void) global_node_page_state(NR_ISOLATED_FILE) + global_node_page_state(NR_UNEVICTABLE); - return (global_node_page_state_pages(NR_SLAB_UNRECLAIMABLE_B) > nr_lru); + return (global_node_page_state(NR_SLAB_UNRECLAIMABLE_B) > nr_lru); } /** diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 58916b3afdab..d16c9388c0b8 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -5372,7 +5372,7 @@ long si_mem_available(void) * items that are in use, and cannot be freed. Cap this estimate at the * low watermark. */ - reclaimable = global_node_page_state_pages(NR_SLAB_RECLAIMABLE_B) + + reclaimable = global_node_page_state(NR_SLAB_RECLAIMABLE_B) + global_node_page_state(NR_KERNEL_MISC_RECLAIMABLE); available += reclaimable - min(reclaimable / 2, wmark_low); @@ -5516,8 +5516,8 @@ void show_free_areas(unsigned int filter, nodemask_t *nodemask) global_node_page_state(NR_UNEVICTABLE), global_node_page_state(NR_FILE_DIRTY), global_node_page_state(NR_WRITEBACK), - global_node_page_state_pages(NR_SLAB_RECLAIMABLE_B), - global_node_page_state_pages(NR_SLAB_UNRECLAIMABLE_B), + global_node_page_state(NR_SLAB_RECLAIMABLE_B), + global_node_page_state(NR_SLAB_UNRECLAIMABLE_B), global_node_page_state(NR_FILE_MAPPED), global_node_page_state(NR_SHMEM), global_zone_page_state(NR_PAGETABLE), @@ -5572,9 +5572,9 @@ void show_free_areas(unsigned int filter, nodemask_t *nodemask) K(node_page_state(pgdat, NR_ANON_THPS)), #endif K(node_page_state(pgdat, NR_WRITEBACK_TEMP)), - node_page_state(pgdat, NR_KERNEL_STACK_B) / SZ_1K, + K(node_page_state(pgdat, NR_KERNEL_STACK_B)), #ifdef CONFIG_SHADOW_CALL_STACK - node_page_state(pgdat, NR_KERNEL_SCS_B) / SZ_1K, + K(node_page_state(pgdat, NR_KERNEL_SCS_B)), #endif pgdat->kswapd_failures >= MAX_RECLAIM_RETRIES ? "yes" : "no"); diff --git a/mm/vmscan.c b/mm/vmscan.c index 469016222cdb..5d3c8fa68979 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -4220,7 +4220,7 @@ int node_reclaim(struct pglist_data *pgdat, gfp_t gfp_mask, unsigned int order) * unmapped file backed pages. */ if (node_pagecache_reclaimable(pgdat) <= pgdat->min_unmapped_pages && - node_page_state_pages(pgdat, NR_SLAB_RECLAIMABLE_B) <= + node_page_state(pgdat, NR_SLAB_RECLAIMABLE_B) <= pgdat->min_slab_pages) return NODE_RECLAIM_FULL; diff --git a/mm/vmstat.c b/mm/vmstat.c index 25751b1d8e2e..b7cdef585efd 100644 --- a/mm/vmstat.c +++ b/mm/vmstat.c @@ -1000,22 +1000,9 @@ unsigned long sum_zone_numa_state(int node, } /* - * Determine the per node value of a stat item. + * Determine the per node value of a stat item. This always returns + * values in pages. */ -unsigned long node_page_state_pages(struct pglist_data *pgdat, - enum node_stat_item item) -{ - long x = atomic_long_read(&pgdat->vm_stat[item]); - -#ifdef CONFIG_SMP - if (x < 0) - x = 0; -#endif - if (vmstat_item_in_bytes(item)) - x >>= PAGE_SHIFT; - return x; -} - unsigned long node_page_state(struct pglist_data *pgdat, enum node_stat_item item) { @@ -1025,6 +1012,8 @@ unsigned long node_page_state(struct pglist_data *pgdat, if (x < 0) x = 0; #endif + if (vmstat_item_in_bytes(item)) + x >>= PAGE_SHIFT; return x; } #endif @@ -1626,7 +1615,7 @@ static void zoneinfo_show_print(struct seq_file *m, pg_data_t *pgdat, seq_printf(m, "\n per-node stats"); for (i = 0; i < NR_VM_NODE_STAT_ITEMS; i++) { seq_printf(m, "\n %-12s %lu", node_stat_name(i), - node_page_state_pages(pgdat, i)); + node_page_state(pgdat, i)); } } seq_printf(m, @@ -1747,7 +1736,7 @@ static void *vmstat_start(struct seq_file *m, loff_t *pos) #endif for (i = 0; i < NR_VM_NODE_STAT_ITEMS; i++) - v[i] = global_node_page_state_pages(i); + v[i] = global_node_page_state(i); v += NR_VM_NODE_STAT_ITEMS; global_dirty_limits(v + NR_DIRTY_BG_THRESHOLD,