From patchwork Fri Dec 11 04:19:48 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 11967205 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E5379C4361B for ; Fri, 11 Dec 2020 04:23:13 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 5155D23ED2 for ; Fri, 11 Dec 2020 04:23:13 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5155D23ED2 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id DBF8D6B005D; Thu, 10 Dec 2020 23:23:12 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D94E16B0068; Thu, 10 Dec 2020 23:23:12 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C5C056B006C; Thu, 10 Dec 2020 23:23:12 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0002.hostedemail.com [216.40.44.2]) by kanga.kvack.org (Postfix) with ESMTP id AE8156B005D for ; Thu, 10 Dec 2020 23:23:12 -0500 (EST) Received: from smtpin03.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 7D7883632 for ; Fri, 11 Dec 2020 04:23:12 +0000 (UTC) X-FDA: 77579706624.03.nest04_561818e273fe Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin03.hostedemail.com (Postfix) with ESMTP id 5CCD028A4E8 for ; Fri, 11 Dec 2020 04:23:12 +0000 (UTC) X-HE-Tag: nest04_561818e273fe X-Filterd-Recvd-Size: 4549 Received: from mail-pg1-f193.google.com (mail-pg1-f193.google.com [209.85.215.193]) by imf29.hostedemail.com (Postfix) with ESMTP for ; Fri, 11 Dec 2020 04:23:11 +0000 (UTC) Received: by mail-pg1-f193.google.com with SMTP id g18so6316633pgk.1 for ; Thu, 10 Dec 2020 20:23:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=FpYaFCBP9ryR0HnwKGfpggL7bn85I6W2LvY3/jvWBMg=; b=T2ahvN7HcruB45VJZG2OJqfIapWawdOD8QtWwwwuD43vjFPlp2f0Yi8Gslq+3aVBN3 QuexiT59I4+K7zRILtyttGcENl7TzkS+jsjA2Oh+lybl033Bptk3XduoWbvXn6xVDsAL 7ee5f0G+Ld7Nk/1Y3Wh4XlEMomkWzrC1dLBL4zDjO8dWk8nn8cZPsKzWKj443KmhKlbA NKCRfX207fMkpWgeZ/0TF0rb0LteP1ZUhUv6MOH1oj64KFiCmTraUoFBRKZ2bpGW2vj5 EG0+5TFj/Kc9+edCH85DQVHPkanMAhz8lGbgK+VXRAPkdeeGVlMCFhSbRk9PNWxS7rQK g/WQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=FpYaFCBP9ryR0HnwKGfpggL7bn85I6W2LvY3/jvWBMg=; b=U8wbP73qmWlkleDJjGyFMhSZlS0KcukldE7cLAoBDjY2drwYiFF/Z22XKPr0wXQ48d FYMTasguqYs4Atv+O8/KFCabqAsfpISg2Jgv0tdP0hc/uNn1yUAi4LwiGC+Je4rcESWG ZeZPt4zabFpQ9ip22+1DlbZ2RPs2Qjovk3sR2N5URCWuSMQaeKv70NTes04OEvPmtzKO MMcRH9OsYpnEqdDOOYuXW0iKNmb/q/WU8AHi43gwYfQN8LKAQ/MSDmZ6E1aplGao+8bG 1KD7HGrOp8MJAPwCTEiLUupf3X5ZyUAUB1xE1mvx0d5gU3G7G79vTkryfMBDEZ9I260p bBmQ== X-Gm-Message-State: AOAM531kY/W5Gq7Qudb90tE1INwY4NJ35NzNgeEcNKY8JFj9RvL55XPu ptPVyKdgiQq6dWNzUhT/0DJyAQ== X-Google-Smtp-Source: ABdhPJz13P4aoeVkSVXowRznFKeGs6JEKzGT5V/ZC2LxOJuCCnhOD8neubcs+2TtN6B0lrJSzDSQTw== X-Received: by 2002:a05:6a00:1596:b029:19d:96b8:6eab with SMTP id u22-20020a056a001596b029019d96b86eabmr9899521pfk.38.1607660590187; Thu, 10 Dec 2020 20:23:10 -0800 (PST) Received: from localhost.localdomain ([103.136.221.66]) by smtp.gmail.com with ESMTPSA id 19sm8623352pfu.85.2020.12.10.20.23.03 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 10 Dec 2020 20:23:09 -0800 (PST) From: Muchun Song To: gregkh@linuxfoundation.org, rafael@kernel.org, adobriyan@gmail.com, akpm@linux-foundation.org, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, hughd@google.com, shakeelb@google.com, guro@fb.com, samitolvanen@google.com, feng.tang@intel.com, neilb@suse.de, iamjoonsoo.kim@lge.com, rdunlap@infradead.org Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, Muchun Song , Michal Hocko Subject: [PATCH v4 1/7] mm: memcontrol: fix NR_ANON_THPS accounting in charge moving Date: Fri, 11 Dec 2020 12:19:48 +0800 Message-Id: <20201211041954.79543-2-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20201211041954.79543-1-songmuchun@bytedance.com> References: <20201211041954.79543-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The unit of NR_ANON_THPS is HPAGE_PMD_NR already. So it should inc/dec by one rather than nr_pages. Fixes: 468c398233da ("mm: memcontrol: switch to native NR_ANON_THPS counter") Signed-off-by: Muchun Song Acked-by: Michal Hocko Acked-by: Johannes Weiner Reviewed-by: Roman Gushchin Acked-by: Pankaj Gupta --- mm/memcontrol.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index b80328f52fb4..8818bf64d6fe 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -5653,10 +5653,8 @@ static int mem_cgroup_move_account(struct page *page, __mod_lruvec_state(from_vec, NR_ANON_MAPPED, -nr_pages); __mod_lruvec_state(to_vec, NR_ANON_MAPPED, nr_pages); if (PageTransHuge(page)) { - __mod_lruvec_state(from_vec, NR_ANON_THPS, - -nr_pages); - __mod_lruvec_state(to_vec, NR_ANON_THPS, - nr_pages); + __dec_lruvec_state(from_vec, NR_ANON_THPS); + __inc_lruvec_state(to_vec, NR_ANON_THPS); } } From patchwork Fri Dec 11 04:19:49 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 11967207 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 80A5CC433FE for ; Fri, 11 Dec 2020 04:23:20 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 1055A23EF1 for ; Fri, 11 Dec 2020 04:23:20 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1055A23EF1 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 9C7106B0068; Thu, 10 Dec 2020 23:23:19 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 9A24A6B006C; Thu, 10 Dec 2020 23:23:19 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8B1E76B006E; Thu, 10 Dec 2020 23:23:19 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0002.hostedemail.com [216.40.44.2]) by kanga.kvack.org (Postfix) with ESMTP id 7425A6B0068 for ; Thu, 10 Dec 2020 23:23:19 -0500 (EST) Received: from smtpin22.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 358AF3632 for ; Fri, 11 Dec 2020 04:23:19 +0000 (UTC) X-FDA: 77579706918.22.roof62_5e0d9e1273fe Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin22.hostedemail.com (Postfix) with ESMTP id 0AA7818038E60 for ; Fri, 11 Dec 2020 04:23:19 +0000 (UTC) X-HE-Tag: roof62_5e0d9e1273fe X-Filterd-Recvd-Size: 11909 Received: from mail-pj1-f65.google.com (mail-pj1-f65.google.com [209.85.216.65]) by imf15.hostedemail.com (Postfix) with ESMTP for ; Fri, 11 Dec 2020 04:23:18 +0000 (UTC) Received: by mail-pj1-f65.google.com with SMTP id v1so1567475pjr.2 for ; Thu, 10 Dec 2020 20:23:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=cU/HrVcwfM3LPFm4rjOQAH8JdzbfJ+ZpL7D7Ks2Uco0=; b=tih4ZHbKA0YgssVEixNJ0hUNRlQ83ass+bYW1yuu3v27Qw+hZwKiLO7xMEW7V9iGfo CL+A5/3YoVRnB1m1CfbOvvdxL6uiv1v8CvG7gKw4Lqm6dLgN8Ih0dD9pkJVjxa2DsOGp HjsANWSDU0LDkKJLCdqXKHlEN7jTGU+zJDrDwyUVrV9i4kCXKTXRf2tAU81mxiMaCZ3J wJj9xeB0NYehtyC1FQ1pBreTX1nJiWNTjK4sEq6AQHFSwQuLlbadTYekoPpxVHudgs4W Ko1GxshKle2QfTsP0MSzCxmnG8MEgEHist32cC9QtFWhohA01FbOh+c+Aayw1n7tTkbJ 9gWQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=cU/HrVcwfM3LPFm4rjOQAH8JdzbfJ+ZpL7D7Ks2Uco0=; b=gU/uavAk37nMZHLwg+ad+slHi3wrn08Ulm9BCurp4n9Y06w9qMfBH/qSPcXtqkxamC V7aLNk2nlp2nTvmxdIkBgwZAIoQk1XdQDD8nKiMlDu9okyRDs7RcxDUvw9duUt+BRWZw /ijXmcI1OM0H0JrPty80PY53UBaCN0S9pN7seGGFuCRzUskN/rQ/omkQcwmEhEnyO2PE Sd+CT5AzfYi4ToSjxNtpPw0ThQJ/JyjzIZkjjy86SEtagum5OBjtyR3F6GAx4QuuWsD1 heSVWAF2o4DIN0R+rA904knKnmamMh8tKwFv8FPEJZNWRQDGNRQNtZ4H0ZW8vKuUq31x r8gQ== X-Gm-Message-State: AOAM533E94rR+cBYIQ4zm7z7nSqSNY+zpbJd7aLER7KIbgJG+ZC56yje vv/ObkiUZBLcpQ/T8O2eKV8TUQ== X-Google-Smtp-Source: ABdhPJwvRIgVYH8myVTOip532m/G1DHFdQPbkxnk9Pu1iXGs/BxE4LZolQ+5F26TEiO0j8O5CqF9nQ== X-Received: by 2002:a17:90b:1b52:: with SMTP id nv18mr11374155pjb.172.1607660597400; Thu, 10 Dec 2020 20:23:17 -0800 (PST) Received: from localhost.localdomain ([103.136.221.66]) by smtp.gmail.com with ESMTPSA id 19sm8623352pfu.85.2020.12.10.20.23.10 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 10 Dec 2020 20:23:16 -0800 (PST) From: Muchun Song To: gregkh@linuxfoundation.org, rafael@kernel.org, adobriyan@gmail.com, akpm@linux-foundation.org, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, hughd@google.com, shakeelb@google.com, guro@fb.com, samitolvanen@google.com, feng.tang@intel.com, neilb@suse.de, iamjoonsoo.kim@lge.com, rdunlap@infradead.org Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, Muchun Song Subject: [PATCH v4 2/7] mm: memcontrol: convert NR_ANON_THPS account to pages Date: Fri, 11 Dec 2020 12:19:49 +0800 Message-Id: <20201211041954.79543-3-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20201211041954.79543-1-songmuchun@bytedance.com> References: <20201211041954.79543-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The unit of NR_ANON_THPS is HPAGE_PMD_NR. Convert the NR_ANON_THPS account to pages. Signed-off-by: Muchun Song --- drivers/base/node.c | 15 +++++++++------ fs/proc/meminfo.c | 2 +- include/linux/mmzone.h | 9 +++++++++ mm/huge_memory.c | 3 ++- mm/memcontrol.c | 20 ++++++-------------- mm/page_alloc.c | 2 +- mm/rmap.c | 7 ++++--- mm/vmstat.c | 11 +++++++++-- 8 files changed, 41 insertions(+), 28 deletions(-) diff --git a/drivers/base/node.c b/drivers/base/node.c index 04f71c7bc3f8..6da0c3508bc9 100644 --- a/drivers/base/node.c +++ b/drivers/base/node.c @@ -461,8 +461,7 @@ static ssize_t node_read_meminfo(struct device *dev, nid, K(sunreclaimable) #ifdef CONFIG_TRANSPARENT_HUGEPAGE , - nid, K(node_page_state(pgdat, NR_ANON_THPS) * - HPAGE_PMD_NR), + nid, K(node_page_state(pgdat, NR_ANON_THPS)), nid, K(node_page_state(pgdat, NR_SHMEM_THPS) * HPAGE_PMD_NR), nid, K(node_page_state(pgdat, NR_SHMEM_PMDMAPPED) * @@ -519,10 +518,14 @@ static ssize_t node_read_vmstat(struct device *dev, sum_zone_numa_state(nid, i)); #endif - for (i = 0; i < NR_VM_NODE_STAT_ITEMS; i++) - len += sysfs_emit_at(buf, len, "%s %lu\n", - node_stat_name(i), - node_page_state_pages(pgdat, i)); + for (i = 0; i < NR_VM_NODE_STAT_ITEMS; i++) { + unsigned long pages = node_page_state_pages(pgdat, i); + + if (vmstat_item_print_in_thp(i)) + pages /= HPAGE_PMD_NR; + len += sysfs_emit_at(buf, len, "%s %lu\n", node_stat_name(i), + pages); + } return len; } diff --git a/fs/proc/meminfo.c b/fs/proc/meminfo.c index d6fc74619625..a635c8a84ddf 100644 --- a/fs/proc/meminfo.c +++ b/fs/proc/meminfo.c @@ -129,7 +129,7 @@ static int meminfo_proc_show(struct seq_file *m, void *v) #ifdef CONFIG_TRANSPARENT_HUGEPAGE show_val_kb(m, "AnonHugePages: ", - global_node_page_state(NR_ANON_THPS) * HPAGE_PMD_NR); + global_node_page_state(NR_ANON_THPS)); show_val_kb(m, "ShmemHugePages: ", global_node_page_state(NR_SHMEM_THPS) * HPAGE_PMD_NR); show_val_kb(m, "ShmemPmdMapped: ", diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index b593316bff3d..4ac95448421c 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -210,6 +210,15 @@ enum node_stat_item { }; /* + * Returns true if the item should prints in THPs. The /proc/vmstat currently + * prints number of anon, file and shmem THPs. But the item is charged in pages. + */ +static __always_inline bool vmstat_item_print_in_thp(enum node_stat_item item) +{ + return item == NR_ANON_THPS; +} + +/* * Returns true if the value is measured in bytes (most vmstat values are * measured in pages). This defines the API part, the internal representation * might be different. diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 10dd3cae5f53..66ec454120de 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -2178,7 +2178,8 @@ static void __split_huge_pmd_locked(struct vm_area_struct *vma, pmd_t *pmd, lock_page_memcg(page); if (atomic_add_negative(-1, compound_mapcount_ptr(page))) { /* Last compound_mapcount is gone. */ - __dec_lruvec_page_state(page, NR_ANON_THPS); + __mod_lruvec_page_state(page, NR_ANON_THPS, + -HPAGE_PMD_NR); if (TestClearPageDoubleMap(page)) { /* No need in mapcount reference anymore */ for (i = 0; i < HPAGE_PMD_NR; i++) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 8818bf64d6fe..b18e25a5cdf3 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -1532,7 +1532,7 @@ static struct memory_stat memory_stats[] = { * on some architectures, the macro of HPAGE_PMD_SIZE is not * constant(e.g. powerpc). */ - { "anon_thp", 0, NR_ANON_THPS }, + { "anon_thp", PAGE_SIZE, NR_ANON_THPS }, { "file_thp", 0, NR_FILE_THPS }, { "shmem_thp", 0, NR_SHMEM_THPS }, #endif @@ -1565,8 +1565,7 @@ static int __init memory_stats_init(void) for (i = 0; i < ARRAY_SIZE(memory_stats); i++) { #ifdef CONFIG_TRANSPARENT_HUGEPAGE - if (memory_stats[i].idx == NR_ANON_THPS || - memory_stats[i].idx == NR_FILE_THPS || + if (memory_stats[i].idx == NR_FILE_THPS || memory_stats[i].idx == NR_SHMEM_THPS) memory_stats[i].ratio = HPAGE_PMD_SIZE; #endif @@ -4088,10 +4087,6 @@ static int memcg_stat_show(struct seq_file *m, void *v) if (memcg1_stats[i] == MEMCG_SWAP && !do_memsw_account()) continue; nr = memcg_page_state_local(memcg, memcg1_stats[i]); -#ifdef CONFIG_TRANSPARENT_HUGEPAGE - if (memcg1_stats[i] == NR_ANON_THPS) - nr *= HPAGE_PMD_NR; -#endif seq_printf(m, "%s %lu\n", memcg1_stat_names[i], nr * PAGE_SIZE); } @@ -4122,10 +4117,6 @@ static int memcg_stat_show(struct seq_file *m, void *v) if (memcg1_stats[i] == MEMCG_SWAP && !do_memsw_account()) continue; nr = memcg_page_state(memcg, memcg1_stats[i]); -#ifdef CONFIG_TRANSPARENT_HUGEPAGE - if (memcg1_stats[i] == NR_ANON_THPS) - nr *= HPAGE_PMD_NR; -#endif seq_printf(m, "total_%s %llu\n", memcg1_stat_names[i], (u64)nr * PAGE_SIZE); } @@ -5653,10 +5644,11 @@ static int mem_cgroup_move_account(struct page *page, __mod_lruvec_state(from_vec, NR_ANON_MAPPED, -nr_pages); __mod_lruvec_state(to_vec, NR_ANON_MAPPED, nr_pages); if (PageTransHuge(page)) { - __dec_lruvec_state(from_vec, NR_ANON_THPS); - __inc_lruvec_state(to_vec, NR_ANON_THPS); + __mod_lruvec_state(from_vec, NR_ANON_THPS, + -nr_pages); + __mod_lruvec_state(to_vec, NR_ANON_THPS, + nr_pages); } - } } else { __mod_lruvec_state(from_vec, NR_FILE_PAGES, -nr_pages); diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 469e28f95ce7..1700f52b7869 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -5580,7 +5580,7 @@ void show_free_areas(unsigned int filter, nodemask_t *nodemask) K(node_page_state(pgdat, NR_SHMEM_THPS) * HPAGE_PMD_NR), K(node_page_state(pgdat, NR_SHMEM_PMDMAPPED) * HPAGE_PMD_NR), - K(node_page_state(pgdat, NR_ANON_THPS) * HPAGE_PMD_NR), + K(node_page_state(pgdat, NR_ANON_THPS)), #endif K(node_page_state(pgdat, NR_WRITEBACK_TEMP)), node_page_state(pgdat, NR_KERNEL_STACK_KB), diff --git a/mm/rmap.c b/mm/rmap.c index 08c56aaf72eb..f59e92e26b61 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -1144,7 +1144,8 @@ void do_page_add_anon_rmap(struct page *page, * disabled. */ if (compound) - __inc_lruvec_page_state(page, NR_ANON_THPS); + __mod_lruvec_page_state(page, NR_ANON_THPS, + HPAGE_PMD_NR); __mod_lruvec_page_state(page, NR_ANON_MAPPED, nr); } @@ -1186,7 +1187,7 @@ void page_add_new_anon_rmap(struct page *page, if (hpage_pincount_available(page)) atomic_set(compound_pincount_ptr(page), 0); - __inc_lruvec_page_state(page, NR_ANON_THPS); + __mod_lruvec_page_state(page, NR_ANON_THPS, HPAGE_PMD_NR); } else { /* Anon THP always mapped first with PMD */ VM_BUG_ON_PAGE(PageTransCompound(page), page); @@ -1292,7 +1293,7 @@ static void page_remove_anon_compound_rmap(struct page *page) if (!IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE)) return; - __dec_lruvec_page_state(page, NR_ANON_THPS); + __mod_lruvec_page_state(page, NR_ANON_THPS, -HPAGE_PMD_NR); if (TestClearPageDoubleMap(page)) { /* diff --git a/mm/vmstat.c b/mm/vmstat.c index 663f49ed1fd6..37c9e7b21e1e 100644 --- a/mm/vmstat.c +++ b/mm/vmstat.c @@ -1624,8 +1624,12 @@ static void zoneinfo_show_print(struct seq_file *m, pg_data_t *pgdat, if (is_zone_first_populated(pgdat, zone)) { seq_printf(m, "\n per-node stats"); for (i = 0; i < NR_VM_NODE_STAT_ITEMS; i++) { + unsigned long pages = node_page_state_pages(pgdat, i); + + if (vmstat_item_print_in_thp(i)) + pages /= HPAGE_PMD_NR; seq_printf(m, "\n %-12s %lu", node_stat_name(i), - node_page_state_pages(pgdat, i)); + pages); } } seq_printf(m, @@ -1745,8 +1749,11 @@ static void *vmstat_start(struct seq_file *m, loff_t *pos) v += NR_VM_NUMA_STAT_ITEMS; #endif - for (i = 0; i < NR_VM_NODE_STAT_ITEMS; i++) + for (i = 0; i < NR_VM_NODE_STAT_ITEMS; i++) { v[i] = global_node_page_state_pages(i); + if (vmstat_item_print_in_thp(i)) + v[i] /= HPAGE_PMD_NR; + } v += NR_VM_NODE_STAT_ITEMS; global_dirty_limits(v + NR_DIRTY_BG_THRESHOLD, From patchwork Fri Dec 11 04:19:50 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 11967209 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E6F53C4361B for ; Fri, 11 Dec 2020 04:23:27 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 538C123ED2 for ; Fri, 11 Dec 2020 04:23:27 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 538C123ED2 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id BE8296B006C; Thu, 10 Dec 2020 23:23:26 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id B99346B006E; Thu, 10 Dec 2020 23:23:26 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A890E6B0070; Thu, 10 Dec 2020 23:23:26 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0222.hostedemail.com [216.40.44.222]) by kanga.kvack.org (Postfix) with ESMTP id 92CD36B006C for ; Thu, 10 Dec 2020 23:23:26 -0500 (EST) Received: from smtpin21.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 63E7B180AD802 for ; Fri, 11 Dec 2020 04:23:26 +0000 (UTC) X-FDA: 77579707212.21.sand85_300cf40273fe Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin21.hostedemail.com (Postfix) with ESMTP id 3C3B3180442C4 for ; Fri, 11 Dec 2020 04:23:26 +0000 (UTC) X-HE-Tag: sand85_300cf40273fe X-Filterd-Recvd-Size: 7657 Received: from mail-pg1-f175.google.com (mail-pg1-f175.google.com [209.85.215.175]) by imf46.hostedemail.com (Postfix) with ESMTP for ; Fri, 11 Dec 2020 04:23:25 +0000 (UTC) Received: by mail-pg1-f175.google.com with SMTP id e2so6293987pgi.5 for ; Thu, 10 Dec 2020 20:23:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=+zNdjlNjaGbBbTl1joNFBgmnXlFZ8pwX/30snPIOtJc=; b=1XiX5y0gJqbD5SSIiumZ0LyPZNM1ekosj+wN+jmqz4Ib2jLVWgjuCp2gKxmvofctEf o3J0QyOC89ir8SyzeYt99WS5VUbpEO1OD0lVZwJV11nzX8zLpcPF4Aat9WzIpoDxUZ/D AnUZwFPC04zWG63iJBONvyedsRogVFB1S6ZRxCLS8ywA8NY36S94ra0DufLlrOnRnawL W4ZogsZtWEgQYVR2y4tzXeLpigm1HAJSHvf7pfe41BBxlbcpPfbdLyLo1YVH9Udzs+f5 Jkyy5hE2kUJeB3FLEnOglA71C3Z67W5YhPHafNMjSIs6bvYaooS0m2QwqcioXcEm+dQA ielA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=+zNdjlNjaGbBbTl1joNFBgmnXlFZ8pwX/30snPIOtJc=; b=cIJh3W8gADeF3ghyPmY6vzFrEbDhqqBsnNUX1LaR6EGZBPHwt7zkGl8yqBW/PKsp7O hEQ1CYgUGVC13c+2GZthscKmpe7bojf6qzBKMYk5fCL4W/JuJtSZy4UCrPzymYinpRXi gD3NxWpSV1eaP9y9xTXxXYfWfEwVnLzU2r2+gsS4J4rKfkDsgoLr2HHKgBXxMOWlhUu9 fcZOYeFKk6Yxb1XXuIDF6JVx3px4hecGEFAVqaF7ncGbW5AIcFlhLKMnFIHiCx324zL4 Okwtw+KJ0VM3eXdnLSCM/hT42D+RqtvqbioZif5pRmjuMnq9P8J2OQku51xdO/ayjvkz sodQ== X-Gm-Message-State: AOAM531JZnnXwvg1MkXKLmUVOUbLIcoooQKV9iM9hpkqdlwy9wlRpx2+ slS6AlMHpW3pnWzg4RRVw5DQIA== X-Google-Smtp-Source: ABdhPJxr55yXFTu3oGTg5HJKqTWWlt1XbAYA22QdUZ72kpiYjt+UFd6FsjrFKMq14umappTGj9ibRA== X-Received: by 2002:a63:da03:: with SMTP id c3mr9789954pgh.133.1607660604856; Thu, 10 Dec 2020 20:23:24 -0800 (PST) Received: from localhost.localdomain ([103.136.221.66]) by smtp.gmail.com with ESMTPSA id 19sm8623352pfu.85.2020.12.10.20.23.17 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 10 Dec 2020 20:23:24 -0800 (PST) From: Muchun Song To: gregkh@linuxfoundation.org, rafael@kernel.org, adobriyan@gmail.com, akpm@linux-foundation.org, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, hughd@google.com, shakeelb@google.com, guro@fb.com, samitolvanen@google.com, feng.tang@intel.com, neilb@suse.de, iamjoonsoo.kim@lge.com, rdunlap@infradead.org Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, Muchun Song Subject: [PATCH v4 3/7] mm: memcontrol: convert NR_FILE_THPS account to pages Date: Fri, 11 Dec 2020 12:19:50 +0800 Message-Id: <20201211041954.79543-4-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20201211041954.79543-1-songmuchun@bytedance.com> References: <20201211041954.79543-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The unit of NR_FILE_THPS is HPAGE_PMD_NR. Converrt NR_FILE_THPS account to pages. Signed-off-by: Muchun Song --- drivers/base/node.c | 3 +-- fs/proc/meminfo.c | 2 +- include/linux/mmzone.h | 3 ++- mm/filemap.c | 2 +- mm/huge_memory.c | 3 ++- mm/khugepaged.c | 2 +- mm/memcontrol.c | 5 ++--- 7 files changed, 10 insertions(+), 10 deletions(-) diff --git a/drivers/base/node.c b/drivers/base/node.c index 6da0c3508bc9..d5952f754911 100644 --- a/drivers/base/node.c +++ b/drivers/base/node.c @@ -466,8 +466,7 @@ static ssize_t node_read_meminfo(struct device *dev, HPAGE_PMD_NR), nid, K(node_page_state(pgdat, NR_SHMEM_PMDMAPPED) * HPAGE_PMD_NR), - nid, K(node_page_state(pgdat, NR_FILE_THPS) * - HPAGE_PMD_NR), + nid, K(node_page_state(pgdat, NR_FILE_THPS)), nid, K(node_page_state(pgdat, NR_FILE_PMDMAPPED) * HPAGE_PMD_NR) #endif diff --git a/fs/proc/meminfo.c b/fs/proc/meminfo.c index a635c8a84ddf..7ea4679880c8 100644 --- a/fs/proc/meminfo.c +++ b/fs/proc/meminfo.c @@ -135,7 +135,7 @@ static int meminfo_proc_show(struct seq_file *m, void *v) show_val_kb(m, "ShmemPmdMapped: ", global_node_page_state(NR_SHMEM_PMDMAPPED) * HPAGE_PMD_NR); show_val_kb(m, "FileHugePages: ", - global_node_page_state(NR_FILE_THPS) * HPAGE_PMD_NR); + global_node_page_state(NR_FILE_THPS)); show_val_kb(m, "FilePmdMapped: ", global_node_page_state(NR_FILE_PMDMAPPED) * HPAGE_PMD_NR); #endif diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index 4ac95448421c..67b6598c9ea4 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -215,7 +215,8 @@ enum node_stat_item { */ static __always_inline bool vmstat_item_print_in_thp(enum node_stat_item item) { - return item == NR_ANON_THPS; + return item == NR_ANON_THPS || + item == NR_FILE_THPS; } /* diff --git a/mm/filemap.c b/mm/filemap.c index 78090ee08ac2..9cc8b3ac9eac 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -207,7 +207,7 @@ static void unaccount_page_cache_page(struct address_space *mapping, if (PageTransHuge(page)) __dec_lruvec_page_state(page, NR_SHMEM_THPS); } else if (PageTransHuge(page)) { - __dec_lruvec_page_state(page, NR_FILE_THPS); + __mod_lruvec_page_state(page, NR_FILE_THPS, -HPAGE_PMD_NR); filemap_nr_thps_dec(mapping); } diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 66ec454120de..1e24165fa53a 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -2748,7 +2748,8 @@ int split_huge_page_to_list(struct page *page, struct list_head *list) if (PageSwapBacked(head)) __dec_lruvec_page_state(head, NR_SHMEM_THPS); else - __dec_lruvec_page_state(head, NR_FILE_THPS); + __mod_lruvec_page_state(head, NR_FILE_THPS, + -HPAGE_PMD_NR); } __split_huge_page(page, list, end); diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 494d3cb0b58a..76b3e064a72a 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1869,7 +1869,7 @@ static void collapse_file(struct mm_struct *mm, if (is_shmem) __inc_lruvec_page_state(new_page, NR_SHMEM_THPS); else { - __inc_lruvec_page_state(new_page, NR_FILE_THPS); + __mod_lruvec_page_state(new_page, NR_FILE_THPS, HPAGE_PMD_NR); filemap_nr_thps_inc(mapping); } diff --git a/mm/memcontrol.c b/mm/memcontrol.c index b18e25a5cdf3..04985c8c6a0a 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -1533,7 +1533,7 @@ static struct memory_stat memory_stats[] = { * constant(e.g. powerpc). */ { "anon_thp", PAGE_SIZE, NR_ANON_THPS }, - { "file_thp", 0, NR_FILE_THPS }, + { "file_thp", PAGE_SIZE, NR_FILE_THPS }, { "shmem_thp", 0, NR_SHMEM_THPS }, #endif { "inactive_anon", PAGE_SIZE, NR_INACTIVE_ANON }, @@ -1565,8 +1565,7 @@ static int __init memory_stats_init(void) for (i = 0; i < ARRAY_SIZE(memory_stats); i++) { #ifdef CONFIG_TRANSPARENT_HUGEPAGE - if (memory_stats[i].idx == NR_FILE_THPS || - memory_stats[i].idx == NR_SHMEM_THPS) + if (memory_stats[i].idx == NR_SHMEM_THPS) memory_stats[i].ratio = HPAGE_PMD_SIZE; #endif VM_BUG_ON(!memory_stats[i].ratio); From patchwork Fri Dec 11 04:19:51 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 11967211 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.7 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,UNWANTED_LANGUAGE_BODY, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5E41FC4361B for ; Fri, 11 Dec 2020 04:23:35 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id EC8BE23ED2 for ; Fri, 11 Dec 2020 04:23:34 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EC8BE23ED2 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 8B4AA6B006E; Thu, 10 Dec 2020 23:23:34 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 8640A6B0070; Thu, 10 Dec 2020 23:23:34 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 77C0E6B0071; Thu, 10 Dec 2020 23:23:34 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0022.hostedemail.com [216.40.44.22]) by kanga.kvack.org (Postfix) with ESMTP id 621FE6B006E for ; Thu, 10 Dec 2020 23:23:34 -0500 (EST) Received: from smtpin25.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 202722485 for ; Fri, 11 Dec 2020 04:23:34 +0000 (UTC) X-FDA: 77579707548.25.cap62_63102d9273fe Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin25.hostedemail.com (Postfix) with ESMTP id 0055F1804E3A1 for ; Fri, 11 Dec 2020 04:23:33 +0000 (UTC) X-HE-Tag: cap62_63102d9273fe X-Filterd-Recvd-Size: 9971 Received: from mail-pg1-f181.google.com (mail-pg1-f181.google.com [209.85.215.181]) by imf32.hostedemail.com (Postfix) with ESMTP for ; Fri, 11 Dec 2020 04:23:33 +0000 (UTC) Received: by mail-pg1-f181.google.com with SMTP id 143so185990pge.10 for ; Thu, 10 Dec 2020 20:23:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=NbJhG/dST+z4diHKFuM1zuN6LseIAyC0A7txgFqKs14=; b=GORbjDWZI6+50DEGT4dnLtiziTalV0Lt8P/furd1Fs3XrM/sZzpPQvk7840/8W+QS7 Uq7HXKX1RrX18znav0P/3a3kxYtMH6NkjZRU26+GCv4Kbo3qjDVfPOkyQQ3p4Wh92kbb NtCmT1EPhw04P9OLkJQzyEtvA+V3E4gAbMOEtguB9i062ms/j4E6atxS6ntEm+1/wHv8 1clU8ncAIiYEXL5poqcfd8klzCtr1AiIoTopGRXAtJSqzloZO1dWTrnbyu7cBgzcLBFu ZdYc1WBtXQo9Y0frkW9q6UlrJ1RGH6Mk9/qoxjlOm6bbTzkEnx1dvwnm+xGXrlh/nvTL 4iTA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=NbJhG/dST+z4diHKFuM1zuN6LseIAyC0A7txgFqKs14=; b=ApSUsgLe8ow8mWRuE1R9wyKurDPJt/BRMu+60M8ve/qcuZELVSoJHAuHExJN1IKTSx ILsMcQeiMYnevHUbD6iVOg9JnwE9gThOkD4q7QUxPH+eM1hbge+Hu1QOM4Cj4KgimGq4 NzD4reJ6d4SGaYgp0TTBfK0fTkRzQzIL3eI3kartpq6KxcvdYfyhQ+5LVEu5TwX8vVtl JF/sj/a5JaMjufUCUK9X5BptFt51l3ZTG4QszUupU7oPIyIRtsETfqb9swRHDQUxkKnY eXWliqSrkh0cGmN3XArZ7tVysrfzQkHy8YCF1YBpugY5wWMoG9Jisb5vn1uKp04kBNbD jIew== X-Gm-Message-State: AOAM531kTx2WxlqTyv8mhpp+SpymSjjNu/hb61vFXv5tXcXN0no34sy5 0HbJGlAqK7c9A+lyli5KEsflx2TkTmJKgn5L X-Google-Smtp-Source: ABdhPJyos71clKQbmypkn6DJcpPrhNuiTt6C+WbTDEwBPzMpExl/Tq/F9HfnC6T3EcmJ8Ml75UOqew== X-Received: by 2002:a62:7e09:0:b029:19e:9fdd:80a3 with SMTP id z9-20020a627e090000b029019e9fdd80a3mr8794253pfc.24.1607660612143; Thu, 10 Dec 2020 20:23:32 -0800 (PST) Received: from localhost.localdomain ([103.136.221.66]) by smtp.gmail.com with ESMTPSA id 19sm8623352pfu.85.2020.12.10.20.23.25 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 10 Dec 2020 20:23:31 -0800 (PST) From: Muchun Song To: gregkh@linuxfoundation.org, rafael@kernel.org, adobriyan@gmail.com, akpm@linux-foundation.org, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, hughd@google.com, shakeelb@google.com, guro@fb.com, samitolvanen@google.com, feng.tang@intel.com, neilb@suse.de, iamjoonsoo.kim@lge.com, rdunlap@infradead.org Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, Muchun Song Subject: [PATCH v4 4/7] mm: memcontrol: convert NR_SHMEM_THPS account to pages Date: Fri, 11 Dec 2020 12:19:51 +0800 Message-Id: <20201211041954.79543-5-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20201211041954.79543-1-songmuchun@bytedance.com> References: <20201211041954.79543-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The unit of NR_SHMEM_THPS is HPAGE_PMD_NR. Convert NR_SHMEM_THPS account to pages Signed-off-by: Muchun Song --- drivers/base/node.c | 3 +-- fs/proc/meminfo.c | 2 +- include/linux/mmzone.h | 3 ++- mm/filemap.c | 2 +- mm/huge_memory.c | 3 ++- mm/khugepaged.c | 2 +- mm/memcontrol.c | 26 ++------------------------ mm/page_alloc.c | 2 +- mm/shmem.c | 3 ++- 9 files changed, 13 insertions(+), 33 deletions(-) diff --git a/drivers/base/node.c b/drivers/base/node.c index d5952f754911..6d5ac6ffb6e1 100644 --- a/drivers/base/node.c +++ b/drivers/base/node.c @@ -462,8 +462,7 @@ static ssize_t node_read_meminfo(struct device *dev, #ifdef CONFIG_TRANSPARENT_HUGEPAGE , nid, K(node_page_state(pgdat, NR_ANON_THPS)), - nid, K(node_page_state(pgdat, NR_SHMEM_THPS) * - HPAGE_PMD_NR), + nid, K(node_page_state(pgdat, NR_SHMEM_THPS)), nid, K(node_page_state(pgdat, NR_SHMEM_PMDMAPPED) * HPAGE_PMD_NR), nid, K(node_page_state(pgdat, NR_FILE_THPS)), diff --git a/fs/proc/meminfo.c b/fs/proc/meminfo.c index 7ea4679880c8..cfb107eaa3e6 100644 --- a/fs/proc/meminfo.c +++ b/fs/proc/meminfo.c @@ -131,7 +131,7 @@ static int meminfo_proc_show(struct seq_file *m, void *v) show_val_kb(m, "AnonHugePages: ", global_node_page_state(NR_ANON_THPS)); show_val_kb(m, "ShmemHugePages: ", - global_node_page_state(NR_SHMEM_THPS) * HPAGE_PMD_NR); + global_node_page_state(NR_SHMEM_THPS)); show_val_kb(m, "ShmemPmdMapped: ", global_node_page_state(NR_SHMEM_PMDMAPPED) * HPAGE_PMD_NR); show_val_kb(m, "FileHugePages: ", diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index 67b6598c9ea4..4f49af38dced 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -216,7 +216,8 @@ enum node_stat_item { static __always_inline bool vmstat_item_print_in_thp(enum node_stat_item item) { return item == NR_ANON_THPS || - item == NR_FILE_THPS; + item == NR_FILE_THPS || + item == NR_SHMEM_THPS; } /* diff --git a/mm/filemap.c b/mm/filemap.c index 9cc8b3ac9eac..c653717b92b6 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -205,7 +205,7 @@ static void unaccount_page_cache_page(struct address_space *mapping, if (PageSwapBacked(page)) { __mod_lruvec_page_state(page, NR_SHMEM, -nr); if (PageTransHuge(page)) - __dec_lruvec_page_state(page, NR_SHMEM_THPS); + __mod_lruvec_page_state(page, NR_SHMEM_THPS, -HPAGE_PMD_NR); } else if (PageTransHuge(page)) { __mod_lruvec_page_state(page, NR_FILE_THPS, -HPAGE_PMD_NR); filemap_nr_thps_dec(mapping); diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 1e24165fa53a..ac552807492e 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -2746,7 +2746,8 @@ int split_huge_page_to_list(struct page *page, struct list_head *list) spin_unlock(&ds_queue->split_queue_lock); if (mapping) { if (PageSwapBacked(head)) - __dec_lruvec_page_state(head, NR_SHMEM_THPS); + __mod_lruvec_page_state(head, NR_SHMEM_THPS, + -HPAGE_PMD_NR); else __mod_lruvec_page_state(head, NR_FILE_THPS, -HPAGE_PMD_NR); diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 76b3e064a72a..2f1c22331188 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1867,7 +1867,7 @@ static void collapse_file(struct mm_struct *mm, } if (is_shmem) - __inc_lruvec_page_state(new_page, NR_SHMEM_THPS); + __mod_lruvec_page_state(new_page, NR_SHMEM_THPS, HPAGE_PMD_NR); else { __mod_lruvec_page_state(new_page, NR_FILE_THPS, HPAGE_PMD_NR); filemap_nr_thps_inc(mapping); diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 04985c8c6a0a..a40797a27f87 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -1515,7 +1515,7 @@ struct memory_stat { unsigned int idx; }; -static struct memory_stat memory_stats[] = { +static const struct memory_stat memory_stats[] = { { "anon", PAGE_SIZE, NR_ANON_MAPPED }, { "file", PAGE_SIZE, NR_FILE_PAGES }, { "kernel_stack", 1024, NR_KERNEL_STACK_KB }, @@ -1527,14 +1527,9 @@ static struct memory_stat memory_stats[] = { { "file_dirty", PAGE_SIZE, NR_FILE_DIRTY }, { "file_writeback", PAGE_SIZE, NR_WRITEBACK }, #ifdef CONFIG_TRANSPARENT_HUGEPAGE - /* - * The ratio will be initialized in memory_stats_init(). Because - * on some architectures, the macro of HPAGE_PMD_SIZE is not - * constant(e.g. powerpc). - */ { "anon_thp", PAGE_SIZE, NR_ANON_THPS }, { "file_thp", PAGE_SIZE, NR_FILE_THPS }, - { "shmem_thp", 0, NR_SHMEM_THPS }, + { "shmem_thp", PAGE_SIZE, NR_SHMEM_THPS }, #endif { "inactive_anon", PAGE_SIZE, NR_INACTIVE_ANON }, { "active_anon", PAGE_SIZE, NR_ACTIVE_ANON }, @@ -1559,23 +1554,6 @@ static struct memory_stat memory_stats[] = { { "workingset_nodereclaim", 1, WORKINGSET_NODERECLAIM }, }; -static int __init memory_stats_init(void) -{ - int i; - - for (i = 0; i < ARRAY_SIZE(memory_stats); i++) { -#ifdef CONFIG_TRANSPARENT_HUGEPAGE - if (memory_stats[i].idx == NR_SHMEM_THPS) - memory_stats[i].ratio = HPAGE_PMD_SIZE; -#endif - VM_BUG_ON(!memory_stats[i].ratio); - VM_BUG_ON(memory_stats[i].idx >= MEMCG_NR_STAT); - } - - return 0; -} -pure_initcall(memory_stats_init); - static char *memory_stat_format(struct mem_cgroup *memcg) { struct seq_buf s; diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 1700f52b7869..720fb5a220b6 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -5577,7 +5577,7 @@ void show_free_areas(unsigned int filter, nodemask_t *nodemask) K(node_page_state(pgdat, NR_WRITEBACK)), K(node_page_state(pgdat, NR_SHMEM)), #ifdef CONFIG_TRANSPARENT_HUGEPAGE - K(node_page_state(pgdat, NR_SHMEM_THPS) * HPAGE_PMD_NR), + K(node_page_state(pgdat, NR_SHMEM_THPS)), K(node_page_state(pgdat, NR_SHMEM_PMDMAPPED) * HPAGE_PMD_NR), K(node_page_state(pgdat, NR_ANON_THPS)), diff --git a/mm/shmem.c b/mm/shmem.c index 53d84d2c9fe5..b2ca3beabc19 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -713,7 +713,8 @@ static int shmem_add_to_page_cache(struct page *page, } if (PageTransHuge(page)) { count_vm_event(THP_FILE_ALLOC); - __inc_lruvec_page_state(page, NR_SHMEM_THPS); + __mod_lruvec_page_state(page, NR_SHMEM_THPS, + HPAGE_PMD_NR); } mapping->nrpages += nr; __mod_lruvec_page_state(page, NR_FILE_PAGES, nr); From patchwork Fri Dec 11 04:19:52 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 11967213 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 95514C433FE for ; Fri, 11 Dec 2020 04:23:42 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 3BF4E23EF1 for ; Fri, 11 Dec 2020 04:23:42 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3BF4E23EF1 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id C3B256B0070; Thu, 10 Dec 2020 23:23:41 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id BE61A6B0071; Thu, 10 Dec 2020 23:23:41 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A3A696B0072; Thu, 10 Dec 2020 23:23:41 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0156.hostedemail.com [216.40.44.156]) by kanga.kvack.org (Postfix) with ESMTP id 8062E6B0070 for ; Thu, 10 Dec 2020 23:23:41 -0500 (EST) Received: from smtpin13.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 4DE75180AD802 for ; Fri, 11 Dec 2020 04:23:41 +0000 (UTC) X-FDA: 77579707842.13.bike48_4803a6f273fe Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin13.hostedemail.com (Postfix) with ESMTP id 2769018140B70 for ; Fri, 11 Dec 2020 04:23:41 +0000 (UTC) X-HE-Tag: bike48_4803a6f273fe X-Filterd-Recvd-Size: 6979 Received: from mail-pl1-f193.google.com (mail-pl1-f193.google.com [209.85.214.193]) by imf39.hostedemail.com (Postfix) with ESMTP for ; Fri, 11 Dec 2020 04:23:40 +0000 (UTC) Received: by mail-pl1-f193.google.com with SMTP id j1so3972216pld.3 for ; Thu, 10 Dec 2020 20:23:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=GQUFW73mdqeVNnWmOCUXrriPRd8OoQbYZkReps7lWmE=; b=IjskIXxMWCN/ytFBKw+DiAXxwBYCNKF8JYU6iVwQmlV4cOw6vNaY1P3GrQ1tyoaM9Y LGNX7hCXCHbchH5rP4RRe4V9p2J6KNK5d+n6pNqTcWLgaQl5e6zgcFFXn26pC5V98oKH ChD3XxNRpjpFKsRgxoWhrRl3CJzYll1nkGHxrs8VD5kbyoR5RDageIyV/Pdrr5APQ0Hm fD0x2O/ifiMYM1Ztriz5/eDvN9WjdQ6SYLXPN14x7R2j3v8AGb3gRuChTUYnqUWV0/pV KX51x3M9Zsv6f1MvSNxvn/1/c4Y4CM9Ne7pKYum2xNsYxOizOxI1ztLeDsH6gWgq/tEu EYfg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=GQUFW73mdqeVNnWmOCUXrriPRd8OoQbYZkReps7lWmE=; b=P2PKQ7qFr5xPhodoSJTDRr/w3AI9PzQTpAHW26+49MNvL9hiGsLv8UBLGvEU4gI66W gaZOUJgib5yINCmJDits+528dy6tM9d6cPujQzrYjN5VOTmzdgTSWiWwDcyWeNEla8b6 ryPjNdDT22q7HmMbQpf8gUobFZqL5pP6hrh9Rkug5smtgRE2uyLRprANYLw0Ud9GjPt5 4RJuMhLmF+wxxi7e8vln3YS25uYQ4c7GdvtZ5pHlhmhgM+SfmQz9g+8ESCLcUjwzWaiX N+rBW5Th9MUvyGXtb4TLscWXKIiGyKaZ/I4uj3dd5PgpKwbGy7ut+LGZLxJZ2CWTSBMq Ym/A== X-Gm-Message-State: AOAM533GpWOFC+NA3mlSYTGXzqlb/QRIoyZYh6RwjThhvgJkh3oLolVt 4bShLYMvHg4jr7DM3OzFCX/dsw== X-Google-Smtp-Source: ABdhPJzeiQSfJfIhgFUu9W2wTkNi1XOhkEXOXCWkca4m+6vxOOnVMFAG744rfQkyKjRaEl4HhunhnQ== X-Received: by 2002:a17:902:b213:b029:db:3a3e:d8ad with SMTP id t19-20020a170902b213b02900db3a3ed8admr9494640plr.73.1607660619419; Thu, 10 Dec 2020 20:23:39 -0800 (PST) Received: from localhost.localdomain ([103.136.221.66]) by smtp.gmail.com with ESMTPSA id 19sm8623352pfu.85.2020.12.10.20.23.32 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 10 Dec 2020 20:23:38 -0800 (PST) From: Muchun Song To: gregkh@linuxfoundation.org, rafael@kernel.org, adobriyan@gmail.com, akpm@linux-foundation.org, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, hughd@google.com, shakeelb@google.com, guro@fb.com, samitolvanen@google.com, feng.tang@intel.com, neilb@suse.de, iamjoonsoo.kim@lge.com, rdunlap@infradead.org Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, Muchun Song Subject: [PATCH v4 5/7] mm: memcontrol: convert NR_SHMEM_PMDMAPPED account to pages Date: Fri, 11 Dec 2020 12:19:52 +0800 Message-Id: <20201211041954.79543-6-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20201211041954.79543-1-songmuchun@bytedance.com> References: <20201211041954.79543-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The unit of NR_SHMEM_PMDMAPPED is HPAGE_PMD_NR. Convert NR_SHMEM_PMDMAPPED account to pages. Signed-off-by: Muchun Song --- drivers/base/node.c | 3 +-- fs/proc/meminfo.c | 2 +- include/linux/mmzone.h | 3 ++- mm/page_alloc.c | 3 +-- mm/rmap.c | 6 ++++-- 5 files changed, 9 insertions(+), 8 deletions(-) diff --git a/drivers/base/node.c b/drivers/base/node.c index 6d5ac6ffb6e1..7a66aefe4e46 100644 --- a/drivers/base/node.c +++ b/drivers/base/node.c @@ -463,8 +463,7 @@ static ssize_t node_read_meminfo(struct device *dev, , nid, K(node_page_state(pgdat, NR_ANON_THPS)), nid, K(node_page_state(pgdat, NR_SHMEM_THPS)), - nid, K(node_page_state(pgdat, NR_SHMEM_PMDMAPPED) * - HPAGE_PMD_NR), + nid, K(node_page_state(pgdat, NR_SHMEM_PMDMAPPED)), nid, K(node_page_state(pgdat, NR_FILE_THPS)), nid, K(node_page_state(pgdat, NR_FILE_PMDMAPPED) * HPAGE_PMD_NR) diff --git a/fs/proc/meminfo.c b/fs/proc/meminfo.c index cfb107eaa3e6..c61f440570f9 100644 --- a/fs/proc/meminfo.c +++ b/fs/proc/meminfo.c @@ -133,7 +133,7 @@ static int meminfo_proc_show(struct seq_file *m, void *v) show_val_kb(m, "ShmemHugePages: ", global_node_page_state(NR_SHMEM_THPS)); show_val_kb(m, "ShmemPmdMapped: ", - global_node_page_state(NR_SHMEM_PMDMAPPED) * HPAGE_PMD_NR); + global_node_page_state(NR_SHMEM_PMDMAPPED)); show_val_kb(m, "FileHugePages: ", global_node_page_state(NR_FILE_THPS)); show_val_kb(m, "FilePmdMapped: ", diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index 4f49af38dced..d6b13b9327be 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -217,7 +217,8 @@ static __always_inline bool vmstat_item_print_in_thp(enum node_stat_item item) { return item == NR_ANON_THPS || item == NR_FILE_THPS || - item == NR_SHMEM_THPS; + item == NR_SHMEM_THPS || + item == NR_SHMEM_PMDMAPPED; } /* diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 720fb5a220b6..575fbfeea4b5 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -5578,8 +5578,7 @@ void show_free_areas(unsigned int filter, nodemask_t *nodemask) K(node_page_state(pgdat, NR_SHMEM)), #ifdef CONFIG_TRANSPARENT_HUGEPAGE K(node_page_state(pgdat, NR_SHMEM_THPS)), - K(node_page_state(pgdat, NR_SHMEM_PMDMAPPED) - * HPAGE_PMD_NR), + K(node_page_state(pgdat, NR_SHMEM_PMDMAPPED)), K(node_page_state(pgdat, NR_ANON_THPS)), #endif K(node_page_state(pgdat, NR_WRITEBACK_TEMP)), diff --git a/mm/rmap.c b/mm/rmap.c index f59e92e26b61..3089ad6bf468 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -1219,7 +1219,8 @@ void page_add_file_rmap(struct page *page, bool compound) if (!atomic_inc_and_test(compound_mapcount_ptr(page))) goto out; if (PageSwapBacked(page)) - __inc_node_page_state(page, NR_SHMEM_PMDMAPPED); + __mod_lruvec_page_state(page, NR_SHMEM_PMDMAPPED, + HPAGE_PMD_NR); else __inc_node_page_state(page, NR_FILE_PMDMAPPED); } else { @@ -1260,7 +1261,8 @@ static void page_remove_file_rmap(struct page *page, bool compound) if (!atomic_add_negative(-1, compound_mapcount_ptr(page))) return; if (PageSwapBacked(page)) - __dec_node_page_state(page, NR_SHMEM_PMDMAPPED); + __mod_lruvec_page_state(page, NR_SHMEM_PMDMAPPED, + -HPAGE_PMD_NR); else __dec_node_page_state(page, NR_FILE_PMDMAPPED); } else { From patchwork Fri Dec 11 04:19:53 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 11967215 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9E621C433FE for ; Fri, 11 Dec 2020 04:23:49 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 5AF85223E0 for ; Fri, 11 Dec 2020 04:23:49 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5AF85223E0 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id ECE4E6B0071; Thu, 10 Dec 2020 23:23:48 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id E7E646B0072; Thu, 10 Dec 2020 23:23:48 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D6F926B0073; Thu, 10 Dec 2020 23:23:48 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0173.hostedemail.com [216.40.44.173]) by kanga.kvack.org (Postfix) with ESMTP id C19746B0071 for ; Thu, 10 Dec 2020 23:23:48 -0500 (EST) Received: from smtpin01.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 800BE1EF2 for ; Fri, 11 Dec 2020 04:23:48 +0000 (UTC) X-FDA: 77579708136.01.metal86_1005061273fe Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin01.hostedemail.com (Postfix) with ESMTP id 5E9CA10046D6F for ; Fri, 11 Dec 2020 04:23:48 +0000 (UTC) X-HE-Tag: metal86_1005061273fe X-Filterd-Recvd-Size: 6237 Received: from mail-pg1-f196.google.com (mail-pg1-f196.google.com [209.85.215.196]) by imf13.hostedemail.com (Postfix) with ESMTP for ; Fri, 11 Dec 2020 04:23:47 +0000 (UTC) Received: by mail-pg1-f196.google.com with SMTP id n1so3571038pge.8 for ; Thu, 10 Dec 2020 20:23:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=c1LVwFTVmqjgTFBygiYf9KRS/M2MDULFm8W3CcAScWk=; b=sHMPxQiX9HU37cGIlQKYuTVGVR1QPCEbEEnvCmq1qHG5L+syE+EtC++5V++Y8LmEN8 gOrCGT9Ye2l+SD2JnOdPsE0b/2uvmNdTkhYjo26CkdGNqNr9/K0qF49nz7ftVV51cipT UsR8aYpkXMvucEHBXXhdrQMlTS8B+g4KXiPyiBUOu5DFjtoy7vnp++5+5xRaG8+FnYe1 8RRDwH6oPVoJbEst3xbvp3AtHixH6JqaIKrZvrkEduKy0YScajgC30DFxvNqqk/Yzz60 IMWwPzI7dR4ebv/zrcaAyU5VMrVt38yjZW51GIFUjhN7kTvBEiDufL0YmDJs4UOyTn3s nVrQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=c1LVwFTVmqjgTFBygiYf9KRS/M2MDULFm8W3CcAScWk=; b=VqS7F7FxLwKO59iwNCWr/2WQXM0JTpJeXQD/XB4KlijBmSW9gocCSUv3yZYG64/uSW SN+b9ElL6yuFRsWpYa5S5GZDLOogFjKCHWfFnnZ8gkZIrwWoq75Z/Ra4EwtO/0iNFVLX ydLfsDu7eBMXN7gKgQaqzuNpxqNcFY18yBipl575Bae3+ZHEIIL6RBECS8ySI8iFJHRw FepwPHY9nW7lkS57oL6WWNzrAvujcfZX+ZH6OqKg71QsCtZRMhUP3481lEfvg0xca6Vk BM4m3NnXdDNxl9jzp2tbt+kRbLfNcWJJv2bxPDsPOWvJSOhOsl1AR1f1WFeB3gbZWaQ5 F67A== X-Gm-Message-State: AOAM532wlUcZEesoN4i2iP3L7SdxuhzZ9bEriJPvta71fSZ9we1yRhzB NE0snsIqSMSE4F0mSuzCmhQzPQ== X-Google-Smtp-Source: ABdhPJzvI52xrdQRDVgHYSZXJEjSxmWy+Pk9gYCoF+D8vHIorOD99kxCXFDzu/WJ7Akt1GmsLOn10Q== X-Received: by 2002:a63:4950:: with SMTP id y16mr9800008pgk.415.1607660626886; Thu, 10 Dec 2020 20:23:46 -0800 (PST) Received: from localhost.localdomain ([103.136.221.66]) by smtp.gmail.com with ESMTPSA id 19sm8623352pfu.85.2020.12.10.20.23.39 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 10 Dec 2020 20:23:46 -0800 (PST) From: Muchun Song To: gregkh@linuxfoundation.org, rafael@kernel.org, adobriyan@gmail.com, akpm@linux-foundation.org, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, hughd@google.com, shakeelb@google.com, guro@fb.com, samitolvanen@google.com, feng.tang@intel.com, neilb@suse.de, iamjoonsoo.kim@lge.com, rdunlap@infradead.org Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, Muchun Song Subject: [PATCH v4 6/7] mm: memcontrol: convert NR_FILE_PMDMAPPED account to pages Date: Fri, 11 Dec 2020 12:19:53 +0800 Message-Id: <20201211041954.79543-7-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20201211041954.79543-1-songmuchun@bytedance.com> References: <20201211041954.79543-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The unit of NR_FILE_PMDMAPPED is HPAGE_PMD_NR. Convert NR_FILE_PMDMAPPED account to pages. Signed-off-by: Muchun Song --- drivers/base/node.c | 3 +-- fs/proc/meminfo.c | 2 +- include/linux/mmzone.h | 3 ++- mm/rmap.c | 6 ++++-- 4 files changed, 8 insertions(+), 6 deletions(-) diff --git a/drivers/base/node.c b/drivers/base/node.c index 7a66aefe4e46..d02d86aec19f 100644 --- a/drivers/base/node.c +++ b/drivers/base/node.c @@ -465,8 +465,7 @@ static ssize_t node_read_meminfo(struct device *dev, nid, K(node_page_state(pgdat, NR_SHMEM_THPS)), nid, K(node_page_state(pgdat, NR_SHMEM_PMDMAPPED)), nid, K(node_page_state(pgdat, NR_FILE_THPS)), - nid, K(node_page_state(pgdat, NR_FILE_PMDMAPPED) * - HPAGE_PMD_NR) + nid, K(node_page_state(pgdat, NR_FILE_PMDMAPPED)) #endif ); len += hugetlb_report_node_meminfo(buf, len, nid); diff --git a/fs/proc/meminfo.c b/fs/proc/meminfo.c index c61f440570f9..6fa761c9cc78 100644 --- a/fs/proc/meminfo.c +++ b/fs/proc/meminfo.c @@ -137,7 +137,7 @@ static int meminfo_proc_show(struct seq_file *m, void *v) show_val_kb(m, "FileHugePages: ", global_node_page_state(NR_FILE_THPS)); show_val_kb(m, "FilePmdMapped: ", - global_node_page_state(NR_FILE_PMDMAPPED) * HPAGE_PMD_NR); + global_node_page_state(NR_FILE_PMDMAPPED)); #endif #ifdef CONFIG_CMA diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index d6b13b9327be..c587343a3884 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -218,7 +218,8 @@ static __always_inline bool vmstat_item_print_in_thp(enum node_stat_item item) return item == NR_ANON_THPS || item == NR_FILE_THPS || item == NR_SHMEM_THPS || - item == NR_SHMEM_PMDMAPPED; + item == NR_SHMEM_PMDMAPPED || + item == NR_FILE_PMDMAPPED; } /* diff --git a/mm/rmap.c b/mm/rmap.c index 3089ad6bf468..e383c5619501 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -1222,7 +1222,8 @@ void page_add_file_rmap(struct page *page, bool compound) __mod_lruvec_page_state(page, NR_SHMEM_PMDMAPPED, HPAGE_PMD_NR); else - __inc_node_page_state(page, NR_FILE_PMDMAPPED); + __mod_lruvec_page_state(page, NR_FILE_PMDMAPPED, + HPAGE_PMD_NR); } else { if (PageTransCompound(page) && page_mapping(page)) { VM_WARN_ON_ONCE(!PageLocked(page)); @@ -1264,7 +1265,8 @@ static void page_remove_file_rmap(struct page *page, bool compound) __mod_lruvec_page_state(page, NR_SHMEM_PMDMAPPED, -HPAGE_PMD_NR); else - __dec_node_page_state(page, NR_FILE_PMDMAPPED); + __mod_lruvec_page_state(page, NR_FILE_PMDMAPPED, + -HPAGE_PMD_NR); } else { if (!atomic_add_negative(-1, &page->_mapcount)) return; From patchwork Fri Dec 11 04:19:54 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 11967217 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 05A13C4361B for ; Fri, 11 Dec 2020 04:23:57 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id AB06E223E0 for ; Fri, 11 Dec 2020 04:23:56 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AB06E223E0 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 3F82C6B0072; Thu, 10 Dec 2020 23:23:56 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 3AB8E6B0073; Thu, 10 Dec 2020 23:23:56 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2C06F6B0074; Thu, 10 Dec 2020 23:23:56 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0026.hostedemail.com [216.40.44.26]) by kanga.kvack.org (Postfix) with ESMTP id 157A16B0072 for ; Thu, 10 Dec 2020 23:23:56 -0500 (EST) Received: from smtpin28.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id D3791181AEF09 for ; Fri, 11 Dec 2020 04:23:55 +0000 (UTC) X-FDA: 77579708430.28.rose53_2d0955c273fe Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin28.hostedemail.com (Postfix) with ESMTP id B854D6C1F for ; Fri, 11 Dec 2020 04:23:55 +0000 (UTC) X-HE-Tag: rose53_2d0955c273fe X-Filterd-Recvd-Size: 9553 Received: from mail-pj1-f68.google.com (mail-pj1-f68.google.com [209.85.216.68]) by imf31.hostedemail.com (Postfix) with ESMTP for ; Fri, 11 Dec 2020 04:23:55 +0000 (UTC) Received: by mail-pj1-f68.google.com with SMTP id m5so1680689pjv.5 for ; Thu, 10 Dec 2020 20:23:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=V6cgE4Bypope/50+jczAJckIDmu+NX283pqYCpMuFa4=; b=WdKBmuSwKfyBRkQlidbqMK4rXhMtOhxX6s7b71K9mAl848EvGutMDRJlI/5zZezDjF SQ1dfYswVirngLEoUH90Ybd55tva4pkIwo6yYYmH5Z16e1VzvqcgRKSNVe3edbgwJXsN /IBh/OGj6mWWWy5qPdsoGqzcO1jHkTITPvq9bVe5Q8ez7e66cSEfqe0n8VpaV3E8mfl2 pAtJL7Y+vDGsXom08S3+05XPIIonieNQ4K6GadmIWTHVRt7gr/gbFKFlqREEfXZ9mEOF 87egGO6IsZrqoAU+a/QIVF29sbkNmWTKCFFpMM1vYuusJC0tddSKqd/AwOXagjz2AEXQ 9GRg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=V6cgE4Bypope/50+jczAJckIDmu+NX283pqYCpMuFa4=; b=JTvGiCcBuEftlFHPFPpAnYDCuBXII4IIJ9WyyWE3bGevv4yxz6kYRHWDpM9v1W4fEx tzg9UJnaHMIevIszVuK+oXiMOrf2Hg+0qSIF/aiTlysGRH49IsEslh+q+NF+WCdWfNNP MNXJdJwObTtOP6slMQ6udXCBOgOUEDt2BRWR344HZ0w4r9kI295KtCFUpq6Y2vHEvDB8 NLIp4hpWbHgLa1to2FcKHMfYLpyuKsCGWtnSd+u6zICs45Lew7Rd45wPbsse9yHVHI94 tK2OqdRRITGynIlIYXYmDIcrxea1Hg1970hO8p5oHfhL2kzBcswJDWfP/Hp9BFs4Yp5S Kvtg== X-Gm-Message-State: AOAM531x6ZPS0l3VXbrYzcvOXOZ3l0ozHwrJ3oHYTU41uGcFXRXycZ/M sO1UcK9yU4vpu5MLTy9sX/53lg== X-Google-Smtp-Source: ABdhPJwdoIX1p/0NJ1YKn2SsyeylTuzKrv0Zp2zZ/VDEYphVudb4Bo3w4+chjSDK2ZbuOTvuVpPmuw== X-Received: by 2002:a17:902:778e:b029:da:feef:8f2d with SMTP id o14-20020a170902778eb02900dafeef8f2dmr2170797pll.25.1607660634412; Thu, 10 Dec 2020 20:23:54 -0800 (PST) Received: from localhost.localdomain ([103.136.221.66]) by smtp.gmail.com with ESMTPSA id 19sm8623352pfu.85.2020.12.10.20.23.47 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 10 Dec 2020 20:23:53 -0800 (PST) From: Muchun Song To: gregkh@linuxfoundation.org, rafael@kernel.org, adobriyan@gmail.com, akpm@linux-foundation.org, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, hughd@google.com, shakeelb@google.com, guro@fb.com, samitolvanen@google.com, feng.tang@intel.com, neilb@suse.de, iamjoonsoo.kim@lge.com, rdunlap@infradead.org Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, Muchun Song Subject: [PATCH v4 7/7] mm: memcontrol: make the slab calculation consistent Date: Fri, 11 Dec 2020 12:19:54 +0800 Message-Id: <20201211041954.79543-8-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20201211041954.79543-1-songmuchun@bytedance.com> References: <20201211041954.79543-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Although the ratio of the slab is one, we also should read the ratio from the related memory_stats instead of hard-coding. And the local variable of size is already the value of slab_unreclaimable. So we do not need to read again. We can drop the ratio in struct memory_stat. This can make the code clean and simple. And get rid of the awkward mix of static and runtime initialization of the memory_stats table. Signed-off-by: Muchun Song --- mm/memcontrol.c | 105 +++++++++++++++++++++++++++++++++++--------------------- 1 file changed, 66 insertions(+), 39 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index a40797a27f87..eec44918d373 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -1511,49 +1511,71 @@ static bool mem_cgroup_wait_acct_move(struct mem_cgroup *memcg) struct memory_stat { const char *name; - unsigned int ratio; unsigned int idx; }; static const struct memory_stat memory_stats[] = { - { "anon", PAGE_SIZE, NR_ANON_MAPPED }, - { "file", PAGE_SIZE, NR_FILE_PAGES }, - { "kernel_stack", 1024, NR_KERNEL_STACK_KB }, - { "pagetables", PAGE_SIZE, NR_PAGETABLE }, - { "percpu", 1, MEMCG_PERCPU_B }, - { "sock", PAGE_SIZE, MEMCG_SOCK }, - { "shmem", PAGE_SIZE, NR_SHMEM }, - { "file_mapped", PAGE_SIZE, NR_FILE_MAPPED }, - { "file_dirty", PAGE_SIZE, NR_FILE_DIRTY }, - { "file_writeback", PAGE_SIZE, NR_WRITEBACK }, + { "anon", NR_ANON_MAPPED }, + { "file", NR_FILE_PAGES }, + { "kernel_stack", NR_KERNEL_STACK_KB }, + { "pagetables", NR_PAGETABLE }, + { "percpu", MEMCG_PERCPU_B }, + { "sock", MEMCG_SOCK }, + { "shmem", NR_SHMEM }, + { "file_mapped", NR_FILE_MAPPED }, + { "file_dirty", NR_FILE_DIRTY }, + { "file_writeback", NR_WRITEBACK }, #ifdef CONFIG_TRANSPARENT_HUGEPAGE - { "anon_thp", PAGE_SIZE, NR_ANON_THPS }, - { "file_thp", PAGE_SIZE, NR_FILE_THPS }, - { "shmem_thp", PAGE_SIZE, NR_SHMEM_THPS }, + { "anon_thp", NR_ANON_THPS }, + { "file_thp", NR_FILE_THPS }, + { "shmem_thp", NR_SHMEM_THPS }, #endif - { "inactive_anon", PAGE_SIZE, NR_INACTIVE_ANON }, - { "active_anon", PAGE_SIZE, NR_ACTIVE_ANON }, - { "inactive_file", PAGE_SIZE, NR_INACTIVE_FILE }, - { "active_file", PAGE_SIZE, NR_ACTIVE_FILE }, - { "unevictable", PAGE_SIZE, NR_UNEVICTABLE }, - - /* - * Note: The slab_reclaimable and slab_unreclaimable must be - * together and slab_reclaimable must be in front. - */ - { "slab_reclaimable", 1, NR_SLAB_RECLAIMABLE_B }, - { "slab_unreclaimable", 1, NR_SLAB_UNRECLAIMABLE_B }, + { "inactive_anon", NR_INACTIVE_ANON }, + { "active_anon", NR_ACTIVE_ANON }, + { "inactive_file", NR_INACTIVE_FILE }, + { "active_file", NR_ACTIVE_FILE }, + { "unevictable", NR_UNEVICTABLE }, + { "slab_reclaimable", NR_SLAB_RECLAIMABLE_B }, + { "slab_unreclaimable", NR_SLAB_UNRECLAIMABLE_B }, /* The memory events */ - { "workingset_refault_anon", 1, WORKINGSET_REFAULT_ANON }, - { "workingset_refault_file", 1, WORKINGSET_REFAULT_FILE }, - { "workingset_activate_anon", 1, WORKINGSET_ACTIVATE_ANON }, - { "workingset_activate_file", 1, WORKINGSET_ACTIVATE_FILE }, - { "workingset_restore_anon", 1, WORKINGSET_RESTORE_ANON }, - { "workingset_restore_file", 1, WORKINGSET_RESTORE_FILE }, - { "workingset_nodereclaim", 1, WORKINGSET_NODERECLAIM }, + { "workingset_refault_anon", WORKINGSET_REFAULT_ANON }, + { "workingset_refault_file", WORKINGSET_REFAULT_FILE }, + { "workingset_activate_anon", WORKINGSET_ACTIVATE_ANON }, + { "workingset_activate_file", WORKINGSET_ACTIVATE_FILE }, + { "workingset_restore_anon", WORKINGSET_RESTORE_ANON }, + { "workingset_restore_file", WORKINGSET_RESTORE_FILE }, + { "workingset_nodereclaim", WORKINGSET_NODERECLAIM }, }; +/* Translate stat items to the correct unit for memory.stat output */ +static int memcg_page_state_unit(int item) +{ + switch (item) { + case MEMCG_PERCPU_B: + case NR_SLAB_RECLAIMABLE_B: + case NR_SLAB_UNRECLAIMABLE_B: + case WORKINGSET_REFAULT_ANON: + case WORKINGSET_REFAULT_FILE: + case WORKINGSET_ACTIVATE_ANON: + case WORKINGSET_ACTIVATE_FILE: + case WORKINGSET_RESTORE_ANON: + case WORKINGSET_RESTORE_FILE: + case WORKINGSET_NODERECLAIM: + return 1; + case NR_KERNEL_STACK_KB: + return SZ_1K; + default: + return PAGE_SIZE; + } +} + +static inline unsigned long memcg_page_state_output(struct mem_cgroup *memcg, + int item) +{ + return memcg_page_state(memcg, item) * memcg_page_state_unit(item); +} + static char *memory_stat_format(struct mem_cgroup *memcg) { struct seq_buf s; @@ -1577,13 +1599,12 @@ static char *memory_stat_format(struct mem_cgroup *memcg) for (i = 0; i < ARRAY_SIZE(memory_stats); i++) { u64 size; - size = memcg_page_state(memcg, memory_stats[i].idx); - size *= memory_stats[i].ratio; + size = memcg_page_state_output(memcg, memory_stats[i].idx); seq_buf_printf(&s, "%s %llu\n", memory_stats[i].name, size); if (unlikely(memory_stats[i].idx == NR_SLAB_UNRECLAIMABLE_B)) { - size = memcg_page_state(memcg, NR_SLAB_RECLAIMABLE_B) + - memcg_page_state(memcg, NR_SLAB_UNRECLAIMABLE_B); + size += memcg_page_state_output(memcg, + NR_SLAB_RECLAIMABLE_B); seq_buf_printf(&s, "slab %llu\n", size); } } @@ -6377,6 +6398,12 @@ static int memory_stat_show(struct seq_file *m, void *v) } #ifdef CONFIG_NUMA +static inline unsigned long lruvec_page_state_output(struct lruvec *lruvec, + int item) +{ + return lruvec_page_state(lruvec, item) * memcg_page_state_unit(item); +} + static int memory_numa_stat_show(struct seq_file *m, void *v) { int i; @@ -6394,8 +6421,8 @@ static int memory_numa_stat_show(struct seq_file *m, void *v) struct lruvec *lruvec; lruvec = mem_cgroup_lruvec(memcg, NODE_DATA(nid)); - size = lruvec_page_state(lruvec, memory_stats[i].idx); - size *= memory_stats[i].ratio; + size = lruvec_page_state_output(lruvec, + memory_stats[i].idx); seq_printf(m, " N%d=%llu", nid, size); } seq_putc(m, '\n');