From patchwork Fri Dec 11 04:19:48 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 11967191 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 65D44C4167B for ; Fri, 11 Dec 2020 04:24:53 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2EF7623EF2 for ; Fri, 11 Dec 2020 04:24:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2395348AbgLKEYN (ORCPT ); Thu, 10 Dec 2020 23:24:13 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36288 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2395350AbgLKEXu (ORCPT ); Thu, 10 Dec 2020 23:23:50 -0500 Received: from mail-pf1-x444.google.com (mail-pf1-x444.google.com [IPv6:2607:f8b0:4864:20::444]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 906FBC061794 for ; Thu, 10 Dec 2020 20:23:10 -0800 (PST) Received: by mail-pf1-x444.google.com with SMTP id p4so6221159pfg.0 for ; Thu, 10 Dec 2020 20:23:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=FpYaFCBP9ryR0HnwKGfpggL7bn85I6W2LvY3/jvWBMg=; b=T2ahvN7HcruB45VJZG2OJqfIapWawdOD8QtWwwwuD43vjFPlp2f0Yi8Gslq+3aVBN3 QuexiT59I4+K7zRILtyttGcENl7TzkS+jsjA2Oh+lybl033Bptk3XduoWbvXn6xVDsAL 7ee5f0G+Ld7Nk/1Y3Wh4XlEMomkWzrC1dLBL4zDjO8dWk8nn8cZPsKzWKj443KmhKlbA NKCRfX207fMkpWgeZ/0TF0rb0LteP1ZUhUv6MOH1oj64KFiCmTraUoFBRKZ2bpGW2vj5 EG0+5TFj/Kc9+edCH85DQVHPkanMAhz8lGbgK+VXRAPkdeeGVlMCFhSbRk9PNWxS7rQK g/WQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=FpYaFCBP9ryR0HnwKGfpggL7bn85I6W2LvY3/jvWBMg=; b=DIWFjfq12fWdwLgwe9A/fKhb/ongly8gk8oqUS7QSeDnu9gdIqkjSq7qWTbF3639PA 82P4P5zQ5tTeVbp81vqHkeGnyziQ2h0dpmE+CgNjmQpacjzk+mO44es3FLKcdsA58sNa PyL+/OPlC3Ycq7rlxSmGjsPTe+NnHl46G1+GsRQSIJoQatE3Ssu8NIyNiDSZjW9ZHPnd OwE2eOLk2CAlpsrfVw5cYFwhUWhxA2OrsWdj5L7UAQqTnzkG38ThX5eYTDyGF/nYvXpx 1WUy6+EI5w34nP+TxDcylgkX9BctG1eGkTl/t41HrJAcZt/b0Bw8uvNmIYKjI7548BXY UPng== X-Gm-Message-State: AOAM533IrS2a3bHLxY2c0vj7fv2xiG+lcxZ4/G540nYPsH7WJbChLanf ZWRE4TRxs0rpzKxPdDCbFIUDJg== X-Google-Smtp-Source: ABdhPJz13P4aoeVkSVXowRznFKeGs6JEKzGT5V/ZC2LxOJuCCnhOD8neubcs+2TtN6B0lrJSzDSQTw== X-Received: by 2002:a05:6a00:1596:b029:19d:96b8:6eab with SMTP id u22-20020a056a001596b029019d96b86eabmr9899521pfk.38.1607660590187; Thu, 10 Dec 2020 20:23:10 -0800 (PST) Received: from localhost.localdomain ([103.136.221.66]) by smtp.gmail.com with ESMTPSA id 19sm8623352pfu.85.2020.12.10.20.23.03 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 10 Dec 2020 20:23:09 -0800 (PST) From: Muchun Song To: gregkh@linuxfoundation.org, rafael@kernel.org, adobriyan@gmail.com, akpm@linux-foundation.org, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, hughd@google.com, shakeelb@google.com, guro@fb.com, samitolvanen@google.com, feng.tang@intel.com, neilb@suse.de, iamjoonsoo.kim@lge.com, rdunlap@infradead.org Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, Muchun Song , Michal Hocko Subject: [PATCH v4 1/7] mm: memcontrol: fix NR_ANON_THPS accounting in charge moving Date: Fri, 11 Dec 2020 12:19:48 +0800 Message-Id: <20201211041954.79543-2-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20201211041954.79543-1-songmuchun@bytedance.com> References: <20201211041954.79543-1-songmuchun@bytedance.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org The unit of NR_ANON_THPS is HPAGE_PMD_NR already. So it should inc/dec by one rather than nr_pages. Fixes: 468c398233da ("mm: memcontrol: switch to native NR_ANON_THPS counter") Signed-off-by: Muchun Song Acked-by: Michal Hocko Acked-by: Johannes Weiner Reviewed-by: Roman Gushchin Acked-by: Pankaj Gupta --- mm/memcontrol.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index b80328f52fb4..8818bf64d6fe 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -5653,10 +5653,8 @@ static int mem_cgroup_move_account(struct page *page, __mod_lruvec_state(from_vec, NR_ANON_MAPPED, -nr_pages); __mod_lruvec_state(to_vec, NR_ANON_MAPPED, nr_pages); if (PageTransHuge(page)) { - __mod_lruvec_state(from_vec, NR_ANON_THPS, - -nr_pages); - __mod_lruvec_state(to_vec, NR_ANON_THPS, - nr_pages); + __dec_lruvec_state(from_vec, NR_ANON_THPS); + __inc_lruvec_state(to_vec, NR_ANON_THPS); } } From patchwork Fri Dec 11 04:19:49 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 11967199 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 42526C2BB9A for ; Fri, 11 Dec 2020 04:26:05 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1848D23FB2 for ; Fri, 11 Dec 2020 04:26:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2395395AbgLKEYv (ORCPT ); Thu, 10 Dec 2020 23:24:51 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36386 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2395370AbgLKEY1 (ORCPT ); Thu, 10 Dec 2020 23:24:27 -0500 Received: from mail-pj1-x1042.google.com (mail-pj1-x1042.google.com [IPv6:2607:f8b0:4864:20::1042]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7C59FC0617B0 for ; Thu, 10 Dec 2020 20:23:18 -0800 (PST) Received: by mail-pj1-x1042.google.com with SMTP id l23so1679497pjg.1 for ; Thu, 10 Dec 2020 20:23:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=cU/HrVcwfM3LPFm4rjOQAH8JdzbfJ+ZpL7D7Ks2Uco0=; b=tih4ZHbKA0YgssVEixNJ0hUNRlQ83ass+bYW1yuu3v27Qw+hZwKiLO7xMEW7V9iGfo CL+A5/3YoVRnB1m1CfbOvvdxL6uiv1v8CvG7gKw4Lqm6dLgN8Ih0dD9pkJVjxa2DsOGp HjsANWSDU0LDkKJLCdqXKHlEN7jTGU+zJDrDwyUVrV9i4kCXKTXRf2tAU81mxiMaCZ3J wJj9xeB0NYehtyC1FQ1pBreTX1nJiWNTjK4sEq6AQHFSwQuLlbadTYekoPpxVHudgs4W Ko1GxshKle2QfTsP0MSzCxmnG8MEgEHist32cC9QtFWhohA01FbOh+c+Aayw1n7tTkbJ 9gWQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=cU/HrVcwfM3LPFm4rjOQAH8JdzbfJ+ZpL7D7Ks2Uco0=; b=Fbccfm7GqW4rQxZ+FY+QqXBHsS6DTm0pJPsQb+sEKjWDmdBvieQXTFzkDC0vwHAbIM lLdChP80xz8WmsXv5trz6AXNqkTUwV1C29HVlgjymdc2IrdfsSFrp8LtG80f1CxPmeWI V6mbNe3U3cn6AqScoB4Soxh/tdUnGMm+EP0ldewwEKHGk/+5DqJ5K7/k7KTRHteC7SKG TjzPSr9bqZJLCIgz6mcFx/FcMC7a8aezOIAEsPIO7R2gNY4XjB5K6NJYiltVUNAmWg2I 9cyrZeaIVSP9Wm4554aKzBZOmEb3+cGZiueR72X/9CCMVPyMf7rQSI+XC1qPFmjBptcQ n/zA== X-Gm-Message-State: AOAM530hpq1m/Hq94XJgNiYfsW50pD7ewSB5uTN65YOdhsg/qUc806h1 ysMqYmnpukKO3DI5U1yrDQtD3Q== X-Google-Smtp-Source: ABdhPJwvRIgVYH8myVTOip532m/G1DHFdQPbkxnk9Pu1iXGs/BxE4LZolQ+5F26TEiO0j8O5CqF9nQ== X-Received: by 2002:a17:90b:1b52:: with SMTP id nv18mr11374155pjb.172.1607660597400; Thu, 10 Dec 2020 20:23:17 -0800 (PST) Received: from localhost.localdomain ([103.136.221.66]) by smtp.gmail.com with ESMTPSA id 19sm8623352pfu.85.2020.12.10.20.23.10 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 10 Dec 2020 20:23:16 -0800 (PST) From: Muchun Song To: gregkh@linuxfoundation.org, rafael@kernel.org, adobriyan@gmail.com, akpm@linux-foundation.org, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, hughd@google.com, shakeelb@google.com, guro@fb.com, samitolvanen@google.com, feng.tang@intel.com, neilb@suse.de, iamjoonsoo.kim@lge.com, rdunlap@infradead.org Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, Muchun Song Subject: [PATCH v4 2/7] mm: memcontrol: convert NR_ANON_THPS account to pages Date: Fri, 11 Dec 2020 12:19:49 +0800 Message-Id: <20201211041954.79543-3-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20201211041954.79543-1-songmuchun@bytedance.com> References: <20201211041954.79543-1-songmuchun@bytedance.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org The unit of NR_ANON_THPS is HPAGE_PMD_NR. Convert the NR_ANON_THPS account to pages. Signed-off-by: Muchun Song --- drivers/base/node.c | 15 +++++++++------ fs/proc/meminfo.c | 2 +- include/linux/mmzone.h | 9 +++++++++ mm/huge_memory.c | 3 ++- mm/memcontrol.c | 20 ++++++-------------- mm/page_alloc.c | 2 +- mm/rmap.c | 7 ++++--- mm/vmstat.c | 11 +++++++++-- 8 files changed, 41 insertions(+), 28 deletions(-) diff --git a/drivers/base/node.c b/drivers/base/node.c index 04f71c7bc3f8..6da0c3508bc9 100644 --- a/drivers/base/node.c +++ b/drivers/base/node.c @@ -461,8 +461,7 @@ static ssize_t node_read_meminfo(struct device *dev, nid, K(sunreclaimable) #ifdef CONFIG_TRANSPARENT_HUGEPAGE , - nid, K(node_page_state(pgdat, NR_ANON_THPS) * - HPAGE_PMD_NR), + nid, K(node_page_state(pgdat, NR_ANON_THPS)), nid, K(node_page_state(pgdat, NR_SHMEM_THPS) * HPAGE_PMD_NR), nid, K(node_page_state(pgdat, NR_SHMEM_PMDMAPPED) * @@ -519,10 +518,14 @@ static ssize_t node_read_vmstat(struct device *dev, sum_zone_numa_state(nid, i)); #endif - for (i = 0; i < NR_VM_NODE_STAT_ITEMS; i++) - len += sysfs_emit_at(buf, len, "%s %lu\n", - node_stat_name(i), - node_page_state_pages(pgdat, i)); + for (i = 0; i < NR_VM_NODE_STAT_ITEMS; i++) { + unsigned long pages = node_page_state_pages(pgdat, i); + + if (vmstat_item_print_in_thp(i)) + pages /= HPAGE_PMD_NR; + len += sysfs_emit_at(buf, len, "%s %lu\n", node_stat_name(i), + pages); + } return len; } diff --git a/fs/proc/meminfo.c b/fs/proc/meminfo.c index d6fc74619625..a635c8a84ddf 100644 --- a/fs/proc/meminfo.c +++ b/fs/proc/meminfo.c @@ -129,7 +129,7 @@ static int meminfo_proc_show(struct seq_file *m, void *v) #ifdef CONFIG_TRANSPARENT_HUGEPAGE show_val_kb(m, "AnonHugePages: ", - global_node_page_state(NR_ANON_THPS) * HPAGE_PMD_NR); + global_node_page_state(NR_ANON_THPS)); show_val_kb(m, "ShmemHugePages: ", global_node_page_state(NR_SHMEM_THPS) * HPAGE_PMD_NR); show_val_kb(m, "ShmemPmdMapped: ", diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index b593316bff3d..4ac95448421c 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -210,6 +210,15 @@ enum node_stat_item { }; /* + * Returns true if the item should prints in THPs. The /proc/vmstat currently + * prints number of anon, file and shmem THPs. But the item is charged in pages. + */ +static __always_inline bool vmstat_item_print_in_thp(enum node_stat_item item) +{ + return item == NR_ANON_THPS; +} + +/* * Returns true if the value is measured in bytes (most vmstat values are * measured in pages). This defines the API part, the internal representation * might be different. diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 10dd3cae5f53..66ec454120de 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -2178,7 +2178,8 @@ static void __split_huge_pmd_locked(struct vm_area_struct *vma, pmd_t *pmd, lock_page_memcg(page); if (atomic_add_negative(-1, compound_mapcount_ptr(page))) { /* Last compound_mapcount is gone. */ - __dec_lruvec_page_state(page, NR_ANON_THPS); + __mod_lruvec_page_state(page, NR_ANON_THPS, + -HPAGE_PMD_NR); if (TestClearPageDoubleMap(page)) { /* No need in mapcount reference anymore */ for (i = 0; i < HPAGE_PMD_NR; i++) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 8818bf64d6fe..b18e25a5cdf3 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -1532,7 +1532,7 @@ static struct memory_stat memory_stats[] = { * on some architectures, the macro of HPAGE_PMD_SIZE is not * constant(e.g. powerpc). */ - { "anon_thp", 0, NR_ANON_THPS }, + { "anon_thp", PAGE_SIZE, NR_ANON_THPS }, { "file_thp", 0, NR_FILE_THPS }, { "shmem_thp", 0, NR_SHMEM_THPS }, #endif @@ -1565,8 +1565,7 @@ static int __init memory_stats_init(void) for (i = 0; i < ARRAY_SIZE(memory_stats); i++) { #ifdef CONFIG_TRANSPARENT_HUGEPAGE - if (memory_stats[i].idx == NR_ANON_THPS || - memory_stats[i].idx == NR_FILE_THPS || + if (memory_stats[i].idx == NR_FILE_THPS || memory_stats[i].idx == NR_SHMEM_THPS) memory_stats[i].ratio = HPAGE_PMD_SIZE; #endif @@ -4088,10 +4087,6 @@ static int memcg_stat_show(struct seq_file *m, void *v) if (memcg1_stats[i] == MEMCG_SWAP && !do_memsw_account()) continue; nr = memcg_page_state_local(memcg, memcg1_stats[i]); -#ifdef CONFIG_TRANSPARENT_HUGEPAGE - if (memcg1_stats[i] == NR_ANON_THPS) - nr *= HPAGE_PMD_NR; -#endif seq_printf(m, "%s %lu\n", memcg1_stat_names[i], nr * PAGE_SIZE); } @@ -4122,10 +4117,6 @@ static int memcg_stat_show(struct seq_file *m, void *v) if (memcg1_stats[i] == MEMCG_SWAP && !do_memsw_account()) continue; nr = memcg_page_state(memcg, memcg1_stats[i]); -#ifdef CONFIG_TRANSPARENT_HUGEPAGE - if (memcg1_stats[i] == NR_ANON_THPS) - nr *= HPAGE_PMD_NR; -#endif seq_printf(m, "total_%s %llu\n", memcg1_stat_names[i], (u64)nr * PAGE_SIZE); } @@ -5653,10 +5644,11 @@ static int mem_cgroup_move_account(struct page *page, __mod_lruvec_state(from_vec, NR_ANON_MAPPED, -nr_pages); __mod_lruvec_state(to_vec, NR_ANON_MAPPED, nr_pages); if (PageTransHuge(page)) { - __dec_lruvec_state(from_vec, NR_ANON_THPS); - __inc_lruvec_state(to_vec, NR_ANON_THPS); + __mod_lruvec_state(from_vec, NR_ANON_THPS, + -nr_pages); + __mod_lruvec_state(to_vec, NR_ANON_THPS, + nr_pages); } - } } else { __mod_lruvec_state(from_vec, NR_FILE_PAGES, -nr_pages); diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 469e28f95ce7..1700f52b7869 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -5580,7 +5580,7 @@ void show_free_areas(unsigned int filter, nodemask_t *nodemask) K(node_page_state(pgdat, NR_SHMEM_THPS) * HPAGE_PMD_NR), K(node_page_state(pgdat, NR_SHMEM_PMDMAPPED) * HPAGE_PMD_NR), - K(node_page_state(pgdat, NR_ANON_THPS) * HPAGE_PMD_NR), + K(node_page_state(pgdat, NR_ANON_THPS)), #endif K(node_page_state(pgdat, NR_WRITEBACK_TEMP)), node_page_state(pgdat, NR_KERNEL_STACK_KB), diff --git a/mm/rmap.c b/mm/rmap.c index 08c56aaf72eb..f59e92e26b61 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -1144,7 +1144,8 @@ void do_page_add_anon_rmap(struct page *page, * disabled. */ if (compound) - __inc_lruvec_page_state(page, NR_ANON_THPS); + __mod_lruvec_page_state(page, NR_ANON_THPS, + HPAGE_PMD_NR); __mod_lruvec_page_state(page, NR_ANON_MAPPED, nr); } @@ -1186,7 +1187,7 @@ void page_add_new_anon_rmap(struct page *page, if (hpage_pincount_available(page)) atomic_set(compound_pincount_ptr(page), 0); - __inc_lruvec_page_state(page, NR_ANON_THPS); + __mod_lruvec_page_state(page, NR_ANON_THPS, HPAGE_PMD_NR); } else { /* Anon THP always mapped first with PMD */ VM_BUG_ON_PAGE(PageTransCompound(page), page); @@ -1292,7 +1293,7 @@ static void page_remove_anon_compound_rmap(struct page *page) if (!IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE)) return; - __dec_lruvec_page_state(page, NR_ANON_THPS); + __mod_lruvec_page_state(page, NR_ANON_THPS, -HPAGE_PMD_NR); if (TestClearPageDoubleMap(page)) { /* diff --git a/mm/vmstat.c b/mm/vmstat.c index 663f49ed1fd6..37c9e7b21e1e 100644 --- a/mm/vmstat.c +++ b/mm/vmstat.c @@ -1624,8 +1624,12 @@ static void zoneinfo_show_print(struct seq_file *m, pg_data_t *pgdat, if (is_zone_first_populated(pgdat, zone)) { seq_printf(m, "\n per-node stats"); for (i = 0; i < NR_VM_NODE_STAT_ITEMS; i++) { + unsigned long pages = node_page_state_pages(pgdat, i); + + if (vmstat_item_print_in_thp(i)) + pages /= HPAGE_PMD_NR; seq_printf(m, "\n %-12s %lu", node_stat_name(i), - node_page_state_pages(pgdat, i)); + pages); } } seq_printf(m, @@ -1745,8 +1749,11 @@ static void *vmstat_start(struct seq_file *m, loff_t *pos) v += NR_VM_NUMA_STAT_ITEMS; #endif - for (i = 0; i < NR_VM_NODE_STAT_ITEMS; i++) + for (i = 0; i < NR_VM_NODE_STAT_ITEMS; i++) { v[i] = global_node_page_state_pages(i); + if (vmstat_item_print_in_thp(i)) + v[i] /= HPAGE_PMD_NR; + } v += NR_VM_NODE_STAT_ITEMS; global_dirty_limits(v + NR_DIRTY_BG_THRESHOLD, From patchwork Fri Dec 11 04:19:50 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 11967197 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 472E6C47434 for ; Fri, 11 Dec 2020 04:26:05 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2CE0D23FB3 for ; Fri, 11 Dec 2020 04:26:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2395389AbgLKEYp (ORCPT ); Thu, 10 Dec 2020 23:24:45 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36388 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2395371AbgLKEY1 (ORCPT ); Thu, 10 Dec 2020 23:24:27 -0500 Received: from mail-pf1-x42f.google.com (mail-pf1-x42f.google.com [IPv6:2607:f8b0:4864:20::42f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 51128C06138C for ; Thu, 10 Dec 2020 20:23:25 -0800 (PST) Received: by mail-pf1-x42f.google.com with SMTP id 11so6205993pfu.4 for ; Thu, 10 Dec 2020 20:23:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=+zNdjlNjaGbBbTl1joNFBgmnXlFZ8pwX/30snPIOtJc=; b=1XiX5y0gJqbD5SSIiumZ0LyPZNM1ekosj+wN+jmqz4Ib2jLVWgjuCp2gKxmvofctEf o3J0QyOC89ir8SyzeYt99WS5VUbpEO1OD0lVZwJV11nzX8zLpcPF4Aat9WzIpoDxUZ/D AnUZwFPC04zWG63iJBONvyedsRogVFB1S6ZRxCLS8ywA8NY36S94ra0DufLlrOnRnawL W4ZogsZtWEgQYVR2y4tzXeLpigm1HAJSHvf7pfe41BBxlbcpPfbdLyLo1YVH9Udzs+f5 Jkyy5hE2kUJeB3FLEnOglA71C3Z67W5YhPHafNMjSIs6bvYaooS0m2QwqcioXcEm+dQA ielA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=+zNdjlNjaGbBbTl1joNFBgmnXlFZ8pwX/30snPIOtJc=; b=kkebhQn8eKR+x2U5xhLUbgEDwzyRFiGhahysHT9m3qS7T12YP7eLWshcypI4iK1wn7 xm7Z4MAUsoUgYDDb/6Kvo/8m4Lx1WfgnuiDQgX+VVTLYLCEQ+XQ9uxRfsmQwcM1YOuqt U7MbIS7ySDCLkVZfzOMgwdzjswb+xH+B9XcJbybHh/e+X1dcYDWGlv5Ck/H3hIre7ADF l76g4AD8sbbZvd7x5qrJhUEdaXpNt1PolPP1Ef8lzjLGjkPJOTpj8TPJhQnvADmtjZFb ne5LJAT//BU6lNl9lJ/r7xsiSHITLH7nBNRH4LDfdkTcEvaW2LMKn+RHsFBG7FjU0Cae o0hQ== X-Gm-Message-State: AOAM533TFDsU4xM9bRqkTBlsnZtwUEr8kbMWmHHlVBlWJFG8Ek+gOQaO 0hDSJAb83kKfi8utE+Cvi0O0Ow== X-Google-Smtp-Source: ABdhPJxr55yXFTu3oGTg5HJKqTWWlt1XbAYA22QdUZ72kpiYjt+UFd6FsjrFKMq14umappTGj9ibRA== X-Received: by 2002:a63:da03:: with SMTP id c3mr9789954pgh.133.1607660604856; Thu, 10 Dec 2020 20:23:24 -0800 (PST) Received: from localhost.localdomain ([103.136.221.66]) by smtp.gmail.com with ESMTPSA id 19sm8623352pfu.85.2020.12.10.20.23.17 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 10 Dec 2020 20:23:24 -0800 (PST) From: Muchun Song To: gregkh@linuxfoundation.org, rafael@kernel.org, adobriyan@gmail.com, akpm@linux-foundation.org, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, hughd@google.com, shakeelb@google.com, guro@fb.com, samitolvanen@google.com, feng.tang@intel.com, neilb@suse.de, iamjoonsoo.kim@lge.com, rdunlap@infradead.org Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, Muchun Song Subject: [PATCH v4 3/7] mm: memcontrol: convert NR_FILE_THPS account to pages Date: Fri, 11 Dec 2020 12:19:50 +0800 Message-Id: <20201211041954.79543-4-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20201211041954.79543-1-songmuchun@bytedance.com> References: <20201211041954.79543-1-songmuchun@bytedance.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org The unit of NR_FILE_THPS is HPAGE_PMD_NR. Converrt NR_FILE_THPS account to pages. Signed-off-by: Muchun Song --- drivers/base/node.c | 3 +-- fs/proc/meminfo.c | 2 +- include/linux/mmzone.h | 3 ++- mm/filemap.c | 2 +- mm/huge_memory.c | 3 ++- mm/khugepaged.c | 2 +- mm/memcontrol.c | 5 ++--- 7 files changed, 10 insertions(+), 10 deletions(-) diff --git a/drivers/base/node.c b/drivers/base/node.c index 6da0c3508bc9..d5952f754911 100644 --- a/drivers/base/node.c +++ b/drivers/base/node.c @@ -466,8 +466,7 @@ static ssize_t node_read_meminfo(struct device *dev, HPAGE_PMD_NR), nid, K(node_page_state(pgdat, NR_SHMEM_PMDMAPPED) * HPAGE_PMD_NR), - nid, K(node_page_state(pgdat, NR_FILE_THPS) * - HPAGE_PMD_NR), + nid, K(node_page_state(pgdat, NR_FILE_THPS)), nid, K(node_page_state(pgdat, NR_FILE_PMDMAPPED) * HPAGE_PMD_NR) #endif diff --git a/fs/proc/meminfo.c b/fs/proc/meminfo.c index a635c8a84ddf..7ea4679880c8 100644 --- a/fs/proc/meminfo.c +++ b/fs/proc/meminfo.c @@ -135,7 +135,7 @@ static int meminfo_proc_show(struct seq_file *m, void *v) show_val_kb(m, "ShmemPmdMapped: ", global_node_page_state(NR_SHMEM_PMDMAPPED) * HPAGE_PMD_NR); show_val_kb(m, "FileHugePages: ", - global_node_page_state(NR_FILE_THPS) * HPAGE_PMD_NR); + global_node_page_state(NR_FILE_THPS)); show_val_kb(m, "FilePmdMapped: ", global_node_page_state(NR_FILE_PMDMAPPED) * HPAGE_PMD_NR); #endif diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index 4ac95448421c..67b6598c9ea4 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -215,7 +215,8 @@ enum node_stat_item { */ static __always_inline bool vmstat_item_print_in_thp(enum node_stat_item item) { - return item == NR_ANON_THPS; + return item == NR_ANON_THPS || + item == NR_FILE_THPS; } /* diff --git a/mm/filemap.c b/mm/filemap.c index 78090ee08ac2..9cc8b3ac9eac 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -207,7 +207,7 @@ static void unaccount_page_cache_page(struct address_space *mapping, if (PageTransHuge(page)) __dec_lruvec_page_state(page, NR_SHMEM_THPS); } else if (PageTransHuge(page)) { - __dec_lruvec_page_state(page, NR_FILE_THPS); + __mod_lruvec_page_state(page, NR_FILE_THPS, -HPAGE_PMD_NR); filemap_nr_thps_dec(mapping); } diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 66ec454120de..1e24165fa53a 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -2748,7 +2748,8 @@ int split_huge_page_to_list(struct page *page, struct list_head *list) if (PageSwapBacked(head)) __dec_lruvec_page_state(head, NR_SHMEM_THPS); else - __dec_lruvec_page_state(head, NR_FILE_THPS); + __mod_lruvec_page_state(head, NR_FILE_THPS, + -HPAGE_PMD_NR); } __split_huge_page(page, list, end); diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 494d3cb0b58a..76b3e064a72a 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1869,7 +1869,7 @@ static void collapse_file(struct mm_struct *mm, if (is_shmem) __inc_lruvec_page_state(new_page, NR_SHMEM_THPS); else { - __inc_lruvec_page_state(new_page, NR_FILE_THPS); + __mod_lruvec_page_state(new_page, NR_FILE_THPS, HPAGE_PMD_NR); filemap_nr_thps_inc(mapping); } diff --git a/mm/memcontrol.c b/mm/memcontrol.c index b18e25a5cdf3..04985c8c6a0a 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -1533,7 +1533,7 @@ static struct memory_stat memory_stats[] = { * constant(e.g. powerpc). */ { "anon_thp", PAGE_SIZE, NR_ANON_THPS }, - { "file_thp", 0, NR_FILE_THPS }, + { "file_thp", PAGE_SIZE, NR_FILE_THPS }, { "shmem_thp", 0, NR_SHMEM_THPS }, #endif { "inactive_anon", PAGE_SIZE, NR_INACTIVE_ANON }, @@ -1565,8 +1565,7 @@ static int __init memory_stats_init(void) for (i = 0; i < ARRAY_SIZE(memory_stats); i++) { #ifdef CONFIG_TRANSPARENT_HUGEPAGE - if (memory_stats[i].idx == NR_FILE_THPS || - memory_stats[i].idx == NR_SHMEM_THPS) + if (memory_stats[i].idx == NR_SHMEM_THPS) memory_stats[i].ratio = HPAGE_PMD_SIZE; #endif VM_BUG_ON(!memory_stats[i].ratio); From patchwork Fri Dec 11 04:19:51 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 11967193 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.9 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,UNWANTED_LANGUAGE_BODY, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 13EACC433FE for ; Fri, 11 Dec 2020 04:25:33 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D472C23EF3 for ; Fri, 11 Dec 2020 04:25:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2395370AbgLKEYv (ORCPT ); Thu, 10 Dec 2020 23:24:51 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36410 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2395376AbgLKEYd (ORCPT ); Thu, 10 Dec 2020 23:24:33 -0500 Received: from mail-pf1-x433.google.com (mail-pf1-x433.google.com [IPv6:2607:f8b0:4864:20::433]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8FFB1C061248 for ; Thu, 10 Dec 2020 20:23:32 -0800 (PST) Received: by mail-pf1-x433.google.com with SMTP id c79so6208079pfc.2 for ; Thu, 10 Dec 2020 20:23:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=NbJhG/dST+z4diHKFuM1zuN6LseIAyC0A7txgFqKs14=; b=GORbjDWZI6+50DEGT4dnLtiziTalV0Lt8P/furd1Fs3XrM/sZzpPQvk7840/8W+QS7 Uq7HXKX1RrX18znav0P/3a3kxYtMH6NkjZRU26+GCv4Kbo3qjDVfPOkyQQ3p4Wh92kbb NtCmT1EPhw04P9OLkJQzyEtvA+V3E4gAbMOEtguB9i062ms/j4E6atxS6ntEm+1/wHv8 1clU8ncAIiYEXL5poqcfd8klzCtr1AiIoTopGRXAtJSqzloZO1dWTrnbyu7cBgzcLBFu ZdYc1WBtXQo9Y0frkW9q6UlrJ1RGH6Mk9/qoxjlOm6bbTzkEnx1dvwnm+xGXrlh/nvTL 4iTA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=NbJhG/dST+z4diHKFuM1zuN6LseIAyC0A7txgFqKs14=; b=PksN4Mhaj1nQwAkaIKoeCADSTkQEbCaBZvp2b/LEKS/23b1GGbxqFjfwrM9V07wqcg 1OjKgGkeSeljerOyvbQJFXJe16fDe4bnVIUwBioaaXYFm+GIVN7YiTANalsOlC8RLzDw qM8l//sIWuGchT3cXcuGRuapXOAJft9v/DbmpWuudoRCe57Qt7GR4WZ/tyFIcusp9u5z TaufdZrwNB0KWohAXEpFW6QQJhDkhbGg8A6QlSh8I42ywox09aheqMPQxTeLgrNvGTk0 ENLIGjGZ5w89Tbkpsk5cPvlL4woti7SE1Opta7azVK88vvikWq5F5Bu/Vpo8Zt8jywUU tiwQ== X-Gm-Message-State: AOAM533+fxeiQyW8jZZFK+Fjb69M+khlWtV7rUAS8I62w9e3W73u/y/C 0fx5M+u3clrGYfv761rekdrGHA== X-Google-Smtp-Source: ABdhPJyos71clKQbmypkn6DJcpPrhNuiTt6C+WbTDEwBPzMpExl/Tq/F9HfnC6T3EcmJ8Ml75UOqew== X-Received: by 2002:a62:7e09:0:b029:19e:9fdd:80a3 with SMTP id z9-20020a627e090000b029019e9fdd80a3mr8794253pfc.24.1607660612143; Thu, 10 Dec 2020 20:23:32 -0800 (PST) Received: from localhost.localdomain ([103.136.221.66]) by smtp.gmail.com with ESMTPSA id 19sm8623352pfu.85.2020.12.10.20.23.25 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 10 Dec 2020 20:23:31 -0800 (PST) From: Muchun Song To: gregkh@linuxfoundation.org, rafael@kernel.org, adobriyan@gmail.com, akpm@linux-foundation.org, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, hughd@google.com, shakeelb@google.com, guro@fb.com, samitolvanen@google.com, feng.tang@intel.com, neilb@suse.de, iamjoonsoo.kim@lge.com, rdunlap@infradead.org Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, Muchun Song Subject: [PATCH v4 4/7] mm: memcontrol: convert NR_SHMEM_THPS account to pages Date: Fri, 11 Dec 2020 12:19:51 +0800 Message-Id: <20201211041954.79543-5-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20201211041954.79543-1-songmuchun@bytedance.com> References: <20201211041954.79543-1-songmuchun@bytedance.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org The unit of NR_SHMEM_THPS is HPAGE_PMD_NR. Convert NR_SHMEM_THPS account to pages Signed-off-by: Muchun Song --- drivers/base/node.c | 3 +-- fs/proc/meminfo.c | 2 +- include/linux/mmzone.h | 3 ++- mm/filemap.c | 2 +- mm/huge_memory.c | 3 ++- mm/khugepaged.c | 2 +- mm/memcontrol.c | 26 ++------------------------ mm/page_alloc.c | 2 +- mm/shmem.c | 3 ++- 9 files changed, 13 insertions(+), 33 deletions(-) diff --git a/drivers/base/node.c b/drivers/base/node.c index d5952f754911..6d5ac6ffb6e1 100644 --- a/drivers/base/node.c +++ b/drivers/base/node.c @@ -462,8 +462,7 @@ static ssize_t node_read_meminfo(struct device *dev, #ifdef CONFIG_TRANSPARENT_HUGEPAGE , nid, K(node_page_state(pgdat, NR_ANON_THPS)), - nid, K(node_page_state(pgdat, NR_SHMEM_THPS) * - HPAGE_PMD_NR), + nid, K(node_page_state(pgdat, NR_SHMEM_THPS)), nid, K(node_page_state(pgdat, NR_SHMEM_PMDMAPPED) * HPAGE_PMD_NR), nid, K(node_page_state(pgdat, NR_FILE_THPS)), diff --git a/fs/proc/meminfo.c b/fs/proc/meminfo.c index 7ea4679880c8..cfb107eaa3e6 100644 --- a/fs/proc/meminfo.c +++ b/fs/proc/meminfo.c @@ -131,7 +131,7 @@ static int meminfo_proc_show(struct seq_file *m, void *v) show_val_kb(m, "AnonHugePages: ", global_node_page_state(NR_ANON_THPS)); show_val_kb(m, "ShmemHugePages: ", - global_node_page_state(NR_SHMEM_THPS) * HPAGE_PMD_NR); + global_node_page_state(NR_SHMEM_THPS)); show_val_kb(m, "ShmemPmdMapped: ", global_node_page_state(NR_SHMEM_PMDMAPPED) * HPAGE_PMD_NR); show_val_kb(m, "FileHugePages: ", diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index 67b6598c9ea4..4f49af38dced 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -216,7 +216,8 @@ enum node_stat_item { static __always_inline bool vmstat_item_print_in_thp(enum node_stat_item item) { return item == NR_ANON_THPS || - item == NR_FILE_THPS; + item == NR_FILE_THPS || + item == NR_SHMEM_THPS; } /* diff --git a/mm/filemap.c b/mm/filemap.c index 9cc8b3ac9eac..c653717b92b6 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -205,7 +205,7 @@ static void unaccount_page_cache_page(struct address_space *mapping, if (PageSwapBacked(page)) { __mod_lruvec_page_state(page, NR_SHMEM, -nr); if (PageTransHuge(page)) - __dec_lruvec_page_state(page, NR_SHMEM_THPS); + __mod_lruvec_page_state(page, NR_SHMEM_THPS, -HPAGE_PMD_NR); } else if (PageTransHuge(page)) { __mod_lruvec_page_state(page, NR_FILE_THPS, -HPAGE_PMD_NR); filemap_nr_thps_dec(mapping); diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 1e24165fa53a..ac552807492e 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -2746,7 +2746,8 @@ int split_huge_page_to_list(struct page *page, struct list_head *list) spin_unlock(&ds_queue->split_queue_lock); if (mapping) { if (PageSwapBacked(head)) - __dec_lruvec_page_state(head, NR_SHMEM_THPS); + __mod_lruvec_page_state(head, NR_SHMEM_THPS, + -HPAGE_PMD_NR); else __mod_lruvec_page_state(head, NR_FILE_THPS, -HPAGE_PMD_NR); diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 76b3e064a72a..2f1c22331188 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1867,7 +1867,7 @@ static void collapse_file(struct mm_struct *mm, } if (is_shmem) - __inc_lruvec_page_state(new_page, NR_SHMEM_THPS); + __mod_lruvec_page_state(new_page, NR_SHMEM_THPS, HPAGE_PMD_NR); else { __mod_lruvec_page_state(new_page, NR_FILE_THPS, HPAGE_PMD_NR); filemap_nr_thps_inc(mapping); diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 04985c8c6a0a..a40797a27f87 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -1515,7 +1515,7 @@ struct memory_stat { unsigned int idx; }; -static struct memory_stat memory_stats[] = { +static const struct memory_stat memory_stats[] = { { "anon", PAGE_SIZE, NR_ANON_MAPPED }, { "file", PAGE_SIZE, NR_FILE_PAGES }, { "kernel_stack", 1024, NR_KERNEL_STACK_KB }, @@ -1527,14 +1527,9 @@ static struct memory_stat memory_stats[] = { { "file_dirty", PAGE_SIZE, NR_FILE_DIRTY }, { "file_writeback", PAGE_SIZE, NR_WRITEBACK }, #ifdef CONFIG_TRANSPARENT_HUGEPAGE - /* - * The ratio will be initialized in memory_stats_init(). Because - * on some architectures, the macro of HPAGE_PMD_SIZE is not - * constant(e.g. powerpc). - */ { "anon_thp", PAGE_SIZE, NR_ANON_THPS }, { "file_thp", PAGE_SIZE, NR_FILE_THPS }, - { "shmem_thp", 0, NR_SHMEM_THPS }, + { "shmem_thp", PAGE_SIZE, NR_SHMEM_THPS }, #endif { "inactive_anon", PAGE_SIZE, NR_INACTIVE_ANON }, { "active_anon", PAGE_SIZE, NR_ACTIVE_ANON }, @@ -1559,23 +1554,6 @@ static struct memory_stat memory_stats[] = { { "workingset_nodereclaim", 1, WORKINGSET_NODERECLAIM }, }; -static int __init memory_stats_init(void) -{ - int i; - - for (i = 0; i < ARRAY_SIZE(memory_stats); i++) { -#ifdef CONFIG_TRANSPARENT_HUGEPAGE - if (memory_stats[i].idx == NR_SHMEM_THPS) - memory_stats[i].ratio = HPAGE_PMD_SIZE; -#endif - VM_BUG_ON(!memory_stats[i].ratio); - VM_BUG_ON(memory_stats[i].idx >= MEMCG_NR_STAT); - } - - return 0; -} -pure_initcall(memory_stats_init); - static char *memory_stat_format(struct mem_cgroup *memcg) { struct seq_buf s; diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 1700f52b7869..720fb5a220b6 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -5577,7 +5577,7 @@ void show_free_areas(unsigned int filter, nodemask_t *nodemask) K(node_page_state(pgdat, NR_WRITEBACK)), K(node_page_state(pgdat, NR_SHMEM)), #ifdef CONFIG_TRANSPARENT_HUGEPAGE - K(node_page_state(pgdat, NR_SHMEM_THPS) * HPAGE_PMD_NR), + K(node_page_state(pgdat, NR_SHMEM_THPS)), K(node_page_state(pgdat, NR_SHMEM_PMDMAPPED) * HPAGE_PMD_NR), K(node_page_state(pgdat, NR_ANON_THPS)), diff --git a/mm/shmem.c b/mm/shmem.c index 53d84d2c9fe5..b2ca3beabc19 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -713,7 +713,8 @@ static int shmem_add_to_page_cache(struct page *page, } if (PageTransHuge(page)) { count_vm_event(THP_FILE_ALLOC); - __inc_lruvec_page_state(page, NR_SHMEM_THPS); + __mod_lruvec_page_state(page, NR_SHMEM_THPS, + HPAGE_PMD_NR); } mapping->nrpages += nr; __mod_lruvec_page_state(page, NR_FILE_PAGES, nr); From patchwork Fri Dec 11 04:19:52 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 11967195 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2BAB4C1B0E3 for ; Fri, 11 Dec 2020 04:26:05 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 02ABC23EF2 for ; Fri, 11 Dec 2020 04:26:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2405310AbgLKEY4 (ORCPT ); Thu, 10 Dec 2020 23:24:56 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36384 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2395381AbgLKEYm (ORCPT ); Thu, 10 Dec 2020 23:24:42 -0500 Received: from mail-pl1-x642.google.com (mail-pl1-x642.google.com [IPv6:2607:f8b0:4864:20::642]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C8CA0C0611CC for ; Thu, 10 Dec 2020 20:23:39 -0800 (PST) Received: by mail-pl1-x642.google.com with SMTP id t18so3979920plo.0 for ; Thu, 10 Dec 2020 20:23:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=GQUFW73mdqeVNnWmOCUXrriPRd8OoQbYZkReps7lWmE=; b=IjskIXxMWCN/ytFBKw+DiAXxwBYCNKF8JYU6iVwQmlV4cOw6vNaY1P3GrQ1tyoaM9Y LGNX7hCXCHbchH5rP4RRe4V9p2J6KNK5d+n6pNqTcWLgaQl5e6zgcFFXn26pC5V98oKH ChD3XxNRpjpFKsRgxoWhrRl3CJzYll1nkGHxrs8VD5kbyoR5RDageIyV/Pdrr5APQ0Hm fD0x2O/ifiMYM1Ztriz5/eDvN9WjdQ6SYLXPN14x7R2j3v8AGb3gRuChTUYnqUWV0/pV KX51x3M9Zsv6f1MvSNxvn/1/c4Y4CM9Ne7pKYum2xNsYxOizOxI1ztLeDsH6gWgq/tEu EYfg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=GQUFW73mdqeVNnWmOCUXrriPRd8OoQbYZkReps7lWmE=; b=plG/weX05DX4QyMUtr4QJBFBPIfro+fZ6iN1NLBuXOEGvNzyijsoo1Hmabn0YQgr4k VJon6cl1rNLy7seVgPZk5WcjyW2yzSh7ZyryBQJjhxRbl+R2kyuv6zO46KSpBfgbwvLJ hnM6dofR5b9Pd8AS3kZxPA9+scnFg6po85S/AboB4AgB+fdZtYp9lJYo0Dm0wpLa2Cp8 d2Ixiloucr/nAVst3gJEvgZYk1i1ps+h4zFW1Z618x0vz9/1V0TwO2YATIXpCgrChubI 5BkW2W3+0uo/4eNKCcmGfdaSQZINlvvlDMh3t4suo+W4jpY5Nk+o66bZLX8Oqlzr8OrG 72bw== X-Gm-Message-State: AOAM533LBv1XO32m6KiqctxjO6CR0hOyvOfLxGJDe++sYuCjsnfxd1XH sBvhX3DZC9ZnGoJHfkXRLJxiDw== X-Google-Smtp-Source: ABdhPJzeiQSfJfIhgFUu9W2wTkNi1XOhkEXOXCWkca4m+6vxOOnVMFAG744rfQkyKjRaEl4HhunhnQ== X-Received: by 2002:a17:902:b213:b029:db:3a3e:d8ad with SMTP id t19-20020a170902b213b02900db3a3ed8admr9494640plr.73.1607660619419; Thu, 10 Dec 2020 20:23:39 -0800 (PST) Received: from localhost.localdomain ([103.136.221.66]) by smtp.gmail.com with ESMTPSA id 19sm8623352pfu.85.2020.12.10.20.23.32 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 10 Dec 2020 20:23:38 -0800 (PST) From: Muchun Song To: gregkh@linuxfoundation.org, rafael@kernel.org, adobriyan@gmail.com, akpm@linux-foundation.org, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, hughd@google.com, shakeelb@google.com, guro@fb.com, samitolvanen@google.com, feng.tang@intel.com, neilb@suse.de, iamjoonsoo.kim@lge.com, rdunlap@infradead.org Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, Muchun Song Subject: [PATCH v4 5/7] mm: memcontrol: convert NR_SHMEM_PMDMAPPED account to pages Date: Fri, 11 Dec 2020 12:19:52 +0800 Message-Id: <20201211041954.79543-6-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20201211041954.79543-1-songmuchun@bytedance.com> References: <20201211041954.79543-1-songmuchun@bytedance.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org The unit of NR_SHMEM_PMDMAPPED is HPAGE_PMD_NR. Convert NR_SHMEM_PMDMAPPED account to pages. Signed-off-by: Muchun Song --- drivers/base/node.c | 3 +-- fs/proc/meminfo.c | 2 +- include/linux/mmzone.h | 3 ++- mm/page_alloc.c | 3 +-- mm/rmap.c | 6 ++++-- 5 files changed, 9 insertions(+), 8 deletions(-) diff --git a/drivers/base/node.c b/drivers/base/node.c index 6d5ac6ffb6e1..7a66aefe4e46 100644 --- a/drivers/base/node.c +++ b/drivers/base/node.c @@ -463,8 +463,7 @@ static ssize_t node_read_meminfo(struct device *dev, , nid, K(node_page_state(pgdat, NR_ANON_THPS)), nid, K(node_page_state(pgdat, NR_SHMEM_THPS)), - nid, K(node_page_state(pgdat, NR_SHMEM_PMDMAPPED) * - HPAGE_PMD_NR), + nid, K(node_page_state(pgdat, NR_SHMEM_PMDMAPPED)), nid, K(node_page_state(pgdat, NR_FILE_THPS)), nid, K(node_page_state(pgdat, NR_FILE_PMDMAPPED) * HPAGE_PMD_NR) diff --git a/fs/proc/meminfo.c b/fs/proc/meminfo.c index cfb107eaa3e6..c61f440570f9 100644 --- a/fs/proc/meminfo.c +++ b/fs/proc/meminfo.c @@ -133,7 +133,7 @@ static int meminfo_proc_show(struct seq_file *m, void *v) show_val_kb(m, "ShmemHugePages: ", global_node_page_state(NR_SHMEM_THPS)); show_val_kb(m, "ShmemPmdMapped: ", - global_node_page_state(NR_SHMEM_PMDMAPPED) * HPAGE_PMD_NR); + global_node_page_state(NR_SHMEM_PMDMAPPED)); show_val_kb(m, "FileHugePages: ", global_node_page_state(NR_FILE_THPS)); show_val_kb(m, "FilePmdMapped: ", diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index 4f49af38dced..d6b13b9327be 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -217,7 +217,8 @@ static __always_inline bool vmstat_item_print_in_thp(enum node_stat_item item) { return item == NR_ANON_THPS || item == NR_FILE_THPS || - item == NR_SHMEM_THPS; + item == NR_SHMEM_THPS || + item == NR_SHMEM_PMDMAPPED; } /* diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 720fb5a220b6..575fbfeea4b5 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -5578,8 +5578,7 @@ void show_free_areas(unsigned int filter, nodemask_t *nodemask) K(node_page_state(pgdat, NR_SHMEM)), #ifdef CONFIG_TRANSPARENT_HUGEPAGE K(node_page_state(pgdat, NR_SHMEM_THPS)), - K(node_page_state(pgdat, NR_SHMEM_PMDMAPPED) - * HPAGE_PMD_NR), + K(node_page_state(pgdat, NR_SHMEM_PMDMAPPED)), K(node_page_state(pgdat, NR_ANON_THPS)), #endif K(node_page_state(pgdat, NR_WRITEBACK_TEMP)), diff --git a/mm/rmap.c b/mm/rmap.c index f59e92e26b61..3089ad6bf468 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -1219,7 +1219,8 @@ void page_add_file_rmap(struct page *page, bool compound) if (!atomic_inc_and_test(compound_mapcount_ptr(page))) goto out; if (PageSwapBacked(page)) - __inc_node_page_state(page, NR_SHMEM_PMDMAPPED); + __mod_lruvec_page_state(page, NR_SHMEM_PMDMAPPED, + HPAGE_PMD_NR); else __inc_node_page_state(page, NR_FILE_PMDMAPPED); } else { @@ -1260,7 +1261,8 @@ static void page_remove_file_rmap(struct page *page, bool compound) if (!atomic_add_negative(-1, compound_mapcount_ptr(page))) return; if (PageSwapBacked(page)) - __dec_node_page_state(page, NR_SHMEM_PMDMAPPED); + __mod_lruvec_page_state(page, NR_SHMEM_PMDMAPPED, + -HPAGE_PMD_NR); else __dec_node_page_state(page, NR_FILE_PMDMAPPED); } else { From patchwork Fri Dec 11 04:19:53 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 11967201 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 029A6C4167B for ; Fri, 11 Dec 2020 04:26:05 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D7D1C23EF1 for ; Fri, 11 Dec 2020 04:26:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2404951AbgLKEYw (ORCPT ); Thu, 10 Dec 2020 23:24:52 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36390 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2395384AbgLKEYm (ORCPT ); Thu, 10 Dec 2020 23:24:42 -0500 Received: from mail-pf1-x443.google.com (mail-pf1-x443.google.com [IPv6:2607:f8b0:4864:20::443]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 409C7C0613CF for ; Thu, 10 Dec 2020 20:23:47 -0800 (PST) Received: by mail-pf1-x443.google.com with SMTP id d2so6211919pfq.5 for ; Thu, 10 Dec 2020 20:23:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=c1LVwFTVmqjgTFBygiYf9KRS/M2MDULFm8W3CcAScWk=; b=sHMPxQiX9HU37cGIlQKYuTVGVR1QPCEbEEnvCmq1qHG5L+syE+EtC++5V++Y8LmEN8 gOrCGT9Ye2l+SD2JnOdPsE0b/2uvmNdTkhYjo26CkdGNqNr9/K0qF49nz7ftVV51cipT UsR8aYpkXMvucEHBXXhdrQMlTS8B+g4KXiPyiBUOu5DFjtoy7vnp++5+5xRaG8+FnYe1 8RRDwH6oPVoJbEst3xbvp3AtHixH6JqaIKrZvrkEduKy0YScajgC30DFxvNqqk/Yzz60 IMWwPzI7dR4ebv/zrcaAyU5VMrVt38yjZW51GIFUjhN7kTvBEiDufL0YmDJs4UOyTn3s nVrQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=c1LVwFTVmqjgTFBygiYf9KRS/M2MDULFm8W3CcAScWk=; b=cD6jYKyw0Zzt0nZcamq4/vNyR6J63O4iyRBH/w5yoY1VtZWplh33feFISIoeNriHEs pSK7YP5N1TE6PxBGbFoIM3tNeoVkFRGohbeGfJfrRB3vZSf3EQx2n3jVtL1NmGyyuQZQ Cb2jYZuYwrMYHoq8FY7m6jO/9R1wIw/O3hOOXy71nPEVk9A6qKROGXlaVE4o/8vVnosm YThSVhejodRTVUIcHnpB/5Qrl0cEF5HmrjKJ2eUlM05P7vF1dIBOkrGMIFMGdZBkhocc NKJn3NcgejjkRhq2IfJpUgLUchB8tBuh/3T2EpuN0NRLehVzJhYiO5kNNNxhw25FAchp rCcA== X-Gm-Message-State: AOAM532PN0/Wz2chlHF2U4mOk2CdWIU9C+qXETPPo2+XS8JdS5JiVZjp xk3L+ANrpjVnk6+tKxINiyj6Lg== X-Google-Smtp-Source: ABdhPJzvI52xrdQRDVgHYSZXJEjSxmWy+Pk9gYCoF+D8vHIorOD99kxCXFDzu/WJ7Akt1GmsLOn10Q== X-Received: by 2002:a63:4950:: with SMTP id y16mr9800008pgk.415.1607660626886; Thu, 10 Dec 2020 20:23:46 -0800 (PST) Received: from localhost.localdomain ([103.136.221.66]) by smtp.gmail.com with ESMTPSA id 19sm8623352pfu.85.2020.12.10.20.23.39 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 10 Dec 2020 20:23:46 -0800 (PST) From: Muchun Song To: gregkh@linuxfoundation.org, rafael@kernel.org, adobriyan@gmail.com, akpm@linux-foundation.org, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, hughd@google.com, shakeelb@google.com, guro@fb.com, samitolvanen@google.com, feng.tang@intel.com, neilb@suse.de, iamjoonsoo.kim@lge.com, rdunlap@infradead.org Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, Muchun Song Subject: [PATCH v4 6/7] mm: memcontrol: convert NR_FILE_PMDMAPPED account to pages Date: Fri, 11 Dec 2020 12:19:53 +0800 Message-Id: <20201211041954.79543-7-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20201211041954.79543-1-songmuchun@bytedance.com> References: <20201211041954.79543-1-songmuchun@bytedance.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org The unit of NR_FILE_PMDMAPPED is HPAGE_PMD_NR. Convert NR_FILE_PMDMAPPED account to pages. Signed-off-by: Muchun Song --- drivers/base/node.c | 3 +-- fs/proc/meminfo.c | 2 +- include/linux/mmzone.h | 3 ++- mm/rmap.c | 6 ++++-- 4 files changed, 8 insertions(+), 6 deletions(-) diff --git a/drivers/base/node.c b/drivers/base/node.c index 7a66aefe4e46..d02d86aec19f 100644 --- a/drivers/base/node.c +++ b/drivers/base/node.c @@ -465,8 +465,7 @@ static ssize_t node_read_meminfo(struct device *dev, nid, K(node_page_state(pgdat, NR_SHMEM_THPS)), nid, K(node_page_state(pgdat, NR_SHMEM_PMDMAPPED)), nid, K(node_page_state(pgdat, NR_FILE_THPS)), - nid, K(node_page_state(pgdat, NR_FILE_PMDMAPPED) * - HPAGE_PMD_NR) + nid, K(node_page_state(pgdat, NR_FILE_PMDMAPPED)) #endif ); len += hugetlb_report_node_meminfo(buf, len, nid); diff --git a/fs/proc/meminfo.c b/fs/proc/meminfo.c index c61f440570f9..6fa761c9cc78 100644 --- a/fs/proc/meminfo.c +++ b/fs/proc/meminfo.c @@ -137,7 +137,7 @@ static int meminfo_proc_show(struct seq_file *m, void *v) show_val_kb(m, "FileHugePages: ", global_node_page_state(NR_FILE_THPS)); show_val_kb(m, "FilePmdMapped: ", - global_node_page_state(NR_FILE_PMDMAPPED) * HPAGE_PMD_NR); + global_node_page_state(NR_FILE_PMDMAPPED)); #endif #ifdef CONFIG_CMA diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index d6b13b9327be..c587343a3884 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -218,7 +218,8 @@ static __always_inline bool vmstat_item_print_in_thp(enum node_stat_item item) return item == NR_ANON_THPS || item == NR_FILE_THPS || item == NR_SHMEM_THPS || - item == NR_SHMEM_PMDMAPPED; + item == NR_SHMEM_PMDMAPPED || + item == NR_FILE_PMDMAPPED; } /* diff --git a/mm/rmap.c b/mm/rmap.c index 3089ad6bf468..e383c5619501 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -1222,7 +1222,8 @@ void page_add_file_rmap(struct page *page, bool compound) __mod_lruvec_page_state(page, NR_SHMEM_PMDMAPPED, HPAGE_PMD_NR); else - __inc_node_page_state(page, NR_FILE_PMDMAPPED); + __mod_lruvec_page_state(page, NR_FILE_PMDMAPPED, + HPAGE_PMD_NR); } else { if (PageTransCompound(page) && page_mapping(page)) { VM_WARN_ON_ONCE(!PageLocked(page)); @@ -1264,7 +1265,8 @@ static void page_remove_file_rmap(struct page *page, bool compound) __mod_lruvec_page_state(page, NR_SHMEM_PMDMAPPED, -HPAGE_PMD_NR); else - __dec_node_page_state(page, NR_FILE_PMDMAPPED); + __mod_lruvec_page_state(page, NR_FILE_PMDMAPPED, + -HPAGE_PMD_NR); } else { if (!atomic_add_negative(-1, &page->_mapcount)) return; From patchwork Fri Dec 11 04:19:54 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 11967241 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 346F0C2BB48 for ; Fri, 11 Dec 2020 04:26:54 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1454523EF2 for ; Fri, 11 Dec 2020 04:26:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2405319AbgLKEZv (ORCPT ); Thu, 10 Dec 2020 23:25:51 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36502 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2405315AbgLKEZH (ORCPT ); Thu, 10 Dec 2020 23:25:07 -0500 Received: from mail-pj1-x1044.google.com (mail-pj1-x1044.google.com [IPv6:2607:f8b0:4864:20::1044]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D829FC061794 for ; Thu, 10 Dec 2020 20:23:54 -0800 (PST) Received: by mail-pj1-x1044.google.com with SMTP id b5so1682067pjl.0 for ; Thu, 10 Dec 2020 20:23:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=V6cgE4Bypope/50+jczAJckIDmu+NX283pqYCpMuFa4=; b=WdKBmuSwKfyBRkQlidbqMK4rXhMtOhxX6s7b71K9mAl848EvGutMDRJlI/5zZezDjF SQ1dfYswVirngLEoUH90Ybd55tva4pkIwo6yYYmH5Z16e1VzvqcgRKSNVe3edbgwJXsN /IBh/OGj6mWWWy5qPdsoGqzcO1jHkTITPvq9bVe5Q8ez7e66cSEfqe0n8VpaV3E8mfl2 pAtJL7Y+vDGsXom08S3+05XPIIonieNQ4K6GadmIWTHVRt7gr/gbFKFlqREEfXZ9mEOF 87egGO6IsZrqoAU+a/QIVF29sbkNmWTKCFFpMM1vYuusJC0tddSKqd/AwOXagjz2AEXQ 9GRg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=V6cgE4Bypope/50+jczAJckIDmu+NX283pqYCpMuFa4=; b=Dc27cVd2l7X1/od7w5VeAcwMBzuCg27pBcL8IGq/TFTw+k6S6CbZOfIvPhOpGC99kN dgd3O60IwZy3XAtnmsI7s+c60erR2B1tYoqOdDqhowfVa4a8pcHHhkTri8fqKVrNp9I0 og/FCu6pe7bWgCvvF7ZvSRmDIWKc+0jqg9BmWYjblIf5otLNyCoZgGqbKpZzaKJ32WsY tB01O781nsLBG6o+vW+9M8pDjauLxNKe8IuRWHjq5C+4Lav/CCXAm/WSXcZcN/sp+zvV WjZW43FPYMGXRJyybNh8df9JOxTNDyQmcvuycCS4221NO1qsq3ztAkfQcOjuAuUDKyDn HCPQ== X-Gm-Message-State: AOAM5335UN+0BVRDItVABsVhKG43rk4Ru7qh43IPYlI6Nm454B3B1kib PuhZC+l37A6MBOxedrexYOBwmA== X-Google-Smtp-Source: ABdhPJwdoIX1p/0NJ1YKn2SsyeylTuzKrv0Zp2zZ/VDEYphVudb4Bo3w4+chjSDK2ZbuOTvuVpPmuw== X-Received: by 2002:a17:902:778e:b029:da:feef:8f2d with SMTP id o14-20020a170902778eb02900dafeef8f2dmr2170797pll.25.1607660634412; Thu, 10 Dec 2020 20:23:54 -0800 (PST) Received: from localhost.localdomain ([103.136.221.66]) by smtp.gmail.com with ESMTPSA id 19sm8623352pfu.85.2020.12.10.20.23.47 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 10 Dec 2020 20:23:53 -0800 (PST) From: Muchun Song To: gregkh@linuxfoundation.org, rafael@kernel.org, adobriyan@gmail.com, akpm@linux-foundation.org, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, hughd@google.com, shakeelb@google.com, guro@fb.com, samitolvanen@google.com, feng.tang@intel.com, neilb@suse.de, iamjoonsoo.kim@lge.com, rdunlap@infradead.org Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, Muchun Song Subject: [PATCH v4 7/7] mm: memcontrol: make the slab calculation consistent Date: Fri, 11 Dec 2020 12:19:54 +0800 Message-Id: <20201211041954.79543-8-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20201211041954.79543-1-songmuchun@bytedance.com> References: <20201211041954.79543-1-songmuchun@bytedance.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Although the ratio of the slab is one, we also should read the ratio from the related memory_stats instead of hard-coding. And the local variable of size is already the value of slab_unreclaimable. So we do not need to read again. We can drop the ratio in struct memory_stat. This can make the code clean and simple. And get rid of the awkward mix of static and runtime initialization of the memory_stats table. Signed-off-by: Muchun Song --- mm/memcontrol.c | 105 +++++++++++++++++++++++++++++++++++--------------------- 1 file changed, 66 insertions(+), 39 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index a40797a27f87..eec44918d373 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -1511,49 +1511,71 @@ static bool mem_cgroup_wait_acct_move(struct mem_cgroup *memcg) struct memory_stat { const char *name; - unsigned int ratio; unsigned int idx; }; static const struct memory_stat memory_stats[] = { - { "anon", PAGE_SIZE, NR_ANON_MAPPED }, - { "file", PAGE_SIZE, NR_FILE_PAGES }, - { "kernel_stack", 1024, NR_KERNEL_STACK_KB }, - { "pagetables", PAGE_SIZE, NR_PAGETABLE }, - { "percpu", 1, MEMCG_PERCPU_B }, - { "sock", PAGE_SIZE, MEMCG_SOCK }, - { "shmem", PAGE_SIZE, NR_SHMEM }, - { "file_mapped", PAGE_SIZE, NR_FILE_MAPPED }, - { "file_dirty", PAGE_SIZE, NR_FILE_DIRTY }, - { "file_writeback", PAGE_SIZE, NR_WRITEBACK }, + { "anon", NR_ANON_MAPPED }, + { "file", NR_FILE_PAGES }, + { "kernel_stack", NR_KERNEL_STACK_KB }, + { "pagetables", NR_PAGETABLE }, + { "percpu", MEMCG_PERCPU_B }, + { "sock", MEMCG_SOCK }, + { "shmem", NR_SHMEM }, + { "file_mapped", NR_FILE_MAPPED }, + { "file_dirty", NR_FILE_DIRTY }, + { "file_writeback", NR_WRITEBACK }, #ifdef CONFIG_TRANSPARENT_HUGEPAGE - { "anon_thp", PAGE_SIZE, NR_ANON_THPS }, - { "file_thp", PAGE_SIZE, NR_FILE_THPS }, - { "shmem_thp", PAGE_SIZE, NR_SHMEM_THPS }, + { "anon_thp", NR_ANON_THPS }, + { "file_thp", NR_FILE_THPS }, + { "shmem_thp", NR_SHMEM_THPS }, #endif - { "inactive_anon", PAGE_SIZE, NR_INACTIVE_ANON }, - { "active_anon", PAGE_SIZE, NR_ACTIVE_ANON }, - { "inactive_file", PAGE_SIZE, NR_INACTIVE_FILE }, - { "active_file", PAGE_SIZE, NR_ACTIVE_FILE }, - { "unevictable", PAGE_SIZE, NR_UNEVICTABLE }, - - /* - * Note: The slab_reclaimable and slab_unreclaimable must be - * together and slab_reclaimable must be in front. - */ - { "slab_reclaimable", 1, NR_SLAB_RECLAIMABLE_B }, - { "slab_unreclaimable", 1, NR_SLAB_UNRECLAIMABLE_B }, + { "inactive_anon", NR_INACTIVE_ANON }, + { "active_anon", NR_ACTIVE_ANON }, + { "inactive_file", NR_INACTIVE_FILE }, + { "active_file", NR_ACTIVE_FILE }, + { "unevictable", NR_UNEVICTABLE }, + { "slab_reclaimable", NR_SLAB_RECLAIMABLE_B }, + { "slab_unreclaimable", NR_SLAB_UNRECLAIMABLE_B }, /* The memory events */ - { "workingset_refault_anon", 1, WORKINGSET_REFAULT_ANON }, - { "workingset_refault_file", 1, WORKINGSET_REFAULT_FILE }, - { "workingset_activate_anon", 1, WORKINGSET_ACTIVATE_ANON }, - { "workingset_activate_file", 1, WORKINGSET_ACTIVATE_FILE }, - { "workingset_restore_anon", 1, WORKINGSET_RESTORE_ANON }, - { "workingset_restore_file", 1, WORKINGSET_RESTORE_FILE }, - { "workingset_nodereclaim", 1, WORKINGSET_NODERECLAIM }, + { "workingset_refault_anon", WORKINGSET_REFAULT_ANON }, + { "workingset_refault_file", WORKINGSET_REFAULT_FILE }, + { "workingset_activate_anon", WORKINGSET_ACTIVATE_ANON }, + { "workingset_activate_file", WORKINGSET_ACTIVATE_FILE }, + { "workingset_restore_anon", WORKINGSET_RESTORE_ANON }, + { "workingset_restore_file", WORKINGSET_RESTORE_FILE }, + { "workingset_nodereclaim", WORKINGSET_NODERECLAIM }, }; +/* Translate stat items to the correct unit for memory.stat output */ +static int memcg_page_state_unit(int item) +{ + switch (item) { + case MEMCG_PERCPU_B: + case NR_SLAB_RECLAIMABLE_B: + case NR_SLAB_UNRECLAIMABLE_B: + case WORKINGSET_REFAULT_ANON: + case WORKINGSET_REFAULT_FILE: + case WORKINGSET_ACTIVATE_ANON: + case WORKINGSET_ACTIVATE_FILE: + case WORKINGSET_RESTORE_ANON: + case WORKINGSET_RESTORE_FILE: + case WORKINGSET_NODERECLAIM: + return 1; + case NR_KERNEL_STACK_KB: + return SZ_1K; + default: + return PAGE_SIZE; + } +} + +static inline unsigned long memcg_page_state_output(struct mem_cgroup *memcg, + int item) +{ + return memcg_page_state(memcg, item) * memcg_page_state_unit(item); +} + static char *memory_stat_format(struct mem_cgroup *memcg) { struct seq_buf s; @@ -1577,13 +1599,12 @@ static char *memory_stat_format(struct mem_cgroup *memcg) for (i = 0; i < ARRAY_SIZE(memory_stats); i++) { u64 size; - size = memcg_page_state(memcg, memory_stats[i].idx); - size *= memory_stats[i].ratio; + size = memcg_page_state_output(memcg, memory_stats[i].idx); seq_buf_printf(&s, "%s %llu\n", memory_stats[i].name, size); if (unlikely(memory_stats[i].idx == NR_SLAB_UNRECLAIMABLE_B)) { - size = memcg_page_state(memcg, NR_SLAB_RECLAIMABLE_B) + - memcg_page_state(memcg, NR_SLAB_UNRECLAIMABLE_B); + size += memcg_page_state_output(memcg, + NR_SLAB_RECLAIMABLE_B); seq_buf_printf(&s, "slab %llu\n", size); } } @@ -6377,6 +6398,12 @@ static int memory_stat_show(struct seq_file *m, void *v) } #ifdef CONFIG_NUMA +static inline unsigned long lruvec_page_state_output(struct lruvec *lruvec, + int item) +{ + return lruvec_page_state(lruvec, item) * memcg_page_state_unit(item); +} + static int memory_numa_stat_show(struct seq_file *m, void *v) { int i; @@ -6394,8 +6421,8 @@ static int memory_numa_stat_show(struct seq_file *m, void *v) struct lruvec *lruvec; lruvec = mem_cgroup_lruvec(memcg, NODE_DATA(nid)); - size = lruvec_page_state(lruvec, memory_stats[i].idx); - size *= memory_stats[i].ratio; + size = lruvec_page_state_output(lruvec, + memory_stats[i].idx); seq_printf(m, " N%d=%llu", nid, size); } seq_putc(m, '\n');