From patchwork Tue Dec 8 04:18:41 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 11957517 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1C70EC4361B for ; Tue, 8 Dec 2020 04:21:02 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id ADB6723A5E for ; Tue, 8 Dec 2020 04:21:01 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org ADB6723A5E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 2DFEF8D001B; Mon, 7 Dec 2020 23:21:01 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 291578D0002; Mon, 7 Dec 2020 23:21:01 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 17C488D001B; Mon, 7 Dec 2020 23:21:01 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0007.hostedemail.com [216.40.44.7]) by kanga.kvack.org (Postfix) with ESMTP id EFE4B8D0002 for ; Mon, 7 Dec 2020 23:21:00 -0500 (EST) Received: from smtpin21.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id B3A738249980 for ; Tue, 8 Dec 2020 04:21:00 +0000 (UTC) X-FDA: 77568814680.21.deer91_091167f273e4 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin21.hostedemail.com (Postfix) with ESMTP id 95449180442C3 for ; Tue, 8 Dec 2020 04:21:00 +0000 (UTC) X-HE-Tag: deer91_091167f273e4 X-Filterd-Recvd-Size: 4391 Received: from mail-pg1-f196.google.com (mail-pg1-f196.google.com [209.85.215.196]) by imf20.hostedemail.com (Postfix) with ESMTP for ; Tue, 8 Dec 2020 04:20:59 +0000 (UTC) Received: by mail-pg1-f196.google.com with SMTP id w16so11017380pga.9 for ; Mon, 07 Dec 2020 20:20:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=WYYxHKy2hNUOwy6YEzcM7TOwb+Qq885tP2jcyrxWyck=; b=gEpUnsZWYOjoACAUsQZDmde/SEQQIOvKt+ueoQYzxcPNg33SDVHk9FF1gWVVaN6rs4 /aPL5PPxMnaE+DPh8DMDUoxFDpMJXOYSZs/IupuC1Q/TVUZZ1023ynmZ+enC3EPTOC3n hj8AGMbUX4SDYfWwFpT6VNLLopbu/Zp7aV83yNWA7hJAPq/+5Jqk9mYqHCVyp0bmK/o6 ZfaRvPqDmHVJAhILi2YKpfJizlkIQL3Y04ky1LvHktsPKsw4k92k6hh6Bxam5CkIg4c+ LHpPGdsTcqgRmcHOKKSym3csLwmcz5qUUiSxWAB9f8fTbZmkoNgnQshKdURnH/7YkbGs mUBA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=WYYxHKy2hNUOwy6YEzcM7TOwb+Qq885tP2jcyrxWyck=; b=LOk0G/15tocxnGRcRk9H4XLYr6iT4LTRcSjBNwdsOMZ0ozMWQ6sAvuxa7CAxIe+jN+ 5uU4WnCLiTOhxlHnhtDsiKem5O744fAOlgCtwoQ6/hv3Mg/GOrgTfEGbrGL328PaufC1 pf2KMJtO5xN8w6YWfvNx0qRVaua6vnYgsIvaybTH/MsbJiIy9/TrqF33zS9ZFthc3Chg iv5igtlT/mjF6K4ptm6LNIyYda7LW+RUortr0L9yp6xtdKkzN8HlsdRv4fddghJd5brw oy3vs1UJdDpBLJBVRy6dROGGJe0pa/hia5tZAlZoSq6gq6whjXoETHJD+N/AAJ8ve6Rp U0kw== X-Gm-Message-State: AOAM533/5qqCOe3KQM9evr+YtXtA4TUhdvqjtCrX6d2oojxwGEW773sK FIAPyoRw3HL/CLz9zkBPxqCoxA== X-Google-Smtp-Source: ABdhPJx4XV3psoCYFQP0G4bYZi/2ONeVaNl9iWtUxxUXrX/TPLVKdBUrKH5so2rbiHM24rrZ0hcmDw== X-Received: by 2002:a17:90b:1213:: with SMTP id gl19mr2312127pjb.232.1607401259049; Mon, 07 Dec 2020 20:20:59 -0800 (PST) Received: from localhost.localdomain ([103.136.221.70]) by smtp.gmail.com with ESMTPSA id mr7sm1031166pjb.31.2020.12.07.20.20.52 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 07 Dec 2020 20:20:58 -0800 (PST) From: Muchun Song To: gregkh@linuxfoundation.org, rafael@kernel.org, adobriyan@gmail.com, akpm@linux-foundation.org, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, hughd@google.com, shakeelb@google.com, guro@fb.com, samitolvanen@google.com, feng.tang@intel.com, neilb@suse.de, iamjoonsoo.kim@lge.com, rdunlap@infradead.org Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, Muchun Song , Michal Hocko Subject: [PATCH v3 1/7] mm: memcontrol: fix NR_ANON_THPS account Date: Tue, 8 Dec 2020 12:18:41 +0800 Message-Id: <20201208041847.72122-2-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20201208041847.72122-1-songmuchun@bytedance.com> References: <20201208041847.72122-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The unit of NR_ANON_THPS is HPAGE_PMD_NR already. So it should inc/dec by one rather than nr_pages. Fixes: 468c398233da ("mm: memcontrol: switch to native NR_ANON_THPS counter") Signed-off-by: Muchun Song Acked-by: Michal Hocko Reviewed-by: Roman Gushchin --- mm/memcontrol.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index b80328f52fb4..8818bf64d6fe 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -5653,10 +5653,8 @@ static int mem_cgroup_move_account(struct page *page, __mod_lruvec_state(from_vec, NR_ANON_MAPPED, -nr_pages); __mod_lruvec_state(to_vec, NR_ANON_MAPPED, nr_pages); if (PageTransHuge(page)) { - __mod_lruvec_state(from_vec, NR_ANON_THPS, - -nr_pages); - __mod_lruvec_state(to_vec, NR_ANON_THPS, - nr_pages); + __dec_lruvec_state(from_vec, NR_ANON_THPS); + __inc_lruvec_state(to_vec, NR_ANON_THPS); } } From patchwork Tue Dec 8 04:18:42 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 11957519 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7BA85C4361B for ; Tue, 8 Dec 2020 04:21:09 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id EA21A23A5E for ; Tue, 8 Dec 2020 04:21:08 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EA21A23A5E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 7721E8D001C; Mon, 7 Dec 2020 23:21:08 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 6FA228D0002; Mon, 7 Dec 2020 23:21:08 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5C2208D001C; Mon, 7 Dec 2020 23:21:08 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0191.hostedemail.com [216.40.44.191]) by kanga.kvack.org (Postfix) with ESMTP id 4293C8D0002 for ; Mon, 7 Dec 2020 23:21:08 -0500 (EST) Received: from smtpin14.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 07A521EE6 for ; Tue, 8 Dec 2020 04:21:08 +0000 (UTC) X-FDA: 77568815016.14.grain55_5a0d0bc273e4 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin14.hostedemail.com (Postfix) with ESMTP id DC1EA18229837 for ; Tue, 8 Dec 2020 04:21:07 +0000 (UTC) X-HE-Tag: grain55_5a0d0bc273e4 X-Filterd-Recvd-Size: 9471 Received: from mail-pg1-f196.google.com (mail-pg1-f196.google.com [209.85.215.196]) by imf24.hostedemail.com (Postfix) with ESMTP for ; Tue, 8 Dec 2020 04:21:07 +0000 (UTC) Received: by mail-pg1-f196.google.com with SMTP id o5so11030299pgm.10 for ; Mon, 07 Dec 2020 20:21:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=GajMgrZPDow1lHdb5TrYOWTS9STm/NsKo1AyMyDYucQ=; b=lIXXeWqDB6bu+1GPGuzOpdPa88aHybT4YvKQJAWYRSzsTdjyXY0E51cyBA7tZ6MjKj mD69JH/4EA51Ber7HMdhbdf05XB6s6yx+QndYPTK9PyPHX/U1MIajRgW3tki/ybTwcCA WFREHSLnyDKENtkdmxTFG35wv1OkGgu0QRWUSsRakdEvc9euMLm+Ocj+iOxvqyFFtnos +5eRne4oKr/vOrBCL2FBDw3W5sFBvPz/Q1HFeeKCHVY6u4Yllry1SrmSvJ6uYktjhhS/ kjp7PPf1xR55XV1txMzncEavNuLB7WvN+xkyeuLQgaLxpDFQ/4b2RXiuAbRl+4yMHxGj V8lw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=GajMgrZPDow1lHdb5TrYOWTS9STm/NsKo1AyMyDYucQ=; b=I54GS+Qs+4a5zFY3RJA1QlyKa1U5eD5x1/5amrGfoNGnOQ2/dT3MV2AYUA09dhHnmF VCl97ukbiSvgirKRkUXWNuo7NT2EtBdghBxeZ8WBuz3sdmPgJdm2XyF126h8+0EsOl44 oPexcVOOooCMse/Luz+XogI3mECcZ9BpC9JCVE9Oz+iCevJHdMeVlhrteqkU9RWzA119 /Nawk8OL215y8DYgCvs6X3Zapqf7raReRKLLd1vOuU+8ayjkhwD3aUUUjkVSGpcYa6fZ PLcnpKJritQo2zHXWDfEQyOe1T+//N7/qF8H/M/FVJdSbEu5j3d58z7eLdfKMhUSwLL8 o49Q== X-Gm-Message-State: AOAM533tbqUHE9k6q2MZhoUl5DEaj/AVL4LUv+WZ8TObW90B2xpfBDJ4 BV8UjssZTprnPUyn93mhD7ftkA== X-Google-Smtp-Source: ABdhPJxyFNqYgZeVS/V/rgrPeQgYYUbYxCdKjyXsjG2QuyRcD52TGWI+Vg+2q1O/txQtjn5Gv8dtjw== X-Received: by 2002:a17:90b:384c:: with SMTP id nl12mr2290341pjb.72.1607401266341; Mon, 07 Dec 2020 20:21:06 -0800 (PST) Received: from localhost.localdomain ([103.136.221.70]) by smtp.gmail.com with ESMTPSA id mr7sm1031166pjb.31.2020.12.07.20.20.59 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 07 Dec 2020 20:21:05 -0800 (PST) From: Muchun Song To: gregkh@linuxfoundation.org, rafael@kernel.org, adobriyan@gmail.com, akpm@linux-foundation.org, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, hughd@google.com, shakeelb@google.com, guro@fb.com, samitolvanen@google.com, feng.tang@intel.com, neilb@suse.de, iamjoonsoo.kim@lge.com, rdunlap@infradead.org Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, Muchun Song Subject: [PATCH v3 2/7] mm: memcontrol: convert NR_ANON_THPS account to pages Date: Tue, 8 Dec 2020 12:18:42 +0800 Message-Id: <20201208041847.72122-3-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20201208041847.72122-1-songmuchun@bytedance.com> References: <20201208041847.72122-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The unit of NR_ANON_THPS is HPAGE_PMD_NR. Convert the NR_ANON_THPS account to pages. Signed-off-by: Muchun Song --- drivers/base/node.c | 3 +-- fs/proc/meminfo.c | 2 +- mm/huge_memory.c | 3 ++- mm/memcontrol.c | 20 ++++++-------------- mm/page_alloc.c | 2 +- mm/rmap.c | 7 ++++--- 6 files changed, 15 insertions(+), 22 deletions(-) diff --git a/drivers/base/node.c b/drivers/base/node.c index 04f71c7bc3f8..ec35cb567940 100644 --- a/drivers/base/node.c +++ b/drivers/base/node.c @@ -461,8 +461,7 @@ static ssize_t node_read_meminfo(struct device *dev, nid, K(sunreclaimable) #ifdef CONFIG_TRANSPARENT_HUGEPAGE , - nid, K(node_page_state(pgdat, NR_ANON_THPS) * - HPAGE_PMD_NR), + nid, K(node_page_state(pgdat, NR_ANON_THPS)), nid, K(node_page_state(pgdat, NR_SHMEM_THPS) * HPAGE_PMD_NR), nid, K(node_page_state(pgdat, NR_SHMEM_PMDMAPPED) * diff --git a/fs/proc/meminfo.c b/fs/proc/meminfo.c index d6fc74619625..a635c8a84ddf 100644 --- a/fs/proc/meminfo.c +++ b/fs/proc/meminfo.c @@ -129,7 +129,7 @@ static int meminfo_proc_show(struct seq_file *m, void *v) #ifdef CONFIG_TRANSPARENT_HUGEPAGE show_val_kb(m, "AnonHugePages: ", - global_node_page_state(NR_ANON_THPS) * HPAGE_PMD_NR); + global_node_page_state(NR_ANON_THPS)); show_val_kb(m, "ShmemHugePages: ", global_node_page_state(NR_SHMEM_THPS) * HPAGE_PMD_NR); show_val_kb(m, "ShmemPmdMapped: ", diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 10dd3cae5f53..66ec454120de 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -2178,7 +2178,8 @@ static void __split_huge_pmd_locked(struct vm_area_struct *vma, pmd_t *pmd, lock_page_memcg(page); if (atomic_add_negative(-1, compound_mapcount_ptr(page))) { /* Last compound_mapcount is gone. */ - __dec_lruvec_page_state(page, NR_ANON_THPS); + __mod_lruvec_page_state(page, NR_ANON_THPS, + -HPAGE_PMD_NR); if (TestClearPageDoubleMap(page)) { /* No need in mapcount reference anymore */ for (i = 0; i < HPAGE_PMD_NR; i++) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 8818bf64d6fe..b18e25a5cdf3 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -1532,7 +1532,7 @@ static struct memory_stat memory_stats[] = { * on some architectures, the macro of HPAGE_PMD_SIZE is not * constant(e.g. powerpc). */ - { "anon_thp", 0, NR_ANON_THPS }, + { "anon_thp", PAGE_SIZE, NR_ANON_THPS }, { "file_thp", 0, NR_FILE_THPS }, { "shmem_thp", 0, NR_SHMEM_THPS }, #endif @@ -1565,8 +1565,7 @@ static int __init memory_stats_init(void) for (i = 0; i < ARRAY_SIZE(memory_stats); i++) { #ifdef CONFIG_TRANSPARENT_HUGEPAGE - if (memory_stats[i].idx == NR_ANON_THPS || - memory_stats[i].idx == NR_FILE_THPS || + if (memory_stats[i].idx == NR_FILE_THPS || memory_stats[i].idx == NR_SHMEM_THPS) memory_stats[i].ratio = HPAGE_PMD_SIZE; #endif @@ -4088,10 +4087,6 @@ static int memcg_stat_show(struct seq_file *m, void *v) if (memcg1_stats[i] == MEMCG_SWAP && !do_memsw_account()) continue; nr = memcg_page_state_local(memcg, memcg1_stats[i]); -#ifdef CONFIG_TRANSPARENT_HUGEPAGE - if (memcg1_stats[i] == NR_ANON_THPS) - nr *= HPAGE_PMD_NR; -#endif seq_printf(m, "%s %lu\n", memcg1_stat_names[i], nr * PAGE_SIZE); } @@ -4122,10 +4117,6 @@ static int memcg_stat_show(struct seq_file *m, void *v) if (memcg1_stats[i] == MEMCG_SWAP && !do_memsw_account()) continue; nr = memcg_page_state(memcg, memcg1_stats[i]); -#ifdef CONFIG_TRANSPARENT_HUGEPAGE - if (memcg1_stats[i] == NR_ANON_THPS) - nr *= HPAGE_PMD_NR; -#endif seq_printf(m, "total_%s %llu\n", memcg1_stat_names[i], (u64)nr * PAGE_SIZE); } @@ -5653,10 +5644,11 @@ static int mem_cgroup_move_account(struct page *page, __mod_lruvec_state(from_vec, NR_ANON_MAPPED, -nr_pages); __mod_lruvec_state(to_vec, NR_ANON_MAPPED, nr_pages); if (PageTransHuge(page)) { - __dec_lruvec_state(from_vec, NR_ANON_THPS); - __inc_lruvec_state(to_vec, NR_ANON_THPS); + __mod_lruvec_state(from_vec, NR_ANON_THPS, + -nr_pages); + __mod_lruvec_state(to_vec, NR_ANON_THPS, + nr_pages); } - } } else { __mod_lruvec_state(from_vec, NR_FILE_PAGES, -nr_pages); diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 469e28f95ce7..1700f52b7869 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -5580,7 +5580,7 @@ void show_free_areas(unsigned int filter, nodemask_t *nodemask) K(node_page_state(pgdat, NR_SHMEM_THPS) * HPAGE_PMD_NR), K(node_page_state(pgdat, NR_SHMEM_PMDMAPPED) * HPAGE_PMD_NR), - K(node_page_state(pgdat, NR_ANON_THPS) * HPAGE_PMD_NR), + K(node_page_state(pgdat, NR_ANON_THPS)), #endif K(node_page_state(pgdat, NR_WRITEBACK_TEMP)), node_page_state(pgdat, NR_KERNEL_STACK_KB), diff --git a/mm/rmap.c b/mm/rmap.c index 08c56aaf72eb..f59e92e26b61 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -1144,7 +1144,8 @@ void do_page_add_anon_rmap(struct page *page, * disabled. */ if (compound) - __inc_lruvec_page_state(page, NR_ANON_THPS); + __mod_lruvec_page_state(page, NR_ANON_THPS, + HPAGE_PMD_NR); __mod_lruvec_page_state(page, NR_ANON_MAPPED, nr); } @@ -1186,7 +1187,7 @@ void page_add_new_anon_rmap(struct page *page, if (hpage_pincount_available(page)) atomic_set(compound_pincount_ptr(page), 0); - __inc_lruvec_page_state(page, NR_ANON_THPS); + __mod_lruvec_page_state(page, NR_ANON_THPS, HPAGE_PMD_NR); } else { /* Anon THP always mapped first with PMD */ VM_BUG_ON_PAGE(PageTransCompound(page), page); @@ -1292,7 +1293,7 @@ static void page_remove_anon_compound_rmap(struct page *page) if (!IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE)) return; - __dec_lruvec_page_state(page, NR_ANON_THPS); + __mod_lruvec_page_state(page, NR_ANON_THPS, -HPAGE_PMD_NR); if (TestClearPageDoubleMap(page)) { /* From patchwork Tue Dec 8 04:18:43 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 11957521 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9764BC433FE for ; Tue, 8 Dec 2020 04:21:16 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 184AD239EB for ; Tue, 8 Dec 2020 04:21:16 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 184AD239EB Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id A08FB8D001D; Mon, 7 Dec 2020 23:21:15 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 9B8788D0002; Mon, 7 Dec 2020 23:21:15 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8A7BA8D001D; Mon, 7 Dec 2020 23:21:15 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0147.hostedemail.com [216.40.44.147]) by kanga.kvack.org (Postfix) with ESMTP id 7230E8D0002 for ; Mon, 7 Dec 2020 23:21:15 -0500 (EST) Received: from smtpin24.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 417223628 for ; Tue, 8 Dec 2020 04:21:15 +0000 (UTC) X-FDA: 77568815310.24.rifle02_1e03b6e273e4 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin24.hostedemail.com (Postfix) with ESMTP id 2303D1A4A0 for ; Tue, 8 Dec 2020 04:21:15 +0000 (UTC) X-HE-Tag: rifle02_1e03b6e273e4 X-Filterd-Recvd-Size: 7251 Received: from mail-pl1-f174.google.com (mail-pl1-f174.google.com [209.85.214.174]) by imf41.hostedemail.com (Postfix) with ESMTP for ; Tue, 8 Dec 2020 04:21:14 +0000 (UTC) Received: by mail-pl1-f174.google.com with SMTP id v3so6299025plz.13 for ; Mon, 07 Dec 2020 20:21:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=3KuLT1Cn/hR9CSFlVwhhGNLVEez8vLOWizoZNvv9uxQ=; b=Vj4JJ+bRYX5Oj4q3X0L8Fh6Gls+2B/SXWUh4NrNW72Aqtu/BDQ3boPiqVkNwJDDs44 IS3iz2hn5UDA/kWICAwIBPkt4krmzbzPFMuds4SgvUdlbQrGw5lcfKEIzVcwjOkdDpvf MB9rfJai8zIHC6WAzQqjJkCfaHNqZ+5uZENHhIpzcRNXNz/jr43WQzYkJ/SWQmdywRja BZ0ty40DqI2t0ZHO1DDDbT0H6eJOJ/rngA3c0K6mew7YyX6o+71G5ojMNfh7mY9TZaWW v6P3wbNlykFS+3tfjxMhGQCfsJBYhy4PDAg3pKBNPZXNazNDcahRsjjIQZeEZNzkfuht l2HQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=3KuLT1Cn/hR9CSFlVwhhGNLVEez8vLOWizoZNvv9uxQ=; b=uHUgoH8fh72gxH0ImUGjZ97e1QnfBselDozv8Auzb1OUD44pmot6ICFS4+hsiqsUVJ dny920W5ROIEGXlWhyWyxiWZX2CBtYl4i39xFPsWGGP0ywaBzAHsKq7VjOJzsqi9O9NN XnVArQPf4+nRJlh8d8beGj+Ok78aCZTVjvmM0OMj4Q2oqZ6PWmO1o3CcZhlQeChsL2ln 6yYtZh0+Z+KKn79HbaDd3jxx3VUCWo4dvUZDZK7xza5eGn6BoaTcAQuMSSkx7eoh5SOY Bt6WTtY125Hm/GiYI8T6edP91LNSp4qL9XUyfTGySS68aaieA880g7ccf58+thxaEA2A g/MQ== X-Gm-Message-State: AOAM531nIPwVuQAfaFxGHNLSmHIz2IBavWPA9wukP/n974iEzIKQxUw4 fE8bm7m5SsaMH+jtxfR3bi54Zg== X-Google-Smtp-Source: ABdhPJwnxAh1QoB4VoC3+au0FbztqWHIniEtcuRmqnWoI3WNUPiQuqySX8qVI42PTtXkXaCGg8QztA== X-Received: by 2002:a17:902:26a:b029:da:af47:77c7 with SMTP id 97-20020a170902026ab02900daaf4777c7mr19699178plc.10.1607401273486; Mon, 07 Dec 2020 20:21:13 -0800 (PST) Received: from localhost.localdomain ([103.136.221.70]) by smtp.gmail.com with ESMTPSA id mr7sm1031166pjb.31.2020.12.07.20.21.06 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 07 Dec 2020 20:21:12 -0800 (PST) From: Muchun Song To: gregkh@linuxfoundation.org, rafael@kernel.org, adobriyan@gmail.com, akpm@linux-foundation.org, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, hughd@google.com, shakeelb@google.com, guro@fb.com, samitolvanen@google.com, feng.tang@intel.com, neilb@suse.de, iamjoonsoo.kim@lge.com, rdunlap@infradead.org Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, Muchun Song Subject: [PATCH v3 3/7] mm: memcontrol: convert NR_FILE_THPS account to pages Date: Tue, 8 Dec 2020 12:18:43 +0800 Message-Id: <20201208041847.72122-4-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20201208041847.72122-1-songmuchun@bytedance.com> References: <20201208041847.72122-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The unit of NR_FILE_THPS is HPAGE_PMD_NR. Converrt NR_FILE_THPS account to pages. Signed-off-by: Muchun Song --- drivers/base/node.c | 3 +-- fs/proc/meminfo.c | 2 +- mm/filemap.c | 2 +- mm/huge_memory.c | 3 ++- mm/khugepaged.c | 2 +- mm/memcontrol.c | 5 ++--- 6 files changed, 8 insertions(+), 9 deletions(-) diff --git a/drivers/base/node.c b/drivers/base/node.c index ec35cb567940..0f752d0fce6f 100644 --- a/drivers/base/node.c +++ b/drivers/base/node.c @@ -466,8 +466,7 @@ static ssize_t node_read_meminfo(struct device *dev, HPAGE_PMD_NR), nid, K(node_page_state(pgdat, NR_SHMEM_PMDMAPPED) * HPAGE_PMD_NR), - nid, K(node_page_state(pgdat, NR_FILE_THPS) * - HPAGE_PMD_NR), + nid, K(node_page_state(pgdat, NR_FILE_THPS)), nid, K(node_page_state(pgdat, NR_FILE_PMDMAPPED) * HPAGE_PMD_NR) #endif diff --git a/fs/proc/meminfo.c b/fs/proc/meminfo.c index a635c8a84ddf..7ea4679880c8 100644 --- a/fs/proc/meminfo.c +++ b/fs/proc/meminfo.c @@ -135,7 +135,7 @@ static int meminfo_proc_show(struct seq_file *m, void *v) show_val_kb(m, "ShmemPmdMapped: ", global_node_page_state(NR_SHMEM_PMDMAPPED) * HPAGE_PMD_NR); show_val_kb(m, "FileHugePages: ", - global_node_page_state(NR_FILE_THPS) * HPAGE_PMD_NR); + global_node_page_state(NR_FILE_THPS)); show_val_kb(m, "FilePmdMapped: ", global_node_page_state(NR_FILE_PMDMAPPED) * HPAGE_PMD_NR); #endif diff --git a/mm/filemap.c b/mm/filemap.c index 78090ee08ac2..9cc8b3ac9eac 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -207,7 +207,7 @@ static void unaccount_page_cache_page(struct address_space *mapping, if (PageTransHuge(page)) __dec_lruvec_page_state(page, NR_SHMEM_THPS); } else if (PageTransHuge(page)) { - __dec_lruvec_page_state(page, NR_FILE_THPS); + __mod_lruvec_page_state(page, NR_FILE_THPS, -HPAGE_PMD_NR); filemap_nr_thps_dec(mapping); } diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 66ec454120de..1e24165fa53a 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -2748,7 +2748,8 @@ int split_huge_page_to_list(struct page *page, struct list_head *list) if (PageSwapBacked(head)) __dec_lruvec_page_state(head, NR_SHMEM_THPS); else - __dec_lruvec_page_state(head, NR_FILE_THPS); + __mod_lruvec_page_state(head, NR_FILE_THPS, + -HPAGE_PMD_NR); } __split_huge_page(page, list, end); diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 494d3cb0b58a..76b3e064a72a 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1869,7 +1869,7 @@ static void collapse_file(struct mm_struct *mm, if (is_shmem) __inc_lruvec_page_state(new_page, NR_SHMEM_THPS); else { - __inc_lruvec_page_state(new_page, NR_FILE_THPS); + __mod_lruvec_page_state(new_page, NR_FILE_THPS, HPAGE_PMD_NR); filemap_nr_thps_inc(mapping); } diff --git a/mm/memcontrol.c b/mm/memcontrol.c index b18e25a5cdf3..04985c8c6a0a 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -1533,7 +1533,7 @@ static struct memory_stat memory_stats[] = { * constant(e.g. powerpc). */ { "anon_thp", PAGE_SIZE, NR_ANON_THPS }, - { "file_thp", 0, NR_FILE_THPS }, + { "file_thp", PAGE_SIZE, NR_FILE_THPS }, { "shmem_thp", 0, NR_SHMEM_THPS }, #endif { "inactive_anon", PAGE_SIZE, NR_INACTIVE_ANON }, @@ -1565,8 +1565,7 @@ static int __init memory_stats_init(void) for (i = 0; i < ARRAY_SIZE(memory_stats); i++) { #ifdef CONFIG_TRANSPARENT_HUGEPAGE - if (memory_stats[i].idx == NR_FILE_THPS || - memory_stats[i].idx == NR_SHMEM_THPS) + if (memory_stats[i].idx == NR_SHMEM_THPS) memory_stats[i].ratio = HPAGE_PMD_SIZE; #endif VM_BUG_ON(!memory_stats[i].ratio); From patchwork Tue Dec 8 04:18:44 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 11957523 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.7 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,UNWANTED_LANGUAGE_BODY, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 90E93C4361B for ; Tue, 8 Dec 2020 04:21:23 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 00012239EB for ; Tue, 8 Dec 2020 04:21:22 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 00012239EB Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 833A28D001E; Mon, 7 Dec 2020 23:21:22 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 7E45E8D0002; Mon, 7 Dec 2020 23:21:22 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6F99E8D001E; Mon, 7 Dec 2020 23:21:22 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0027.hostedemail.com [216.40.44.27]) by kanga.kvack.org (Postfix) with ESMTP id 56DE78D0002 for ; Mon, 7 Dec 2020 23:21:22 -0500 (EST) Received: from smtpin17.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 1EADE1EE6 for ; Tue, 8 Dec 2020 04:21:22 +0000 (UTC) X-FDA: 77568815604.17.box97_3e0d353273e4 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin17.hostedemail.com (Postfix) with ESMTP id F3BD8180D0180 for ; Tue, 8 Dec 2020 04:21:21 +0000 (UTC) X-HE-Tag: box97_3e0d353273e4 X-Filterd-Recvd-Size: 9423 Received: from mail-pf1-f179.google.com (mail-pf1-f179.google.com [209.85.210.179]) by imf49.hostedemail.com (Postfix) with ESMTP for ; Tue, 8 Dec 2020 04:21:21 +0000 (UTC) Received: by mail-pf1-f179.google.com with SMTP id t8so12634669pfg.8 for ; Mon, 07 Dec 2020 20:21:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ilOzfJzaEHpIeYQ1I62uxEC1j7ecDDpVYKn/PbVz7hk=; b=XAcT6IZjvIsK78V1kU3fVc+3JSdnxeTFMwOXOyo1Mnai2TKC6UDSmEwqqVjAFRutkW uuG4pVNg6Ow8O3d3GXo8Hfh+9u8Kg+zVpKEcrbzFdFdQvhMEGG66tBeI3NsWzGtPRvgY cQVSE6cRx7HoI5o0resydxFRLl7i0FO0vzYYr1gZRb5qT6p19Ix32bjMiGEgcxCcN4iY oLrIVgYyKWWA5ouTpU/GBragMXQZCawZqnxrhA6Rg6OMuzkYU7H03OfPQIMnXE6VwH6+ jf29aQOLRTWPVW7xdHqgTuKDy3icUydEJ52U9MsdQ40reeh6f9fe9loy8KkwfDAk3Tx0 i8lg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ilOzfJzaEHpIeYQ1I62uxEC1j7ecDDpVYKn/PbVz7hk=; b=h9EoqKADWshw6OsrWwkeSsEKBvTZz6+wM0pecUNeLQfRqT6Sef8zNwMPcoZX0QjAvR 4EUB9WOkfb4s6P9Zr82EWPDpYJAMIBy9GhOsZKj0zGnQCxmdG43ezssjrqk+URfn4gwg ROvPCy2VYwoiDjylRdv9V/3LkRO5EpV2LwurRzSsV5yv2ASfo1Eh3fWX5Qb2MTkh1rDP CsI/jRelJsgmghGUs6HCk1IcKvMjhP1jEmrMSVf2JsLmNNioAw9Ndileqg+Q/lp15Y5A mJbLbTw1/+aE9IdY/VYayi+UD2X8pnW6C6Of2lkxn2/lhRIUTX0UMNv12TUFpohmfU2n Q03w== X-Gm-Message-State: AOAM530NoRp6jL126OMLX7SCSO8Qzmr7IfYiUQl3/O1850pAuqxSTwnV HTyqkxc3Xs0SPsaTcoST4ZOkgQ== X-Google-Smtp-Source: ABdhPJyja+iimN7aQkpSVwz43jQR93Eu/rJTDpXWJywNtThoOb+74KO9VtjSDgusudI0r76oNw4OKg== X-Received: by 2002:a17:90a:7ac8:: with SMTP id b8mr2238562pjl.109.1607401280680; Mon, 07 Dec 2020 20:21:20 -0800 (PST) Received: from localhost.localdomain ([103.136.221.70]) by smtp.gmail.com with ESMTPSA id mr7sm1031166pjb.31.2020.12.07.20.21.13 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 07 Dec 2020 20:21:20 -0800 (PST) From: Muchun Song To: gregkh@linuxfoundation.org, rafael@kernel.org, adobriyan@gmail.com, akpm@linux-foundation.org, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, hughd@google.com, shakeelb@google.com, guro@fb.com, samitolvanen@google.com, feng.tang@intel.com, neilb@suse.de, iamjoonsoo.kim@lge.com, rdunlap@infradead.org Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, Muchun Song Subject: [PATCH v3 4/7] mm: memcontrol: convert NR_SHMEM_THPS account to pages Date: Tue, 8 Dec 2020 12:18:44 +0800 Message-Id: <20201208041847.72122-5-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20201208041847.72122-1-songmuchun@bytedance.com> References: <20201208041847.72122-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The unit of NR_SHMEM_THPS is HPAGE_PMD_NR. Convert NR_SHMEM_THPS account to pages Signed-off-by: Muchun Song --- drivers/base/node.c | 3 +-- fs/proc/meminfo.c | 2 +- mm/filemap.c | 2 +- mm/huge_memory.c | 3 ++- mm/khugepaged.c | 2 +- mm/memcontrol.c | 26 ++------------------------ mm/page_alloc.c | 2 +- mm/shmem.c | 3 ++- 8 files changed, 11 insertions(+), 32 deletions(-) diff --git a/drivers/base/node.c b/drivers/base/node.c index 0f752d0fce6f..a7f298ae693a 100644 --- a/drivers/base/node.c +++ b/drivers/base/node.c @@ -462,8 +462,7 @@ static ssize_t node_read_meminfo(struct device *dev, #ifdef CONFIG_TRANSPARENT_HUGEPAGE , nid, K(node_page_state(pgdat, NR_ANON_THPS)), - nid, K(node_page_state(pgdat, NR_SHMEM_THPS) * - HPAGE_PMD_NR), + nid, K(node_page_state(pgdat, NR_SHMEM_THPS)), nid, K(node_page_state(pgdat, NR_SHMEM_PMDMAPPED) * HPAGE_PMD_NR), nid, K(node_page_state(pgdat, NR_FILE_THPS)), diff --git a/fs/proc/meminfo.c b/fs/proc/meminfo.c index 7ea4679880c8..cfb107eaa3e6 100644 --- a/fs/proc/meminfo.c +++ b/fs/proc/meminfo.c @@ -131,7 +131,7 @@ static int meminfo_proc_show(struct seq_file *m, void *v) show_val_kb(m, "AnonHugePages: ", global_node_page_state(NR_ANON_THPS)); show_val_kb(m, "ShmemHugePages: ", - global_node_page_state(NR_SHMEM_THPS) * HPAGE_PMD_NR); + global_node_page_state(NR_SHMEM_THPS)); show_val_kb(m, "ShmemPmdMapped: ", global_node_page_state(NR_SHMEM_PMDMAPPED) * HPAGE_PMD_NR); show_val_kb(m, "FileHugePages: ", diff --git a/mm/filemap.c b/mm/filemap.c index 9cc8b3ac9eac..c653717b92b6 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -205,7 +205,7 @@ static void unaccount_page_cache_page(struct address_space *mapping, if (PageSwapBacked(page)) { __mod_lruvec_page_state(page, NR_SHMEM, -nr); if (PageTransHuge(page)) - __dec_lruvec_page_state(page, NR_SHMEM_THPS); + __mod_lruvec_page_state(page, NR_SHMEM_THPS, -HPAGE_PMD_NR); } else if (PageTransHuge(page)) { __mod_lruvec_page_state(page, NR_FILE_THPS, -HPAGE_PMD_NR); filemap_nr_thps_dec(mapping); diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 1e24165fa53a..ac552807492e 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -2746,7 +2746,8 @@ int split_huge_page_to_list(struct page *page, struct list_head *list) spin_unlock(&ds_queue->split_queue_lock); if (mapping) { if (PageSwapBacked(head)) - __dec_lruvec_page_state(head, NR_SHMEM_THPS); + __mod_lruvec_page_state(head, NR_SHMEM_THPS, + -HPAGE_PMD_NR); else __mod_lruvec_page_state(head, NR_FILE_THPS, -HPAGE_PMD_NR); diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 76b3e064a72a..2f1c22331188 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1867,7 +1867,7 @@ static void collapse_file(struct mm_struct *mm, } if (is_shmem) - __inc_lruvec_page_state(new_page, NR_SHMEM_THPS); + __mod_lruvec_page_state(new_page, NR_SHMEM_THPS, HPAGE_PMD_NR); else { __mod_lruvec_page_state(new_page, NR_FILE_THPS, HPAGE_PMD_NR); filemap_nr_thps_inc(mapping); diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 04985c8c6a0a..a40797a27f87 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -1515,7 +1515,7 @@ struct memory_stat { unsigned int idx; }; -static struct memory_stat memory_stats[] = { +static const struct memory_stat memory_stats[] = { { "anon", PAGE_SIZE, NR_ANON_MAPPED }, { "file", PAGE_SIZE, NR_FILE_PAGES }, { "kernel_stack", 1024, NR_KERNEL_STACK_KB }, @@ -1527,14 +1527,9 @@ static struct memory_stat memory_stats[] = { { "file_dirty", PAGE_SIZE, NR_FILE_DIRTY }, { "file_writeback", PAGE_SIZE, NR_WRITEBACK }, #ifdef CONFIG_TRANSPARENT_HUGEPAGE - /* - * The ratio will be initialized in memory_stats_init(). Because - * on some architectures, the macro of HPAGE_PMD_SIZE is not - * constant(e.g. powerpc). - */ { "anon_thp", PAGE_SIZE, NR_ANON_THPS }, { "file_thp", PAGE_SIZE, NR_FILE_THPS }, - { "shmem_thp", 0, NR_SHMEM_THPS }, + { "shmem_thp", PAGE_SIZE, NR_SHMEM_THPS }, #endif { "inactive_anon", PAGE_SIZE, NR_INACTIVE_ANON }, { "active_anon", PAGE_SIZE, NR_ACTIVE_ANON }, @@ -1559,23 +1554,6 @@ static struct memory_stat memory_stats[] = { { "workingset_nodereclaim", 1, WORKINGSET_NODERECLAIM }, }; -static int __init memory_stats_init(void) -{ - int i; - - for (i = 0; i < ARRAY_SIZE(memory_stats); i++) { -#ifdef CONFIG_TRANSPARENT_HUGEPAGE - if (memory_stats[i].idx == NR_SHMEM_THPS) - memory_stats[i].ratio = HPAGE_PMD_SIZE; -#endif - VM_BUG_ON(!memory_stats[i].ratio); - VM_BUG_ON(memory_stats[i].idx >= MEMCG_NR_STAT); - } - - return 0; -} -pure_initcall(memory_stats_init); - static char *memory_stat_format(struct mem_cgroup *memcg) { struct seq_buf s; diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 1700f52b7869..720fb5a220b6 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -5577,7 +5577,7 @@ void show_free_areas(unsigned int filter, nodemask_t *nodemask) K(node_page_state(pgdat, NR_WRITEBACK)), K(node_page_state(pgdat, NR_SHMEM)), #ifdef CONFIG_TRANSPARENT_HUGEPAGE - K(node_page_state(pgdat, NR_SHMEM_THPS) * HPAGE_PMD_NR), + K(node_page_state(pgdat, NR_SHMEM_THPS)), K(node_page_state(pgdat, NR_SHMEM_PMDMAPPED) * HPAGE_PMD_NR), K(node_page_state(pgdat, NR_ANON_THPS)), diff --git a/mm/shmem.c b/mm/shmem.c index 53d84d2c9fe5..b2ca3beabc19 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -713,7 +713,8 @@ static int shmem_add_to_page_cache(struct page *page, } if (PageTransHuge(page)) { count_vm_event(THP_FILE_ALLOC); - __inc_lruvec_page_state(page, NR_SHMEM_THPS); + __mod_lruvec_page_state(page, NR_SHMEM_THPS, + HPAGE_PMD_NR); } mapping->nrpages += nr; __mod_lruvec_page_state(page, NR_FILE_PAGES, nr); From patchwork Tue Dec 8 04:18:45 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 11957541 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AA4C3C4167B for ; Tue, 8 Dec 2020 04:21:30 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 5F44A23A61 for ; Tue, 8 Dec 2020 04:21:30 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5F44A23A61 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id DB2698D001F; Mon, 7 Dec 2020 23:21:29 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D61BC8D0002; Mon, 7 Dec 2020 23:21:29 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C78158D001F; Mon, 7 Dec 2020 23:21:29 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0060.hostedemail.com [216.40.44.60]) by kanga.kvack.org (Postfix) with ESMTP id AE3F98D0002 for ; Mon, 7 Dec 2020 23:21:29 -0500 (EST) Received: from smtpin04.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 7E0A6181AEF21 for ; Tue, 8 Dec 2020 04:21:29 +0000 (UTC) X-FDA: 77568815898.04.paper63_310cc88273e4 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin04.hostedemail.com (Postfix) with ESMTP id 6267480081D8 for ; Tue, 8 Dec 2020 04:21:29 +0000 (UTC) X-HE-Tag: paper63_310cc88273e4 X-Filterd-Recvd-Size: 6485 Received: from mail-pf1-f196.google.com (mail-pf1-f196.google.com [209.85.210.196]) by imf41.hostedemail.com (Postfix) with ESMTP for ; Tue, 8 Dec 2020 04:21:28 +0000 (UTC) Received: by mail-pf1-f196.google.com with SMTP id c79so12642021pfc.2 for ; Mon, 07 Dec 2020 20:21:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=5QtzUPSaCCxeqzmaLwiZLdmUDR0T79YdbdBWjz3bjU0=; b=mc4LQi+8Dvw0LMhxU20Dluvm2ZSyZowOBRFKbsg4+E2QKQkG+5WEUvL3sq7+H+KsIG /2PbPyQsG1YW1ToGRNXGF9EmUdWNsHJTkjDCNcOvPqwNrTOViEnPA2kmUGOPQ9xIR8bm YGC4aheG0ammcsnwywGtpfGNKK9Zd3GigZ61K7sz44/qSRPuehM2Qqwo12E4wpkhvrH9 aAnnFsNssqM9nCLjet0TxIL/KzaOE4fRuZwnTtNc5sEgPGlBOlMLBSSz0y72K1ctDj6D S51rJ0KDs5C1EOhBpJH1OXIpV3SRLdo7Raf1opYzy5xE0kanXE5ccDXXUhjDIkAzh3T0 X7fg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=5QtzUPSaCCxeqzmaLwiZLdmUDR0T79YdbdBWjz3bjU0=; b=IZZKUgaBiarkpBPDTByb0e12rTBptNFb1IPFJP80PsM/z+/sATMQoDICEObJAqG4ZH SM5EQSHBv7zmueiyQPSyrYIV/jj8/IiLoi3REUoGc7StjqJ0WIIuuxI8WhuulHQhSSBu 1mCsEP1cMSmTXH51ofAhiyFEN8ugPvIZ1PsJrP+0DaXTcgRoUeqE2bv67yBVxCGIpAQJ HZv0iszBpSkv0R10gbVeCSBcXvQs/MyrvWQrbAmM5UYaOh8pRW151NITL7K7gBLEDDLZ CwgyamTamtIIjCnP/USHNHAtbVqFe7g3jVer+GB/6wzpipE02EcznLlj2/eNUxT/Fh+5 WURw== X-Gm-Message-State: AOAM531K1EbhtwEHaMcehv22IGkxrbhPSv3Xd5pAhO+I7I1N1Y2eTn7z Qg0tP8UVMES6EyYMIR7E89gdnA== X-Google-Smtp-Source: ABdhPJwgTvXrSNNUdeX86od1g83iYDUqnnM9L9ZoCFjA9qyAxMxXscN568YbBrQgdQtIM9Jk5wEabQ== X-Received: by 2002:a62:5c87:0:b029:197:5f13:b66c with SMTP id q129-20020a625c870000b02901975f13b66cmr19288373pfb.73.1607401288139; Mon, 07 Dec 2020 20:21:28 -0800 (PST) Received: from localhost.localdomain ([103.136.221.70]) by smtp.gmail.com with ESMTPSA id mr7sm1031166pjb.31.2020.12.07.20.21.21 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 07 Dec 2020 20:21:27 -0800 (PST) From: Muchun Song To: gregkh@linuxfoundation.org, rafael@kernel.org, adobriyan@gmail.com, akpm@linux-foundation.org, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, hughd@google.com, shakeelb@google.com, guro@fb.com, samitolvanen@google.com, feng.tang@intel.com, neilb@suse.de, iamjoonsoo.kim@lge.com, rdunlap@infradead.org Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, Muchun Song Subject: [PATCH v3 5/7] mm: memcontrol: convert NR_SHMEM_PMDMAPPED account to pages Date: Tue, 8 Dec 2020 12:18:45 +0800 Message-Id: <20201208041847.72122-6-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20201208041847.72122-1-songmuchun@bytedance.com> References: <20201208041847.72122-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The unit of NR_SHMEM_PMDMAPPED is HPAGE_PMD_NR. Convert NR_SHMEM_PMDMAPPED account to pages. Signed-off-by: Muchun Song --- drivers/base/node.c | 3 +-- fs/proc/meminfo.c | 2 +- mm/page_alloc.c | 3 +-- mm/rmap.c | 6 ++++-- 4 files changed, 7 insertions(+), 7 deletions(-) diff --git a/drivers/base/node.c b/drivers/base/node.c index a7f298ae693a..45293084bb19 100644 --- a/drivers/base/node.c +++ b/drivers/base/node.c @@ -463,8 +463,7 @@ static ssize_t node_read_meminfo(struct device *dev, , nid, K(node_page_state(pgdat, NR_ANON_THPS)), nid, K(node_page_state(pgdat, NR_SHMEM_THPS)), - nid, K(node_page_state(pgdat, NR_SHMEM_PMDMAPPED) * - HPAGE_PMD_NR), + nid, K(node_page_state(pgdat, NR_SHMEM_PMDMAPPED)), nid, K(node_page_state(pgdat, NR_FILE_THPS)), nid, K(node_page_state(pgdat, NR_FILE_PMDMAPPED) * HPAGE_PMD_NR) diff --git a/fs/proc/meminfo.c b/fs/proc/meminfo.c index cfb107eaa3e6..c61f440570f9 100644 --- a/fs/proc/meminfo.c +++ b/fs/proc/meminfo.c @@ -133,7 +133,7 @@ static int meminfo_proc_show(struct seq_file *m, void *v) show_val_kb(m, "ShmemHugePages: ", global_node_page_state(NR_SHMEM_THPS)); show_val_kb(m, "ShmemPmdMapped: ", - global_node_page_state(NR_SHMEM_PMDMAPPED) * HPAGE_PMD_NR); + global_node_page_state(NR_SHMEM_PMDMAPPED)); show_val_kb(m, "FileHugePages: ", global_node_page_state(NR_FILE_THPS)); show_val_kb(m, "FilePmdMapped: ", diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 720fb5a220b6..575fbfeea4b5 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -5578,8 +5578,7 @@ void show_free_areas(unsigned int filter, nodemask_t *nodemask) K(node_page_state(pgdat, NR_SHMEM)), #ifdef CONFIG_TRANSPARENT_HUGEPAGE K(node_page_state(pgdat, NR_SHMEM_THPS)), - K(node_page_state(pgdat, NR_SHMEM_PMDMAPPED) - * HPAGE_PMD_NR), + K(node_page_state(pgdat, NR_SHMEM_PMDMAPPED)), K(node_page_state(pgdat, NR_ANON_THPS)), #endif K(node_page_state(pgdat, NR_WRITEBACK_TEMP)), diff --git a/mm/rmap.c b/mm/rmap.c index f59e92e26b61..3089ad6bf468 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -1219,7 +1219,8 @@ void page_add_file_rmap(struct page *page, bool compound) if (!atomic_inc_and_test(compound_mapcount_ptr(page))) goto out; if (PageSwapBacked(page)) - __inc_node_page_state(page, NR_SHMEM_PMDMAPPED); + __mod_lruvec_page_state(page, NR_SHMEM_PMDMAPPED, + HPAGE_PMD_NR); else __inc_node_page_state(page, NR_FILE_PMDMAPPED); } else { @@ -1260,7 +1261,8 @@ static void page_remove_file_rmap(struct page *page, bool compound) if (!atomic_add_negative(-1, compound_mapcount_ptr(page))) return; if (PageSwapBacked(page)) - __dec_node_page_state(page, NR_SHMEM_PMDMAPPED); + __mod_lruvec_page_state(page, NR_SHMEM_PMDMAPPED, + -HPAGE_PMD_NR); else __dec_node_page_state(page, NR_FILE_PMDMAPPED); } else { From patchwork Tue Dec 8 04:18:46 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 11957543 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7D481C4361B for ; Tue, 8 Dec 2020 04:21:38 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 16DA7239EB for ; Tue, 8 Dec 2020 04:21:38 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 16DA7239EB Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 994D68D0020; Mon, 7 Dec 2020 23:21:37 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 942FD8D0002; Mon, 7 Dec 2020 23:21:37 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7BD1E8D0020; Mon, 7 Dec 2020 23:21:37 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0248.hostedemail.com [216.40.44.248]) by kanga.kvack.org (Postfix) with ESMTP id 653648D0002 for ; Mon, 7 Dec 2020 23:21:37 -0500 (EST) Received: from smtpin27.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 2C5E6180AD804 for ; Tue, 8 Dec 2020 04:21:37 +0000 (UTC) X-FDA: 77568816234.27.nut30_2600303273e4 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin27.hostedemail.com (Postfix) with ESMTP id 0BD453D663 for ; Tue, 8 Dec 2020 04:21:37 +0000 (UTC) X-HE-Tag: nut30_2600303273e4 X-Filterd-Recvd-Size: 5756 Received: from mail-pf1-f194.google.com (mail-pf1-f194.google.com [209.85.210.194]) by imf09.hostedemail.com (Postfix) with ESMTP for ; Tue, 8 Dec 2020 04:21:36 +0000 (UTC) Received: by mail-pf1-f194.google.com with SMTP id p4so6759808pfg.0 for ; Mon, 07 Dec 2020 20:21:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=l2/ZiqpsepZIKX1SS9g/oGWRoEH2RxsiZeFtxG7E1hY=; b=S8M0B/X4Ja/6hprP34VU2OQyUfKlylvqpIPM6XSSxRwqhRjKKLpZJOl0k6OEp+xaYM mZ2X4yVhKR6L/pN12zw2Kffg4h2R5TXsaSxK64AkNIV/pqAL2H6TDD3E3Uo9Gx68AhzA HutJNM4eV9CpjbgsgUvgPLLi7RszFvMHaOZU8sSio5P3BRJ8T7HRjO4OtanKlh9daBRX MoP/XcIJ6ymzAuMIQ0BcfLyS6eEw7pB2ttIGnhB+4WZVZC9vkxWdAdvKJvU+GC2sgr8B HfNcRSzFzYPba9SXniBut3LclEGo4j0FShH6TiKAk1eNJCFfKs6d6ZcuCSUwUgEAjO/L FYzQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=l2/ZiqpsepZIKX1SS9g/oGWRoEH2RxsiZeFtxG7E1hY=; b=HirRZbID8G1TjJO66I2jIJWBAnjBl5maNVa/Gq8WIkF/y2gs7fQEdIuzalYMXOcr7o 72EiLQXbtCOe9AUSV9eMFKvTdByfjqqPvuvMGk4PaH0IE16DaE3+rT2lPaAXatikzeNF yOTARVkgH9ayZg14bP7/U6ULAqNwhyozWdLOnL6TGOStQvVPuuERToUTQRWf/nlvXpCl YJS93G01Z4XnJb8qCoCgr8O6RyKzThEBkqQt5nLj+Rjcj1XdTjb+m/+HPAwG2cXnqMvt Nd6OU2uFCSx/7bDYN5HLe4ZaetVbQ2knWj91nOCpreQMD4j3wOTKmO6gAhKm1duF+Afq jYyw== X-Gm-Message-State: AOAM5312IG/vUw/OGEHnm/VK+6bh1bhprd6+r/Hkgbtfp+fjQDyBv2v0 MAeyQUft7t2NKCI+uOvHH3Jt4g== X-Google-Smtp-Source: ABdhPJymx+vIWOX3gvWNwXYz6IZpIemiL3RX0BEW6eUqRsqAvsi7ntancYcLOEluQuMi/s75zDV16g== X-Received: by 2002:a62:5e81:0:b029:197:baa5:1792 with SMTP id s123-20020a625e810000b0290197baa51792mr18804110pfb.80.1607401295606; Mon, 07 Dec 2020 20:21:35 -0800 (PST) Received: from localhost.localdomain ([103.136.221.70]) by smtp.gmail.com with ESMTPSA id mr7sm1031166pjb.31.2020.12.07.20.21.28 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 07 Dec 2020 20:21:35 -0800 (PST) From: Muchun Song To: gregkh@linuxfoundation.org, rafael@kernel.org, adobriyan@gmail.com, akpm@linux-foundation.org, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, hughd@google.com, shakeelb@google.com, guro@fb.com, samitolvanen@google.com, feng.tang@intel.com, neilb@suse.de, iamjoonsoo.kim@lge.com, rdunlap@infradead.org Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, Muchun Song Subject: [PATCH v3 6/7] mm: memcontrol: convert NR_FILE_PMDMAPPED account to pages Date: Tue, 8 Dec 2020 12:18:46 +0800 Message-Id: <20201208041847.72122-7-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20201208041847.72122-1-songmuchun@bytedance.com> References: <20201208041847.72122-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The unit of NR_FILE_PMDMAPPED is HPAGE_PMD_NR. Convert NR_FILE_PMDMAPPED account to pages. Signed-off-by: Muchun Song --- drivers/base/node.c | 3 +-- fs/proc/meminfo.c | 2 +- mm/rmap.c | 6 ++++-- 3 files changed, 6 insertions(+), 5 deletions(-) diff --git a/drivers/base/node.c b/drivers/base/node.c index 45293084bb19..04505beca104 100644 --- a/drivers/base/node.c +++ b/drivers/base/node.c @@ -465,8 +465,7 @@ static ssize_t node_read_meminfo(struct device *dev, nid, K(node_page_state(pgdat, NR_SHMEM_THPS)), nid, K(node_page_state(pgdat, NR_SHMEM_PMDMAPPED)), nid, K(node_page_state(pgdat, NR_FILE_THPS)), - nid, K(node_page_state(pgdat, NR_FILE_PMDMAPPED) * - HPAGE_PMD_NR) + nid, K(node_page_state(pgdat, NR_FILE_PMDMAPPED)) #endif ); len += hugetlb_report_node_meminfo(buf, len, nid); diff --git a/fs/proc/meminfo.c b/fs/proc/meminfo.c index c61f440570f9..6fa761c9cc78 100644 --- a/fs/proc/meminfo.c +++ b/fs/proc/meminfo.c @@ -137,7 +137,7 @@ static int meminfo_proc_show(struct seq_file *m, void *v) show_val_kb(m, "FileHugePages: ", global_node_page_state(NR_FILE_THPS)); show_val_kb(m, "FilePmdMapped: ", - global_node_page_state(NR_FILE_PMDMAPPED) * HPAGE_PMD_NR); + global_node_page_state(NR_FILE_PMDMAPPED)); #endif #ifdef CONFIG_CMA diff --git a/mm/rmap.c b/mm/rmap.c index 3089ad6bf468..e383c5619501 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -1222,7 +1222,8 @@ void page_add_file_rmap(struct page *page, bool compound) __mod_lruvec_page_state(page, NR_SHMEM_PMDMAPPED, HPAGE_PMD_NR); else - __inc_node_page_state(page, NR_FILE_PMDMAPPED); + __mod_lruvec_page_state(page, NR_FILE_PMDMAPPED, + HPAGE_PMD_NR); } else { if (PageTransCompound(page) && page_mapping(page)) { VM_WARN_ON_ONCE(!PageLocked(page)); @@ -1264,7 +1265,8 @@ static void page_remove_file_rmap(struct page *page, bool compound) __mod_lruvec_page_state(page, NR_SHMEM_PMDMAPPED, -HPAGE_PMD_NR); else - __dec_node_page_state(page, NR_FILE_PMDMAPPED); + __mod_lruvec_page_state(page, NR_FILE_PMDMAPPED, + -HPAGE_PMD_NR); } else { if (!atomic_add_negative(-1, &page->_mapcount)) return; From patchwork Tue Dec 8 04:18:47 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 11957545 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C6136C4361B for ; Tue, 8 Dec 2020 04:21:45 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 50826239EB for ; Tue, 8 Dec 2020 04:21:45 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 50826239EB Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id C535A8D0021; Mon, 7 Dec 2020 23:21:44 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id C02F08D0002; Mon, 7 Dec 2020 23:21:44 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B18728D0021; Mon, 7 Dec 2020 23:21:44 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0198.hostedemail.com [216.40.44.198]) by kanga.kvack.org (Postfix) with ESMTP id 974C08D0002 for ; Mon, 7 Dec 2020 23:21:44 -0500 (EST) Received: from smtpin29.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 6654D1EE6 for ; Tue, 8 Dec 2020 04:21:44 +0000 (UTC) X-FDA: 77568816528.29.sand92_410816f273e4 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin29.hostedemail.com (Postfix) with ESMTP id 46E64180868E8 for ; Tue, 8 Dec 2020 04:21:44 +0000 (UTC) X-HE-Tag: sand92_410816f273e4 X-Filterd-Recvd-Size: 9627 Received: from mail-pl1-f193.google.com (mail-pl1-f193.google.com [209.85.214.193]) by imf41.hostedemail.com (Postfix) with ESMTP for ; Tue, 8 Dec 2020 04:21:43 +0000 (UTC) Received: by mail-pl1-f193.google.com with SMTP id r4so2186149pls.11 for ; Mon, 07 Dec 2020 20:21:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=iuruSM5Wk5IQuXnGIYHAC1ej2c7Wmgf5EexDWUENUOw=; b=JvSBmHPkXQn9tqltsKxRwqdG3L4xMxx3IUPXwtPqKOYEyGp238OQgLQ7jDXPKt24e3 sYkRd9arNELgXCsNFausQ1G47Il3mclukxuUag5+wS39WymDZvYjin+7g0neRTqG8Y9n VO9rU6zNwtU41CReh9zDnH2Ljy1keWq+zuZwRvTfmgKg83qpHAMoB4/mXprbbYLETe6G RBRnybRoEHBts/t70AtrP2bQhcUwjOJ8oh3mfZrTGORG02pzBeqFJEBegTzngrCn72XV gPVe66zYzpbG9bk4zkDzACtObij3N/ESHJn/3qupwhqJAPbI9bj6K33pWLyewDQtt+LB wCXg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=iuruSM5Wk5IQuXnGIYHAC1ej2c7Wmgf5EexDWUENUOw=; b=F3nEK6pJey8ilVbxhuXdYaJaeUFMihjBOqhEmeTcFqbBP1ZwAR0pITOl3AW0m2hV29 PLJlAK6LFUI0BFcUkbmJOMUuiv5S6Vf5GyfCxv3vlo4d23B6pDWsZWjNxveb9ek83pVN JbYZYWCILIw/KSb5N1HzKnS9Dzeyt/loTHgc+O5gXW9svM4si6racOd4mPpzkXzFlrEG TCXI0uCHJkROD6OB2xdjp6OYqrOxVUU0BzfzS7L4vPo2ajV0B6g/Wcb8W++6vVn7Toy5 Sbw6VbnmOWglthSoN7BB1rhGGs0ArqQBOaVNLrKWeYwZOoOeXG2EeHEhMK/EGKJPatbw sncw== X-Gm-Message-State: AOAM532xEy3eh6ztlxKReSQwF6JjxC7Z7Wka9i9aH5MbvGa3ZNGNF7nM 6ji+cDkbVTW1c5wJZChVGvLv1w== X-Google-Smtp-Source: ABdhPJwCAN4zm3sxSIMbaeUGH8Q36ijCLpemWCrzTBw2fM2/G6fQIIQjIkxW+ToQpB618UjA8kaeLg== X-Received: by 2002:a17:902:6803:b029:d6:cf9d:2cfb with SMTP id h3-20020a1709026803b02900d6cf9d2cfbmr19526373plk.55.1607401302801; Mon, 07 Dec 2020 20:21:42 -0800 (PST) Received: from localhost.localdomain ([103.136.221.70]) by smtp.gmail.com with ESMTPSA id mr7sm1031166pjb.31.2020.12.07.20.21.35 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 07 Dec 2020 20:21:42 -0800 (PST) From: Muchun Song To: gregkh@linuxfoundation.org, rafael@kernel.org, adobriyan@gmail.com, akpm@linux-foundation.org, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, hughd@google.com, shakeelb@google.com, guro@fb.com, samitolvanen@google.com, feng.tang@intel.com, neilb@suse.de, iamjoonsoo.kim@lge.com, rdunlap@infradead.org Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, Muchun Song Subject: [PATCH v3 7/7] mm: memcontrol: make the slab calculation consistent Date: Tue, 8 Dec 2020 12:18:47 +0800 Message-Id: <20201208041847.72122-8-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20201208041847.72122-1-songmuchun@bytedance.com> References: <20201208041847.72122-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Although the ratio of the slab is one, we also should read the ratio from the related memory_stats instead of hard-coding. And the local variable of size is already the value of slab_unreclaimable. So we do not need to read again. We can drop the ratio in struct memory_stat. This can make the code clean and simple. And get rid of the awkward mix of static and runtime initialization of the memory_stats table. Signed-off-by: Muchun Song --- mm/memcontrol.c | 112 ++++++++++++++++++++++++++++++++++++-------------------- 1 file changed, 73 insertions(+), 39 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index a40797a27f87..841ea37cc123 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -1511,49 +1511,78 @@ static bool mem_cgroup_wait_acct_move(struct mem_cgroup *memcg) struct memory_stat { const char *name; - unsigned int ratio; unsigned int idx; }; static const struct memory_stat memory_stats[] = { - { "anon", PAGE_SIZE, NR_ANON_MAPPED }, - { "file", PAGE_SIZE, NR_FILE_PAGES }, - { "kernel_stack", 1024, NR_KERNEL_STACK_KB }, - { "pagetables", PAGE_SIZE, NR_PAGETABLE }, - { "percpu", 1, MEMCG_PERCPU_B }, - { "sock", PAGE_SIZE, MEMCG_SOCK }, - { "shmem", PAGE_SIZE, NR_SHMEM }, - { "file_mapped", PAGE_SIZE, NR_FILE_MAPPED }, - { "file_dirty", PAGE_SIZE, NR_FILE_DIRTY }, - { "file_writeback", PAGE_SIZE, NR_WRITEBACK }, + { "anon", NR_ANON_MAPPED }, + { "file", NR_FILE_PAGES }, + { "kernel_stack", NR_KERNEL_STACK_KB }, + { "pagetables", NR_PAGETABLE }, + { "percpu", MEMCG_PERCPU_B }, + { "sock", MEMCG_SOCK }, + { "shmem", NR_SHMEM }, + { "file_mapped", NR_FILE_MAPPED }, + { "file_dirty", NR_FILE_DIRTY }, + { "file_writeback", NR_WRITEBACK }, #ifdef CONFIG_TRANSPARENT_HUGEPAGE - { "anon_thp", PAGE_SIZE, NR_ANON_THPS }, - { "file_thp", PAGE_SIZE, NR_FILE_THPS }, - { "shmem_thp", PAGE_SIZE, NR_SHMEM_THPS }, + { "anon_thp", NR_ANON_THPS }, + { "file_thp", NR_FILE_THPS }, + { "shmem_thp", NR_SHMEM_THPS }, #endif - { "inactive_anon", PAGE_SIZE, NR_INACTIVE_ANON }, - { "active_anon", PAGE_SIZE, NR_ACTIVE_ANON }, - { "inactive_file", PAGE_SIZE, NR_INACTIVE_FILE }, - { "active_file", PAGE_SIZE, NR_ACTIVE_FILE }, - { "unevictable", PAGE_SIZE, NR_UNEVICTABLE }, - - /* - * Note: The slab_reclaimable and slab_unreclaimable must be - * together and slab_reclaimable must be in front. - */ - { "slab_reclaimable", 1, NR_SLAB_RECLAIMABLE_B }, - { "slab_unreclaimable", 1, NR_SLAB_UNRECLAIMABLE_B }, + { "inactive_anon", NR_INACTIVE_ANON }, + { "active_anon", NR_ACTIVE_ANON }, + { "inactive_file", NR_INACTIVE_FILE }, + { "active_file", NR_ACTIVE_FILE }, + { "unevictable", NR_UNEVICTABLE }, + { "slab_reclaimable", NR_SLAB_RECLAIMABLE_B }, + { "slab_unreclaimable", NR_SLAB_UNRECLAIMABLE_B }, /* The memory events */ - { "workingset_refault_anon", 1, WORKINGSET_REFAULT_ANON }, - { "workingset_refault_file", 1, WORKINGSET_REFAULT_FILE }, - { "workingset_activate_anon", 1, WORKINGSET_ACTIVATE_ANON }, - { "workingset_activate_file", 1, WORKINGSET_ACTIVATE_FILE }, - { "workingset_restore_anon", 1, WORKINGSET_RESTORE_ANON }, - { "workingset_restore_file", 1, WORKINGSET_RESTORE_FILE }, - { "workingset_nodereclaim", 1, WORKINGSET_NODERECLAIM }, + { "workingset_refault_anon", WORKINGSET_REFAULT_ANON }, + { "workingset_refault_file", WORKINGSET_REFAULT_FILE }, + { "workingset_activate_anon", WORKINGSET_ACTIVATE_ANON }, + { "workingset_activate_file", WORKINGSET_ACTIVATE_FILE }, + { "workingset_restore_anon", WORKINGSET_RESTORE_ANON }, + { "workingset_restore_file", WORKINGSET_RESTORE_FILE }, + { "workingset_nodereclaim", WORKINGSET_NODERECLAIM }, }; +/* Translate stat items to the correct unit for memory.stat output */ +static int memcg_page_state_unit(int item) +{ + int unit; + + switch (item) { + case MEMCG_PERCPU_B: + case NR_SLAB_RECLAIMABLE_B: + case NR_SLAB_UNRECLAIMABLE_B: + case WORKINGSET_REFAULT_ANON: + case WORKINGSET_REFAULT_FILE: + case WORKINGSET_ACTIVATE_ANON: + case WORKINGSET_ACTIVATE_FILE: + case WORKINGSET_RESTORE_ANON: + case WORKINGSET_RESTORE_FILE: + case WORKINGSET_NODERECLAIM: + unit = 1; + break; + case NR_KERNEL_STACK_KB: + unit = SZ_1K; + break; + default: + unit = PAGE_SIZE; + break; + } + + return unit; +} + +static inline unsigned long memcg_page_state_output(struct mem_cgroup *memcg, + int item) +{ + return memcg_page_state(memcg, item) * memcg_page_state_unit(item); +} + static char *memory_stat_format(struct mem_cgroup *memcg) { struct seq_buf s; @@ -1577,13 +1606,12 @@ static char *memory_stat_format(struct mem_cgroup *memcg) for (i = 0; i < ARRAY_SIZE(memory_stats); i++) { u64 size; - size = memcg_page_state(memcg, memory_stats[i].idx); - size *= memory_stats[i].ratio; + size = memcg_page_state_output(memcg, memory_stats[i].idx); seq_buf_printf(&s, "%s %llu\n", memory_stats[i].name, size); if (unlikely(memory_stats[i].idx == NR_SLAB_UNRECLAIMABLE_B)) { - size = memcg_page_state(memcg, NR_SLAB_RECLAIMABLE_B) + - memcg_page_state(memcg, NR_SLAB_UNRECLAIMABLE_B); + size += memcg_page_state_output(memcg, + NR_SLAB_RECLAIMABLE_B); seq_buf_printf(&s, "slab %llu\n", size); } } @@ -6377,6 +6405,12 @@ static int memory_stat_show(struct seq_file *m, void *v) } #ifdef CONFIG_NUMA +static inline unsigned long lruvec_page_state_output(struct lruvec *lruvec, + int item) +{ + return lruvec_page_state(lruvec, item) * memcg_page_state_unit(item); +} + static int memory_numa_stat_show(struct seq_file *m, void *v) { int i; @@ -6394,8 +6428,8 @@ static int memory_numa_stat_show(struct seq_file *m, void *v) struct lruvec *lruvec; lruvec = mem_cgroup_lruvec(memcg, NODE_DATA(nid)); - size = lruvec_page_state(lruvec, memory_stats[i].idx); - size *= memory_stats[i].ratio; + size = lruvec_page_state_output(lruvec, + memory_stats[i].idx); seq_printf(m, " N%d=%llu", nid, size); } seq_putc(m, '\n');