From patchwork Sun Dec 6 10:14:40 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 11953885 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 083D6C4361B for ; Sun, 6 Dec 2020 10:17:03 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 6B9BF22D2C for ; Sun, 6 Dec 2020 10:17:02 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6B9BF22D2C Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 0BCCF6B0068; Sun, 6 Dec 2020 05:17:02 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 090348D0002; Sun, 6 Dec 2020 05:17:02 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EC1E26B0071; Sun, 6 Dec 2020 05:17:01 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0156.hostedemail.com [216.40.44.156]) by kanga.kvack.org (Postfix) with ESMTP id D53FD6B0068 for ; Sun, 6 Dec 2020 05:17:01 -0500 (EST) Received: from smtpin21.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 97610181AEF10 for ; Sun, 6 Dec 2020 10:17:01 +0000 (UTC) X-FDA: 77562454242.21.skin75_381359d273d4 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin21.hostedemail.com (Postfix) with ESMTP id 7405F180442C2 for ; Sun, 6 Dec 2020 10:17:01 +0000 (UTC) X-HE-Tag: skin75_381359d273d4 X-Filterd-Recvd-Size: 4471 Received: from mail-pf1-f193.google.com (mail-pf1-f193.google.com [209.85.210.193]) by imf38.hostedemail.com (Postfix) with ESMTP for ; Sun, 6 Dec 2020 10:17:00 +0000 (UTC) Received: by mail-pf1-f193.google.com with SMTP id s21so6998415pfu.13 for ; Sun, 06 Dec 2020 02:17:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=JLGZC7sv5nAUpuhCShqzZAhxLbvYoPKLszNPrb+RhvY=; b=sDQL9F0PUFel7IRE81/3eSpUKyMdwDgTZi3UBN+42b+bU2PNE/t3ymmlZSkl6G0zUE NSCG35LdEVF++dP/WfDe2t2uqtwSFpsGuaN57CFdrOLHb4WjfZ3jys2v2GYv1slCoxpV lHGI0MzqMCRhrpIXGMhucScdPb77XwS1gVQ8DA3u+zB/gPDofVk7OR0MNcSQkNxlPfaJ 1RyynIXs88y3N+8Tuq6M1bZWNQoh5e5Y9f2FauDg2E2dRqtshDAZubBtA0H7xFytfcsr bP6zs0AWlZzZkTE8tZ0EnxPaZLjkWuzOMlhnw7qO6H+KSceypFyZdtq4GtFb8oTd85Yc RNxQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=JLGZC7sv5nAUpuhCShqzZAhxLbvYoPKLszNPrb+RhvY=; b=HAo00RHZaH+X8+4BDi0AlqosBuRS0pOPIfdVLec4zLWFtI1Iw9U5VY7GyhmuVlZUhr Y1V+gX55f9znzWKoZw3muQZN/oGRt4D8NX0Ot9FjuBYnH/MOMur/bYNMtvaIJLhdHOw+ KOz2bgtlRq0Yl/xEd/XvdMZYg2L9ie/WbTgrWIpH8mtAdjZOMxqEtY+BTAYtPvnRRJIt +iz2mjZseFGGbyTY+U/Quyf9h/uKo5bOwv9Qw24rduCXW40edYa/jxkewa0WuuwYXTz/ RF5gRxUUVurZWWkslTDilsYcaSEQY0HRDO9WlCVkKEgQayycj1fjD5FlRhbxn+BIeOGs h4kg== X-Gm-Message-State: AOAM531KYGFLBf1P/yLlMMN5YESUCSndSdj2it6J2P8SiqpupJvhZR9B geYPGMa7shRT+PBRwTmnC4t+8w== X-Google-Smtp-Source: ABdhPJwrEym39f3aZ6PyYuujbHaEjIbhRNOiqu9p7TRph3J6zvXaoYG5vwozGaIuyksCOLbrQRARnQ== X-Received: by 2002:a62:88c3:0:b029:18c:3203:efb7 with SMTP id l186-20020a6288c30000b029018c3203efb7mr11447624pfd.33.1607249819828; Sun, 06 Dec 2020 02:16:59 -0800 (PST) Received: from localhost.localdomain ([103.136.221.70]) by smtp.gmail.com with ESMTPSA id g16sm10337657pfb.201.2020.12.06.02.16.51 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 06 Dec 2020 02:16:59 -0800 (PST) From: Muchun Song To: gregkh@linuxfoundation.org, rafael@kernel.org, adobriyan@gmail.com, akpm@linux-foundation.org, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, hughd@google.com, will@kernel.org, guro@fb.com, rppt@kernel.org, tglx@linutronix.de, esyr@redhat.com, peterx@redhat.com, krisman@collabora.com, surenb@google.com, avagin@openvz.org, elver@google.com, rdunlap@infradead.org, iamjoonsoo.kim@lge.com Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, Muchun Song Subject: [RESEND PATCH v2 01/12] mm: memcontrol: fix NR_ANON_THPS account Date: Sun, 6 Dec 2020 18:14:40 +0800 Message-Id: <20201206101451.14706-2-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20201206101451.14706-1-songmuchun@bytedance.com> References: <20201206101451.14706-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The unit of NR_ANON_THPS is HPAGE_PMD_NR already. So it should inc/dec by one rather than nr_pages. Fixes: 468c398233da ("mm: memcontrol: switch to native NR_ANON_THPS counter") Signed-off-by: Muchun Song Acked-by: Michal Hocko Acked-by: Johannes Weiner --- mm/memcontrol.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 22d9bd688d6d..695dedf8687a 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -5634,10 +5634,8 @@ static int mem_cgroup_move_account(struct page *page, __mod_lruvec_state(from_vec, NR_ANON_MAPPED, -nr_pages); __mod_lruvec_state(to_vec, NR_ANON_MAPPED, nr_pages); if (PageTransHuge(page)) { - __mod_lruvec_state(from_vec, NR_ANON_THPS, - -nr_pages); - __mod_lruvec_state(to_vec, NR_ANON_THPS, - nr_pages); + __dec_lruvec_state(from_vec, NR_ANON_THPS); + __inc_lruvec_state(to_vec, NR_ANON_THPS); } } From patchwork Sun Dec 6 10:14:41 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 11953887 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9FEA0C4361B for ; Sun, 6 Dec 2020 10:17:11 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 2F283230FA for ; Sun, 6 Dec 2020 10:17:11 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2F283230FA Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id A549A8D0002; Sun, 6 Dec 2020 05:17:10 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id A05696B0071; Sun, 6 Dec 2020 05:17:10 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8F5FC8D0002; Sun, 6 Dec 2020 05:17:10 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0136.hostedemail.com [216.40.44.136]) by kanga.kvack.org (Postfix) with ESMTP id 78DAD6B0070 for ; Sun, 6 Dec 2020 05:17:10 -0500 (EST) Received: from smtpin11.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 2EA248249980 for ; Sun, 6 Dec 2020 10:17:10 +0000 (UTC) X-FDA: 77562454620.11.mind70_190d434273d4 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin11.hostedemail.com (Postfix) with ESMTP id 0641E180F8B80 for ; Sun, 6 Dec 2020 10:17:10 +0000 (UTC) X-HE-Tag: mind70_190d434273d4 X-Filterd-Recvd-Size: 9534 Received: from mail-pl1-f196.google.com (mail-pl1-f196.google.com [209.85.214.196]) by imf37.hostedemail.com (Postfix) with ESMTP for ; Sun, 6 Dec 2020 10:17:09 +0000 (UTC) Received: by mail-pl1-f196.google.com with SMTP id v3so5570267plz.13 for ; Sun, 06 Dec 2020 02:17:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=tb1F00/nSlox2NKAPVic2CylQOjgm2ke+JSF79slOJI=; b=uAhtAkvDbhKkFqddHAKBOhza/mhkmYbQ1gO6u++TYaJcsXVMyP3DxJmUjnpMeUG3SW XaMHSAVvWel0CQp+XAMCU1fy5kVW4g/nQk1Ljm3WTrKzYFBut8uMBDb529RBGkfk+A2v M/rf2Vw6BzysfA+PgBb0lHWSxh7zZIVcUHPgsevcH+FXhSayuBVPKaI2/sRJUjVwZ3sh mDVTvJagBELE8ENh8vCH8AgbeDky1F6PQf5oWry/i1XGz24pO3nc2eO7pWTPkntDwSlm P1bSiP4lHuYOVRc4yTGX5e4058PUKvlIZS8LbLvOAgCUcIs6oHfH+7dSCl4EgNqeLuj7 eYVQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=tb1F00/nSlox2NKAPVic2CylQOjgm2ke+JSF79slOJI=; b=ToykJ6R9VV1ykAjxgY3eaTZ3TcMpjr9cPem9NKWAMuT6W/dD7xgnfSd0gLbfBtnWb7 eTHrI0ekC770PZy6pVsWVMJ7/wjTmPwSqLpb1Fg77juBQMY+i8iqbfa81yS3TEAod5mR JvdbWjF3+vqviVj5suyZLzW/eU5D54SsiqefJrdozJl3nz7P8OcyJlFvZjCAPfaZeBYu RlNSCobyGEoedoZyYHscnPZEzypOoJkW2iTKjugK+NKqD/9/70StsPvCUjM149zk9KcV 6IiZDwZXyv67p4TbW/HqfuzLmD2F/LgSal13LLWQSIlSH4TJT9AIL/5mvecvSu4Ddjkh 59QA== X-Gm-Message-State: AOAM533sTVv3FZ090X+Q/JpZomiL30Z7wN8ZdBGe70/M/85b4btJmRQV XnewL8F7cPkNbjyJVW8Vqu4pZw== X-Google-Smtp-Source: ABdhPJzEBo93k11SoyRC/DlZb7e56oJgos7Jvh1hQZakYwMm4rGixAiJhZdQfEqnglOZ1ymwlJS7CA== X-Received: by 2002:a17:90a:bd0b:: with SMTP id y11mr8078590pjr.25.1607249828160; Sun, 06 Dec 2020 02:17:08 -0800 (PST) Received: from localhost.localdomain ([103.136.221.70]) by smtp.gmail.com with ESMTPSA id g16sm10337657pfb.201.2020.12.06.02.17.00 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 06 Dec 2020 02:17:07 -0800 (PST) From: Muchun Song To: gregkh@linuxfoundation.org, rafael@kernel.org, adobriyan@gmail.com, akpm@linux-foundation.org, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, hughd@google.com, will@kernel.org, guro@fb.com, rppt@kernel.org, tglx@linutronix.de, esyr@redhat.com, peterx@redhat.com, krisman@collabora.com, surenb@google.com, avagin@openvz.org, elver@google.com, rdunlap@infradead.org, iamjoonsoo.kim@lge.com Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, Muchun Song Subject: [RESEND PATCH v2 02/12] mm: memcontrol: convert NR_ANON_THPS account to pages Date: Sun, 6 Dec 2020 18:14:41 +0800 Message-Id: <20201206101451.14706-3-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20201206101451.14706-1-songmuchun@bytedance.com> References: <20201206101451.14706-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Convert the NR_ANON_THPS account to pages. Signed-off-by: Muchun Song --- drivers/base/node.c | 3 +-- fs/proc/meminfo.c | 2 +- mm/huge_memory.c | 3 ++- mm/memcontrol.c | 20 ++++++-------------- mm/page_alloc.c | 2 +- mm/rmap.c | 7 ++++--- 6 files changed, 15 insertions(+), 22 deletions(-) diff --git a/drivers/base/node.c b/drivers/base/node.c index 6ffa470e2984..7ebe4c2f64d1 100644 --- a/drivers/base/node.c +++ b/drivers/base/node.c @@ -461,8 +461,7 @@ static ssize_t node_read_meminfo(struct device *dev, nid, K(sunreclaimable) #ifdef CONFIG_TRANSPARENT_HUGEPAGE , - nid, K(node_page_state(pgdat, NR_ANON_THPS) * - HPAGE_PMD_NR), + nid, K(node_page_state(pgdat, NR_ANON_THPS)), nid, K(node_page_state(pgdat, NR_SHMEM_THPS) * HPAGE_PMD_NR), nid, K(node_page_state(pgdat, NR_SHMEM_PMDMAPPED) * diff --git a/fs/proc/meminfo.c b/fs/proc/meminfo.c index 887a5532e449..1f7e1945c313 100644 --- a/fs/proc/meminfo.c +++ b/fs/proc/meminfo.c @@ -129,7 +129,7 @@ static int meminfo_proc_show(struct seq_file *m, void *v) #ifdef CONFIG_TRANSPARENT_HUGEPAGE show_val_kb(m, "AnonHugePages: ", - global_node_page_state(NR_ANON_THPS) * HPAGE_PMD_NR); + global_node_page_state(NR_ANON_THPS)); show_val_kb(m, "ShmemHugePages: ", global_node_page_state(NR_SHMEM_THPS) * HPAGE_PMD_NR); show_val_kb(m, "ShmemPmdMapped: ", diff --git a/mm/huge_memory.c b/mm/huge_memory.c index c94dfc0cc1a3..1a13e1dab3ec 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -2178,7 +2178,8 @@ static void __split_huge_pmd_locked(struct vm_area_struct *vma, pmd_t *pmd, lock_page_memcg(page); if (atomic_add_negative(-1, compound_mapcount_ptr(page))) { /* Last compound_mapcount is gone. */ - __dec_lruvec_page_state(page, NR_ANON_THPS); + __mod_lruvec_page_state(page, NR_ANON_THPS, + -HPAGE_PMD_NR); if (TestClearPageDoubleMap(page)) { /* No need in mapcount reference anymore */ for (i = 0; i < HPAGE_PMD_NR; i++) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 695dedf8687a..2700c1db5a1a 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -1513,7 +1513,7 @@ static struct memory_stat memory_stats[] = { * on some architectures, the macro of HPAGE_PMD_SIZE is not * constant(e.g. powerpc). */ - { "anon_thp", 0, NR_ANON_THPS }, + { "anon_thp", PAGE_SIZE, NR_ANON_THPS }, { "file_thp", 0, NR_FILE_THPS }, { "shmem_thp", 0, NR_SHMEM_THPS }, #endif @@ -1546,8 +1546,7 @@ static int __init memory_stats_init(void) for (i = 0; i < ARRAY_SIZE(memory_stats); i++) { #ifdef CONFIG_TRANSPARENT_HUGEPAGE - if (memory_stats[i].idx == NR_ANON_THPS || - memory_stats[i].idx == NR_FILE_THPS || + if (memory_stats[i].idx == NR_FILE_THPS || memory_stats[i].idx == NR_SHMEM_THPS) memory_stats[i].ratio = HPAGE_PMD_SIZE; #endif @@ -4069,10 +4068,6 @@ static int memcg_stat_show(struct seq_file *m, void *v) if (memcg1_stats[i] == MEMCG_SWAP && !do_memsw_account()) continue; nr = memcg_page_state_local(memcg, memcg1_stats[i]); -#ifdef CONFIG_TRANSPARENT_HUGEPAGE - if (memcg1_stats[i] == NR_ANON_THPS) - nr *= HPAGE_PMD_NR; -#endif seq_printf(m, "%s %lu\n", memcg1_stat_names[i], nr * PAGE_SIZE); } @@ -4103,10 +4098,6 @@ static int memcg_stat_show(struct seq_file *m, void *v) if (memcg1_stats[i] == MEMCG_SWAP && !do_memsw_account()) continue; nr = memcg_page_state(memcg, memcg1_stats[i]); -#ifdef CONFIG_TRANSPARENT_HUGEPAGE - if (memcg1_stats[i] == NR_ANON_THPS) - nr *= HPAGE_PMD_NR; -#endif seq_printf(m, "total_%s %llu\n", memcg1_stat_names[i], (u64)nr * PAGE_SIZE); } @@ -5634,10 +5625,11 @@ static int mem_cgroup_move_account(struct page *page, __mod_lruvec_state(from_vec, NR_ANON_MAPPED, -nr_pages); __mod_lruvec_state(to_vec, NR_ANON_MAPPED, nr_pages); if (PageTransHuge(page)) { - __dec_lruvec_state(from_vec, NR_ANON_THPS); - __inc_lruvec_state(to_vec, NR_ANON_THPS); + __mod_lruvec_state(from_vec, NR_ANON_THPS, + -nr_pages); + __mod_lruvec_state(to_vec, NR_ANON_THPS, + nr_pages); } - } } else { __mod_lruvec_state(from_vec, NR_FILE_PAGES, -nr_pages); diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 56e603eea1dd..f97ca98d361f 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -5570,7 +5570,7 @@ void show_free_areas(unsigned int filter, nodemask_t *nodemask) K(node_page_state(pgdat, NR_SHMEM_THPS) * HPAGE_PMD_NR), K(node_page_state(pgdat, NR_SHMEM_PMDMAPPED) * HPAGE_PMD_NR), - K(node_page_state(pgdat, NR_ANON_THPS) * HPAGE_PMD_NR), + K(node_page_state(pgdat, NR_ANON_THPS)), #endif K(node_page_state(pgdat, NR_WRITEBACK_TEMP)), node_page_state(pgdat, NR_KERNEL_STACK_KB), diff --git a/mm/rmap.c b/mm/rmap.c index 08c56aaf72eb..f59e92e26b61 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -1144,7 +1144,8 @@ void do_page_add_anon_rmap(struct page *page, * disabled. */ if (compound) - __inc_lruvec_page_state(page, NR_ANON_THPS); + __mod_lruvec_page_state(page, NR_ANON_THPS, + HPAGE_PMD_NR); __mod_lruvec_page_state(page, NR_ANON_MAPPED, nr); } @@ -1186,7 +1187,7 @@ void page_add_new_anon_rmap(struct page *page, if (hpage_pincount_available(page)) atomic_set(compound_pincount_ptr(page), 0); - __inc_lruvec_page_state(page, NR_ANON_THPS); + __mod_lruvec_page_state(page, NR_ANON_THPS, HPAGE_PMD_NR); } else { /* Anon THP always mapped first with PMD */ VM_BUG_ON_PAGE(PageTransCompound(page), page); @@ -1292,7 +1293,7 @@ static void page_remove_anon_compound_rmap(struct page *page) if (!IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE)) return; - __dec_lruvec_page_state(page, NR_ANON_THPS); + __mod_lruvec_page_state(page, NR_ANON_THPS, -HPAGE_PMD_NR); if (TestClearPageDoubleMap(page)) { /* From patchwork Sun Dec 6 10:14:42 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 11953889 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D95DBC4361B for ; Sun, 6 Dec 2020 10:17:19 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 4C6BA22D2C for ; Sun, 6 Dec 2020 10:17:19 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4C6BA22D2C Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id E27F78D0003; Sun, 6 Dec 2020 05:17:18 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id DD55E6B0071; Sun, 6 Dec 2020 05:17:18 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CA05D8D0003; Sun, 6 Dec 2020 05:17:18 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0177.hostedemail.com [216.40.44.177]) by kanga.kvack.org (Postfix) with ESMTP id B3A476B0070 for ; Sun, 6 Dec 2020 05:17:18 -0500 (EST) Received: from smtpin28.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 7722C8249980 for ; Sun, 6 Dec 2020 10:17:18 +0000 (UTC) X-FDA: 77562454956.28.ants22_2817021273d4 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin28.hostedemail.com (Postfix) with ESMTP id 559BF6D70 for ; Sun, 6 Dec 2020 10:17:18 +0000 (UTC) X-HE-Tag: ants22_2817021273d4 X-Filterd-Recvd-Size: 7315 Received: from mail-pg1-f180.google.com (mail-pg1-f180.google.com [209.85.215.180]) by imf04.hostedemail.com (Postfix) with ESMTP for ; Sun, 6 Dec 2020 10:17:17 +0000 (UTC) Received: by mail-pg1-f180.google.com with SMTP id m9so6461382pgb.4 for ; Sun, 06 Dec 2020 02:17:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=i2WmQI9Q0soqzGFU1vD0NNfzSFXtCmp1bcFtyNd4sgI=; b=Ms1jKhSa+qgJDEXhLIQ4samWqeYhB2vhUcYFQ9kIBfeKOFDxvVzFk87TrixWrRD6Rr +vaXPRp3mQ8tdRqpuDAFtslhGGYCDSz9lAeIhR+7d3Au95mDEpciTi58EJCUbOV74boF irHf7UgHfqeYxqPOZMX694NrAsulQCs6XWfG4XayIwUmc6E7MAPpDhiT8R6HB7wMt/sK SNrLW1p5/PnRMUlJ+0fav+HcOTb1EAz0rIYHqUEVW3GpZpT1zdHfzbuDuyQrNuUcrO3L MmsEJi3uDyzEQa2sWlVjoK6VcSbmc9uIkM5UCU+YCzAwjoR5Wi2Lk1tuSgR+J6AkErEc KAhQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=i2WmQI9Q0soqzGFU1vD0NNfzSFXtCmp1bcFtyNd4sgI=; b=bBuL/kb/mlktPuOj+mZlWmz9gmslNCn/qxZBxFZvmFXRh+KEXYwBACzR9YH6c4qo/T YIkYKObN6Vps56tmrXHlBBhwVsxUrF9ahmlsEdkPSWzx4vg5qn/lRuUV6uK/4JP8ZNLZ sAqJeYvo5VhTPMj7hDKwolfEF1Di+Q6wnLX1HGkc/W984ECNXeiD3JbmLMdujKuMhWdT pJnMy7AaO/kTp6YWZRBtesuoF1ef0P1JdOOVITubTCf0E30bTwF9zlwZcrXqemIE1/aJ DZi9Lu7bXzjvKvfrX6j8mwk3ShYx1nP6OC+wxgye7cJ+xX5TVDoijLTedlGG8zC/lH4S rX5w== X-Gm-Message-State: AOAM531Z9d7YKw/QqJ7oAjGroe+cHHIZJNRT7kMJ1xQMhZ1cDi6XQWO/ Fd/QLyQVzIHO8S6zfomKny5ERQ== X-Google-Smtp-Source: ABdhPJxX4VV3xHN6Lrb6jwWVudCM8xvy7cP+M5Gcg95AHBIo8KyIJoYWtLsufEfRR9NNE3e7JelnGw== X-Received: by 2002:a62:8f03:0:b029:18c:968a:4cf9 with SMTP id n3-20020a628f030000b029018c968a4cf9mr11487716pfd.59.1607249836594; Sun, 06 Dec 2020 02:17:16 -0800 (PST) Received: from localhost.localdomain ([103.136.221.70]) by smtp.gmail.com with ESMTPSA id g16sm10337657pfb.201.2020.12.06.02.17.08 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 06 Dec 2020 02:17:16 -0800 (PST) From: Muchun Song To: gregkh@linuxfoundation.org, rafael@kernel.org, adobriyan@gmail.com, akpm@linux-foundation.org, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, hughd@google.com, will@kernel.org, guro@fb.com, rppt@kernel.org, tglx@linutronix.de, esyr@redhat.com, peterx@redhat.com, krisman@collabora.com, surenb@google.com, avagin@openvz.org, elver@google.com, rdunlap@infradead.org, iamjoonsoo.kim@lge.com Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, Muchun Song Subject: [RESEND PATCH v2 03/12] mm: memcontrol: convert NR_FILE_THPS account to pages Date: Sun, 6 Dec 2020 18:14:42 +0800 Message-Id: <20201206101451.14706-4-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20201206101451.14706-1-songmuchun@bytedance.com> References: <20201206101451.14706-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Converrt NR_FILE_THPS account to pages. Signed-off-by: Muchun Song --- drivers/base/node.c | 3 +-- fs/proc/meminfo.c | 2 +- mm/filemap.c | 2 +- mm/huge_memory.c | 3 ++- mm/khugepaged.c | 2 +- mm/memcontrol.c | 5 ++--- 6 files changed, 8 insertions(+), 9 deletions(-) diff --git a/drivers/base/node.c b/drivers/base/node.c index 7ebe4c2f64d1..2db28acdaa4f 100644 --- a/drivers/base/node.c +++ b/drivers/base/node.c @@ -466,8 +466,7 @@ static ssize_t node_read_meminfo(struct device *dev, HPAGE_PMD_NR), nid, K(node_page_state(pgdat, NR_SHMEM_PMDMAPPED) * HPAGE_PMD_NR), - nid, K(node_page_state(pgdat, NR_FILE_THPS) * - HPAGE_PMD_NR), + nid, K(node_page_state(pgdat, NR_FILE_THPS)), nid, K(node_page_state(pgdat, NR_FILE_PMDMAPPED) * HPAGE_PMD_NR) #endif diff --git a/fs/proc/meminfo.c b/fs/proc/meminfo.c index 1f7e1945c313..f4157f26cbf5 100644 --- a/fs/proc/meminfo.c +++ b/fs/proc/meminfo.c @@ -135,7 +135,7 @@ static int meminfo_proc_show(struct seq_file *m, void *v) show_val_kb(m, "ShmemPmdMapped: ", global_node_page_state(NR_SHMEM_PMDMAPPED) * HPAGE_PMD_NR); show_val_kb(m, "FileHugePages: ", - global_node_page_state(NR_FILE_THPS) * HPAGE_PMD_NR); + global_node_page_state(NR_FILE_THPS)); show_val_kb(m, "FilePmdMapped: ", global_node_page_state(NR_FILE_PMDMAPPED) * HPAGE_PMD_NR); #endif diff --git a/mm/filemap.c b/mm/filemap.c index 28e7309c29c8..c4dcb1144883 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -206,7 +206,7 @@ static void unaccount_page_cache_page(struct address_space *mapping, if (PageTransHuge(page)) __dec_lruvec_page_state(page, NR_SHMEM_THPS); } else if (PageTransHuge(page)) { - __dec_lruvec_page_state(page, NR_FILE_THPS); + __mod_lruvec_page_state(page, NR_FILE_THPS, -HPAGE_PMD_NR); filemap_nr_thps_dec(mapping); } diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 1a13e1dab3ec..37840bdeaad0 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -2748,7 +2748,8 @@ int split_huge_page_to_list(struct page *page, struct list_head *list) if (PageSwapBacked(head)) __dec_lruvec_page_state(head, NR_SHMEM_THPS); else - __dec_lruvec_page_state(head, NR_FILE_THPS); + __mod_lruvec_page_state(head, NR_FILE_THPS, + -HPAGE_PMD_NR); } __split_huge_page(page, list, end); diff --git a/mm/khugepaged.c b/mm/khugepaged.c index ad316d2e1fee..1e1ced2208d0 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1859,7 +1859,7 @@ static void collapse_file(struct mm_struct *mm, if (is_shmem) __inc_lruvec_page_state(new_page, NR_SHMEM_THPS); else { - __inc_lruvec_page_state(new_page, NR_FILE_THPS); + __mod_lruvec_page_state(new_page, NR_FILE_THPS, HPAGE_PMD_NR); filemap_nr_thps_inc(mapping); } diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 2700c1db5a1a..c4557de2b211 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -1514,7 +1514,7 @@ static struct memory_stat memory_stats[] = { * constant(e.g. powerpc). */ { "anon_thp", PAGE_SIZE, NR_ANON_THPS }, - { "file_thp", 0, NR_FILE_THPS }, + { "file_thp", PAGE_SIZE, NR_FILE_THPS }, { "shmem_thp", 0, NR_SHMEM_THPS }, #endif { "inactive_anon", PAGE_SIZE, NR_INACTIVE_ANON }, @@ -1546,8 +1546,7 @@ static int __init memory_stats_init(void) for (i = 0; i < ARRAY_SIZE(memory_stats); i++) { #ifdef CONFIG_TRANSPARENT_HUGEPAGE - if (memory_stats[i].idx == NR_FILE_THPS || - memory_stats[i].idx == NR_SHMEM_THPS) + if (memory_stats[i].idx == NR_SHMEM_THPS) memory_stats[i].ratio = HPAGE_PMD_SIZE; #endif VM_BUG_ON(!memory_stats[i].ratio); From patchwork Sun Dec 6 10:14:43 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 11953891 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.8 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,UNWANTED_LANGUAGE_BODY, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CD89AC433FE for ; Sun, 6 Dec 2020 10:17:27 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 7782722D2C for ; Sun, 6 Dec 2020 10:17:27 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7782722D2C Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 12E108D0005; Sun, 6 Dec 2020 05:17:27 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 08E7C6B0071; Sun, 6 Dec 2020 05:17:26 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E73D98D0005; Sun, 6 Dec 2020 05:17:26 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0129.hostedemail.com [216.40.44.129]) by kanga.kvack.org (Postfix) with ESMTP id CDDA06B0070 for ; Sun, 6 Dec 2020 05:17:26 -0500 (EST) Received: from smtpin10.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 9AD69362D for ; Sun, 6 Dec 2020 10:17:26 +0000 (UTC) X-FDA: 77562455292.10.burn78_260f0ac273d4 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin10.hostedemail.com (Postfix) with ESMTP id 8A11316A4B7 for ; Sun, 6 Dec 2020 10:17:26 +0000 (UTC) X-HE-Tag: burn78_260f0ac273d4 X-Filterd-Recvd-Size: 9536 Received: from mail-pl1-f169.google.com (mail-pl1-f169.google.com [209.85.214.169]) by imf39.hostedemail.com (Postfix) with ESMTP for ; Sun, 6 Dec 2020 10:17:25 +0000 (UTC) Received: by mail-pl1-f169.google.com with SMTP id j1so5598675pld.3 for ; Sun, 06 Dec 2020 02:17:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=jqJ1l3kUFANKBttrjy8eqm/Bfbx0CB9v1pkRqyyS8to=; b=KmA+ORfQc43rX3WMdlp11TQtIwpSJKpCxx5KoM4F25X7T30VLyaba67ZPkmP5cSM7c PKM6k/+aB9OVyO/8dQ+UJ86/av3LnTXnc1p/BRBzA0G00SzeU8PcqZ9Urmcf9zAXuAoZ xXzf2hK7TGTUPRpZVFDts8Ltd9Zsw1mZc60SLFI1uD+HV5B26N+YKp2clMDnKZYVQEBU AL7xPd2zVYSTvthcCID8/4MrAWZvFilD4o1eUb1vzOhLiYzYXGP8SjCSKOeJEG3OUwKx hyLQXCGOesN4uzI5XVZPYMZuoUNXP9kiHZSDYi9/0RQPtXc/uXCNVFSZxXt75fOvG6Ln Zo+g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=jqJ1l3kUFANKBttrjy8eqm/Bfbx0CB9v1pkRqyyS8to=; b=RUQc4ReV+OGdglzISkmqbez4xxUBeW5gw49j2uf1ju4XikSnpcZFflbG1CYzT3wx9a tDxk0nSccoBIUKTcIc+yPs/xVnFc5c+BMQTcnURW5Yyp3Eyb02VrKpHjftPnnEwEp+Rl nKZSTsw8XlU2RRr/CzTQSaptpoy1lPa92qe7nVEL/0xGdfAGUIsRftXxOdSLhcfmUIuo 9drCEMzYS7YoLc0tirOhHDw2mvO9V/UoeALfiPba+UiEOmYxBK7CxDRuA4dIfN55TOEG OUJZM5KLYHb36GZoYzryi+r+gn4MkUllbxjA5RBGkViQsIGuDHYIiCCQafFgZtzwCrPc 2H8w== X-Gm-Message-State: AOAM5338LsvcEV5OcaU/nU/LYB9yhpm5qktoCqFEj8UwMkF8TCwfS5j+ 12P8qKoWWKsibB30COqNFiyqQA== X-Google-Smtp-Source: ABdhPJyATYkKaw9y7vPdixMLM0rw9slgfozfwgOvQwnqmmhL7taHvYOu1U+qJ+3OrLbTIC6GDGWDUg== X-Received: by 2002:a17:902:7b97:b029:d8:ec6e:5c28 with SMTP id w23-20020a1709027b97b02900d8ec6e5c28mr11410388pll.40.1607249845181; Sun, 06 Dec 2020 02:17:25 -0800 (PST) Received: from localhost.localdomain ([103.136.221.70]) by smtp.gmail.com with ESMTPSA id g16sm10337657pfb.201.2020.12.06.02.17.17 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 06 Dec 2020 02:17:24 -0800 (PST) From: Muchun Song To: gregkh@linuxfoundation.org, rafael@kernel.org, adobriyan@gmail.com, akpm@linux-foundation.org, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, hughd@google.com, will@kernel.org, guro@fb.com, rppt@kernel.org, tglx@linutronix.de, esyr@redhat.com, peterx@redhat.com, krisman@collabora.com, surenb@google.com, avagin@openvz.org, elver@google.com, rdunlap@infradead.org, iamjoonsoo.kim@lge.com Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, Muchun Song Subject: [RESEND PATCH v2 04/12] mm: memcontrol: convert NR_SHMEM_THPS account to pages Date: Sun, 6 Dec 2020 18:14:43 +0800 Message-Id: <20201206101451.14706-5-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20201206101451.14706-1-songmuchun@bytedance.com> References: <20201206101451.14706-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Convert NR_SHMEM_THPS account to pages Signed-off-by: Muchun Song --- drivers/base/node.c | 3 +-- fs/proc/meminfo.c | 2 +- mm/filemap.c | 2 +- mm/huge_memory.c | 3 ++- mm/khugepaged.c | 2 +- mm/memcontrol.c | 26 ++------------------------ mm/page_alloc.c | 2 +- mm/shmem.c | 3 ++- 8 files changed, 11 insertions(+), 32 deletions(-) diff --git a/drivers/base/node.c b/drivers/base/node.c index 2db28acdaa4f..3e1094717e40 100644 --- a/drivers/base/node.c +++ b/drivers/base/node.c @@ -462,8 +462,7 @@ static ssize_t node_read_meminfo(struct device *dev, #ifdef CONFIG_TRANSPARENT_HUGEPAGE , nid, K(node_page_state(pgdat, NR_ANON_THPS)), - nid, K(node_page_state(pgdat, NR_SHMEM_THPS) * - HPAGE_PMD_NR), + nid, K(node_page_state(pgdat, NR_SHMEM_THPS)), nid, K(node_page_state(pgdat, NR_SHMEM_PMDMAPPED) * HPAGE_PMD_NR), nid, K(node_page_state(pgdat, NR_FILE_THPS)), diff --git a/fs/proc/meminfo.c b/fs/proc/meminfo.c index f4157f26cbf5..b4d8a6ee822d 100644 --- a/fs/proc/meminfo.c +++ b/fs/proc/meminfo.c @@ -131,7 +131,7 @@ static int meminfo_proc_show(struct seq_file *m, void *v) show_val_kb(m, "AnonHugePages: ", global_node_page_state(NR_ANON_THPS)); show_val_kb(m, "ShmemHugePages: ", - global_node_page_state(NR_SHMEM_THPS) * HPAGE_PMD_NR); + global_node_page_state(NR_SHMEM_THPS)); show_val_kb(m, "ShmemPmdMapped: ", global_node_page_state(NR_SHMEM_PMDMAPPED) * HPAGE_PMD_NR); show_val_kb(m, "FileHugePages: ", diff --git a/mm/filemap.c b/mm/filemap.c index c4dcb1144883..5fdefbbc1bc2 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -204,7 +204,7 @@ static void unaccount_page_cache_page(struct address_space *mapping, if (PageSwapBacked(page)) { __mod_lruvec_page_state(page, NR_SHMEM, -nr); if (PageTransHuge(page)) - __dec_lruvec_page_state(page, NR_SHMEM_THPS); + __mod_lruvec_page_state(page, NR_SHMEM_THPS, -HPAGE_PMD_NR); } else if (PageTransHuge(page)) { __mod_lruvec_page_state(page, NR_FILE_THPS, -HPAGE_PMD_NR); filemap_nr_thps_dec(mapping); diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 37840bdeaad0..0e8541bd9f50 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -2746,7 +2746,8 @@ int split_huge_page_to_list(struct page *page, struct list_head *list) spin_unlock(&ds_queue->split_queue_lock); if (mapping) { if (PageSwapBacked(head)) - __dec_lruvec_page_state(head, NR_SHMEM_THPS); + __mod_lruvec_page_state(head, NR_SHMEM_THPS, + -HPAGE_PMD_NR); else __mod_lruvec_page_state(head, NR_FILE_THPS, -HPAGE_PMD_NR); diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 1e1ced2208d0..4fe79ccfc312 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1857,7 +1857,7 @@ static void collapse_file(struct mm_struct *mm, } if (is_shmem) - __inc_lruvec_page_state(new_page, NR_SHMEM_THPS); + __mod_lruvec_page_state(new_page, NR_SHMEM_THPS, HPAGE_PMD_NR); else { __mod_lruvec_page_state(new_page, NR_FILE_THPS, HPAGE_PMD_NR); filemap_nr_thps_inc(mapping); diff --git a/mm/memcontrol.c b/mm/memcontrol.c index c4557de2b211..6d4365d2fd1c 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -1497,7 +1497,7 @@ struct memory_stat { unsigned int idx; }; -static struct memory_stat memory_stats[] = { +static const struct memory_stat memory_stats[] = { { "anon", PAGE_SIZE, NR_ANON_MAPPED }, { "file", PAGE_SIZE, NR_FILE_PAGES }, { "kernel_stack", 1024, NR_KERNEL_STACK_KB }, @@ -1508,14 +1508,9 @@ static struct memory_stat memory_stats[] = { { "file_dirty", PAGE_SIZE, NR_FILE_DIRTY }, { "file_writeback", PAGE_SIZE, NR_WRITEBACK }, #ifdef CONFIG_TRANSPARENT_HUGEPAGE - /* - * The ratio will be initialized in memory_stats_init(). Because - * on some architectures, the macro of HPAGE_PMD_SIZE is not - * constant(e.g. powerpc). - */ { "anon_thp", PAGE_SIZE, NR_ANON_THPS }, { "file_thp", PAGE_SIZE, NR_FILE_THPS }, - { "shmem_thp", 0, NR_SHMEM_THPS }, + { "shmem_thp", PAGE_SIZE, NR_SHMEM_THPS }, #endif { "inactive_anon", PAGE_SIZE, NR_INACTIVE_ANON }, { "active_anon", PAGE_SIZE, NR_ACTIVE_ANON }, @@ -1540,23 +1535,6 @@ static struct memory_stat memory_stats[] = { { "workingset_nodereclaim", 1, WORKINGSET_NODERECLAIM }, }; -static int __init memory_stats_init(void) -{ - int i; - - for (i = 0; i < ARRAY_SIZE(memory_stats); i++) { -#ifdef CONFIG_TRANSPARENT_HUGEPAGE - if (memory_stats[i].idx == NR_SHMEM_THPS) - memory_stats[i].ratio = HPAGE_PMD_SIZE; -#endif - VM_BUG_ON(!memory_stats[i].ratio); - VM_BUG_ON(memory_stats[i].idx >= MEMCG_NR_STAT); - } - - return 0; -} -pure_initcall(memory_stats_init); - static char *memory_stat_format(struct mem_cgroup *memcg) { struct seq_buf s; diff --git a/mm/page_alloc.c b/mm/page_alloc.c index f97ca98d361f..b6a79196e870 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -5567,7 +5567,7 @@ void show_free_areas(unsigned int filter, nodemask_t *nodemask) K(node_page_state(pgdat, NR_WRITEBACK)), K(node_page_state(pgdat, NR_SHMEM)), #ifdef CONFIG_TRANSPARENT_HUGEPAGE - K(node_page_state(pgdat, NR_SHMEM_THPS) * HPAGE_PMD_NR), + K(node_page_state(pgdat, NR_SHMEM_THPS)), K(node_page_state(pgdat, NR_SHMEM_PMDMAPPED) * HPAGE_PMD_NR), K(node_page_state(pgdat, NR_ANON_THPS)), diff --git a/mm/shmem.c b/mm/shmem.c index 5da4f1a3e663..ea5d8c9ccb5b 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -713,7 +713,8 @@ static int shmem_add_to_page_cache(struct page *page, } if (PageTransHuge(page)) { count_vm_event(THP_FILE_ALLOC); - __inc_lruvec_page_state(page, NR_SHMEM_THPS); + __mod_lruvec_page_state(page, NR_SHMEM_THPS, + HPAGE_PMD_NR); } mapping->nrpages += nr; __mod_lruvec_page_state(page, NR_FILE_PAGES, nr); From patchwork Sun Dec 6 10:14:44 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 11953893 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 15F5EC433FE for ; Sun, 6 Dec 2020 10:17:37 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 94AC7230FA for ; Sun, 6 Dec 2020 10:17:36 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 94AC7230FA Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 39E5F6B0070; Sun, 6 Dec 2020 05:17:36 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 373B68D0007; Sun, 6 Dec 2020 05:17:36 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 23D9B8D0006; Sun, 6 Dec 2020 05:17:36 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0148.hostedemail.com [216.40.44.148]) by kanga.kvack.org (Postfix) with ESMTP id 093FE6B0070 for ; Sun, 6 Dec 2020 05:17:36 -0500 (EST) Received: from smtpin04.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id BA58B8249980 for ; Sun, 6 Dec 2020 10:17:35 +0000 (UTC) X-FDA: 77562455670.04.trip09_600046c273d4 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin04.hostedemail.com (Postfix) with ESMTP id 9DBA8800433D for ; Sun, 6 Dec 2020 10:17:35 +0000 (UTC) X-HE-Tag: trip09_600046c273d4 X-Filterd-Recvd-Size: 6489 Received: from mail-pj1-f51.google.com (mail-pj1-f51.google.com [209.85.216.51]) by imf40.hostedemail.com (Postfix) with ESMTP for ; Sun, 6 Dec 2020 10:17:35 +0000 (UTC) Received: by mail-pj1-f51.google.com with SMTP id o7so5743850pjj.2 for ; Sun, 06 Dec 2020 02:17:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=dMdFQuYVxvcZn8v/8mwYCe8GX7PJ+nj7kDaLzG21jFc=; b=IK7YsvdANv45P9bPp/+tcViV/5ZRgaIqVfZiGOFYl4AT+LDlornq1SJTCzpg34ziSS alDGdrMAfAtUerS0JIvbwSFBVnTCSwpGj1vioyYSEm0l1ZNifTM47TmHD1Wm7jShBBsS JR8Yb3XlKjGUnBPCc2cDSgs3KJPPtBJ+g/cmzdAYqsZpBys3F3GY6iVjK/h2prjwkc3o qfMpVKOzzEXgZSpClDBkBsNQ7/43uK7PqHOx07K20kjKHeCvkA8i/49owagWlouNNsBT Sf5aTqmaU2G2M6httDGpi0ZLfknJoyCbIUHrh42l5rIAL+p0BHsDdrasQSp7AzO/kHoA E8mA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=dMdFQuYVxvcZn8v/8mwYCe8GX7PJ+nj7kDaLzG21jFc=; b=nhFjx9m+bemQyMjiJPTETYc/yzZynZ3cpg2vWJtS1iWOslpLeNKy7+R3e/fCu5GNQv Wh9SreSutnRdrHPDW95ea7AHPEdPKUT6SoQAPCg9fvvDRENOShaXP72BUK5Ab+jE+5Js /AQfqSLI1jzuW2l/x1ee/ti87bXbckQSRpuEMl30KJnqp3N4vuQB14ZaSpk3ExVVauz5 Eo13fuhEcSt4ArUOQAq30iUx1am9K7c0gHZGAPPo8H5HU+Xi/F5BXrGb5ICg7h94gHF+ HFMfGE6MK88LfKU4RfUBwbgzqAEhxtPg85YCEbbeGHs/N8h/vEES7YcvnkwuAf/0nHiA /xnQ== X-Gm-Message-State: AOAM530QH3FlgWksdWSUQlmGbs8iJ4pWFrinQ+X58sfjj9MRe5JBIbXw 0Ww0bTfd9ssVREDxsx5iBa8ziQ== X-Google-Smtp-Source: ABdhPJzYhUQEoJu2OmvjmllpIGf6IO82ScoCt1uZsQ3igOUoPB/moId0BftRZgOTLUmgdDW/V8B93A== X-Received: by 2002:a17:90b:1987:: with SMTP id mv7mr11947217pjb.66.1607249854013; Sun, 06 Dec 2020 02:17:34 -0800 (PST) Received: from localhost.localdomain ([103.136.221.70]) by smtp.gmail.com with ESMTPSA id g16sm10337657pfb.201.2020.12.06.02.17.25 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 06 Dec 2020 02:17:33 -0800 (PST) From: Muchun Song To: gregkh@linuxfoundation.org, rafael@kernel.org, adobriyan@gmail.com, akpm@linux-foundation.org, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, hughd@google.com, will@kernel.org, guro@fb.com, rppt@kernel.org, tglx@linutronix.de, esyr@redhat.com, peterx@redhat.com, krisman@collabora.com, surenb@google.com, avagin@openvz.org, elver@google.com, rdunlap@infradead.org, iamjoonsoo.kim@lge.com Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, Muchun Song Subject: [RESEND PATCH v2 05/12] mm: memcontrol: convert NR_SHMEM_PMDMAPPED account to pages Date: Sun, 6 Dec 2020 18:14:44 +0800 Message-Id: <20201206101451.14706-6-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20201206101451.14706-1-songmuchun@bytedance.com> References: <20201206101451.14706-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Convert NR_SHMEM_PMDMAPPED account to pages. Signed-off-by: Muchun Song --- drivers/base/node.c | 3 +-- fs/proc/meminfo.c | 2 +- mm/page_alloc.c | 3 +-- mm/rmap.c | 6 ++++-- 4 files changed, 7 insertions(+), 7 deletions(-) diff --git a/drivers/base/node.c b/drivers/base/node.c index 3e1094717e40..e5abc6144dab 100644 --- a/drivers/base/node.c +++ b/drivers/base/node.c @@ -463,8 +463,7 @@ static ssize_t node_read_meminfo(struct device *dev, , nid, K(node_page_state(pgdat, NR_ANON_THPS)), nid, K(node_page_state(pgdat, NR_SHMEM_THPS)), - nid, K(node_page_state(pgdat, NR_SHMEM_PMDMAPPED) * - HPAGE_PMD_NR), + nid, K(node_page_state(pgdat, NR_SHMEM_PMDMAPPED)), nid, K(node_page_state(pgdat, NR_FILE_THPS)), nid, K(node_page_state(pgdat, NR_FILE_PMDMAPPED) * HPAGE_PMD_NR) diff --git a/fs/proc/meminfo.c b/fs/proc/meminfo.c index b4d8a6ee822d..84886b2cc2f7 100644 --- a/fs/proc/meminfo.c +++ b/fs/proc/meminfo.c @@ -133,7 +133,7 @@ static int meminfo_proc_show(struct seq_file *m, void *v) show_val_kb(m, "ShmemHugePages: ", global_node_page_state(NR_SHMEM_THPS)); show_val_kb(m, "ShmemPmdMapped: ", - global_node_page_state(NR_SHMEM_PMDMAPPED) * HPAGE_PMD_NR); + global_node_page_state(NR_SHMEM_PMDMAPPED)); show_val_kb(m, "FileHugePages: ", global_node_page_state(NR_FILE_THPS)); show_val_kb(m, "FilePmdMapped: ", diff --git a/mm/page_alloc.c b/mm/page_alloc.c index b6a79196e870..d103513b3e4f 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -5568,8 +5568,7 @@ void show_free_areas(unsigned int filter, nodemask_t *nodemask) K(node_page_state(pgdat, NR_SHMEM)), #ifdef CONFIG_TRANSPARENT_HUGEPAGE K(node_page_state(pgdat, NR_SHMEM_THPS)), - K(node_page_state(pgdat, NR_SHMEM_PMDMAPPED) - * HPAGE_PMD_NR), + K(node_page_state(pgdat, NR_SHMEM_PMDMAPPED)), K(node_page_state(pgdat, NR_ANON_THPS)), #endif K(node_page_state(pgdat, NR_WRITEBACK_TEMP)), diff --git a/mm/rmap.c b/mm/rmap.c index f59e92e26b61..3089ad6bf468 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -1219,7 +1219,8 @@ void page_add_file_rmap(struct page *page, bool compound) if (!atomic_inc_and_test(compound_mapcount_ptr(page))) goto out; if (PageSwapBacked(page)) - __inc_node_page_state(page, NR_SHMEM_PMDMAPPED); + __mod_lruvec_page_state(page, NR_SHMEM_PMDMAPPED, + HPAGE_PMD_NR); else __inc_node_page_state(page, NR_FILE_PMDMAPPED); } else { @@ -1260,7 +1261,8 @@ static void page_remove_file_rmap(struct page *page, bool compound) if (!atomic_add_negative(-1, compound_mapcount_ptr(page))) return; if (PageSwapBacked(page)) - __dec_node_page_state(page, NR_SHMEM_PMDMAPPED); + __mod_lruvec_page_state(page, NR_SHMEM_PMDMAPPED, + -HPAGE_PMD_NR); else __dec_node_page_state(page, NR_FILE_PMDMAPPED); } else { From patchwork Sun Dec 6 10:14:45 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 11953895 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A8BB6C4361B for ; Sun, 6 Dec 2020 10:17:45 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 4C25B22D2C for ; Sun, 6 Dec 2020 10:17:45 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4C25B22D2C Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id DD5E16B0071; Sun, 6 Dec 2020 05:17:44 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D87498D0007; Sun, 6 Dec 2020 05:17:44 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C297A8D0006; Sun, 6 Dec 2020 05:17:44 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0086.hostedemail.com [216.40.44.86]) by kanga.kvack.org (Postfix) with ESMTP id ADF146B0071 for ; Sun, 6 Dec 2020 05:17:44 -0500 (EST) Received: from smtpin13.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 765251EE6 for ; Sun, 6 Dec 2020 10:17:44 +0000 (UTC) X-FDA: 77562456048.13.thing94_400c646273d4 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin13.hostedemail.com (Postfix) with ESMTP id 45E1718140B67 for ; Sun, 6 Dec 2020 10:17:44 +0000 (UTC) X-HE-Tag: thing94_400c646273d4 X-Filterd-Recvd-Size: 5813 Received: from mail-pf1-f173.google.com (mail-pf1-f173.google.com [209.85.210.173]) by imf39.hostedemail.com (Postfix) with ESMTP for ; Sun, 6 Dec 2020 10:17:43 +0000 (UTC) Received: by mail-pf1-f173.google.com with SMTP id f9so6414153pfc.11 for ; Sun, 06 Dec 2020 02:17:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=47URTm8Dh9aVQTOnIPzCjSnQ3r0uw/hpFPQHZdAjXJk=; b=ZFrm0RAsDKjEspMJB3v1V7ooekc7E7o5kl6qta47nxPfgFfA0L3kv9YJmSyWr+/B31 JHbHQopIMAu9OZDsv9OzAyQyEvdtpOUYOU4uChMVGIF7Rha4Kuyjtf6rvPmWRFq5r0Hb YpLTSeIofdez0J4pdps2gKdUCVEdB8v5HEpGQlyPdg9S6BnGVAvnNjCt9SvzcI6vyvP0 AnBBUWKm3mxAbbRDBv+EBHvtTprB++jbhIeX1HnMXf4B51kCpzqpciDO6VLGjCSqEfqj l7aae0EI4r2sW1m8S0lkLyNyXIeCmU8aRNz4NeRgVcg0KuTQ+ZB92ejjuYcxaPQNk/6O FZjQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=47URTm8Dh9aVQTOnIPzCjSnQ3r0uw/hpFPQHZdAjXJk=; b=PPJ+JSL4F6qx7Wt7H5NPvBcTmpWW/RJ0PpC+NJAlKxsxZq8c+Kfv4meGp3vakLUarA bph/B3ZOWhSzVQdQ87SjdBth8HUsrXUX5YpckZL5XGPhbGu+rRia+YVwr/1m6bzSbFEP IZ2AqTQvH5/cBNujqy8sMPoig39Ie0N9r6Lyi11Qiefn0hKRMl/RCJ0CFZYl0r0s95t2 e6nbV8ZS9XFlgOUwOzeHfLubMyO2ajk0S5IGvzaQESxXSVOCnZMDto414JyCTE2wQ3Zc PNedGBYMPj5K0idjZrsGwyDRSjIcZxhONp4Q4XfZyXQzJWqtVGqc+WJ+A+PtCknDZN3m CYmQ== X-Gm-Message-State: AOAM531Bhvp6M+6BDTJV80nf+imBItKspkUXX8Q/eKvy5QbIQ+P7uBkz SW+jywvCLevWXxZ7FRg2s2B6yQ== X-Google-Smtp-Source: ABdhPJyRYGohzoTg4dhzau/id+aOZcLwu77zYAllJcJ4h4jgoseBbrsSpJiXHpTE4u8OWvmslmKRBw== X-Received: by 2002:a62:6346:0:b029:198:3d33:feb with SMTP id x67-20020a6263460000b02901983d330febmr11740529pfb.8.1607249862830; Sun, 06 Dec 2020 02:17:42 -0800 (PST) Received: from localhost.localdomain ([103.136.221.70]) by smtp.gmail.com with ESMTPSA id g16sm10337657pfb.201.2020.12.06.02.17.34 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 06 Dec 2020 02:17:42 -0800 (PST) From: Muchun Song To: gregkh@linuxfoundation.org, rafael@kernel.org, adobriyan@gmail.com, akpm@linux-foundation.org, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, hughd@google.com, will@kernel.org, guro@fb.com, rppt@kernel.org, tglx@linutronix.de, esyr@redhat.com, peterx@redhat.com, krisman@collabora.com, surenb@google.com, avagin@openvz.org, elver@google.com, rdunlap@infradead.org, iamjoonsoo.kim@lge.com Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, Muchun Song Subject: [RESEND PATCH v2 06/12] mm: memcontrol: convert NR_FILE_PMDMAPPED account to pages Date: Sun, 6 Dec 2020 18:14:45 +0800 Message-Id: <20201206101451.14706-7-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20201206101451.14706-1-songmuchun@bytedance.com> References: <20201206101451.14706-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Convert NR_FILE_PMDMAPPED account to pages Signed-off-by: Muchun Song --- drivers/base/node.c | 3 +-- fs/proc/meminfo.c | 2 +- mm/rmap.c | 6 ++++-- 3 files changed, 6 insertions(+), 5 deletions(-) diff --git a/drivers/base/node.c b/drivers/base/node.c index e5abc6144dab..f77652e6339f 100644 --- a/drivers/base/node.c +++ b/drivers/base/node.c @@ -465,8 +465,7 @@ static ssize_t node_read_meminfo(struct device *dev, nid, K(node_page_state(pgdat, NR_SHMEM_THPS)), nid, K(node_page_state(pgdat, NR_SHMEM_PMDMAPPED)), nid, K(node_page_state(pgdat, NR_FILE_THPS)), - nid, K(node_page_state(pgdat, NR_FILE_PMDMAPPED) * - HPAGE_PMD_NR) + nid, K(node_page_state(pgdat, NR_FILE_PMDMAPPED)) #endif ); len += hugetlb_report_node_meminfo(buf, len, nid); diff --git a/fs/proc/meminfo.c b/fs/proc/meminfo.c index 84886b2cc2f7..5a83012d8b72 100644 --- a/fs/proc/meminfo.c +++ b/fs/proc/meminfo.c @@ -137,7 +137,7 @@ static int meminfo_proc_show(struct seq_file *m, void *v) show_val_kb(m, "FileHugePages: ", global_node_page_state(NR_FILE_THPS)); show_val_kb(m, "FilePmdMapped: ", - global_node_page_state(NR_FILE_PMDMAPPED) * HPAGE_PMD_NR); + global_node_page_state(NR_FILE_PMDMAPPED)); #endif #ifdef CONFIG_CMA diff --git a/mm/rmap.c b/mm/rmap.c index 3089ad6bf468..e383c5619501 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -1222,7 +1222,8 @@ void page_add_file_rmap(struct page *page, bool compound) __mod_lruvec_page_state(page, NR_SHMEM_PMDMAPPED, HPAGE_PMD_NR); else - __inc_node_page_state(page, NR_FILE_PMDMAPPED); + __mod_lruvec_page_state(page, NR_FILE_PMDMAPPED, + HPAGE_PMD_NR); } else { if (PageTransCompound(page) && page_mapping(page)) { VM_WARN_ON_ONCE(!PageLocked(page)); @@ -1264,7 +1265,8 @@ static void page_remove_file_rmap(struct page *page, bool compound) __mod_lruvec_page_state(page, NR_SHMEM_PMDMAPPED, -HPAGE_PMD_NR); else - __dec_node_page_state(page, NR_FILE_PMDMAPPED); + __mod_lruvec_page_state(page, NR_FILE_PMDMAPPED, + -HPAGE_PMD_NR); } else { if (!atomic_add_negative(-1, &page->_mapcount)) return; From patchwork Sun Dec 6 10:14:46 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 11953897 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 94F7BC433FE for ; Sun, 6 Dec 2020 10:17:54 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 3435122D2C for ; Sun, 6 Dec 2020 10:17:54 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3435122D2C Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id B64BD6B0072; Sun, 6 Dec 2020 05:17:53 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id B15E18D0007; Sun, 6 Dec 2020 05:17:53 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A05E08D0006; Sun, 6 Dec 2020 05:17:53 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0136.hostedemail.com [216.40.44.136]) by kanga.kvack.org (Postfix) with ESMTP id 8C5C06B0072 for ; Sun, 6 Dec 2020 05:17:53 -0500 (EST) Received: from smtpin20.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 5E2D0181AEF10 for ; Sun, 6 Dec 2020 10:17:53 +0000 (UTC) X-FDA: 77562456426.20.laugh46_6316689273d4 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin20.hostedemail.com (Postfix) with ESMTP id 3C5DC180C07AB for ; Sun, 6 Dec 2020 10:17:53 +0000 (UTC) X-HE-Tag: laugh46_6316689273d4 X-Filterd-Recvd-Size: 7502 Received: from mail-pl1-f193.google.com (mail-pl1-f193.google.com [209.85.214.193]) by imf05.hostedemail.com (Postfix) with ESMTP for ; Sun, 6 Dec 2020 10:17:52 +0000 (UTC) Received: by mail-pl1-f193.google.com with SMTP id t18so5599781plo.0 for ; Sun, 06 Dec 2020 02:17:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=OffO4J0XNbKZKsjRiAzQ4jXTCwqv9VY0gL30Fq4+1Jk=; b=H94p/twke3EGgzTfnzLG5zYHpHqM5U9I4h/8S6xaN8YSvCZ9DO+xCRNJwC6Z/juFsl M6Tsvms6S5YZ3AfjawkS+pa4BYclyYqZ7ErIOUHfuViHuPEQJKE7YUPDUVxKXrSe9f+W /RfGFNjF/dCQOKuXuj5rOX6YluRTAYjaWtfsjUt0UVOlFiCQyJQdkNVKiKC+nmLF4x2S t+ClAU7xY+b1CeVAoG4FMpAXmde0vMVwPKAXitdvLV6I6kKeLmjwwcVcjKO7y8jbA/++ 6pgQQpVzBIBijbMxqL/JI/EWPnG4lZP2c+/SrwDWsGQ9CTdpscRKJYHtqtnwm4wK7gPA DPPg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=OffO4J0XNbKZKsjRiAzQ4jXTCwqv9VY0gL30Fq4+1Jk=; b=bktxWvVEuuvJnWLN0fUdH4Y4vzzdnUk/PXHdkOaML8eDrYxwTTnEkPqONCrnLIjP/k 5BUXzZ5/VBcUXPvS/ANKxba9tLD4QsGhRbj3ggQBNSrTCqS4YbpCcxdKVePfWd3oJR8w wWw7BnRQzB1cxY+bVywFrxONnBVAVexTlmzLtDkK87a9XDymB0ySYzApp7jSxQEWL6gZ fmCOj+TSkvknU36irXFvJCZy84OBoPvi/pQkyJ+HmSJusGnVmvOOxyB3I5nsNb3uW0YH 3rV8hshtJ/Ar3wVEdyhaPGapvaKjoWpsc5IfTR73TibAFzzsX03H0wvt5tzwjS8yKbjt Vh1w== X-Gm-Message-State: AOAM533hpx3g+PTq0i/JIauQ0CnZAeb7a4b9gdocTz8eU+tUQ66KmK2a DsfOn2wx6JG0UgFQZlrm1VtzaA== X-Google-Smtp-Source: ABdhPJyaSoVwqS1hV86Jy9HYtaaTN3KW8cLPtOEHhySz3KTEmRALmhiqeeK5DCu0P0C+H1csGtcR0Q== X-Received: by 2002:a17:90a:e386:: with SMTP id b6mr11649470pjz.134.1607249871646; Sun, 06 Dec 2020 02:17:51 -0800 (PST) Received: from localhost.localdomain ([103.136.221.70]) by smtp.gmail.com with ESMTPSA id g16sm10337657pfb.201.2020.12.06.02.17.43 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 06 Dec 2020 02:17:51 -0800 (PST) From: Muchun Song To: gregkh@linuxfoundation.org, rafael@kernel.org, adobriyan@gmail.com, akpm@linux-foundation.org, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, hughd@google.com, will@kernel.org, guro@fb.com, rppt@kernel.org, tglx@linutronix.de, esyr@redhat.com, peterx@redhat.com, krisman@collabora.com, surenb@google.com, avagin@openvz.org, elver@google.com, rdunlap@infradead.org, iamjoonsoo.kim@lge.com Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, Muchun Song Subject: [RESEND PATCH v2 07/12] mm: memcontrol: convert kernel stack account to bytes Date: Sun, 6 Dec 2020 18:14:46 +0800 Message-Id: <20201206101451.14706-8-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20201206101451.14706-1-songmuchun@bytedance.com> References: <20201206101451.14706-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The kernel stack account is the one that counts in KiB. This patch convert it from KiB to byte. Signed-off-by: Muchun Song --- drivers/base/node.c | 2 +- fs/proc/meminfo.c | 2 +- include/linux/mmzone.h | 2 +- kernel/fork.c | 8 ++++---- mm/memcontrol.c | 2 +- mm/page_alloc.c | 2 +- 6 files changed, 9 insertions(+), 9 deletions(-) diff --git a/drivers/base/node.c b/drivers/base/node.c index f77652e6339f..92a75bad35c9 100644 --- a/drivers/base/node.c +++ b/drivers/base/node.c @@ -446,7 +446,7 @@ static ssize_t node_read_meminfo(struct device *dev, nid, K(node_page_state(pgdat, NR_FILE_MAPPED)), nid, K(node_page_state(pgdat, NR_ANON_MAPPED)), nid, K(i.sharedram), - nid, node_page_state(pgdat, NR_KERNEL_STACK_KB), + nid, node_page_state(pgdat, NR_KERNEL_STACK_B) / SZ_1K, #ifdef CONFIG_SHADOW_CALL_STACK nid, node_page_state(pgdat, NR_KERNEL_SCS_KB), #endif diff --git a/fs/proc/meminfo.c b/fs/proc/meminfo.c index 5a83012d8b72..799a537d4218 100644 --- a/fs/proc/meminfo.c +++ b/fs/proc/meminfo.c @@ -101,7 +101,7 @@ static int meminfo_proc_show(struct seq_file *m, void *v) show_val_kb(m, "SReclaimable: ", sreclaimable); show_val_kb(m, "SUnreclaim: ", sunreclaim); seq_printf(m, "KernelStack: %8lu kB\n", - global_node_page_state(NR_KERNEL_STACK_KB)); + global_node_page_state(NR_KERNEL_STACK_B) / SZ_1K); #ifdef CONFIG_SHADOW_CALL_STACK seq_printf(m, "ShadowCallStack:%8lu kB\n", global_node_page_state(NR_KERNEL_SCS_KB)); diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index 15132adaa233..bd34416293ec 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -202,7 +202,7 @@ enum node_stat_item { NR_KERNEL_MISC_RECLAIMABLE, /* reclaimable non-slab kernel pages */ NR_FOLL_PIN_ACQUIRED, /* via: pin_user_page(), gup flag: FOLL_PIN */ NR_FOLL_PIN_RELEASED, /* pages returned via unpin_user_page() */ - NR_KERNEL_STACK_KB, /* measured in KiB */ + NR_KERNEL_STACK_B, /* measured in byte */ #if IS_ENABLED(CONFIG_SHADOW_CALL_STACK) NR_KERNEL_SCS_KB, /* measured in KiB */ #endif diff --git a/kernel/fork.c b/kernel/fork.c index 345f378e104d..2913d7c43dcb 100644 --- a/kernel/fork.c +++ b/kernel/fork.c @@ -382,11 +382,11 @@ static void account_kernel_stack(struct task_struct *tsk, int account) /* All stack pages are in the same node. */ if (vm) - mod_lruvec_page_state(vm->pages[0], NR_KERNEL_STACK_KB, - account * (THREAD_SIZE / 1024)); + mod_lruvec_page_state(vm->pages[0], NR_KERNEL_STACK_B, + account * THREAD_SIZE); else - mod_lruvec_kmem_state(stack, NR_KERNEL_STACK_KB, - account * (THREAD_SIZE / 1024)); + mod_lruvec_kmem_state(stack, NR_KERNEL_STACK_B, + account * THREAD_SIZE); } static int memcg_charge_kernel_stack(struct task_struct *tsk) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 6d4365d2fd1c..48d70c1ad301 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -1500,7 +1500,7 @@ struct memory_stat { static const struct memory_stat memory_stats[] = { { "anon", PAGE_SIZE, NR_ANON_MAPPED }, { "file", PAGE_SIZE, NR_FILE_PAGES }, - { "kernel_stack", 1024, NR_KERNEL_STACK_KB }, + { "kernel_stack", 1, NR_KERNEL_STACK_B }, { "percpu", 1, MEMCG_PERCPU_B }, { "sock", PAGE_SIZE, MEMCG_SOCK }, { "shmem", PAGE_SIZE, NR_SHMEM }, diff --git a/mm/page_alloc.c b/mm/page_alloc.c index d103513b3e4f..d2821ba7f682 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -5572,7 +5572,7 @@ void show_free_areas(unsigned int filter, nodemask_t *nodemask) K(node_page_state(pgdat, NR_ANON_THPS)), #endif K(node_page_state(pgdat, NR_WRITEBACK_TEMP)), - node_page_state(pgdat, NR_KERNEL_STACK_KB), + node_page_state(pgdat, NR_KERNEL_STACK_B) / SZ_1K, #ifdef CONFIG_SHADOW_CALL_STACK node_page_state(pgdat, NR_KERNEL_SCS_KB), #endif From patchwork Sun Dec 6 10:14:47 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 11953899 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 10DDAC4361B for ; Sun, 6 Dec 2020 10:18:06 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id BE31F22D2C for ; Sun, 6 Dec 2020 10:18:05 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BE31F22D2C Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 6349B6B005D; Sun, 6 Dec 2020 05:18:05 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 5E6706B0068; Sun, 6 Dec 2020 05:18:05 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4FD2F6B0073; Sun, 6 Dec 2020 05:18:05 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0144.hostedemail.com [216.40.44.144]) by kanga.kvack.org (Postfix) with ESMTP id 3A59B6B005D for ; Sun, 6 Dec 2020 05:18:05 -0500 (EST) Received: from smtpin20.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 0245E180AD804 for ; Sun, 6 Dec 2020 10:18:05 +0000 (UTC) X-FDA: 77562456930.20.use37_3f04dd9273d4 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin20.hostedemail.com (Postfix) with ESMTP id DA76E180C0609 for ; Sun, 6 Dec 2020 10:18:04 +0000 (UTC) X-HE-Tag: use37_3f04dd9273d4 X-Filterd-Recvd-Size: 6502 Received: from mail-pf1-f194.google.com (mail-pf1-f194.google.com [209.85.210.194]) by imf27.hostedemail.com (Postfix) with ESMTP for ; Sun, 6 Dec 2020 10:18:04 +0000 (UTC) Received: by mail-pf1-f194.google.com with SMTP id w6so7024348pfu.1 for ; Sun, 06 Dec 2020 02:18:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=IzVbUHzHNutFlJQXX/d4rCoABm7bks/8Sv9HD05GfEo=; b=poeTINc8GQTJYdjLzUjOwVcyWKpDqMJuexxM4Jb6DPltaseMinhqh/fI4za09P7SpJ ueTfpbdcaNT8FRgCzXBnvchsMG19r9qFb6dmWplY0+dV4prCug2v2aNQDXQeD4f4ITSk EoVkV9OIP/YW0/Sa7OyTso7ZG60JcGENiXRqAvbK4UuODW674qsOoXE/pNVvUrmgshUS 85409VfqkcMfK8Xr4ByeyYmoKcONo5Ce9w22Krdh3gyADCxxiAkKBnxjcsgF8uV5S52E vCvZZh3sftwph8jcw8YfxrYT88oU5Tt46j5Q212IwnFvnbLVED2uTA1ydilqWk7ajFo5 i5mQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=IzVbUHzHNutFlJQXX/d4rCoABm7bks/8Sv9HD05GfEo=; b=jb1qy6j46ENzP81Qa8x/iBuy82UwibCCDlgos50Pf32ZYJ+39OXW/FF/ShrCAfn1uq jXUcu17cxXCgHavWDH+/W/2HURK6Q0j+oz9iCRDHfQFtjobjp8zcsBmGngdk3kP+OQQJ zr8s/mSuvXA42o1mXSWJaLXE47UNX3LM34TDWwKekuGlnaGdq/LLC8JsTwukKdiV/xyO tmrY8h0vm7h34baquIW6pjLIOuRb9aFMzV6hfyWVTQqQetUt+UmBmcfangmJiGtfq3bL 2pWhmLbu6KHPSmesSUJaemtK82bIf2PeJ97Xmcyq77CvL851yUvBO0MZMCg1NG7+4Jt+ jT+A== X-Gm-Message-State: AOAM532nbqigMWIdYwD7YsQ3r3D2e3SzRPAdQfnA3H/eHfHgZ02YzMOV 8A0IOGra8F9CPpA+j5OGCtsyjw== X-Google-Smtp-Source: ABdhPJwOE+mifEeZdUynoRMflnzW+cfbd7QXeq8UPaKhUO7NDUpn7fwwqZ66YSE0/Dk/xryAVB2Udg== X-Received: by 2002:aa7:9501:0:b029:155:3b11:d5c4 with SMTP id b1-20020aa795010000b02901553b11d5c4mr11232823pfp.76.1607249883472; Sun, 06 Dec 2020 02:18:03 -0800 (PST) Received: from localhost.localdomain ([103.136.221.70]) by smtp.gmail.com with ESMTPSA id g16sm10337657pfb.201.2020.12.06.02.17.51 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 06 Dec 2020 02:18:02 -0800 (PST) From: Muchun Song To: gregkh@linuxfoundation.org, rafael@kernel.org, adobriyan@gmail.com, akpm@linux-foundation.org, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, hughd@google.com, will@kernel.org, guro@fb.com, rppt@kernel.org, tglx@linutronix.de, esyr@redhat.com, peterx@redhat.com, krisman@collabora.com, surenb@google.com, avagin@openvz.org, elver@google.com, rdunlap@infradead.org, iamjoonsoo.kim@lge.com Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, Muchun Song Subject: [RESEND PATCH v2 08/12] mm: memcontrol: convert NR_KERNEL_SCS_KB account to bytes Date: Sun, 6 Dec 2020 18:14:47 +0800 Message-Id: <20201206101451.14706-9-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20201206101451.14706-1-songmuchun@bytedance.com> References: <20201206101451.14706-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Convert NR_KERNEL_SCS_KB account to bytes Signed-off-by: Muchun Song --- drivers/base/node.c | 2 +- fs/proc/meminfo.c | 2 +- include/linux/mmzone.h | 2 +- kernel/scs.c | 4 ++-- mm/page_alloc.c | 2 +- 5 files changed, 6 insertions(+), 6 deletions(-) diff --git a/drivers/base/node.c b/drivers/base/node.c index 92a75bad35c9..bc01ce0b2fcd 100644 --- a/drivers/base/node.c +++ b/drivers/base/node.c @@ -448,7 +448,7 @@ static ssize_t node_read_meminfo(struct device *dev, nid, K(i.sharedram), nid, node_page_state(pgdat, NR_KERNEL_STACK_B) / SZ_1K, #ifdef CONFIG_SHADOW_CALL_STACK - nid, node_page_state(pgdat, NR_KERNEL_SCS_KB), + nid, node_page_state(pgdat, NR_KERNEL_SCS_B) / SZ_1K, #endif nid, K(sum_zone_node_page_state(nid, NR_PAGETABLE)), nid, 0UL, diff --git a/fs/proc/meminfo.c b/fs/proc/meminfo.c index 799a537d4218..69895e83d4fc 100644 --- a/fs/proc/meminfo.c +++ b/fs/proc/meminfo.c @@ -104,7 +104,7 @@ static int meminfo_proc_show(struct seq_file *m, void *v) global_node_page_state(NR_KERNEL_STACK_B) / SZ_1K); #ifdef CONFIG_SHADOW_CALL_STACK seq_printf(m, "ShadowCallStack:%8lu kB\n", - global_node_page_state(NR_KERNEL_SCS_KB)); + global_node_page_state(NR_KERNEL_SCS_B) / SZ_1K); #endif show_val_kb(m, "PageTables: ", global_zone_page_state(NR_PAGETABLE)); diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index bd34416293ec..1f9c83778629 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -204,7 +204,7 @@ enum node_stat_item { NR_FOLL_PIN_RELEASED, /* pages returned via unpin_user_page() */ NR_KERNEL_STACK_B, /* measured in byte */ #if IS_ENABLED(CONFIG_SHADOW_CALL_STACK) - NR_KERNEL_SCS_KB, /* measured in KiB */ + NR_KERNEL_SCS_B, /* measured in byte */ #endif NR_VM_NODE_STAT_ITEMS }; diff --git a/kernel/scs.c b/kernel/scs.c index 4ff4a7ba0094..8db89c932ddc 100644 --- a/kernel/scs.c +++ b/kernel/scs.c @@ -17,8 +17,8 @@ static void __scs_account(void *s, int account) { struct page *scs_page = virt_to_page(s); - mod_node_page_state(page_pgdat(scs_page), NR_KERNEL_SCS_KB, - account * (SCS_SIZE / SZ_1K)); + mod_node_page_state(page_pgdat(scs_page), NR_KERNEL_SCS_B, + account * SCS_SIZE); } static void *scs_alloc(int node) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index d2821ba7f682..58916b3afdab 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -5574,7 +5574,7 @@ void show_free_areas(unsigned int filter, nodemask_t *nodemask) K(node_page_state(pgdat, NR_WRITEBACK_TEMP)), node_page_state(pgdat, NR_KERNEL_STACK_B) / SZ_1K, #ifdef CONFIG_SHADOW_CALL_STACK - node_page_state(pgdat, NR_KERNEL_SCS_KB), + node_page_state(pgdat, NR_KERNEL_SCS_B) / SZ_1K, #endif pgdat->kswapd_failures >= MAX_RECLAIM_RETRIES ? "yes" : "no"); From patchwork Sun Dec 6 10:14:48 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 11953901 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A5C19C433FE for ; Sun, 6 Dec 2020 10:18:16 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 549B022D2C for ; Sun, 6 Dec 2020 10:18:16 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 549B022D2C Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id B3E9A6B0068; Sun, 6 Dec 2020 05:18:15 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id AEF336B0073; Sun, 6 Dec 2020 05:18:15 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9DF5C6B0074; Sun, 6 Dec 2020 05:18:15 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0075.hostedemail.com [216.40.44.75]) by kanga.kvack.org (Postfix) with ESMTP id 8805B6B0068 for ; Sun, 6 Dec 2020 05:18:15 -0500 (EST) Received: from smtpin11.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 4D7E1181AEF10 for ; Sun, 6 Dec 2020 10:18:15 +0000 (UTC) X-FDA: 77562457350.11.honey25_040cd82273d4 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin11.hostedemail.com (Postfix) with ESMTP id 13D8D180F8B80 for ; Sun, 6 Dec 2020 10:18:15 +0000 (UTC) X-HE-Tag: honey25_040cd82273d4 X-Filterd-Recvd-Size: 6557 Received: from mail-pj1-f68.google.com (mail-pj1-f68.google.com [209.85.216.68]) by imf42.hostedemail.com (Postfix) with ESMTP for ; Sun, 6 Dec 2020 10:18:14 +0000 (UTC) Received: by mail-pj1-f68.google.com with SMTP id o7so5744296pjj.2 for ; Sun, 06 Dec 2020 02:18:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=jxjzCofAcULr7bnQMbpOvZCW6tNXMo722rzO5X3X+g0=; b=l0t429mzGcnZFwxy/27EHeyds85DHU2/SvU/cTTFIMKpTZQ13vGzScVjnNJc1Qfgdl tq3J8LH2Po4yAdE4Qo1uNZ9GfP2dCXBM7HpnjMHacvON/LTEfdfapf3sJCtF6f1t+m// Rdasl44TWp0U1bs3j6e3qWDr9Qn4cvq1JzncWvKY7tQL+4hmLSOUSIIajJobIPEmCai1 ok61QC0poafIrVv2iaxO+AbBLSIJ7BNrJjU/r+EtWVpKKiZe8qX3AGUSS983k1LeN9AL G+VQSyqTH4X1j0tI7iZy9yvMHaPD1FXpVWoKPjzR3GdmseCxF/0+PH5flDA0QOufjbFW Swlw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=jxjzCofAcULr7bnQMbpOvZCW6tNXMo722rzO5X3X+g0=; b=iuqHlf2IIW6GMVu2xVf5l+uYrO+XCSaIK0Wp/qv9VCcu9Em1fUMzUwPcJLMZ/VNuvQ IiEFhpuVJ3kdJjKL4hT4SlvDXtll/yGx2SlPLeyPAfnc2mwPsy1wne0q9oe1aLPA1o+h BJ4II6sIgLF7ndz3lP+yXZY3ODE9m4fycloElfTjoPopFMYF0GWagGwJFbXY/t/ccjG0 WY2JAWWAeSUQ6R+hg89x62XGaYZRk3KtP9Ny82TcbeJo6GigHL+7E0V14jJjDuYLa+t0 McTVC5z1TJtkUS2NNqYub7AJg3w0veQ1nK3+iyJQ3g0OseRdDEqCN10AwkzjJ7NHfGV0 XVPA== X-Gm-Message-State: AOAM533/kqB1XhfpIp/rgXMnc87cB/FfnLhn5srPqBiJ2emmfRA5Qjum LWRWzEPhN5vylSJQ569iYmvOFA== X-Google-Smtp-Source: ABdhPJwhfw1qSbAuBw+1RM7qEp36SUEwopFNg3Q2irPAgag3zrJxkL8XTXGjZkVkY1IB6k2hYZijDg== X-Received: by 2002:a17:90b:390b:: with SMTP id ob11mr11955031pjb.66.1607249893914; Sun, 06 Dec 2020 02:18:13 -0800 (PST) Received: from localhost.localdomain ([103.136.221.70]) by smtp.gmail.com with ESMTPSA id g16sm10337657pfb.201.2020.12.06.02.18.03 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 06 Dec 2020 02:18:13 -0800 (PST) From: Muchun Song To: gregkh@linuxfoundation.org, rafael@kernel.org, adobriyan@gmail.com, akpm@linux-foundation.org, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, hughd@google.com, will@kernel.org, guro@fb.com, rppt@kernel.org, tglx@linutronix.de, esyr@redhat.com, peterx@redhat.com, krisman@collabora.com, surenb@google.com, avagin@openvz.org, elver@google.com, rdunlap@infradead.org, iamjoonsoo.kim@lge.com Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, Muchun Song Subject: [RESEND PATCH v2 09/12] mm: memcontrol: convert vmstat slab counters to bytes Date: Sun, 6 Dec 2020 18:14:48 +0800 Message-Id: <20201206101451.14706-10-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20201206101451.14706-1-songmuchun@bytedance.com> References: <20201206101451.14706-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: the global and per-node counters are stored in pages, however memcg and lruvec counters are stored in bytes. This scheme looks weird. So convert all vmstat slab counters to bytes. Signed-off-by: Muchun Song --- include/linux/vmstat.h | 17 ++++++++++------- mm/vmstat.c | 21 ++++++++++----------- 2 files changed, 20 insertions(+), 18 deletions(-) diff --git a/include/linux/vmstat.h b/include/linux/vmstat.h index 322dcbfcc933..fd1a3d5d4926 100644 --- a/include/linux/vmstat.h +++ b/include/linux/vmstat.h @@ -197,18 +197,26 @@ static inline unsigned long global_node_page_state_pages(enum node_stat_item item) { long x = atomic_long_read(&vm_node_stat[item]); + #ifdef CONFIG_SMP if (x < 0) x = 0; #endif + if (vmstat_item_in_bytes(item)) + x >>= PAGE_SHIFT; return x; } static inline unsigned long global_node_page_state(enum node_stat_item item) { - VM_WARN_ON_ONCE(vmstat_item_in_bytes(item)); + long x = atomic_long_read(&vm_node_stat[item]); - return global_node_page_state_pages(item); + VM_WARN_ON_ONCE(vmstat_item_in_bytes(item)); +#ifdef CONFIG_SMP + if (x < 0) + x = 0; +#endif + return x; } static inline unsigned long zone_page_state(struct zone *zone, @@ -312,11 +320,6 @@ static inline void __mod_zone_page_state(struct zone *zone, static inline void __mod_node_page_state(struct pglist_data *pgdat, enum node_stat_item item, int delta) { - if (vmstat_item_in_bytes(item)) { - VM_WARN_ON_ONCE(delta & (PAGE_SIZE - 1)); - delta >>= PAGE_SHIFT; - } - node_page_state_add(delta, pgdat, item); } diff --git a/mm/vmstat.c b/mm/vmstat.c index 8d77ee426e22..7fb0c7cb9516 100644 --- a/mm/vmstat.c +++ b/mm/vmstat.c @@ -345,11 +345,6 @@ void __mod_node_page_state(struct pglist_data *pgdat, enum node_stat_item item, long x; long t; - if (vmstat_item_in_bytes(item)) { - VM_WARN_ON_ONCE(delta & (PAGE_SIZE - 1)); - delta >>= PAGE_SHIFT; - } - x = delta + __this_cpu_read(*p); t = __this_cpu_read(pcp->stat_threshold); @@ -554,11 +549,6 @@ static inline void mod_node_state(struct pglist_data *pgdat, s8 __percpu *p = pcp->vm_node_stat_diff + item; long o, n, t, z; - if (vmstat_item_in_bytes(item)) { - VM_WARN_ON_ONCE(delta & (PAGE_SIZE - 1)); - delta >>= PAGE_SHIFT; - } - do { z = 0; /* overflow to node counters */ @@ -1012,19 +1002,28 @@ unsigned long node_page_state_pages(struct pglist_data *pgdat, enum node_stat_item item) { long x = atomic_long_read(&pgdat->vm_stat[item]); + #ifdef CONFIG_SMP if (x < 0) x = 0; #endif + if (vmstat_item_in_bytes(item)) + x >>= PAGE_SHIFT; return x; } unsigned long node_page_state(struct pglist_data *pgdat, enum node_stat_item item) { + long x = atomic_long_read(&pgdat->vm_stat[item]); + VM_WARN_ON_ONCE(vmstat_item_in_bytes(item)); - return node_page_state_pages(pgdat, item); +#ifdef CONFIG_SMP + if (x < 0) + x = 0; +#endif + return x; } #endif From patchwork Sun Dec 6 10:14:49 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 11953903 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 965CFC4361B for ; Sun, 6 Dec 2020 10:18:25 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 3F4BA22D2C for ; Sun, 6 Dec 2020 10:18:25 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3F4BA22D2C Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id D87828D0003; Sun, 6 Dec 2020 05:18:24 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D38196B0074; Sun, 6 Dec 2020 05:18:24 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C276B8D0003; Sun, 6 Dec 2020 05:18:24 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0164.hostedemail.com [216.40.44.164]) by kanga.kvack.org (Postfix) with ESMTP id AD7246B0073 for ; Sun, 6 Dec 2020 05:18:24 -0500 (EST) Received: from smtpin03.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 7C7941EE6 for ; Sun, 6 Dec 2020 10:18:24 +0000 (UTC) X-FDA: 77562457728.03.car43_480aaf5273d4 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin03.hostedemail.com (Postfix) with ESMTP id 4201028A4EB for ; Sun, 6 Dec 2020 10:18:24 +0000 (UTC) X-HE-Tag: car43_480aaf5273d4 X-Filterd-Recvd-Size: 8777 Received: from mail-pl1-f194.google.com (mail-pl1-f194.google.com [209.85.214.194]) by imf40.hostedemail.com (Postfix) with ESMTP for ; Sun, 6 Dec 2020 10:18:23 +0000 (UTC) Received: by mail-pl1-f194.google.com with SMTP id t6so37852plq.1 for ; Sun, 06 Dec 2020 02:18:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=cL9BVbVQLgAGW6GpXAhL1X3CA8w9xPKU91tirmQZrTs=; b=0MtFDJWkKOQQMDcBAWzMx6gdC5YkqHBAIYhp+BQWqaLCRwKSjpNgXt0q+GE5YDnhq7 zZlx16g/g5Hty2Kbo00woChd1R2HTbXvLREFGr+dgsJNg5wqsoVf732VjIl8t0tx/guG xKJjwyDtJ23mJ3u2LiGrZt8158SzBOEHqu1EONODMauwN44TsNCh+5mOLcjf6KWCRj+Q PpDhsHo4azjOMj55FDu9gE2qP5mrbjysGxlYeo4D7bHzSkfdcbJN0sB1YfNpHorDM+rk KTXv8MboLM+aP9XEdWHdigIwbkC+YMT4rLQKIz5VS8XacdkjzRDyPoEiVJiUVpjvTk70 B/IQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=cL9BVbVQLgAGW6GpXAhL1X3CA8w9xPKU91tirmQZrTs=; b=KwObw7VvRTSXZRvGTjAD7PM6qufoINlZJzxiEPAPJW+g0w0xkg7J2gevFVRTSPi21w ogIZom4V19wl8ht76vuLikPM/Lt4A0g25ozB4dyndiIJPrzP6hemgskTChGIjGH8r8ZA xD88sp8GVsYK4AhlBCcwP1HxxzZ9AZMWMVnlhh+HTwK6nr8ZRw6jIJOiSD3IyQLYA/mE 1x9c9Swt4OUBY9RRscDTMUqCEjPazvtnAzqY7B3cIabwi5K9I3N92PMkpklzbHAQ9KWY IV3ijN7A/91LiuIAHwZcflbRTHwD1eY/tkrqFHm57cDQNLRhUGNdNE/kFESG9ADCHDuK bmXw== X-Gm-Message-State: AOAM5327J6Md89QdQ5GZlY8/X1r3voIwq93V73p99e+I64nQTeeK8K8K lefMCZhrt4w2g5mQ8mgmiXOZQQ== X-Google-Smtp-Source: ABdhPJz5DNYCSJVhDue++Ah/epNUxyRIwmJ9vSLViSgMOWI7q8z87Dx9FtBFYEBh1GWmEzjkBxvqzQ== X-Received: by 2002:a17:902:b582:b029:d6:6008:264d with SMTP id a2-20020a170902b582b02900d66008264dmr11050550pls.80.1607249902812; Sun, 06 Dec 2020 02:18:22 -0800 (PST) Received: from localhost.localdomain ([103.136.221.70]) by smtp.gmail.com with ESMTPSA id g16sm10337657pfb.201.2020.12.06.02.18.14 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 06 Dec 2020 02:18:22 -0800 (PST) From: Muchun Song To: gregkh@linuxfoundation.org, rafael@kernel.org, adobriyan@gmail.com, akpm@linux-foundation.org, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, hughd@google.com, will@kernel.org, guro@fb.com, rppt@kernel.org, tglx@linutronix.de, esyr@redhat.com, peterx@redhat.com, krisman@collabora.com, surenb@google.com, avagin@openvz.org, elver@google.com, rdunlap@infradead.org, iamjoonsoo.kim@lge.com Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, Muchun Song Subject: [RESEND PATCH v2 10/12] mm: memcontrol: scale stat_threshold for byted-sized vmstat Date: Sun, 6 Dec 2020 18:14:49 +0800 Message-Id: <20201206101451.14706-11-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20201206101451.14706-1-songmuchun@bytedance.com> References: <20201206101451.14706-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Some vmstat counters are being accounted in bytes not pages, so the stat_threshold should also scale to bytes. The vmstat counters are already long type for memcg (can reference to struct lruvec_stat). For the global per-node vmstat counters also can scale to long. But the maximum vmstat threshold is 125, so the type of s32 is enough. Signed-off-by: Muchun Song --- include/linux/mmzone.h | 17 ++++++----------- include/linux/vmstat.h | 1 - mm/vmstat.c | 24 +++++++++++++----------- 3 files changed, 19 insertions(+), 23 deletions(-) diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index 1f9c83778629..d53328551225 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -216,17 +216,12 @@ enum node_stat_item { */ static __always_inline bool vmstat_item_in_bytes(int idx) { - /* - * Global and per-node slab counters track slab pages. - * It's expected that changes are multiples of PAGE_SIZE. - * Internally values are stored in pages. - * - * Per-memcg and per-lruvec counters track memory, consumed - * by individual slab objects. These counters are actually - * byte-precise. - */ return (idx == NR_SLAB_RECLAIMABLE_B || - idx == NR_SLAB_UNRECLAIMABLE_B); + idx == NR_SLAB_UNRECLAIMABLE_B || +#if IS_ENABLED(CONFIG_SHADOW_CALL_STACK) + idx == NR_KERNEL_SCS_B || +#endif + idx == NR_KERNEL_STACK_B); } /* @@ -340,7 +335,7 @@ struct per_cpu_pageset { struct per_cpu_nodestat { s8 stat_threshold; - s8 vm_node_stat_diff[NR_VM_NODE_STAT_ITEMS]; + s32 vm_node_stat_diff[NR_VM_NODE_STAT_ITEMS]; }; #endif /* !__GENERATING_BOUNDS.H */ diff --git a/include/linux/vmstat.h b/include/linux/vmstat.h index fd1a3d5d4926..afd84dc2398c 100644 --- a/include/linux/vmstat.h +++ b/include/linux/vmstat.h @@ -211,7 +211,6 @@ static inline unsigned long global_node_page_state(enum node_stat_item item) { long x = atomic_long_read(&vm_node_stat[item]); - VM_WARN_ON_ONCE(vmstat_item_in_bytes(item)); #ifdef CONFIG_SMP if (x < 0) x = 0; diff --git a/mm/vmstat.c b/mm/vmstat.c index 7fb0c7cb9516..25751b1d8e2e 100644 --- a/mm/vmstat.c +++ b/mm/vmstat.c @@ -341,13 +341,15 @@ void __mod_node_page_state(struct pglist_data *pgdat, enum node_stat_item item, long delta) { struct per_cpu_nodestat __percpu *pcp = pgdat->per_cpu_nodestats; - s8 __percpu *p = pcp->vm_node_stat_diff + item; + s32 __percpu *p = pcp->vm_node_stat_diff + item; long x; long t; x = delta + __this_cpu_read(*p); t = __this_cpu_read(pcp->stat_threshold); + if (vmstat_item_in_bytes(item)) + t <<= PAGE_SHIFT; if (unlikely(abs(x) > t)) { node_page_state_add(x, pgdat, item); @@ -399,15 +401,15 @@ void __inc_zone_state(struct zone *zone, enum zone_stat_item item) void __inc_node_state(struct pglist_data *pgdat, enum node_stat_item item) { struct per_cpu_nodestat __percpu *pcp = pgdat->per_cpu_nodestats; - s8 __percpu *p = pcp->vm_node_stat_diff + item; - s8 v, t; + s32 __percpu *p = pcp->vm_node_stat_diff + item; + s32 v, t; VM_WARN_ON_ONCE(vmstat_item_in_bytes(item)); v = __this_cpu_inc_return(*p); t = __this_cpu_read(pcp->stat_threshold); if (unlikely(v > t)) { - s8 overstep = t >> 1; + s32 overstep = t >> 1; node_page_state_add(v + overstep, pgdat, item); __this_cpu_write(*p, -overstep); @@ -445,8 +447,8 @@ void __dec_zone_state(struct zone *zone, enum zone_stat_item item) void __dec_node_state(struct pglist_data *pgdat, enum node_stat_item item) { struct per_cpu_nodestat __percpu *pcp = pgdat->per_cpu_nodestats; - s8 __percpu *p = pcp->vm_node_stat_diff + item; - s8 v, t; + s32 __percpu *p = pcp->vm_node_stat_diff + item; + s32 v, t; VM_WARN_ON_ONCE(vmstat_item_in_bytes(item)); @@ -546,7 +548,7 @@ static inline void mod_node_state(struct pglist_data *pgdat, enum node_stat_item item, int delta, int overstep_mode) { struct per_cpu_nodestat __percpu *pcp = pgdat->per_cpu_nodestats; - s8 __percpu *p = pcp->vm_node_stat_diff + item; + s32 __percpu *p = pcp->vm_node_stat_diff + item; long o, n, t, z; do { @@ -563,6 +565,8 @@ static inline void mod_node_state(struct pglist_data *pgdat, * for all cpus in a node. */ t = this_cpu_read(pcp->stat_threshold); + if (vmstat_item_in_bytes(item)) + t <<= PAGE_SHIFT; o = this_cpu_read(*p); n = delta + o; @@ -829,7 +833,7 @@ static int refresh_cpu_vm_stats(bool do_pagesets) struct per_cpu_nodestat __percpu *p = pgdat->per_cpu_nodestats; for (i = 0; i < NR_VM_NODE_STAT_ITEMS; i++) { - int v; + s32 v; v = this_cpu_xchg(p->vm_node_stat_diff[i], 0); if (v) { @@ -899,7 +903,7 @@ void cpu_vm_stats_fold(int cpu) for (i = 0; i < NR_VM_NODE_STAT_ITEMS; i++) if (p->vm_node_stat_diff[i]) { - int v; + s32 v; v = p->vm_node_stat_diff[i]; p->vm_node_stat_diff[i] = 0; @@ -1017,8 +1021,6 @@ unsigned long node_page_state(struct pglist_data *pgdat, { long x = atomic_long_read(&pgdat->vm_stat[item]); - VM_WARN_ON_ONCE(vmstat_item_in_bytes(item)); - #ifdef CONFIG_SMP if (x < 0) x = 0; From patchwork Sun Dec 6 10:14:50 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 11953905 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2604AC4361B for ; Sun, 6 Dec 2020 10:18:34 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id B85D422D2C for ; Sun, 6 Dec 2020 10:18:33 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B85D422D2C Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 60E038D0005; Sun, 6 Dec 2020 05:18:33 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 5BD126B0074; Sun, 6 Dec 2020 05:18:33 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4D32E8D0005; Sun, 6 Dec 2020 05:18:33 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0059.hostedemail.com [216.40.44.59]) by kanga.kvack.org (Postfix) with ESMTP id 373536B0073 for ; Sun, 6 Dec 2020 05:18:33 -0500 (EST) Received: from smtpin29.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id ED196362D for ; Sun, 6 Dec 2020 10:18:32 +0000 (UTC) X-FDA: 77562458064.29.rat31_560f7cf273d4 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin29.hostedemail.com (Postfix) with ESMTP id CD74F180868E2 for ; Sun, 6 Dec 2020 10:18:32 +0000 (UTC) X-HE-Tag: rat31_560f7cf273d4 X-Filterd-Recvd-Size: 9659 Received: from mail-pj1-f68.google.com (mail-pj1-f68.google.com [209.85.216.68]) by imf47.hostedemail.com (Postfix) with ESMTP for ; Sun, 6 Dec 2020 10:18:32 +0000 (UTC) Received: by mail-pj1-f68.google.com with SMTP id iq13so5664183pjb.3 for ; Sun, 06 Dec 2020 02:18:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=KwRaMRI+Sr6xQo3P1J4+HoZOIYaqIcm7QaRM3zeaHfM=; b=f/VNxHgtOj2/ijqqrMgBgEUMKgTq1TDJOBO6yN20rM9HpH8TxNGjJYzwKTkp3RG5sD USGASwcjMY+URzaWD2oF4Pv1KZGFh4zftUJ6WMVXbHKNpARcJxdsDopTTjMW+4Bx1CMk f2c5UACjKkSxlREuzyENkckmbtZVOQTFTlnlEPmMSdM0RyQlwfpdOGuZmJ9MMihGnp35 vxeZEdaUuGsIF96EnguTyhCMMADUD3KzIYz3IyE0jVXLFmkeoxxKPbhM1wOXunzhe9Jw 9p3eJ/ZZXPy00wlkw4joCyNiL5Fs2TV2Gpkjx954WMxsVeup4myYhUKVdDRCrwp027Q2 Ndsw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=KwRaMRI+Sr6xQo3P1J4+HoZOIYaqIcm7QaRM3zeaHfM=; b=IUkfOA9kH0qSrLttbkD4yN7N97Gg7QvpznDvRTvDVt9tmEwsQvKYzbD4RdWbka2BXR AT9V68eONPVZMt3p0zgZFubhJ/JDxDWhUSNvcVP5o+CUx8KWF5UBWR/wmG9VZSuyIYBP 6TvUXdsXRLDetYqBaZZsdrGOh5zyuwnJeZn9MPw1WPgHFE0V10IOip9CVWxspIO10whA jEcV37zTmlQA2ljirGFb2hM1JhxSUMSVcGqz0MV5/qSbJrHcLCMVVV7w4T1Vpa4Wgo9S GbG3HtWHp8vGFgFqvO6qDl2ZL8Hha6sT7KgUAEUTPTb5aZXwLbKExEITlCloBxR0lih6 QbrA== X-Gm-Message-State: AOAM533+EdcOpadsl6u6sFAcHh0yl1iXrU6VeqD5TnvW+gASD/+Fz65v gxWCSWFOAzbq5p+V6fT1B0Wumg== X-Google-Smtp-Source: ABdhPJxLE9y1Wu89jVTjYxdBJC2dpD5Vw0WGeqkY8mH2oUPiW1ZCgwzB4TvQGoJ/pALX8b1qIIx9ng== X-Received: by 2002:a17:90a:bf88:: with SMTP id d8mr12070599pjs.102.1607249911546; Sun, 06 Dec 2020 02:18:31 -0800 (PST) Received: from localhost.localdomain ([103.136.221.70]) by smtp.gmail.com with ESMTPSA id g16sm10337657pfb.201.2020.12.06.02.18.23 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 06 Dec 2020 02:18:30 -0800 (PST) From: Muchun Song To: gregkh@linuxfoundation.org, rafael@kernel.org, adobriyan@gmail.com, akpm@linux-foundation.org, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, hughd@google.com, will@kernel.org, guro@fb.com, rppt@kernel.org, tglx@linutronix.de, esyr@redhat.com, peterx@redhat.com, krisman@collabora.com, surenb@google.com, avagin@openvz.org, elver@google.com, rdunlap@infradead.org, iamjoonsoo.kim@lge.com Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, Muchun Song Subject: [RESEND PATCH v2 11/12] mm: memcontrol: make the slab calculation consistent Date: Sun, 6 Dec 2020 18:14:50 +0800 Message-Id: <20201206101451.14706-12-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20201206101451.14706-1-songmuchun@bytedance.com> References: <20201206101451.14706-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Although the ratio of the slab is one, we also should read the ratio from the related memory_stats instead of hard-coding. And the local variable of size is already the value of slab_unreclaimable. So we do not need to read again. The unit of the vmstat counters are either pages or bytes now. So we can drop the ratio in struct memory_stat. This can make the code clean and simple. And get rid of the awkward mix of static and runtime initialization of the memory_stats table. Signed-off-by: Muchun Song --- mm/memcontrol.c | 108 ++++++++++++++++++++++++++++++++++++-------------------- 1 file changed, 70 insertions(+), 38 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 48d70c1ad301..49fbcf003bf5 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -1493,48 +1493,71 @@ static bool mem_cgroup_wait_acct_move(struct mem_cgroup *memcg) struct memory_stat { const char *name; - unsigned int ratio; unsigned int idx; }; static const struct memory_stat memory_stats[] = { - { "anon", PAGE_SIZE, NR_ANON_MAPPED }, - { "file", PAGE_SIZE, NR_FILE_PAGES }, - { "kernel_stack", 1, NR_KERNEL_STACK_B }, - { "percpu", 1, MEMCG_PERCPU_B }, - { "sock", PAGE_SIZE, MEMCG_SOCK }, - { "shmem", PAGE_SIZE, NR_SHMEM }, - { "file_mapped", PAGE_SIZE, NR_FILE_MAPPED }, - { "file_dirty", PAGE_SIZE, NR_FILE_DIRTY }, - { "file_writeback", PAGE_SIZE, NR_WRITEBACK }, + { "anon", NR_ANON_MAPPED }, + { "file", NR_FILE_PAGES }, + { "kernel_stack", NR_KERNEL_STACK_B }, + { "percpu", MEMCG_PERCPU_B }, + { "sock", MEMCG_SOCK }, + { "shmem", NR_SHMEM }, + { "file_mapped", NR_FILE_MAPPED }, + { "file_dirty", NR_FILE_DIRTY }, + { "file_writeback", NR_WRITEBACK }, #ifdef CONFIG_TRANSPARENT_HUGEPAGE - { "anon_thp", PAGE_SIZE, NR_ANON_THPS }, - { "file_thp", PAGE_SIZE, NR_FILE_THPS }, - { "shmem_thp", PAGE_SIZE, NR_SHMEM_THPS }, + { "anon_thp", NR_ANON_THPS }, + { "file_thp", NR_FILE_THPS }, + { "shmem_thp", NR_SHMEM_THPS }, #endif - { "inactive_anon", PAGE_SIZE, NR_INACTIVE_ANON }, - { "active_anon", PAGE_SIZE, NR_ACTIVE_ANON }, - { "inactive_file", PAGE_SIZE, NR_INACTIVE_FILE }, - { "active_file", PAGE_SIZE, NR_ACTIVE_FILE }, - { "unevictable", PAGE_SIZE, NR_UNEVICTABLE }, - - /* - * Note: The slab_reclaimable and slab_unreclaimable must be - * together and slab_reclaimable must be in front. - */ - { "slab_reclaimable", 1, NR_SLAB_RECLAIMABLE_B }, - { "slab_unreclaimable", 1, NR_SLAB_UNRECLAIMABLE_B }, + { "inactive_anon", NR_INACTIVE_ANON }, + { "active_anon", NR_ACTIVE_ANON }, + { "inactive_file", NR_INACTIVE_FILE }, + { "active_file", NR_ACTIVE_FILE }, + { "unevictable", NR_UNEVICTABLE }, + { "slab_reclaimable", NR_SLAB_RECLAIMABLE_B }, + { "slab_unreclaimable", NR_SLAB_UNRECLAIMABLE_B }, /* The memory events */ - { "workingset_refault_anon", 1, WORKINGSET_REFAULT_ANON }, - { "workingset_refault_file", 1, WORKINGSET_REFAULT_FILE }, - { "workingset_activate_anon", 1, WORKINGSET_ACTIVATE_ANON }, - { "workingset_activate_file", 1, WORKINGSET_ACTIVATE_FILE }, - { "workingset_restore_anon", 1, WORKINGSET_RESTORE_ANON }, - { "workingset_restore_file", 1, WORKINGSET_RESTORE_FILE }, - { "workingset_nodereclaim", 1, WORKINGSET_NODERECLAIM }, + { "workingset_refault_anon", WORKINGSET_REFAULT_ANON }, + { "workingset_refault_file", WORKINGSET_REFAULT_FILE }, + { "workingset_activate_anon", WORKINGSET_ACTIVATE_ANON }, + { "workingset_activate_file", WORKINGSET_ACTIVATE_FILE }, + { "workingset_restore_anon", WORKINGSET_RESTORE_ANON }, + { "workingset_restore_file", WORKINGSET_RESTORE_FILE }, + { "workingset_nodereclaim", WORKINGSET_NODERECLAIM }, }; +/* Translate stat items to the correct unit for memory.stat output */ +static int memcg_page_state_unit(int item) +{ + int unit; + + switch (item) { + case WORKINGSET_REFAULT_ANON: + case WORKINGSET_REFAULT_FILE: + case WORKINGSET_ACTIVATE_ANON: + case WORKINGSET_ACTIVATE_FILE: + case WORKINGSET_RESTORE_ANON: + case WORKINGSET_RESTORE_FILE: + case WORKINGSET_NODERECLAIM: + unit = 1; + break; + default: + unit = memcg_stat_item_in_bytes(item) ? 1 : PAGE_SIZE; + break; + } + + return unit; +} + +static inline unsigned long memcg_page_state_output(struct mem_cgroup *memcg, + int item) +{ + return memcg_page_state(memcg, item) * memcg_page_state_unit(item); +} + static char *memory_stat_format(struct mem_cgroup *memcg) { struct seq_buf s; @@ -1558,13 +1581,16 @@ static char *memory_stat_format(struct mem_cgroup *memcg) for (i = 0; i < ARRAY_SIZE(memory_stats); i++) { u64 size; - size = memcg_page_state(memcg, memory_stats[i].idx); - size *= memory_stats[i].ratio; + size = memcg_page_state_output(memcg, memory_stats[i].idx); seq_buf_printf(&s, "%s %llu\n", memory_stats[i].name, size); + /* + * We are printing reclaimable, unreclaimable of the slab + * and the sum of both. + */ if (unlikely(memory_stats[i].idx == NR_SLAB_UNRECLAIMABLE_B)) { - size = memcg_page_state(memcg, NR_SLAB_RECLAIMABLE_B) + - memcg_page_state(memcg, NR_SLAB_UNRECLAIMABLE_B); + size += memcg_page_state_output(memcg, + NR_SLAB_RECLAIMABLE_B); seq_buf_printf(&s, "slab %llu\n", size); } } @@ -6358,6 +6384,12 @@ static int memory_stat_show(struct seq_file *m, void *v) } #ifdef CONFIG_NUMA +static inline unsigned long lruvec_page_state_output(struct lruvec *lruvec, + int item) +{ + return lruvec_page_state(lruvec, item) * memcg_page_state_unit(item); +} + static int memory_numa_stat_show(struct seq_file *m, void *v) { int i; @@ -6375,8 +6407,8 @@ static int memory_numa_stat_show(struct seq_file *m, void *v) struct lruvec *lruvec; lruvec = mem_cgroup_lruvec(memcg, NODE_DATA(nid)); - size = lruvec_page_state(lruvec, memory_stats[i].idx); - size *= memory_stats[i].ratio; + size = lruvec_page_state_output(lruvec, + memory_stats[i].idx); seq_printf(m, " N%d=%llu", nid, size); } seq_putc(m, '\n'); From patchwork Sun Dec 6 10:14:51 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 11953907 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6BCECC4361B for ; Sun, 6 Dec 2020 10:18:42 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 162DB230FB for ; Sun, 6 Dec 2020 10:18:42 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 162DB230FB Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id B17636B0073; Sun, 6 Dec 2020 05:18:41 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id AC6388D0007; Sun, 6 Dec 2020 05:18:41 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9DB468D0006; Sun, 6 Dec 2020 05:18:41 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0115.hostedemail.com [216.40.44.115]) by kanga.kvack.org (Postfix) with ESMTP id 87C206B0073 for ; Sun, 6 Dec 2020 05:18:41 -0500 (EST) Received: from smtpin05.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 5737A180AD806 for ; Sun, 6 Dec 2020 10:18:41 +0000 (UTC) X-FDA: 77562458442.05.kitty19_0d11749273d4 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin05.hostedemail.com (Postfix) with ESMTP id 37F8B18001BE2 for ; Sun, 6 Dec 2020 10:18:41 +0000 (UTC) X-HE-Tag: kitty19_0d11749273d4 X-Filterd-Recvd-Size: 13532 Received: from mail-pf1-f194.google.com (mail-pf1-f194.google.com [209.85.210.194]) by imf36.hostedemail.com (Postfix) with ESMTP for ; Sun, 6 Dec 2020 10:18:40 +0000 (UTC) Received: by mail-pf1-f194.google.com with SMTP id 131so7001466pfb.9 for ; Sun, 06 Dec 2020 02:18:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=lP6NOXY9P1Y1tdnLUE8WJ9oCZQOprYuVn3yHjkNbfog=; b=ZcB+MrFfITKTQD6yHNcSXXGhcuAfJVfq9Vc0JYQrQO8x+y4QfNb3GbTp1Bw/q0Xh1C SRPp2uOrMxEWh2MTi3HsJw/K2uDSI7BPt7W9VaUyVAlrworEQCgS3w0i4ICgyZ0NW2Zx rpIPH2IBLP/NecuXoWwQwHU8HJfMMqjmeilA0qISzpe3Ua5VcpfB1uHFE8rwuzE/+BBu dU61wVmkRUSnIPY6q3zmuFpZhfYSc1xyARC+ZWy/++zR4K0ULoqC3Ec/hYdfb+kc7cKa 6FL91dJW9nMFXkchdG699Ol55L2kOxnfCBh+PsPo6Ib+GHG2WmGfHHoPHmJdZUhbgFZi R05g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=lP6NOXY9P1Y1tdnLUE8WJ9oCZQOprYuVn3yHjkNbfog=; b=GZDBTiYXHjBQIUrbXHoLdZaisasVvL9fGPEMZUKua8JChCtBvHAECCFJjX6s3Blh6X 0fY2fOWW6OVJu5Xhl5hRaYoL5wNGWUQxPv/SeswI0AiuMTkqZAWTmcvqIAEhwEtkgMPB gH2ZlKZIu989B406b22hnes88r/w0Li10F3UecylKOHX0Z+iruAFWEU63dlCwkucr93K PXGTddsEpX9nmfDRiob/beuW2eDQ51Ilp3NxqzGQx4rWQUKwMO5+8Lmp9q+OV8E5/j+y Z5sShgZBDEbPT7hd5I/z8U36xnziBBD1Tm8FLCSVXqrHTj7ikrqVXJwNHdQYHKsmNRs2 kekw== X-Gm-Message-State: AOAM530urFjw9IVwDq1W42T5B1ynstO0Xhf1+PzmUWvNeX/4qiTUOBA0 AsvHhasTaNsTNm+fvUYGtKVkkw== X-Google-Smtp-Source: ABdhPJyTR5IEC8wjJwZanqPdp0TYqYuwBV1k98LMJDgi/aO+lbtgHvOt9jKwuJZe7B/Wxcytd89bHw== X-Received: by 2002:aa7:991c:0:b029:19d:b276:1ecb with SMTP id z28-20020aa7991c0000b029019db2761ecbmr11518358pff.73.1607249919971; Sun, 06 Dec 2020 02:18:39 -0800 (PST) Received: from localhost.localdomain ([103.136.221.70]) by smtp.gmail.com with ESMTPSA id g16sm10337657pfb.201.2020.12.06.02.18.31 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 06 Dec 2020 02:18:39 -0800 (PST) From: Muchun Song To: gregkh@linuxfoundation.org, rafael@kernel.org, adobriyan@gmail.com, akpm@linux-foundation.org, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, hughd@google.com, will@kernel.org, guro@fb.com, rppt@kernel.org, tglx@linutronix.de, esyr@redhat.com, peterx@redhat.com, krisman@collabora.com, surenb@google.com, avagin@openvz.org, elver@google.com, rdunlap@infradead.org, iamjoonsoo.kim@lge.com Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, Muchun Song Subject: [RESEND PATCH v2 12/12] mm: memcontrol: remove {global_}node_page_state_pages Date: Sun, 6 Dec 2020 18:14:51 +0800 Message-Id: <20201206101451.14706-13-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20201206101451.14706-1-songmuchun@bytedance.com> References: <20201206101451.14706-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Now the unit of the vmstat counters are either pages or bytes. So we can adjust the node_page_state to always returns values in pages and remove the node_page_state_pages. Signed-off-by: Muchun Song --- drivers/base/node.c | 10 +++++----- fs/proc/meminfo.c | 12 ++++++------ include/linux/vmstat.h | 17 +---------------- kernel/power/snapshot.c | 2 +- mm/oom_kill.c | 2 +- mm/page_alloc.c | 10 +++++----- mm/vmscan.c | 2 +- mm/vmstat.c | 23 ++++++----------------- 8 files changed, 26 insertions(+), 52 deletions(-) diff --git a/drivers/base/node.c b/drivers/base/node.c index bc01ce0b2fcd..42298e3552e5 100644 --- a/drivers/base/node.c +++ b/drivers/base/node.c @@ -374,8 +374,8 @@ static ssize_t node_read_meminfo(struct device *dev, unsigned long sreclaimable, sunreclaimable; si_meminfo_node(&i, nid); - sreclaimable = node_page_state_pages(pgdat, NR_SLAB_RECLAIMABLE_B); - sunreclaimable = node_page_state_pages(pgdat, NR_SLAB_UNRECLAIMABLE_B); + sreclaimable = node_page_state(pgdat, NR_SLAB_RECLAIMABLE_B); + sunreclaimable = node_page_state(pgdat, NR_SLAB_UNRECLAIMABLE_B); len = sysfs_emit_at(buf, len, "Node %d MemTotal: %8lu kB\n" "Node %d MemFree: %8lu kB\n" @@ -446,9 +446,9 @@ static ssize_t node_read_meminfo(struct device *dev, nid, K(node_page_state(pgdat, NR_FILE_MAPPED)), nid, K(node_page_state(pgdat, NR_ANON_MAPPED)), nid, K(i.sharedram), - nid, node_page_state(pgdat, NR_KERNEL_STACK_B) / SZ_1K, + nid, K(node_page_state(pgdat, NR_KERNEL_STACK_B)), #ifdef CONFIG_SHADOW_CALL_STACK - nid, node_page_state(pgdat, NR_KERNEL_SCS_B) / SZ_1K, + nid, K(node_page_state(pgdat, NR_KERNEL_SCS_B)), #endif nid, K(sum_zone_node_page_state(nid, NR_PAGETABLE)), nid, 0UL, @@ -517,7 +517,7 @@ static ssize_t node_read_vmstat(struct device *dev, for (i = 0; i < NR_VM_NODE_STAT_ITEMS; i++) len += sysfs_emit_at(buf, len, "%s %lu\n", node_stat_name(i), - node_page_state_pages(pgdat, i)); + node_page_state(pgdat, i)); return len; } diff --git a/fs/proc/meminfo.c b/fs/proc/meminfo.c index 69895e83d4fc..95ea5f062161 100644 --- a/fs/proc/meminfo.c +++ b/fs/proc/meminfo.c @@ -52,8 +52,8 @@ static int meminfo_proc_show(struct seq_file *m, void *v) pages[lru] = global_node_page_state(NR_LRU_BASE + lru); available = si_mem_available(); - sreclaimable = global_node_page_state_pages(NR_SLAB_RECLAIMABLE_B); - sunreclaim = global_node_page_state_pages(NR_SLAB_UNRECLAIMABLE_B); + sreclaimable = global_node_page_state(NR_SLAB_RECLAIMABLE_B); + sunreclaim = global_node_page_state(NR_SLAB_UNRECLAIMABLE_B); show_val_kb(m, "MemTotal: ", i.totalram); show_val_kb(m, "MemFree: ", i.freeram); @@ -100,11 +100,11 @@ static int meminfo_proc_show(struct seq_file *m, void *v) show_val_kb(m, "Slab: ", sreclaimable + sunreclaim); show_val_kb(m, "SReclaimable: ", sreclaimable); show_val_kb(m, "SUnreclaim: ", sunreclaim); - seq_printf(m, "KernelStack: %8lu kB\n", - global_node_page_state(NR_KERNEL_STACK_B) / SZ_1K); + show_val_kb(m, "KernelStack: ", + global_node_page_state(NR_KERNEL_STACK_B)); #ifdef CONFIG_SHADOW_CALL_STACK - seq_printf(m, "ShadowCallStack:%8lu kB\n", - global_node_page_state(NR_KERNEL_SCS_B) / SZ_1K); + show_val_kb(m, "ShadowCallStack:", + global_node_page_state(NR_KERNEL_SCS_B)); #endif show_val_kb(m, "PageTables: ", global_zone_page_state(NR_PAGETABLE)); diff --git a/include/linux/vmstat.h b/include/linux/vmstat.h index afd84dc2398c..ae821e016fdd 100644 --- a/include/linux/vmstat.h +++ b/include/linux/vmstat.h @@ -193,8 +193,7 @@ static inline unsigned long global_zone_page_state(enum zone_stat_item item) return x; } -static inline -unsigned long global_node_page_state_pages(enum node_stat_item item) +static inline unsigned long global_node_page_state(enum node_stat_item item) { long x = atomic_long_read(&vm_node_stat[item]); @@ -207,17 +206,6 @@ unsigned long global_node_page_state_pages(enum node_stat_item item) return x; } -static inline unsigned long global_node_page_state(enum node_stat_item item) -{ - long x = atomic_long_read(&vm_node_stat[item]); - -#ifdef CONFIG_SMP - if (x < 0) - x = 0; -#endif - return x; -} - static inline unsigned long zone_page_state(struct zone *zone, enum zone_stat_item item) { @@ -258,12 +246,9 @@ extern unsigned long sum_zone_node_page_state(int node, extern unsigned long sum_zone_numa_state(int node, enum numa_stat_item item); extern unsigned long node_page_state(struct pglist_data *pgdat, enum node_stat_item item); -extern unsigned long node_page_state_pages(struct pglist_data *pgdat, - enum node_stat_item item); #else #define sum_zone_node_page_state(node, item) global_zone_page_state(item) #define node_page_state(node, item) global_node_page_state(item) -#define node_page_state_pages(node, item) global_node_page_state_pages(item) #endif /* CONFIG_NUMA */ #ifdef CONFIG_SMP diff --git a/kernel/power/snapshot.c b/kernel/power/snapshot.c index d63560e1cf87..664520bdaa20 100644 --- a/kernel/power/snapshot.c +++ b/kernel/power/snapshot.c @@ -1705,7 +1705,7 @@ static unsigned long minimum_image_size(unsigned long saveable) { unsigned long size; - size = global_node_page_state_pages(NR_SLAB_RECLAIMABLE_B) + size = global_node_page_state(NR_SLAB_RECLAIMABLE_B) + global_node_page_state(NR_ACTIVE_ANON) + global_node_page_state(NR_INACTIVE_ANON) + global_node_page_state(NR_ACTIVE_FILE) diff --git a/mm/oom_kill.c b/mm/oom_kill.c index 04b19b7b5435..73861473c7d4 100644 --- a/mm/oom_kill.c +++ b/mm/oom_kill.c @@ -188,7 +188,7 @@ static bool should_dump_unreclaim_slab(void) global_node_page_state(NR_ISOLATED_FILE) + global_node_page_state(NR_UNEVICTABLE); - return (global_node_page_state_pages(NR_SLAB_UNRECLAIMABLE_B) > nr_lru); + return (global_node_page_state(NR_SLAB_UNRECLAIMABLE_B) > nr_lru); } /** diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 58916b3afdab..d16c9388c0b8 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -5372,7 +5372,7 @@ long si_mem_available(void) * items that are in use, and cannot be freed. Cap this estimate at the * low watermark. */ - reclaimable = global_node_page_state_pages(NR_SLAB_RECLAIMABLE_B) + + reclaimable = global_node_page_state(NR_SLAB_RECLAIMABLE_B) + global_node_page_state(NR_KERNEL_MISC_RECLAIMABLE); available += reclaimable - min(reclaimable / 2, wmark_low); @@ -5516,8 +5516,8 @@ void show_free_areas(unsigned int filter, nodemask_t *nodemask) global_node_page_state(NR_UNEVICTABLE), global_node_page_state(NR_FILE_DIRTY), global_node_page_state(NR_WRITEBACK), - global_node_page_state_pages(NR_SLAB_RECLAIMABLE_B), - global_node_page_state_pages(NR_SLAB_UNRECLAIMABLE_B), + global_node_page_state(NR_SLAB_RECLAIMABLE_B), + global_node_page_state(NR_SLAB_UNRECLAIMABLE_B), global_node_page_state(NR_FILE_MAPPED), global_node_page_state(NR_SHMEM), global_zone_page_state(NR_PAGETABLE), @@ -5572,9 +5572,9 @@ void show_free_areas(unsigned int filter, nodemask_t *nodemask) K(node_page_state(pgdat, NR_ANON_THPS)), #endif K(node_page_state(pgdat, NR_WRITEBACK_TEMP)), - node_page_state(pgdat, NR_KERNEL_STACK_B) / SZ_1K, + K(node_page_state(pgdat, NR_KERNEL_STACK_B)), #ifdef CONFIG_SHADOW_CALL_STACK - node_page_state(pgdat, NR_KERNEL_SCS_B) / SZ_1K, + K(node_page_state(pgdat, NR_KERNEL_SCS_B)), #endif pgdat->kswapd_failures >= MAX_RECLAIM_RETRIES ? "yes" : "no"); diff --git a/mm/vmscan.c b/mm/vmscan.c index 469016222cdb..5d3c8fa68979 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -4220,7 +4220,7 @@ int node_reclaim(struct pglist_data *pgdat, gfp_t gfp_mask, unsigned int order) * unmapped file backed pages. */ if (node_pagecache_reclaimable(pgdat) <= pgdat->min_unmapped_pages && - node_page_state_pages(pgdat, NR_SLAB_RECLAIMABLE_B) <= + node_page_state(pgdat, NR_SLAB_RECLAIMABLE_B) <= pgdat->min_slab_pages) return NODE_RECLAIM_FULL; diff --git a/mm/vmstat.c b/mm/vmstat.c index 25751b1d8e2e..b7cdef585efd 100644 --- a/mm/vmstat.c +++ b/mm/vmstat.c @@ -1000,22 +1000,9 @@ unsigned long sum_zone_numa_state(int node, } /* - * Determine the per node value of a stat item. + * Determine the per node value of a stat item. This always returns + * values in pages. */ -unsigned long node_page_state_pages(struct pglist_data *pgdat, - enum node_stat_item item) -{ - long x = atomic_long_read(&pgdat->vm_stat[item]); - -#ifdef CONFIG_SMP - if (x < 0) - x = 0; -#endif - if (vmstat_item_in_bytes(item)) - x >>= PAGE_SHIFT; - return x; -} - unsigned long node_page_state(struct pglist_data *pgdat, enum node_stat_item item) { @@ -1025,6 +1012,8 @@ unsigned long node_page_state(struct pglist_data *pgdat, if (x < 0) x = 0; #endif + if (vmstat_item_in_bytes(item)) + x >>= PAGE_SHIFT; return x; } #endif @@ -1626,7 +1615,7 @@ static void zoneinfo_show_print(struct seq_file *m, pg_data_t *pgdat, seq_printf(m, "\n per-node stats"); for (i = 0; i < NR_VM_NODE_STAT_ITEMS; i++) { seq_printf(m, "\n %-12s %lu", node_stat_name(i), - node_page_state_pages(pgdat, i)); + node_page_state(pgdat, i)); } } seq_printf(m, @@ -1747,7 +1736,7 @@ static void *vmstat_start(struct seq_file *m, loff_t *pos) #endif for (i = 0; i < NR_VM_NODE_STAT_ITEMS; i++) - v[i] = global_node_page_state_pages(i); + v[i] = global_node_page_state(i); v += NR_VM_NODE_STAT_ITEMS; global_dirty_limits(v + NR_DIRTY_BG_THRESHOLD,