From patchwork Sat Dec 5 13:02:16 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 11953163 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8779BC4361A for ; Sat, 5 Dec 2020 15:42:45 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5AAA322C9F for ; Sat, 5 Dec 2020 15:42:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726950AbgLEPkd (ORCPT ); Sat, 5 Dec 2020 10:40:33 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42108 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727301AbgLEP3O (ORCPT ); Sat, 5 Dec 2020 10:29:14 -0500 Received: from mail-pf1-x442.google.com (mail-pf1-x442.google.com [IPv6:2607:f8b0:4864:20::442]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3DCC0C094242 for ; Sat, 5 Dec 2020 05:02:52 -0800 (PST) Received: by mail-pf1-x442.google.com with SMTP id q10so5764958pfn.0 for ; Sat, 05 Dec 2020 05:02:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=CowKSBe21ybUCNZpbXy3BdqlRA0xnHhkE4QdTXQDr3k=; b=1E/sfXgHaC4PaGNOQGTv5UbJpGKRfVFVfNwfoMAvioxro0OeUnyGU7+KbNrPOmzCJB Uc+HeD8TjLwjqb5MX1YfYNo3gGGM7+9qKPJNif8ipbHCrN6r7rYKmOi8fBHe67OyeSBy Y1P4LecS8PHQ1OnnbNTZD/cwAKl6OFG9NvKKfrPw4bK9RC4BPsZNj48E9G7vqAjSylgL 5pvEUV3NYJodopVNMOcvf6Kls5MhenBu7FAvV7gfzrSFhzJ2wLUsvGk6BhmB+JtCXpVw obRyNkYZAykzB3ugmrBdd4T4Vi3xL+S4QOJd9QNRYELJOY2AWIKE8QOyBmaBnF3al2AA yeLA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=CowKSBe21ybUCNZpbXy3BdqlRA0xnHhkE4QdTXQDr3k=; b=CsTzSEuKup1+pnJxOBu+LU6QeHlXdPWnPzeceB6PI6Abs3WIvkimvXP3YSSspOvmJ0 6AcgCEHolhvTK7R9/6CGsMFiXFzY7BUbh9gXB8jcsncN/xiRWvU9GVl4LneT5QJzZXiE QH3SuTRw/9pMpyekr7Bee92hK1EivIPUC6vpNRZlh9xgTdRlrSH1sjqRnrv5e9+eU0hK br+a80SRLBeoXl662mBj7flIwdBF9MH+/B2JsplixE1Qk3xn5uuOkpFWNTvY6Fl5LdRw 0/O2T+fjAhQPZq5CnBPIXiQZwMBeUhdfDQrnaI12VwjhPeHUBz5w3Lr6jyO9Vvepr4xX yo3g== X-Gm-Message-State: AOAM532y9du6rKtZd77JQtKCA2EPh7iVrWuyDC5Irea5pV3V34CHrBxi Mm0yEyZvWDSEIkZWbmDXWF5jiA== X-Google-Smtp-Source: ABdhPJx+y6G4rm2gt0gKnr+PKMwRmh0AURFtJ3VRUjZceruiOVQhaXh+OwSXp+cpSpnljOk2jlgDNQ== X-Received: by 2002:a63:2202:: with SMTP id i2mr11434673pgi.63.1607173371732; Sat, 05 Dec 2020 05:02:51 -0800 (PST) Received: from localhost.bytedance.net ([103.136.220.120]) by smtp.gmail.com with ESMTPSA id kb12sm5047790pjb.2.2020.12.05.05.02.43 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sat, 05 Dec 2020 05:02:51 -0800 (PST) From: Muchun Song To: gregkh@linuxfoundation.org, rafael@kernel.org, adobriyan@gmail.com, akpm@linux-foundation.org, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, hughd@google.com, will@kernel.org, guro@fb.com, rppt@kernel.org, tglx@linutronix.de, esyr@redhat.com, peterx@redhat.com, krisman@collabora.com, surenb@google.com, avagin@openvz.org, elver@google.com, rdunlap@infradead.org, iamjoonsoo.kim@lge.com Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, Muchun Song Subject: [PATCH 1/9] mm: vmstat: fix stat_threshold for NR_KERNEL_STACK_KB Date: Sat, 5 Dec 2020 21:02:16 +0800 Message-Id: <20201205130224.81607-2-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20201205130224.81607-1-songmuchun@bytedance.com> References: <20201205130224.81607-1-songmuchun@bytedance.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org The kernel stack is being accounted in KiB not page, so the stat_threshold should also adjust to byte. Signed-off-by: Muchun Song --- mm/vmstat.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/mm/vmstat.c b/mm/vmstat.c index 8d77ee426e22..f7857a7052e4 100644 --- a/mm/vmstat.c +++ b/mm/vmstat.c @@ -353,6 +353,8 @@ void __mod_node_page_state(struct pglist_data *pgdat, enum node_stat_item item, x = delta + __this_cpu_read(*p); t = __this_cpu_read(pcp->stat_threshold); + if (unlikely(item == NR_KERNEL_STACK_KB)) + t <<= PAGE_SHIFT; if (unlikely(abs(x) > t)) { node_page_state_add(x, pgdat, item); @@ -573,6 +575,8 @@ static inline void mod_node_state(struct pglist_data *pgdat, * for all cpus in a node. */ t = this_cpu_read(pcp->stat_threshold); + if (unlikely(item == NR_KERNEL_STACK_KB)) + t <<= PAGE_SHIFT; o = this_cpu_read(*p); n = delta + o; From patchwork Sat Dec 5 13:02:17 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 11953165 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1660BC4361A for ; Sat, 5 Dec 2020 15:47:44 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id DD9FB2312B for ; Sat, 5 Dec 2020 15:47:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726515AbgLEPmo (ORCPT ); Sat, 5 Dec 2020 10:42:44 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42098 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727353AbgLEP3O (ORCPT ); Sat, 5 Dec 2020 10:29:14 -0500 Received: from mail-pj1-x1043.google.com (mail-pj1-x1043.google.com [IPv6:2607:f8b0:4864:20::1043]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EF8D9C09424E for ; Sat, 5 Dec 2020 05:03:00 -0800 (PST) Received: by mail-pj1-x1043.google.com with SMTP id b5so893231pjl.0 for ; Sat, 05 Dec 2020 05:03:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=JLGZC7sv5nAUpuhCShqzZAhxLbvYoPKLszNPrb+RhvY=; b=iXNSJ08wIZGakHjgbIcjXIwKHGYw6UmpSfo0Eb6nim+XGWMNO6A9YVj0zeZ0dYSQFb VCVr64gPtj6hVu4sogDtwUSNROzxv6XvOqYT+pBJFDMLmdj3FCafR6WDEUGrUGpa/Gry XXAUfe11y0QLCmPB2oAJqeUXRXxHhQFQISCUJ2uu0NDV/c2iML7Jx2DS5LxoZdVrz+Dt JWQKZovT5QiiunDzry2OtZZ0K6Qd39WLeMyA2wk/HngpFlpKfv9fU/7JTGbFE+lpi8zb 9xP0mmYdGoGIHA3F9+ShWXuADJv7UxVumiQ3BAl7ffDiAPAwAgcfHt+lMzY/FNoFgLKl JnvQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=JLGZC7sv5nAUpuhCShqzZAhxLbvYoPKLszNPrb+RhvY=; b=Bnvg97gpAsEctf1PER29PBT+hTrxeTT/FxFOFZuaR9Chk9PaNlwNKh7dMd3NBpRz3R 3SJpaYMh6SLnI0z5dNLwlVJxDWV9708V34g0zsUY6Ga6pifV9ToM5ueAF72Kx6nlrsXp EHFXS+UMZN0+wwpEbac4Ii+M9vun8uFEoYlyn8+IwBOnlSxL49soR+vvALTWH+Sr49ZY hMxcNjLqtmi2b9U6L77Wk3MSb4QE64JyfRfKz51yAIG25JmNRzMxPT+dETmZvLFZS31e C9MnhlBUMKqqTg48L/Git0NvqFctxb6wxIuAVkgOKB8axYIPALqGD6Llh7pnFEXvJFTl biaw== X-Gm-Message-State: AOAM530GnkTUFa+LaYOtBzCc311UUmqPAa7ImWo/PHIfBFhR5R2kjqiJ 59EmlVMejm6IbZVRwoQepKIT0Q== X-Google-Smtp-Source: ABdhPJw6o9CKoD5YBzWC7VW/NxM7IRg3JXfDBPySbZAj0r3+Q0HFX4zEhlsXQ8gE6TyP2+R2ML2Aow== X-Received: by 2002:a17:902:b58a:b029:d7:d45c:481c with SMTP id a10-20020a170902b58ab02900d7d45c481cmr8074051pls.55.1607173380527; Sat, 05 Dec 2020 05:03:00 -0800 (PST) Received: from localhost.bytedance.net ([103.136.220.120]) by smtp.gmail.com with ESMTPSA id kb12sm5047790pjb.2.2020.12.05.05.02.52 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sat, 05 Dec 2020 05:02:59 -0800 (PST) From: Muchun Song To: gregkh@linuxfoundation.org, rafael@kernel.org, adobriyan@gmail.com, akpm@linux-foundation.org, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, hughd@google.com, will@kernel.org, guro@fb.com, rppt@kernel.org, tglx@linutronix.de, esyr@redhat.com, peterx@redhat.com, krisman@collabora.com, surenb@google.com, avagin@openvz.org, elver@google.com, rdunlap@infradead.org, iamjoonsoo.kim@lge.com Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, Muchun Song Subject: [PATCH 2/9] mm: memcontrol: fix NR_ANON_THPS account Date: Sat, 5 Dec 2020 21:02:17 +0800 Message-Id: <20201205130224.81607-3-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20201205130224.81607-1-songmuchun@bytedance.com> References: <20201205130224.81607-1-songmuchun@bytedance.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org The unit of NR_ANON_THPS is HPAGE_PMD_NR already. So it should inc/dec by one rather than nr_pages. Fixes: 468c398233da ("mm: memcontrol: switch to native NR_ANON_THPS counter") Signed-off-by: Muchun Song --- mm/memcontrol.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 22d9bd688d6d..695dedf8687a 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -5634,10 +5634,8 @@ static int mem_cgroup_move_account(struct page *page, __mod_lruvec_state(from_vec, NR_ANON_MAPPED, -nr_pages); __mod_lruvec_state(to_vec, NR_ANON_MAPPED, nr_pages); if (PageTransHuge(page)) { - __mod_lruvec_state(from_vec, NR_ANON_THPS, - -nr_pages); - __mod_lruvec_state(to_vec, NR_ANON_THPS, - nr_pages); + __dec_lruvec_state(from_vec, NR_ANON_THPS); + __inc_lruvec_state(to_vec, NR_ANON_THPS); } } From patchwork Sat Dec 5 13:02:18 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 11953271 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A71E1C4361B for ; Sat, 5 Dec 2020 18:33:02 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 642EC229C4 for ; Sat, 5 Dec 2020 18:33:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729230AbgLEScX (ORCPT ); Sat, 5 Dec 2020 13:32:23 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41970 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728722AbgLES3P (ORCPT ); Sat, 5 Dec 2020 13:29:15 -0500 Received: from mail-pg1-x543.google.com (mail-pg1-x543.google.com [IPv6:2607:f8b0:4864:20::543]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 51559C09424F for ; Sat, 5 Dec 2020 05:03:09 -0800 (PST) Received: by mail-pg1-x543.google.com with SMTP id e23so5287103pgk.12 for ; Sat, 05 Dec 2020 05:03:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ibu+HPexsNJ6GCHgTCs1EnvihRdwIFloguIfEoBcFm0=; b=RIWJqFEUtUvKu8Fg1Zcim0D0ygw/emSaDfHLDuSMZVi1l6jU98f0+j1WCvxvvrXmbM Zz7Sr/edGmZx8HWClxGoOS136rCnQ1gGAPZhN7ffez6dlZGHiW+lhfJkjQnXlF/DmcGV j3hVd1mClJVrhdcn5tFzL/2qmSGBKn5QetJBoFw03uBntP75pIbJLNCJCW5g8A95YVar h1pf9RM+JYZAqyk9myiNWcGDxI851yBOCBx8Uf5/huiuzSMioyS+Ke4ayojmQfH/XBos gO+Ml53OUJiZ8lF7EYb8uvp4uvrtkfNLiPCa5d/7BXjnzxxV8VQ0Gw4FOZU7yaEC3a0S 4IZQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ibu+HPexsNJ6GCHgTCs1EnvihRdwIFloguIfEoBcFm0=; b=FjAgaB07oqPGe8xavS+Pb4qZ6PvtlIcuxgOi5fxJ3a4Fb2Kj2vRtL/0viajnfZBDmC /ojQFJal9ToWCMhVQdB3ba997DWPzua5gmofQYZX98hewbQVTvi5aVhpNcXUOVYD90s9 BOW1ONKaDrIvQM4hLKZ/fdVa+CsDS2lD1HjlYfmnFQp0BpFps8L0GIqj1RaFDYYRr9GC 7Cym+k2s0SMUGYFx0MBR8trxHtCBz2Vkz0wjz5f4oy7juVP6FFEgxi35l0dc+9ByBGui 6gLDu74GPhU/jXScKE6WfZtwJkpyKd3qzU5QfBCzYtmtmCwAX8MNqrD12nqOs5f0T3A+ 5ELQ== X-Gm-Message-State: AOAM530/KVpEosTo1hd5OXJKOV7XBclywX4sqqOVxMH2X0yKs73s6kLt R7MFm8iB4YJK/0mV7XgX1sseQQ== X-Google-Smtp-Source: ABdhPJxGItJGcsCAEef158uYkZotzF/UVDU0ARQIri7AoCY66ltru3tqE9faWvDHTvvNRxMmQE/Xbw== X-Received: by 2002:a65:5944:: with SMTP id g4mr11607407pgu.446.1607173388919; Sat, 05 Dec 2020 05:03:08 -0800 (PST) Received: from localhost.bytedance.net ([103.136.220.120]) by smtp.gmail.com with ESMTPSA id kb12sm5047790pjb.2.2020.12.05.05.03.01 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sat, 05 Dec 2020 05:03:08 -0800 (PST) From: Muchun Song To: gregkh@linuxfoundation.org, rafael@kernel.org, adobriyan@gmail.com, akpm@linux-foundation.org, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, hughd@google.com, will@kernel.org, guro@fb.com, rppt@kernel.org, tglx@linutronix.de, esyr@redhat.com, peterx@redhat.com, krisman@collabora.com, surenb@google.com, avagin@openvz.org, elver@google.com, rdunlap@infradead.org, iamjoonsoo.kim@lge.com Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, Muchun Song Subject: [PATCH 3/9] mm: memcontrol: convert kernel stack account to byte-sized Date: Sat, 5 Dec 2020 21:02:18 +0800 Message-Id: <20201205130224.81607-4-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20201205130224.81607-1-songmuchun@bytedance.com> References: <20201205130224.81607-1-songmuchun@bytedance.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org The kernel stack account is the only one that counts in KiB. This patch convert it from KiB to byte. Signed-off-by: Muchun Song --- drivers/base/node.c | 2 +- fs/proc/meminfo.c | 2 +- include/linux/mmzone.h | 2 +- kernel/fork.c | 8 ++++---- mm/memcontrol.c | 2 +- mm/page_alloc.c | 2 +- mm/vmstat.c | 4 ++-- 7 files changed, 11 insertions(+), 11 deletions(-) diff --git a/drivers/base/node.c b/drivers/base/node.c index 6ffa470e2984..855886a6ba0e 100644 --- a/drivers/base/node.c +++ b/drivers/base/node.c @@ -446,7 +446,7 @@ static ssize_t node_read_meminfo(struct device *dev, nid, K(node_page_state(pgdat, NR_FILE_MAPPED)), nid, K(node_page_state(pgdat, NR_ANON_MAPPED)), nid, K(i.sharedram), - nid, node_page_state(pgdat, NR_KERNEL_STACK_KB), + nid, node_page_state(pgdat, NR_KERNEL_STACK_B) / 1024, #ifdef CONFIG_SHADOW_CALL_STACK nid, node_page_state(pgdat, NR_KERNEL_SCS_KB), #endif diff --git a/fs/proc/meminfo.c b/fs/proc/meminfo.c index 887a5532e449..c396b6cfba82 100644 --- a/fs/proc/meminfo.c +++ b/fs/proc/meminfo.c @@ -101,7 +101,7 @@ static int meminfo_proc_show(struct seq_file *m, void *v) show_val_kb(m, "SReclaimable: ", sreclaimable); show_val_kb(m, "SUnreclaim: ", sunreclaim); seq_printf(m, "KernelStack: %8lu kB\n", - global_node_page_state(NR_KERNEL_STACK_KB)); + global_node_page_state(NR_KERNEL_STACK_B) / 1024); #ifdef CONFIG_SHADOW_CALL_STACK seq_printf(m, "ShadowCallStack:%8lu kB\n", global_node_page_state(NR_KERNEL_SCS_KB)); diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index 15132adaa233..bd34416293ec 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -202,7 +202,7 @@ enum node_stat_item { NR_KERNEL_MISC_RECLAIMABLE, /* reclaimable non-slab kernel pages */ NR_FOLL_PIN_ACQUIRED, /* via: pin_user_page(), gup flag: FOLL_PIN */ NR_FOLL_PIN_RELEASED, /* pages returned via unpin_user_page() */ - NR_KERNEL_STACK_KB, /* measured in KiB */ + NR_KERNEL_STACK_B, /* measured in byte */ #if IS_ENABLED(CONFIG_SHADOW_CALL_STACK) NR_KERNEL_SCS_KB, /* measured in KiB */ #endif diff --git a/kernel/fork.c b/kernel/fork.c index 345f378e104d..2913d7c43dcb 100644 --- a/kernel/fork.c +++ b/kernel/fork.c @@ -382,11 +382,11 @@ static void account_kernel_stack(struct task_struct *tsk, int account) /* All stack pages are in the same node. */ if (vm) - mod_lruvec_page_state(vm->pages[0], NR_KERNEL_STACK_KB, - account * (THREAD_SIZE / 1024)); + mod_lruvec_page_state(vm->pages[0], NR_KERNEL_STACK_B, + account * THREAD_SIZE); else - mod_lruvec_kmem_state(stack, NR_KERNEL_STACK_KB, - account * (THREAD_SIZE / 1024)); + mod_lruvec_kmem_state(stack, NR_KERNEL_STACK_B, + account * THREAD_SIZE); } static int memcg_charge_kernel_stack(struct task_struct *tsk) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 695dedf8687a..a7ec79dcb7dc 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -1500,7 +1500,7 @@ struct memory_stat { static struct memory_stat memory_stats[] = { { "anon", PAGE_SIZE, NR_ANON_MAPPED }, { "file", PAGE_SIZE, NR_FILE_PAGES }, - { "kernel_stack", 1024, NR_KERNEL_STACK_KB }, + { "kernel_stack", 1, NR_KERNEL_STACK_B }, { "percpu", 1, MEMCG_PERCPU_B }, { "sock", PAGE_SIZE, MEMCG_SOCK }, { "shmem", PAGE_SIZE, NR_SHMEM }, diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 56e603eea1dd..c28f8e1f1ef6 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -5573,7 +5573,7 @@ void show_free_areas(unsigned int filter, nodemask_t *nodemask) K(node_page_state(pgdat, NR_ANON_THPS) * HPAGE_PMD_NR), #endif K(node_page_state(pgdat, NR_WRITEBACK_TEMP)), - node_page_state(pgdat, NR_KERNEL_STACK_KB), + node_page_state(pgdat, NR_KERNEL_STACK_B) / 1024, #ifdef CONFIG_SHADOW_CALL_STACK node_page_state(pgdat, NR_KERNEL_SCS_KB), #endif diff --git a/mm/vmstat.c b/mm/vmstat.c index f7857a7052e4..3e3bcaf7ba7e 100644 --- a/mm/vmstat.c +++ b/mm/vmstat.c @@ -353,7 +353,7 @@ void __mod_node_page_state(struct pglist_data *pgdat, enum node_stat_item item, x = delta + __this_cpu_read(*p); t = __this_cpu_read(pcp->stat_threshold); - if (unlikely(item == NR_KERNEL_STACK_KB)) + if (unlikely(item == NR_KERNEL_STACK_B)) t <<= PAGE_SHIFT; if (unlikely(abs(x) > t)) { @@ -575,7 +575,7 @@ static inline void mod_node_state(struct pglist_data *pgdat, * for all cpus in a node. */ t = this_cpu_read(pcp->stat_threshold); - if (unlikely(item == NR_KERNEL_STACK_KB)) + if (unlikely(item == NR_KERNEL_STACK_B)) t <<= PAGE_SHIFT; o = this_cpu_read(*p); From patchwork Sat Dec 5 13:02:19 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 11953269 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 44800C4361B for ; Sat, 5 Dec 2020 18:32:23 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 10DC523340 for ; Sat, 5 Dec 2020 18:32:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728631AbgLESb5 (ORCPT ); Sat, 5 Dec 2020 13:31:57 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42020 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726851AbgLES3c (ORCPT ); Sat, 5 Dec 2020 13:29:32 -0500 Received: from mail-pl1-x643.google.com (mail-pl1-x643.google.com [IPv6:2607:f8b0:4864:20::643]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1527DC094252 for ; Sat, 5 Dec 2020 05:03:18 -0800 (PST) Received: by mail-pl1-x643.google.com with SMTP id x4so4658145pln.8 for ; Sat, 05 Dec 2020 05:03:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=e/ByzOD9hz0YZpM+oDBw2TLHq41qvMWpZTD0cqZGx4I=; b=EPbjwaB9bJDW0LzCf2yQYjATnBpm0Th60ZbMY9bsk2dPEMEfNS9bPgB7ty5q5wOzBB UiHdsfr9ogMWYH7TvvzqvrO6EoPVUMNK1x+1ydqTdreqCEu+/KpnzwSr0w73mgKDJGvk aSR5MzsuSDls159UCz7r42D8qQO+2u/g1NP4uciuI15nyxVO1pkuIM9G3tOWAtjcTqzk uOnFwiQfpKH13sRYjk/L7DMll/r1662AJqcNsxRL22WLA3xqWdJgYAeMUUZl0HR2bTDd ZEnm6lHxNfhzUNl6pyRHHeB29a8nUbf0+Zz2VjLBtAfGCI9mVhTBW6JycPut5UccvzLv 8ILg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=e/ByzOD9hz0YZpM+oDBw2TLHq41qvMWpZTD0cqZGx4I=; b=k/xkx69HWX3FSgRXJd8jBJRgzr16eiMa9H7PmsY+MQ8UGkBtKSCSLHpIjTXvykj149 u4wkXcYZI8JJNL1EHuWJ+i/h+v1H+iRkjeTPcjylS1adhhTPsQ/hZG2RJ5JxGre/NsYd jZK7ut25iccG4JqVQUX1oo/X56aTUURZvGdhYK5HNwnUlUmy5Qim0ricne56CNi7qLNV FpBTijQq/eJRbe7lJYrZCxOQUPBLv2sTlx2O0di81ap23Yp3XMetsSK4RUP+wPVh2adM OCpuEG68bdRbc+1rL9yuxU3k/aZbaq99iEY3RP2HdW2ry077IxTouGIc/EZRMXMhJc4E FofQ== X-Gm-Message-State: AOAM53013HtqtEFfIp7BKuB3pZMBebC0ZKRUXOUc8h9WWS+b3o5XSU4y /MsQ34HVPMkuz/4U1PVpvNgyyg== X-Google-Smtp-Source: ABdhPJzxxvs7f3dkKh/KtxLlbQ1Ivsj2ntK724skGrCgujhkDGeVXhFTomMBpOV1Wdd0fWXsK7Jm7A== X-Received: by 2002:a17:902:ed11:b029:da:3137:2695 with SMTP id b17-20020a170902ed11b02900da31372695mr8311259pld.1.1607173397620; Sat, 05 Dec 2020 05:03:17 -0800 (PST) Received: from localhost.bytedance.net ([103.136.220.120]) by smtp.gmail.com with ESMTPSA id kb12sm5047790pjb.2.2020.12.05.05.03.09 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sat, 05 Dec 2020 05:03:17 -0800 (PST) From: Muchun Song To: gregkh@linuxfoundation.org, rafael@kernel.org, adobriyan@gmail.com, akpm@linux-foundation.org, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, hughd@google.com, will@kernel.org, guro@fb.com, rppt@kernel.org, tglx@linutronix.de, esyr@redhat.com, peterx@redhat.com, krisman@collabora.com, surenb@google.com, avagin@openvz.org, elver@google.com, rdunlap@infradead.org, iamjoonsoo.kim@lge.com Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, Muchun Song Subject: [PATCH 4/9] mm: memcontrol: convert NR_ANON_THPS account to pages Date: Sat, 5 Dec 2020 21:02:19 +0800 Message-Id: <20201205130224.81607-5-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20201205130224.81607-1-songmuchun@bytedance.com> References: <20201205130224.81607-1-songmuchun@bytedance.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Convert the NR_ANON_THPS account to pages. Signed-off-by: Muchun Song --- drivers/base/node.c | 3 +-- fs/proc/meminfo.c | 2 +- mm/huge_memory.c | 3 ++- mm/memcontrol.c | 20 ++++++-------------- mm/page_alloc.c | 2 +- mm/rmap.c | 7 ++++--- 6 files changed, 15 insertions(+), 22 deletions(-) diff --git a/drivers/base/node.c b/drivers/base/node.c index 855886a6ba0e..05c369e93e16 100644 --- a/drivers/base/node.c +++ b/drivers/base/node.c @@ -461,8 +461,7 @@ static ssize_t node_read_meminfo(struct device *dev, nid, K(sunreclaimable) #ifdef CONFIG_TRANSPARENT_HUGEPAGE , - nid, K(node_page_state(pgdat, NR_ANON_THPS) * - HPAGE_PMD_NR), + nid, K(node_page_state(pgdat, NR_ANON_THPS)), nid, K(node_page_state(pgdat, NR_SHMEM_THPS) * HPAGE_PMD_NR), nid, K(node_page_state(pgdat, NR_SHMEM_PMDMAPPED) * diff --git a/fs/proc/meminfo.c b/fs/proc/meminfo.c index c396b6cfba82..f447ac191d24 100644 --- a/fs/proc/meminfo.c +++ b/fs/proc/meminfo.c @@ -129,7 +129,7 @@ static int meminfo_proc_show(struct seq_file *m, void *v) #ifdef CONFIG_TRANSPARENT_HUGEPAGE show_val_kb(m, "AnonHugePages: ", - global_node_page_state(NR_ANON_THPS) * HPAGE_PMD_NR); + global_node_page_state(NR_ANON_THPS)); show_val_kb(m, "ShmemHugePages: ", global_node_page_state(NR_SHMEM_THPS) * HPAGE_PMD_NR); show_val_kb(m, "ShmemPmdMapped: ", diff --git a/mm/huge_memory.c b/mm/huge_memory.c index c94dfc0cc1a3..1a13e1dab3ec 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -2178,7 +2178,8 @@ static void __split_huge_pmd_locked(struct vm_area_struct *vma, pmd_t *pmd, lock_page_memcg(page); if (atomic_add_negative(-1, compound_mapcount_ptr(page))) { /* Last compound_mapcount is gone. */ - __dec_lruvec_page_state(page, NR_ANON_THPS); + __mod_lruvec_page_state(page, NR_ANON_THPS, + -HPAGE_PMD_NR); if (TestClearPageDoubleMap(page)) { /* No need in mapcount reference anymore */ for (i = 0; i < HPAGE_PMD_NR; i++) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index a7ec79dcb7dc..39cb7f1b00d3 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -1513,7 +1513,7 @@ static struct memory_stat memory_stats[] = { * on some architectures, the macro of HPAGE_PMD_SIZE is not * constant(e.g. powerpc). */ - { "anon_thp", 0, NR_ANON_THPS }, + { "anon_thp", PAGE_SIZE, NR_ANON_THPS }, { "file_thp", 0, NR_FILE_THPS }, { "shmem_thp", 0, NR_SHMEM_THPS }, #endif @@ -1546,8 +1546,7 @@ static int __init memory_stats_init(void) for (i = 0; i < ARRAY_SIZE(memory_stats); i++) { #ifdef CONFIG_TRANSPARENT_HUGEPAGE - if (memory_stats[i].idx == NR_ANON_THPS || - memory_stats[i].idx == NR_FILE_THPS || + if (memory_stats[i].idx == NR_FILE_THPS || memory_stats[i].idx == NR_SHMEM_THPS) memory_stats[i].ratio = HPAGE_PMD_SIZE; #endif @@ -4069,10 +4068,6 @@ static int memcg_stat_show(struct seq_file *m, void *v) if (memcg1_stats[i] == MEMCG_SWAP && !do_memsw_account()) continue; nr = memcg_page_state_local(memcg, memcg1_stats[i]); -#ifdef CONFIG_TRANSPARENT_HUGEPAGE - if (memcg1_stats[i] == NR_ANON_THPS) - nr *= HPAGE_PMD_NR; -#endif seq_printf(m, "%s %lu\n", memcg1_stat_names[i], nr * PAGE_SIZE); } @@ -4103,10 +4098,6 @@ static int memcg_stat_show(struct seq_file *m, void *v) if (memcg1_stats[i] == MEMCG_SWAP && !do_memsw_account()) continue; nr = memcg_page_state(memcg, memcg1_stats[i]); -#ifdef CONFIG_TRANSPARENT_HUGEPAGE - if (memcg1_stats[i] == NR_ANON_THPS) - nr *= HPAGE_PMD_NR; -#endif seq_printf(m, "total_%s %llu\n", memcg1_stat_names[i], (u64)nr * PAGE_SIZE); } @@ -5634,10 +5625,11 @@ static int mem_cgroup_move_account(struct page *page, __mod_lruvec_state(from_vec, NR_ANON_MAPPED, -nr_pages); __mod_lruvec_state(to_vec, NR_ANON_MAPPED, nr_pages); if (PageTransHuge(page)) { - __dec_lruvec_state(from_vec, NR_ANON_THPS); - __inc_lruvec_state(to_vec, NR_ANON_THPS); + __mod_lruvec_state(from_vec, NR_ANON_THPS, + -nr_pages); + __mod_lruvec_state(to_vec, NR_ANON_THPS, + nr_pages); } - } } else { __mod_lruvec_state(from_vec, NR_FILE_PAGES, -nr_pages); diff --git a/mm/page_alloc.c b/mm/page_alloc.c index c28f8e1f1ef6..fabdbb340806 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -5570,7 +5570,7 @@ void show_free_areas(unsigned int filter, nodemask_t *nodemask) K(node_page_state(pgdat, NR_SHMEM_THPS) * HPAGE_PMD_NR), K(node_page_state(pgdat, NR_SHMEM_PMDMAPPED) * HPAGE_PMD_NR), - K(node_page_state(pgdat, NR_ANON_THPS) * HPAGE_PMD_NR), + K(node_page_state(pgdat, NR_ANON_THPS)), #endif K(node_page_state(pgdat, NR_WRITEBACK_TEMP)), node_page_state(pgdat, NR_KERNEL_STACK_B) / 1024, diff --git a/mm/rmap.c b/mm/rmap.c index 08c56aaf72eb..f59e92e26b61 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -1144,7 +1144,8 @@ void do_page_add_anon_rmap(struct page *page, * disabled. */ if (compound) - __inc_lruvec_page_state(page, NR_ANON_THPS); + __mod_lruvec_page_state(page, NR_ANON_THPS, + HPAGE_PMD_NR); __mod_lruvec_page_state(page, NR_ANON_MAPPED, nr); } @@ -1186,7 +1187,7 @@ void page_add_new_anon_rmap(struct page *page, if (hpage_pincount_available(page)) atomic_set(compound_pincount_ptr(page), 0); - __inc_lruvec_page_state(page, NR_ANON_THPS); + __mod_lruvec_page_state(page, NR_ANON_THPS, HPAGE_PMD_NR); } else { /* Anon THP always mapped first with PMD */ VM_BUG_ON_PAGE(PageTransCompound(page), page); @@ -1292,7 +1293,7 @@ static void page_remove_anon_compound_rmap(struct page *page) if (!IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE)) return; - __dec_lruvec_page_state(page, NR_ANON_THPS); + __mod_lruvec_page_state(page, NR_ANON_THPS, -HPAGE_PMD_NR); if (TestClearPageDoubleMap(page)) { /* From patchwork Sat Dec 5 13:02:20 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 11953311 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B4117C4167B for ; Sat, 5 Dec 2020 18:40:32 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8CECA22EBF for ; Sat, 5 Dec 2020 18:40:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728936AbgLESTY (ORCPT ); Sat, 5 Dec 2020 13:19:24 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36492 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727993AbgLERx0 (ORCPT ); Sat, 5 Dec 2020 12:53:26 -0500 Received: from mail-pg1-x52b.google.com (mail-pg1-x52b.google.com [IPv6:2607:f8b0:4864:20::52b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DC76EC094256 for ; Sat, 5 Dec 2020 05:03:26 -0800 (PST) Received: by mail-pg1-x52b.google.com with SMTP id f17so5310742pge.6 for ; Sat, 05 Dec 2020 05:03:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=OePvIMZNKUN5Y3sDnnNomk9XQLTkzEVnsUhWUrUZogc=; b=WoXWAA5JojNk8kyuAW6gJqdYWZlwQnOCLR7Q5khaKLVJcZPcegnRLzSkXjT9L44vo4 gMebG3LUcJpd2o/hfnDrx4QCainJdKIEHFCRezmXO/GNHXBG9b2zM/Yco6il+mw18Asw tpK8F/iN8ncgW1Sxop6wxdpkNTxUPGdYyCudrLqVxLPR593KPuLvVBTG5O2waEp4DfcD s3KeMl2fsXxi7SgNOuX/HFQh9P1829Cd5x34/Bgx20Ck01y1SManhV9Mt+5pthxcfMdS GjItzyfUc2ZeOaiZrxZIVCKUoTsPsOpkHw/r/BPFjr0rpuXXqpMbDQgJQMkzu0tXt1jK uJcA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=OePvIMZNKUN5Y3sDnnNomk9XQLTkzEVnsUhWUrUZogc=; b=Y3MKSjNol5/Vyi5pLB33iJ4tx1GMRaWzUwbGHxS6gdujEWp3DkKVlYNuDmo09uZpB7 dTHAN/3pJR/XW5ujDMUNfKWB5o8AaddA+7WqA66O49SK9BoCfZBhLn3a+gOcx69/HETb +476hrt3Em6nhMSauDAtaUpHzqAvfyDPKPhhPmyMUWRcuajO9zP90Ln2aIw1qIXxa0e5 o+Gi3vfqLGUTKjs1+sl/LbTvSw5QIaqmqgTCz9IBuSxaTBjQQu6ILGR4IFR95B3d6AOp ShP8cfN3kDIjII4gHiYfC3tBepjBzJ48cRMXo1nz1/ncAVioI3u80nPeBWBWbfUIBLPa fy4g== X-Gm-Message-State: AOAM532GWMUyE8kec1kBe59J4f1uswXWZ0gBDwQXWzlaw1VApmfHKEf9 UlnIB34KCUg70ohd6eoPr2+/Ig== X-Google-Smtp-Source: ABdhPJx3PGE321iKpyLYXtBK9QOB0oxnrpnk2BUwF7TQjHOd7yU12KO+bBfach6O7MPRuv53jDc8aA== X-Received: by 2002:a63:985:: with SMTP id 127mr11363907pgj.449.1607173406403; Sat, 05 Dec 2020 05:03:26 -0800 (PST) Received: from localhost.bytedance.net ([103.136.220.120]) by smtp.gmail.com with ESMTPSA id kb12sm5047790pjb.2.2020.12.05.05.03.17 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sat, 05 Dec 2020 05:03:25 -0800 (PST) From: Muchun Song To: gregkh@linuxfoundation.org, rafael@kernel.org, adobriyan@gmail.com, akpm@linux-foundation.org, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, hughd@google.com, will@kernel.org, guro@fb.com, rppt@kernel.org, tglx@linutronix.de, esyr@redhat.com, peterx@redhat.com, krisman@collabora.com, surenb@google.com, avagin@openvz.org, elver@google.com, rdunlap@infradead.org, iamjoonsoo.kim@lge.com Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, Muchun Song Subject: [PATCH 5/9] mm: memcontrol: convert NR_FILE_THPS account to pages Date: Sat, 5 Dec 2020 21:02:20 +0800 Message-Id: <20201205130224.81607-6-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20201205130224.81607-1-songmuchun@bytedance.com> References: <20201205130224.81607-1-songmuchun@bytedance.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Converrt NR_FILE_THPS account to pages. Signed-off-by: Muchun Song --- drivers/base/node.c | 3 +-- fs/proc/meminfo.c | 2 +- mm/filemap.c | 2 +- mm/huge_memory.c | 3 ++- mm/khugepaged.c | 2 +- mm/memcontrol.c | 5 ++--- 6 files changed, 8 insertions(+), 9 deletions(-) diff --git a/drivers/base/node.c b/drivers/base/node.c index 05c369e93e16..f6a9521bbcf8 100644 --- a/drivers/base/node.c +++ b/drivers/base/node.c @@ -466,8 +466,7 @@ static ssize_t node_read_meminfo(struct device *dev, HPAGE_PMD_NR), nid, K(node_page_state(pgdat, NR_SHMEM_PMDMAPPED) * HPAGE_PMD_NR), - nid, K(node_page_state(pgdat, NR_FILE_THPS) * - HPAGE_PMD_NR), + nid, K(node_page_state(pgdat, NR_FILE_THPS)), nid, K(node_page_state(pgdat, NR_FILE_PMDMAPPED) * HPAGE_PMD_NR) #endif diff --git a/fs/proc/meminfo.c b/fs/proc/meminfo.c index f447ac191d24..9b2cb770326e 100644 --- a/fs/proc/meminfo.c +++ b/fs/proc/meminfo.c @@ -135,7 +135,7 @@ static int meminfo_proc_show(struct seq_file *m, void *v) show_val_kb(m, "ShmemPmdMapped: ", global_node_page_state(NR_SHMEM_PMDMAPPED) * HPAGE_PMD_NR); show_val_kb(m, "FileHugePages: ", - global_node_page_state(NR_FILE_THPS) * HPAGE_PMD_NR); + global_node_page_state(NR_FILE_THPS)); show_val_kb(m, "FilePmdMapped: ", global_node_page_state(NR_FILE_PMDMAPPED) * HPAGE_PMD_NR); #endif diff --git a/mm/filemap.c b/mm/filemap.c index 28e7309c29c8..c4dcb1144883 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -206,7 +206,7 @@ static void unaccount_page_cache_page(struct address_space *mapping, if (PageTransHuge(page)) __dec_lruvec_page_state(page, NR_SHMEM_THPS); } else if (PageTransHuge(page)) { - __dec_lruvec_page_state(page, NR_FILE_THPS); + __mod_lruvec_page_state(page, NR_FILE_THPS, -HPAGE_PMD_NR); filemap_nr_thps_dec(mapping); } diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 1a13e1dab3ec..37840bdeaad0 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -2748,7 +2748,8 @@ int split_huge_page_to_list(struct page *page, struct list_head *list) if (PageSwapBacked(head)) __dec_lruvec_page_state(head, NR_SHMEM_THPS); else - __dec_lruvec_page_state(head, NR_FILE_THPS); + __mod_lruvec_page_state(head, NR_FILE_THPS, + -HPAGE_PMD_NR); } __split_huge_page(page, list, end); diff --git a/mm/khugepaged.c b/mm/khugepaged.c index ad316d2e1fee..1e1ced2208d0 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1859,7 +1859,7 @@ static void collapse_file(struct mm_struct *mm, if (is_shmem) __inc_lruvec_page_state(new_page, NR_SHMEM_THPS); else { - __inc_lruvec_page_state(new_page, NR_FILE_THPS); + __mod_lruvec_page_state(new_page, NR_FILE_THPS, HPAGE_PMD_NR); filemap_nr_thps_inc(mapping); } diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 39cb7f1b00d3..dce76dddac61 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -1514,7 +1514,7 @@ static struct memory_stat memory_stats[] = { * constant(e.g. powerpc). */ { "anon_thp", PAGE_SIZE, NR_ANON_THPS }, - { "file_thp", 0, NR_FILE_THPS }, + { "file_thp", PAGE_SIZE, NR_FILE_THPS }, { "shmem_thp", 0, NR_SHMEM_THPS }, #endif { "inactive_anon", PAGE_SIZE, NR_INACTIVE_ANON }, @@ -1546,8 +1546,7 @@ static int __init memory_stats_init(void) for (i = 0; i < ARRAY_SIZE(memory_stats); i++) { #ifdef CONFIG_TRANSPARENT_HUGEPAGE - if (memory_stats[i].idx == NR_FILE_THPS || - memory_stats[i].idx == NR_SHMEM_THPS) + if (memory_stats[i].idx == NR_SHMEM_THPS) memory_stats[i].ratio = HPAGE_PMD_SIZE; #endif VM_BUG_ON(!memory_stats[i].ratio); From patchwork Sat Dec 5 13:02:21 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 11953309 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.9 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,UNWANTED_LANGUAGE_BODY, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B8BAAC18E57 for ; Sat, 5 Dec 2020 18:40:07 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9CB9F2310B for ; Sat, 5 Dec 2020 18:40:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728987AbgLESTZ (ORCPT ); Sat, 5 Dec 2020 13:19:25 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36752 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728066AbgLERzL (ORCPT ); Sat, 5 Dec 2020 12:55:11 -0500 Received: from mail-pl1-x62a.google.com (mail-pl1-x62a.google.com [IPv6:2607:f8b0:4864:20::62a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 395F1C094258 for ; Sat, 5 Dec 2020 05:03:35 -0800 (PST) Received: by mail-pl1-x62a.google.com with SMTP id v3so4639324plz.13 for ; Sat, 05 Dec 2020 05:03:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=S7mNi5PKMwLVHFP7gDd3IF0GOynsG77QNA4M+vj3DJk=; b=gCJzTjfCAu0GltWm/hyzcdft7AkWXt2V/lS0x+vLZl92dYwWf3S82oKszSt3ZzKOIh WJVAhhbIWRORnK/Q43dPs6cgOgzg4pM3xTRgx3tWJK4eM22mlCVn5OmI+Zmu3mN2tYc+ vaNJLGxSuk6PCdg3PEtKUb3d4tMZnBo0ZXJ4hqFbSA/El0ixN3PLTNvU+QkLm6/93Rs2 O2v5DKU+SdeEUy3Yfk1YFKrshqnCBSJcCUDKoNT6yL6HkTahhKM4nfAMWpL0Fbc7xRq3 TChN0DPtaAZiu7p3qKGo80phGyYpuCMaIuqq93KisF6L6ItFlt8klNxm5rp8Aftk9Mt/ CCQw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=S7mNi5PKMwLVHFP7gDd3IF0GOynsG77QNA4M+vj3DJk=; b=nZyRDgC62E5eyfFtGp8yvrwwI4DGFQeOKnRvFW+HfUYmpKTmJqr99EjWsDp6Jyh7od +/dp7BqhONIisPSK6KKCzEHL3anaYpJQdQJuz4xlaJjPaaFg4a9CfO+tcvuRdaoYOqBJ jXJGw9bCJhalmR3dLe/sbI0+7a5iMbHhYPH+GNGUah9GVkSn1SAn06rMQbJzN2PMMrOx Tm1AI8bFf+V9xf1LCleuHYW/FiTgT6uHEEvGtfvRKilHiq+wUes7AVVyFssbAo51e47+ 3zRVP+UA1bTe03cIfQ/DrIzieofpR6XfOlwP0MENViyaWhXOq3sFKJqLJmY+8vlBfiUj 7VLw== X-Gm-Message-State: AOAM532XttrTEGkM56dyxqweHW21gZmJqCAHlCujW+k4a2QkRfPG3ksc 18/vgXG2SOCViikgwXJQSYoo9g== X-Google-Smtp-Source: ABdhPJzppmPzDrR6zK3HJU3/qpakcgNl1o9PfNQFCZG1Z6v8tqR+jDK1GSjnKubs3798DaEWIBLcTA== X-Received: by 2002:a17:902:8506:b029:d9:6dd:45ea with SMTP id bj6-20020a1709028506b02900d906dd45eamr7956386plb.72.1607173414728; Sat, 05 Dec 2020 05:03:34 -0800 (PST) Received: from localhost.bytedance.net ([103.136.220.120]) by smtp.gmail.com with ESMTPSA id kb12sm5047790pjb.2.2020.12.05.05.03.26 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sat, 05 Dec 2020 05:03:34 -0800 (PST) From: Muchun Song To: gregkh@linuxfoundation.org, rafael@kernel.org, adobriyan@gmail.com, akpm@linux-foundation.org, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, hughd@google.com, will@kernel.org, guro@fb.com, rppt@kernel.org, tglx@linutronix.de, esyr@redhat.com, peterx@redhat.com, krisman@collabora.com, surenb@google.com, avagin@openvz.org, elver@google.com, rdunlap@infradead.org, iamjoonsoo.kim@lge.com Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, Muchun Song Subject: [PATCH 6/9] mm: memcontrol: convert NR_SHMEM_THPS account to pages Date: Sat, 5 Dec 2020 21:02:21 +0800 Message-Id: <20201205130224.81607-7-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20201205130224.81607-1-songmuchun@bytedance.com> References: <20201205130224.81607-1-songmuchun@bytedance.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Convert NR_SHMEM_THPS account to pages Signed-off-by: Muchun Song --- drivers/base/node.c | 3 +-- fs/proc/meminfo.c | 2 +- mm/filemap.c | 2 +- mm/huge_memory.c | 3 ++- mm/khugepaged.c | 2 +- mm/memcontrol.c | 26 ++------------------------ mm/page_alloc.c | 2 +- mm/shmem.c | 3 ++- 8 files changed, 11 insertions(+), 32 deletions(-) diff --git a/drivers/base/node.c b/drivers/base/node.c index f6a9521bbcf8..a64f9c5484a0 100644 --- a/drivers/base/node.c +++ b/drivers/base/node.c @@ -462,8 +462,7 @@ static ssize_t node_read_meminfo(struct device *dev, #ifdef CONFIG_TRANSPARENT_HUGEPAGE , nid, K(node_page_state(pgdat, NR_ANON_THPS)), - nid, K(node_page_state(pgdat, NR_SHMEM_THPS) * - HPAGE_PMD_NR), + nid, K(node_page_state(pgdat, NR_SHMEM_THPS)), nid, K(node_page_state(pgdat, NR_SHMEM_PMDMAPPED) * HPAGE_PMD_NR), nid, K(node_page_state(pgdat, NR_FILE_THPS)), diff --git a/fs/proc/meminfo.c b/fs/proc/meminfo.c index 9b2cb770326e..574779b6e48c 100644 --- a/fs/proc/meminfo.c +++ b/fs/proc/meminfo.c @@ -131,7 +131,7 @@ static int meminfo_proc_show(struct seq_file *m, void *v) show_val_kb(m, "AnonHugePages: ", global_node_page_state(NR_ANON_THPS)); show_val_kb(m, "ShmemHugePages: ", - global_node_page_state(NR_SHMEM_THPS) * HPAGE_PMD_NR); + global_node_page_state(NR_SHMEM_THPS)); show_val_kb(m, "ShmemPmdMapped: ", global_node_page_state(NR_SHMEM_PMDMAPPED) * HPAGE_PMD_NR); show_val_kb(m, "FileHugePages: ", diff --git a/mm/filemap.c b/mm/filemap.c index c4dcb1144883..5fdefbbc1bc2 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -204,7 +204,7 @@ static void unaccount_page_cache_page(struct address_space *mapping, if (PageSwapBacked(page)) { __mod_lruvec_page_state(page, NR_SHMEM, -nr); if (PageTransHuge(page)) - __dec_lruvec_page_state(page, NR_SHMEM_THPS); + __mod_lruvec_page_state(page, NR_SHMEM_THPS, -HPAGE_PMD_NR); } else if (PageTransHuge(page)) { __mod_lruvec_page_state(page, NR_FILE_THPS, -HPAGE_PMD_NR); filemap_nr_thps_dec(mapping); diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 37840bdeaad0..0e8541bd9f50 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -2746,7 +2746,8 @@ int split_huge_page_to_list(struct page *page, struct list_head *list) spin_unlock(&ds_queue->split_queue_lock); if (mapping) { if (PageSwapBacked(head)) - __dec_lruvec_page_state(head, NR_SHMEM_THPS); + __mod_lruvec_page_state(head, NR_SHMEM_THPS, + -HPAGE_PMD_NR); else __mod_lruvec_page_state(head, NR_FILE_THPS, -HPAGE_PMD_NR); diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 1e1ced2208d0..4fe79ccfc312 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1857,7 +1857,7 @@ static void collapse_file(struct mm_struct *mm, } if (is_shmem) - __inc_lruvec_page_state(new_page, NR_SHMEM_THPS); + __mod_lruvec_page_state(new_page, NR_SHMEM_THPS, HPAGE_PMD_NR); else { __mod_lruvec_page_state(new_page, NR_FILE_THPS, HPAGE_PMD_NR); filemap_nr_thps_inc(mapping); diff --git a/mm/memcontrol.c b/mm/memcontrol.c index dce76dddac61..48d70c1ad301 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -1497,7 +1497,7 @@ struct memory_stat { unsigned int idx; }; -static struct memory_stat memory_stats[] = { +static const struct memory_stat memory_stats[] = { { "anon", PAGE_SIZE, NR_ANON_MAPPED }, { "file", PAGE_SIZE, NR_FILE_PAGES }, { "kernel_stack", 1, NR_KERNEL_STACK_B }, @@ -1508,14 +1508,9 @@ static struct memory_stat memory_stats[] = { { "file_dirty", PAGE_SIZE, NR_FILE_DIRTY }, { "file_writeback", PAGE_SIZE, NR_WRITEBACK }, #ifdef CONFIG_TRANSPARENT_HUGEPAGE - /* - * The ratio will be initialized in memory_stats_init(). Because - * on some architectures, the macro of HPAGE_PMD_SIZE is not - * constant(e.g. powerpc). - */ { "anon_thp", PAGE_SIZE, NR_ANON_THPS }, { "file_thp", PAGE_SIZE, NR_FILE_THPS }, - { "shmem_thp", 0, NR_SHMEM_THPS }, + { "shmem_thp", PAGE_SIZE, NR_SHMEM_THPS }, #endif { "inactive_anon", PAGE_SIZE, NR_INACTIVE_ANON }, { "active_anon", PAGE_SIZE, NR_ACTIVE_ANON }, @@ -1540,23 +1535,6 @@ static struct memory_stat memory_stats[] = { { "workingset_nodereclaim", 1, WORKINGSET_NODERECLAIM }, }; -static int __init memory_stats_init(void) -{ - int i; - - for (i = 0; i < ARRAY_SIZE(memory_stats); i++) { -#ifdef CONFIG_TRANSPARENT_HUGEPAGE - if (memory_stats[i].idx == NR_SHMEM_THPS) - memory_stats[i].ratio = HPAGE_PMD_SIZE; -#endif - VM_BUG_ON(!memory_stats[i].ratio); - VM_BUG_ON(memory_stats[i].idx >= MEMCG_NR_STAT); - } - - return 0; -} -pure_initcall(memory_stats_init); - static char *memory_stat_format(struct mem_cgroup *memcg) { struct seq_buf s; diff --git a/mm/page_alloc.c b/mm/page_alloc.c index fabdbb340806..8fb9f3d38b67 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -5567,7 +5567,7 @@ void show_free_areas(unsigned int filter, nodemask_t *nodemask) K(node_page_state(pgdat, NR_WRITEBACK)), K(node_page_state(pgdat, NR_SHMEM)), #ifdef CONFIG_TRANSPARENT_HUGEPAGE - K(node_page_state(pgdat, NR_SHMEM_THPS) * HPAGE_PMD_NR), + K(node_page_state(pgdat, NR_SHMEM_THPS)), K(node_page_state(pgdat, NR_SHMEM_PMDMAPPED) * HPAGE_PMD_NR), K(node_page_state(pgdat, NR_ANON_THPS)), diff --git a/mm/shmem.c b/mm/shmem.c index 5da4f1a3e663..ea5d8c9ccb5b 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -713,7 +713,8 @@ static int shmem_add_to_page_cache(struct page *page, } if (PageTransHuge(page)) { count_vm_event(THP_FILE_ALLOC); - __inc_lruvec_page_state(page, NR_SHMEM_THPS); + __mod_lruvec_page_state(page, NR_SHMEM_THPS, + HPAGE_PMD_NR); } mapping->nrpages += nr; __mod_lruvec_page_state(page, NR_FILE_PAGES, nr); From patchwork Sat Dec 5 13:02:22 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 11953267 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B74FFC433FE for ; Sat, 5 Dec 2020 18:31:23 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8211622D01 for ; Sat, 5 Dec 2020 18:31:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726254AbgLESaz (ORCPT ); Sat, 5 Dec 2020 13:30:55 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42128 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725996AbgLES3k (ORCPT ); Sat, 5 Dec 2020 13:29:40 -0500 Received: from mail-pf1-x42b.google.com (mail-pf1-x42b.google.com [IPv6:2607:f8b0:4864:20::42b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7DDF3C09425B for ; Sat, 5 Dec 2020 05:03:43 -0800 (PST) Received: by mail-pf1-x42b.google.com with SMTP id w6so5760495pfu.1 for ; Sat, 05 Dec 2020 05:03:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=HZOUEjYnrw/jKRYqpi7q020fqNgK9R8UvUzzMbw50zY=; b=neYDpg4r2y9kA8bSIzVIJ2XJBnhDklsaXMPeNsZUDajIujwhGMeEFhY+XngVY5qv8u IYh5151ZcZtLBQsFRPs2SNjyK1NhJq+NVy4gswyaAdkA94CsNPNg/atj6y+aNTBSlYD6 gNQrw6rGfg8uRBsZgcdDNhrcRCWvx5RjKJVD3sAabpSf3vnHCWmmGbHI3pywzLcmNDOZ sP2YEZeBeHFVXYlVwSw1cKnhg4GAgCFJVxmzFgOOwCzgTMcu7CxRdpdjfhnn+IkHGoqI P/Z7nXl9EfZYolJ9/kaAWPLl8ZUvmXv3zGIY+9XoTHJKnfUatUnrDe/8gBSjCCnvGYaM MxqQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=HZOUEjYnrw/jKRYqpi7q020fqNgK9R8UvUzzMbw50zY=; b=G7RfJa12iydUMi+lxFF47neDgZBgvSso/nvuarkT3O5oX9BVdzYpOdr1/A5Fb6+4h9 8/4KO/KCPgiaD6xx+0LmAWHxdaodJfbIFw1r1DVT2BW789jUmvsRc5ieIRhucAkERSz5 CmSVxi2LPMW3R5ewIwnlVSF0CAu5RMatuvEozIlUWk0waSOrqLfxUTkGxjBXA1s3n7U6 7HOPKUESqeCMINfC4QWepkWxIUXGWbtvBZlmliqFv0/ZSRkPeA82wBkuixkjA17OT11I c3WmCm3Ma33X4yy/TOkvbxsoIqbytVC/sKVuKTv4Y08wPZ9fAaFWu09zjo6xAflpSlLV owZg== X-Gm-Message-State: AOAM530yJjV76Z8+o5oSwtkWyQyJGYKVUllp9zvUQriEBIwSVlRdLbJ+ 979QaESOJy1uWlEQwzbdCzOMxg== X-Google-Smtp-Source: ABdhPJwqvYfk7WrXCeek5zIgmaL6JHMd+SCgDfry5zzeu3DJFq3BmX1JwmtE9dLTcxi/BJFMeSUD5A== X-Received: by 2002:a62:1b06:0:b029:19d:d05d:f67a with SMTP id b6-20020a621b060000b029019dd05df67amr2471401pfb.78.1607173423110; Sat, 05 Dec 2020 05:03:43 -0800 (PST) Received: from localhost.bytedance.net ([103.136.220.120]) by smtp.gmail.com with ESMTPSA id kb12sm5047790pjb.2.2020.12.05.05.03.35 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sat, 05 Dec 2020 05:03:42 -0800 (PST) From: Muchun Song To: gregkh@linuxfoundation.org, rafael@kernel.org, adobriyan@gmail.com, akpm@linux-foundation.org, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, hughd@google.com, will@kernel.org, guro@fb.com, rppt@kernel.org, tglx@linutronix.de, esyr@redhat.com, peterx@redhat.com, krisman@collabora.com, surenb@google.com, avagin@openvz.org, elver@google.com, rdunlap@infradead.org, iamjoonsoo.kim@lge.com Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, Muchun Song Subject: [PATCH 7/9] mm: memcontrol: convert NR_SHMEM_PMDMAPPED account to pages Date: Sat, 5 Dec 2020 21:02:22 +0800 Message-Id: <20201205130224.81607-8-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20201205130224.81607-1-songmuchun@bytedance.com> References: <20201205130224.81607-1-songmuchun@bytedance.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Convert NR_SHMEM_PMDMAPPED account to pages. Signed-off-by: Muchun Song --- drivers/base/node.c | 3 +-- fs/proc/meminfo.c | 2 +- mm/page_alloc.c | 3 +-- mm/rmap.c | 6 ++++-- 4 files changed, 7 insertions(+), 7 deletions(-) diff --git a/drivers/base/node.c b/drivers/base/node.c index a64f9c5484a0..fe90888f90a8 100644 --- a/drivers/base/node.c +++ b/drivers/base/node.c @@ -463,8 +463,7 @@ static ssize_t node_read_meminfo(struct device *dev, , nid, K(node_page_state(pgdat, NR_ANON_THPS)), nid, K(node_page_state(pgdat, NR_SHMEM_THPS)), - nid, K(node_page_state(pgdat, NR_SHMEM_PMDMAPPED) * - HPAGE_PMD_NR), + nid, K(node_page_state(pgdat, NR_SHMEM_PMDMAPPED)), nid, K(node_page_state(pgdat, NR_FILE_THPS)), nid, K(node_page_state(pgdat, NR_FILE_PMDMAPPED) * HPAGE_PMD_NR) diff --git a/fs/proc/meminfo.c b/fs/proc/meminfo.c index 574779b6e48c..b2bff8359497 100644 --- a/fs/proc/meminfo.c +++ b/fs/proc/meminfo.c @@ -133,7 +133,7 @@ static int meminfo_proc_show(struct seq_file *m, void *v) show_val_kb(m, "ShmemHugePages: ", global_node_page_state(NR_SHMEM_THPS)); show_val_kb(m, "ShmemPmdMapped: ", - global_node_page_state(NR_SHMEM_PMDMAPPED) * HPAGE_PMD_NR); + global_node_page_state(NR_SHMEM_PMDMAPPED)); show_val_kb(m, "FileHugePages: ", global_node_page_state(NR_FILE_THPS)); show_val_kb(m, "FilePmdMapped: ", diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 8fb9f3d38b67..ddaa1dcd6e38 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -5568,8 +5568,7 @@ void show_free_areas(unsigned int filter, nodemask_t *nodemask) K(node_page_state(pgdat, NR_SHMEM)), #ifdef CONFIG_TRANSPARENT_HUGEPAGE K(node_page_state(pgdat, NR_SHMEM_THPS)), - K(node_page_state(pgdat, NR_SHMEM_PMDMAPPED) - * HPAGE_PMD_NR), + K(node_page_state(pgdat, NR_SHMEM_PMDMAPPED)), K(node_page_state(pgdat, NR_ANON_THPS)), #endif K(node_page_state(pgdat, NR_WRITEBACK_TEMP)), diff --git a/mm/rmap.c b/mm/rmap.c index f59e92e26b61..3089ad6bf468 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -1219,7 +1219,8 @@ void page_add_file_rmap(struct page *page, bool compound) if (!atomic_inc_and_test(compound_mapcount_ptr(page))) goto out; if (PageSwapBacked(page)) - __inc_node_page_state(page, NR_SHMEM_PMDMAPPED); + __mod_lruvec_page_state(page, NR_SHMEM_PMDMAPPED, + HPAGE_PMD_NR); else __inc_node_page_state(page, NR_FILE_PMDMAPPED); } else { @@ -1260,7 +1261,8 @@ static void page_remove_file_rmap(struct page *page, bool compound) if (!atomic_add_negative(-1, compound_mapcount_ptr(page))) return; if (PageSwapBacked(page)) - __dec_node_page_state(page, NR_SHMEM_PMDMAPPED); + __mod_lruvec_page_state(page, NR_SHMEM_PMDMAPPED, + -HPAGE_PMD_NR); else __dec_node_page_state(page, NR_FILE_PMDMAPPED); } else { From patchwork Sat Dec 5 13:02:23 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 11953261 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 84A2FC4167B for ; Sat, 5 Dec 2020 18:30:56 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5E131230FE for ; Sat, 5 Dec 2020 18:30:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729076AbgLESar (ORCPT ); Sat, 5 Dec 2020 13:30:47 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42152 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726254AbgLES3k (ORCPT ); Sat, 5 Dec 2020 13:29:40 -0500 Received: from mail-pl1-x636.google.com (mail-pl1-x636.google.com [IPv6:2607:f8b0:4864:20::636]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5AC33C09425F for ; Sat, 5 Dec 2020 05:03:52 -0800 (PST) Received: by mail-pl1-x636.google.com with SMTP id t18so4669695plo.0 for ; Sat, 05 Dec 2020 05:03:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=FtP7qi7mK1L0PfFz3nd+UCw8YfODDif7rcyX5iGzCy0=; b=YBqE9JIpZZHp59F2B1fcdbYxWsPsKLsDcStvI6zVPyor7Qvy6DPn+rQUINr+d2lrr4 xPUtcPPYHneb4wBbmPXr94aIzyBHQ7s06HpUtYiINlmoL0Z9oSXqzyz2b8usnRFtG8WX 8/xIK4F8m/q/JyZxdQ+wzyn3+xrk1F/Nb8P5zcWAz2qcvUvsoLk4pRTF4PzYe0q9+/Sj OjpvPukcqXmtU7DA7WYe9o5kUelIpmxnRsnIt9T/Vk8LMUk2fsls8CUUTbcaClLy/AO7 iTdJmEw8xWDYF82QC/sshrA+U9zeszVwAh8YxD682t9P0lFeQmroJ+4MTEjqGahXpBRV kqZg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=FtP7qi7mK1L0PfFz3nd+UCw8YfODDif7rcyX5iGzCy0=; b=bRutFOcVDxPIhfPWiIw6wvqcHDTqnj5v0AYcFzq+g5cwJhaMRFkyW9W0ZSf1s6cktF 9VlbJXEKRSf/P/JUfO2Fm7h3ahlwch9UVhIZn5UKaMktzl1ibbL0Aw1golvEWkytBtvE +Xv/E628V9i6bMV89TsaWYYb6ogZG5xrJadZqsCgzxaR8DlbXxUzSmEWAdkbxa4msHjg 3IYQ+hwX4b35YN9fDywBXBoefZpzyrvRKK2n/xEf16pkZEm8X4irkvuFjh544CRI8N96 ee1c9Y87HgJYigm4vw1eI02WnSB8AY/1KeDLRQg4Rm6BZ1amWBSlFr4QsTpiub+OTs7U sUFg== X-Gm-Message-State: AOAM530Hh+HGPZtDjtukOjYYQ54MwVBYjFlhjeem3vHySArWdmBHa3k4 b+4039kZPHLDEpTjVIbdQhta1w== X-Google-Smtp-Source: ABdhPJxUntSBpIscnDoUmmMVMARNsl+XmN2HuKXM1HmNHVZCkOQBbP+bouFyJZkLYx+IaQCeb/ju+A== X-Received: by 2002:a17:902:7297:b029:da:861e:eae1 with SMTP id d23-20020a1709027297b02900da861eeae1mr8308288pll.8.1607173431888; Sat, 05 Dec 2020 05:03:51 -0800 (PST) Received: from localhost.bytedance.net ([103.136.220.120]) by smtp.gmail.com with ESMTPSA id kb12sm5047790pjb.2.2020.12.05.05.03.43 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sat, 05 Dec 2020 05:03:51 -0800 (PST) From: Muchun Song To: gregkh@linuxfoundation.org, rafael@kernel.org, adobriyan@gmail.com, akpm@linux-foundation.org, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, hughd@google.com, will@kernel.org, guro@fb.com, rppt@kernel.org, tglx@linutronix.de, esyr@redhat.com, peterx@redhat.com, krisman@collabora.com, surenb@google.com, avagin@openvz.org, elver@google.com, rdunlap@infradead.org, iamjoonsoo.kim@lge.com Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, Muchun Song Subject: [PATCH 8/9] mm: memcontrol: convert NR_FILE_PMDMAPPED account to pages Date: Sat, 5 Dec 2020 21:02:23 +0800 Message-Id: <20201205130224.81607-9-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20201205130224.81607-1-songmuchun@bytedance.com> References: <20201205130224.81607-1-songmuchun@bytedance.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Convert NR_FILE_PMDMAPPED account to pages Signed-off-by: Muchun Song --- drivers/base/node.c | 3 +-- fs/proc/meminfo.c | 2 +- mm/rmap.c | 6 ++++-- 3 files changed, 6 insertions(+), 5 deletions(-) diff --git a/drivers/base/node.c b/drivers/base/node.c index fe90888f90a8..679215cdeb87 100644 --- a/drivers/base/node.c +++ b/drivers/base/node.c @@ -465,8 +465,7 @@ static ssize_t node_read_meminfo(struct device *dev, nid, K(node_page_state(pgdat, NR_SHMEM_THPS)), nid, K(node_page_state(pgdat, NR_SHMEM_PMDMAPPED)), nid, K(node_page_state(pgdat, NR_FILE_THPS)), - nid, K(node_page_state(pgdat, NR_FILE_PMDMAPPED) * - HPAGE_PMD_NR) + nid, K(node_page_state(pgdat, NR_FILE_PMDMAPPED)) #endif ); len += hugetlb_report_node_meminfo(buf, len, nid); diff --git a/fs/proc/meminfo.c b/fs/proc/meminfo.c index b2bff8359497..a0828df01721 100644 --- a/fs/proc/meminfo.c +++ b/fs/proc/meminfo.c @@ -137,7 +137,7 @@ static int meminfo_proc_show(struct seq_file *m, void *v) show_val_kb(m, "FileHugePages: ", global_node_page_state(NR_FILE_THPS)); show_val_kb(m, "FilePmdMapped: ", - global_node_page_state(NR_FILE_PMDMAPPED) * HPAGE_PMD_NR); + global_node_page_state(NR_FILE_PMDMAPPED)); #endif #ifdef CONFIG_CMA diff --git a/mm/rmap.c b/mm/rmap.c index 3089ad6bf468..e383c5619501 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -1222,7 +1222,8 @@ void page_add_file_rmap(struct page *page, bool compound) __mod_lruvec_page_state(page, NR_SHMEM_PMDMAPPED, HPAGE_PMD_NR); else - __inc_node_page_state(page, NR_FILE_PMDMAPPED); + __mod_lruvec_page_state(page, NR_FILE_PMDMAPPED, + HPAGE_PMD_NR); } else { if (PageTransCompound(page) && page_mapping(page)) { VM_WARN_ON_ONCE(!PageLocked(page)); @@ -1264,7 +1265,8 @@ static void page_remove_file_rmap(struct page *page, bool compound) __mod_lruvec_page_state(page, NR_SHMEM_PMDMAPPED, -HPAGE_PMD_NR); else - __dec_node_page_state(page, NR_FILE_PMDMAPPED); + __mod_lruvec_page_state(page, NR_FILE_PMDMAPPED, + -HPAGE_PMD_NR); } else { if (!atomic_add_negative(-1, &page->_mapcount)) return; From patchwork Sat Dec 5 13:02:24 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 11953387 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E1CF7C4361A for ; Sat, 5 Dec 2020 18:45:49 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8FF4A22B37 for ; Sat, 5 Dec 2020 18:45:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726763AbgLEQtA (ORCPT ); Sat, 5 Dec 2020 11:49:00 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53742 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726249AbgLEQoo (ORCPT ); Sat, 5 Dec 2020 11:44:44 -0500 Received: from mail-pg1-x542.google.com (mail-pg1-x542.google.com [IPv6:2607:f8b0:4864:20::542]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 67B08C094261 for ; Sat, 5 Dec 2020 05:04:00 -0800 (PST) Received: by mail-pg1-x542.google.com with SMTP id f17so5311308pge.6 for ; Sat, 05 Dec 2020 05:04:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=zRyHqu7TjhWPvcpqYaxOFrZwdcjevZ28RME9caXsylo=; b=m/rCaxXSUDZQW5kZo+JO83uk75DwwkHyUjCYnCvBEjao+vuJqHqVIEezB6KH3XIyi5 Dgg7e7bhqIU+ulk7wSR5kaHrVKRhCTeq0ZeQ1SOdDvnW/krZOG84FMk1XjFSl3Gb5OA/ 29lrO3IUcLbskByH+gbY2/Gt6L7lOsWGhVy/alHNFwX9S/XI4gcZ2E/GcP+rpwEZk7pn 9fjlHjhoMrcAJISDAy3zXTPYLjZDT2GBLcvH/iDzDBn2OVE1zuUonlDXFPhct9Cv9wad UZfE9qtqO5ZyR2285m16+LNjcvQUliBK1kJd2IZDLJzbgOPiMd9PZK0hJlhGTNjo6nwu OwZg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=zRyHqu7TjhWPvcpqYaxOFrZwdcjevZ28RME9caXsylo=; b=J/I7TAA16P0BV1vQ9fRAy0WMjvoJ1dczj/vfo0giDztw1+kUm+mCd63iCYzVNGUwa6 2yMVNa4Mevz30hJUFM8G7px9pLzxP0OMjFVVkNuu6UBFpgNcVQU2i9oezy5ZDCAkg7jk mQ+1IvXXXFqmKEI+ej6EPDV5Eu0+SfLjTqRu6ZEjrbMv6sq5JxfI1JfbtrQEtvwMvPMH nuwYTNrezU4buRXFLHJnJP475SBNMWnzF5UMDJTgM/+smkicGwZ5/9rq6Cx58pQCMjUt MrhhIIFK+xcJl0uPUgTk80rD/kSt94dnxDFMw5Tz4uEgaoXOWR4pH13aZcfgKVHEkE6V Aq5g== X-Gm-Message-State: AOAM532GytEhFYjD+8SB4ex9pyn3rlcJczz8rcIhEhssoe8EEQDwKxXc RjItGz2L5vFtD8C5NApms2WLcg== X-Google-Smtp-Source: ABdhPJxvXcELEy8kd2pseS9d6RB7ZGMaFuIBTwCAKgjIlZ4MBPJrlUVBm5AwpRtLy3Jl9rT7WT0uxA== X-Received: by 2002:a05:6a00:13a3:b029:18b:d5d2:196 with SMTP id t35-20020a056a0013a3b029018bd5d20196mr8333119pfg.62.1607173439991; Sat, 05 Dec 2020 05:03:59 -0800 (PST) Received: from localhost.bytedance.net ([103.136.220.120]) by smtp.gmail.com with ESMTPSA id kb12sm5047790pjb.2.2020.12.05.05.03.52 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sat, 05 Dec 2020 05:03:59 -0800 (PST) From: Muchun Song To: gregkh@linuxfoundation.org, rafael@kernel.org, adobriyan@gmail.com, akpm@linux-foundation.org, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, hughd@google.com, will@kernel.org, guro@fb.com, rppt@kernel.org, tglx@linutronix.de, esyr@redhat.com, peterx@redhat.com, krisman@collabora.com, surenb@google.com, avagin@openvz.org, elver@google.com, rdunlap@infradead.org, iamjoonsoo.kim@lge.com Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, Muchun Song Subject: [PATCH 9/9] mm: memcontrol: make the slab calculation consistent Date: Sat, 5 Dec 2020 21:02:24 +0800 Message-Id: <20201205130224.81607-10-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20201205130224.81607-1-songmuchun@bytedance.com> References: <20201205130224.81607-1-songmuchun@bytedance.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Although the ratio of the slab is one, we also should read the ratio from the related memory_stats instead of hard-coding. And the local variable of size is already the value of slab_unreclaimable. So we do not need to read again. Signed-off-by: Muchun Song --- mm/memcontrol.c | 112 +++++++++++++++++++++++++++++++++++++------------------- 1 file changed, 74 insertions(+), 38 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 48d70c1ad301..0e9e8ca7e770 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -1493,48 +1493,75 @@ static bool mem_cgroup_wait_acct_move(struct mem_cgroup *memcg) struct memory_stat { const char *name; - unsigned int ratio; unsigned int idx; }; static const struct memory_stat memory_stats[] = { - { "anon", PAGE_SIZE, NR_ANON_MAPPED }, - { "file", PAGE_SIZE, NR_FILE_PAGES }, - { "kernel_stack", 1, NR_KERNEL_STACK_B }, - { "percpu", 1, MEMCG_PERCPU_B }, - { "sock", PAGE_SIZE, MEMCG_SOCK }, - { "shmem", PAGE_SIZE, NR_SHMEM }, - { "file_mapped", PAGE_SIZE, NR_FILE_MAPPED }, - { "file_dirty", PAGE_SIZE, NR_FILE_DIRTY }, - { "file_writeback", PAGE_SIZE, NR_WRITEBACK }, + { "anon", NR_ANON_MAPPED }, + { "file", NR_FILE_PAGES }, + { "kernel_stack", NR_KERNEL_STACK_B }, + { "percpu", MEMCG_PERCPU_B }, + { "sock", MEMCG_SOCK }, + { "shmem", NR_SHMEM }, + { "file_mapped", NR_FILE_MAPPED }, + { "file_dirty", NR_FILE_DIRTY }, + { "file_writeback", NR_WRITEBACK }, #ifdef CONFIG_TRANSPARENT_HUGEPAGE - { "anon_thp", PAGE_SIZE, NR_ANON_THPS }, - { "file_thp", PAGE_SIZE, NR_FILE_THPS }, - { "shmem_thp", PAGE_SIZE, NR_SHMEM_THPS }, + { "anon_thp", NR_ANON_THPS }, + { "file_thp", NR_FILE_THPS }, + { "shmem_thp", NR_SHMEM_THPS }, #endif - { "inactive_anon", PAGE_SIZE, NR_INACTIVE_ANON }, - { "active_anon", PAGE_SIZE, NR_ACTIVE_ANON }, - { "inactive_file", PAGE_SIZE, NR_INACTIVE_FILE }, - { "active_file", PAGE_SIZE, NR_ACTIVE_FILE }, - { "unevictable", PAGE_SIZE, NR_UNEVICTABLE }, - - /* - * Note: The slab_reclaimable and slab_unreclaimable must be - * together and slab_reclaimable must be in front. - */ - { "slab_reclaimable", 1, NR_SLAB_RECLAIMABLE_B }, - { "slab_unreclaimable", 1, NR_SLAB_UNRECLAIMABLE_B }, + { "inactive_anon", NR_INACTIVE_ANON }, + { "active_anon", NR_ACTIVE_ANON }, + { "inactive_file", NR_INACTIVE_FILE }, + { "active_file", NR_ACTIVE_FILE }, + { "unevictable", NR_UNEVICTABLE }, + { "slab_reclaimable", NR_SLAB_RECLAIMABLE_B }, + { "slab_unreclaimable", NR_SLAB_UNRECLAIMABLE_B }, /* The memory events */ - { "workingset_refault_anon", 1, WORKINGSET_REFAULT_ANON }, - { "workingset_refault_file", 1, WORKINGSET_REFAULT_FILE }, - { "workingset_activate_anon", 1, WORKINGSET_ACTIVATE_ANON }, - { "workingset_activate_file", 1, WORKINGSET_ACTIVATE_FILE }, - { "workingset_restore_anon", 1, WORKINGSET_RESTORE_ANON }, - { "workingset_restore_file", 1, WORKINGSET_RESTORE_FILE }, - { "workingset_nodereclaim", 1, WORKINGSET_NODERECLAIM }, + { "workingset_refault_anon", WORKINGSET_REFAULT_ANON }, + { "workingset_refault_file", WORKINGSET_REFAULT_FILE }, + { "workingset_activate_anon", WORKINGSET_ACTIVATE_ANON }, + { "workingset_activate_file", WORKINGSET_ACTIVATE_FILE }, + { "workingset_restore_anon", WORKINGSET_RESTORE_ANON }, + { "workingset_restore_file", WORKINGSET_RESTORE_FILE }, + { "workingset_nodereclaim", WORKINGSET_NODERECLAIM }, }; +/* Translate stat items to the correct unit for memory.stat output */ +static int memcg_page_state_unit(int item) +{ + int unit; + + switch (item) { + case NR_SLAB_RECLAIMABLE_B: + case NR_SLAB_UNRECLAIMABLE_B: + case WORKINGSET_REFAULT_ANON: + case WORKINGSET_REFAULT_FILE: + case WORKINGSET_ACTIVATE_ANON: + case WORKINGSET_ACTIVATE_FILE: + case WORKINGSET_RESTORE_ANON: + case WORKINGSET_RESTORE_FILE: + case WORKINGSET_NODERECLAIM: + case MEMCG_PERCPU_B: + case NR_KERNEL_STACK_B: + unit = 1; + break; + default: + unit = PAGE_SIZE; + break; + } + + return unit; +} + +static inline unsigned long memcg_page_state_output(struct mem_cgroup *memcg, + int item) +{ + return memcg_page_state(memcg, item) * memcg_page_state_unit(item); +} + static char *memory_stat_format(struct mem_cgroup *memcg) { struct seq_buf s; @@ -1558,13 +1585,16 @@ static char *memory_stat_format(struct mem_cgroup *memcg) for (i = 0; i < ARRAY_SIZE(memory_stats); i++) { u64 size; - size = memcg_page_state(memcg, memory_stats[i].idx); - size *= memory_stats[i].ratio; + size = memcg_page_state_output(memcg, memory_stats[i].idx); seq_buf_printf(&s, "%s %llu\n", memory_stats[i].name, size); + /* + * We are printing reclaimable, unreclaimable of the slab + * and the sum of both. + */ if (unlikely(memory_stats[i].idx == NR_SLAB_UNRECLAIMABLE_B)) { - size = memcg_page_state(memcg, NR_SLAB_RECLAIMABLE_B) + - memcg_page_state(memcg, NR_SLAB_UNRECLAIMABLE_B); + size += memcg_page_state_output(memcg, + NR_SLAB_RECLAIMABLE_B); seq_buf_printf(&s, "slab %llu\n", size); } } @@ -6358,6 +6388,12 @@ static int memory_stat_show(struct seq_file *m, void *v) } #ifdef CONFIG_NUMA +static inline unsigned long lruvec_page_state_output(struct lruvec *lruvec, + int item) +{ + return lruvec_page_state(lruvec, item) * memcg_page_state_unit(item); +} + static int memory_numa_stat_show(struct seq_file *m, void *v) { int i; @@ -6375,8 +6411,8 @@ static int memory_numa_stat_show(struct seq_file *m, void *v) struct lruvec *lruvec; lruvec = mem_cgroup_lruvec(memcg, NODE_DATA(nid)); - size = lruvec_page_state(lruvec, memory_stats[i].idx); - size *= memory_stats[i].ratio; + size = lruvec_page_state_output(lruvec, + memory_stats[i].idx); seq_printf(m, " N%d=%llu", nid, size); } seq_putc(m, '\n');