From patchwork Sun Dec 6 10:14:48 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 11953901 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A5C19C433FE for ; Sun, 6 Dec 2020 10:18:16 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 549B022D2C for ; Sun, 6 Dec 2020 10:18:16 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 549B022D2C Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id B3E9A6B0068; Sun, 6 Dec 2020 05:18:15 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id AEF336B0073; Sun, 6 Dec 2020 05:18:15 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9DF5C6B0074; Sun, 6 Dec 2020 05:18:15 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0075.hostedemail.com [216.40.44.75]) by kanga.kvack.org (Postfix) with ESMTP id 8805B6B0068 for ; Sun, 6 Dec 2020 05:18:15 -0500 (EST) Received: from smtpin11.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 4D7E1181AEF10 for ; Sun, 6 Dec 2020 10:18:15 +0000 (UTC) X-FDA: 77562457350.11.honey25_040cd82273d4 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin11.hostedemail.com (Postfix) with ESMTP id 13D8D180F8B80 for ; Sun, 6 Dec 2020 10:18:15 +0000 (UTC) X-HE-Tag: honey25_040cd82273d4 X-Filterd-Recvd-Size: 6557 Received: from mail-pj1-f68.google.com (mail-pj1-f68.google.com [209.85.216.68]) by imf42.hostedemail.com (Postfix) with ESMTP for ; Sun, 6 Dec 2020 10:18:14 +0000 (UTC) Received: by mail-pj1-f68.google.com with SMTP id o7so5744296pjj.2 for ; Sun, 06 Dec 2020 02:18:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=jxjzCofAcULr7bnQMbpOvZCW6tNXMo722rzO5X3X+g0=; b=l0t429mzGcnZFwxy/27EHeyds85DHU2/SvU/cTTFIMKpTZQ13vGzScVjnNJc1Qfgdl tq3J8LH2Po4yAdE4Qo1uNZ9GfP2dCXBM7HpnjMHacvON/LTEfdfapf3sJCtF6f1t+m// Rdasl44TWp0U1bs3j6e3qWDr9Qn4cvq1JzncWvKY7tQL+4hmLSOUSIIajJobIPEmCai1 ok61QC0poafIrVv2iaxO+AbBLSIJ7BNrJjU/r+EtWVpKKiZe8qX3AGUSS983k1LeN9AL G+VQSyqTH4X1j0tI7iZy9yvMHaPD1FXpVWoKPjzR3GdmseCxF/0+PH5flDA0QOufjbFW Swlw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=jxjzCofAcULr7bnQMbpOvZCW6tNXMo722rzO5X3X+g0=; b=iuqHlf2IIW6GMVu2xVf5l+uYrO+XCSaIK0Wp/qv9VCcu9Em1fUMzUwPcJLMZ/VNuvQ IiEFhpuVJ3kdJjKL4hT4SlvDXtll/yGx2SlPLeyPAfnc2mwPsy1wne0q9oe1aLPA1o+h BJ4II6sIgLF7ndz3lP+yXZY3ODE9m4fycloElfTjoPopFMYF0GWagGwJFbXY/t/ccjG0 WY2JAWWAeSUQ6R+hg89x62XGaYZRk3KtP9Ny82TcbeJo6GigHL+7E0V14jJjDuYLa+t0 McTVC5z1TJtkUS2NNqYub7AJg3w0veQ1nK3+iyJQ3g0OseRdDEqCN10AwkzjJ7NHfGV0 XVPA== X-Gm-Message-State: AOAM533/kqB1XhfpIp/rgXMnc87cB/FfnLhn5srPqBiJ2emmfRA5Qjum LWRWzEPhN5vylSJQ569iYmvOFA== X-Google-Smtp-Source: ABdhPJwhfw1qSbAuBw+1RM7qEp36SUEwopFNg3Q2irPAgag3zrJxkL8XTXGjZkVkY1IB6k2hYZijDg== X-Received: by 2002:a17:90b:390b:: with SMTP id ob11mr11955031pjb.66.1607249893914; Sun, 06 Dec 2020 02:18:13 -0800 (PST) Received: from localhost.localdomain ([103.136.221.70]) by smtp.gmail.com with ESMTPSA id g16sm10337657pfb.201.2020.12.06.02.18.03 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 06 Dec 2020 02:18:13 -0800 (PST) From: Muchun Song To: gregkh@linuxfoundation.org, rafael@kernel.org, adobriyan@gmail.com, akpm@linux-foundation.org, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, hughd@google.com, will@kernel.org, guro@fb.com, rppt@kernel.org, tglx@linutronix.de, esyr@redhat.com, peterx@redhat.com, krisman@collabora.com, surenb@google.com, avagin@openvz.org, elver@google.com, rdunlap@infradead.org, iamjoonsoo.kim@lge.com Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, Muchun Song Subject: [RESEND PATCH v2 09/12] mm: memcontrol: convert vmstat slab counters to bytes Date: Sun, 6 Dec 2020 18:14:48 +0800 Message-Id: <20201206101451.14706-10-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20201206101451.14706-1-songmuchun@bytedance.com> References: <20201206101451.14706-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: the global and per-node counters are stored in pages, however memcg and lruvec counters are stored in bytes. This scheme looks weird. So convert all vmstat slab counters to bytes. Signed-off-by: Muchun Song --- include/linux/vmstat.h | 17 ++++++++++------- mm/vmstat.c | 21 ++++++++++----------- 2 files changed, 20 insertions(+), 18 deletions(-) diff --git a/include/linux/vmstat.h b/include/linux/vmstat.h index 322dcbfcc933..fd1a3d5d4926 100644 --- a/include/linux/vmstat.h +++ b/include/linux/vmstat.h @@ -197,18 +197,26 @@ static inline unsigned long global_node_page_state_pages(enum node_stat_item item) { long x = atomic_long_read(&vm_node_stat[item]); + #ifdef CONFIG_SMP if (x < 0) x = 0; #endif + if (vmstat_item_in_bytes(item)) + x >>= PAGE_SHIFT; return x; } static inline unsigned long global_node_page_state(enum node_stat_item item) { - VM_WARN_ON_ONCE(vmstat_item_in_bytes(item)); + long x = atomic_long_read(&vm_node_stat[item]); - return global_node_page_state_pages(item); + VM_WARN_ON_ONCE(vmstat_item_in_bytes(item)); +#ifdef CONFIG_SMP + if (x < 0) + x = 0; +#endif + return x; } static inline unsigned long zone_page_state(struct zone *zone, @@ -312,11 +320,6 @@ static inline void __mod_zone_page_state(struct zone *zone, static inline void __mod_node_page_state(struct pglist_data *pgdat, enum node_stat_item item, int delta) { - if (vmstat_item_in_bytes(item)) { - VM_WARN_ON_ONCE(delta & (PAGE_SIZE - 1)); - delta >>= PAGE_SHIFT; - } - node_page_state_add(delta, pgdat, item); } diff --git a/mm/vmstat.c b/mm/vmstat.c index 8d77ee426e22..7fb0c7cb9516 100644 --- a/mm/vmstat.c +++ b/mm/vmstat.c @@ -345,11 +345,6 @@ void __mod_node_page_state(struct pglist_data *pgdat, enum node_stat_item item, long x; long t; - if (vmstat_item_in_bytes(item)) { - VM_WARN_ON_ONCE(delta & (PAGE_SIZE - 1)); - delta >>= PAGE_SHIFT; - } - x = delta + __this_cpu_read(*p); t = __this_cpu_read(pcp->stat_threshold); @@ -554,11 +549,6 @@ static inline void mod_node_state(struct pglist_data *pgdat, s8 __percpu *p = pcp->vm_node_stat_diff + item; long o, n, t, z; - if (vmstat_item_in_bytes(item)) { - VM_WARN_ON_ONCE(delta & (PAGE_SIZE - 1)); - delta >>= PAGE_SHIFT; - } - do { z = 0; /* overflow to node counters */ @@ -1012,19 +1002,28 @@ unsigned long node_page_state_pages(struct pglist_data *pgdat, enum node_stat_item item) { long x = atomic_long_read(&pgdat->vm_stat[item]); + #ifdef CONFIG_SMP if (x < 0) x = 0; #endif + if (vmstat_item_in_bytes(item)) + x >>= PAGE_SHIFT; return x; } unsigned long node_page_state(struct pglist_data *pgdat, enum node_stat_item item) { + long x = atomic_long_read(&pgdat->vm_stat[item]); + VM_WARN_ON_ONCE(vmstat_item_in_bytes(item)); - return node_page_state_pages(pgdat, item); +#ifdef CONFIG_SMP + if (x < 0) + x = 0; +#endif + return x; } #endif