From patchwork Thu Sep 21 08:10:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yosry Ahmed X-Patchwork-Id: 13393759 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6B99AE706E5 for ; Thu, 21 Sep 2023 08:11:09 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5D5716B0195; Thu, 21 Sep 2023 04:11:08 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4C1F66B0198; Thu, 21 Sep 2023 04:11:08 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2EC8B6B01F0; Thu, 21 Sep 2023 04:11:08 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 13BE86B0195 for ; Thu, 21 Sep 2023 04:11:08 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id CFEBA8102F for ; Thu, 21 Sep 2023 08:11:07 +0000 (UTC) X-FDA: 81259884174.25.487737F Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) by imf22.hostedemail.com (Postfix) with ESMTP id 22348C001D for ; Thu, 21 Sep 2023 08:11:05 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=1KtSUNu1; spf=pass (imf22.hostedemail.com: domain of 3mfoLZQoKCOwmcgfmOVaSRUccUZS.QcaZWbil-aaYjOQY.cfU@flex--yosryahmed.bounces.google.com designates 209.85.128.202 as permitted sender) smtp.mailfrom=3mfoLZQoKCOwmcgfmOVaSRUccUZS.QcaZWbil-aaYjOQY.cfU@flex--yosryahmed.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1695283866; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=VUfaq4QWGo6Pgq9+Gc0pYjCdwcXd4ikm64XsTJ+V7zI=; b=G8d4TlZZ9LfjDppLt2/A/YEBMLnbaiuKcajYNMoFpBMvD28w+ocXo0r7Lehf6XgkSLBA4V rTxHrdZLUV5TPoCsNEzMCw5uEe70DjIatsxzytvlgxXFck218hALwcVhdKzTW6Ip4wGVYw ZHIBUQosjM/F9ChIW8KRNerH91kHxsI= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1695283866; a=rsa-sha256; cv=none; b=LZ+BwcoVDhunrgVM9a+FlDxbdZBS4ViKuStJSZ8mrr43zoQss5p3/2XRBnbgw9UuMAoKa0 l8oXRNOtCOcv5qGzc6AMqZFaGatAy0Ci6gBwWsyhb+aZjQYvOW8/ZsCKoMBkmy4zWoMrqR ff94Xb2iNyzIZEQ6g6MRNhYVu5N1a8w= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=1KtSUNu1; spf=pass (imf22.hostedemail.com: domain of 3mfoLZQoKCOwmcgfmOVaSRUccUZS.QcaZWbil-aaYjOQY.cfU@flex--yosryahmed.bounces.google.com designates 209.85.128.202 as permitted sender) smtp.mailfrom=3mfoLZQoKCOwmcgfmOVaSRUccUZS.QcaZWbil-aaYjOQY.cfU@flex--yosryahmed.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-59e78032ef9so10082157b3.2 for ; Thu, 21 Sep 2023 01:11:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1695283865; x=1695888665; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=VUfaq4QWGo6Pgq9+Gc0pYjCdwcXd4ikm64XsTJ+V7zI=; b=1KtSUNu1FsLC7ZYWkGZiLAVKN6Y8P1vkRyAg0llhiyZReNLYxhQP0199zs3psCZj9+ A1Zm7xp9wvMpeciLNuKqGlCaoo473FpEYlPU8M8hDegc450l5i3zpLQ3Kkou9St4soxI OHa8auqL4c7wmb+Ed/luhDGN11dYRqO9Nk+iaPKEHKm+DgeKag78YqoyNwygAayuRaTq LRaMqQccw43eJwTRfG5321GLtZCkGGO6a1Il0S/y1I5o0Ap1L2XkROFfS13Mem01SlMX G5Ftwd5iyuVbCuyd1OCbyUiOzq+AY8b2F9+WzrdM6bjJpp1oDS15xJVtDST6UFbn+fip xigA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695283865; x=1695888665; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=VUfaq4QWGo6Pgq9+Gc0pYjCdwcXd4ikm64XsTJ+V7zI=; b=lkhdYX79RVEBX4pnIOK8CzwSUCnp5ynSKr4I8vWGnod9Pa0kBWrfuzQnUy5UoQpYhs NUZFVvLmrXarg96liCAmMBBxJiIHTiJN9v19dLmp+O4dIdgh6PUvCvnYt/e6lrDtzWzP xO0OYWuSlMLIvfEOd+pelNAT1ghZk7M8CBWK0C1dj7jhfOBLuHiz7fhF8Wr7fatRCwbS dFMjsvdUOx9BXY9X61yf6jlzvTp/2TK1VOgm7wJA9HaOOOjHLZCF3XP1WgXk5rgtMDRY o1DrgSz7ZbycvXPsHyhrvOEk6111y3qoNIDU3I1inMLpFALzwX0r0GTm6cUQQ9Gt5j8L s/9Q== X-Gm-Message-State: AOJu0YxU5QLNIPRQdJh+eIt3i/d3Lxa5onGv4KaMqfCdrEv32r1DzARx 5wrJ1hDIY6iBtDLD1L7uHNiwjfP5e9RstRvj X-Google-Smtp-Source: AGHT+IFBE/oPIuCJlO97tHApRysO3VNA0vvMKpAee95h86ybMubWU1dHAIDYmvCWr5CQMyHTNET+SuyM9GansnHQ X-Received: from yosry.c.googlers.com ([fda3:e722:ac3:cc00:20:ed76:c0a8:29b4]) (user=yosryahmed job=sendgmr) by 2002:a81:b656:0:b0:59b:f493:813c with SMTP id h22-20020a81b656000000b0059bf493813cmr74625ywk.9.1695283865279; Thu, 21 Sep 2023 01:11:05 -0700 (PDT) Date: Thu, 21 Sep 2023 08:10:54 +0000 In-Reply-To: <20230921081057.3440885-1-yosryahmed@google.com> Mime-Version: 1.0 References: <20230921081057.3440885-1-yosryahmed@google.com> X-Mailer: git-send-email 2.42.0.459.ge4e396fd5e-goog Message-ID: <20230921081057.3440885-3-yosryahmed@google.com> Subject: [PATCH 2/5] mm: memcg: move vmstats structs definition above flushing code From: Yosry Ahmed To: Andrew Morton Cc: Johannes Weiner , Michal Hocko , Roman Gushchin , Shakeel Butt , Muchun Song , Ivan Babrou , Tejun Heo , " =?utf-8?q?Michal_Koutn=C3=BD?= " , Waiman Long , kernel-team@cloudflare.com, Wei Xu , Greg Thelen , linux-mm@kvack.org, cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, Yosry Ahmed X-Rspamd-Queue-Id: 22348C001D X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: jn7sob6xsawar439ri43de3y7jpj9kt6 X-HE-Tag: 1695283865-418841 X-HE-Meta: U2FsdGVkX19ufXBSq1kqsRloaj0fccC0+lPeYgZD9pN2/7YZMq/96JgflAJitz8FjKQYCSaLgMfdayS/56nh3kNE7k75JmjkxttqVb5ijD08UOOrg1BOSsRIzcAI/lhHbvxUbX1uBlzDjo//qN/8qpfwXptDpeIiKfhO5g/Q9JianCMZD1u6fQolIp6W4vr/gEuWKRl71sSXSJKTWznom/60zz/uhbL4R1cYKVFDgyJinc6Bk5r1ei4+7yYDMd0l2U3V3q2jbgH0ayaE4zVaxewE//7FwtokdWY3n2eGBvhD4EU8Mx1JPzIf+coiOHt5rGR+0vKmf6mt7QwMpiwzMvEgJJAMzJHsdTkNMWkEXGtk1pUYiOPhZoo2WqSwwC9kPGeHLKaXLwQw6kPp36Ox6RzwSGwJWknGmDiluG/+wvUuvV6RvbPUaqM8p34JYOCkATiOcnPTc8b/oBAUl8IUoKlmplsZbDqdkQvDBKixINsATj5Nc9Eacb1Jmu3I/Cq/Vr3oTgbIyYtV/8WisnaiGkOi9GZQKz5G6mltJm4/IgX92I/fmMXPicwV97amtGXYgc+Jq7DxeRr1s+qah8hnc08OohXuk6HdR5tS5cw62Muf5WDWsR7bPKBuOXeHpzzdLe53dqCj09LqNf95WHUFSfbRAi9lTl1PGtlxGy3ErwGnQEX+Cqml2q+4RbTllroXm/+pfNvYBe19JjuE12vhJY1HXziMnvPWTpiHuTsTSs5eopWMl1Pjad/1W3DI2o0ocHU4vHE//3ajjKX7xUnwO0cnntj8Cu4dOYsxCfKySFiQJ8RSYL8ZtjGuqWlaN3kCm4FzPMCjmYmCjzVQfDnjlA6VKAh4LD31rnxtUw3nbDsKNTVhwhu7l0S3FXPmH0v0FVeSr4AG/EYe4p/GoB0FaMaveQJlVe6fwbur5+YrUO2oKTfW0wQf+xpSjk8BOjlhoRfcYky0av0RL9FVLz8 oNb6nGEw srR8q80jFXEkVQsm3ah1oMLQp2sXU/cw+luu93lu86/UVgyZnKnzEPayzKE7YyCp2L32tHkrIi1TFKVxsAhcRNpU6p9biCxiMlTbZq830IygXHdbd5scmlkebS2FjBTWBjv7kRQ2Kdfuf61OaI4T8aezoa5opodeGOaqUb4kr1PoJ5sneXHY37C3/okKWMNvTLP8kAEHeUIneWaUoZpAxn7sMjNys92NhfO9GIdtJ79WYcuTboC+iSzLJyj3aictCPMO8X3cP4IKGjq1LqRHDgchh2GSZIzZmHvd4X1VIT8LzZbjJX1Lqj0jECE77wxsohKc7Kx6NF2KuOt7RN8Pn/dq/KLcr8hcLNTgtbIMTUW4xInXWY13z8xYx63Vov0l7V9bNSUYhmoMuIZjUCvxAnfXkfwgPkYUsBGKkg+FfbZqM5NDZYU7G+miPs6X731YfgasRD7NWQyNndwO5aDje5jPFNVs7nFMOUsJN+JerxgHQUOY+KNejzxeFhSHcww7UL+aU+iKNIyJqV4PMEjXjEb0aEqAyAZCD0MNyPYnF74CE0oRl5rPZSOcbH8RSTwLu1tXeFA27LQM9VBUyhv6Am+5E9LwbLexYxlvzfrp+ZOAeBhK4v9Qnrn5793AbVnD9uLTy X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The following patch will make use of those structs in the flushing code, so move their definitions (and a few other dependencies) a little bit up to reduce the diff noise in the following patch. No functional change intended. Signed-off-by: Yosry Ahmed --- mm/memcontrol.c | 146 ++++++++++++++++++++++++------------------------ 1 file changed, 73 insertions(+), 73 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 49562dfdeab2..ef7ad66a9e4c 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -570,6 +570,79 @@ mem_cgroup_largest_soft_limit_node(struct mem_cgroup_tree_per_node *mctz) return mz; } +/* Subset of vm_event_item to report for memcg event stats */ +static const unsigned int memcg_vm_event_stat[] = { + PGPGIN, + PGPGOUT, + PGSCAN_KSWAPD, + PGSCAN_DIRECT, + PGSCAN_KHUGEPAGED, + PGSTEAL_KSWAPD, + PGSTEAL_DIRECT, + PGSTEAL_KHUGEPAGED, + PGFAULT, + PGMAJFAULT, + PGREFILL, + PGACTIVATE, + PGDEACTIVATE, + PGLAZYFREE, + PGLAZYFREED, +#if defined(CONFIG_MEMCG_KMEM) && defined(CONFIG_ZSWAP) + ZSWPIN, + ZSWPOUT, +#endif +#ifdef CONFIG_TRANSPARENT_HUGEPAGE + THP_FAULT_ALLOC, + THP_COLLAPSE_ALLOC, + THP_SWPOUT, + THP_SWPOUT_FALLBACK, +#endif +}; + +#define NR_MEMCG_EVENTS ARRAY_SIZE(memcg_vm_event_stat) +static int mem_cgroup_events_index[NR_VM_EVENT_ITEMS] __read_mostly; + +static void init_memcg_events(void) +{ + int i; + + for (i = 0; i < NR_MEMCG_EVENTS; ++i) + mem_cgroup_events_index[memcg_vm_event_stat[i]] = i + 1; +} + +static inline int memcg_events_index(enum vm_event_item idx) +{ + return mem_cgroup_events_index[idx] - 1; +} + +struct memcg_vmstats_percpu { + /* Local (CPU and cgroup) page state & events */ + long state[MEMCG_NR_STAT]; + unsigned long events[NR_MEMCG_EVENTS]; + + /* Delta calculation for lockless upward propagation */ + long state_prev[MEMCG_NR_STAT]; + unsigned long events_prev[NR_MEMCG_EVENTS]; + + /* Cgroup1: threshold notifications & softlimit tree updates */ + unsigned long nr_page_events; + unsigned long targets[MEM_CGROUP_NTARGETS]; +}; + +struct memcg_vmstats { + /* Aggregated (CPU and subtree) page state & events */ + long state[MEMCG_NR_STAT]; + unsigned long events[NR_MEMCG_EVENTS]; + + /* Non-hierarchical (CPU aggregated) page state & events */ + long state_local[MEMCG_NR_STAT]; + unsigned long events_local[NR_MEMCG_EVENTS]; + + /* Pending child counts during tree propagation */ + long state_pending[MEMCG_NR_STAT]; + unsigned long events_pending[NR_MEMCG_EVENTS]; +}; + /* * memcg and lruvec stats flushing * @@ -681,79 +754,6 @@ static void flush_memcg_stats_dwork(struct work_struct *w) queue_delayed_work(system_unbound_wq, &stats_flush_dwork, FLUSH_TIME); } -/* Subset of vm_event_item to report for memcg event stats */ -static const unsigned int memcg_vm_event_stat[] = { - PGPGIN, - PGPGOUT, - PGSCAN_KSWAPD, - PGSCAN_DIRECT, - PGSCAN_KHUGEPAGED, - PGSTEAL_KSWAPD, - PGSTEAL_DIRECT, - PGSTEAL_KHUGEPAGED, - PGFAULT, - PGMAJFAULT, - PGREFILL, - PGACTIVATE, - PGDEACTIVATE, - PGLAZYFREE, - PGLAZYFREED, -#if defined(CONFIG_MEMCG_KMEM) && defined(CONFIG_ZSWAP) - ZSWPIN, - ZSWPOUT, -#endif -#ifdef CONFIG_TRANSPARENT_HUGEPAGE - THP_FAULT_ALLOC, - THP_COLLAPSE_ALLOC, - THP_SWPOUT, - THP_SWPOUT_FALLBACK, -#endif -}; - -#define NR_MEMCG_EVENTS ARRAY_SIZE(memcg_vm_event_stat) -static int mem_cgroup_events_index[NR_VM_EVENT_ITEMS] __read_mostly; - -static void init_memcg_events(void) -{ - int i; - - for (i = 0; i < NR_MEMCG_EVENTS; ++i) - mem_cgroup_events_index[memcg_vm_event_stat[i]] = i + 1; -} - -static inline int memcg_events_index(enum vm_event_item idx) -{ - return mem_cgroup_events_index[idx] - 1; -} - -struct memcg_vmstats_percpu { - /* Local (CPU and cgroup) page state & events */ - long state[MEMCG_NR_STAT]; - unsigned long events[NR_MEMCG_EVENTS]; - - /* Delta calculation for lockless upward propagation */ - long state_prev[MEMCG_NR_STAT]; - unsigned long events_prev[NR_MEMCG_EVENTS]; - - /* Cgroup1: threshold notifications & softlimit tree updates */ - unsigned long nr_page_events; - unsigned long targets[MEM_CGROUP_NTARGETS]; -}; - -struct memcg_vmstats { - /* Aggregated (CPU and subtree) page state & events */ - long state[MEMCG_NR_STAT]; - unsigned long events[NR_MEMCG_EVENTS]; - - /* Non-hierarchical (CPU aggregated) page state & events */ - long state_local[MEMCG_NR_STAT]; - unsigned long events_local[NR_MEMCG_EVENTS]; - - /* Pending child counts during tree propagation */ - long state_pending[MEMCG_NR_STAT]; - unsigned long events_pending[NR_MEMCG_EVENTS]; -}; - unsigned long memcg_page_state(struct mem_cgroup *memcg, int idx) { long x = READ_ONCE(memcg->vmstats->state[idx]);