From patchwork Tue Apr 30 06:06:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shakeel Butt X-Patchwork-Id: 13648287 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 162E4C10F16 for ; Tue, 30 Apr 2024 06:06:29 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 87B296B0095; Tue, 30 Apr 2024 02:06:28 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8039C6B0096; Tue, 30 Apr 2024 02:06:28 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 67DEC6B0098; Tue, 30 Apr 2024 02:06:28 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 424A26B0095 for ; Tue, 30 Apr 2024 02:06:28 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id E70B3120601 for ; Tue, 30 Apr 2024 06:06:27 +0000 (UTC) X-FDA: 82065163614.13.62CC85B Received: from out-175.mta0.migadu.com (out-175.mta0.migadu.com [91.218.175.175]) by imf29.hostedemail.com (Postfix) with ESMTP id 3CC3012000D for ; Tue, 30 Apr 2024 06:06:25 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=gWysJ5rJ; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf29.hostedemail.com: domain of shakeel.butt@linux.dev designates 91.218.175.175 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1714457186; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Wvw793twJ1F4/KqGjGhIdLueYM54shZE8VvQWV77fqg=; b=uDFaCY5U+DWTA2x65uL5Ew++rbKHsU42tsILkQiQfNO04NkZ0ypUg33Ne0myuMBuZbeKmn Bj1f/TiIRkJwKCLcSbeoQMd9Y1ezDIjBj6QeLNQE4VBSVPEoIKRo5apk0oZREdbLwgTO/O mJNYkKTEvd5ZQ5q15D3PKFMCckKF194= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=gWysJ5rJ; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf29.hostedemail.com: domain of shakeel.butt@linux.dev designates 91.218.175.175 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1714457186; a=rsa-sha256; cv=none; b=KLGJ6VaNcfksP3ozprgSTagcqo4PV+09BoKmOlqV8ybt8XgGdLe3sA7CM2x3YvrAkh4yVB 6f/wEmq8zAkHL1Bwlx2oLQhXzCrPf+lXMJixtYvpHXILuOveYxLpONOa3+nVz0N5nOvgZw mZRVuOUAflSEkCueSuhOVfCpB4E2L50= X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1714457184; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Wvw793twJ1F4/KqGjGhIdLueYM54shZE8VvQWV77fqg=; b=gWysJ5rJLgTNd7+7sPEXX3lDa/aX41N7fiFaoVJoqBw2GLQoTmpvYSXD+j56elG71KufDu 5H3chniFPjB0Kb5+t8HX4JvT1HtVlVG8HX9EX65NGXLDJ1eBuARlgTh71roMJCtj96Mtw2 +qgkfyNgcm3UlGBUDOM5HOeHYGXdz14= From: Shakeel Butt To: Andrew Morton , Johannes Weiner , Michal Hocko , Roman Gushchin , Muchun Song , Yosry Ahmed , "T . J . Mercier" Cc: kernel-team@meta.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v3 1/8] memcg: reduce memory size of mem_cgroup_events_index Date: Mon, 29 Apr 2024 23:06:05 -0700 Message-ID: <20240430060612.2171650-2-shakeel.butt@linux.dev> In-Reply-To: <20240430060612.2171650-1-shakeel.butt@linux.dev> References: <20240430060612.2171650-1-shakeel.butt@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 3CC3012000D X-Stat-Signature: 3ruchfyspu55yk9io3137so3ppmmxe4w X-Rspam-User: X-HE-Tag: 1714457185-458312 X-HE-Meta: U2FsdGVkX18Rjxk3S5JGnX6f2/2xfzgByfRiZA+JkQfigbpUTWhAcYpPkHP8umIdgXaYdINfQWhyaZfC2x4tqCnCPoqwEoxK0x7hDNjqe75rgsvimguP6Uj1qhdaER0ANbIFJZWCA0UUfvRbBe3MirMPDFPq/GM6wUCfBDhx98zPC3RMP1ZbQdbZ0q/wOb3Irl/the1UnTcCXfv8IGi3wU+y09aUv+i84OYaCooxc93vULUcs1FwWQf8OIpZW0zr/4agH4E+GObFhgmCiJr/Y42eF0BkA3wSnEuTmikxOjTDnNVoHRyUEldP2QgDvwW8qE2+iI6Ls8IPGPCzbwFPlwYZTLHTsso9IX5m1kNscbSb5vWQaiQBoRFdMpY4FlHXsLZ7sCl0K3zU4go5xGkHG7qFsK6IdpmHAHPEyT0MjWjz7VipELc6BaeJ8TgCQQlwupDnmbdHm4yfNXVKWZbst/v2v5aefsciIOryufmisjZ03xgWkRU92+bUiqtdJYXqZ8znrk7kil8qhxEhgK4ZVAXRYmmDts/u1uVcIK2sL5efNv8NVT6+Ys3wR0qu0qzMur9nRJyS8EHob5Vm5D0GZF68CcHt4foeXPQKaMx6Z1LgEatfnWir5oiCecdEtzr0ZUV1FlRf08FwQhy98m64TJJPuj5RPZvnseH8wVxUm+e+rbS/E+sWTN2NaGkF7BBgu235Qai952Af5ZnMEPlipHsWWbP1Cp/9wB41cj6CRrWEjWPQBiv1/A3Y5N7sz6a7JuyDcde/r85ili3zQY8o7w8s1AlInhxTOd+xhGqiywGPQKGt8weXWMH63zBnDLfl+nWNHTRbcssgVrQDeM+DP5cjB2xgHH/oyxvsKLUn+Twf6WGnJEwf61wWpRRBfo1JR5T8V0KAqcAm+R4vWUnim0WxmeeFzeAb+cQbUWz3r7esjADwUbuD7Y68h0nQGlrk1T+UKYqLgvGazPm2HXo uE03bqrf H1kj3fUzp6XcuAsIfhXLEsaj7MrwBH6inWac7DacME5TBpuwdoFqJJzNNj+NIAY6OuttN5wXSrg/bplDh7GwllZdcc+2VKSjS5Otnzbh2Xi5DlelgNtD8/M+quY8/0awVson8 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: mem_cgroup_events_index is a translation table to get the right index of the memcg relevant entry for the general vm_event_item. At the moment, it is defined as integer array. However on a typical system the max entry of vm_event_item (NR_VM_EVENT_ITEMS) is 113, so we don't need to use int as storage type of the array. For now just use int8_t as type and add a BUILD_BUG_ON() and will switch to short once NR_VM_EVENT_ITEMS touches 127. Another benefit of this change is that the translation table fits in 2 cachelines while previously it would require 8 cachelines (assuming 64 bytes cachesline). Signed-off-by: Shakeel Butt Reviewed-by: Roman Gushchin Reviewed-by: Yosry Ahmed Reviewed-by: T.J. Mercier --- Changes since v2: - Used S8_MAX instead of 127 - Update commit message based on Yosry's feedback. mm/memcontrol.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 602ad5faad4d..c146187cda9c 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -607,11 +607,13 @@ static const unsigned int memcg_vm_event_stat[] = { }; #define NR_MEMCG_EVENTS ARRAY_SIZE(memcg_vm_event_stat) -static int mem_cgroup_events_index[NR_VM_EVENT_ITEMS] __read_mostly; +static int8_t mem_cgroup_events_index[NR_VM_EVENT_ITEMS] __read_mostly; static void init_memcg_events(void) { - int i; + int8_t i; + + BUILD_BUG_ON(NR_VM_EVENT_ITEMS >= S8_MAX); for (i = 0; i < NR_MEMCG_EVENTS; ++i) mem_cgroup_events_index[memcg_vm_event_stat[i]] = i + 1; From patchwork Tue Apr 30 06:06:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shakeel Butt X-Patchwork-Id: 13648288 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B88B8C4345F for ; Tue, 30 Apr 2024 06:06:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4AAF96B0096; Tue, 30 Apr 2024 02:06:31 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4331F6B0098; Tue, 30 Apr 2024 02:06:31 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 211986B0099; Tue, 30 Apr 2024 02:06:31 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id E79E46B0096 for ; Tue, 30 Apr 2024 02:06:30 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 7D39BA0F29 for ; Tue, 30 Apr 2024 06:06:30 +0000 (UTC) X-FDA: 82065163740.16.C6EB41B Received: from out-183.mta0.migadu.com (out-183.mta0.migadu.com [91.218.175.183]) by imf29.hostedemail.com (Postfix) with ESMTP id CBEE0120003 for ; Tue, 30 Apr 2024 06:06:28 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=je6PYpsu; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf29.hostedemail.com: domain of shakeel.butt@linux.dev designates 91.218.175.183 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1714457189; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=TJurZ26Ei0Ypt+z9OF2t2IjIht8hsBt7wgbBwSaQpf8=; b=fmJwxCnJywGj9IAHWn48gpxkxDdHQdBFXIcZNiZSitx8yswlmJ9lfxSjN1SXYc9HdXifO+ QRogV2lKrhA9ADmcEJCxBRcZAeWk599g6y7YtoaxsPOftXC4NNbye5ALJm6+Els8D3kKGk F+WTGITLa1Ji0RT4YTXrcYqq3dzPq8E= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1714457189; a=rsa-sha256; cv=none; b=lpDIVdKshDpo4qh1EVv83adoLor2YuJUnmdsJaVqS5Hbw2v2aDYpZox1cHbBSbShVW+T6c iU6QKu4o3y9TufunHy4Jh/J3I52bxWCXLaOL+BI8YsAdmDLiffWLDmDI1n35wT69erUbdx +SoYP6ISMalNhPi9dw5YnP9L+oK+b1I= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=je6PYpsu; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf29.hostedemail.com: domain of shakeel.butt@linux.dev designates 91.218.175.183 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1714457187; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=TJurZ26Ei0Ypt+z9OF2t2IjIht8hsBt7wgbBwSaQpf8=; b=je6PYpsuWMMT4ClDbZOM+FDe6Wa7t5KPl7NGfnZGypdcykVsLQ7bzMC/DUlhoH4PbjhXOw kSljsOgicb7sCAwL0LnTAbu6ZwNNVnO66ZdtMa3H8yCDKLM4rtyAKijVBa7NOETK6dJIAT WERHD4+A0LUkC53y/Rl7J/LAm+rUWzM= From: Shakeel Butt To: Andrew Morton , Johannes Weiner , Michal Hocko , Roman Gushchin , Muchun Song , Yosry Ahmed , "T . J . Mercier" Cc: kernel-team@meta.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v3 2/8] memcg: dynamically allocate lruvec_stats Date: Mon, 29 Apr 2024 23:06:06 -0700 Message-ID: <20240430060612.2171650-3-shakeel.butt@linux.dev> In-Reply-To: <20240430060612.2171650-1-shakeel.butt@linux.dev> References: <20240430060612.2171650-1-shakeel.butt@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Rspamd-Queue-Id: CBEE0120003 X-Rspam-User: X-Rspamd-Server: rspam03 X-Stat-Signature: 8wxujr919xcgmdie894bi1m51oirkjah X-HE-Tag: 1714457188-228854 X-HE-Meta: U2FsdGVkX18J0fYf1Px7/T3l5WXlKnshwGtITIktsEb1R0waX/rSjWigC7oxLw2k81XDIq6msvz3stSiwbPtIxzB8dFM1WjVP3Xd2Oc3rDMjzih4gY1C4Z56ooEsHJ/k/Z/nV+XqgyKaY9PYfpMD0LuAA4Pg3zyQRMBVOmmCOzZ6+iyO02zYDM5sQ+kjBK684Wa34tGlTyVo5Ctn29mAR94aKZQbctWWV82e/HDbBYACsv/qLecaO09d/TpGrH/I7PbiodESgExaiyj3cM8BorhmsxMjLKyP55lwIDBjBBhs4KR1BhwbnwOugGr8YnRahQDZApmhe2C2mA8P3gtmGPQSIRQaaSBVnKxObfNOyWZsYCBCU2l4TFv0ywl1RiMI3n6A4nrLxEmWbke8wFicCUv/q1j8YgjMAenS1XXNfEKEgJM+rC1cSF1TI1UpNNcbuiZAPrtjjpPksYQxAMQrvMDqY8se0Pl8vb0CAjJToaVFSQL56hJlQ6ANYwvc8mNZ6ENP70fdBc9+gCVBqUerG8u4QesnW+C/bARdxmtcbjZhzRRfXM9/vKlLcNRxJ1ofbuAILWFj3ZdmL00iYAsSgvHkRbFeg7W+ZsFLsWmmUyGQucs52wEbKNE9BlXcBJGO1WL3pgt8VbwwCcF7xBXY1Z8WnSisUTIsxYdNZeTg6jysyl0J4NeQWJgxm3Xe15zRmUYkFr4TOKSMubUNvcONy0BOFDHhoh7n+CwirlOEykPjGtWE+vZyDSd1JsgQCtGbGmeNrcZT/6bDt/9wckp3/dkDGJdaILFTfrriD+Hi+zU7cWud15r7a+EK0Bo8TPjgK2wutDgUOrAmKKkQYx1nXbn6sAbNSmzg1zz//CLIziGXqyyBAJGvwfCuGTsApph9BLtyvnXKFvmvr7km1vwtImVo2B8EfkQ/2xJaTSykz3X/zDW9E7VYL8IUgA9OKf9QirFra3a0VyywIcK3sUX I/Ie2RpQ eVWkcPbF2+3EhyBrrch4+wGxdZe47QldkCJK5HrG0SttVga9lAkJP4bGwhDzca3LshhH6fdQQSry9Pg2qM9s+8Tc4htBclgGfBfgEr+A/sBFXH4ZSLlCllQQOtAkQI+p262pG1lvCwvPIZyGwpD/fNpqqWA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: To decouple the dependency of lruvec_stats on NR_VM_NODE_STAT_ITEMS, we need to dynamically allocate lruvec_stats in the mem_cgroup_per_node structure. Also move the definition of lruvec_stats_percpu and lruvec_stats and related functions to the memcontrol.c to facilitate later patches. No functional changes in the patch. Signed-off-by: Shakeel Butt Reviewed-by: Yosry Ahmed Reviewed-by: T.J. Mercier --- Changes since v2: - N/A include/linux/memcontrol.h | 62 +++------------------------ mm/memcontrol.c | 87 ++++++++++++++++++++++++++++++++------ 2 files changed, 81 insertions(+), 68 deletions(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 9aba0d0462ca..ab8a6e884375 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -83,6 +83,8 @@ enum mem_cgroup_events_target { struct memcg_vmstats_percpu; struct memcg_vmstats; +struct lruvec_stats_percpu; +struct lruvec_stats; struct mem_cgroup_reclaim_iter { struct mem_cgroup *position; @@ -90,25 +92,6 @@ struct mem_cgroup_reclaim_iter { unsigned int generation; }; -struct lruvec_stats_percpu { - /* Local (CPU and cgroup) state */ - long state[NR_VM_NODE_STAT_ITEMS]; - - /* Delta calculation for lockless upward propagation */ - long state_prev[NR_VM_NODE_STAT_ITEMS]; -}; - -struct lruvec_stats { - /* Aggregated (CPU and subtree) state */ - long state[NR_VM_NODE_STAT_ITEMS]; - - /* Non-hierarchical (CPU aggregated) state */ - long state_local[NR_VM_NODE_STAT_ITEMS]; - - /* Pending child counts during tree propagation */ - long state_pending[NR_VM_NODE_STAT_ITEMS]; -}; - /* * per-node information in memory controller. */ @@ -116,7 +99,7 @@ struct mem_cgroup_per_node { struct lruvec lruvec; struct lruvec_stats_percpu __percpu *lruvec_stats_percpu; - struct lruvec_stats lruvec_stats; + struct lruvec_stats *lruvec_stats; unsigned long lru_zone_size[MAX_NR_ZONES][NR_LRU_LISTS]; @@ -1037,42 +1020,9 @@ static inline void mod_memcg_page_state(struct page *page, } unsigned long memcg_page_state(struct mem_cgroup *memcg, int idx); - -static inline unsigned long lruvec_page_state(struct lruvec *lruvec, - enum node_stat_item idx) -{ - struct mem_cgroup_per_node *pn; - long x; - - if (mem_cgroup_disabled()) - return node_page_state(lruvec_pgdat(lruvec), idx); - - pn = container_of(lruvec, struct mem_cgroup_per_node, lruvec); - x = READ_ONCE(pn->lruvec_stats.state[idx]); -#ifdef CONFIG_SMP - if (x < 0) - x = 0; -#endif - return x; -} - -static inline unsigned long lruvec_page_state_local(struct lruvec *lruvec, - enum node_stat_item idx) -{ - struct mem_cgroup_per_node *pn; - long x = 0; - - if (mem_cgroup_disabled()) - return node_page_state(lruvec_pgdat(lruvec), idx); - - pn = container_of(lruvec, struct mem_cgroup_per_node, lruvec); - x = READ_ONCE(pn->lruvec_stats.state_local[idx]); -#ifdef CONFIG_SMP - if (x < 0) - x = 0; -#endif - return x; -} +unsigned long lruvec_page_state(struct lruvec *lruvec, enum node_stat_item idx); +unsigned long lruvec_page_state_local(struct lruvec *lruvec, + enum node_stat_item idx); void mem_cgroup_flush_stats(struct mem_cgroup *memcg); void mem_cgroup_flush_stats_ratelimited(struct mem_cgroup *memcg); diff --git a/mm/memcontrol.c b/mm/memcontrol.c index c146187cda9c..7126459ec56a 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -576,6 +576,60 @@ mem_cgroup_largest_soft_limit_node(struct mem_cgroup_tree_per_node *mctz) return mz; } +struct lruvec_stats_percpu { + /* Local (CPU and cgroup) state */ + long state[NR_VM_NODE_STAT_ITEMS]; + + /* Delta calculation for lockless upward propagation */ + long state_prev[NR_VM_NODE_STAT_ITEMS]; +}; + +struct lruvec_stats { + /* Aggregated (CPU and subtree) state */ + long state[NR_VM_NODE_STAT_ITEMS]; + + /* Non-hierarchical (CPU aggregated) state */ + long state_local[NR_VM_NODE_STAT_ITEMS]; + + /* Pending child counts during tree propagation */ + long state_pending[NR_VM_NODE_STAT_ITEMS]; +}; + +unsigned long lruvec_page_state(struct lruvec *lruvec, enum node_stat_item idx) +{ + struct mem_cgroup_per_node *pn; + long x; + + if (mem_cgroup_disabled()) + return node_page_state(lruvec_pgdat(lruvec), idx); + + pn = container_of(lruvec, struct mem_cgroup_per_node, lruvec); + x = READ_ONCE(pn->lruvec_stats->state[idx]); +#ifdef CONFIG_SMP + if (x < 0) + x = 0; +#endif + return x; +} + +unsigned long lruvec_page_state_local(struct lruvec *lruvec, + enum node_stat_item idx) +{ + struct mem_cgroup_per_node *pn; + long x = 0; + + if (mem_cgroup_disabled()) + return node_page_state(lruvec_pgdat(lruvec), idx); + + pn = container_of(lruvec, struct mem_cgroup_per_node, lruvec); + x = READ_ONCE(pn->lruvec_stats->state_local[idx]); +#ifdef CONFIG_SMP + if (x < 0) + x = 0; +#endif + return x; +} + /* Subset of vm_event_item to report for memcg event stats */ static const unsigned int memcg_vm_event_stat[] = { PGPGIN, @@ -5491,18 +5545,25 @@ static int alloc_mem_cgroup_per_node_info(struct mem_cgroup *memcg, int node) if (!pn) return 1; + pn->lruvec_stats = kzalloc_node(sizeof(struct lruvec_stats), GFP_KERNEL, + node); + if (!pn->lruvec_stats) + goto fail; + pn->lruvec_stats_percpu = alloc_percpu_gfp(struct lruvec_stats_percpu, GFP_KERNEL_ACCOUNT); - if (!pn->lruvec_stats_percpu) { - kfree(pn); - return 1; - } + if (!pn->lruvec_stats_percpu) + goto fail; lruvec_init(&pn->lruvec); pn->memcg = memcg; memcg->nodeinfo[node] = pn; return 0; +fail: + kfree(pn->lruvec_stats); + kfree(pn); + return 1; } static void free_mem_cgroup_per_node_info(struct mem_cgroup *memcg, int node) @@ -5513,6 +5574,7 @@ static void free_mem_cgroup_per_node_info(struct mem_cgroup *memcg, int node) return; free_percpu(pn->lruvec_stats_percpu); + kfree(pn->lruvec_stats); kfree(pn); } @@ -5865,18 +5927,19 @@ static void mem_cgroup_css_rstat_flush(struct cgroup_subsys_state *css, int cpu) for_each_node_state(nid, N_MEMORY) { struct mem_cgroup_per_node *pn = memcg->nodeinfo[nid]; - struct mem_cgroup_per_node *ppn = NULL; + struct lruvec_stats *lstats = pn->lruvec_stats; + struct lruvec_stats *plstats = NULL; struct lruvec_stats_percpu *lstatc; if (parent) - ppn = parent->nodeinfo[nid]; + plstats = parent->nodeinfo[nid]->lruvec_stats; lstatc = per_cpu_ptr(pn->lruvec_stats_percpu, cpu); for (i = 0; i < NR_VM_NODE_STAT_ITEMS; i++) { - delta = pn->lruvec_stats.state_pending[i]; + delta = lstats->state_pending[i]; if (delta) - pn->lruvec_stats.state_pending[i] = 0; + lstats->state_pending[i] = 0; delta_cpu = 0; v = READ_ONCE(lstatc->state[i]); @@ -5887,12 +5950,12 @@ static void mem_cgroup_css_rstat_flush(struct cgroup_subsys_state *css, int cpu) } if (delta_cpu) - pn->lruvec_stats.state_local[i] += delta_cpu; + lstats->state_local[i] += delta_cpu; if (delta) { - pn->lruvec_stats.state[i] += delta; - if (ppn) - ppn->lruvec_stats.state_pending[i] += delta; + lstats->state[i] += delta; + if (plstats) + plstats->state_pending[i] += delta; } } } From patchwork Tue Apr 30 06:06:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shakeel Butt X-Patchwork-Id: 13648289 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 96D49C4345F for ; Tue, 30 Apr 2024 06:06:34 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 260546B0099; Tue, 30 Apr 2024 02:06:34 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1E9266B009A; Tue, 30 Apr 2024 02:06:34 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0B1A36B009B; Tue, 30 Apr 2024 02:06:34 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id DA7CB6B0099 for ; Tue, 30 Apr 2024 02:06:33 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 547061A0313 for ; Tue, 30 Apr 2024 06:06:33 +0000 (UTC) X-FDA: 82065163866.22.3F32336 Received: from out-182.mta0.migadu.com (out-182.mta0.migadu.com [91.218.175.182]) by imf11.hostedemail.com (Postfix) with ESMTP id 928F640011 for ; Tue, 30 Apr 2024 06:06:31 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=PeTHaWtH; spf=pass (imf11.hostedemail.com: domain of shakeel.butt@linux.dev designates 91.218.175.182 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev; dmarc=pass (policy=none) header.from=linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1714457191; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=5r5hMtfswp5qoj9glT20FCqOKYLmIdRrP0lNWsirtps=; b=HsvAdTX8AP5hY39z8TB+Pu3iYGt6Ijclwj+HlxoAIIe1sVbcyF1KDy2gWRTxj5KTpVC02N YqV9VEbcH9p8MobRhGxf+2yUOfPBAEzcZHdVyjFjz/RjWTjxtnHPnSzcHMrsXzi9kCPNNC Hxd0z2zHefGYhulO7jcKAckQHm+lHxM= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1714457191; a=rsa-sha256; cv=none; b=oMATbWNqMxgXlQ+5janmjNaYU16zJjtdujYatcVUDPLO0EVdPMk3LvHnV2OSbwzyL2x4+a W6bKqHtDqmziUfsiEsWnbf5aTMlJqqw5l+1nimmhqiu1JHXErc7k9dw8gBYdjN5YONAh4k b2paNArbrNY8eqTgKQOkWI/dKRmjuIA= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=PeTHaWtH; spf=pass (imf11.hostedemail.com: domain of shakeel.butt@linux.dev designates 91.218.175.182 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev; dmarc=pass (policy=none) header.from=linux.dev X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1714457190; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5r5hMtfswp5qoj9glT20FCqOKYLmIdRrP0lNWsirtps=; b=PeTHaWtHFNHgJkCXWCF/Ai07h2OyJw4wbWxj/hRwj0ynslSaUgA9yIq+xHiiRg6Is0gE5T dBPacjZehCXpuz9tiwffhX/AyogEmjHQ/UiWLdnv0irunhDrAvkdSSYKPHm5ydKRd5y1H8 xYbZIifAU4bef22QTLtYPQfKgrHJZPI= From: Shakeel Butt To: Andrew Morton , Johannes Weiner , Michal Hocko , Roman Gushchin , Muchun Song , Yosry Ahmed , "T . J . Mercier" Cc: kernel-team@meta.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v3 3/8] mm: memcg: account memory used for memcg vmstats and lruvec stats Date: Mon, 29 Apr 2024 23:06:07 -0700 Message-ID: <20240430060612.2171650-4-shakeel.butt@linux.dev> In-Reply-To: <20240430060612.2171650-1-shakeel.butt@linux.dev> References: <20240430060612.2171650-1-shakeel.butt@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Rspam-User: X-Stat-Signature: xnoyrgptzkgdwykn18xqoewnnb4h8b4p X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: 928F640011 X-HE-Tag: 1714457191-60108 X-HE-Meta: U2FsdGVkX19Xjs/ixjGBd4ireM/N3+i4FabruC/Kmgi9J/2lpLRcgxDUcnXgEih3QYQRiqjlf7brOlr24EhUIfc4AFL7Yatv26FpYDu+TIhoHubAXnqARtp0lisVQvYSxVxp/FjOk0qnlTO3bKiRZnMczzuzTsC/fSHDUM1FMraFfIpC8hsmwHTbBHm2DNPjGMVDUPlxxkN6VOO81Pkih63i+HBgWKakuz5yg7Zh1P+e/7/citqQ+f402tucL4apMOlT6vj4hp0oR2zAtWjLPdJ1oLHnHLnbsiJ5YDwKEv0H8VAjw0y6r7UJv6MEEJBUSvSYQBo7SC5iItDcOYtfK9K6Fhj6mKKlZvybM9bzwDBpYUjJpbVTwrVU9ln8Xx9EvFCYbkdDQGEQFIUqp5rWmFruxyhqq6L/pMzFcFwz4//ZBzt2/K6QS1dS3rad1yahZhzg/SUsG3posFbzunV0SlgzzkjCHLEzEpgIjrpF2xdzx2NCkb3YoJXY2K/8vYbkDhbWeNAv9/dXw+BCoBzfKJP0NrB/jeu+oueiJj8vVOCIKHg/+GVPbuhQdYiDy5JyJVUSFSIRn6HgXzCWS4hYiM98fiH4oTDOf/eVrrrR8pSIWRRdd8EVT7NyWRLPXZOygi+4rSpl0ipgpLFNfO41XPNnZnqP10WXJ0ny0DDkSbW0dpDlYq3Gj9IfOQtX51G+HATa9x8Wc1DYj1L41/wixqba4itZFaz85WGcG4DVU/qDL4YbeGKwRFoJ0+Pm6L5mZU++Cx6GRvUW76UuI/tGWs85mzekCWjlTVC2qXBGEP3lDB0J1//wnUPfN56xccVWpHD/5UfD8kDEIa74HCioAD1R+oUWjFP3KxGCo31kTOiRkX6aSLKzy6YrlfsSoaA8zoTe2slZZXpAK96uAXf9ZVvruODb0noLsecLFprN+AgSjTqpMLWBu/PDjkELHTZun4wV4hLDihBCd5xFNJw /RuPa5Iu JpDpVL7DHlAANNmjMtUZkbctJXiLDQv5U0y6NmR+sQq4L4Dl/XfgavqH/ZCLFrY9KTCPdgh9nsqGR/fs= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Roman Gushchin The percpu memory used by memcg's memory statistics is already accounted. For consistency, let's enable accounting for vmstats and lruvec stats as well. Signed-off-by: Roman Gushchin Signed-off-by: Shakeel Butt Reviewed-by: Yosry Ahmed Reviewed-by: T.J. Mercier --- mm/memcontrol.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 7126459ec56a..434cff91b65e 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -5545,8 +5545,8 @@ static int alloc_mem_cgroup_per_node_info(struct mem_cgroup *memcg, int node) if (!pn) return 1; - pn->lruvec_stats = kzalloc_node(sizeof(struct lruvec_stats), GFP_KERNEL, - node); + pn->lruvec_stats = kzalloc_node(sizeof(struct lruvec_stats), + GFP_KERNEL_ACCOUNT, node); if (!pn->lruvec_stats) goto fail; @@ -5617,7 +5617,8 @@ static struct mem_cgroup *mem_cgroup_alloc(struct mem_cgroup *parent) goto fail; } - memcg->vmstats = kzalloc(sizeof(struct memcg_vmstats), GFP_KERNEL); + memcg->vmstats = kzalloc(sizeof(struct memcg_vmstats), + GFP_KERNEL_ACCOUNT); if (!memcg->vmstats) goto fail; From patchwork Tue Apr 30 06:06:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shakeel Butt X-Patchwork-Id: 13648290 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 666ECC4345F for ; Tue, 30 Apr 2024 06:06:37 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EE4A56B0087; Tue, 30 Apr 2024 02:06:36 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E6E426B008C; Tue, 30 Apr 2024 02:06:36 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CE8FB6B009B; Tue, 30 Apr 2024 02:06:36 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id A40706B0087 for ; Tue, 30 Apr 2024 02:06:36 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 62E561C0628 for ; Tue, 30 Apr 2024 06:06:36 +0000 (UTC) X-FDA: 82065163992.03.BEFDB00 Received: from out-180.mta0.migadu.com (out-180.mta0.migadu.com [91.218.175.180]) by imf15.hostedemail.com (Postfix) with ESMTP id ACD99A0025 for ; Tue, 30 Apr 2024 06:06:34 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=eT6UOJAv; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf15.hostedemail.com: domain of shakeel.butt@linux.dev designates 91.218.175.180 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1714457194; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=46jIpUDCGTb729LeCv+9t97jXtyDL66VCoduqMx+CMM=; b=bSSUI3mREfCoUI/dIs2UmAjmRm2taNDFyW+WMsA9ioc7u74Da5FbKmXcU7/llEZlh7/Ya2 VoVVOVlq2du/yPf0sXDlEgU9PwxmUkt4qdCvhwVL7TBQiVq7coiMZ4plRg5wlYsk7LTTca vBEzKQPzQEDBjKuKX34KOc4aAYfmOAo= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1714457194; a=rsa-sha256; cv=none; b=TOp3Nl53fXBV+0zOzArZbzXUEjjLnZ0RlP00K47QENjOCShjzb7t7ytFWxY1do/WMm7bun qWQvaHFtrXhYQWhDm8Nw2hSZkUnuVQtfLr6UFLaFkZ+PD98SOmL47EC5EppGntekTZqM8e 3UWAi0rYX/0Qwb7UyiI/c4iYssFpZHE= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=eT6UOJAv; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf15.hostedemail.com: domain of shakeel.butt@linux.dev designates 91.218.175.180 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1714457193; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=46jIpUDCGTb729LeCv+9t97jXtyDL66VCoduqMx+CMM=; b=eT6UOJAv9lSCNtyA6T8DFFhJFSoH7G/OxSaLKTW2ca6EPrjCF4awZ5/G+GTTdGnfIt7dO/ gGZQHMBCXl1ffB5QTlzSqRSn06kHWxZkfxhUyvdzoXBxyPrBOndD3fBF9ArOJEv3kU3e6u 9yY3/ETZZnAG1RZYdU7lE+4P4vGERZI= From: Shakeel Butt To: Andrew Morton , Johannes Weiner , Michal Hocko , Roman Gushchin , Muchun Song , Yosry Ahmed , "T . J . Mercier" Cc: kernel-team@meta.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v3 4/8] memcg: reduce memory for the lruvec and memcg stats Date: Mon, 29 Apr 2024 23:06:08 -0700 Message-ID: <20240430060612.2171650-5-shakeel.butt@linux.dev> In-Reply-To: <20240430060612.2171650-1-shakeel.butt@linux.dev> References: <20240430060612.2171650-1-shakeel.butt@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Rspamd-Queue-Id: ACD99A0025 X-Rspam-User: X-Rspamd-Server: rspam03 X-Stat-Signature: 6rju4uw5ftbkofc8dmb1w8c4kdjqm14p X-HE-Tag: 1714457194-550447 X-HE-Meta: U2FsdGVkX190PttDoMizYMOZJpJ4dcMgUlTnNQ6ETQ/wN6755/j9lynFRltL6+PskS0W7Zy0W57aoVfMEBKv+3BgQAwELOw+9o9WmEYiMozfCQv0fBgMH7gIfYG7GKyusKBaXjwfEqGrVhDHroIoGdZhB+Rul2z5n4ER191cdyf5nPD5ryfdiEupNiTP9ab7/Gomzzls7AoZ51pjAGFQcEDPTlVAHoB4Q0GdbdJFFfcIQ+IgWrp1OKKBxvRdpXqx1avyIgzZGzqkCu4HJsD5AZQ+U9dfsoq9mEPJ9Q87Kl5uS0ZHgj73kJpyAvkrBsgsagGOO2YicgUCAR7v1SUT/+xHACFkexLj9Ctxo20sYlKL74IwLEFnBdewi2ec44OMzt/GmZbSzIw1DBC9i1ADMDl86MBSRK08G7LqcHInr1FZ+v88SMZB0khsLZMW8wBbJhrTabcQ2JwcL3cD5x2q5mDXqJZYQ8HASbYoO6ERjSa6vSVJjHumhOP6KVO3TKtvoYFkPRzK2ggHMhZpwIWA0DWSnnW9qvA6mqJfM6mnf0ZsxY/uUqCffKm4bPE8ZftAl21VxNJHIL94oXmPDypYdjxbwEdO8+QpWkDAFXxl/Ryh7C2i15bLto5dTRSa+ujh5r4I1IFaxsC/ySatMe5/6cIlAonbck6E/IOhJYqbUJoYYCQRsRTyFBKnbcn/eWUhV9Gp2U8SIiKW6+zbSjYQm1cYTyNzQpt+K5T2kbqmtiR+JHdSFbBYdA0OsHrp9iXN08d8yqfloEdfx/mqARcTYVmDZ98Y9mzvRJmSw7HwfVNHAnSezBXNpI421OkgtZJlsZajJg+SAE0rHIHccChp5Xif+T3AxLVYs5MiS/K4YQFlc9boAs34APTq17W0aE7g5g1f35WbHtiY9f9bdVBkE5ha5xijPknYIsrCX8gwFW59p+9e2y7u/jBsHnyoDqp0G2s9idlxKYEh6ZCYcQu 12m675mE 5bSx6+ixXnybFqUhc8wRd3+sz6+JaPFjw/OEVOB5wuunJDGINJcckUtiVNNWbJalz7AbTcpyDKq6h9Yg= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: At the moment, the amount of memory allocated for stats related structs in the mem_cgroup corresponds to the size of enum node_stat_item. However not all fields in enum node_stat_item has corresponding memcg stats. So, let's use indirection mechanism similar to the one used for memcg vmstats management. For a given x86_64 config, the size of stats with and without patch is: structs size in bytes w/o with struct lruvec_stats 1128 648 struct lruvec_stats_percpu 752 432 struct memcg_vmstats 1832 1352 struct memcg_vmstats_percpu 1280 960 The memory savings is further compounded by the fact that these structs are allocated for each cpu and for each node. To be precise, for each memcg the memory saved would be: Memory saved = ((21 * 3 * NR_NODES) + (21 * 2 * NR_NODS * NR_CPUS) + (21 * 3) + (21 * 2 * NR_CPUS)) * sizeof(long) Where 21 is the number of fields eliminated. Signed-off-by: Shakeel Butt --- Changes since v2: - N/A mm/memcontrol.c | 138 ++++++++++++++++++++++++++++++++++++++++-------- 1 file changed, 115 insertions(+), 23 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 434cff91b65e..f424c5b2ba9b 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -576,35 +576,105 @@ mem_cgroup_largest_soft_limit_node(struct mem_cgroup_tree_per_node *mctz) return mz; } +/* Subset of node_stat_item for memcg stats */ +static const unsigned int memcg_node_stat_items[] = { + NR_INACTIVE_ANON, + NR_ACTIVE_ANON, + NR_INACTIVE_FILE, + NR_ACTIVE_FILE, + NR_UNEVICTABLE, + NR_SLAB_RECLAIMABLE_B, + NR_SLAB_UNRECLAIMABLE_B, + WORKINGSET_REFAULT_ANON, + WORKINGSET_REFAULT_FILE, + WORKINGSET_ACTIVATE_ANON, + WORKINGSET_ACTIVATE_FILE, + WORKINGSET_RESTORE_ANON, + WORKINGSET_RESTORE_FILE, + WORKINGSET_NODERECLAIM, + NR_ANON_MAPPED, + NR_FILE_MAPPED, + NR_FILE_PAGES, + NR_FILE_DIRTY, + NR_WRITEBACK, + NR_SHMEM, + NR_SHMEM_THPS, + NR_FILE_THPS, + NR_ANON_THPS, + NR_KERNEL_STACK_KB, + NR_PAGETABLE, + NR_SECONDARY_PAGETABLE, +#ifdef CONFIG_SWAP + NR_SWAPCACHE, +#endif +}; + +static const unsigned int memcg_stat_items[] = { + MEMCG_SWAP, + MEMCG_SOCK, + MEMCG_PERCPU_B, + MEMCG_VMALLOC, + MEMCG_KMEM, + MEMCG_ZSWAP_B, + MEMCG_ZSWAPPED, +}; + +#define NR_MEMCG_NODE_STAT_ITEMS ARRAY_SIZE(memcg_node_stat_items) +#define NR_MEMCG_STATS (NR_MEMCG_NODE_STAT_ITEMS + ARRAY_SIZE(memcg_stat_items)) +static int8_t mem_cgroup_stats_index[MEMCG_NR_STAT] __read_mostly; + +static void init_memcg_stats(void) +{ + int8_t i, j = 0; + + /* Switch to short once this failure occurs. */ + BUILD_BUG_ON(NR_MEMCG_STATS >= 127 /* INT8_MAX */); + + for (i = 0; i < NR_MEMCG_NODE_STAT_ITEMS; ++i) + mem_cgroup_stats_index[memcg_node_stat_items[i]] = ++j; + + for (i = 0; i < ARRAY_SIZE(memcg_stat_items); ++i) + mem_cgroup_stats_index[memcg_stat_items[i]] = ++j; +} + +static inline int memcg_stats_index(int idx) +{ + return mem_cgroup_stats_index[idx] - 1; +} + struct lruvec_stats_percpu { /* Local (CPU and cgroup) state */ - long state[NR_VM_NODE_STAT_ITEMS]; + long state[NR_MEMCG_NODE_STAT_ITEMS]; /* Delta calculation for lockless upward propagation */ - long state_prev[NR_VM_NODE_STAT_ITEMS]; + long state_prev[NR_MEMCG_NODE_STAT_ITEMS]; }; struct lruvec_stats { /* Aggregated (CPU and subtree) state */ - long state[NR_VM_NODE_STAT_ITEMS]; + long state[NR_MEMCG_NODE_STAT_ITEMS]; /* Non-hierarchical (CPU aggregated) state */ - long state_local[NR_VM_NODE_STAT_ITEMS]; + long state_local[NR_MEMCG_NODE_STAT_ITEMS]; /* Pending child counts during tree propagation */ - long state_pending[NR_VM_NODE_STAT_ITEMS]; + long state_pending[NR_MEMCG_NODE_STAT_ITEMS]; }; unsigned long lruvec_page_state(struct lruvec *lruvec, enum node_stat_item idx) { struct mem_cgroup_per_node *pn; - long x; + long x = 0; + int i; if (mem_cgroup_disabled()) return node_page_state(lruvec_pgdat(lruvec), idx); - pn = container_of(lruvec, struct mem_cgroup_per_node, lruvec); - x = READ_ONCE(pn->lruvec_stats->state[idx]); + i = memcg_stats_index(idx); + if (i >= 0) { + pn = container_of(lruvec, struct mem_cgroup_per_node, lruvec); + x = READ_ONCE(pn->lruvec_stats->state[i]); + } #ifdef CONFIG_SMP if (x < 0) x = 0; @@ -617,12 +687,16 @@ unsigned long lruvec_page_state_local(struct lruvec *lruvec, { struct mem_cgroup_per_node *pn; long x = 0; + int i; if (mem_cgroup_disabled()) return node_page_state(lruvec_pgdat(lruvec), idx); - pn = container_of(lruvec, struct mem_cgroup_per_node, lruvec); - x = READ_ONCE(pn->lruvec_stats->state_local[idx]); + i = memcg_stats_index(idx); + if (i >= 0) { + pn = container_of(lruvec, struct mem_cgroup_per_node, lruvec); + x = READ_ONCE(pn->lruvec_stats->state_local[i]); + } #ifdef CONFIG_SMP if (x < 0) x = 0; @@ -689,11 +763,11 @@ struct memcg_vmstats_percpu { /* The above should fit a single cacheline for memcg_rstat_updated() */ /* Local (CPU and cgroup) page state & events */ - long state[MEMCG_NR_STAT]; + long state[NR_MEMCG_STATS]; unsigned long events[NR_MEMCG_EVENTS]; /* Delta calculation for lockless upward propagation */ - long state_prev[MEMCG_NR_STAT]; + long state_prev[NR_MEMCG_STATS]; unsigned long events_prev[NR_MEMCG_EVENTS]; /* Cgroup1: threshold notifications & softlimit tree updates */ @@ -703,15 +777,15 @@ struct memcg_vmstats_percpu { struct memcg_vmstats { /* Aggregated (CPU and subtree) page state & events */ - long state[MEMCG_NR_STAT]; + long state[NR_MEMCG_STATS]; unsigned long events[NR_MEMCG_EVENTS]; /* Non-hierarchical (CPU aggregated) page state & events */ - long state_local[MEMCG_NR_STAT]; + long state_local[NR_MEMCG_STATS]; unsigned long events_local[NR_MEMCG_EVENTS]; /* Pending child counts during tree propagation */ - long state_pending[MEMCG_NR_STAT]; + long state_pending[NR_MEMCG_STATS]; unsigned long events_pending[NR_MEMCG_EVENTS]; /* Stats updates since the last flush */ @@ -844,7 +918,13 @@ static void flush_memcg_stats_dwork(struct work_struct *w) unsigned long memcg_page_state(struct mem_cgroup *memcg, int idx) { - long x = READ_ONCE(memcg->vmstats->state[idx]); + long x; + int i = memcg_stats_index(idx); + + if (i < 0) + return 0; + + x = READ_ONCE(memcg->vmstats->state[i]); #ifdef CONFIG_SMP if (x < 0) x = 0; @@ -876,18 +956,25 @@ static int memcg_state_val_in_pages(int idx, int val) */ void __mod_memcg_state(struct mem_cgroup *memcg, int idx, int val) { - if (mem_cgroup_disabled()) + int i = memcg_stats_index(idx); + + if (mem_cgroup_disabled() || i < 0) return; - __this_cpu_add(memcg->vmstats_percpu->state[idx], val); + __this_cpu_add(memcg->vmstats_percpu->state[i], val); memcg_rstat_updated(memcg, memcg_state_val_in_pages(idx, val)); } /* idx can be of type enum memcg_stat_item or node_stat_item. */ static unsigned long memcg_page_state_local(struct mem_cgroup *memcg, int idx) { - long x = READ_ONCE(memcg->vmstats->state_local[idx]); + long x; + int i = memcg_stats_index(idx); + + if (i < 0) + return 0; + x = READ_ONCE(memcg->vmstats->state_local[i]); #ifdef CONFIG_SMP if (x < 0) x = 0; @@ -901,6 +988,10 @@ static void __mod_memcg_lruvec_state(struct lruvec *lruvec, { struct mem_cgroup_per_node *pn; struct mem_cgroup *memcg; + int i = memcg_stats_index(idx); + + if (i < 0) + return; pn = container_of(lruvec, struct mem_cgroup_per_node, lruvec); memcg = pn->memcg; @@ -930,10 +1021,10 @@ static void __mod_memcg_lruvec_state(struct lruvec *lruvec, } /* Update memcg */ - __this_cpu_add(memcg->vmstats_percpu->state[idx], val); + __this_cpu_add(memcg->vmstats_percpu->state[i], val); /* Update lruvec */ - __this_cpu_add(pn->lruvec_stats_percpu->state[idx], val); + __this_cpu_add(pn->lruvec_stats_percpu->state[i], val); memcg_rstat_updated(memcg, memcg_state_val_in_pages(idx, val)); memcg_stats_unlock(); @@ -5702,6 +5793,7 @@ mem_cgroup_css_alloc(struct cgroup_subsys_state *parent_css) page_counter_init(&memcg->kmem, &parent->kmem); page_counter_init(&memcg->tcpmem, &parent->tcpmem); } else { + init_memcg_stats(); init_memcg_events(); page_counter_init(&memcg->memory, NULL); page_counter_init(&memcg->swap, NULL); @@ -5873,7 +5965,7 @@ static void mem_cgroup_css_rstat_flush(struct cgroup_subsys_state *css, int cpu) statc = per_cpu_ptr(memcg->vmstats_percpu, cpu); - for (i = 0; i < MEMCG_NR_STAT; i++) { + for (i = 0; i < NR_MEMCG_STATS; i++) { /* * Collect the aggregated propagation counts of groups * below us. We're in a per-cpu loop here and this is @@ -5937,7 +6029,7 @@ static void mem_cgroup_css_rstat_flush(struct cgroup_subsys_state *css, int cpu) lstatc = per_cpu_ptr(pn->lruvec_stats_percpu, cpu); - for (i = 0; i < NR_VM_NODE_STAT_ITEMS; i++) { + for (i = 0; i < NR_MEMCG_NODE_STAT_ITEMS; i++) { delta = lstats->state_pending[i]; if (delta) lstats->state_pending[i] = 0; From patchwork Tue Apr 30 06:06:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shakeel Butt X-Patchwork-Id: 13648291 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B4170C4345F for ; Tue, 30 Apr 2024 06:06:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4D3666B009E; Tue, 30 Apr 2024 02:06:42 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 45C676B009F; Tue, 30 Apr 2024 02:06:42 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2AD166B00A0; Tue, 30 Apr 2024 02:06:42 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 0A1696B009E for ; Tue, 30 Apr 2024 02:06:42 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id AB3F3A0BF9 for ; Tue, 30 Apr 2024 06:06:41 +0000 (UTC) X-FDA: 82065164202.10.B7EF6A6 Received: from out-179.mta1.migadu.com (out-179.mta1.migadu.com [95.215.58.179]) by imf19.hostedemail.com (Postfix) with ESMTP id 1965A1A000F for ; Tue, 30 Apr 2024 06:06:38 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b="EfBe/UdG"; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf19.hostedemail.com: domain of shakeel.butt@linux.dev designates 95.215.58.179 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1714457199; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=BPVK+LGjMG4ohSIE1zDoj8EpOU3OQH41GwgZeY5ZUQE=; b=B9jV8KnEnLmA1DCzm+OIWBWy2Xg7Em+REMj1lSqeEUZOcrhRzsoC8geV1dlOnhKXlMTPAF W7GVxHdaUU6fFKOCnYzGjxIm26ODnG3EZFliueuDLBxZJV1Pjpdut/Xqu619BEk3Wy4UBh n6YacvNAA0nKPK2PPDzRteBiVab8/gE= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b="EfBe/UdG"; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf19.hostedemail.com: domain of shakeel.butt@linux.dev designates 95.215.58.179 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1714457199; a=rsa-sha256; cv=none; b=pkAeLGiV4vSpB0ZiOqt9d7297b5YVDUtuTQglJBnDi53VQWRCEoybwGaj2f2c5Tnt3fq9D Dh52wSaZC3qJB4oBHLLxhxseRvbNT0WTwy65Jaanc24fEj5CtU37CQKHPnJ0rwgLvSwLf4 YvKE1+7u2ma1uyhKuLgHqXEaYOfF52U= X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1714457197; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=BPVK+LGjMG4ohSIE1zDoj8EpOU3OQH41GwgZeY5ZUQE=; b=EfBe/UdGoVlt2JhWhQf0xF720ZRjNkbnEaL968MBYRXAEfDgAk6LqFh+Rm50VUpYIDY/cs lTYQ4CduLYDHFatXqJEXF81zWjZVFefdLKzR+Vz4EU5vqzA/O01NyIDzRvas+cBk/Dt79Q tgleknm1AKtzrboCI0KYvo7eigvehRc= From: Shakeel Butt To: Andrew Morton , Johannes Weiner , Michal Hocko , Roman Gushchin , Muchun Song , Yosry Ahmed , "T . J . Mercier" Cc: kernel-team@meta.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v3 5/8] memcg: cleanup __mod_memcg_lruvec_state Date: Mon, 29 Apr 2024 23:06:09 -0700 Message-ID: <20240430060612.2171650-6-shakeel.butt@linux.dev> In-Reply-To: <20240430060612.2171650-1-shakeel.butt@linux.dev> References: <20240430060612.2171650-1-shakeel.butt@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Rspam-User: X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 1965A1A000F X-Stat-Signature: ufi3gna4hefkfuxs6zx8at6o7ij64es8 X-HE-Tag: 1714457198-92178 X-HE-Meta: U2FsdGVkX1/yY+VlrnKX9TL/ZJCZ9Xm9y+KTwhKqQ7V/5ULKsMNnxhsyEBa1M/IfrsgZbdKQwNQYzZ6+m5JruqmjgLZd8HlB6c437fFlIbfcvuLHnHuZOK42r6JQ0x8y6XRrWRh6paLDZt1qNkn+r3P150nqSQH+WvBiV4+Sgq1JDQYzceVSeEf0ahIlnC4OtPlf3pkvFjNiRYxcCRqdlbVQS9RssrEWhdUjwP+qOaq4COMRI6xtsqizaqBreQ3dW5saL2FuX1vGBS2b9KqUbOsWVP2VRoIYSYzH9yDcW6hxTE/77rnWb0Bph/HOibq7p6CBDKgiwfAucVN/BPNqFfwXoI9Durnx0xl8N0nY52TXvKmGmWwLOlShurJE+MTtzTgGwmARBcSFJ+wPRlgO9zLA8vyNET+OAvjlSIIxjtCRj5A/Dc67OPD4a3KMoUqdmO0BLVCH5y44I3h5KU403xC+tDANga9Jm+Tx1izMCjAG4B204+Xm+x/Y4b8muhC/Jv1sOddyFMqTHddlBvaHNbOk6e79qnC4l+hLTUI3BqXEWbcWEpDYW8vgwo4Ip+QxbFWscc4pO7CcjDFBjqKB6UHgJWw6p21wax8DtB/sQTuc7SjWPqm6mGXkO8wPIwTXgWLspz13RVuhyRptRI7apCzANr14ezHhg2ObeAj3L02aFbYKHfAa3qZwXBk0by4x40GCc9N4ZkgqIvxOUQMqNL+SmiETA03aeluSJE3u2pKv6a5FNeXoL9gNWb0mTSlVQrKLbYfhKp26OBqi8VMJJ4Ygef3746R0ge2Ct6Rw9eSjZ6/CPDK9zV8brF9aWm6kWCZZlskZB5ohtn9MkpRFL7nRoKv4LODAhYdVQKGc2TDzJ8PyksAPK7TxJNXCAYRb1wbx67rBOcH4J75sDEX10lKbEQEXZXZNbXpLmF4B4CKxlpPRqcGJexR70LXjeEQ8PJtJ5h3gL3s7tWJ1A2W b9/zEuyS EaK6N8YLaOL4pLHYNMcbosLbKsH8nGqbH5qKB9Osx6XNulwdUxgUeD/hyBSREEfcKt57EdQK8yJxoti2JIQeGxvnT7iH4wVFdMWWhMoCmQuIURpLYr0PMQYrrc/caiheJKazITNDsxMpq7wMeec4BqmT1Cw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: There are no memcg specific stats for NR_SHMEM_PMDMAPPED and NR_FILE_PMDMAPPED. Let's remove them. Signed-off-by: Shakeel Butt Reviewed-by: Yosry Ahmed Reviewed-by: Roman Gushchin Reviewed-by: T.J. Mercier --- Changes since v2: - N/A mm/memcontrol.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index f424c5b2ba9b..df94abc0088f 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -1008,8 +1008,6 @@ static void __mod_memcg_lruvec_state(struct lruvec *lruvec, case NR_ANON_MAPPED: case NR_FILE_MAPPED: case NR_ANON_THPS: - case NR_SHMEM_PMDMAPPED: - case NR_FILE_PMDMAPPED: if (WARN_ON_ONCE(!in_task())) pr_warn("stat item index: %d\n", idx); break; From patchwork Tue Apr 30 06:06:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shakeel Butt X-Patchwork-Id: 13648292 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EA23CC25B10 for ; Tue, 30 Apr 2024 06:06:44 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5BDC46B009F; Tue, 30 Apr 2024 02:06:43 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5446D6B00A0; Tue, 30 Apr 2024 02:06:43 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 370326B00A1; Tue, 30 Apr 2024 02:06:43 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 14C1F6B009F for ; Tue, 30 Apr 2024 02:06:43 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id C7055802B8 for ; Tue, 30 Apr 2024 06:06:42 +0000 (UTC) X-FDA: 82065164244.02.F4C5BD6 Received: from out-182.mta0.migadu.com (out-182.mta0.migadu.com [91.218.175.182]) by imf28.hostedemail.com (Postfix) with ESMTP id 2095BC0028 for ; Tue, 30 Apr 2024 06:06:40 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=Zvjx05Vg; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf28.hostedemail.com: domain of shakeel.butt@linux.dev designates 91.218.175.182 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1714457201; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=3q1W7D1q9MXTrJoQi5E1RFDlpl05owU67WZ+nEUBEsw=; b=jEE2WytLx/I7r5M+FMwkjOVTU+kdhmF+vru6OnwLbvWkqZJPXHfosuO7N2aTj10DyBx3RK 6rE6ciBt2qC+9/AJKqqWMdQbabbwGlUTbKXBwEHwDU++hnbWVnTS1ytM7RUN0bxsWDffzx TNYoauAtlUvWo72FXGmroDqasTh7xJs= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=Zvjx05Vg; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf28.hostedemail.com: domain of shakeel.butt@linux.dev designates 91.218.175.182 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1714457201; a=rsa-sha256; cv=none; b=Ih9JpcGwLukqJVcOwNZeBTn1pBTQdK5Wvfkj+02JzlliYLVRWQfVwe+jye86Sy1qzKy0Kd nQe54rkwtpmvgwsT1e9tDwJTFW5zEoFaI2pe9LfAYv/X1xpR9PV1qSu+L7VJxWFieUW9RZ lXx1UpzOatCO+3K07CJ1Y8TF23Yrqwk= X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1714457199; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3q1W7D1q9MXTrJoQi5E1RFDlpl05owU67WZ+nEUBEsw=; b=Zvjx05VgmEQtihTxz/c69px6v2O5rBQ6CAXVV5XJDCN9QIGwtVdWfgW0aW1WLezr8zeDVJ zKAt36dlwhmlOfD/FK11SMqB6MnAnnB0oRd07xODCZ2IYBioYy8jixmJkk1+4QTpOQfCkT WFnehtYOPj0Jpa6WCliKDbOMN+qGLUo= From: Shakeel Butt To: Andrew Morton , Johannes Weiner , Michal Hocko , Roman Gushchin , Muchun Song , Yosry Ahmed , "T . J . Mercier" Cc: kernel-team@meta.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v3 6/8] mm: cleanup WORKINGSET_NODES in workingset Date: Mon, 29 Apr 2024 23:06:10 -0700 Message-ID: <20240430060612.2171650-7-shakeel.butt@linux.dev> In-Reply-To: <20240430060612.2171650-1-shakeel.butt@linux.dev> References: <20240430060612.2171650-1-shakeel.butt@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Stat-Signature: rc66dky8xha9piy13prjiidwhm1ipi8x X-Rspamd-Queue-Id: 2095BC0028 X-Rspam-User: X-Rspamd-Server: rspam12 X-HE-Tag: 1714457200-761931 X-HE-Meta: U2FsdGVkX18BFh0TdmtcS7HQFvyXBCjmkFqngobgMBpgdoZmBHANKc2EvSUAjdeZBgeK/Bm9v5YQcJuGYR8AEhU+++/AoFU5BrVeBKMmdO8G8n1YOAnhejgIapVsD5UF5bYxeMIbFP5ixOyHSLrYFsEMfPL9Ht39Rgu12aTlfleYnQTcULc4aiLWnRV4oriRMkWcC1rC2rNISsREoCemHJp6rqr3Pz2gQzOWT7jMITuIZiIpZIec4AFzMg8eZwOHw9+WutiiegXZ4FJOzMAp1rvUL+IZv/1HEeMXM3LGwQJVmCQnEV9WqJMuzAma1mnzfYOCAumBuz+BZ0jAW5P/buINxL6JYYgILfnmF4gHAjeBT4qb98J9niHobIAYJ9VqrErgQ5vFcfAXRllxjDqQ9iWwTSeDrObmNiHr66OGZcUemVJ4MkfkI1kui9fChLzyAQqMpNQutwKe98U94tB4J8UDcVsJYzcy39t0PESpQY1L+ZethBSIge0KtljuYMpX+csINvBE3A9buuxcnqoXyGuKcIS2KQZZI+1BbF1I0dF7mPsMT3tX3yUkJqr/xayWMC0dsj4h0ro1nJmKc3p+zsuWfv09sCoqp1XZOmuAaKD2/SEhjs1dNktEV89k+QnjKALf0egSVypGRI+88MdxHsEx9vE+u2B4Yb6AesfJvP+Epka7Vj0pVTTXjW3FnYiURVrOZyuU6mXFAWaDGsUH3OiE6bVg/AqdE/Q3eaew3rYhGJ6inP3I5AbSvJZXqljY37MBm1b9rCW8NkvS9X8iGwlmIToSetPTSKOyiKvb0BYCgrSYy4IU3+wituo+oy6I9XJRtHONkTg964adptsK7XyDNi+0CXMtShLDD8brecGb8UcNNuFmDwRndgsNlCJsDqD+1BdhHjwPgqqVwpz+HOnIJwE9914fni/xWY5E6CYK5aYjNfzElCgseHN75D1vBBOeglvsqgnZVjepATw +rPMxenL EQDZS6XwmQWin82+9ewy9SCFKfRI4uPOgEjn37LtIvS1XMkaVJVD7uU1iEYptDAaEy89rVm36n9y40+iQKSYerLMjwioVlhoCZlIOnTzrkZuileakcGhlnOKALVE/Nxop7daD X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: WORKINGSET_NODES is not exposed in the memcg stats and thus there is no need to use the memcg specific stat update functions for it. In future if we decide to expose WORKINGSET_NODES in the memcg stats, we can revert this patch. Signed-off-by: Shakeel Butt Reviewed-by: Roman Gushchin Reviewed-by: T.J. Mercier --- Changes since v2: - N/A mm/workingset.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/mm/workingset.c b/mm/workingset.c index f2a0ecaf708d..c22adb93622a 100644 --- a/mm/workingset.c +++ b/mm/workingset.c @@ -618,6 +618,7 @@ struct list_lru shadow_nodes; void workingset_update_node(struct xa_node *node) { struct address_space *mapping; + struct page *page = virt_to_page(node); /* * Track non-empty nodes that contain only shadow entries; @@ -633,12 +634,12 @@ void workingset_update_node(struct xa_node *node) if (node->count && node->count == node->nr_values) { if (list_empty(&node->private_list)) { list_lru_add_obj(&shadow_nodes, &node->private_list); - __inc_lruvec_kmem_state(node, WORKINGSET_NODES); + __inc_node_page_state(page, WORKINGSET_NODES); } } else { if (!list_empty(&node->private_list)) { list_lru_del_obj(&shadow_nodes, &node->private_list); - __dec_lruvec_kmem_state(node, WORKINGSET_NODES); + __dec_node_page_state(page, WORKINGSET_NODES); } } } @@ -742,7 +743,7 @@ static enum lru_status shadow_lru_isolate(struct list_head *item, } list_lru_isolate(lru, item); - __dec_lruvec_kmem_state(node, WORKINGSET_NODES); + __dec_node_page_state(virt_to_page(node), WORKINGSET_NODES); spin_unlock(lru_lock); From patchwork Tue Apr 30 06:06:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shakeel Butt X-Patchwork-Id: 13648293 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0B2A4C4345F for ; Tue, 30 Apr 2024 06:06:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 150896B00A3; Tue, 30 Apr 2024 02:06:46 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0E5646B00A2; Tue, 30 Apr 2024 02:06:46 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E6F816B00A3; Tue, 30 Apr 2024 02:06:45 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id C07136B00A1 for ; Tue, 30 Apr 2024 02:06:45 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 7CA5B14058E for ; Tue, 30 Apr 2024 06:06:45 +0000 (UTC) X-FDA: 82065164370.09.113A0DC Received: from out-173.mta0.migadu.com (out-173.mta0.migadu.com [91.218.175.173]) by imf22.hostedemail.com (Postfix) with ESMTP id D8AB8C0010 for ; Tue, 30 Apr 2024 06:06:43 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=i9Ajc1vV; spf=pass (imf22.hostedemail.com: domain of shakeel.butt@linux.dev designates 91.218.175.173 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev; dmarc=pass (policy=none) header.from=linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1714457204; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=/V685kde9L0ZctsmCACzkCSs3YoKScmeI639UOEVavY=; b=T3xDyNhapJmYn+pLBip02xppqqaYT2Yc0sGpe2/NdxxYrb5gdnUnpTO9kzOGZJ23R4ahhM tTFMEq/0F2GLd2tJSXcG3QKYRyBa8dMUdUbKSXcChurZ2k5Z/SmeaA88ceQZUdSEeBPooc ylYtbWuYWZacfa6jlrd+YA6DJLjK69I= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=i9Ajc1vV; spf=pass (imf22.hostedemail.com: domain of shakeel.butt@linux.dev designates 91.218.175.173 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev; dmarc=pass (policy=none) header.from=linux.dev ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1714457204; a=rsa-sha256; cv=none; b=DkcvfPRGd19RZGndf36lu/2hB4qVQ+0ZLSwYEQMf+DTajGIZfG7fdV/0XpwdbIlBz6S2t+ DM2FbWBP/ROcaSUhbLcckSD4cgqx7Qg9wJfj59YzRjksqEe2PEENXUZs4jou1tUuV60+Ia Ka/7ZOPHwK2flO/iZK2BWAQ2dUqd9ow= X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1714457202; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/V685kde9L0ZctsmCACzkCSs3YoKScmeI639UOEVavY=; b=i9Ajc1vVEAxx3DPEALD4PHoTBLkXVrvxG8ZT0+gtHZi8WqtrmLvE1uboXg7LzmLXkBmgAe mkiSrRVvWlwUHmiKEHwyPJOySqzpYOfr9raHJAytZqjOw0fL57/fyFm58R4EVIawHdQKE1 7rcnHurz3ckYjl7kQ8vgmKrnNFzRbXU= From: Shakeel Butt To: Andrew Morton , Johannes Weiner , Michal Hocko , Roman Gushchin , Muchun Song , Yosry Ahmed , "T . J . Mercier" Cc: kernel-team@meta.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v3 7/8] memcg: warn for unexpected events and stats Date: Mon, 29 Apr 2024 23:06:11 -0700 Message-ID: <20240430060612.2171650-8-shakeel.butt@linux.dev> In-Reply-To: <20240430060612.2171650-1-shakeel.butt@linux.dev> References: <20240430060612.2171650-1-shakeel.butt@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Stat-Signature: 9hswke3u455sqx4uysa1mj4gafkcfyag X-Rspam-User: X-Rspamd-Queue-Id: D8AB8C0010 X-Rspamd-Server: rspam05 X-HE-Tag: 1714457203-275623 X-HE-Meta: U2FsdGVkX197xhx3oTyfyYWi6AdUYvUPYwedoFqgtEXzyKRVp5naXKPb8XWph/Z+cOpRz2/PNNNfLHGnIiDbtpFZ7vjyy6r0NbYbQMPKwg1GdwHh8rAOLbOBVWyoM9w65ubL6K40mWwyLVehE1VxSsjKUoePJt/DcNDdv1ra9z6BC0CicYvxx2Kfop7xxdfUoEFliAvG4JFocipS2ZIibZcyO/20PB6FtCTqniU8urCj1KsFD2CDYSEVJHPsVjL2uTkizXBv/Va0YeMzyLJJIzN46ElpX4r5BvXU5CqqldsgjP7CNBv9jGS/VzJ+KiY1k5uFFhNzDCtRRDT7wQRoh4OHz10OjFE6aRcMjO/n5wKAm/bUN4LpamPg2t2Q2dXtK3EnUidVz9+ykIxB3A3P3ZhNa/ETDzq5rsE1jmEeGahRyYoqorvLOmyc7JdJhrcSli+EcorHVIl0Qp+PFnOgmxthS0a7I7/0XqLQzssiTo935DxNYB+wdqdlcYZmT5nshIq9CEZcx/0Bv/+me7h4UbNUY8MImNkJlMq1NbvBGdXRCtOOyft0mbU1ofPHvWGs1R3sz/jbDJFEPhP0NC9KjdNehKRCCAfw70m+sqfLZauDthmk3g+8pUmwgGmehHCORkorPu15a68w07nnQ8hUQllP7Rx446pVvrpCs9W5Wh6rS7iUJ2ZZoc86IX8ACuVyUHYDg1ciyftPLBXSITi35hI6QxqDaZRgoZbTJ41dVmiOP+1sZYFp3C+ZiE2TqH6OtPRmjTSRQNeNBEp8tjV4Tj4yoiBYf80xm10BpWbrFGrMrqFNdNuNuLaH5SUXBRRw007WRAjOEfwwJ1zPmqRrLQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: To reduce memory usage by the memcg events and stats, the kernel uses indirection table and only allocate stats and events which are being used by the memcg code. To make this more robust, let's add warnings where unexpected stats and events indexes are used. Signed-off-by: Shakeel Butt --- Changes since v2: - Based on feedback from Johannes, switched to WARN_ONCE() from pr_warn_once(). mm/memcontrol.c | 55 ++++++++++++++++++++++++++++--------------------- 1 file changed, 32 insertions(+), 23 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index df94abc0088f..72e36977a96e 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -664,17 +664,18 @@ struct lruvec_stats { unsigned long lruvec_page_state(struct lruvec *lruvec, enum node_stat_item idx) { struct mem_cgroup_per_node *pn; - long x = 0; + long x; int i; if (mem_cgroup_disabled()) return node_page_state(lruvec_pgdat(lruvec), idx); i = memcg_stats_index(idx); - if (i >= 0) { - pn = container_of(lruvec, struct mem_cgroup_per_node, lruvec); - x = READ_ONCE(pn->lruvec_stats->state[i]); - } + if (WARN_ONCE(i < 0, "%s: missing stat item %d\n", __func__, idx)) + return 0; + + pn = container_of(lruvec, struct mem_cgroup_per_node, lruvec); + x = READ_ONCE(pn->lruvec_stats->state[i]); #ifdef CONFIG_SMP if (x < 0) x = 0; @@ -686,17 +687,18 @@ unsigned long lruvec_page_state_local(struct lruvec *lruvec, enum node_stat_item idx) { struct mem_cgroup_per_node *pn; - long x = 0; + long x; int i; if (mem_cgroup_disabled()) return node_page_state(lruvec_pgdat(lruvec), idx); i = memcg_stats_index(idx); - if (i >= 0) { - pn = container_of(lruvec, struct mem_cgroup_per_node, lruvec); - x = READ_ONCE(pn->lruvec_stats->state_local[i]); - } + if (WARN_ONCE(i < 0, "%s: missing stat item %d\n", __func__, idx)) + return 0; + + pn = container_of(lruvec, struct mem_cgroup_per_node, lruvec); + x = READ_ONCE(pn->lruvec_stats->state_local[i]); #ifdef CONFIG_SMP if (x < 0) x = 0; @@ -921,7 +923,7 @@ unsigned long memcg_page_state(struct mem_cgroup *memcg, int idx) long x; int i = memcg_stats_index(idx); - if (i < 0) + if (WARN_ONCE(i < 0, "%s: missing stat item %d\n", __func__, idx)) return 0; x = READ_ONCE(memcg->vmstats->state[i]); @@ -958,7 +960,10 @@ void __mod_memcg_state(struct mem_cgroup *memcg, int idx, int val) { int i = memcg_stats_index(idx); - if (mem_cgroup_disabled() || i < 0) + if (mem_cgroup_disabled()) + return; + + if (WARN_ONCE(i < 0, "%s: missing stat item %d\n", __func__, idx)) return; __this_cpu_add(memcg->vmstats_percpu->state[i], val); @@ -971,7 +976,7 @@ static unsigned long memcg_page_state_local(struct mem_cgroup *memcg, int idx) long x; int i = memcg_stats_index(idx); - if (i < 0) + if (WARN_ONCE(i < 0, "%s: missing stat item %d\n", __func__, idx)) return 0; x = READ_ONCE(memcg->vmstats->state_local[i]); @@ -990,7 +995,7 @@ static void __mod_memcg_lruvec_state(struct lruvec *lruvec, struct mem_cgroup *memcg; int i = memcg_stats_index(idx); - if (i < 0) + if (WARN_ONCE(i < 0, "%s: missing stat item %d\n", __func__, idx)) return; pn = container_of(lruvec, struct mem_cgroup_per_node, lruvec); @@ -1104,34 +1109,38 @@ void __mod_lruvec_kmem_state(void *p, enum node_stat_item idx, int val) void __count_memcg_events(struct mem_cgroup *memcg, enum vm_event_item idx, unsigned long count) { - int index = memcg_events_index(idx); + int i = memcg_events_index(idx); - if (mem_cgroup_disabled() || index < 0) + if (mem_cgroup_disabled()) + return; + + if (WARN_ONCE(i < 0, "%s: missing stat item %d\n", __func__, idx)) return; memcg_stats_lock(); - __this_cpu_add(memcg->vmstats_percpu->events[index], count); + __this_cpu_add(memcg->vmstats_percpu->events[i], count); memcg_rstat_updated(memcg, count); memcg_stats_unlock(); } static unsigned long memcg_events(struct mem_cgroup *memcg, int event) { - int index = memcg_events_index(event); + int i = memcg_events_index(event); - if (index < 0) + if (WARN_ONCE(i < 0, "%s: missing stat item %d\n", __func__, event)) return 0; - return READ_ONCE(memcg->vmstats->events[index]); + + return READ_ONCE(memcg->vmstats->events[i]); } static unsigned long memcg_events_local(struct mem_cgroup *memcg, int event) { - int index = memcg_events_index(event); + int i = memcg_events_index(event); - if (index < 0) + if (WARN_ONCE(i < 0, "%s: missing stat item %d\n", __func__, event)) return 0; - return READ_ONCE(memcg->vmstats->events_local[index]); + return READ_ONCE(memcg->vmstats->events_local[i]); } static void mem_cgroup_charge_statistics(struct mem_cgroup *memcg, From patchwork Tue Apr 30 06:06:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shakeel Butt X-Patchwork-Id: 13648294 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 11FEAC4345F for ; Tue, 30 Apr 2024 06:06:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 919BD6B00A2; Tue, 30 Apr 2024 02:06:50 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8795A6B00A4; Tue, 30 Apr 2024 02:06:50 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6CB426B00A5; Tue, 30 Apr 2024 02:06:50 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 439FA6B00A2 for ; Tue, 30 Apr 2024 02:06:50 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 070CAA0F29 for ; Tue, 30 Apr 2024 06:06:50 +0000 (UTC) X-FDA: 82065164580.29.B6ED7A1 Received: from out-170.mta1.migadu.com (out-170.mta1.migadu.com [95.215.58.170]) by imf24.hostedemail.com (Postfix) with ESMTP id 5715618000D for ; Tue, 30 Apr 2024 06:06:48 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=rotHBScM; spf=pass (imf24.hostedemail.com: domain of shakeel.butt@linux.dev designates 95.215.58.170 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev; dmarc=pass (policy=none) header.from=linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1714457208; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=oQu7rbPfAQy6XdrYuqRfpS3viClFCHuzUf1lmpnIBnw=; b=eZBOBFuB6BVMIIZiFu/RpXa/O6vIelbF0J8prAuGQWtQo+gpc+bTbGBSdyJ3pawS8MFZgK Xb0ZyxnkkKLFeZeKkmkZQDp7Zk37G2GgNYTncEcpCiwIN1PW0PniS11tEdp/JmpjNIM2nr B5ofbUI1jhom8de+VrlWeWlnjFJKUdk= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=rotHBScM; spf=pass (imf24.hostedemail.com: domain of shakeel.butt@linux.dev designates 95.215.58.170 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev; dmarc=pass (policy=none) header.from=linux.dev ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1714457208; a=rsa-sha256; cv=none; b=y/2izXr1UwZgb/UpewhbWTy2ssRKtpZH/01HTI4yg2MO8u9/xDHtyjFVLHHkphJHJWFFLn CgszMfpBYEGQByW+/sp4P4drLzLeCEfwdEAlpX/MQH/w4Q82y6/eYg2ltsUZH42oqbKbzp 8jPmIDgMrkXS3BlXNcn6i+wsU60Y7j0= X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1714457207; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=oQu7rbPfAQy6XdrYuqRfpS3viClFCHuzUf1lmpnIBnw=; b=rotHBScM1JlySEyfeor2cpEW+V9EK+9UeCwpDcgSfKbo+f1dWWgvcQN7mi7VyWHyq1/fRf IzfXcU8ln7O4xZudZdXn5uue6q1VbcyhdHtiuhipO4lqg7CJmp+yB/63oyYllZKOXFUrxN tFEkBtSRbv0k1HFIJIKt5si+4wW9Xvk= From: Shakeel Butt To: Andrew Morton , Johannes Weiner , Michal Hocko , Roman Gushchin , Muchun Song , Yosry Ahmed , "T . J . Mercier" Cc: kernel-team@meta.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v3 8/8] memcg: use proper type for mod_memcg_state Date: Mon, 29 Apr 2024 23:06:12 -0700 Message-ID: <20240430060612.2171650-9-shakeel.butt@linux.dev> In-Reply-To: <20240430060612.2171650-1-shakeel.butt@linux.dev> References: <20240430060612.2171650-1-shakeel.butt@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Rspam-User: X-Rspamd-Queue-Id: 5715618000D X-Rspamd-Server: rspam06 X-Stat-Signature: r5y6j7hra3crz5ac8kyoebcs9wb8944c X-HE-Tag: 1714457208-791498 X-HE-Meta: U2FsdGVkX18zMIgeuQpykdK45iD+5axM5jhuYVOELjmNmpg6IoF94YdeAAg4hWyJ9w81sRadUot18HheHh3gyl44RIajqQ+L8r9m2xl12aMOn5Twm2XBJKtzmAkF/3FOnv1mqR0XmsO0W2vRL24U/bDDiQ0wgzy6E39dcD11NSnFbmLCHni2h0emIexgdYSixnByJ4i8QZ1iNQB0GAAu1ZZJe6DYMCGjThzg48PzvJogFFBMB6gmfcSi/skfrzGvKuxs7F/Bw9DiowV+0rizcpHxmmzltlLH76rJNNmk25Ax+8JQpuYQFtlBMKPqcD0dN5n/n1yO4poEGK0t/ah+ZbPwAQfxqqn/Nndf8dvbIku4rKoM+IKTTC/RPvbU+f3JaTQT8tWvoxJJSnI0Mhod+hp7AtOVSKJ4o5w4Msms+0CIaKYE8liA8VPl/iSq7+qvkt5SFGehoQCoeygNZPWdMfJFtmZORE6/CdHLCoewR2/35wDIW+oN+xXhilESJyilslm7Vu7/60OobYYZ2yAJfdlb9lrqD4cQt+YlCpvfrqR2MA3urTR1r7RaG+vXKmwXwIuXj04DZQK9EnWq2vGVvB4kRWJXWQ89w1d/13xGtXZR7esehXJiCTB2MPlmtVq5g6D6HoNZ98YdXTetouzRuPxFiQyXPOkaEFjU2yH47mDw3AxcX/mG59U4i5M/+9JCA/F2NP2JL6h14PvOCUkKD+FcX3zL/z0Ige2u622lEovF9Kwd0vlwWqJaaV2sCNUAITeBJArfCej46/MJIzaHEV4EMMl80FdlZDc4zoPNjuLXzS08Dp42jd26B6QSyueQVOE9efYg6mnE7JHW8Q881bliPT3OS7lJkOXk/wwXxXDjoEEBSd3yiICsJPbodUBAIpk2EAaemzdoGKiEqi6F73kXnoRZCClkDHGf6j5f1yBaXRLBQGxGeHfSx/Jlj9Bb1+apV0fNKfVqee8Whwd Sb6dHqKm Waz2HQR2sVfJfu11ipdcqMtffqyWyeDNz5VhexDNmVZo1bz2UG1r+PweKlqjF+MgG49bzB0O7fU4GpXeAv1FXhu7fVx1VGRaIHx/khKNII0ptpyfLgjpGQv9LAZOhDFgBz7cC X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The memcg stats update functions can take arbitrary integer but the only input which make sense is enum memcg_stat_item and we don't want these functions to be called with arbitrary integer, so replace the parameter type with enum memcg_stat_item and compiler will be able to warn if memcg stat update functions are called with incorrect index value. Signed-off-by: Shakeel Butt Reviewed-by: T.J. Mercier --- Change since v2: - Fixed whitespace issue based on TJ's suggestion. include/linux/memcontrol.h | 13 +++++++------ mm/memcontrol.c | 3 ++- 2 files changed, 9 insertions(+), 7 deletions(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index ab8a6e884375..030d34e9d117 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -974,7 +974,8 @@ void mem_cgroup_print_oom_group(struct mem_cgroup *memcg); void folio_memcg_lock(struct folio *folio); void folio_memcg_unlock(struct folio *folio); -void __mod_memcg_state(struct mem_cgroup *memcg, int idx, int val); +void __mod_memcg_state(struct mem_cgroup *memcg, enum memcg_stat_item idx, + int val); /* try to stablize folio_memcg() for all the pages in a memcg */ static inline bool mem_cgroup_trylock_pages(struct mem_cgroup *memcg) @@ -995,7 +996,7 @@ static inline void mem_cgroup_unlock_pages(void) /* idx can be of type enum memcg_stat_item or node_stat_item */ static inline void mod_memcg_state(struct mem_cgroup *memcg, - int idx, int val) + enum memcg_stat_item idx, int val) { unsigned long flags; @@ -1005,7 +1006,7 @@ static inline void mod_memcg_state(struct mem_cgroup *memcg, } static inline void mod_memcg_page_state(struct page *page, - int idx, int val) + enum memcg_stat_item idx, int val) { struct mem_cgroup *memcg; @@ -1491,19 +1492,19 @@ static inline void mem_cgroup_print_oom_group(struct mem_cgroup *memcg) } static inline void __mod_memcg_state(struct mem_cgroup *memcg, - int idx, + enum memcg_stat_item idx, int nr) { } static inline void mod_memcg_state(struct mem_cgroup *memcg, - int idx, + enum memcg_stat_item idx, int nr) { } static inline void mod_memcg_page_state(struct page *page, - int idx, int val) + enum memcg_stat_item idx, int val) { } diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 72e36977a96e..f5fc16b918ba 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -956,7 +956,8 @@ static int memcg_state_val_in_pages(int idx, int val) * @idx: the stat item - can be enum memcg_stat_item or enum node_stat_item * @val: delta to add to the counter, can be negative */ -void __mod_memcg_state(struct mem_cgroup *memcg, int idx, int val) +void __mod_memcg_state(struct mem_cgroup *memcg, enum memcg_stat_item idx, + int val) { int i = memcg_stats_index(idx);