From patchwork Thu Aug 15 05:04:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shakeel Butt X-Patchwork-Id: 13764397 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8F61AC3DA7F for ; Thu, 15 Aug 2024 05:05:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2B2F66B0088; Thu, 15 Aug 2024 01:05:26 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 23CA66B0089; Thu, 15 Aug 2024 01:05:26 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0DCE46B008A; Thu, 15 Aug 2024 01:05:26 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id DD0316B0088 for ; Thu, 15 Aug 2024 01:05:25 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 998F4A1284 for ; Thu, 15 Aug 2024 05:05:25 +0000 (UTC) X-FDA: 82453291410.25.324DA47 Received: from out-188.mta1.migadu.com (out-188.mta1.migadu.com [95.215.58.188]) by imf20.hostedemail.com (Postfix) with ESMTP id C1B731C000B for ; Thu, 15 Aug 2024 05:05:23 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=Y3kcxohf; spf=pass (imf20.hostedemail.com: domain of shakeel.butt@linux.dev designates 95.215.58.188 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev; dmarc=pass (policy=none) header.from=linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723698288; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ZSO/XXBnC6h5HUDwl//DSPS7ZBS0AJFyTcAs4FnSlKM=; b=v4qMa5EAy43Aa5aRA1KYvjunHiQVuKuWmDm7eENGbCegEOvUymwZFFsdvo2nlYEjFq9Oo7 EIiaZI4c676hKAchXiDkQ2MdEKW9dV9jIW2f52GHuDGDqCUYMFlcu5aV4NDPgqnG/Gio4y 2uKK2tpPllGzSlDEyKnn1ngSbIAtiXs= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=Y3kcxohf; spf=pass (imf20.hostedemail.com: domain of shakeel.butt@linux.dev designates 95.215.58.188 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev; dmarc=pass (policy=none) header.from=linux.dev ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723698288; a=rsa-sha256; cv=none; b=qxADS+HiaidMva9v6qAxFjcIFBL9hcfePq7KnRVC8h7Fn5PhZEIaPL9jqIXC75AkJnsx6/ 4fRuMjIZmYi5BuH+LKud6g/vdzymRGUCuuTGvHTvA+7dz5/M29psmxs3Nm8nrzgpb3gsSC ia4ue/LAeOcTja5dIdse7Hxm45wQzEg= X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1723698321; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ZSO/XXBnC6h5HUDwl//DSPS7ZBS0AJFyTcAs4FnSlKM=; b=Y3kcxohfENTdgij0QcKxQZsWg+v9DKuVZAQt+WIo6/kh+sJq5aMjVvqJzasgusRpF9OpZT gARH4N24V2jEu6GDgdAkIZdC6QiYtivkRDstetOViqplsW/rx1LIKf6Mil+1gj881wDMvS zRrXs9GfToHXTJpGDkYSVVyes7vMhXo= From: Shakeel Butt To: Andrew Morton Cc: Johannes Weiner , Michal Hocko , Roman Gushchin , Muchun Song , "T . J . Mercier" , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Meta kernel team , cgroups@vger.kernel.org Subject: [PATCH 1/7] memcg: move v1 only percpu stats in separate struct Date: Wed, 14 Aug 2024 22:04:47 -0700 Message-ID: <20240815050453.1298138-2-shakeel.butt@linux.dev> In-Reply-To: <20240815050453.1298138-1-shakeel.butt@linux.dev> References: <20240815050453.1298138-1-shakeel.butt@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Rspamd-Server: rspam03 X-Rspam-User: X-Rspamd-Queue-Id: C1B731C000B X-Stat-Signature: 634e4hsodjpsgqgz5yzkroouq7115j8s X-HE-Tag: 1723698323-289000 X-HE-Meta: U2FsdGVkX19ZW84/zza9mLsXi8G2o92IGG9/mx0+1rkYm6pz12mJEgbREEDhn8qd6cTmwhUSH4v+FDB1O7vZh/jPxWcigpn6ogdKk8+9bQwmh+kmjoyjmUfYEgEiJ4vFgzpRJBZWpFOYgig1pToa+Uo6ROQwelHL7Agymf5cbAwA85SlTC80XrRX98jd8Q1gQuS1sNYWbN8HR5MZ7D0QAIQueo67qCj9ejzM/BDnBUh+2rM8wHlXHoEjRv6fh36Vx2WyWSDEzTw8/Wno28AT0PX5ahg8Tzfju9f8Iat0ZKdlQIR+YgNvrs/5yLpV2f1USm/NZsxx2OujDm6/9m2Io2rC8kXrnAVEIJutx8LosKSAOx5LkMjDxIld1cPn1oyW40yACpJu8v7iKRb+EZZHmxOcNE9U1qSk7A96vixwgR86La7+H1tSdhpU5uE/XTO8oIvmZlkH8dexUseWQ/jj9CoZnmaCXhYG5BMiimy986vc715l8dDggkZSkWooz+5/p1Uxj/gS3dolqdODkpyC3xV4xCfNP7hkKhMIKG3RSDra9pdCkIiZs0Qgp1HQ+sandHvzctiC4sK4aP1q+nJCZ0a3mWxF8Z7M+0VwIKDMisWpNhCW1tSHFjge8zqzuwXT0WsMvySuzX8459y+Yw9M5p7QywZLNVwD84cHvN/8MqZM+UAPx6FR6IfkiKI1rhJTiE8lq/kqM2aNGcmpY2p0xJKkz8h45mpVzXA23nRMVlnj3fUb9ZglNnd2v76G3yrNzGTh+VuNhbbQ0F3S3SBTkIVW2bPABWEoz/2qyeAO7nqmFEkMZoVgKQGFkSVEFuonOZtzVB/xot5xJuk9oa/rO8/1ThVv8ouDca8x8Okht58+DXsBU9KwSqLigu1CsX2xiNX+8NudhocAdp4mkNrflQapkTU0e3BG/ArqcYcybNziSFMyF+w9c43McJaqK8ddBN1fbNa7AzB8jQLNqbU koFxRsov GbD5bZ+C62z9a6zjrTcUDugGubf9EZ+4jzyAiLWU0/3rMA9DUvNJVeQOLX6D8aSstji5vHNkUU1JKHHUx2kSUWnRaJp9wWD6jSAmjk/qtYWP+7js2YMaz3Zl4cQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: At the moment struct memcg_vmstats_percpu contains two v1 only fields which consumes memory even when CONFIG_MEMCG_V1 is not enabled. In addition there are v1 only functions accessing them and are in the main memcontrol source file and can not be moved to v1 only source file due to these fields. Let's move these fields into their own struct. Later patches will move the functions accessing them to v1 source file and only allocate these fields when CONFIG_MEMCG_V1 is enabled. Signed-off-by: Shakeel Butt Acked-by: Roman Gushchin --- include/linux/memcontrol.h | 2 ++ mm/memcontrol-v1.h | 19 +++++++++++++++++++ mm/memcontrol.c | 18 +++++++++--------- 3 files changed, 30 insertions(+), 9 deletions(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 90ecd2dbca06..e21a1541adeb 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -70,6 +70,7 @@ struct mem_cgroup_id { }; struct memcg_vmstats_percpu; +struct memcg1_events_percpu; struct memcg_vmstats; struct lruvec_stats_percpu; struct lruvec_stats; @@ -254,6 +255,7 @@ struct mem_cgroup { struct list_head objcg_list; struct memcg_vmstats_percpu __percpu *vmstats_percpu; + struct memcg1_events_percpu __percpu *events_percpu; #ifdef CONFIG_CGROUP_WRITEBACK struct list_head cgwb_list; diff --git a/mm/memcontrol-v1.h b/mm/memcontrol-v1.h index 56d7eaa98274..8feccecf8e2a 100644 --- a/mm/memcontrol-v1.h +++ b/mm/memcontrol-v1.h @@ -56,6 +56,12 @@ enum mem_cgroup_events_target { MEM_CGROUP_NTARGETS, }; +/* Cgroup1: threshold notifications & softlimit tree updates */ +struct memcg1_events_percpu { + unsigned long nr_page_events; + unsigned long targets[MEM_CGROUP_NTARGETS]; +}; + bool mem_cgroup_event_ratelimit(struct mem_cgroup *memcg, enum mem_cgroup_events_target target); unsigned long mem_cgroup_usage(struct mem_cgroup *memcg, bool swap); @@ -69,6 +75,19 @@ unsigned long memcg_page_state_output(struct mem_cgroup *memcg, int item); unsigned long memcg_page_state_local_output(struct mem_cgroup *memcg, int item); int memory_stat_show(struct seq_file *m, void *v); +static inline bool memcg1_alloc_events(struct mem_cgroup *memcg) +{ + memcg->events_percpu = alloc_percpu_gfp(struct memcg1_events_percpu, + GFP_KERNEL_ACCOUNT); + return !!memcg->events_percpu; +} + +static inline void memcg1_free_events(struct mem_cgroup *memcg) +{ + if (memcg->events_percpu) + free_percpu(memcg->events_percpu); +} + /* Cgroup v1-specific declarations */ #ifdef CONFIG_MEMCG_V1 void memcg1_memcg_init(struct mem_cgroup *memcg); diff --git a/mm/memcontrol.c b/mm/memcontrol.c index dacf4fec4541..66d60bb56f91 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -485,10 +485,6 @@ struct memcg_vmstats_percpu { /* Delta calculation for lockless upward propagation */ long state_prev[MEMCG_VMSTAT_SIZE]; unsigned long events_prev[NR_MEMCG_EVENTS]; - - /* Cgroup1: threshold notifications & softlimit tree updates */ - unsigned long nr_page_events; - unsigned long targets[MEM_CGROUP_NTARGETS]; } ____cacheline_aligned; struct memcg_vmstats { @@ -865,7 +861,7 @@ void mem_cgroup_charge_statistics(struct mem_cgroup *memcg, int nr_pages) nr_pages = -nr_pages; /* for event */ } - __this_cpu_add(memcg->vmstats_percpu->nr_page_events, nr_pages); + __this_cpu_add(memcg->events_percpu->nr_page_events, nr_pages); } bool mem_cgroup_event_ratelimit(struct mem_cgroup *memcg, @@ -873,8 +869,8 @@ bool mem_cgroup_event_ratelimit(struct mem_cgroup *memcg, { unsigned long val, next; - val = __this_cpu_read(memcg->vmstats_percpu->nr_page_events); - next = __this_cpu_read(memcg->vmstats_percpu->targets[target]); + val = __this_cpu_read(memcg->events_percpu->nr_page_events); + next = __this_cpu_read(memcg->events_percpu->targets[target]); /* from time_after() in jiffies.h */ if ((long)(next - val) < 0) { switch (target) { @@ -887,7 +883,7 @@ bool mem_cgroup_event_ratelimit(struct mem_cgroup *memcg, default: break; } - __this_cpu_write(memcg->vmstats_percpu->targets[target], next); + __this_cpu_write(memcg->events_percpu->targets[target], next); return true; } return false; @@ -3510,6 +3506,7 @@ static void __mem_cgroup_free(struct mem_cgroup *memcg) for_each_node(node) free_mem_cgroup_per_node_info(memcg, node); + memcg1_free_events(memcg); kfree(memcg->vmstats); free_percpu(memcg->vmstats_percpu); kfree(memcg); @@ -3549,6 +3546,9 @@ static struct mem_cgroup *mem_cgroup_alloc(struct mem_cgroup *parent) if (!memcg->vmstats_percpu) goto fail; + if (!memcg1_alloc_events(memcg)) + goto fail; + for_each_possible_cpu(cpu) { if (parent) pstatc = per_cpu_ptr(parent->vmstats_percpu, cpu); @@ -4664,7 +4664,7 @@ static void uncharge_batch(const struct uncharge_gather *ug) local_irq_save(flags); __count_memcg_events(ug->memcg, PGPGOUT, ug->pgpgout); - __this_cpu_add(ug->memcg->vmstats_percpu->nr_page_events, ug->nr_memory); + __this_cpu_add(ug->memcg->events_percpu->nr_page_events, ug->nr_memory); memcg1_check_events(ug->memcg, ug->nid); local_irq_restore(flags); From patchwork Thu Aug 15 05:04:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shakeel Butt X-Patchwork-Id: 13764398 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CC4A3C52D7F for ; Thu, 15 Aug 2024 05:05:28 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6371C6B008A; Thu, 15 Aug 2024 01:05:28 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5BF6C6B008C; Thu, 15 Aug 2024 01:05:28 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4600A6B0092; Thu, 15 Aug 2024 01:05:28 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 221E66B008A for ; Thu, 15 Aug 2024 01:05:28 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id BBF9D120DBB for ; Thu, 15 Aug 2024 05:05:27 +0000 (UTC) X-FDA: 82453291494.11.1B085A1 Received: from out-187.mta1.migadu.com (out-187.mta1.migadu.com [95.215.58.187]) by imf29.hostedemail.com (Postfix) with ESMTP id 143BE12001F for ; Thu, 15 Aug 2024 05:05:25 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=X0EawjGD; spf=pass (imf29.hostedemail.com: domain of shakeel.butt@linux.dev designates 95.215.58.187 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev; dmarc=pass (policy=none) header.from=linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723698254; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=3kd/o0yNhsTl6XgkxdFjWyNP+hd5ZpFtgpy4OdScm7c=; b=3Ymq8AyEqMWke8pHWCUbV8puVk+7KfXiEGFwv94rvzWst1fgE29LtEtDDrbUDjZW6hScph XZk1l2c7fq3PdMMCqfB5nO5Q6eU0bvTWeMBm4YUOLg6mVQjpb9TpHRQGOErDL3Wcq2uZoY SPoA+x8TK+j3n/ml04ff0ymJDcnc0J8= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723698254; a=rsa-sha256; cv=none; b=3PPl6jct/syMFESriP/3oBXok1ZRs56qWQIvtxm549ahbub1KwiaKjgmO6WODSB5InsUeT z5tI7QWr3HS7cnmhlXs9M2ldKIg3hTc/qQjsmO+fKyJsRRG9f0euUAsVDO/wkf93cbXOSa MxICxEOrjduA+qZB+EytoyaLtQtXhM8= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=X0EawjGD; spf=pass (imf29.hostedemail.com: domain of shakeel.butt@linux.dev designates 95.215.58.187 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev; dmarc=pass (policy=none) header.from=linux.dev X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1723698324; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3kd/o0yNhsTl6XgkxdFjWyNP+hd5ZpFtgpy4OdScm7c=; b=X0EawjGDGBt0x9QlW3FR9g3CeR2jLvJN8b92rhYl0C8WSp8MKMgYF8pjXvM+UazgUhT92s 7sLOcrK9t6peoDGdE7HiOQtlxG83UkRbwLpUgeCDl9mkGc0cWuKmJtSd96jllCUoH5ISwe 0Vf/bgMX7pYM3B6MiFwezeZns1BD21c= From: Shakeel Butt To: Andrew Morton Cc: Johannes Weiner , Michal Hocko , Roman Gushchin , Muchun Song , "T . J . Mercier" , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Meta kernel team , cgroups@vger.kernel.org Subject: [PATCH 2/7] memcg: move mem_cgroup_event_ratelimit to v1 code Date: Wed, 14 Aug 2024 22:04:48 -0700 Message-ID: <20240815050453.1298138-3-shakeel.butt@linux.dev> In-Reply-To: <20240815050453.1298138-1-shakeel.butt@linux.dev> References: <20240815050453.1298138-1-shakeel.butt@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Rspamd-Queue-Id: 143BE12001F X-Stat-Signature: cku93w4h5e6o7uk3nfr5uhfnn7i8fc6u X-Rspamd-Server: rspam09 X-Rspam-User: X-HE-Tag: 1723698325-789250 X-HE-Meta: U2FsdGVkX18uO3avTzPfoTeD8j2Kyv938MEdanrljFxNJDww8ogBH1bXj2bNtPv8feTaHJMmq4aHMvXhc4oKhW2mD8XlrBtRi//HAO0Sk3SxUGhIot6MA8of13uNiBLrDp6Xo7ZBpvgHfQH5QkleulUOusz0DRlAmyUZjcG+XIDjH1HfA4mECDRVgijfJ68x+tgx0ALd0B3LGTXi9kg5hS1ifSEB1+CAdK/rsb8eJ+fd5pWQYvI94+PGY68ULi1a0nsM4eWCZBVhUktq8xJ8SPNq7Sft0n4/Gg4ciLKYsYJ/GGLaIeg5EaL9918eRu79/+R+kh8uxHLtGNlCdHAQxtJxV6zujtN3t9ZXpz2Qo7DiP6FJ9A2YWfXdvKCIOwxnwYH45ieEYsdK496uxwsJIOqR/2dyO16yMHiB4imipa8Hd27sCv86bKf9MA4c2wDFuxi38p359bJcWumXFWWgqlgCh0ufYgMz/3RjxDDwIG6YBuUrN98WnZXK6WxaAHRvTM+mHw0b8KCDnGCxw9xnFTkLU5EfSipF/8Vkbhh+6hFvXLDR+zw4AhnkEM6MIIXSS+gjUo9kwcqnmEBb/xll1RoWDB1gmnvJtu445OX0dL0iagWUnhbK5H3heBw4KhJd0/OQ1NeePyE9fIDEhzwej51RIAg+ugqRX5rsOl1SZEhBpIDN9Y94jehz+q8jeLTpQo/Acu1ihvOcsUjYT+3YvR/g3R3i1Qpos/8/wDhJ/4Nq2tf570zNBSwuSr9ga4tA7A4NyIMDH+EAT/B2c1mC/j+lvXGVICtn7qjByy8vTrtMmx8yuFHbaPSwrFgE68zYIi26/yC9SzDDngMe8pRKBhoDtfOdUf+GikNEU6XSRADjx1nqg8P381RzaE9kE0Xp/CQwEKNc290KXae7GYgdYSTobG3fX6IFKbDUaXXyUljRUuFv0u4MFBv2KlI+Btimrd2b4E3whszZQdLAMqT 4oShLgRy rgtfSas7JhOZnpVa3h+5xUK7lnLOh07sAg/zKWxXuJiU0DyG5JBzwtmxLozHpJgLEaZRn6J00d6+0JvVcIPGASd/sv9TLz+qg6G/mWZrCloicnDNNprbxkehTRr0jQhUfyMriTo0bDq0GReJSqO9kt2mHgg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: There are no callers of mem_cgroup_event_ratelimit() in the v2 code. Move it to v1 only code and rename it to memcg1_event_ratelimit(). Signed-off-by: Shakeel Butt --- mm/memcontrol-v1.c | 32 ++++++++++++++++++++++++++++++-- mm/memcontrol-v1.h | 2 -- mm/memcontrol.c | 28 ---------------------------- 3 files changed, 30 insertions(+), 32 deletions(-) diff --git a/mm/memcontrol-v1.c b/mm/memcontrol-v1.c index 52aecdae2c28..0ce1807ba468 100644 --- a/mm/memcontrol-v1.c +++ b/mm/memcontrol-v1.c @@ -1439,6 +1439,34 @@ static void mem_cgroup_threshold(struct mem_cgroup *memcg) } } +#define THRESHOLDS_EVENTS_TARGET 128 +#define SOFTLIMIT_EVENTS_TARGET 1024 + +static bool memcg1_event_ratelimit(struct mem_cgroup *memcg, + enum mem_cgroup_events_target target) +{ + unsigned long val, next; + + val = __this_cpu_read(memcg->events_percpu->nr_page_events); + next = __this_cpu_read(memcg->events_percpu->targets[target]); + /* from time_after() in jiffies.h */ + if ((long)(next - val) < 0) { + switch (target) { + case MEM_CGROUP_TARGET_THRESH: + next = val + THRESHOLDS_EVENTS_TARGET; + break; + case MEM_CGROUP_TARGET_SOFTLIMIT: + next = val + SOFTLIMIT_EVENTS_TARGET; + break; + default: + break; + } + __this_cpu_write(memcg->events_percpu->targets[target], next); + return true; + } + return false; +} + /* * Check events in order. * @@ -1449,11 +1477,11 @@ void memcg1_check_events(struct mem_cgroup *memcg, int nid) return; /* threshold event is triggered in finer grain than soft limit */ - if (unlikely(mem_cgroup_event_ratelimit(memcg, + if (unlikely(memcg1_event_ratelimit(memcg, MEM_CGROUP_TARGET_THRESH))) { bool do_softlimit; - do_softlimit = mem_cgroup_event_ratelimit(memcg, + do_softlimit = memcg1_event_ratelimit(memcg, MEM_CGROUP_TARGET_SOFTLIMIT); mem_cgroup_threshold(memcg); if (unlikely(do_softlimit)) diff --git a/mm/memcontrol-v1.h b/mm/memcontrol-v1.h index 8feccecf8e2a..fb7d439f19de 100644 --- a/mm/memcontrol-v1.h +++ b/mm/memcontrol-v1.h @@ -62,8 +62,6 @@ struct memcg1_events_percpu { unsigned long targets[MEM_CGROUP_NTARGETS]; }; -bool mem_cgroup_event_ratelimit(struct mem_cgroup *memcg, - enum mem_cgroup_events_target target); unsigned long mem_cgroup_usage(struct mem_cgroup *memcg, bool swap); void drain_all_stock(struct mem_cgroup *root_memcg); diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 66d60bb56f91..7ea511119567 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -95,9 +95,6 @@ static bool cgroup_memory_nobpf __ro_after_init; static DECLARE_WAIT_QUEUE_HEAD(memcg_cgwb_frn_waitq); #endif -#define THRESHOLDS_EVENTS_TARGET 128 -#define SOFTLIMIT_EVENTS_TARGET 1024 - static inline bool task_is_dying(void) { return tsk_is_oom_victim(current) || fatal_signal_pending(current) || @@ -864,31 +861,6 @@ void mem_cgroup_charge_statistics(struct mem_cgroup *memcg, int nr_pages) __this_cpu_add(memcg->events_percpu->nr_page_events, nr_pages); } -bool mem_cgroup_event_ratelimit(struct mem_cgroup *memcg, - enum mem_cgroup_events_target target) -{ - unsigned long val, next; - - val = __this_cpu_read(memcg->events_percpu->nr_page_events); - next = __this_cpu_read(memcg->events_percpu->targets[target]); - /* from time_after() in jiffies.h */ - if ((long)(next - val) < 0) { - switch (target) { - case MEM_CGROUP_TARGET_THRESH: - next = val + THRESHOLDS_EVENTS_TARGET; - break; - case MEM_CGROUP_TARGET_SOFTLIMIT: - next = val + SOFTLIMIT_EVENTS_TARGET; - break; - default: - break; - } - __this_cpu_write(memcg->events_percpu->targets[target], next); - return true; - } - return false; -} - struct mem_cgroup *mem_cgroup_from_task(struct task_struct *p) { /* From patchwork Thu Aug 15 05:04:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shakeel Butt X-Patchwork-Id: 13764399 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B3036C52D7F for ; Thu, 15 Aug 2024 05:05:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 43C116B008C; Thu, 15 Aug 2024 01:05:31 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3762F6B0092; Thu, 15 Aug 2024 01:05:31 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1F1996B0093; Thu, 15 Aug 2024 01:05:31 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id F0C5D6B008C for ; Thu, 15 Aug 2024 01:05:30 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id AC763120F81 for ; Thu, 15 Aug 2024 05:05:30 +0000 (UTC) X-FDA: 82453291620.19.6F3DCB9 Received: from out-175.mta0.migadu.com (out-175.mta0.migadu.com [91.218.175.175]) by imf23.hostedemail.com (Postfix) with ESMTP id DB205140003 for ; Thu, 15 Aug 2024 05:05:28 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=SSufFwWC; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf23.hostedemail.com: domain of shakeel.butt@linux.dev designates 91.218.175.175 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723698248; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=zygVeQZHpWPOjOtuooWei0ElZsFf2qNUmjbZYRpm8Xo=; b=5/Cyy8fsSjxt35pz8A00kbub4b1+oQ0C76B43x1u9d8fMYmmSZjkmxX/E+4VXaWkL3iAzk 49Wtng8vuWei9EbJdzl8PohW9lj0OWgaV9PimBut71OokAXO+goNAzwmM2aU9c598SvQxE Hr9UgzVhmLsws49+QbiICj4lna4xGfg= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723698248; a=rsa-sha256; cv=none; b=nWmn9Oex8hA8MZVvTAfghEuEI7x9Z/dRn/We7qLEGXQLTXw7j2V6jBiSGs+/RjpNXfq+zu PzmzGD5b7Rda5SvI2gB85ViSV4YUAJiQrbsUkDa4VC99ZViMWLNR/8veFUBLIY6gZMN0mY VWPCDgTw6Jbw3OumP9ZtoWvouYuP+cU= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=SSufFwWC; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf23.hostedemail.com: domain of shakeel.butt@linux.dev designates 91.218.175.175 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1723698327; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=zygVeQZHpWPOjOtuooWei0ElZsFf2qNUmjbZYRpm8Xo=; b=SSufFwWCrJB8eaH9sXXX9w33eVNxPwPO5BWCNuheXLzNVABd7WZoEprGHuGIUwlN5GrBBN bVriG0DCZ4Y2PnWVKHGPrswvS5dnKDx0K7ZRYh+l9vgp2IrJN7Zpuj595kZK5teC4UFSfz Cy3IRk3vBQKDfZD2kKNw68eArrfs2y8= From: Shakeel Butt To: Andrew Morton Cc: Johannes Weiner , Michal Hocko , Roman Gushchin , Muchun Song , "T . J . Mercier" , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Meta kernel team , cgroups@vger.kernel.org Subject: [PATCH 3/7] memcg: move mem_cgroup_charge_statistics to v1 code Date: Wed, 14 Aug 2024 22:04:49 -0700 Message-ID: <20240815050453.1298138-4-shakeel.butt@linux.dev> In-Reply-To: <20240815050453.1298138-1-shakeel.butt@linux.dev> References: <20240815050453.1298138-1-shakeel.butt@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: DB205140003 X-Stat-Signature: i9s8xck5nztyxwgf8mri31e51ddjzzq6 X-Rspam-User: X-HE-Tag: 1723698328-320292 X-HE-Meta: U2FsdGVkX19bdblgoAZCmmUukgBartzNhLRWSWGu5cPEYYb67W92HpE6pppeU6hotxfs8i5yXcNbxdY8RRC5liCOcA4UtSWYNT1qagXIGOxfoFtNJ5xHqYgejY0xg04nK3ntplogHceQtdIzDKskLUsTu62lQWmFtv9QqpLcMtBEIHqC+Ld2aieId8DIv+Lkp6tWaRhJ17INdWw3iJZyZnhNVxbmnAekM43W1jFJws+gknoBc8QPnu2stAe+sg7Nh+l6YKJrJgoo1E7tKz4ufUuKqckSrvJEad0g7CiS+OIksogYtUo0DZlJSnPU2lT5N3azkYDCItW3GQEhnkJGEln3tCy+XkGPcuTcifNYihb0XLsqKJU625GK4zMff/99PlnD37atyQzy0OiyCSdYPs4CB0YgABZ/ZTPjq0Zjoo3sw/DaMzMxxHIrdk3BX24jnNrdzdvhr6SwtTQkb0AaAmwYk6cOkhY2KEcjMDvp9cTOVvy9Ms1SeESFHOoDAVGSLhRGgQAeuJwxPSHMr2rB6r9uhQ6f0FArImJshC/QE0XDfCPsNG7znv1RTsv5HACThXTGesvPSHKqkh5VOgcyp5id7uDrEDHNb2u9DLLrInxyZfEVEcTI7pwdQDvg2c6kM4zn0tgy131AYLHP0jq2RMtTi5jMY8Uj3XrR45Zsq6JzwmJoV3Xr4zLzpHRiYCvMVfM7kZzJb+UF+qszpWd5PNQvQE+buFd69DCgb8cJwfbQDMgU2zZKR65At5xyFcRK7n+zCeDv1U0J8oWsGE+EhdnUKKOQccFqvfO0M4I84uwh9ZuY3WpwAOzMz3zd3yjXBXU/wPBT6uNZs13mk7/qtHUoza+shUq8QscK4bm+ZPgmyv9Ktzv3zQTTfC0ANiwzpXXqsJgtQpoBDimcDJqh4/giVDwU4sH9+GAC9jOrB27aNNdf8HRaOnNnGTkeE7RqIR0K4OiVpXksQTPnXnM 1OKjI43f +rrfa9TgsfcPkby5IG3ZLhzMCtTfWhjtHX8+Qho4nNiCR1u78uzh8gm7vfKHVo+egCSg5HPJFfL1arCJDxHFyKMkxtIDXNbK4iY4yTsfSdWp2GZELpC6gb5BBJ3W+qwHHx3moIyXYoQf2aTqXXTWAFZX5oA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: There are no callers of mem_cgroup_charge_statistics() in the v2 code base, so move it to the v1 only code and rename it to memcg1_charge_statistics(). Signed-off-by: Shakeel Butt --- mm/memcontrol-v1.c | 17 +++++++++++++++-- mm/memcontrol-v1.h | 3 ++- mm/memcontrol.c | 19 +++---------------- 3 files changed, 20 insertions(+), 19 deletions(-) diff --git a/mm/memcontrol-v1.c b/mm/memcontrol-v1.c index 0ce1807ba468..73587e6417c5 100644 --- a/mm/memcontrol-v1.c +++ b/mm/memcontrol-v1.c @@ -853,9 +853,9 @@ static int mem_cgroup_move_account(struct folio *folio, nid = folio_nid(folio); local_irq_disable(); - mem_cgroup_charge_statistics(to, nr_pages); + memcg1_charge_statistics(to, nr_pages); memcg1_check_events(to, nid); - mem_cgroup_charge_statistics(from, -nr_pages); + memcg1_charge_statistics(from, -nr_pages); memcg1_check_events(from, nid); local_irq_enable(); out: @@ -1439,6 +1439,19 @@ static void mem_cgroup_threshold(struct mem_cgroup *memcg) } } +void memcg1_charge_statistics(struct mem_cgroup *memcg, int nr_pages) +{ + /* pagein of a big page is an event. So, ignore page size */ + if (nr_pages > 0) + __count_memcg_events(memcg, PGPGIN, 1); + else { + __count_memcg_events(memcg, PGPGOUT, 1); + nr_pages = -nr_pages; /* for event */ + } + + __this_cpu_add(memcg->events_percpu->nr_page_events, nr_pages); +} + #define THRESHOLDS_EVENTS_TARGET 128 #define SOFTLIMIT_EVENTS_TARGET 1024 diff --git a/mm/memcontrol-v1.h b/mm/memcontrol-v1.h index fb7d439f19de..ef72d0b7c5c6 100644 --- a/mm/memcontrol-v1.h +++ b/mm/memcontrol-v1.h @@ -7,7 +7,6 @@ /* Cgroup v1 and v2 common declarations */ -void mem_cgroup_charge_statistics(struct mem_cgroup *memcg, int nr_pages); int try_charge_memcg(struct mem_cgroup *memcg, gfp_t gfp_mask, unsigned int nr_pages); @@ -116,6 +115,7 @@ bool memcg1_oom_prepare(struct mem_cgroup *memcg, bool *locked); void memcg1_oom_finish(struct mem_cgroup *memcg, bool locked); void memcg1_oom_recover(struct mem_cgroup *memcg); +void memcg1_charge_statistics(struct mem_cgroup *memcg, int nr_pages); void memcg1_check_events(struct mem_cgroup *memcg, int nid); void memcg1_stat_format(struct mem_cgroup *memcg, struct seq_buf *s); @@ -147,6 +147,7 @@ static inline bool memcg1_oom_prepare(struct mem_cgroup *memcg, bool *locked) { static inline void memcg1_oom_finish(struct mem_cgroup *memcg, bool locked) {} static inline void memcg1_oom_recover(struct mem_cgroup *memcg) {} +static inline void memcg1_charge_statistics(struct mem_cgroup *memcg, int nr_pages) {} static inline void memcg1_check_events(struct mem_cgroup *memcg, int nid) {} static inline void memcg1_stat_format(struct mem_cgroup *memcg, struct seq_buf *s) {} diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 7ea511119567..f8db9924d5dc 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -848,19 +848,6 @@ unsigned long memcg_events_local(struct mem_cgroup *memcg, int event) return READ_ONCE(memcg->vmstats->events_local[i]); } -void mem_cgroup_charge_statistics(struct mem_cgroup *memcg, int nr_pages) -{ - /* pagein of a big page is an event. So, ignore page size */ - if (nr_pages > 0) - __count_memcg_events(memcg, PGPGIN, 1); - else { - __count_memcg_events(memcg, PGPGOUT, 1); - nr_pages = -nr_pages; /* for event */ - } - - __this_cpu_add(memcg->events_percpu->nr_page_events, nr_pages); -} - struct mem_cgroup *mem_cgroup_from_task(struct task_struct *p) { /* @@ -2398,7 +2385,7 @@ void mem_cgroup_commit_charge(struct folio *folio, struct mem_cgroup *memcg) commit_charge(folio, memcg); local_irq_disable(); - mem_cgroup_charge_statistics(memcg, folio_nr_pages(folio)); + memcg1_charge_statistics(memcg, folio_nr_pages(folio)); memcg1_check_events(memcg, folio_nid(folio)); local_irq_enable(); } @@ -4775,7 +4762,7 @@ void mem_cgroup_replace_folio(struct folio *old, struct folio *new) commit_charge(new, memcg); local_irq_save(flags); - mem_cgroup_charge_statistics(memcg, nr_pages); + memcg1_charge_statistics(memcg, nr_pages); memcg1_check_events(memcg, folio_nid(new)); local_irq_restore(flags); } @@ -5020,7 +5007,7 @@ void mem_cgroup_swapout(struct folio *folio, swp_entry_t entry) * only synchronisation we have for updating the per-CPU variables. */ memcg_stats_lock(); - mem_cgroup_charge_statistics(memcg, -nr_entries); + memcg1_charge_statistics(memcg, -nr_entries); memcg_stats_unlock(); memcg1_check_events(memcg, folio_nid(folio)); From patchwork Thu Aug 15 05:04:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shakeel Butt X-Patchwork-Id: 13764400 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 75B75C52D7F for ; Thu, 15 Aug 2024 05:05:34 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 01D046B0096; Thu, 15 Aug 2024 01:05:34 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id EE85F6B0095; Thu, 15 Aug 2024 01:05:33 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D3C5E6B0096; Thu, 15 Aug 2024 01:05:33 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 9E7E06B0093 for ; Thu, 15 Aug 2024 01:05:33 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 5596D80E15 for ; Thu, 15 Aug 2024 05:05:33 +0000 (UTC) X-FDA: 82453291746.11.AB79167 Received: from out-176.mta1.migadu.com (out-176.mta1.migadu.com [95.215.58.176]) by imf20.hostedemail.com (Postfix) with ESMTP id 944B11C0009 for ; Thu, 15 Aug 2024 05:05:31 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=NYVUPZKR; spf=pass (imf20.hostedemail.com: domain of shakeel.butt@linux.dev designates 95.215.58.176 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev; dmarc=pass (policy=none) header.from=linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723698274; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=0zs8JI6aVxhuUibp1uzMfh5rP7bzFpoEcwMysAA/4Ek=; b=LOq8v6GZteFeSgo/cGM6AuZQOvuQ6ixl48tKdRcv7bI4iW5s2S7XKanRxqiKUvV125go9U QRDXC05Rx+xHc1vMwM1SKgAlkUpKi3Pz9vWXWfBYo+qDi93jL9b+34z5Gc3iF8PNccL0u0 fKz7LIxxqLjrTgkRh1/vl3Y1TOKv4Mg= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=NYVUPZKR; spf=pass (imf20.hostedemail.com: domain of shakeel.butt@linux.dev designates 95.215.58.176 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev; dmarc=pass (policy=none) header.from=linux.dev ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723698274; a=rsa-sha256; cv=none; b=q73kld2C3fs+bBCVHw5nFzDhA0qExhzgI17z/dST1WJ5biS19GgS2n4bDOIVTEJWSLyYak YwbtUMhSH9yLU2t9av0XNr05UQC2slowhFCyMg7lGvn/D50MWfQQBKJXmkuP17Cms5HCJI XG7tZ/GZAJ801enhe4euTsvZhimHclE= X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1723698330; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0zs8JI6aVxhuUibp1uzMfh5rP7bzFpoEcwMysAA/4Ek=; b=NYVUPZKRDyftYd4jWKz6eJxpMyBBS3VYXLMedsgvZjz0oPrIH3OYf1IEUOh5sCV80li/Yu lRk4xUdMr+RC9MPZFLr5K+eHQNx/j4b06zQD++GNIoNGumkjKOsymupdndD2XKl7eIJPZ/ fZZxpp/v/SLIuu1a9AMlglYO45I0+R4= From: Shakeel Butt To: Andrew Morton Cc: Johannes Weiner , Michal Hocko , Roman Gushchin , Muchun Song , "T . J . Mercier" , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Meta kernel team , cgroups@vger.kernel.org Subject: [PATCH 4/7] memcg: move v1 events and statistics code to v1 file Date: Wed, 14 Aug 2024 22:04:50 -0700 Message-ID: <20240815050453.1298138-5-shakeel.butt@linux.dev> In-Reply-To: <20240815050453.1298138-1-shakeel.butt@linux.dev> References: <20240815050453.1298138-1-shakeel.butt@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Stat-Signature: 6y3ynw75673ijzse9b7n7rwn8yzc64i8 X-Rspam-User: X-Rspamd-Queue-Id: 944B11C0009 X-Rspamd-Server: rspam02 X-HE-Tag: 1723698331-477053 X-HE-Meta: U2FsdGVkX19LE4ikmuuJm3RQ2M6WVqCCIgeVGMjYkENhIo3hvDAtz02tXugaQBWoNb5wE5EDNGe8UAFLY7yDhdvYalsaQaI2A1K6gBxKkTHCwsyik29XDo9Mz/VjV9sb1v0fp2Hp3J9x9Dqb+x/oxKwz+inMlcE6Mr67gZyvUHH2dG0JctGBaaXKxtqcENoV2JUqzH9TXUIUdrI42SSUgHtKSmJZBRktntTqVH4FrUnN7vO9hAr7ArBUHkdRHOQj5dP0r820cqaoWBmguH5f6QQe6KyAFerrm0p2DgdfXegdZorxKhXwtWpTURdS83iWRdjxphyKOjmOq42DgshWgIdG5HRsT5Gu/mPHXUdgY0dsujRyk7x8Q82Xe87SdCtXDG9kaB1fWZJRiBUsHGxeZH2XZAIj8X2/CNmgceKSttCMyzidQS1mHfTjYPfVrDus2rETsJn/XamgmF9nBJ8jf14NGV+3A5PGW6lusE/+YDynXWIY5K3ua5VKbrrcriHQ6t11lTbCIvBBScvezVJ+O+vEcw4A+f2qx9kwLLd00gOksjooGawTxDFTkpTenaG6c597Cpg+/q8yYIVvgrAZIGhEGQI345+qxDBRAWhLMcOXrYCCeBFzUOiWSasHrErfi8gwyYaLdmrskDKJPIlsxnuI4+AtVigfy96vuJQd6AnCotSXB5Gl6Rjo67uCjI1Bev3bXmOxssgBbnj97k6DpyTCYjkVmmMN5BhXjxW8NSlHr5vkekegtVu41s9FL9nzVzjgzfbmMv+EmVjYsOgXHnzXkei6LI5qP0+rm6ItbBvRYTewBOXp840Fs/JeUyxcRrAP7zy1L6NxSy5DgrGYSPi8mKnffLmDFdDCwZFqV2CG7JxbCSBUCMfQzHGeFhLiNyWY06ocylO4+9pcHS8cbk0jklgAIbJSf5OVCAl09IkC+yq3fEu6zcaCyHrbmd4BmdJ+ejzaKm3cG4I3Uyt bboTNPot leb8F6AGRxMMypfabCleP4keuH2uTuyYpBDu1BQo+iBQF53G6YHEnWXtl2XP4VxWnvkRcz1smpVSbK0t4P7Q9gI8BnrPXZTOx18LR+2qubhPVQ+t/z5Ab6Mge3Q== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Currently the common code path for charge commit, swapout and batched uncharge are executing v1 only code which is completely useless for the v2 deployments where CONFIG_MEMCG_V1 is disabled. In addition, it is mucking with IRQs which might be slow on some architectures. Let's move all of this code to v1 only code and remove them from v2 only deployments. Signed-off-by: Shakeel Butt --- mm/memcontrol-v1.c | 37 +++++++++++++++++++++++++++++++++++++ mm/memcontrol-v1.h | 14 ++++++++++++++ mm/memcontrol.c | 33 ++++----------------------------- 3 files changed, 55 insertions(+), 29 deletions(-) diff --git a/mm/memcontrol-v1.c b/mm/memcontrol-v1.c index 73587e6417c5..ffb7246b3f35 100644 --- a/mm/memcontrol-v1.c +++ b/mm/memcontrol-v1.c @@ -1502,6 +1502,43 @@ void memcg1_check_events(struct mem_cgroup *memcg, int nid) } } +void memcg1_commit_charge(struct folio *folio, struct mem_cgroup *memcg) +{ + unsigned long flags; + + local_irq_save(flags); + memcg1_charge_statistics(memcg, folio_nr_pages(folio)); + memcg1_check_events(memcg, folio_nid(folio)); + local_irq_restore(flags); +} + +void memcg1_swapout(struct folio *folio, struct mem_cgroup *memcg) +{ + /* + * Interrupts should be disabled here because the caller holds the + * i_pages lock which is taken with interrupts-off. It is + * important here to have the interrupts disabled because it is the + * only synchronisation we have for updating the per-CPU variables. + */ + preempt_disable_nested(); + VM_WARN_ON_IRQS_ENABLED(); + memcg1_charge_statistics(memcg, -folio_nr_pages(folio)); + preempt_enable_nested(); + memcg1_check_events(memcg, folio_nid(folio)); +} + +void memcg1_uncharge_batch(struct mem_cgroup *memcg, unsigned long pgpgout, + unsigned long nr_memory, int nid) +{ + unsigned long flags; + + local_irq_save(flags); + __count_memcg_events(memcg, PGPGOUT, pgpgout); + __this_cpu_add(memcg->events_percpu->nr_page_events, nr_memory); + memcg1_check_events(memcg, nid); + local_irq_restore(flags); +} + static int compare_thresholds(const void *a, const void *b) { const struct mem_cgroup_threshold *_a = a; diff --git a/mm/memcontrol-v1.h b/mm/memcontrol-v1.h index ef72d0b7c5c6..376d021a2bf4 100644 --- a/mm/memcontrol-v1.h +++ b/mm/memcontrol-v1.h @@ -118,6 +118,11 @@ void memcg1_oom_recover(struct mem_cgroup *memcg); void memcg1_charge_statistics(struct mem_cgroup *memcg, int nr_pages); void memcg1_check_events(struct mem_cgroup *memcg, int nid); +void memcg1_commit_charge(struct folio *folio, struct mem_cgroup *memcg); +void memcg1_swapout(struct folio *folio, struct mem_cgroup *memcg); +void memcg1_uncharge_batch(struct mem_cgroup *memcg, unsigned long pgpgout, + unsigned long nr_memory, int nid); + void memcg1_stat_format(struct mem_cgroup *memcg, struct seq_buf *s); void memcg1_account_kmem(struct mem_cgroup *memcg, int nr_pages); @@ -150,6 +155,15 @@ static inline void memcg1_oom_recover(struct mem_cgroup *memcg) {} static inline void memcg1_charge_statistics(struct mem_cgroup *memcg, int nr_pages) {} static inline void memcg1_check_events(struct mem_cgroup *memcg, int nid) {} +static inline void memcg1_commit_charge(struct folio *folio, + struct mem_cgroup *memcg) {} + +static inline void memcg1_swapout(struct folio *folio, struct mem_cgroup *memcg) {} + +static inline void memcg1_uncharge_batch(struct mem_cgroup *memcg, + unsigned long pgpgout, + unsigned long nr_memory, int nid) {} + static inline void memcg1_stat_format(struct mem_cgroup *memcg, struct seq_buf *s) {} static inline void memcg1_account_kmem(struct mem_cgroup *memcg, int nr_pages) {} diff --git a/mm/memcontrol.c b/mm/memcontrol.c index f8db9924d5dc..c4b06f26ccfd 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -2383,11 +2383,7 @@ void mem_cgroup_commit_charge(struct folio *folio, struct mem_cgroup *memcg) { css_get(&memcg->css); commit_charge(folio, memcg); - - local_irq_disable(); - memcg1_charge_statistics(memcg, folio_nr_pages(folio)); - memcg1_check_events(memcg, folio_nid(folio)); - local_irq_enable(); + memcg1_commit_charge(folio, memcg); } static inline void __mod_objcg_mlstate(struct obj_cgroup *objcg, @@ -4608,8 +4604,6 @@ static inline void uncharge_gather_clear(struct uncharge_gather *ug) static void uncharge_batch(const struct uncharge_gather *ug) { - unsigned long flags; - if (ug->nr_memory) { page_counter_uncharge(&ug->memcg->memory, ug->nr_memory); if (do_memsw_account()) @@ -4621,11 +4615,7 @@ static void uncharge_batch(const struct uncharge_gather *ug) memcg1_oom_recover(ug->memcg); } - local_irq_save(flags); - __count_memcg_events(ug->memcg, PGPGOUT, ug->pgpgout); - __this_cpu_add(ug->memcg->events_percpu->nr_page_events, ug->nr_memory); - memcg1_check_events(ug->memcg, ug->nid); - local_irq_restore(flags); + memcg1_uncharge_batch(ug->memcg, ug->pgpgout, ug->nr_memory, ug->nid); /* drop reference from uncharge_folio */ css_put(&ug->memcg->css); @@ -4732,7 +4722,6 @@ void mem_cgroup_replace_folio(struct folio *old, struct folio *new) { struct mem_cgroup *memcg; long nr_pages = folio_nr_pages(new); - unsigned long flags; VM_BUG_ON_FOLIO(!folio_test_locked(old), old); VM_BUG_ON_FOLIO(!folio_test_locked(new), new); @@ -4760,11 +4749,7 @@ void mem_cgroup_replace_folio(struct folio *old, struct folio *new) css_get(&memcg->css); commit_charge(new, memcg); - - local_irq_save(flags); - memcg1_charge_statistics(memcg, nr_pages); - memcg1_check_events(memcg, folio_nid(new)); - local_irq_restore(flags); + memcg1_commit_charge(new, memcg); } /** @@ -5000,17 +4985,7 @@ void mem_cgroup_swapout(struct folio *folio, swp_entry_t entry) page_counter_uncharge(&memcg->memsw, nr_entries); } - /* - * Interrupts should be disabled here because the caller holds the - * i_pages lock which is taken with interrupts-off. It is - * important here to have the interrupts disabled because it is the - * only synchronisation we have for updating the per-CPU variables. - */ - memcg_stats_lock(); - memcg1_charge_statistics(memcg, -nr_entries); - memcg_stats_unlock(); - memcg1_check_events(memcg, folio_nid(folio)); - + memcg1_swapout(folio, memcg); css_put(&memcg->css); } From patchwork Thu Aug 15 05:04:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shakeel Butt X-Patchwork-Id: 13764401 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A183DC52D7F for ; Thu, 15 Aug 2024 05:05:38 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 352936B0098; Thu, 15 Aug 2024 01:05:38 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2B1706B0099; Thu, 15 Aug 2024 01:05:38 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 153806B009A; Thu, 15 Aug 2024 01:05:38 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id E96E76B0098 for ; Thu, 15 Aug 2024 01:05:37 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 793391C4D3A for ; Thu, 15 Aug 2024 05:05:37 +0000 (UTC) X-FDA: 82453291914.19.07CF4BE Received: from out-183.mta0.migadu.com (out-183.mta0.migadu.com [91.218.175.183]) by imf02.hostedemail.com (Postfix) with ESMTP id 96FA280023 for ; Thu, 15 Aug 2024 05:05:35 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=OMht0cRZ; spf=pass (imf02.hostedemail.com: domain of shakeel.butt@linux.dev designates 91.218.175.183 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev; dmarc=pass (policy=none) header.from=linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723698278; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=oBl+yNWVDTrBvPczXHA6yxAB2QIqH+IAeny01+DfM7g=; b=t7AZkBz8/oJPBevwaeg+l7hZd4Zm6GFbve9bby/hGaP/MaKcwoiQEBJ22CAVtsG6H9f3+R xFSFGpnKamXGSnxxTMu9190UtjG5fSGC4j1Ba4UAAEoiGzAFg+w0k0VHiL01n1cMasnOXM FqLHJxQpp12G0/Z5nCa2PsB4iqLwxq8= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=OMht0cRZ; spf=pass (imf02.hostedemail.com: domain of shakeel.butt@linux.dev designates 91.218.175.183 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev; dmarc=pass (policy=none) header.from=linux.dev ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723698278; a=rsa-sha256; cv=none; b=x8pAHU+FSTmP0ibZnkm3Yis22BpRNVEjtc2P9zu63uyzp0Hi5J3XqtzfF0Jv5PG94nkG5R yZ3n9CpjbDnOAhz+QJOKU8xyR5fnlcxcKYoluMmkQ/CEXiMbjc0nRHpryubfGPPxRJuS7W MSw4V8oEclibHuGqOzy4Vq/psRZ6Syc= X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1723698334; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=oBl+yNWVDTrBvPczXHA6yxAB2QIqH+IAeny01+DfM7g=; b=OMht0cRZt5CcDvh+ZY+LOgDLFGiDcR3YmhPy5UbQyi0DzGomb5EhU8GsdkP452q20TOv1z t+sdwOObMDH8uPitMzDC/boZ97f+rQ5TZcmRIq9RbN3yTFNZ5QEIEpHnAt2hD1ZbzSVGBv 4U5zzA23HbLNZpDzNDuvZZfoaBgON4I= From: Shakeel Butt To: Andrew Morton Cc: Johannes Weiner , Michal Hocko , Roman Gushchin , Muchun Song , "T . J . Mercier" , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Meta kernel team , cgroups@vger.kernel.org Subject: [PATCH 5/7] memcg: make v1 only functions static Date: Wed, 14 Aug 2024 22:04:51 -0700 Message-ID: <20240815050453.1298138-6-shakeel.butt@linux.dev> In-Reply-To: <20240815050453.1298138-1-shakeel.butt@linux.dev> References: <20240815050453.1298138-1-shakeel.butt@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Stat-Signature: kn3d9gbyiirhae8rf77hhjdbke3npoxg X-Rspam-User: X-Rspamd-Queue-Id: 96FA280023 X-Rspamd-Server: rspam02 X-HE-Tag: 1723698335-437507 X-HE-Meta: U2FsdGVkX19CV45IWVwdPp7i41bwFNU1zC4D4stAOB4yX1b4WpZw/GYb7YDwTm4ckt24H3Hx8hIcd1cuhYsNGKH+sGASm3TW7ErBsTKY0DCBnV1HKtlFYWYLSe1WwBKN5KmXLbVFdt1T//L/92Z8VwNQjr39wR5A2ikMEkxNmPCMOF46lIjmJTDFDQmMdJwAnmxzJTpzol5Cc1uA2gnTrURxStGCplflqYwa1QdIXoQx0KD4PvB+yPxaVBZcKXmuL8X9qJZDbm31QvHrclG6XfzKaf5CmHlMYDwJ3sE6gad8iiN/vDsCFvSFhs8PA6/PpvZzhswqbY5b70CZWPiuzw3h0cK+nOKlKMxyhftJwm4XWTS2pOcmxtPMHD3WV1qRV4cz0xaTJ8b7MzTEULgPbmo3Ilxu3kLZp3gl5yyUZP6TrgCdPkmTtYOpQB0hNsXr3ZkQ1B253khMLcXEYBkymBMot1T6cu6JFJdMiz3BAQ306NeOXCS5qnge5AMN1VGrvAJSLcGyK0jvx51UfV+yNpuPkhr2QY3pPv3LehxUHpYX4brymF9AE6oDmPDtyIhf0PUZdeZ9t/tqOWUaqbjQZV7DcE3k+oTf9mJl1lzq0Uq8vFSkW7kDi6fdOzbIwFyV7h2ObK4qrcuxCQ5jr/jjngtoX9iw67QOKje4TzJaw5Qk0S+wRr0l+vCpcYV4dZUt2Oofw6DQ+yE//OsFtFg7eatXmd6+k3t2rKM/xnMRAafX+xTCpG/q7ZVhGHqPDyca0oZZRgV/0MxyJh14TSgx+HHwbm7IwxaEZ8FQad+ua+J7B4vLiRvKJV4aI3OyY4HfAUz7nnSIEj0sDNhBTaV5VRPba6VGXIIp91aqh9NVnncSGP8oomnSTxhPEa8FFFxI/54HI5uR2Ba36RXzEYM0kTKqBOBoodHmWq72CHPhd7BHzuTC97+9KbbdykO+MlEfJGgX/w1VOTas3Qb9MaR a6WpD3bK aYQ2s/Qh4iQeOO2Czj8CHZ3SSlQZawNtMy9fpgvnijk0QMm3wBDjfCsC1gTuqfFgyc9eN8q9tmJ4YTLSxeVjxKg8ENczViKK+1O1RjyvrTVkXffQy211GdGfy8w== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The functions memcg1_charge_statistics() and memcg1_check_events() are never used outside of v1 source file. So, make them static. Signed-off-by: Shakeel Butt --- mm/memcontrol-v1.c | 7 +++++-- mm/memcontrol-v1.h | 6 ------ 2 files changed, 5 insertions(+), 8 deletions(-) diff --git a/mm/memcontrol-v1.c b/mm/memcontrol-v1.c index ffb7246b3f35..0589d08c1599 100644 --- a/mm/memcontrol-v1.c +++ b/mm/memcontrol-v1.c @@ -742,6 +742,9 @@ static struct page *mc_handle_file_pte(struct vm_area_struct *vma, return folio_file_page(folio, index); } +static void memcg1_check_events(struct mem_cgroup *memcg, int nid); +static void memcg1_charge_statistics(struct mem_cgroup *memcg, int nr_pages); + /** * mem_cgroup_move_account - move account of the folio * @folio: The folio. @@ -1439,7 +1442,7 @@ static void mem_cgroup_threshold(struct mem_cgroup *memcg) } } -void memcg1_charge_statistics(struct mem_cgroup *memcg, int nr_pages) +static void memcg1_charge_statistics(struct mem_cgroup *memcg, int nr_pages) { /* pagein of a big page is an event. So, ignore page size */ if (nr_pages > 0) @@ -1484,7 +1487,7 @@ static bool memcg1_event_ratelimit(struct mem_cgroup *memcg, * Check events in order. * */ -void memcg1_check_events(struct mem_cgroup *memcg, int nid) +static void memcg1_check_events(struct mem_cgroup *memcg, int nid) { if (IS_ENABLED(CONFIG_PREEMPT_RT)) return; diff --git a/mm/memcontrol-v1.h b/mm/memcontrol-v1.h index 376d021a2bf4..0a9f3f9c2362 100644 --- a/mm/memcontrol-v1.h +++ b/mm/memcontrol-v1.h @@ -115,9 +115,6 @@ bool memcg1_oom_prepare(struct mem_cgroup *memcg, bool *locked); void memcg1_oom_finish(struct mem_cgroup *memcg, bool locked); void memcg1_oom_recover(struct mem_cgroup *memcg); -void memcg1_charge_statistics(struct mem_cgroup *memcg, int nr_pages); -void memcg1_check_events(struct mem_cgroup *memcg, int nid); - void memcg1_commit_charge(struct folio *folio, struct mem_cgroup *memcg); void memcg1_swapout(struct folio *folio, struct mem_cgroup *memcg); void memcg1_uncharge_batch(struct mem_cgroup *memcg, unsigned long pgpgout, @@ -152,9 +149,6 @@ static inline bool memcg1_oom_prepare(struct mem_cgroup *memcg, bool *locked) { static inline void memcg1_oom_finish(struct mem_cgroup *memcg, bool locked) {} static inline void memcg1_oom_recover(struct mem_cgroup *memcg) {} -static inline void memcg1_charge_statistics(struct mem_cgroup *memcg, int nr_pages) {} -static inline void memcg1_check_events(struct mem_cgroup *memcg, int nid) {} - static inline void memcg1_commit_charge(struct folio *folio, struct mem_cgroup *memcg) {} From patchwork Thu Aug 15 05:04:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shakeel Butt X-Patchwork-Id: 13764402 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9F28EC52D7F for ; Thu, 15 Aug 2024 05:05:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2B2F26B0099; Thu, 15 Aug 2024 01:05:41 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 23D336B009A; Thu, 15 Aug 2024 01:05:41 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0B71A6B009B; Thu, 15 Aug 2024 01:05:41 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id E13196B0099 for ; Thu, 15 Aug 2024 01:05:40 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 59EFDC1301 for ; Thu, 15 Aug 2024 05:05:40 +0000 (UTC) X-FDA: 82453292040.29.67028E4 Received: from out-184.mta0.migadu.com (out-184.mta0.migadu.com [91.218.175.184]) by imf28.hostedemail.com (Postfix) with ESMTP id AB33BC0014 for ; Thu, 15 Aug 2024 05:05:38 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=WP7X0cSU; spf=pass (imf28.hostedemail.com: domain of shakeel.butt@linux.dev designates 91.218.175.184 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev; dmarc=pass (policy=none) header.from=linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723698281; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=VYCZowvOZdc7ruHIIQXRY16w2zXEYkmh9LCBLV9ujoU=; b=jB35UxdaE+M5Nrx/O7E+KZnzLsJdPD8/g2PdgbNmcIcyfgULEYwkz62XQ8xYaWo1roZdOf p8+oQmU5ue9KGQO799auubbtMmrNnznIrLSnrkyL3BxdfRHJdtsTwTBTomMD4++pFaDvtb Cm8J6wdYRd6p7GRaUBfNJJ9zl8ySRQU= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=WP7X0cSU; spf=pass (imf28.hostedemail.com: domain of shakeel.butt@linux.dev designates 91.218.175.184 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev; dmarc=pass (policy=none) header.from=linux.dev ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723698281; a=rsa-sha256; cv=none; b=hdWlwnucskl6jhAaNTEVfntsRCh1+iSU50kTc6c/JlF1oe2yHAp467EJ2BbamSkhCoTRh0 tCLI+TKAtH3OfzQCYg4CgVUPw1KBvpHC5BQvYINxIjo7ekXAOcLunZmXHbCB1Zqp5eU+RE a5W31YwyNAm8BEDHmAcUu+FoIvr28KY= X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1723698337; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=VYCZowvOZdc7ruHIIQXRY16w2zXEYkmh9LCBLV9ujoU=; b=WP7X0cSUyHiZ4+RCUxJqB0xZEH6v5O2C7+QSCEYDQ2TRI05iHw9COqAeX0BwDbAN6DiHiA CjD2EZQ3q3DpSHkQKw1zthwY9fNApeaGXkV4jW0SMc4aMnVWGioC/q1zXwiZuRlA1DIV4k UpSi9xYB1LjAJK/yHmXtSeIAHYa2K1g= From: Shakeel Butt To: Andrew Morton Cc: Johannes Weiner , Michal Hocko , Roman Gushchin , Muchun Song , "T . J . Mercier" , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Meta kernel team , cgroups@vger.kernel.org Subject: [PATCH 6/7] memcg: allocate v1 event percpu only on v1 deployment Date: Wed, 14 Aug 2024 22:04:52 -0700 Message-ID: <20240815050453.1298138-7-shakeel.butt@linux.dev> In-Reply-To: <20240815050453.1298138-1-shakeel.butt@linux.dev> References: <20240815050453.1298138-1-shakeel.butt@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Stat-Signature: pddaibms4btbcae996zdmco9yse6ic6s X-Rspam-User: X-Rspamd-Queue-Id: AB33BC0014 X-Rspamd-Server: rspam02 X-HE-Tag: 1723698338-940107 X-HE-Meta: U2FsdGVkX1/RKKaqURJkhM71lPf00SJYiWPjU4PInhQTm48EohtJX5/coOnyy2Ukmo0RcPcacmkypTlCeLE3XPIBs8Q7oGj6ON1auZ9y0zi7qhKhgf9SzBanaCJhDnPxS+rAqnBP6fqZVXTv5mzsAKocZYdQSsV7YWQT1V4ntFlMXgf4mQ98jaoKPp/fKX7VoqbnSDmne25C3RzqtAJdoZhd4kMYMSSjSs7ahkB2+fOjudn1nSQ7KM9yCEq2FwVq99c9L/H3GXUnQ3sBbOd+I7unrwZc+qr6nUAvzQKnlkkokSqyVoiCFkVxERuS8Ou+nVRMT5dfedjOLgiFawK35OxZMqRnxtzzUQG/zTUYBvXcmL4qnTdaZULayF6ttTAvBVXG+SV3X31/ZK8HB5TvndfjnC2peLdhNIqvt/XO2Oygko2y9J6tnZfycoS9vHxXgJ7ZTkGTfsSKu0Ov5EpX8rh4RPg2K82H/9Qjul/wCKOoPgbiXn3kQVjBst+nOgOpLKWivTZBtVuFUihalnAi6j3LS3DDWy9e9UUb9jouGpFsXcTRwIbskskFBcJOmSx+yX/GOrDR9s4L4hajLymiDdgMc/CaisahkvBuymvpniEqdxF54YcYdJxBz1uEEKxcpyfk1HaaldOEDwbPSu5xQ8dVYY67Lmd6rbl6cpB6MOaBcGGmPdjep9MjNXBe7a1bS56wruzwj5z8lA1QVKX2fTRL+Y2fBcEOtf7K64YFYC4PjO6uEyc0riH5juRdO9WK30mFlBpRS6I6+lqYQTjnFZEugXgFk5Z6TimmR13jr0Ff6/zApFXUJZvtNmHs25Q2Km7UbPShy/8d/8eIMEv9ZA4lZwCPhIcgMpr5U3xY/PsEM/5qTnvCNeCmTPckfWSGD8SIwRrRkK+DtzYsZyxTIyfHGYMbqLiuvxKr+bCRvzGM4zX2TuCqPB8szTG1ulHfhB2eb+AeEJHZGUVBEtQ bgGCs3wu p99vfmueJ2yXsKCSkLV4ZhnM7MKsi+7e6f2rX2ubthL5bDuhdviBpCMcx+TJ5yO/dHDF+5bU5jGJwXV6Za6GJrcSLdb1QLvmYUj7n2qT23tgt8WdC3Sj+BzUl6Q== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Currently memcg->events_percpu gets allocated on v2 deployments. Let's move the allocation to v1 only codebase. This is not needed in v2. Signed-off-by: Shakeel Butt --- include/linux/memcontrol.h | 3 ++- mm/memcontrol-v1.c | 19 +++++++++++++++++++ mm/memcontrol-v1.h | 26 +++++++------------------- 3 files changed, 28 insertions(+), 20 deletions(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index e21a1541adeb..1f86d01d3b97 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -255,7 +255,6 @@ struct mem_cgroup { struct list_head objcg_list; struct memcg_vmstats_percpu __percpu *vmstats_percpu; - struct memcg1_events_percpu __percpu *events_percpu; #ifdef CONFIG_CGROUP_WRITEBACK struct list_head cgwb_list; @@ -277,6 +276,8 @@ struct mem_cgroup { struct page_counter kmem; /* v1 only */ struct page_counter tcpmem; /* v1 only */ + struct memcg1_events_percpu __percpu *events_percpu; + unsigned long soft_limit; /* protected by memcg_oom_lock */ diff --git a/mm/memcontrol-v1.c b/mm/memcontrol-v1.c index 0589d08c1599..81d8819f13cd 100644 --- a/mm/memcontrol-v1.c +++ b/mm/memcontrol-v1.c @@ -1442,6 +1442,12 @@ static void mem_cgroup_threshold(struct mem_cgroup *memcg) } } +/* Cgroup1: threshold notifications & softlimit tree updates */ +struct memcg1_events_percpu { + unsigned long nr_page_events; + unsigned long targets[MEM_CGROUP_NTARGETS]; +}; + static void memcg1_charge_statistics(struct mem_cgroup *memcg, int nr_pages) { /* pagein of a big page is an event. So, ignore page size */ @@ -3049,6 +3055,19 @@ bool memcg1_charge_skmem(struct mem_cgroup *memcg, unsigned int nr_pages, return false; } +bool memcg1_alloc_events(struct mem_cgroup *memcg) +{ + memcg->events_percpu = alloc_percpu_gfp(struct memcg1_events_percpu, + GFP_KERNEL_ACCOUNT); + return !!memcg->events_percpu; +} + +void memcg1_free_events(struct mem_cgroup *memcg) +{ + if (memcg->events_percpu) + free_percpu(memcg->events_percpu); +} + static int __init memcg1_init(void) { int node; diff --git a/mm/memcontrol-v1.h b/mm/memcontrol-v1.h index 0a9f3f9c2362..3bb8b3030e61 100644 --- a/mm/memcontrol-v1.h +++ b/mm/memcontrol-v1.h @@ -55,12 +55,6 @@ enum mem_cgroup_events_target { MEM_CGROUP_NTARGETS, }; -/* Cgroup1: threshold notifications & softlimit tree updates */ -struct memcg1_events_percpu { - unsigned long nr_page_events; - unsigned long targets[MEM_CGROUP_NTARGETS]; -}; - unsigned long mem_cgroup_usage(struct mem_cgroup *memcg, bool swap); void drain_all_stock(struct mem_cgroup *root_memcg); @@ -72,21 +66,12 @@ unsigned long memcg_page_state_output(struct mem_cgroup *memcg, int item); unsigned long memcg_page_state_local_output(struct mem_cgroup *memcg, int item); int memory_stat_show(struct seq_file *m, void *v); -static inline bool memcg1_alloc_events(struct mem_cgroup *memcg) -{ - memcg->events_percpu = alloc_percpu_gfp(struct memcg1_events_percpu, - GFP_KERNEL_ACCOUNT); - return !!memcg->events_percpu; -} - -static inline void memcg1_free_events(struct mem_cgroup *memcg) -{ - if (memcg->events_percpu) - free_percpu(memcg->events_percpu); -} - /* Cgroup v1-specific declarations */ #ifdef CONFIG_MEMCG_V1 + +bool memcg1_alloc_events(struct mem_cgroup *memcg); +void memcg1_free_events(struct mem_cgroup *memcg); + void memcg1_memcg_init(struct mem_cgroup *memcg); void memcg1_remove_from_trees(struct mem_cgroup *memcg); @@ -139,6 +124,9 @@ extern struct cftype mem_cgroup_legacy_files[]; #else /* CONFIG_MEMCG_V1 */ +static inline bool memcg1_alloc_events(struct mem_cgroup *memcg) { return true; } +static inline void memcg1_free_events(struct mem_cgroup *memcg) {} + static inline void memcg1_memcg_init(struct mem_cgroup *memcg) {} static inline void memcg1_remove_from_trees(struct mem_cgroup *memcg) {} static inline void memcg1_soft_limit_reset(struct mem_cgroup *memcg) {} From patchwork Thu Aug 15 05:04:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shakeel Butt X-Patchwork-Id: 13764403 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D840AC3DA7F for ; Thu, 15 Aug 2024 05:05:43 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 65F216B009B; Thu, 15 Aug 2024 01:05:43 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5E5B16B009C; Thu, 15 Aug 2024 01:05:43 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 43AFF6B009D; Thu, 15 Aug 2024 01:05:43 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 22BB76B009B for ; Thu, 15 Aug 2024 01:05:43 -0400 (EDT) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id D1DB9413AA for ; Thu, 15 Aug 2024 05:05:42 +0000 (UTC) X-FDA: 82453292124.28.851D630 Received: from out-181.mta0.migadu.com (out-181.mta0.migadu.com [91.218.175.181]) by imf06.hostedemail.com (Postfix) with ESMTP id 2F10718000A for ; Thu, 15 Aug 2024 05:05:40 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=olL28lu+; spf=pass (imf06.hostedemail.com: domain of shakeel.butt@linux.dev designates 91.218.175.181 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev; dmarc=pass (policy=none) header.from=linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723698284; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=4eAu+FADTIrdS5fQdJHfQzYaBndTq6CMzABmov4I1A8=; b=NKAKhBiMvt5CMRUXtwjEy4SadT1EVPA9MdFrxL1A+233zcwK+F6kOsGaFRfmsiJQw7Da0p zt9WMLOMcFXyOG5XX1rOBCzdVX+Bpmig1w9/DFU52KpoZJby1yvhH7MeJg2G+hUUxfJfYj B0VYjPv+5jXekVbJVFFr6jtK/EdjKyc= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=olL28lu+; spf=pass (imf06.hostedemail.com: domain of shakeel.butt@linux.dev designates 91.218.175.181 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev; dmarc=pass (policy=none) header.from=linux.dev ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723698284; a=rsa-sha256; cv=none; b=K+r1N4qRkRHRZ0K6f74C0Wju5IWbb7cAYjGbR7Tu23dXZeteYItfUHuggnls6ZkQr309wY rhPEHMaqAlTCjLAUWjzVpwk8iANmqoJi4+P6ZG3UkPhRwSEq3rDFaZN01u9Lq8in4zD30G r+0/+c564Kgf8YJWNkegdmL/bh7pENg= X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1723698339; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4eAu+FADTIrdS5fQdJHfQzYaBndTq6CMzABmov4I1A8=; b=olL28lu+B/zDWmkEuIbo69rLnNvVgmsmXU8CRoAVPnl5y2BaGzfXFuxPoLOVcbY0rq6D7C 8ngmcAcKiy7BJ07ngoQh7T+Um+zErT+SdESm4nRgRT4kPTIwD8f8Cr0ONEhBBEfoMRXMou nC2ywtg16+j82Juz7oq0kph1B4s92u0= From: Shakeel Butt To: Andrew Morton Cc: Johannes Weiner , Michal Hocko , Roman Gushchin , Muchun Song , "T . J . Mercier" , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Meta kernel team , cgroups@vger.kernel.org Subject: [PATCH 7/7] memcg: make PGPGIN and PGPGOUT v1 only Date: Wed, 14 Aug 2024 22:04:53 -0700 Message-ID: <20240815050453.1298138-8-shakeel.butt@linux.dev> In-Reply-To: <20240815050453.1298138-1-shakeel.butt@linux.dev> References: <20240815050453.1298138-1-shakeel.butt@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Stat-Signature: 3hnigrs4okbr91tcyh49gui59fo7q6xw X-Rspam-User: X-Rspamd-Queue-Id: 2F10718000A X-Rspamd-Server: rspam02 X-HE-Tag: 1723698340-716350 X-HE-Meta: U2FsdGVkX19UlpcP1g+VOaUvB/gsP9USfBFhrMQqC/utwrMTEKgoMMlSlwMn9nsSANSsrbeDfQUjoqRp2r4/IIPGHUc2HzNUviCTx9w2St5QBy4pPF2Pw77E7wUqwR89FT5TfAKhg5wsOPqvC2HYqG5aUjBgdljgj5kJK9b2fEyE2rAlPjK3SPBh+LUcScnScCy/ZdX71fXgZDuIHxLXtiiabxwLDKOMG/AIK1dr2SixF/XO7vs+U1k75AD3ag9HX/QcRrfusP02x4GzFE8kVSmsdOivj0hv05vowX5NQoifJMeGkgrR71iyTPa7DhHb/gzYpyHn0mNCtjNsMMNtKaBCDOdba+5mZ6Nrvsf+oeB0ehz5uI4MYM2+tEO7K7kpILyVDJ6CRYRq6DQGqAScdnKIENnCHje6qJ935tkObnLaCNVO1UYlQ6EYgWoulhH5m+PItuYDegse7HoNv1eHn1zNfugFstoKmyzMQbKqYtc/m+OPu7HdWU5okOOe4bWTJ8lFoSAbvPg6xqH0VL78JOEFYYlHJhN/UBJHh0Lom4pR94hex3BNX+GBAOhp46vvVMRAKkqqB91GIFeMejsaBRXg6Pj1ajgoSGJgv2tyt68bBCPto3R4/nf/LjVONy7GU015qnYeUCo/Kq05ws4qAYXzTbB70Gd0lUoa3S19hT0kU54hH71vf+NtZh+0quQ+GPh416dUwP2O78x69reaZG0EjPYjZTqujLGEyeqlPOU8E/e+rHtSxo32lHmqrNlAZSk9hXs6vHgt6f21qABKZ0NqbKq0an7so/nP6vmMq+sFLRiF4q0+n5+Dyr4BS62QsqNBROZsETDKhs2UXG1wwyvZBwJ9OOMoZqde4G4p0RLc6s7MzCk0USiLonYoK2FbA08QxDw6a/xAjhQF7CBZBZCGx70UvuQs+wSqsM5yQ+Jrp22CO24FnESF3W+2snsMg/vgOF/do/BwTToihnE asdQ+kJG ckEX+DSNZxH1DOImh8F7BHXLuVaRF/DMmSdmngMOXUr/H+rmSrGvJVu6U7spzOC6tdhZeIF5kqOtrVpj5MdWq6r9/t6w/LSjR7Jk3mxCKrnurOcLE7vf03oP4dg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Currently PGPGIN and PGPGOUT are used and exposed in the memcg v1 only code. So, let's put them under CONFIG_MEMCG_V1. Signed-off-by: Shakeel Butt --- mm/memcontrol.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index c4b06f26ccfd..9932074c617a 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -411,8 +411,10 @@ unsigned long lruvec_page_state_local(struct lruvec *lruvec, /* Subset of vm_event_item to report for memcg event stats */ static const unsigned int memcg_vm_event_stat[] = { +#ifdef CONFIG_MEMCG_V1 PGPGIN, PGPGOUT, +#endif PGSCAN_KSWAPD, PGSCAN_DIRECT, PGSCAN_KHUGEPAGED, @@ -1461,10 +1463,11 @@ static void memcg_stat_format(struct mem_cgroup *memcg, struct seq_buf *s) memcg_events(memcg, PGSTEAL_KHUGEPAGED)); for (i = 0; i < ARRAY_SIZE(memcg_vm_event_stat); i++) { +#ifdef CONFIG_MEMCG_V1 if (memcg_vm_event_stat[i] == PGPGIN || memcg_vm_event_stat[i] == PGPGOUT) continue; - +#endif seq_buf_printf(s, "%s %lu\n", vm_event_name(memcg_vm_event_stat[i]), memcg_events(memcg, memcg_vm_event_stat[i]));