From patchwork Thu Aug 15 05:04:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shakeel Butt X-Patchwork-Id: 13764397 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8F61AC3DA7F for ; Thu, 15 Aug 2024 05:05:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2B2F66B0088; Thu, 15 Aug 2024 01:05:26 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 23CA66B0089; Thu, 15 Aug 2024 01:05:26 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0DCE46B008A; Thu, 15 Aug 2024 01:05:26 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id DD0316B0088 for ; Thu, 15 Aug 2024 01:05:25 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 998F4A1284 for ; Thu, 15 Aug 2024 05:05:25 +0000 (UTC) X-FDA: 82453291410.25.324DA47 Received: from out-188.mta1.migadu.com (out-188.mta1.migadu.com [95.215.58.188]) by imf20.hostedemail.com (Postfix) with ESMTP id C1B731C000B for ; Thu, 15 Aug 2024 05:05:23 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=Y3kcxohf; spf=pass (imf20.hostedemail.com: domain of shakeel.butt@linux.dev designates 95.215.58.188 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev; dmarc=pass (policy=none) header.from=linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723698288; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ZSO/XXBnC6h5HUDwl//DSPS7ZBS0AJFyTcAs4FnSlKM=; b=v4qMa5EAy43Aa5aRA1KYvjunHiQVuKuWmDm7eENGbCegEOvUymwZFFsdvo2nlYEjFq9Oo7 EIiaZI4c676hKAchXiDkQ2MdEKW9dV9jIW2f52GHuDGDqCUYMFlcu5aV4NDPgqnG/Gio4y 2uKK2tpPllGzSlDEyKnn1ngSbIAtiXs= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=Y3kcxohf; spf=pass (imf20.hostedemail.com: domain of shakeel.butt@linux.dev designates 95.215.58.188 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev; dmarc=pass (policy=none) header.from=linux.dev ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723698288; a=rsa-sha256; cv=none; b=qxADS+HiaidMva9v6qAxFjcIFBL9hcfePq7KnRVC8h7Fn5PhZEIaPL9jqIXC75AkJnsx6/ 4fRuMjIZmYi5BuH+LKud6g/vdzymRGUCuuTGvHTvA+7dz5/M29psmxs3Nm8nrzgpb3gsSC ia4ue/LAeOcTja5dIdse7Hxm45wQzEg= X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1723698321; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ZSO/XXBnC6h5HUDwl//DSPS7ZBS0AJFyTcAs4FnSlKM=; b=Y3kcxohfENTdgij0QcKxQZsWg+v9DKuVZAQt+WIo6/kh+sJq5aMjVvqJzasgusRpF9OpZT gARH4N24V2jEu6GDgdAkIZdC6QiYtivkRDstetOViqplsW/rx1LIKf6Mil+1gj881wDMvS zRrXs9GfToHXTJpGDkYSVVyes7vMhXo= From: Shakeel Butt To: Andrew Morton Cc: Johannes Weiner , Michal Hocko , Roman Gushchin , Muchun Song , "T . J . Mercier" , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Meta kernel team , cgroups@vger.kernel.org Subject: [PATCH 1/7] memcg: move v1 only percpu stats in separate struct Date: Wed, 14 Aug 2024 22:04:47 -0700 Message-ID: <20240815050453.1298138-2-shakeel.butt@linux.dev> In-Reply-To: <20240815050453.1298138-1-shakeel.butt@linux.dev> References: <20240815050453.1298138-1-shakeel.butt@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Rspamd-Server: rspam03 X-Rspam-User: X-Rspamd-Queue-Id: C1B731C000B X-Stat-Signature: 634e4hsodjpsgqgz5yzkroouq7115j8s X-HE-Tag: 1723698323-289000 X-HE-Meta: U2FsdGVkX19ZW84/zza9mLsXi8G2o92IGG9/mx0+1rkYm6pz12mJEgbREEDhn8qd6cTmwhUSH4v+FDB1O7vZh/jPxWcigpn6ogdKk8+9bQwmh+kmjoyjmUfYEgEiJ4vFgzpRJBZWpFOYgig1pToa+Uo6ROQwelHL7Agymf5cbAwA85SlTC80XrRX98jd8Q1gQuS1sNYWbN8HR5MZ7D0QAIQueo67qCj9ejzM/BDnBUh+2rM8wHlXHoEjRv6fh36Vx2WyWSDEzTw8/Wno28AT0PX5ahg8Tzfju9f8Iat0ZKdlQIR+YgNvrs/5yLpV2f1USm/NZsxx2OujDm6/9m2Io2rC8kXrnAVEIJutx8LosKSAOx5LkMjDxIld1cPn1oyW40yACpJu8v7iKRb+EZZHmxOcNE9U1qSk7A96vixwgR86La7+H1tSdhpU5uE/XTO8oIvmZlkH8dexUseWQ/jj9CoZnmaCXhYG5BMiimy986vc715l8dDggkZSkWooz+5/p1Uxj/gS3dolqdODkpyC3xV4xCfNP7hkKhMIKG3RSDra9pdCkIiZs0Qgp1HQ+sandHvzctiC4sK4aP1q+nJCZ0a3mWxF8Z7M+0VwIKDMisWpNhCW1tSHFjge8zqzuwXT0WsMvySuzX8459y+Yw9M5p7QywZLNVwD84cHvN/8MqZM+UAPx6FR6IfkiKI1rhJTiE8lq/kqM2aNGcmpY2p0xJKkz8h45mpVzXA23nRMVlnj3fUb9ZglNnd2v76G3yrNzGTh+VuNhbbQ0F3S3SBTkIVW2bPABWEoz/2qyeAO7nqmFEkMZoVgKQGFkSVEFuonOZtzVB/xot5xJuk9oa/rO8/1ThVv8ouDca8x8Okht58+DXsBU9KwSqLigu1CsX2xiNX+8NudhocAdp4mkNrflQapkTU0e3BG/ArqcYcybNziSFMyF+w9c43McJaqK8ddBN1fbNa7AzB8jQLNqbU koFxRsov GbD5bZ+C62z9a6zjrTcUDugGubf9EZ+4jzyAiLWU0/3rMA9DUvNJVeQOLX6D8aSstji5vHNkUU1JKHHUx2kSUWnRaJp9wWD6jSAmjk/qtYWP+7js2YMaz3Zl4cQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: At the moment struct memcg_vmstats_percpu contains two v1 only fields which consumes memory even when CONFIG_MEMCG_V1 is not enabled. In addition there are v1 only functions accessing them and are in the main memcontrol source file and can not be moved to v1 only source file due to these fields. Let's move these fields into their own struct. Later patches will move the functions accessing them to v1 source file and only allocate these fields when CONFIG_MEMCG_V1 is enabled. Signed-off-by: Shakeel Butt Acked-by: Roman Gushchin --- include/linux/memcontrol.h | 2 ++ mm/memcontrol-v1.h | 19 +++++++++++++++++++ mm/memcontrol.c | 18 +++++++++--------- 3 files changed, 30 insertions(+), 9 deletions(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 90ecd2dbca06..e21a1541adeb 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -70,6 +70,7 @@ struct mem_cgroup_id { }; struct memcg_vmstats_percpu; +struct memcg1_events_percpu; struct memcg_vmstats; struct lruvec_stats_percpu; struct lruvec_stats; @@ -254,6 +255,7 @@ struct mem_cgroup { struct list_head objcg_list; struct memcg_vmstats_percpu __percpu *vmstats_percpu; + struct memcg1_events_percpu __percpu *events_percpu; #ifdef CONFIG_CGROUP_WRITEBACK struct list_head cgwb_list; diff --git a/mm/memcontrol-v1.h b/mm/memcontrol-v1.h index 56d7eaa98274..8feccecf8e2a 100644 --- a/mm/memcontrol-v1.h +++ b/mm/memcontrol-v1.h @@ -56,6 +56,12 @@ enum mem_cgroup_events_target { MEM_CGROUP_NTARGETS, }; +/* Cgroup1: threshold notifications & softlimit tree updates */ +struct memcg1_events_percpu { + unsigned long nr_page_events; + unsigned long targets[MEM_CGROUP_NTARGETS]; +}; + bool mem_cgroup_event_ratelimit(struct mem_cgroup *memcg, enum mem_cgroup_events_target target); unsigned long mem_cgroup_usage(struct mem_cgroup *memcg, bool swap); @@ -69,6 +75,19 @@ unsigned long memcg_page_state_output(struct mem_cgroup *memcg, int item); unsigned long memcg_page_state_local_output(struct mem_cgroup *memcg, int item); int memory_stat_show(struct seq_file *m, void *v); +static inline bool memcg1_alloc_events(struct mem_cgroup *memcg) +{ + memcg->events_percpu = alloc_percpu_gfp(struct memcg1_events_percpu, + GFP_KERNEL_ACCOUNT); + return !!memcg->events_percpu; +} + +static inline void memcg1_free_events(struct mem_cgroup *memcg) +{ + if (memcg->events_percpu) + free_percpu(memcg->events_percpu); +} + /* Cgroup v1-specific declarations */ #ifdef CONFIG_MEMCG_V1 void memcg1_memcg_init(struct mem_cgroup *memcg); diff --git a/mm/memcontrol.c b/mm/memcontrol.c index dacf4fec4541..66d60bb56f91 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -485,10 +485,6 @@ struct memcg_vmstats_percpu { /* Delta calculation for lockless upward propagation */ long state_prev[MEMCG_VMSTAT_SIZE]; unsigned long events_prev[NR_MEMCG_EVENTS]; - - /* Cgroup1: threshold notifications & softlimit tree updates */ - unsigned long nr_page_events; - unsigned long targets[MEM_CGROUP_NTARGETS]; } ____cacheline_aligned; struct memcg_vmstats { @@ -865,7 +861,7 @@ void mem_cgroup_charge_statistics(struct mem_cgroup *memcg, int nr_pages) nr_pages = -nr_pages; /* for event */ } - __this_cpu_add(memcg->vmstats_percpu->nr_page_events, nr_pages); + __this_cpu_add(memcg->events_percpu->nr_page_events, nr_pages); } bool mem_cgroup_event_ratelimit(struct mem_cgroup *memcg, @@ -873,8 +869,8 @@ bool mem_cgroup_event_ratelimit(struct mem_cgroup *memcg, { unsigned long val, next; - val = __this_cpu_read(memcg->vmstats_percpu->nr_page_events); - next = __this_cpu_read(memcg->vmstats_percpu->targets[target]); + val = __this_cpu_read(memcg->events_percpu->nr_page_events); + next = __this_cpu_read(memcg->events_percpu->targets[target]); /* from time_after() in jiffies.h */ if ((long)(next - val) < 0) { switch (target) { @@ -887,7 +883,7 @@ bool mem_cgroup_event_ratelimit(struct mem_cgroup *memcg, default: break; } - __this_cpu_write(memcg->vmstats_percpu->targets[target], next); + __this_cpu_write(memcg->events_percpu->targets[target], next); return true; } return false; @@ -3510,6 +3506,7 @@ static void __mem_cgroup_free(struct mem_cgroup *memcg) for_each_node(node) free_mem_cgroup_per_node_info(memcg, node); + memcg1_free_events(memcg); kfree(memcg->vmstats); free_percpu(memcg->vmstats_percpu); kfree(memcg); @@ -3549,6 +3546,9 @@ static struct mem_cgroup *mem_cgroup_alloc(struct mem_cgroup *parent) if (!memcg->vmstats_percpu) goto fail; + if (!memcg1_alloc_events(memcg)) + goto fail; + for_each_possible_cpu(cpu) { if (parent) pstatc = per_cpu_ptr(parent->vmstats_percpu, cpu); @@ -4664,7 +4664,7 @@ static void uncharge_batch(const struct uncharge_gather *ug) local_irq_save(flags); __count_memcg_events(ug->memcg, PGPGOUT, ug->pgpgout); - __this_cpu_add(ug->memcg->vmstats_percpu->nr_page_events, ug->nr_memory); + __this_cpu_add(ug->memcg->events_percpu->nr_page_events, ug->nr_memory); memcg1_check_events(ug->memcg, ug->nid); local_irq_restore(flags);