From patchwork Fri Jan 3 01:50:12 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: JP Kobryn X-Patchwork-Id: 13925083 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D5276E77197 for ; Fri, 3 Jan 2025 01:50:36 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F1B7F6B0082; Thu, 2 Jan 2025 20:50:35 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id ECBA76B0083; Thu, 2 Jan 2025 20:50:35 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CF5876B0085; Thu, 2 Jan 2025 20:50:35 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id B479A6B0082 for ; Thu, 2 Jan 2025 20:50:35 -0500 (EST) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 565A6AEB6D for ; Fri, 3 Jan 2025 01:50:35 +0000 (UTC) X-FDA: 82964457660.20.08D9DAA Received: from mail-pl1-f169.google.com (mail-pl1-f169.google.com [209.85.214.169]) by imf30.hostedemail.com (Postfix) with ESMTP id 8FA6B80009 for ; Fri, 3 Jan 2025 01:48:55 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=FxjhjS5O; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf30.hostedemail.com: domain of inwardvessel@gmail.com designates 209.85.214.169 as permitted sender) smtp.mailfrom=inwardvessel@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1735868985; a=rsa-sha256; cv=none; b=x1fXdRMj9bZZ/2lCQ+Dp/eTJ6wXMzsI99c9VoTOrcRK9bf/BdA6Rftkzj387/uurtW0I/w FwvxSf6GpFaKkxBhehZ7DE5BmUTpvPj+21VepdG+3CvjyATs8je/btF5Aa36Yxi7KLHB1D MNOQzh74iq9iq27lHM5GknaPfUTEL/M= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=FxjhjS5O; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf30.hostedemail.com: domain of inwardvessel@gmail.com designates 209.85.214.169 as permitted sender) smtp.mailfrom=inwardvessel@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1735868985; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=04f8f3o/yaDsG08ligEdR1CTtfIwVHAWlueUkfYDoXo=; b=b8Sy+sdqsmtLi4nhtJnXIt47sTxoxgEXKJ45qXvmxRNE+9KSVVam5jPsXv/Ux1zabQL4Ml oYfOfJklZu6g7pVUi1bc++V3mZh2NqvQrmH6gqycYn1JbNNRqh9eijk36EiaJOK6pIZSQC Ca8/gYa18vT+/lCy1jFpfR+bu3rQj/M= Received: by mail-pl1-f169.google.com with SMTP id d9443c01a7336-2165cb60719so165630055ad.0 for ; Thu, 02 Jan 2025 17:50:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1735869032; x=1736473832; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=04f8f3o/yaDsG08ligEdR1CTtfIwVHAWlueUkfYDoXo=; b=FxjhjS5ONE97NMTJgwFIuIMYES5CpHrp4hqpfIRNjPi2sCAPZWEL8oGRsLVeYY9C/+ nia23ONyNzyCpIpU+RXGjGST3+jsO7K0h4bLC8nLuPGXn1rzInzb0TMXhDdJlDjI9y1w ZfqglP98XVe+2hZmUVcQXKTbWjTNQkkR6gaQD7awPPmkmbnOyAp2xxB/8+6dKhTu/chf uGXNJkqdr/YN72mM2g/OwCPVDEOOLt4a3JQdp+TIPuYOO/152sCiJIf48cPVwNxR0P3N G7h8splDNsOEVca8sPtV90caRBiXDwgg36qyC452guPsFUT3L8saRaZvnkcazWR0Ziv8 usGg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1735869032; x=1736473832; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=04f8f3o/yaDsG08ligEdR1CTtfIwVHAWlueUkfYDoXo=; b=v4V9PwkPI0LhGDY/rUUbTyp3HtCFpTULaqbNPcpbxmlatu88/vpgsL4bOvv6HO2uSF 7rlotFG4qyg27OkcFAuyQP036jVuU/SG31qebXepkch8NpmWQc9JFMBtIrALW/pvTuSV DwT+yMnPdKKike6qm9+9rZ/hf4qutOmb5ckFgsp0mpSycpzncBRiwM7oxDMLFrqFk2lP GOH9/XvNh2Cnxl5Q/WnQzVmvFOMbmbpCR4/kiG2TgYK1/dq6MZHut7KpiWEUThiNcL/o 0aIv68AM0G6xC9vK1E6aOMv/N+yfKCA2giFT+Gnxou8EIvrfw5sEilmmsoRUZK1bfm/j 0FAQ== X-Gm-Message-State: AOJu0YzgoHqu0+IjSE1S89xJlU9S2ItLBXbDKgKimpmRbS67r1cmpW+x Vjensmzdqp9xyXyyiphARVx8LKJSx8eyL9xUqpJpXFir9MahJb2Q X-Gm-Gg: ASbGncvUGSJgIjxcyJmEuRU4jZMfUUo08pY9pIMj12HqcfZ7QAX9iZFbIYBI9YkfQI7 BbvibPoWET31fHOVW7ThQGlZuj/y/zmJK6h1koqyvzt3yzIUb3otuBexGuVaSQo8aBygFGEll8B l5XsAIhtIBYElXI6qhFTJ4iqCPk4d9AQTQOvxmEtEL1MbwP4e0LSIQsnnf++fQA5bMxhhBK+dkH fu2VwEW/z6YtupgWuHEvMS7xnQavRJQXRCsAe7eQwpGbLPiFUaK581JcEtBLdO1vUn5gBS9cFHB rI+ufCN15CCwGE4zsg== X-Google-Smtp-Source: AGHT+IGDhdL6/CNaSo3pGtV64+fqNxC6hz2SLueRvx2WGBRXmMrSjYEpM/Emx32nG9JYyUBV8OcXLA== X-Received: by 2002:a17:903:234f:b0:216:4b6f:ddda with SMTP id d9443c01a7336-219e6f0e6c3mr678754295ad.35.1735869032286; Thu, 02 Jan 2025 17:50:32 -0800 (PST) Received: from saturn.. (c-67-188-127-15.hsd1.ca.comcast.net. [67.188.127.15]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-219dca04ce7sm228851505ad.283.2025.01.02.17.50.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 02 Jan 2025 17:50:31 -0800 (PST) From: JP Kobryn To: shakeel.butt@linux.dev, tj@kernel.org, mhocko@kernel.org, hannes@cmpxchg.org, yosryahmed@google.com, akpm@linux-foundation.org Cc: linux-mm@kvack.org, cgroups@vger.kernel.org Subject: [RFC PATCH 1/9 v2] cgroup: change cgroup to css in rstat updated and flush api Date: Thu, 2 Jan 2025 17:50:12 -0800 Message-ID: <20250103015020.78547-2-inwardvessel@gmail.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250103015020.78547-1-inwardvessel@gmail.com> References: <20250103015020.78547-1-inwardvessel@gmail.com> MIME-Version: 1.0 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 8FA6B80009 X-Stat-Signature: mhjw77nz7nj5j4thuqf8e1xx43pupjqp X-Rspam-User: X-HE-Tag: 1735868935-980412 X-HE-Meta: U2FsdGVkX18/2ud/62aIy37O1ciV4QqVIONc3xmgD4MT0D7Q4NN6AI9Yvc7FaNpbgIi+njbOGZiduzC/vIbehKxrjwno+N1pjNYSV9KgLeTdgidgHLam1TuvExAx77C+7HcdoVK9PocYR+ZTH1RJ8GR63BZsRSMEbYRqLXEqgHnlBtO4mXteMR6ztVKTzZX8il8efV19fKqjONyIxnO7H3db/pDJauZLIWPJaojWlisabQsbnkiaWDv373Q/qrsB3q+BFxpLkiOEcr90j4IOaC0aPEtsnlvrO97BxmfSyGSitqePJNez6SOYzUNv0GUUyywostlG6TcHMSVGynV1gMXmNyto3A4ZiOc0G0paoa9JN+/l/vM/X5DVjGG8wuvj6yAa1kX6NVDlQ8vDxMUj4bouKMrSEAUHkzc5UECDMDWh8M+AHZcqw8u9r+HrkIjdd0rOMePDir2HzHI5c3OW/dbjdCfhwGo8Wj8f1u4oo5pl+1PyW/9eGqiNGHkZmu6QKV+zn8H31PcSlTaRiY1xXkprvGpTEUEjniBYc3XFaCZIH3dvOHdRLeFipiyURgFbzy/MVU5iKbospeBHuAQG0FlGV0w+GNjV2vcSbfWXGfcEA/hMKM+omQpu+b1yxR9ux5snURFotCxM8Ct+aE61Wjp4yB4S5kqxH9MMQrTdqGRKfDmLZWT5ORVNVlkc4Md2CnhM6rfSAudOBywRG0i8vw1th8PfiE3fJPby48uSBVh2Ttx/nKRf9FFjOZWHfTDURmMcJQlrXeT+hyGRhUqnFtbPXfSQvf1bOIJDiuFaGkqlOxL9C/sM/cjdWPxCQ4YtPs4qX8Sek1OIdH6MC5bnngQB1VtrgyfLVaYtsWna6w4Iu6+X47iiifmGsPMvJdgnORczzjBB30c4huVIAe0jK1JD5dsGb2D48H1fYeN5BLVz/harSIm6z9j1T1Qo0FzwOKMdNdsyaFmL7WI19b4 x+tclD3s edjLIKXSCH1XW3fHHAEe7oG+ugsB4leScAFpmOL/i3t0bThk63B7Iup08nAQY+4GY4E0VgsGR/ryNK16SYCsCElmWaXrLS3t9xpI7tiuvICHitkJ91GBzKO1qJqx9BhemZM/QZ3pU6W3Ep4h6jvjnQneyA3zwaqIzZwVjdyRbjxl0f/Mxye7KF6M/UojShcRZcN4RXHcN+gzCCt6klaxoIX/Nhk1jNVpIXRu5rH0tLoXME/UKJP4wG7YPgOE+VVICxNriWHZUsTin0qQOUpT4LTdu+SR6v/tputi7m3jwSk5H8+zKkzUc6vZtIN27LzaygCWkkoF7yZzCL+Q+jLtFWUGj8febiIKz/jLBx8Zo+QK6Wayev0yCL2xzcQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Change rstat flush and updated API calls to accept a cgroup_subsys_state instead of a cgroup. Signed-off-by: JP Kobryn --- block/blk-cgroup.c | 4 ++-- include/linux/cgroup.h | 8 ++++---- kernel/cgroup/cgroup.c | 4 ++-- kernel/cgroup/rstat.c | 28 +++++++++++++++++++--------- mm/memcontrol.c | 4 ++-- 5 files changed, 29 insertions(+), 19 deletions(-) diff --git a/block/blk-cgroup.c b/block/blk-cgroup.c index 45a395862fbc..98e586ae21ec 100644 --- a/block/blk-cgroup.c +++ b/block/blk-cgroup.c @@ -1200,7 +1200,7 @@ static int blkcg_print_stat(struct seq_file *sf, void *v) if (!seq_css(sf)->parent) blkcg_fill_root_iostats(); else - cgroup_rstat_flush(blkcg->css.cgroup); + cgroup_rstat_flush(&blkcg->css); rcu_read_lock(); hlist_for_each_entry_rcu(blkg, &blkcg->blkg_list, blkcg_node) { @@ -2183,7 +2183,7 @@ void blk_cgroup_bio_start(struct bio *bio) } u64_stats_update_end_irqrestore(&bis->sync, flags); - cgroup_rstat_updated(blkcg->css.cgroup, cpu); + cgroup_rstat_updated(&blkcg->css, cpu); put_cpu(); } diff --git a/include/linux/cgroup.h b/include/linux/cgroup.h index f8ef47f8a634..eec970622419 100644 --- a/include/linux/cgroup.h +++ b/include/linux/cgroup.h @@ -687,10 +687,10 @@ static inline void cgroup_path_from_kernfs_id(u64 id, char *buf, size_t buflen) /* * cgroup scalable recursive statistics. */ -void cgroup_rstat_updated(struct cgroup *cgrp, int cpu); -void cgroup_rstat_flush(struct cgroup *cgrp); -void cgroup_rstat_flush_hold(struct cgroup *cgrp); -void cgroup_rstat_flush_release(struct cgroup *cgrp); +void cgroup_rstat_updated(struct cgroup_subsys_state *css, int cpu); +void cgroup_rstat_flush(struct cgroup_subsys_state *css); +void cgroup_rstat_flush_hold(struct cgroup_subsys_state *css); +void cgroup_rstat_flush_release(struct cgroup_subsys_state *css); /* * Basic resource stats. diff --git a/kernel/cgroup/cgroup.c b/kernel/cgroup/cgroup.c index 43c949824814..fdddd5ec5f3c 100644 --- a/kernel/cgroup/cgroup.c +++ b/kernel/cgroup/cgroup.c @@ -5465,7 +5465,7 @@ static void css_release_work_fn(struct work_struct *work) /* css release path */ if (!list_empty(&css->rstat_css_node)) { - cgroup_rstat_flush(cgrp); + cgroup_rstat_flush(css); list_del_rcu(&css->rstat_css_node); } @@ -5493,7 +5493,7 @@ static void css_release_work_fn(struct work_struct *work) /* cgroup release path */ TRACE_CGROUP_PATH(release, cgrp); - cgroup_rstat_flush(cgrp); + cgroup_rstat_flush(css); spin_lock_irq(&css_set_lock); for (tcgrp = cgroup_parent(cgrp); tcgrp; diff --git a/kernel/cgroup/rstat.c b/kernel/cgroup/rstat.c index 5877974ece92..1ed0f3aab0d9 100644 --- a/kernel/cgroup/rstat.c +++ b/kernel/cgroup/rstat.c @@ -82,8 +82,9 @@ void _cgroup_rstat_cpu_unlock(raw_spinlock_t *cpu_lock, int cpu, * rstat_cpu->updated_children list. See the comment on top of * cgroup_rstat_cpu definition for details. */ -__bpf_kfunc void cgroup_rstat_updated(struct cgroup *cgrp, int cpu) +__bpf_kfunc void cgroup_rstat_updated(struct cgroup_subsys_state *css, int cpu) { + struct cgroup *cgrp = css->cgroup; raw_spinlock_t *cpu_lock = per_cpu_ptr(&cgroup_rstat_cpu_lock, cpu); unsigned long flags; @@ -346,8 +347,10 @@ static void cgroup_rstat_flush_locked(struct cgroup *cgrp) * * This function may block. */ -__bpf_kfunc void cgroup_rstat_flush(struct cgroup *cgrp) +__bpf_kfunc void cgroup_rstat_flush(struct cgroup_subsys_state *css) { + struct cgroup *cgrp = css->cgroup; + might_sleep(); __cgroup_rstat_lock(cgrp, -1); @@ -364,9 +367,11 @@ __bpf_kfunc void cgroup_rstat_flush(struct cgroup *cgrp) * * This function may block. */ -void cgroup_rstat_flush_hold(struct cgroup *cgrp) +void cgroup_rstat_flush_hold(struct cgroup_subsys_state *css) __acquires(&cgroup_rstat_lock) { + struct cgroup *cgrp = css->cgroup; + might_sleep(); __cgroup_rstat_lock(cgrp, -1); cgroup_rstat_flush_locked(cgrp); @@ -376,9 +381,11 @@ void cgroup_rstat_flush_hold(struct cgroup *cgrp) * cgroup_rstat_flush_release - release cgroup_rstat_flush_hold() * @cgrp: cgroup used by tracepoint */ -void cgroup_rstat_flush_release(struct cgroup *cgrp) +void cgroup_rstat_flush_release(struct cgroup_subsys_state *css) __releases(&cgroup_rstat_lock) { + struct cgroup *cgrp = css->cgroup; + __cgroup_rstat_unlock(cgrp, -1); } @@ -408,7 +415,7 @@ void cgroup_rstat_exit(struct cgroup *cgrp) { int cpu; - cgroup_rstat_flush(cgrp); + cgroup_rstat_flush(&cgrp->self); /* sanity check */ for_each_possible_cpu(cpu) { @@ -512,8 +519,10 @@ static void cgroup_base_stat_cputime_account_end(struct cgroup *cgrp, struct cgroup_rstat_cpu *rstatc, unsigned long flags) { + struct cgroup_subsys_state *css = &cgrp->self; + u64_stats_update_end_irqrestore(&rstatc->bsync, flags); - cgroup_rstat_updated(cgrp, smp_processor_id()); + cgroup_rstat_updated(css, smp_processor_id()); put_cpu_ptr(rstatc); } @@ -612,16 +621,17 @@ static void cgroup_force_idle_show(struct seq_file *seq, struct cgroup_base_stat void cgroup_base_stat_cputime_show(struct seq_file *seq) { - struct cgroup *cgrp = seq_css(seq)->cgroup; + struct cgroup_subsys_state *css = seq_css(seq); + struct cgroup *cgrp = css->cgroup; u64 usage, utime, stime, ntime; if (cgroup_parent(cgrp)) { - cgroup_rstat_flush_hold(cgrp); + cgroup_rstat_flush_hold(css); usage = cgrp->bstat.cputime.sum_exec_runtime; cputime_adjust(&cgrp->bstat.cputime, &cgrp->prev_cputime, &utime, &stime); ntime = cgrp->bstat.ntime; - cgroup_rstat_flush_release(cgrp); + cgroup_rstat_flush_release(css); } else { /* cgrp->bstat of root is not actually used, reuse it */ root_cgroup_cputime(&cgrp->bstat); diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 7b3503d12aaf..97552476b844 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -579,7 +579,7 @@ static inline void memcg_rstat_updated(struct mem_cgroup *memcg, int val) if (!val) return; - cgroup_rstat_updated(memcg->css.cgroup, cpu); + cgroup_rstat_updated(&memcg->css, cpu); statc = this_cpu_ptr(memcg->vmstats_percpu); for (; statc; statc = statc->parent) { stats_updates = READ_ONCE(statc->stats_updates) + abs(val); @@ -611,7 +611,7 @@ static void __mem_cgroup_flush_stats(struct mem_cgroup *memcg, bool force) if (mem_cgroup_is_root(memcg)) WRITE_ONCE(flush_last_time, jiffies_64); - cgroup_rstat_flush(memcg->css.cgroup); + cgroup_rstat_flush(&memcg->css); } /* From patchwork Fri Jan 3 01:50:13 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: JP Kobryn X-Patchwork-Id: 13925084 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2CB6CE77188 for ; Fri, 3 Jan 2025 01:50:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7C7FC6B0083; Thu, 2 Jan 2025 20:50:37 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 751FB6B0085; Thu, 2 Jan 2025 20:50:37 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 57C046B0088; Thu, 2 Jan 2025 20:50:37 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 3C9686B0083 for ; Thu, 2 Jan 2025 20:50:37 -0500 (EST) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id E22FF80239 for ; Fri, 3 Jan 2025 01:50:36 +0000 (UTC) X-FDA: 82964459676.30.1C720CE Received: from mail-pj1-f47.google.com (mail-pj1-f47.google.com [209.85.216.47]) by imf15.hostedemail.com (Postfix) with ESMTP id 2806CA0006 for ; Fri, 3 Jan 2025 01:49:00 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b="bx0n/+pF"; spf=pass (imf15.hostedemail.com: domain of inwardvessel@gmail.com designates 209.85.216.47 as permitted sender) smtp.mailfrom=inwardvessel@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1735869001; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=HgCq5Td2Z/AmRDTJOLyjzlVyZ7Vs1foFuhquS5rZUBM=; b=nypksiZ5PEez1MIi/MilTqfVDTHFsUkZDjJhzW9Tk0kAk0eJo57Axqx1ddceG/Qi/h9bsP 3WxYaVujI6383YMYxIp+K/6xPYIgPJD4t7Ovx9uFSbSOAnPPV+TnbxuTIwJBS2btJRDlKX ++Z1eBI05OLaRgEERmQObDXr2fCFECc= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1735869001; a=rsa-sha256; cv=none; b=GT7B2gg3eL9L1gDjgk60vHhuaPKUudk8qY5eC5XpFfOYD5BSDIkCPPunvzHtKqpaBEHh4p YCB73W0Gl249Ax9a7kFld0zyoSBeRP62CZZaHB2XYubE1rygCiAwtayWsHhxhXdevr9u5h mFw1v681Z1RQxLEj+Z/n9SWJw975MnA= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b="bx0n/+pF"; spf=pass (imf15.hostedemail.com: domain of inwardvessel@gmail.com designates 209.85.216.47 as permitted sender) smtp.mailfrom=inwardvessel@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-pj1-f47.google.com with SMTP id 98e67ed59e1d1-2ef72924e53so16575637a91.3 for ; Thu, 02 Jan 2025 17:50:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1735869033; x=1736473833; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=HgCq5Td2Z/AmRDTJOLyjzlVyZ7Vs1foFuhquS5rZUBM=; b=bx0n/+pF+i2b56Jb7ewUl/ytetXqY3ByDHh/6H9rjW5EhSglQ3VYAIpzHuZnBS3awA Ua+aWCoX6GmBRgRzJ0gXt1IKWrFhjcQI+gCZu9BMDitABEkbxtkYYtg0a37kir5ivMyg sb5esOd9+q3xo1NFeMHW325BcdQbfw4zLpJ0n8+oTbfNs2J61KvmjbeuVMCNCSmHIcW/ 6DZNkUAMJdKgkx0X9e0R32shkYchBX4WbtY728zoBCCX1FTIMzqysjnj6xEOVpvzPBy7 L/3VU6H9WBHmrPR8sNpwziIfU0lq4aLjwa/M5UgZfx5pVDJJvca+TGskEsnzEBbaVK3c DpVw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1735869033; x=1736473833; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=HgCq5Td2Z/AmRDTJOLyjzlVyZ7Vs1foFuhquS5rZUBM=; b=Yhht4h/rbIecX2+RwFiFQC6HeebwIPY0DUyISCxuZf4qV9wXYXtoMkijpjAy0LZx3t 8lQNEZiHJeReDKspF9/w6AjEE2rPL+bF46i4MX0OPihvNuECLpP1Bf4TmzDMlUyZmgWg hSVd/0CDOPha6ph+oieAaQvDfvOskcXZJI7sHko5lYxHveAydFW70+8rfTcWaFn7+Qky Gc8vEzWEd5gqqMzaCaYWLxd4iTfqpQ74QzQOOm/2ddjg1CiigoQVbjCMBZAH2NPQ/duJ pJVxJ22QC5gnT147JZBWjauHkY8XNIu/iLVE6PEJmwmIA+HB3K03YOLFnWrph0/iH28g lG0w== X-Gm-Message-State: AOJu0Yz2OJ5a9R0vjZ1hRh2AW3OjaTPv5Wp5vT5scQYNu7vUGUqc303N v0ja7ym0TPiLXK2uaPM7xtH8hhxRsom7yEko/G5VxLHVmXe/aXaq X-Gm-Gg: ASbGnctc3GwZlnkDK1hdUyUIoYoXXr+rhYbvYgazosNwNqnTc1Bw28Z5CD69OX0AjUg FUkVbILhfYQxlwtiWSO16/EEnp18Hywm9p+BD1BF5PkmFxQyMhKSQ5s2kMR2JDsto4kozFtnx7Z 6GAyDIwUencedVILspMAS5xnGFs6qpy8ELWH/RFBusdPA5BpKUECre5RlXJzqI6g1LN7ZnybxXd G3GfJ8Lmzq0ZgXZqmyDdq8AgclFzzDDsQ/g67wVLZWtAismCgUBHMduJz4aE6fADX6Nm5fjmQx2 KiKYFFV0udWBcwBmVA== X-Google-Smtp-Source: AGHT+IGkWtACbpMmHueLMAjxpHU9KrMkr+iF3LJPrQIgx8bENjfG+waIs+Dzt0sDPxQQ6X70fr4+hw== X-Received: by 2002:a17:90a:e18f:b0:2ee:c291:765a with SMTP id 98e67ed59e1d1-2f452debd4fmr80002639a91.8.1735869033613; Thu, 02 Jan 2025 17:50:33 -0800 (PST) Received: from saturn.. (c-67-188-127-15.hsd1.ca.comcast.net. [67.188.127.15]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-219dca04ce7sm228851505ad.283.2025.01.02.17.50.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 02 Jan 2025 17:50:33 -0800 (PST) From: JP Kobryn To: shakeel.butt@linux.dev, tj@kernel.org, mhocko@kernel.org, hannes@cmpxchg.org, yosryahmed@google.com, akpm@linux-foundation.org Cc: linux-mm@kvack.org, cgroups@vger.kernel.org Subject: [RFC PATCH 2/9 v2] cgroup: change cgroup to css in rstat internal flush and lock funcs Date: Thu, 2 Jan 2025 17:50:13 -0800 Message-ID: <20250103015020.78547-3-inwardvessel@gmail.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250103015020.78547-1-inwardvessel@gmail.com> References: <20250103015020.78547-1-inwardvessel@gmail.com> MIME-Version: 1.0 X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 2806CA0006 X-Stat-Signature: ai1r8kc6cgnindzid3o7mnn7yk6rixxn X-Rspam-User: X-HE-Tag: 1735868940-760580 X-HE-Meta: U2FsdGVkX18sHeuRS5QshZx/I3VyWsQL/IUMCqOry3I/WRjspphm4NodlfvXiuPXa/6o723FiV4E6VZaOIr4RmGP0pO4VlmQHOgUa4oRLt2uD+884e+Q2SClb2/VAV+wS+E15vRt/9esE5nPkQZngE+6436fvWu8v0B4rj7pngm3Zjnbs7tUyaUEKGkuvgE0rKYnHPMDyMR81no7EeH8Hj0bBDEi6FXmkRs8G6Q3X4CrGr+UF2A3NYkhWVcjxi25x3Cgj1Y3zy8FZqpfgQT2MRNucdUqqSmMdTaPjuleZdh1KuKFXK6QNqn5LdR4Y9fNHMqDm0ovz/0hzh/862iOdP6OKk2puQwhrils29taXeqeQp7HHnJfIPLTQww9QAZ1C5nixOexXYhKr0Qb5Sx96zkwmGA1rNwT2zG0OJ12qt7bW49UGMN1uUdtwmvEB+EMTRgQDyMKkvULfICv3chV3gFjAyl/zIynoyUGBll8tSN0LIhBUTQdWb11E2MqbAdY7gykNe88Y0o115SG5ZjQlITWTeQpqfb7UvI/lxVkW4Gj0/7sxJxdlk26wdeF9zdBCRW+ERdq91LFECriL/yMPEm3ibBIoee1BkK0Z3+omlhSv3XcC514ZP10SXYVXEyROmthiYV6g7cwLixKwd01IiIuHq+tJWhEf93RgBrndjqYOezjda0wx3u7MdnJdpkJnOUX3uEPHV0LNOWUB496weo/2+saq/3EAmyqAVm5EokTrUiSuV+9b6mttxyF3Rs5+190nJxqCDrLP5vPMjM+5QBMKcIwO0XmrG4Ip5UiMck0fe0nY7XLT7o4jEyVvzSoATq62vQ4uTx1LtLTQFnCxclTntBNcTG8fl5drf4ucmvx2QhSwubjDI8xPf6Wic4SVWkBjgs9/LDOj9r4tpS1czwqPndA2z5r4ZLWtXITuH64VjftqkPKBga2B6zUV+p/FNHBLVP0KoVhSCND3B0 +Bw6Ni9S WYU/doSZdXKAI3wIeTS/pdymTlGVOk7pqdr9osWvSD43PsDNyexjOnWFSAYavZDNQmy09N62KRbUVqCb0e0McK2hrQAguLEB9+KS01/o4308H+8EwUr5p9U1TVuA16PAUIr/KZkFdJO9CIaIt8xDHADQos58GBeuPnmaoaUPB+u8/z4arGXlf+VTFX7TwKP8vk3s4KnJwu1eURv1unvY21rAIwSNzE8pleWKN1rVOtbGwOr4l0mMh3S4JK9k9zhLPoe4Hd0r0h3rN3t+j9XZWPEFELiSwpDtkomDb5kp4vvLEBe2nrIFLzH38e1gsxMGDPx7F8Gow86CrsAIL+Vywv8a3Tb0WbopUsCUIkwb1Ie5rW9nfvccP/yX/hw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.014009, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Change internal rstat functions to accept a cgroup_subsys_state instead of a cgroup. Signed-off-by: JP Kobryn --- kernel/cgroup/rstat.c | 45 ++++++++++++++++++++++--------------------- 1 file changed, 23 insertions(+), 22 deletions(-) diff --git a/kernel/cgroup/rstat.c b/kernel/cgroup/rstat.c index 1ed0f3aab0d9..1b7ef8690a09 100644 --- a/kernel/cgroup/rstat.c +++ b/kernel/cgroup/rstat.c @@ -201,8 +201,10 @@ static struct cgroup *cgroup_rstat_push_children(struct cgroup *head, * within the children list and terminated by the parent cgroup. An exception * here is the cgroup root whose updated_next can be self terminated. */ -static struct cgroup *cgroup_rstat_updated_list(struct cgroup *root, int cpu) +static struct cgroup *cgroup_rstat_updated_list(struct cgroup_subsys_state *root_css, + int cpu) { + struct cgroup *root = root_css->cgroup; raw_spinlock_t *cpu_lock = per_cpu_ptr(&cgroup_rstat_cpu_lock, cpu); struct cgroup_rstat_cpu *rstatc = cgroup_rstat_cpu(root, cpu); struct cgroup *head = NULL, *parent, *child; @@ -280,9 +282,11 @@ __bpf_hook_end(); * value -1 is used when obtaining the main lock else this is the CPU * number processed last. */ -static inline void __cgroup_rstat_lock(struct cgroup *cgrp, int cpu_in_loop) +static inline void __cgroup_rstat_lock(struct cgroup_subsys_state *css, + int cpu_in_loop) __acquires(&cgroup_rstat_lock) { + struct cgroup *cgrp = css->cgroup; bool contended; contended = !spin_trylock_irq(&cgroup_rstat_lock); @@ -293,15 +297,18 @@ static inline void __cgroup_rstat_lock(struct cgroup *cgrp, int cpu_in_loop) trace_cgroup_rstat_locked(cgrp, cpu_in_loop, contended); } -static inline void __cgroup_rstat_unlock(struct cgroup *cgrp, int cpu_in_loop) +static inline void __cgroup_rstat_unlock(struct cgroup_subsys_state *css, + int cpu_in_loop) __releases(&cgroup_rstat_lock) { + struct cgroup *cgrp = css->cgroup; + trace_cgroup_rstat_unlock(cgrp, cpu_in_loop, false); spin_unlock_irq(&cgroup_rstat_lock); } /* see cgroup_rstat_flush() */ -static void cgroup_rstat_flush_locked(struct cgroup *cgrp) +static void cgroup_rstat_flush_locked(struct cgroup_subsys_state *css) __releases(&cgroup_rstat_lock) __acquires(&cgroup_rstat_lock) { int cpu; @@ -309,27 +316,27 @@ static void cgroup_rstat_flush_locked(struct cgroup *cgrp) lockdep_assert_held(&cgroup_rstat_lock); for_each_possible_cpu(cpu) { - struct cgroup *pos = cgroup_rstat_updated_list(cgrp, cpu); + struct cgroup *pos = cgroup_rstat_updated_list(css, cpu); for (; pos; pos = pos->rstat_flush_next) { - struct cgroup_subsys_state *css; + struct cgroup_subsys_state *css_iter; cgroup_base_stat_flush(pos, cpu); bpf_rstat_flush(pos, cgroup_parent(pos), cpu); rcu_read_lock(); - list_for_each_entry_rcu(css, &pos->rstat_css_list, + list_for_each_entry_rcu(css_iter, &pos->rstat_css_list, rstat_css_node) - css->ss->css_rstat_flush(css, cpu); + css_iter->ss->css_rstat_flush(css_iter, cpu); rcu_read_unlock(); } /* play nice and yield if necessary */ if (need_resched() || spin_needbreak(&cgroup_rstat_lock)) { - __cgroup_rstat_unlock(cgrp, cpu); + __cgroup_rstat_unlock(css, cpu); if (!cond_resched()) cpu_relax(); - __cgroup_rstat_lock(cgrp, cpu); + __cgroup_rstat_lock(css, cpu); } } } @@ -349,13 +356,11 @@ static void cgroup_rstat_flush_locked(struct cgroup *cgrp) */ __bpf_kfunc void cgroup_rstat_flush(struct cgroup_subsys_state *css) { - struct cgroup *cgrp = css->cgroup; - might_sleep(); - __cgroup_rstat_lock(cgrp, -1); - cgroup_rstat_flush_locked(cgrp); - __cgroup_rstat_unlock(cgrp, -1); + __cgroup_rstat_lock(css, -1); + cgroup_rstat_flush_locked(css); + __cgroup_rstat_unlock(css, -1); } /** @@ -370,11 +375,9 @@ __bpf_kfunc void cgroup_rstat_flush(struct cgroup_subsys_state *css) void cgroup_rstat_flush_hold(struct cgroup_subsys_state *css) __acquires(&cgroup_rstat_lock) { - struct cgroup *cgrp = css->cgroup; - might_sleep(); - __cgroup_rstat_lock(cgrp, -1); - cgroup_rstat_flush_locked(cgrp); + __cgroup_rstat_lock(css, -1); + cgroup_rstat_flush_locked(css); } /** @@ -384,9 +387,7 @@ void cgroup_rstat_flush_hold(struct cgroup_subsys_state *css) void cgroup_rstat_flush_release(struct cgroup_subsys_state *css) __releases(&cgroup_rstat_lock) { - struct cgroup *cgrp = css->cgroup; - - __cgroup_rstat_unlock(cgrp, -1); + __cgroup_rstat_unlock(css, -1); } int cgroup_rstat_init(struct cgroup *cgrp) From patchwork Fri Jan 3 01:50:14 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: JP Kobryn X-Patchwork-Id: 13925085 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CDF1BE77198 for ; Fri, 3 Jan 2025 01:50:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AFD286B0088; Thu, 2 Jan 2025 20:50:38 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id AAE396B0089; Thu, 2 Jan 2025 20:50:38 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 83C626B008A; Thu, 2 Jan 2025 20:50:38 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 64FC66B0088 for ; Thu, 2 Jan 2025 20:50:38 -0500 (EST) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id E2470AEB7C for ; Fri, 3 Jan 2025 01:50:37 +0000 (UTC) X-FDA: 82964457744.17.DC52D1E Received: from mail-pl1-f169.google.com (mail-pl1-f169.google.com [209.85.214.169]) by imf02.hostedemail.com (Postfix) with ESMTP id 2CBEA80005 for ; Fri, 3 Jan 2025 01:48:57 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=Xp4aMFQo; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf02.hostedemail.com: domain of inwardvessel@gmail.com designates 209.85.214.169 as permitted sender) smtp.mailfrom=inwardvessel@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1735868988; a=rsa-sha256; cv=none; b=LrYgdKxLge2JOqlEWhp/XxOVd4322SEEhCCITnKcJhUxcp8+LHP15yoIh5MP8WcH4NQApI Wyqk7EbXQwzmXWwbWhcvVMaQLlMrq6YVF72O/ULLOytqTku9Cibe/zRS5vNvN8zeqrQuoo 99Ho4IcLBqoUFxF3em0Jjs4yw35OslA= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=Xp4aMFQo; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf02.hostedemail.com: domain of inwardvessel@gmail.com designates 209.85.214.169 as permitted sender) smtp.mailfrom=inwardvessel@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1735868988; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=0fygAlv7C86dgt6jCWO9W9TZivz+mHpPPlFRRnfsUsg=; b=TW/9fuFm0ZC06Q9hDbXVRHU4aTUOFR0v88KibHWUpk/vRGh5tsOPE4/qZEHLZojrRwh9Bf dWm/9IQytaOPR6O51J6Tz5eaKvDDo833wW0pZ37pyteafVtgwIT4NuSOLqYMPEs5XCFk3I T7nIg/I/vvLwwJdKfEs8PR4vcVV1IXQ= Received: by mail-pl1-f169.google.com with SMTP id d9443c01a7336-21683192bf9so162871535ad.3 for ; Thu, 02 Jan 2025 17:50:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1735869035; x=1736473835; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=0fygAlv7C86dgt6jCWO9W9TZivz+mHpPPlFRRnfsUsg=; b=Xp4aMFQoHcNbn0rKGXebbWY5IpeY8sqi/Nf/6wu/OK/Y2iI7akRj+Z75jRY7tOdxwk KOMnpza3Q+opqlHVPAQk2Z5ocpoygrmdg62UZG3j4z21QLy0wR9LCXrZlyo3C7zJeTLU cg1BF86SynQbBrwTTCvrNBj/Avs5t70U4tIxTPDkqzwz+JfHuqy8mvVak0EEsDeBjLuy 65orQL6Di/MzCFdtTQjOWALDYl804CVastWdMvtOuI4Y2Wlvq8abz+FWI6RB2TOVgizu POqJWn75k+BiWEymANqJ8XdM13MYt+mrqaw4osyDQmsy3dXkxrXm9kl8f4AK9ZSqXkAZ 4s7A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1735869035; x=1736473835; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=0fygAlv7C86dgt6jCWO9W9TZivz+mHpPPlFRRnfsUsg=; b=d529C0Jztn20OdC5wj82Lg68EdfZ3Q32OWg7iBjocC6sMgyNfX5mDVtTxybeBjX8Ru 3dezhs0F2Dc2NVyqhFpDLVw/zMsa0oFpD2+97kiY3D4McK7p4wNK+q1uxusqzjOTGNgt DS4xb12GDjSCj1zHs/eMdsZIN/CWgG16SlcKaVmZI/zpzUsAYtP6+a+oMVS81IRYmxTv zRYLgEIvGfcouXVksIPWkQco3CtIUcfaWHf1GXXW552ulfOD2m7IVkBF683ZKc/yWnwj QYmoA+FXGzhb3O3bSzjEw7UXRxsxVKGccC6xo39T/gR2rt2f0qDEk5q9kvj/4ArRvEJb O9fw== X-Gm-Message-State: AOJu0Yyy86oRYRa7iU6a4Xk2IZm3wm829RiiQgKL6m5gzBx8LVyQK5NN 8PtwgxOclvxKphqLX+Qn6aL0Mvr67YuBltCizXdEqgNBIbXImq0s X-Gm-Gg: ASbGncu3P3YunuGX+OAIYSCrIcz5wHjbFwDu+n5ZyflwtzPOTf9tYAOBPfvLMKVb58M eAa9a+Xij3xQz+ihgo99ikbY76NYiMke3NpGRxMONGdWrFqvq1gcwWl3rljaXl9s2jHiNn39BtM Is4tuzbJJf+i2n0/paQoq88gFcUiJ2P4cXm+WjJJuPBRFaX8G4ijT9i68m3N32W/VCx53PxkZbw yJ2AZsJoyNlguSyPcdV9EnY8zQ853nrQL52tqI+uZBWhKX4hp1fNObQB8W4TM/ztRj3Sou3uqD2 7MgHDOYxOtkCCh7Zpw== X-Google-Smtp-Source: AGHT+IFUq6pIKACZFu16YUBj6vJ2nI7etocge0+ZlELfgHcsYpw2HURYCpbJolc7uELa/ETm08W0tQ== X-Received: by 2002:a17:902:c951:b0:215:a039:738 with SMTP id d9443c01a7336-219e6e8928emr709570885ad.5.1735869034959; Thu, 02 Jan 2025 17:50:34 -0800 (PST) Received: from saturn.. (c-67-188-127-15.hsd1.ca.comcast.net. [67.188.127.15]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-219dca04ce7sm228851505ad.283.2025.01.02.17.50.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 02 Jan 2025 17:50:34 -0800 (PST) From: JP Kobryn To: shakeel.butt@linux.dev, tj@kernel.org, mhocko@kernel.org, hannes@cmpxchg.org, yosryahmed@google.com, akpm@linux-foundation.org Cc: linux-mm@kvack.org, cgroups@vger.kernel.org Subject: [RFC PATCH 3/9 v2] cgroup: change cgroup to css in rstat init and exit api Date: Thu, 2 Jan 2025 17:50:14 -0800 Message-ID: <20250103015020.78547-4-inwardvessel@gmail.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250103015020.78547-1-inwardvessel@gmail.com> References: <20250103015020.78547-1-inwardvessel@gmail.com> MIME-Version: 1.0 X-Stat-Signature: ydudtkt6hwsa8nwkd5ngw9qzqyd8cxoz X-Rspam-User: X-Rspamd-Queue-Id: 2CBEA80005 X-Rspamd-Server: rspam08 X-HE-Tag: 1735868937-971665 X-HE-Meta: U2FsdGVkX1+fCEmMja76KHb1tXxDLOUhvXcN1iTLm2UfV4X6mCpbIhBjnl0wRerp+ofE2wHrMO+f5pPao1oTC4sPsm1yYbNVhdrxljAakB7dyWeP1qG+Tyk3Ewrz3qeDjzxkV6/EZltQk4nFYEjVBPSQqSz9XCc5GXtdeHiNSJ/BqMBGTBJ+RDaS05hf3ChAAaV2gb4OPWAHifsioEudZwNoaSU5u6aFGYII4QDWX/L8fUBr6xO0w/4PsrHLFY7NgUGQRPr1RCLjLKgGgLRzE4FROCUBiYSR7xQIw3QsOTBmWeXgoXZU8z4bupMZUHCZmXXkMRQM+GyiMMXnffnCQsvkhuKvQvzDjiFffhQboC6lHo1GMBIZgxz+S0hIVCLrDNGoAYDEZeYaHQF0sLW3olgELeV2LTObG9SCzTTOiDhcqTNAo5XeZ/25cZfr3t0r8CgQ8Hgd8J3S+0J3I6huyME3xA41PwKJ1YAubjJPmZKgmqdRP394xMwMm80PwaOynotu0JIuMiqLoWSPEUaziZ1x7QzyGaVOGKbzy6Eu66+cNHiEgLU/jCR0XYCBozyquhC1ac2GNCLnAuTweq/Q+aIAm2Pm1WkrDgN2+NiOpzlIXC/6C0S8MvfwM6gaCTxQOXcPbEiaSiP/6OFakO0n/gmDSQuWXn0HkF9O9sNC+cW46lhyRRLUaQC+f8KT5rj+FRChri7t6vl9WfwqttmYfLwaJaDVp8TUaksckbCIqAYgV+Q+Ttd/aa7GbEGPoXdRoXKtKXyBqIglNabUkVCGzdfKMP7oIwM+fRJR3YQuizmmWR25RrpNOZ1sBDHUtt9lMof/bnJv6S02px4eRzGXp5i5745AIND5SEQeWuhNdwB4jTleqw5ZxgJIvwtK3fs25CRj2GakukA98inSUF3pqUvlnzMcodrhSjI80gkOOCsGpOrpqvCcScGp5odW/cawPTwYgF5MQRS3vgXAaZu dn5i7gXY Q0kev0QsM37jw/8j6jAcJO8qeh9Gk0F86OclkHzwMUZu/6A7kQChf8736PpPXyauTQKQ99f99ZlfiJaPyu08aOi0uy0y/CiNesDoliVLus+C3sTya0N+OwZyuvo2HoP+S8muqQUlCFYRXKOztQuheXx1tCzTs+Q1nvQR0SyHqyWg57j/wMoe2O/8+QVnIXlufKbsHsKuSrmhAD6iWgpbUNOHgdeteplT7nEpbU+HnVvG7UrNfEqdH6h0JmX8Sd8Jz9j9z0/I7HCfr/1Xu0MEhDsM7SGO7PVLlYuNpkxJByhbayX+nMYfrNzpqa39IWkitsZzp3r0yZ4HUAgaojzmDgglSWa0VkUtzYzZdQjUkkg/MrNO4UU1fo4jb3w== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Change rstat init and exit API calls to accept a cgroup_subsys_state instead of a cgroup. Signed-off-by: JP Kobryn --- kernel/cgroup/cgroup-internal.h | 4 ++-- kernel/cgroup/cgroup.c | 16 ++++++++++------ kernel/cgroup/rstat.c | 6 ++++-- 3 files changed, 16 insertions(+), 10 deletions(-) diff --git a/kernel/cgroup/cgroup-internal.h b/kernel/cgroup/cgroup-internal.h index c964dd7ff967..87d062baff90 100644 --- a/kernel/cgroup/cgroup-internal.h +++ b/kernel/cgroup/cgroup-internal.h @@ -269,8 +269,8 @@ int cgroup_task_count(const struct cgroup *cgrp); /* * rstat.c */ -int cgroup_rstat_init(struct cgroup *cgrp); -void cgroup_rstat_exit(struct cgroup *cgrp); +int cgroup_rstat_init(struct cgroup_subsys_state *css); +void cgroup_rstat_exit(struct cgroup_subsys_state *css); void cgroup_rstat_boot(void); void cgroup_base_stat_cputime_show(struct seq_file *seq); diff --git a/kernel/cgroup/cgroup.c b/kernel/cgroup/cgroup.c index fdddd5ec5f3c..848e09f433c0 100644 --- a/kernel/cgroup/cgroup.c +++ b/kernel/cgroup/cgroup.c @@ -1358,7 +1358,7 @@ static void cgroup_destroy_root(struct cgroup_root *root) cgroup_unlock(); - cgroup_rstat_exit(cgrp); + cgroup_rstat_exit(&cgrp->self); kernfs_destroy_root(root->kf_root); cgroup_free_root(root); } @@ -2132,7 +2132,7 @@ int cgroup_setup_root(struct cgroup_root *root, u16 ss_mask) if (ret) goto destroy_root; - ret = cgroup_rstat_init(root_cgrp); + ret = cgroup_rstat_init(&root_cgrp->self); if (ret) goto destroy_root; @@ -2174,7 +2174,7 @@ int cgroup_setup_root(struct cgroup_root *root, u16 ss_mask) goto out; exit_stats: - cgroup_rstat_exit(root_cgrp); + cgroup_rstat_exit(&root_cgrp->self); destroy_root: kernfs_destroy_root(root->kf_root); root->kf_root = NULL; @@ -5435,7 +5435,7 @@ static void css_free_rwork_fn(struct work_struct *work) cgroup_put(cgroup_parent(cgrp)); kernfs_put(cgrp->kn); psi_cgroup_free(cgrp); - cgroup_rstat_exit(cgrp); + cgroup_rstat_exit(css); kfree(cgrp); } else { /* @@ -5686,7 +5686,11 @@ static struct cgroup *cgroup_create(struct cgroup *parent, const char *name, if (ret) goto out_free_cgrp; - ret = cgroup_rstat_init(cgrp); + /* init self cgroup early so css->cgroup is valid within cgroup_rstat_init() + * note that this will go away in a subsequent patch in this series + */ + cgrp->self.cgroup = cgrp; + ret = cgroup_rstat_init(&cgrp->self); if (ret) goto out_cancel_ref; @@ -5779,7 +5783,7 @@ static struct cgroup *cgroup_create(struct cgroup *parent, const char *name, out_kernfs_remove: kernfs_remove(cgrp->kn); out_stat_exit: - cgroup_rstat_exit(cgrp); + cgroup_rstat_exit(&cgrp->self); out_cancel_ref: percpu_ref_exit(&cgrp->self.refcnt); out_free_cgrp: diff --git a/kernel/cgroup/rstat.c b/kernel/cgroup/rstat.c index 1b7ef8690a09..01a5c185b02a 100644 --- a/kernel/cgroup/rstat.c +++ b/kernel/cgroup/rstat.c @@ -390,8 +390,9 @@ void cgroup_rstat_flush_release(struct cgroup_subsys_state *css) __cgroup_rstat_unlock(css, -1); } -int cgroup_rstat_init(struct cgroup *cgrp) +int cgroup_rstat_init(struct cgroup_subsys_state *css) { + struct cgroup *cgrp = css->cgroup; int cpu; /* the root cgrp has rstat_cpu preallocated */ @@ -412,8 +413,9 @@ int cgroup_rstat_init(struct cgroup *cgrp) return 0; } -void cgroup_rstat_exit(struct cgroup *cgrp) +void cgroup_rstat_exit(struct cgroup_subsys_state *css) { + struct cgroup *cgrp = css->cgroup; int cpu; cgroup_rstat_flush(&cgrp->self); From patchwork Fri Jan 3 01:50:15 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: JP Kobryn X-Patchwork-Id: 13925086 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D61A0E77197 for ; Fri, 3 Jan 2025 01:50:44 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 334966B0089; Thu, 2 Jan 2025 20:50:40 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 2E58B6B008A; Thu, 2 Jan 2025 20:50:40 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 136496B008C; Thu, 2 Jan 2025 20:50:40 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id E3C176B0089 for ; Thu, 2 Jan 2025 20:50:39 -0500 (EST) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 92475AEB6E for ; Fri, 3 Jan 2025 01:50:39 +0000 (UTC) X-FDA: 82964460180.15.9B49EB3 Received: from mail-pl1-f172.google.com (mail-pl1-f172.google.com [209.85.214.172]) by imf07.hostedemail.com (Postfix) with ESMTP id E47A34000F for ; Fri, 3 Jan 2025 01:49:03 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b="M5rdg/CO"; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf07.hostedemail.com: domain of inwardvessel@gmail.com designates 209.85.214.172 as permitted sender) smtp.mailfrom=inwardvessel@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1735868990; a=rsa-sha256; cv=none; b=w+UrlwEMRd9tGuYS8jNV4quH4DB8qrzhJbzezz//VkvSwFR/P4496ezyM4evuVWzUmAPJf H7Y3LfZPQfUxSjMUQAz7zCZ7fO87bvjQErz8Lt2wFBoCquD8W1yL1I4l/LwyQ5ZysnSWM2 3SrfPpDhbKy4H1XlPCz6xnZpjo1V5NI= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b="M5rdg/CO"; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf07.hostedemail.com: domain of inwardvessel@gmail.com designates 209.85.214.172 as permitted sender) smtp.mailfrom=inwardvessel@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1735868990; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=13LzpkMgO0b7mnyCmlNhqmibgyHhCXjzFtjSKM6aNYI=; b=w8CGIXPvRUNcjvhvGtP8Svl1zHCZS3J0dDYkyHVgXvwTHY3oHct4m9DvnK/2zo4TqydhX9 MMZhbR1fENpgkH+LHnXl0dzMiItnBqppeNQyY9AKkIXWurfQsc7RGxVugNkhOEkm5OJVq7 8y/EAPEOb+xRgtJoW6LdiZrCeJvQ/Rw= Received: by mail-pl1-f172.google.com with SMTP id d9443c01a7336-2166651f752so189385695ad.3 for ; Thu, 02 Jan 2025 17:50:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1735869036; x=1736473836; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=13LzpkMgO0b7mnyCmlNhqmibgyHhCXjzFtjSKM6aNYI=; b=M5rdg/CO3tJ85wTL4NJOSDzmBhEjdCWaFkIUHi0AqtbZyYV4DrhW0yCIFG4DKUqHkJ MsKPYalSB+boszBqryZ1qRKGUgoU7wgr5HUXmzUezria2Sbyyjo3RFiwvLHEg59FJwR7 HClXE6MDPGNWL+Rfr5Q4TdmZB3YlmTRCDvBd1RsGidAN1Nie1Aivc3y6cTWKYNL/0845 nzAtvMZIi6lTzrY1hmMEB2lLII66C3edBipe4zpNSRdltIHq/dGZ9dEvC9KVId3Dhf0j UUx0d8xJeXIW9QjQVzCfURgLN/eMHwJniONQzISPazO9EXX3Mji2ve2gYHzECn5roF5Z 6RgA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1735869036; x=1736473836; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=13LzpkMgO0b7mnyCmlNhqmibgyHhCXjzFtjSKM6aNYI=; b=oo+OF2dyR+V87YTACsCrQsCvO+WCuRKW0gmLW9nQ09QU0v5MK32maU378qGDa3E2se HNzTfFcDa76IWcsnvNJg0l5P4leqRtAwvWGAG3H98znpYR7U3rNo96W9QzgSNvA37dLC /1xSfct8ojexpXFVXmVebf+cYP9bq+gQ9N9ee6MykI49QlRVz9Nu69NPuoZQ3A/nmg/b g47ZMkHVESYrvvBHwO348yLlvTBebj0UAAnnEqhNQO6vaGQmYF1n6V8VnbSTKtOY9dx8 KMTYv+Q0XjT+yClWaamQMaOOE+Ww8Gg4YQyBrAASUIW9PzftzYnwgiHHOfLXll1P1yYG hEWA== X-Gm-Message-State: AOJu0Yz11OacC3KezkKNlg6nXe5fO7RjfwsGCVmk2jRqOF+k/GzsbvRg LwYt7PJtXtGPWinJ1sXqctWuk6JC58QKOK9CDX5Z17ALFY/o4kpd X-Gm-Gg: ASbGnctEilGuYI56HMMyflCpczILqody29wKGH9EmIbU1JHsIe1ZvQjwr57qjb7gkKO 51ocmIZPiXg5c4+wvNaNYIbiHb8SMK9AUhb/TGdKSyLDogJToFbf8t2HxizA6EhXcGtC2HE4gE7 qwyLE/DUKo+di7rzNGriQfAFqAmqVAPvJCfVwVsbIbKZbyjMpGHC8iwl38xCfwlP7uWjbfanC2d iIiLaAZwWBnxESbfXaIw+6WO5hbW8kN3NHnSpxFFc1d7IAR32I3FEq2Ju9RNNJaaAeuJgPOf1+u guEMZ0/7CPGpxDwK4g== X-Google-Smtp-Source: AGHT+IEWZ1I3vK8+zRzdBRThBatrKNxc73IbhcbqXpn8Rkn5yh7Bv+OzoAldowPIPiXgzKBnPFiCjw== X-Received: by 2002:a17:903:24e:b0:215:5625:885b with SMTP id d9443c01a7336-219e6f28552mr729349425ad.52.1735869036439; Thu, 02 Jan 2025 17:50:36 -0800 (PST) Received: from saturn.. (c-67-188-127-15.hsd1.ca.comcast.net. [67.188.127.15]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-219dca04ce7sm228851505ad.283.2025.01.02.17.50.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 02 Jan 2025 17:50:35 -0800 (PST) From: JP Kobryn To: shakeel.butt@linux.dev, tj@kernel.org, mhocko@kernel.org, hannes@cmpxchg.org, yosryahmed@google.com, akpm@linux-foundation.org Cc: linux-mm@kvack.org, cgroups@vger.kernel.org Subject: [RFC PATCH 4/9 v2] cgroup: split rstat from cgroup into separate css Date: Thu, 2 Jan 2025 17:50:15 -0800 Message-ID: <20250103015020.78547-5-inwardvessel@gmail.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250103015020.78547-1-inwardvessel@gmail.com> References: <20250103015020.78547-1-inwardvessel@gmail.com> MIME-Version: 1.0 X-Stat-Signature: b78doqntwrb9r6na7c6qduw3osbziww9 X-Rspam-User: X-Rspamd-Queue-Id: E47A34000F X-Rspamd-Server: rspam08 X-HE-Tag: 1735868943-610708 X-HE-Meta: U2FsdGVkX1+E8VYToOhvIsKW6m+4glBhNeHRiDa9Ydvm/B6b4d+ZyAyBRJp4usGCHK3xxX+ALWVEKQMolu0OZ3eHoL2+5MRrKWPZkLOyCrFUTyxijcDwdXKzncQqzddy2KfUb8TK9lwhZqkitVZvn9n3jD86dyo074rMOaBBCmJ/Q2XfRf2/98UkQ8QLMKmgqiPvxcZVQa7sNkg0RSpxZ7b6I3wDIxdgqT6UOWMuYMaIJNppK+q9+h1LbL/gZlIRvWcnlwbNbCkXM76zGlFLwf/Ibkhirf/bBmP9yaXlZbKqlZeXTm41BKhU5z5BsStp3o2YN156Y8ezvS0ZkgFgHvm6F84QgW6yoiQesk5VcjPECxc76zvUZyKKHOXZt9Trg+tnco0F+YX7C8Sgmph6//XGsxHyPjOpQnT9COnpDJ7hvq8NZiUfDzjH3cO6E7LX1XJXSsV0sa1jdaZ3Db8zYC6RzyxyoiVTAwIjpZkx6rIGzn/k+vpShqu1a5huKfHJEVLJzoKaqHGt+NKmml8lZzH5sYzyhAV/Mvxie7C7ZzBpt8lS/Tt5+CS8V42+aYefqUsl7RY+aUXnSNqe+K3sy3rDohhJfE1GTU8aRad9hWemm+nQxWolj/kkOzvJwLDK0LdXYbA7OfcrzFTcbOK98ScIDhjMwhHwG9yZXAewogMcquQ1RMz3I+vO/9kuQSjwxlikvdGtqBu7KLgy6jNHDCKgIE5m94bJ3PakzH9EZClgoaGJ80aQosj/fZNilGtUfdOc9FBRSmVCh5H3eB9rVzcJGo8F935mRhoYckXg/iqACGQg4Ts23t9NoU7Uob1KG1pRcAG6bp4nvTYVsW03bycbllfrXXtH9th7738m6aNqHQRLqIzlS/Bi0XSMLtdsYCBcd0X89PEwtHM1iE6oUh3aEK9EyCjrTCh7tQIjVFIa8aY3KZYE2VBRtxwUadpI7Z783OV84CcCc2ZCYOl 5L3dEgNM S6PLcJfB2/Dot9MjNpn16DN/50qAlT4siQDAbQ4oJ14qq9J2lWeyG+ELGkLzXe4sSWfL4YMZikd7BYlNKChAiig6ls604MWBGgm4dxN39FoR2gdSvZiBp5Gi4l0HpkwP7GHiJb4InO7rxmGidP7PdKx3MF3IwlpscuqUyfOx42atOCwWSHr1FEEluUyLdRuzLyniL2R1EWZlJexJ9Y8azUBcdTorLdzt4oavoAUHE/mlV/f+l6C6W9Rn9RhQlGu15nnhwW7WGGqf558MbHdLv34VzcsP7PV0/ySGw09jfiMKsL7rSNVOtt7kjKE/9Y0Tbe1ZKFJroda/+qFVGt1aTpYQZcSOZNs7fjBF4JdkU2scAmBpE0oIy1LuPlQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Move the rstat entities off of the cgroup struct and onto the cgroup_subsys_state struct. Adjust related code to reflect this new ownership. Signed-off-by: JP Kobryn --- include/linux/cgroup-defs.h | 40 ++++++++-------- kernel/cgroup/cgroup.c | 65 ++++++++++++++++++-------- kernel/cgroup/rstat.c | 92 ++++++++++++++++++------------------- 3 files changed, 111 insertions(+), 86 deletions(-) diff --git a/include/linux/cgroup-defs.h b/include/linux/cgroup-defs.h index 1b20d2d8ef7c..1932f8ae7995 100644 --- a/include/linux/cgroup-defs.h +++ b/include/linux/cgroup-defs.h @@ -180,6 +180,24 @@ struct cgroup_subsys_state { struct list_head sibling; struct list_head children; + /* per-cpu recursive resource statistics */ + struct cgroup_rstat_cpu __percpu *rstat_cpu; + struct list_head rstat_css_list; + + /* + * Add padding to separate the read mostly rstat_cpu and + * rstat_css_list into a different cacheline from the following + * rstat_flush_next and *bstat fields which can have frequent updates. + */ + CACHELINE_PADDING(_pad_); + + /* + * A singly-linked list of cgroup structures to be rstat flushed. + * This is a scratch field to be used exclusively by + * cgroup_rstat_flush_locked() and protected by cgroup_rstat_lock. + */ + struct cgroup_subsys_state *rstat_flush_next; + /* flush target list anchored at cgrp->rstat_css_list */ struct list_head rstat_css_node; @@ -389,8 +407,8 @@ struct cgroup_rstat_cpu { * * Protected by per-cpu cgroup_rstat_cpu_lock. */ - struct cgroup *updated_children; /* terminated by self cgroup */ - struct cgroup *updated_next; /* NULL iff not on the list */ + struct cgroup_subsys_state *updated_children; /* terminated by self cgroup */ + struct cgroup_subsys_state *updated_next; /* NULL iff not on the list */ }; struct cgroup_freezer_state { @@ -516,24 +534,6 @@ struct cgroup { struct cgroup *dom_cgrp; struct cgroup *old_dom_cgrp; /* used while enabling threaded */ - /* per-cpu recursive resource statistics */ - struct cgroup_rstat_cpu __percpu *rstat_cpu; - struct list_head rstat_css_list; - - /* - * Add padding to separate the read mostly rstat_cpu and - * rstat_css_list into a different cacheline from the following - * rstat_flush_next and *bstat fields which can have frequent updates. - */ - CACHELINE_PADDING(_pad_); - - /* - * A singly-linked list of cgroup structures to be rstat flushed. - * This is a scratch field to be used exclusively by - * cgroup_rstat_flush_locked() and protected by cgroup_rstat_lock. - */ - struct cgroup *rstat_flush_next; - /* cgroup basic resource statistics */ struct cgroup_base_stat last_bstat; struct cgroup_base_stat bstat; diff --git a/kernel/cgroup/cgroup.c b/kernel/cgroup/cgroup.c index 848e09f433c0..96a2d15fe5e9 100644 --- a/kernel/cgroup/cgroup.c +++ b/kernel/cgroup/cgroup.c @@ -164,7 +164,7 @@ static struct static_key_true *cgroup_subsys_on_dfl_key[] = { static DEFINE_PER_CPU(struct cgroup_rstat_cpu, cgrp_dfl_root_rstat_cpu); /* the default hierarchy */ -struct cgroup_root cgrp_dfl_root = { .cgrp.rstat_cpu = &cgrp_dfl_root_rstat_cpu }; +struct cgroup_root cgrp_dfl_root = { .cgrp.self.rstat_cpu = &cgrp_dfl_root_rstat_cpu }; EXPORT_SYMBOL_GPL(cgrp_dfl_root); /* @@ -1826,6 +1826,7 @@ int rebind_subsystems(struct cgroup_root *dst_root, u16 ss_mask) struct cgroup_root *src_root = ss->root; struct cgroup *scgrp = &src_root->cgrp; struct cgroup_subsys_state *css = cgroup_css(scgrp, ss); + struct cgroup_subsys_state *dcss = cgroup_css(dcgrp, ss); struct css_set *cset, *cset_pos; struct css_task_iter *it; @@ -1867,7 +1868,7 @@ int rebind_subsystems(struct cgroup_root *dst_root, u16 ss_mask) list_del_rcu(&css->rstat_css_node); synchronize_rcu(); list_add_rcu(&css->rstat_css_node, - &dcgrp->rstat_css_list); + &dcss->rstat_css_list); } /* default hierarchy doesn't enable controllers by default */ @@ -2052,7 +2053,6 @@ static void init_cgroup_housekeeping(struct cgroup *cgrp) cgrp->dom_cgrp = cgrp; cgrp->max_descendants = INT_MAX; cgrp->max_depth = INT_MAX; - INIT_LIST_HEAD(&cgrp->rstat_css_list); prev_cputime_init(&cgrp->prev_cputime); for_each_subsys(ss, ssid) @@ -2088,7 +2088,8 @@ int cgroup_setup_root(struct cgroup_root *root, u16 ss_mask) struct cgroup *root_cgrp = &root->cgrp; struct kernfs_syscall_ops *kf_sops; struct css_set *cset; - int i, ret; + struct cgroup_subsys *ss; + int i, ret, ssid; lockdep_assert_held(&cgroup_mutex); @@ -2132,10 +2133,6 @@ int cgroup_setup_root(struct cgroup_root *root, u16 ss_mask) if (ret) goto destroy_root; - ret = cgroup_rstat_init(&root_cgrp->self); - if (ret) - goto destroy_root; - ret = rebind_subsystems(root, ss_mask); if (ret) goto exit_stats; @@ -2174,7 +2171,10 @@ int cgroup_setup_root(struct cgroup_root *root, u16 ss_mask) goto out; exit_stats: - cgroup_rstat_exit(&root_cgrp->self); + for_each_subsys(ss, ssid) { + struct cgroup_subsys_state *css = init_css_set.subsys[ssid]; + cgroup_rstat_exit(css); + } destroy_root: kernfs_destroy_root(root->kf_root); root->kf_root = NULL; @@ -3229,6 +3229,10 @@ static int cgroup_apply_control_enable(struct cgroup *cgrp) int ssid, ret; cgroup_for_each_live_descendant_pre(dsct, d_css, cgrp) { + ret = cgroup_rstat_init(&dsct->self); + if (ret) + return ret; + for_each_subsys(ss, ssid) { struct cgroup_subsys_state *css = cgroup_css(dsct, ss); @@ -3239,6 +3243,10 @@ static int cgroup_apply_control_enable(struct cgroup *cgrp) css = css_create(dsct, ss); if (IS_ERR(css)) return PTR_ERR(css); + + ret = cgroup_rstat_init(css); + if (ret) + goto err_free_css; } WARN_ON_ONCE(percpu_ref_is_dying(&css->refcnt)); @@ -3252,6 +3260,20 @@ static int cgroup_apply_control_enable(struct cgroup *cgrp) } return 0; + +err_free_css: + cgroup_for_each_live_descendant_pre(dsct, d_css, cgrp) { + cgroup_rstat_exit(&dsct->self); + + for_each_subsys(ss, ssid) { + struct cgroup_subsys_state *css = cgroup_css(dsct, ss); + + if (css != &dsct->self) + cgroup_rstat_exit(css); + } + } + + return ret; } /** @@ -5403,6 +5425,7 @@ static void css_free_rwork_fn(struct work_struct *work) struct cgroup_subsys_state, destroy_rwork); struct cgroup_subsys *ss = css->ss; struct cgroup *cgrp = css->cgroup; + int ssid; percpu_ref_exit(&css->refcnt); @@ -5435,7 +5458,12 @@ static void css_free_rwork_fn(struct work_struct *work) cgroup_put(cgroup_parent(cgrp)); kernfs_put(cgrp->kn); psi_cgroup_free(cgrp); - cgroup_rstat_exit(css); + for_each_subsys(ss, ssid) { + struct cgroup_subsys_state *css = cgrp->subsys[ssid]; + + if (css) + cgroup_rstat_exit(css); + } kfree(cgrp); } else { /* @@ -5541,6 +5569,7 @@ static void init_and_link_css(struct cgroup_subsys_state *css, css->id = -1; INIT_LIST_HEAD(&css->sibling); INIT_LIST_HEAD(&css->children); + INIT_LIST_HEAD(&css->rstat_css_list); INIT_LIST_HEAD(&css->rstat_css_node); css->serial_nr = css_serial_nr_next++; atomic_set(&css->online_cnt, 0); @@ -5551,7 +5580,7 @@ static void init_and_link_css(struct cgroup_subsys_state *css, } if (ss->css_rstat_flush) - list_add_rcu(&css->rstat_css_node, &cgrp->rstat_css_list); + list_add_rcu(&css->rstat_css_node, &css->rstat_css_list); BUG_ON(cgroup_css(cgrp, ss)); } @@ -5686,14 +5715,6 @@ static struct cgroup *cgroup_create(struct cgroup *parent, const char *name, if (ret) goto out_free_cgrp; - /* init self cgroup early so css->cgroup is valid within cgroup_rstat_init() - * note that this will go away in a subsequent patch in this series - */ - cgrp->self.cgroup = cgrp; - ret = cgroup_rstat_init(&cgrp->self); - if (ret) - goto out_cancel_ref; - /* create the directory */ kn = kernfs_create_dir_ns(parent->kn, name, mode, current_fsuid(), current_fsgid(), @@ -5784,7 +5805,6 @@ static struct cgroup *cgroup_create(struct cgroup *parent, const char *name, kernfs_remove(cgrp->kn); out_stat_exit: cgroup_rstat_exit(&cgrp->self); -out_cancel_ref: percpu_ref_exit(&cgrp->self.refcnt); out_free_cgrp: kfree(cgrp); @@ -6189,6 +6209,8 @@ int __init cgroup_init(void) cgroup_unlock(); for_each_subsys(ss, ssid) { + struct cgroup_subsys_state *css; + if (ss->early_init) { struct cgroup_subsys_state *css = init_css_set.subsys[ss->id]; @@ -6200,6 +6222,9 @@ int __init cgroup_init(void) cgroup_init_subsys(ss, false); } + css = init_css_set.subsys[ss->id]; + BUG_ON(cgroup_rstat_init(css)); + list_add_tail(&init_css_set.e_cset_node[ssid], &cgrp_dfl_root.cgrp.e_csets[ssid]); diff --git a/kernel/cgroup/rstat.c b/kernel/cgroup/rstat.c index 01a5c185b02a..4381eb9ac426 100644 --- a/kernel/cgroup/rstat.c +++ b/kernel/cgroup/rstat.c @@ -14,9 +14,10 @@ static DEFINE_PER_CPU(raw_spinlock_t, cgroup_rstat_cpu_lock); static void cgroup_base_stat_flush(struct cgroup *cgrp, int cpu); -static struct cgroup_rstat_cpu *cgroup_rstat_cpu(struct cgroup *cgrp, int cpu) +static struct cgroup_rstat_cpu *css_rstat_cpu( + struct cgroup_subsys_state *css, int cpu) { - return per_cpu_ptr(cgrp->rstat_cpu, cpu); + return per_cpu_ptr(css->rstat_cpu, cpu); } /* @@ -96,15 +97,16 @@ __bpf_kfunc void cgroup_rstat_updated(struct cgroup_subsys_state *css, int cpu) * instead of NULL, we can tell whether @cgrp is on the list by * testing the next pointer for NULL. */ - if (data_race(cgroup_rstat_cpu(cgrp, cpu)->updated_next)) + if (data_race(css_rstat_cpu(css, cpu)->updated_next)) return; flags = _cgroup_rstat_cpu_lock(cpu_lock, cpu, cgrp, true); /* put @cgrp and all ancestors on the corresponding updated lists */ while (true) { - struct cgroup_rstat_cpu *rstatc = cgroup_rstat_cpu(cgrp, cpu); - struct cgroup *parent = cgroup_parent(cgrp); + struct cgroup_rstat_cpu *rstatc = css_rstat_cpu(css, cpu); + struct cgroup_subsys_state *parent = css->parent +; struct cgroup_rstat_cpu *prstatc; /* @@ -116,15 +118,15 @@ __bpf_kfunc void cgroup_rstat_updated(struct cgroup_subsys_state *css, int cpu) /* Root has no parent to link it to, but mark it busy */ if (!parent) { - rstatc->updated_next = cgrp; + rstatc->updated_next = css; break; } - prstatc = cgroup_rstat_cpu(parent, cpu); + prstatc = css_rstat_cpu(parent, cpu); rstatc->updated_next = prstatc->updated_children; - prstatc->updated_children = cgrp; + prstatc->updated_children = css; - cgrp = parent; + css = parent; } _cgroup_rstat_cpu_unlock(cpu_lock, cpu, cgrp, flags, true); @@ -142,12 +144,13 @@ __bpf_kfunc void cgroup_rstat_updated(struct cgroup_subsys_state *css, int cpu) * into a singly linked list built from the tail backward like "pushing" * cgroups into a stack. The root is pushed by the caller. */ -static struct cgroup *cgroup_rstat_push_children(struct cgroup *head, - struct cgroup *child, int cpu) +static struct cgroup_subsys_state *cgroup_rstat_push_children( + struct cgroup_subsys_state *head, + struct cgroup_subsys_state *child, int cpu) { - struct cgroup *chead = child; /* Head of child cgroup level */ - struct cgroup *ghead = NULL; /* Head of grandchild cgroup level */ - struct cgroup *parent, *grandchild; + struct cgroup_subsys_state *chead = child; /* Head of child cgroup level */ + struct cgroup_subsys_state *ghead = NULL; /* Head of grandchild cgroup level */ + struct cgroup_subsys_state *parent, *grandchild; struct cgroup_rstat_cpu *crstatc; child->rstat_flush_next = NULL; @@ -156,13 +159,13 @@ static struct cgroup *cgroup_rstat_push_children(struct cgroup *head, while (chead) { child = chead; chead = child->rstat_flush_next; - parent = cgroup_parent(child); + parent = child->parent; /* updated_next is parent cgroup terminated */ while (child != parent) { child->rstat_flush_next = head; head = child; - crstatc = cgroup_rstat_cpu(child, cpu); + crstatc = css_rstat_cpu(child, cpu); grandchild = crstatc->updated_children; if (grandchild != child) { /* Push the grand child to the next level */ @@ -201,16 +204,15 @@ static struct cgroup *cgroup_rstat_push_children(struct cgroup *head, * within the children list and terminated by the parent cgroup. An exception * here is the cgroup root whose updated_next can be self terminated. */ -static struct cgroup *cgroup_rstat_updated_list(struct cgroup_subsys_state *root_css, - int cpu) +static struct cgroup_subsys_state *cgroup_rstat_updated_list( + struct cgroup_subsys_state *root, int cpu) { - struct cgroup *root = root_css->cgroup; raw_spinlock_t *cpu_lock = per_cpu_ptr(&cgroup_rstat_cpu_lock, cpu); - struct cgroup_rstat_cpu *rstatc = cgroup_rstat_cpu(root, cpu); - struct cgroup *head = NULL, *parent, *child; + struct cgroup_rstat_cpu *rstatc = css_rstat_cpu(root, cpu); + struct cgroup_subsys_state *head = NULL, *parent, *child; unsigned long flags; - flags = _cgroup_rstat_cpu_lock(cpu_lock, cpu, root, false); + flags = _cgroup_rstat_cpu_lock(cpu_lock, cpu, root->cgroup, false); /* Return NULL if this subtree is not on-list */ if (!rstatc->updated_next) @@ -220,17 +222,17 @@ static struct cgroup *cgroup_rstat_updated_list(struct cgroup_subsys_state *root * Unlink @root from its parent. As the updated_children list is * singly linked, we have to walk it to find the removal point. */ - parent = cgroup_parent(root); + parent = root->parent; if (parent) { struct cgroup_rstat_cpu *prstatc; - struct cgroup **nextp; + struct cgroup_subsys_state **nextp; - prstatc = cgroup_rstat_cpu(parent, cpu); + prstatc = css_rstat_cpu(parent, cpu); nextp = &prstatc->updated_children; while (*nextp != root) { struct cgroup_rstat_cpu *nrstatc; - nrstatc = cgroup_rstat_cpu(*nextp, cpu); + nrstatc = css_rstat_cpu(*nextp, cpu); WARN_ON_ONCE(*nextp == parent); nextp = &nrstatc->updated_next; } @@ -247,7 +249,7 @@ static struct cgroup *cgroup_rstat_updated_list(struct cgroup_subsys_state *root if (child != root) head = cgroup_rstat_push_children(head, child, cpu); unlock_ret: - _cgroup_rstat_cpu_unlock(cpu_lock, cpu, root, flags, false); + _cgroup_rstat_cpu_unlock(cpu_lock, cpu, root->cgroup, flags, false); return head; } @@ -316,13 +318,13 @@ static void cgroup_rstat_flush_locked(struct cgroup_subsys_state *css) lockdep_assert_held(&cgroup_rstat_lock); for_each_possible_cpu(cpu) { - struct cgroup *pos = cgroup_rstat_updated_list(css, cpu); + struct cgroup_subsys_state *pos = cgroup_rstat_updated_list(css, cpu); for (; pos; pos = pos->rstat_flush_next) { struct cgroup_subsys_state *css_iter; - cgroup_base_stat_flush(pos, cpu); - bpf_rstat_flush(pos, cgroup_parent(pos), cpu); + cgroup_base_stat_flush(pos->cgroup, cpu); + bpf_rstat_flush(pos->cgroup, cgroup_parent(pos->cgroup), cpu); rcu_read_lock(); list_for_each_entry_rcu(css_iter, &pos->rstat_css_list, @@ -392,21 +394,20 @@ void cgroup_rstat_flush_release(struct cgroup_subsys_state *css) int cgroup_rstat_init(struct cgroup_subsys_state *css) { - struct cgroup *cgrp = css->cgroup; int cpu; - /* the root cgrp has rstat_cpu preallocated */ - if (!cgrp->rstat_cpu) { - cgrp->rstat_cpu = alloc_percpu(struct cgroup_rstat_cpu); - if (!cgrp->rstat_cpu) + /* the root cgrp css has rstat_cpu preallocated */ + if (!css->rstat_cpu) { + css->rstat_cpu = alloc_percpu(struct cgroup_rstat_cpu); + if (!css->rstat_cpu) return -ENOMEM; } /* ->updated_children list is self terminated */ for_each_possible_cpu(cpu) { - struct cgroup_rstat_cpu *rstatc = cgroup_rstat_cpu(cgrp, cpu); + struct cgroup_rstat_cpu *rstatc = css_rstat_cpu(css, cpu); - rstatc->updated_children = cgrp; + rstatc->updated_children = css; u64_stats_init(&rstatc->bsync); } @@ -415,22 +416,21 @@ int cgroup_rstat_init(struct cgroup_subsys_state *css) void cgroup_rstat_exit(struct cgroup_subsys_state *css) { - struct cgroup *cgrp = css->cgroup; int cpu; - cgroup_rstat_flush(&cgrp->self); + cgroup_rstat_flush(css); /* sanity check */ for_each_possible_cpu(cpu) { - struct cgroup_rstat_cpu *rstatc = cgroup_rstat_cpu(cgrp, cpu); + struct cgroup_rstat_cpu *rstatc = css_rstat_cpu(css, cpu); - if (WARN_ON_ONCE(rstatc->updated_children != cgrp) || + if (WARN_ON_ONCE(rstatc->updated_children != css) || WARN_ON_ONCE(rstatc->updated_next)) return; } - free_percpu(cgrp->rstat_cpu); - cgrp->rstat_cpu = NULL; + free_percpu(css->rstat_cpu); + css->rstat_cpu = NULL; } void __init cgroup_rstat_boot(void) @@ -471,7 +471,7 @@ static void cgroup_base_stat_sub(struct cgroup_base_stat *dst_bstat, static void cgroup_base_stat_flush(struct cgroup *cgrp, int cpu) { - struct cgroup_rstat_cpu *rstatc = cgroup_rstat_cpu(cgrp, cpu); + struct cgroup_rstat_cpu *rstatc = css_rstat_cpu(&cgrp->self, cpu); struct cgroup *parent = cgroup_parent(cgrp); struct cgroup_rstat_cpu *prstatc; struct cgroup_base_stat delta; @@ -501,7 +501,7 @@ static void cgroup_base_stat_flush(struct cgroup *cgrp, int cpu) cgroup_base_stat_add(&cgrp->last_bstat, &delta); delta = rstatc->subtree_bstat; - prstatc = cgroup_rstat_cpu(parent, cpu); + prstatc = css_rstat_cpu(&parent->self, cpu); cgroup_base_stat_sub(&delta, &rstatc->last_subtree_bstat); cgroup_base_stat_add(&prstatc->subtree_bstat, &delta); cgroup_base_stat_add(&rstatc->last_subtree_bstat, &delta); @@ -513,7 +513,7 @@ cgroup_base_stat_cputime_account_begin(struct cgroup *cgrp, unsigned long *flags { struct cgroup_rstat_cpu *rstatc; - rstatc = get_cpu_ptr(cgrp->rstat_cpu); + rstatc = get_cpu_ptr(cgrp->self.rstat_cpu); *flags = u64_stats_update_begin_irqsave(&rstatc->bsync); return rstatc; } From patchwork Fri Jan 3 01:50:16 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: JP Kobryn X-Patchwork-Id: 13925087 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BA8D9E77198 for ; Fri, 3 Jan 2025 01:50:47 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 83DD06B008A; Thu, 2 Jan 2025 20:50:41 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 7C7C16B0092; Thu, 2 Jan 2025 20:50:41 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6188A6B0093; Thu, 2 Jan 2025 20:50:41 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 3A6516B008A for ; Thu, 2 Jan 2025 20:50:41 -0500 (EST) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id DE6A5140108 for ; Fri, 3 Jan 2025 01:50:40 +0000 (UTC) X-FDA: 82964459466.06.1C88C66 Received: from mail-pl1-f170.google.com (mail-pl1-f170.google.com [209.85.214.170]) by imf30.hostedemail.com (Postfix) with ESMTP id 1A0BC80007 for ; Fri, 3 Jan 2025 01:49:00 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=ZashkFF0; spf=pass (imf30.hostedemail.com: domain of inwardvessel@gmail.com designates 209.85.214.170 as permitted sender) smtp.mailfrom=inwardvessel@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1735869000; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=06Rq6Yp6bkyCHegIVI2ycFM+KAFm/5mAM9hGlEHX3pE=; b=PHSCs3gjdhQzE4V35NQ2OmDIvO9U8tJPwUA9cMj74VIncvPohcGyXfLVWpczSioMCI/4kE AlSmyDZLhbIEFf6D3sqihVyNoe076CZK14myrOC6B/I9n93wyCmrjZ4SCRLI4oEuAok8RJ xhDhkxj3/gqKesWvpbT+mQ8PADAWvnY= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=ZashkFF0; spf=pass (imf30.hostedemail.com: domain of inwardvessel@gmail.com designates 209.85.214.170 as permitted sender) smtp.mailfrom=inwardvessel@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1735869000; a=rsa-sha256; cv=none; b=CJmoe48RQN0RmxbHd9Nhub3qYc1JvPSR9oXkxnabH7XrIK6LFfIq+zm13i7j8zbggfWclo uFvOf4tADxZcNCrAvvCTZKETd9DKKws1YJJRCAptBwAi/SKAk4FmM4EWIWPcwKU4P4gxwD i/EjcUdW55LM7uWk/0OlcTSLpEMzfxs= Received: by mail-pl1-f170.google.com with SMTP id d9443c01a7336-21619108a6bso151367345ad.3 for ; Thu, 02 Jan 2025 17:50:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1735869038; x=1736473838; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=06Rq6Yp6bkyCHegIVI2ycFM+KAFm/5mAM9hGlEHX3pE=; b=ZashkFF0FtmGZnc/XZzsXMDE+f7JAOyqOhkHhTjf3yZjXMb2WzOoXjJmErPkDMEHik X7T94oHCyr+XFwjsMcQSGXC7xz04I7gO/QOjxS3oglD986R46fg8lEKp9mA7DYlTALwK S1CePxVKm1LGtI0pD6yMY6gfGuGHAP3+p6ZPQ3yBXBnssN/0z1xK/FyKG6LBOwOJDiNo 7HtCY/6CfqlGdB10JfKbzIxwVAgyVefxSvltPuITfrrGj/qTDnnrygvlXw7YMJllxXFC +Xfk0XVxll72aiKag8MywliH2OZEkm2ciKpF7nBXZnStvq9gHJuAN1ziMx5dfUQZRRot 3ThA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1735869038; x=1736473838; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=06Rq6Yp6bkyCHegIVI2ycFM+KAFm/5mAM9hGlEHX3pE=; b=HMFZ+Bv4G6q6o1RACz+AfPc5tsKYNmlBsVtp2a8U0J+nZIQdhhInzGyhWUtvaODC0F lRGUwS9eSuwO9NVBLReyoxZwD2sgmNhkqlho3/jvbMgTx0x/e2lnEFbx315TA/SWYYCT wVt+1lpDGupBm9HNo989zE1JVE9IXMll1UCiG03XhQo3Q/hk2ehbmX6iioiJfk0gMJcP wq/4aUDlWK5HZ6+aLvle19Gs5VP0ISlFkp0+TdsHCqRJ0xvGSTQ3jhWkcjte3cxKWQVE 2Je7E06PaRwhOxu7Kh+xWPYiCB0mcKVjcG8SiX+ZkNK2QZNRZn48f4WtRmNY3GR1+G5O Cfyg== X-Gm-Message-State: AOJu0YzljuFo6UwPyryTxvtUZT0uFIi+tfW8saXcB4cbLNTSLiThC4TB 8ywbNaF1a6iPRdqo+lDwmmgtzeX7kMuGM/RA6ULSGJ6AaixNbq1L X-Gm-Gg: ASbGnct11lkQjq+JrHB5Y26tPP6wtMf4pgIUuAE3itDUKhJLtJJBhCVvntwJT/sLu4Q vyef3Qsyg7HAehe+xVsvF1WIDMwzQBQ3hn1dlH/DtqIUiYkSQE3EEOW44aYPDPqN1zsjvlPEQlm FQnrmS+NF4ym/1PuQXNW1GvjDHv1ThGoX/BASgpOO7mZtb8cj0XDquJW55joKAtiSxndqBrR+c/ bZargueAX5sl+kh1ITHB7f6n04rSOheoyk9c2RLo8WLddmWsOw2wlEJAu9AnLKzvHU+iN5qzu+M ne9kTTvju9w65K7ZjA== X-Google-Smtp-Source: AGHT+IGrZJLbbkVIwX5AQ1w7ifn+2yxZj+rAsWC13phN3JNek86G54n2zYjuYiKjyl8m0KsBcHLbEg== X-Received: by 2002:a17:903:230c:b0:216:1543:195d with SMTP id d9443c01a7336-219e6eb3a5dmr635218695ad.25.1735869037897; Thu, 02 Jan 2025 17:50:37 -0800 (PST) Received: from saturn.. (c-67-188-127-15.hsd1.ca.comcast.net. [67.188.127.15]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-219dca04ce7sm228851505ad.283.2025.01.02.17.50.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 02 Jan 2025 17:50:37 -0800 (PST) From: JP Kobryn To: shakeel.butt@linux.dev, tj@kernel.org, mhocko@kernel.org, hannes@cmpxchg.org, yosryahmed@google.com, akpm@linux-foundation.org Cc: linux-mm@kvack.org, cgroups@vger.kernel.org Subject: [RFC PATCH 5/9 v2] cgroup: separate locking between base css and others Date: Thu, 2 Jan 2025 17:50:16 -0800 Message-ID: <20250103015020.78547-6-inwardvessel@gmail.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250103015020.78547-1-inwardvessel@gmail.com> References: <20250103015020.78547-1-inwardvessel@gmail.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 1A0BC80007 X-Rspamd-Server: rspam12 X-Stat-Signature: 7cnhg3jzb9983eabbnzgkosaijm5uhtx X-Rspam-User: X-HE-Tag: 1735868940-909535 X-HE-Meta: U2FsdGVkX19fenkl/4YKy+meFwxcDDYzRgMpWd+Ug6ZDVEiKXNoQjfADj0LFzTV4Tm/X+N62eRa9jU/jqtLQMujxZZNjOjAckRuWcpLj6WWeo17k/el5c65XaWcsPdDnPXbrWwqgjpV1OP3nC+TVnIdryD9a03EV1NvgbwYsWjm1Xw93w5h2u5VYl1DR4qDlzqFMXYj8CMhOlZQJP6GkBvHzEWYMHA3cZBqmWIv2yqovuEeU6DWogSZFFfjk3Q8F3jkdR2gX59UmnNONsmG3MfsdJqFyYTUXDwY8YTSyNfnd2bBCOlv+9eGdKrLj/57DC92Y9RY8LU4vnhql9lesGl7qw5OPWmi1rogXpSnbdMgkSgU22UDudmDiYO5+WaZzKijOh9UnFoyHkfk/LEl73owlH8Uot9lxiKutjypVZST1FMahzpchpC64pKg96BniJFGX+0npZ0BV40tUstSJ7Q84sIxnZnB7i8K8otu67FZj+JlQygm6qfmoqamy/AQpvcgCZXQ600HZUnNs7tEruJZPJ4khTT/t98v34dwi+v6AghmPm5FBIzImRfKcOyaHrRm3YHUOWJXTol+UkvhDqbiYL6hToQvkle2HGKUtug5Se0qpVI1illuPYQZFfS49vU0Rlr7xiwm0PBJwycBLdxqpl3seCFvy3WePe6k+zfvVpOQIvG8PJiAicqwa0ZTn1mw9UBUtMwUiKXXXUNgXYQV9nJISRwS16oYwV+D4UteRK3pmH+cgqC0qoO2iQjtHBkY2uhDZcyf73YepDFlOZbt4j7ss7LTdt440i4iqOQgivqsuWafAKYPYfTYjuJyDWsdLmFIVuNUFjcEvm1SHwzV7aIWtDzChzmNeaVdgkab4vWtbQoCet5B/OhOMDq/b3J3TIfUbhxriJwAGm03zHERFbfpf5HCzsMXvTp0ME9O0b0K4PXqn/gORYSzikUMcrsu9LufJFLHUORBAx++ h/3ZvNjd AhTuYL9sS6C+zrvipUMOUD9FgUG7LoLJ5Yfo5r4AtXYBImQyI4Eh4A28EU1/cGbwbADYvEs/ympP/sRehW2zTnTQE8QC4/aXYpGr6VcOccHZhk4arb6ligywqJDjCrthfdNwV4ZuLmewX0QJM31YzPHdoY12x+xGeOqzonoj0HOR4b3Q6MA58I2y7iVK71eGv1dGbetCi0ycCdx3iKMJyjO0cBV6aUi3GUe0YuKroZnK20kaBCwIP2pJt5AhSllwJ/ACz2Mr/zf9MkvaSQFUTGTN2qkY5wTq+4IOgLcgWzeB0GgdLTFDdVPuAdWISppYSntHRU5WiVM+87gHIGMfERo4ZvFvqp5x/VKGSEJBhfKJtkBhRsuI8KMbJ8g== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Separate locks can be used to eliminate contention between subsystems that make use of rstat. The base stats also get their own lock. Where applicable, check for the existence of a subsystem pointer to determine if the given cgroup_subsys_state is the base css or not for deciding which lock to take. Signed-off-by: JP Kobryn --- include/linux/cgroup-defs.h | 2 + kernel/cgroup/rstat.c | 92 +++++++++++++++++++++++++------------ 2 files changed, 65 insertions(+), 29 deletions(-) diff --git a/include/linux/cgroup-defs.h b/include/linux/cgroup-defs.h index 1932f8ae7995..4d87519ff023 100644 --- a/include/linux/cgroup-defs.h +++ b/include/linux/cgroup-defs.h @@ -790,6 +790,8 @@ struct cgroup_subsys { * specifies the mask of subsystems that this one depends on. */ unsigned int depends_on; + + spinlock_t rstat_lock; }; extern struct percpu_rw_semaphore cgroup_threadgroup_rwsem; diff --git a/kernel/cgroup/rstat.c b/kernel/cgroup/rstat.c index 4381eb9ac426..958bdccf0359 100644 --- a/kernel/cgroup/rstat.c +++ b/kernel/cgroup/rstat.c @@ -9,8 +9,9 @@ #include -static DEFINE_SPINLOCK(cgroup_rstat_lock); -static DEFINE_PER_CPU(raw_spinlock_t, cgroup_rstat_cpu_lock); +static DEFINE_SPINLOCK(cgroup_rstat_base_lock); +static DEFINE_PER_CPU(raw_spinlock_t, cgroup_rstat_base_cpu_lock); +static DEFINE_PER_CPU(raw_spinlock_t, cgroup_rstat_cpu_lock[CGROUP_SUBSYS_COUNT]); static void cgroup_base_stat_flush(struct cgroup *cgrp, int cpu); @@ -86,7 +87,7 @@ void _cgroup_rstat_cpu_unlock(raw_spinlock_t *cpu_lock, int cpu, __bpf_kfunc void cgroup_rstat_updated(struct cgroup_subsys_state *css, int cpu) { struct cgroup *cgrp = css->cgroup; - raw_spinlock_t *cpu_lock = per_cpu_ptr(&cgroup_rstat_cpu_lock, cpu); + raw_spinlock_t *cpu_lock; unsigned long flags; /* @@ -100,6 +101,11 @@ __bpf_kfunc void cgroup_rstat_updated(struct cgroup_subsys_state *css, int cpu) if (data_race(css_rstat_cpu(css, cpu)->updated_next)) return; + if (css->ss) + cpu_lock = per_cpu_ptr(&cgroup_rstat_cpu_lock[css->ss->id], cpu); + else + cpu_lock = per_cpu_ptr(&cgroup_rstat_base_cpu_lock, cpu); + flags = _cgroup_rstat_cpu_lock(cpu_lock, cpu, cgrp, true); /* put @cgrp and all ancestors on the corresponding updated lists */ @@ -207,11 +213,16 @@ static struct cgroup_subsys_state *cgroup_rstat_push_children( static struct cgroup_subsys_state *cgroup_rstat_updated_list( struct cgroup_subsys_state *root, int cpu) { - raw_spinlock_t *cpu_lock = per_cpu_ptr(&cgroup_rstat_cpu_lock, cpu); struct cgroup_rstat_cpu *rstatc = css_rstat_cpu(root, cpu); struct cgroup_subsys_state *head = NULL, *parent, *child; + raw_spinlock_t *cpu_lock; unsigned long flags; + if (root->ss) + cpu_lock = per_cpu_ptr(&cgroup_rstat_cpu_lock[root->ss->id], cpu); + else + cpu_lock = per_cpu_ptr(&cgroup_rstat_base_cpu_lock, cpu); + flags = _cgroup_rstat_cpu_lock(cpu_lock, cpu, root->cgroup, false); /* Return NULL if this subtree is not on-list */ @@ -285,37 +296,44 @@ __bpf_hook_end(); * number processed last. */ static inline void __cgroup_rstat_lock(struct cgroup_subsys_state *css, - int cpu_in_loop) - __acquires(&cgroup_rstat_lock) + spinlock_t *lock, int cpu_in_loop) + __acquires(lock) { struct cgroup *cgrp = css->cgroup; bool contended; - contended = !spin_trylock_irq(&cgroup_rstat_lock); + contended = !spin_trylock_irq(lock); if (contended) { trace_cgroup_rstat_lock_contended(cgrp, cpu_in_loop, contended); - spin_lock_irq(&cgroup_rstat_lock); + spin_lock_irq(lock); } trace_cgroup_rstat_locked(cgrp, cpu_in_loop, contended); } static inline void __cgroup_rstat_unlock(struct cgroup_subsys_state *css, - int cpu_in_loop) - __releases(&cgroup_rstat_lock) + spinlock_t *lock, int cpu_in_loop) + __releases(lock) { struct cgroup *cgrp = css->cgroup; trace_cgroup_rstat_unlock(cgrp, cpu_in_loop, false); - spin_unlock_irq(&cgroup_rstat_lock); + spin_unlock_irq(lock); } /* see cgroup_rstat_flush() */ static void cgroup_rstat_flush_locked(struct cgroup_subsys_state *css) - __releases(&cgroup_rstat_lock) __acquires(&cgroup_rstat_lock) + __releases(&css->ss->rstat_lock) __acquires(&css->ss->rstat_lock) { + spinlock_t *lock; int cpu; - lockdep_assert_held(&cgroup_rstat_lock); + if (!css->ss) { + pr_warn("cannot use generic flush on base subsystem\n"); + return; + } + + lock = &css->ss->rstat_lock; + lockdep_assert_held(lock); for_each_possible_cpu(cpu) { struct cgroup_subsys_state *pos = cgroup_rstat_updated_list(css, cpu); @@ -334,11 +352,11 @@ static void cgroup_rstat_flush_locked(struct cgroup_subsys_state *css) } /* play nice and yield if necessary */ - if (need_resched() || spin_needbreak(&cgroup_rstat_lock)) { - __cgroup_rstat_unlock(css, cpu); + if (need_resched() || spin_needbreak(lock)) { + __cgroup_rstat_unlock(css, lock, cpu); if (!cond_resched()) cpu_relax(); - __cgroup_rstat_lock(css, cpu); + __cgroup_rstat_lock(css, lock, cpu); } } } @@ -358,11 +376,22 @@ static void cgroup_rstat_flush_locked(struct cgroup_subsys_state *css) */ __bpf_kfunc void cgroup_rstat_flush(struct cgroup_subsys_state *css) { + spinlock_t *lock; + + if (!css->ss) { + int cpu; + + for_each_possible_cpu(cpu) + cgroup_base_stat_flush(css->cgroup, cpu); + return; + } + might_sleep(); - __cgroup_rstat_lock(css, -1); + lock = &css->ss->rstat_lock; + __cgroup_rstat_lock(css, lock, -1); cgroup_rstat_flush_locked(css); - __cgroup_rstat_unlock(css, -1); + __cgroup_rstat_unlock(css, lock, -1); } /** @@ -374,11 +403,11 @@ __bpf_kfunc void cgroup_rstat_flush(struct cgroup_subsys_state *css) * * This function may block. */ -void cgroup_rstat_flush_hold(struct cgroup_subsys_state *css) - __acquires(&cgroup_rstat_lock) +static void cgroup_rstat_base_flush_hold(struct cgroup_subsys_state *css) + __acquires(&cgroup_rstat_base_lock) { might_sleep(); - __cgroup_rstat_lock(css, -1); + __cgroup_rstat_lock(css, &cgroup_rstat_base_lock, -1); cgroup_rstat_flush_locked(css); } @@ -386,10 +415,10 @@ void cgroup_rstat_flush_hold(struct cgroup_subsys_state *css) * cgroup_rstat_flush_release - release cgroup_rstat_flush_hold() * @cgrp: cgroup used by tracepoint */ -void cgroup_rstat_flush_release(struct cgroup_subsys_state *css) - __releases(&cgroup_rstat_lock) +static void cgroup_rstat_base_flush_release(struct cgroup_subsys_state *css) + __releases(&cgroup_rstat_base_lock) { - __cgroup_rstat_unlock(css, -1); + __cgroup_rstat_unlock(css, &cgroup_rstat_base_lock, -1); } int cgroup_rstat_init(struct cgroup_subsys_state *css) @@ -435,10 +464,15 @@ void cgroup_rstat_exit(struct cgroup_subsys_state *css) void __init cgroup_rstat_boot(void) { - int cpu; + struct cgroup_subsys *ss; + int cpu, ssid; + + for_each_possible_cpu(cpu) { + raw_spin_lock_init(per_cpu_ptr(&cgroup_rstat_base_cpu_lock, cpu)); - for_each_possible_cpu(cpu) - raw_spin_lock_init(per_cpu_ptr(&cgroup_rstat_cpu_lock, cpu)); + for_each_subsys(ss, ssid) + raw_spin_lock_init(per_cpu_ptr(&cgroup_rstat_cpu_lock[ssid], cpu)); + } } /* @@ -629,12 +663,12 @@ void cgroup_base_stat_cputime_show(struct seq_file *seq) u64 usage, utime, stime, ntime; if (cgroup_parent(cgrp)) { - cgroup_rstat_flush_hold(css); + cgroup_rstat_base_flush_hold(css); usage = cgrp->bstat.cputime.sum_exec_runtime; cputime_adjust(&cgrp->bstat.cputime, &cgrp->prev_cputime, &utime, &stime); ntime = cgrp->bstat.ntime; - cgroup_rstat_flush_release(css); + cgroup_rstat_base_flush_release(css); } else { /* cgrp->bstat of root is not actually used, reuse it */ root_cgroup_cputime(&cgrp->bstat); From patchwork Fri Jan 3 01:50:17 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: JP Kobryn X-Patchwork-Id: 13925088 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6AA51E77197 for ; Fri, 3 Jan 2025 01:50:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B267F6B0092; Thu, 2 Jan 2025 20:50:42 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id AD8386B0093; Thu, 2 Jan 2025 20:50:42 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9265F6B0095; Thu, 2 Jan 2025 20:50:42 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 696EE6B0092 for ; Thu, 2 Jan 2025 20:50:42 -0500 (EST) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 344DD140227 for ; Fri, 3 Jan 2025 01:50:42 +0000 (UTC) X-FDA: 82964459130.19.F346A26 Received: from mail-pl1-f180.google.com (mail-pl1-f180.google.com [209.85.214.180]) by imf03.hostedemail.com (Postfix) with ESMTP id 2BE6820004 for ; Fri, 3 Jan 2025 01:50:17 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=TsjOGJkZ; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf03.hostedemail.com: domain of inwardvessel@gmail.com designates 209.85.214.180 as permitted sender) smtp.mailfrom=inwardvessel@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1735868992; a=rsa-sha256; cv=none; b=MHfdqNBtNhLcTWub2ImQu8q0Isje6CdE2wGVAYeDtfoRKZ7KzDBomhlMTxTP50PFnuqbfK zMu7aE0xUqHvp+bYrzafStrB41BIUf2km2vmCmB9oRofAutj9rCOLdO4jEXjk0Qp+znuMG gofgMQroxlDCCZzS0uKCkrKFnNUyZaQ= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=TsjOGJkZ; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf03.hostedemail.com: domain of inwardvessel@gmail.com designates 209.85.214.180 as permitted sender) smtp.mailfrom=inwardvessel@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1735868992; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Ov79tpw4GuAmAWp+brDBEww96U26TW/4D2kYy+uPe/w=; b=nf2FFJQYDlpZ583C3nSxd7LAcNPieUOkvWG4m0G9Q7oe3YzxoImSddL6N7diGad4bugu0Y Rl6zz96IDcgKA0qfwjOv7dJXmSoZwOLcS8lHpkCKBAypiOui5w4fp2KSNkIqkYggINebpk V0PiVHdgNutajQjpbHPD8gfwN5HQ9rU= Received: by mail-pl1-f180.google.com with SMTP id d9443c01a7336-2165448243fso196304565ad.1 for ; Thu, 02 Jan 2025 17:50:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1735869039; x=1736473839; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Ov79tpw4GuAmAWp+brDBEww96U26TW/4D2kYy+uPe/w=; b=TsjOGJkZG+ZH7k3AGNlOssTL+Pa9eQ7DHdDD6THN0XIL7CicgRBYQPxgpbUZtJ6T6p wEsLYlfCl59xSJlSLRoQeVbnujZG9qUWR3cRQ/rqdHjfQOMvLXTCfLewxOtYNe6CwFhQ xwsUW/ZCCeNQpthuarusKI1Cbddxghi60VUFv29qR5vIizL7HsV6vxGAPPV2uwU4GIYB r5rwPJszppCgMqQRBYl8KLIR9AEWrDrPhgiEDr7d9z5MPfpxVmEIq1nJFxAXuHGxrSUx CxiFW3pQsHbb2JQOtJhwVdOlJ/GtkdYOLcHyZLyZODHglJR+1TkdRNM0+/sFx8gq9+h2 eVhQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1735869039; x=1736473839; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Ov79tpw4GuAmAWp+brDBEww96U26TW/4D2kYy+uPe/w=; b=G1MOYWScshmFFDl1SGwVe1IqgRCRte64CCUv/wctmjBpe0nUBS2lp8L41R7uwLzWz/ 8nHdA8eAIlLqrKLQdgNZagOTbsqGhU6tlWMrBamwbcfbJ3Wqi7s/rRH8uOGfosau2lCN 13XiZEstzPaBpqMO2wXHVRIeQGFc4JdZaTEhaDTkJv91t0DS60VQEkt/Z9nOY0J23fmd o649tEYRdF1gJQOfehrQQzkmrkLPv6So4Exsj336HsskyvzEl3prtvSaiepz+VVV6bFr Ai+8Otna+mUHaa3MVrXegCCkhtjVocOIZI1PyAH4pRUN6q1LWHODc4u4Vvj3slKVjrEl Vkfg== X-Gm-Message-State: AOJu0Yxjaq41DH22w/f5z+9MfkmfeK5djmHzh/1TAiyMXYbY+1rSxUm+ vCKI8jBvwc+viRJuNJM8zYTNNgTA14W0xAUhADvV69pDr9PIn2Tb X-Gm-Gg: ASbGnctHtPBJ/5lbM5JG62Wyic2k3pqqVuvSKmC+vPxo7P24YdYAXBJH2vzIug8oa6T JcolYM/IsgbySE9DSMnCVLM8OYRulp8p2SH80Aeb3auoAGbixX23JrcrLQ+/1zoSZ5HL2NmrISO X56Wci0RZ8ynv7LAgopydzOXpUWJrtVHyHY/X7RmnvHF0GCWeqSoppzPmrZpmvuVtbMvQujQFlo tMITDCsAkoQhSUcbTROHfgKl5N8psStC8POPf2pQFe2yShsJgO5hMGSUW1UK60zBuWS3A8SN4dZ dwO2uNCKZoJYvwZDOw== X-Google-Smtp-Source: AGHT+IHLu3wo59GI10fAuqab6yxqPVBVuB+WZNMyKnuCfl4ZNRDH7NjVsB8okoZW+Ytc6M8DT1bsCQ== X-Received: by 2002:a17:903:2449:b0:215:3a42:dc17 with SMTP id d9443c01a7336-219e6e8bbabmr593714305ad.7.1735869039266; Thu, 02 Jan 2025 17:50:39 -0800 (PST) Received: from saturn.. (c-67-188-127-15.hsd1.ca.comcast.net. [67.188.127.15]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-219dca04ce7sm228851505ad.283.2025.01.02.17.50.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 02 Jan 2025 17:50:38 -0800 (PST) From: JP Kobryn To: shakeel.butt@linux.dev, tj@kernel.org, mhocko@kernel.org, hannes@cmpxchg.org, yosryahmed@google.com, akpm@linux-foundation.org Cc: linux-mm@kvack.org, cgroups@vger.kernel.org Subject: [RFC PATCH 6/9 v2] cgroup: isolate base stat flush Date: Thu, 2 Jan 2025 17:50:17 -0800 Message-ID: <20250103015020.78547-7-inwardvessel@gmail.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250103015020.78547-1-inwardvessel@gmail.com> References: <20250103015020.78547-1-inwardvessel@gmail.com> MIME-Version: 1.0 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 2BE6820004 X-Stat-Signature: ewb4q8aepwpd8ggr61kokhw31uafmt81 X-Rspam-User: X-HE-Tag: 1735869017-106112 X-HE-Meta: U2FsdGVkX19CG2FG31uhYePXCnVwCUB+Solia+CZaN7D+Iudw6cjQNNNZMX/6GEeEE+gp8zgw+/m4mOrqyGEMI1N2AEg6qjzxmhHgmfurFTpR3w1cfJNbpdaCgQmGflAQbDmCO6uD4erS5H4XJVsne3v1Djv9P8x/3YuT0xWnGPSuQREwohOwwdiiMSljqRa2qZN9AKDnZ/Qbw+Wm4YlQUPY+A/J/zjP17y5m6rOuK4veA6AEGlw0ImSv3unZnLJ2DhJmbdefUz1rcCr5tXuAZ98T7pMVWUDzO+MzDpU9l9SKdD6LyLvzIHxpkt7szik6Tb3iCt9GA3wM7+r2bUhgCVnb7FTT5Fz5rgVgM2284GK4Z6zyLE+BEfy03rp+2EQ4cIY2CyBpauckft0gkkIej8mYsFtwtzNiOCC3HJKyj0MRZ1CbP7ZwNxIl6nClyGuTz2YAww/Kg1mY56JyMELg32mD4EKAU5EVfgDlFNlDSqHI+MnbVK1hzdFsRHwQxb27tixJ6Jj7Wo6i0NUBgSJKPChbqRsvfUSFRFtp44cvHSB/ygJ2xQfTJH/SEu/zek9AtNNPxt8zMjFqY6bs+vYmcAeOBO8GntcBXu68Bqq6z+YvfP05+ecjV+UdjvufnO2kx6ftejDfrggwbKj0SiX1v51Nqb4mJhins9uFEt11Uxp/KqN9bRiNe1H3DOUt5n8OUVAd0t9aRqgogzDllx4LM90MJQoRr6IO4r23ZDKHgs1vCpa+kUu16EI9FavQEFS2rC7IeHjBy9FTMnpAmwx+/3H/wqVugp7qWa+4N5ck8AvzBx8OVxqBv+TTIRkusK444qLJFUZw3PSqyJMytKp5ICpoj1QIHFSmaXZbyUCwL8z/ZC29LzjEVioTwcRX0vN7ih8dIr1Yxs16ucRezV+7fyH1G2O9JSqg4bQx1mzt84qrc5fSWfJ3XF9PvVSTSGwjEFlj7pZz9nWvpDKiBr 4m6lJvwD cqnrGHhjSOdrj4JmEY/cjNeDQEiVRNdYFQ8uFUTRmBWcC8bnGAQDyqqJ+vjYnHXJnq7SJm2LeseOLskw582H2oVEzkisCebpzULWYpXVVMDpf3LH5tQq1vXv7W69mk5mE3L/WregrkjZ6OZsLHmFh2ugUbu1Li/BtS6Z4jI7IxREE2wtnHjJr5i+GR7GG2e8GzzMgrzTKzjiZM70l9Cyvk4NFRYCaauOntEfdt6cu6EEW0HjSL4prj/tWdKKVa25ogbO3hiOmiqDsZOs9nynOQ8qITehPLpfrRtSJXy1TYxP0+INVKb+bzs0cfpkVwHARqbm5bjgeY6rE11192EmsLIP0sUO75nFWG5QGQty1MzecFIFXt2OwwwcLCQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000007, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Remove the base stat flushing from the generic css flushing routine. Provide a direct way to flush the base stats and make use of it when the cpu stats are read. Signed-off-by: JP Kobryn --- kernel/cgroup/rstat.c | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/kernel/cgroup/rstat.c b/kernel/cgroup/rstat.c index 958bdccf0359..92a46b960be1 100644 --- a/kernel/cgroup/rstat.c +++ b/kernel/cgroup/rstat.c @@ -320,6 +320,14 @@ static inline void __cgroup_rstat_unlock(struct cgroup_subsys_state *css, spin_unlock_irq(lock); } +static void cgroup_rstat_base_flush_locked(struct cgroup *cgrp) +{ + int cpu; + + for_each_possible_cpu(cpu) + cgroup_base_stat_flush(cgrp, cpu); +} + /* see cgroup_rstat_flush() */ static void cgroup_rstat_flush_locked(struct cgroup_subsys_state *css) __releases(&css->ss->rstat_lock) __acquires(&css->ss->rstat_lock) @@ -341,7 +349,6 @@ static void cgroup_rstat_flush_locked(struct cgroup_subsys_state *css) for (; pos; pos = pos->rstat_flush_next) { struct cgroup_subsys_state *css_iter; - cgroup_base_stat_flush(pos->cgroup, cpu); bpf_rstat_flush(pos->cgroup, cgroup_parent(pos->cgroup), cpu); rcu_read_lock(); @@ -408,7 +415,7 @@ static void cgroup_rstat_base_flush_hold(struct cgroup_subsys_state *css) { might_sleep(); __cgroup_rstat_lock(css, &cgroup_rstat_base_lock, -1); - cgroup_rstat_flush_locked(css); + cgroup_rstat_base_flush_locked(css->cgroup); } /** From patchwork Fri Jan 3 01:50:18 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: JP Kobryn X-Patchwork-Id: 13925089 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E6837E77197 for ; Fri, 3 Jan 2025 01:50:52 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 50BC46B0095; Thu, 2 Jan 2025 20:50:44 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 4B9256B0096; Thu, 2 Jan 2025 20:50:44 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 30EAA6B0098; Thu, 2 Jan 2025 20:50:44 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 0F4A96B0095 for ; Thu, 2 Jan 2025 20:50:44 -0500 (EST) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id BB57D1A0153 for ; Fri, 3 Jan 2025 01:50:43 +0000 (UTC) X-FDA: 82964459970.23.C3931CE Received: from mail-pl1-f179.google.com (mail-pl1-f179.google.com [209.85.214.179]) by imf17.hostedemail.com (Postfix) with ESMTP id 0B46E40015 for ; Fri, 3 Jan 2025 01:50:02 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=EdRf91tz; spf=pass (imf17.hostedemail.com: domain of inwardvessel@gmail.com designates 209.85.214.179 as permitted sender) smtp.mailfrom=inwardvessel@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1735868989; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=z0/95vxHq4AbTjjm1qqTnNQIEsjcgRpvj31jjZiEdxU=; b=c6jGSRQXRupSNZ0fBaTF4x1s8m8j4MJwQ8qx4Y1OyY/fNWqnrIaQ7TkcaKxrSo2K63gfeA 7tRs+dmVawBM1Ak/lAln0gbsv568UTDD3F6X/8BTex7K2YJoa0tvElYgQry8QaZqV48FOu V/s+9bvqfCy2MRaKxqB4KQ2qH9YUO9A= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=EdRf91tz; spf=pass (imf17.hostedemail.com: domain of inwardvessel@gmail.com designates 209.85.214.179 as permitted sender) smtp.mailfrom=inwardvessel@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1735868989; a=rsa-sha256; cv=none; b=I5u/veVFjalsMrkKaCKZyz7kyxxAAZfWlIS7Ae+AaioD3xn2SlYSOYFFWcd4gcxHl0zyL8 JFSlXvCd9BEMcPhbxNTRw5h7NNnARDuN0pCkYfnP1loTTdSg+ogQ5XVKmbZnl+fAhZBojB vsaJDVrQ9908Wci3zuVvxaEJ1R28eIc= Received: by mail-pl1-f179.google.com with SMTP id d9443c01a7336-21669fd5c7cso165407305ad.3 for ; Thu, 02 Jan 2025 17:50:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1735869040; x=1736473840; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=z0/95vxHq4AbTjjm1qqTnNQIEsjcgRpvj31jjZiEdxU=; b=EdRf91tzTAyxKeQRWiPNaCWMTVrI4i/F8iiOLMR3dmm8ttJIp5C++MIghqa02q4K0c klQq9nGavUuLIB1c9CbRxWaxa3JTGB3WG/3r6vlUBjpk7Kds1vvhnnGJx4mqSmhhkxMA j21YcLF7T4f5TXnYEA75ktr7GDMe0aupdVAwfhz0xWvc0RSfyYzFkGnGGYO0IDg4HNEt YDAi/rnqGPVqbhV84QatKFTL4kPiVT+m12M1hu8xWw/ZbZdJtPNOF+dHFWm3jgCdfGrZ zr1HhAYY/qx7dqoJFE717Scalih7OCk50lgVCqQD7jCz/4WExD3FT4iy1AqDaRvyCvKz AVvA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1735869040; x=1736473840; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=z0/95vxHq4AbTjjm1qqTnNQIEsjcgRpvj31jjZiEdxU=; b=EOrVmsB3mY4O7UWPo2f01rGhtr4/Q/2dqyzB5V5Uq+oxwizZFnUnP0PaCrZfL3GrDG g9Veub75A2oFxkyk7dGN37bAeGi5S/0QwS4yS+6kjO29PFa48GXSDzQ+WcOzA0q17YSs BHyLG/rrTu/o7QPsrtfv4fyAHrdlKcdorpOJpOArD1dOUAt0w3oHqti5SI1Q7CwcmmnS to3MXcAmPEG3cp4sUVZrVvj+KNBQR/axvs+nFfiJ1vxlsesjSCslw2jZMV6mYg4jauO8 xgt9qO2R24YboVi4CfBxCYbW36rzar0Kq91DJzng7YBRhuRaXlWpS2f0SBBYhperpiP6 LnmA== X-Gm-Message-State: AOJu0YxBgOOYbS3BXSJOYPlNUTTxO+4ItmNjBYOofmj2ohWK/4uxK/ye rme1LN4LjVrrBocCGwkTek8oxxLvARDz6z5bogPc+j5LtIGHSUg8 X-Gm-Gg: ASbGnctlcixOQpK7PsgOzyASzcYSrhslo8x2aPQMrob1pu+Sh8pwTolm7xnIyVaGyF9 n26RzNthCMlrX4k4sRW8wRQPTg1IJiNjRWovdgYgx0F1qZyc4i4GnRzOyjM5dRrAe6dkO0UhzWV Ka15D1iXnOXqtrqnTXba2DdcGCv98RTrEfSDgfLNd7I0XySnC7vcAtBPkwjEyrj3QJzHtM0r95B 0vN3QyeX1UvSbXZ2oQxzF4D3AhV22Hv4sz6wKPBJ89u/PwetN8bTL4Qj6lwUob16IufYdoYcZgK ZynH/931YPVg0IfY5w== X-Google-Smtp-Source: AGHT+IHpW6kPJoz3uQ31McNNuWFPISD9A55LMmIKwxPMCRkUhBP25MkbHaNyVJ9vNwtjAbpRrI4k1w== X-Received: by 2002:a17:902:f64b:b0:216:779a:d5f3 with SMTP id d9443c01a7336-219e6ea1c00mr709992435ad.14.1735869040635; Thu, 02 Jan 2025 17:50:40 -0800 (PST) Received: from saturn.. (c-67-188-127-15.hsd1.ca.comcast.net. [67.188.127.15]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-219dca04ce7sm228851505ad.283.2025.01.02.17.50.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 02 Jan 2025 17:50:40 -0800 (PST) From: JP Kobryn To: shakeel.butt@linux.dev, tj@kernel.org, mhocko@kernel.org, hannes@cmpxchg.org, yosryahmed@google.com, akpm@linux-foundation.org Cc: linux-mm@kvack.org, cgroups@vger.kernel.org Subject: [RFC PATCH 7/9 v2] cgroup: remove unneeded rcu list Date: Thu, 2 Jan 2025 17:50:18 -0800 Message-ID: <20250103015020.78547-8-inwardvessel@gmail.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250103015020.78547-1-inwardvessel@gmail.com> References: <20250103015020.78547-1-inwardvessel@gmail.com> MIME-Version: 1.0 X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 0B46E40015 X-Rspam-User: X-Stat-Signature: 9zjg58fh4r6ukhqxj9xe1txwaty8up4n X-HE-Tag: 1735869002-42385 X-HE-Meta: U2FsdGVkX19OVTzjPxafEB6P3gmE1xSN5GnVpe44FPoGVgA+TRpqiOleR4E0kKobvsl9cqth1obNM0wPL52LGXaPGBi3DkojKdNwkTYUQAYgG+mlfDREbJgeS99jjn6XYymLBbISztHIO5uqSbkxNaxCs6AfLeCuWuFVWg7rf9j07IQRkQ0DEz/7WhrUp793MBIecnW705thkStM373QxzVWaAm7n6p/ErNl3oKsMmXqE76RrxwbtSGaZRMTUrljgKuyGsy6KwDPkdbsKrK4+kKpn+97MG3jouLyjnVpHu2YBGiEx5p1PK3TYiDnCtdNmtqSC5GR/GZ+az5LC/vqF6BeZDvhh5Tp56xbhQrLiUW+yWQonCPldv0Jh8rPPllZk2tfe2jLJpAXUje4L+7+oXpcyajGZv3KcMuJbkY8DozDWstVYelCV+KQh9mO87if1tTay3/7c3UTu8TNp6zdBAVQY+NKv5sK+b4kJDxEBlcMtI/hy6pgNsUBvdIao2Nu4MkYqg5aHl5eXEbADE0rEryd6B8jA+u8GGl7XtDQLs97cfU8uLlL6HB92JkwTWrYMCQfCKW3AMJsjN4jqP9w4ja0TdxQejd9dJNQLzQNxfgMGjjxfn863ymxzMBjxGhGIrcbmQBlYJ9XzKCkQX0dJhKP16tnc885TUI5jKAdybzhQ2teEkcqnkt3tnPqaKHy7ARe/sC5gfFW1m+eIAcOLDa2ppIRIknLffU79RymVkOzFgoBMDGP94KfAPKDReySbee/kOd/edytLtandYip3O8Ksa/Dr0uyEJGkAua2WNint4ltCZ065FSMLi6ZfZx+LYyeKA+DJxz4VsdNVy/7D1HLFAnMquDBRReo3JVgfnXZY3YkZK+HQOIHSEAMWDHvNsC+v+7Q4M0FFYg4xjbSJQj4M8ipsZe1I9pVI2d28Cw7b+u+OAjZ1rRBbRcLvucgYi5ghCrv/sz9mVcSaBK b8ygbRKU 03lnFEixnVgAsZ28DkSv5GMeqI5U6/YhAEv9REEGRb1Mh2pb5szM2GXtF5LaLnsJyyCO0SZUBjxjGPGFs3xzhxSCXjXs/PdTQd9wWY6DmlZZfysFeGD2efeYsh52tu5IlmzGT7iUC2G0zHTl2cirnTUAt+Go0UQIpT2nozpTJbNLEFOn/tzGhiqxyzHarg9s28id2CabRTIjNwZ+UN1zJuS17j1CZShl2pL0glT8nsf4SqutMR3wMkPmuEjLbRpund9zUAfWA11CnYZrhJDDsjRcnjlFqtxfOccXDKVb9QluUFKm0Zv9KFfI168AjbkfQkGay+j9AGbLj8UBHivkyjyiQ/rPJip0AZjToHT/2GsN7EhX2g2k8/+3sJw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Since the cgroup_subsystem_state now owns the rstat tree, the list management previously done on the cgroup to keep track of which subsystem states are participating in rstat is no longer needed. Signed-off-by: JP Kobryn --- include/linux/cgroup-defs.h | 11 ----------- kernel/cgroup/cgroup.c | 20 +------------------- kernel/cgroup/rstat.c | 9 +-------- 3 files changed, 2 insertions(+), 38 deletions(-) diff --git a/include/linux/cgroup-defs.h b/include/linux/cgroup-defs.h index 4d87519ff023..836260c422a0 100644 --- a/include/linux/cgroup-defs.h +++ b/include/linux/cgroup-defs.h @@ -182,14 +182,6 @@ struct cgroup_subsys_state { /* per-cpu recursive resource statistics */ struct cgroup_rstat_cpu __percpu *rstat_cpu; - struct list_head rstat_css_list; - - /* - * Add padding to separate the read mostly rstat_cpu and - * rstat_css_list into a different cacheline from the following - * rstat_flush_next and *bstat fields which can have frequent updates. - */ - CACHELINE_PADDING(_pad_); /* * A singly-linked list of cgroup structures to be rstat flushed. @@ -198,9 +190,6 @@ struct cgroup_subsys_state { */ struct cgroup_subsys_state *rstat_flush_next; - /* flush target list anchored at cgrp->rstat_css_list */ - struct list_head rstat_css_node; - /* * PI: Subsys-unique ID. 0 is unused and root is always 1. The * matching css can be looked up using css_from_id(). diff --git a/kernel/cgroup/cgroup.c b/kernel/cgroup/cgroup.c index 96a2d15fe5e9..a36ed3995c6f 100644 --- a/kernel/cgroup/cgroup.c +++ b/kernel/cgroup/cgroup.c @@ -1826,7 +1826,6 @@ int rebind_subsystems(struct cgroup_root *dst_root, u16 ss_mask) struct cgroup_root *src_root = ss->root; struct cgroup *scgrp = &src_root->cgrp; struct cgroup_subsys_state *css = cgroup_css(scgrp, ss); - struct cgroup_subsys_state *dcss = cgroup_css(dcgrp, ss); struct css_set *cset, *cset_pos; struct css_task_iter *it; @@ -1864,13 +1863,6 @@ int rebind_subsystems(struct cgroup_root *dst_root, u16 ss_mask) } spin_unlock_irq(&css_set_lock); - if (ss->css_rstat_flush) { - list_del_rcu(&css->rstat_css_node); - synchronize_rcu(); - list_add_rcu(&css->rstat_css_node, - &dcss->rstat_css_list); - } - /* default hierarchy doesn't enable controllers by default */ dst_root->subsys_mask |= 1 << ssid; if (dst_root == &cgrp_dfl_root) { @@ -5491,11 +5483,7 @@ static void css_release_work_fn(struct work_struct *work) if (ss) { struct cgroup *parent_cgrp; - /* css release path */ - if (!list_empty(&css->rstat_css_node)) { - cgroup_rstat_flush(css); - list_del_rcu(&css->rstat_css_node); - } + cgroup_rstat_flush(css); cgroup_idr_replace(&ss->css_idr, NULL, css->id); if (ss->css_released) @@ -5569,8 +5557,6 @@ static void init_and_link_css(struct cgroup_subsys_state *css, css->id = -1; INIT_LIST_HEAD(&css->sibling); INIT_LIST_HEAD(&css->children); - INIT_LIST_HEAD(&css->rstat_css_list); - INIT_LIST_HEAD(&css->rstat_css_node); css->serial_nr = css_serial_nr_next++; atomic_set(&css->online_cnt, 0); @@ -5579,9 +5565,6 @@ static void init_and_link_css(struct cgroup_subsys_state *css, css_get(css->parent); } - if (ss->css_rstat_flush) - list_add_rcu(&css->rstat_css_node, &css->rstat_css_list); - BUG_ON(cgroup_css(cgrp, ss)); } @@ -5687,7 +5670,6 @@ static struct cgroup_subsys_state *css_create(struct cgroup *cgrp, err_list_del: list_del_rcu(&css->sibling); err_free_css: - list_del_rcu(&css->rstat_css_node); INIT_RCU_WORK(&css->destroy_rwork, css_free_rwork_fn); queue_rcu_work(cgroup_destroy_wq, &css->destroy_rwork); return ERR_PTR(err); diff --git a/kernel/cgroup/rstat.c b/kernel/cgroup/rstat.c index 92a46b960be1..c52e8429c75d 100644 --- a/kernel/cgroup/rstat.c +++ b/kernel/cgroup/rstat.c @@ -347,15 +347,8 @@ static void cgroup_rstat_flush_locked(struct cgroup_subsys_state *css) struct cgroup_subsys_state *pos = cgroup_rstat_updated_list(css, cpu); for (; pos; pos = pos->rstat_flush_next) { - struct cgroup_subsys_state *css_iter; - bpf_rstat_flush(pos->cgroup, cgroup_parent(pos->cgroup), cpu); - - rcu_read_lock(); - list_for_each_entry_rcu(css_iter, &pos->rstat_css_list, - rstat_css_node) - css_iter->ss->css_rstat_flush(css_iter, cpu); - rcu_read_unlock(); + pos->ss->css_rstat_flush(pos, cpu); } /* play nice and yield if necessary */ From patchwork Fri Jan 3 01:50:19 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: JP Kobryn X-Patchwork-Id: 13925090 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C7340E77197 for ; Fri, 3 Jan 2025 01:50:55 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9EE776B0096; Thu, 2 Jan 2025 20:50:45 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 99DC76B0098; Thu, 2 Jan 2025 20:50:45 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7C9BF6B0099; Thu, 2 Jan 2025 20:50:45 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 5C5C56B0096 for ; Thu, 2 Jan 2025 20:50:45 -0500 (EST) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 1B7E81C6A85 for ; Fri, 3 Jan 2025 01:50:45 +0000 (UTC) X-FDA: 82964461062.02.83AC915 Received: from mail-pl1-f179.google.com (mail-pl1-f179.google.com [209.85.214.179]) by imf23.hostedemail.com (Postfix) with ESMTP id 36E3414000A for ; Fri, 3 Jan 2025 01:50:08 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=eZc5xQ0Q; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf23.hostedemail.com: domain of inwardvessel@gmail.com designates 209.85.214.179 as permitted sender) smtp.mailfrom=inwardvessel@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1735868995; a=rsa-sha256; cv=none; b=mUtc/Xw1e664zrk3BZAy6zqnb1ZAXasTAGLda8BY85RYpsBGHr3ttdJ6FVu3FiBWqG7ien KN2PU076BV+ZuJVX3VEsa96lE3FpHl2q4Y8sbJNLXOK7So1hCEHCm4T+PRX8XcA/0AkRJ0 Fq3RNWULX94OXbY1ZiKi3Oby582fA+M= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=eZc5xQ0Q; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf23.hostedemail.com: domain of inwardvessel@gmail.com designates 209.85.214.179 as permitted sender) smtp.mailfrom=inwardvessel@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1735868995; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=yySB7M2h6IxrKtXZnUHVaoTBxDNRO4MzaqO/1vgyTwU=; b=GiwNbX3vPGB72OZE4JyrxEMyngwusFqvboU/3vyVkeS6sG778j/AAZe4h3zHnqX0Km/esC U8CD4ra+mbAxN8u6aNYAi/6i1hXmL8fWdDtCdcodjT3RlXDfOvi4FZJorIlmhq2szEQpls Gti1pETS2gkzGRLzvNIrzSZHUupgMeo= Received: by mail-pl1-f179.google.com with SMTP id d9443c01a7336-2163dc5155fso160546175ad.0 for ; Thu, 02 Jan 2025 17:50:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1735869042; x=1736473842; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=yySB7M2h6IxrKtXZnUHVaoTBxDNRO4MzaqO/1vgyTwU=; b=eZc5xQ0QYi3/XjzmM061Uzen5EgT6jFoHfjVLD/VB3t/Zamgw/Q+lpWXOEKkeqx/Gl WH2Xd445MUogIXze0t4BgFwqaDk9N+FdJU7M+uyZCf3WpKe1g9QTOP+IkIJ7FI/Qesth lOIMETs1Hgipm6lGOWaufl3c+Qm0sAdWxPr2izNmBHHXZk0SmQjKMSTS1XRFlQQ21WDx MtDYYD99srwqpQweSUJoIVGZVP9hBxeeqWhF7FYZciajOsXozm1kxD4veAOFcr4gyLOY Kd7UMlVNbSnwhTSALZM6oTlt/6KcziC4kAb20cj/BodNyFJt0c3e2HMvJPpaq0CLJtPn E+/w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1735869042; x=1736473842; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=yySB7M2h6IxrKtXZnUHVaoTBxDNRO4MzaqO/1vgyTwU=; b=ultvZuUrk4CcFox+k+ZrUh6HeMYMwDlULjaroHli6y5A7nabpLs1Hn5DWIXPowGRsk IVbcMj7T/F3yaNTxUIzK16VP4zvw/n2DmvhbojcbKxQTTWF+RCqgm4533wkF1IoiANBN c4HiAUkoZTLUxYNd6Ds3go8x2OFttH5OGc1W8oTeEvAeCbRiBDF3jOAr+L1KPAEgKTKZ W6ZBy3XxZgUgdIYMSWAgP8pC9UY4bGKl+UYlQ1v63uorRFo9wCovmhEBWqP/zkbS37nz JvgVskOVxJk9wnIVs+fj+Yl8Hw/vTHjY+2HQ5mLsHQUYS/ZHHz/4yf/8lPy8lOwA4ETo sMhw== X-Gm-Message-State: AOJu0YxF5ulBkxs7PTeSQkRKjPU0tvKGEJU4S+7I98BBozcPl+qtjKOQ yyZvZWweO8sB7zR7yK+Monk1ZLy+kxz55H8wml4cnGDDSnNK5R/W X-Gm-Gg: ASbGncubhek2840jDzinbEcPY/oJpEHOJ1Oa3hfJ87c7h8aWuND2vIVMx1sAbyA36KY bDekbpjTiIOTS6kRK2Aos135xrIz+6Z0OcnHE02M8eOF8fUcagw+YV8FmyW8CJfURooIbwct7cq be9HPrrRA1/ywRtZ2n383mXwGqz2+Os82A2m+Q5AlG7zzpH2iK3KUE7Nr8v42mfUHsm0el+Pp1D cgDnVV2TXLJv0psQcxBiX7YdtnhKNBdrJE8IoYE9zkn+KqdbfLRTGtCFQ8guSZiymvvKI18xCiT 9N22evDQ4/e9PvowoA== X-Google-Smtp-Source: AGHT+IECu6ABKn1pd4uHiROyoxYhMZo/Ay/F4QxXtatl8UwP6Qp0oTpLfZ1n23xS7eENlQtjkGSayA== X-Received: by 2002:a17:902:da87:b0:216:554a:212c with SMTP id d9443c01a7336-219e6f12d63mr582089575ad.46.1735869042055; Thu, 02 Jan 2025 17:50:42 -0800 (PST) Received: from saturn.. (c-67-188-127-15.hsd1.ca.comcast.net. [67.188.127.15]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-219dca04ce7sm228851505ad.283.2025.01.02.17.50.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 02 Jan 2025 17:50:41 -0800 (PST) From: JP Kobryn To: shakeel.butt@linux.dev, tj@kernel.org, mhocko@kernel.org, hannes@cmpxchg.org, yosryahmed@google.com, akpm@linux-foundation.org Cc: linux-mm@kvack.org, cgroups@vger.kernel.org Subject: [RFC PATCH 8/9 v2] cgroup: remove bpf rstat flush from css generic flush Date: Thu, 2 Jan 2025 17:50:19 -0800 Message-ID: <20250103015020.78547-9-inwardvessel@gmail.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250103015020.78547-1-inwardvessel@gmail.com> References: <20250103015020.78547-1-inwardvessel@gmail.com> MIME-Version: 1.0 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 36E3414000A X-Stat-Signature: fmr5d3mdmif91iseewg448wcczkh4ws6 X-Rspam-User: X-HE-Tag: 1735869008-902256 X-HE-Meta: U2FsdGVkX1+C1RQ5mDPvfIwBiflBLd9A3649RBRSjGugFlbMjxbeZC6DJyZuPi28f3FCTNa9uByjWuHvxXreCseV7w6m8xpVosbon8atSoR0kxft3FYguRTr8UM73W0hJdY05uHqoRedhIWLJyfa3z6fo/2fCtmhWrtjFiSBRAu+bDwegkVFHDVxmdS4ae04+nyF7BD1z8T+YxjMQ2e7JRuryLJEKN9IzXFurGtROkdm968XURcEbpfQxTRhAgYe6b+r1H0LXiIws4WxuxtEuYdQ95kP8X+B2j4tGLIbUpDWErdAIsdtH5PdcHzKYR1y/jpAt0QZfzEyO+giR2REG1KtqTm91YqT1eyx2NbNf/Oa87YfMViYPf3OQqRbVzL+gu87jF99Azi4gPySw3UnO+1YMlJWb5Ep1wJeSm1nQK9tTJurGEF2EGvhcXM1kdlHhaW45AFJ0p8B8EohsIGi4QZxcWBcvwEutRInTaTSCodR0SG7sWSYqZ8Q4aLC18QbQF11MKsDCOZuO+yj88ZhW7iQYyp0XzWs5E7O8bpOHLJ9Ph9pzRGoJmRJNwLllyq0x61q7VK39DLzle6/mYOkB7OJC/Jkoz3ZBUF0Srl59ql9VDpKfbI/SEOfV+1M+kjfvg8tz4xcJpODsoPwpVBwRPA2+EvF2qULSu4D/1DjgVuW7XiBU91p7jJV9nYGgFBSB8+iPMSlIapHlMZlSG/ycf3odT6YUB4MOwPOOjgRWl7UAuWZm2R++A7ui2o7YFWntedakx+BiQAZb37knHcu7EuQ/EJrOyTa7mtwOTjmOPltbNOylNK0UppODcbe0syc/yuJOi0/TmFwSOCV1Vq8kbUSJAnRyX62irgrS/FNeZrzyazMjh6TqIlqWWZ29s7EZl2SuSYhkwlW76xlESpOh2h/dxCWbDP9psRwXL8i3RDiWPxkayIbFM2YnF665M2NWx08fe+C4GBNZh4iQpw fkQ7kUg9 HhYsNOmf7x4sqdpxrMzTjbEUj8PkyQdi8wNZ1KAIAa7xJ8JyKJjuTFVjVw6WfEeiCsz6tgEab8FNH0TjE1RUj2PLiPx1S7X4Du795x38nOv7ncRilTWm9RroNI0vm7Wir2ZPLlIJgQyxOfx/MxplGfdgVY+yQNkUUlbKxt7nugeD3srz79bMg2jvYSDsBlo25kLPk0OVFaaP5P/F0+tK1diJkL94XzrAm9CpJgTFuUbeYvn53KTc3/DAmwdQILY+E/qS+gc/bYEzV9FIXHrY1FUNDhcb3qedQt9OO78gz9YVemhi7DSNxmsiH+gLozpUv5glZTTsR/QvpQxfhJX0Ic5/I68Fdi4yWZN7LaITkX9BLJfnrjBjxmUtgZA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000467, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Remove the bpf-specific flush call from the generic subsystem flush. Leave it up to bpf programs to manually flush any subsystems desired by using the kfunc cgroup_rstat_flush(). Signed-off-by: JP Kobryn --- kernel/cgroup/rstat.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/kernel/cgroup/rstat.c b/kernel/cgroup/rstat.c index c52e8429c75d..03effaaf09a4 100644 --- a/kernel/cgroup/rstat.c +++ b/kernel/cgroup/rstat.c @@ -346,10 +346,8 @@ static void cgroup_rstat_flush_locked(struct cgroup_subsys_state *css) for_each_possible_cpu(cpu) { struct cgroup_subsys_state *pos = cgroup_rstat_updated_list(css, cpu); - for (; pos; pos = pos->rstat_flush_next) { - bpf_rstat_flush(pos->cgroup, cgroup_parent(pos->cgroup), cpu); + for (; pos; pos = pos->rstat_flush_next) pos->ss->css_rstat_flush(pos, cpu); - } /* play nice and yield if necessary */ if (need_resched() || spin_needbreak(lock)) { From patchwork Fri Jan 3 01:50:20 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: JP Kobryn X-Patchwork-Id: 13925091 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 873B9E77188 for ; Fri, 3 Jan 2025 01:50:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1B09D6B0098; Thu, 2 Jan 2025 20:50:47 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 09B706B009A; Thu, 2 Jan 2025 20:50:47 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DE4BC6B009B; Thu, 2 Jan 2025 20:50:46 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id BDB146B0098 for ; Thu, 2 Jan 2025 20:50:46 -0500 (EST) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 78B78A037F for ; Fri, 3 Jan 2025 01:50:46 +0000 (UTC) X-FDA: 82964459676.06.FA5F63D Received: from mail-pl1-f173.google.com (mail-pl1-f173.google.com [209.85.214.173]) by imf04.hostedemail.com (Postfix) with ESMTP id D415A40009 for ; Fri, 3 Jan 2025 01:49:51 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=HtviQq5x; spf=pass (imf04.hostedemail.com: domain of inwardvessel@gmail.com designates 209.85.214.173 as permitted sender) smtp.mailfrom=inwardvessel@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1735869010; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ZpckZ+t0825F6IGB976GW0oaJ6tqis13qibxlRS1UMw=; b=zwFGLtl9Us2lfMDmCSlrROKrik83zbUbXOpIVcGbhbl3xVsE9DRzUAfWQMdbl2f3Q94Bni HWhgGTHNI1hpW7qYQNSaa/35TB+CYAybnOyGQRSC9P/r70ICrBP1Ta38BxqTBJ3COiB3Ju K5r9Ecu3DLgl1Wj9YjpHuw8GLA3A7h8= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1735869010; a=rsa-sha256; cv=none; b=k8UOFeGmtvutoHCAsJUCn68hbpHs38n4f2RcChsvrkmXbaLAfcAaibfYh/9GO9Oj7K48YY Smu1WR8iqXV2mwv2IOFykigby9G8Ktt7kLt4xePCvxaPQuQt/WLL9HTKCSF7QJzHGQ3qCp aBuiAyEhAlEl543+1SkZxTUFPic4Me0= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=HtviQq5x; spf=pass (imf04.hostedemail.com: domain of inwardvessel@gmail.com designates 209.85.214.173 as permitted sender) smtp.mailfrom=inwardvessel@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-pl1-f173.google.com with SMTP id d9443c01a7336-2166360285dso163859575ad.1 for ; Thu, 02 Jan 2025 17:50:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1735869043; x=1736473843; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ZpckZ+t0825F6IGB976GW0oaJ6tqis13qibxlRS1UMw=; b=HtviQq5xBQBeHFTaWwMK5Bxf5G2hpqZgsQ4Su3r7mGyooiWsT9fBm6EyQOaiy4Mbor e5i47/50XEqVgJJdWaKDj/HGqBpxBu3X6Bq8FFgQA6/JtmLnmkZ4RLgn47WZAFl3MXdI qildsX0hFdQ9G5leTU4E8a28LkWyyhNH8GaCqjJyD5fEcXZitkTvERPeWgalb3Or/3HW Yz2pyCci2eZAlv4xr4GaHm3Sj6FMzmFft7PdAv1NTUq5l3m+lZWzM6GLmG/cUQvMYQbH +5RMe/BVHiKjFrRh0fPonkjs64EcXy86NPVpSWSBr1l4UFgCXAYcpOiPU3KdMvJMNqWD jU5Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1735869043; x=1736473843; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ZpckZ+t0825F6IGB976GW0oaJ6tqis13qibxlRS1UMw=; b=IMYNM1ZgQrRJ6J9xHXWeLtPX3NMk3Op8Evyyo/TpSGZ0avhNNZHQdSRtaukU7P9UeW XAFU8RcIliSXj3ZVloV+l9SGboFh1xfFIp92umXPQpqpXZFun1jjSAqJgfzciAwB8a+d yfveLH2jG/5yH2lzmIAIA3o3LzNenWYoQ4ueybYeVo09WNLbnvu+zSkuXzlpNGCHV900 31hl5WkaX6qvOEwH4Fpt9k5IhlD38H/R5/0j8gPWJrX4pIldf622/8ILsNAw7xZbthdv fUeezYVXIr4KT/4E8vNjfI0HVYaRTDbuy3eA9NVF75tfa9JUBLCXom4NFifI7JZAeRNl VGKQ== X-Gm-Message-State: AOJu0Ywo+QAG0mdsZV4ncNpvcJJ8arjia2XVHG/Ss30m/AObUn+287hT BwtzJ3yMQ4JSJr8ZUQGL7gZn+PxTUm82bsLwf/06pJmFuKz/wnsa X-Gm-Gg: ASbGncsse4Oi7wqfR1yIZkY8cFYwF5GwEQn8FFu+47HJxw0i6tnqGj9fkfIUbXyzFrU IbvNYzf3sr8thZYVNh9BOyPe9DSGul2J8rMbKkhh7klEp9p4NUNP4G457M5jncbRRU2juxna+1d V7mOleifGVLqzmvIyW8QRZQzT/rXFQ+QRWeH2jgzeFvT1Wy1Y7aBV+xk5wAUAdosj1oIj3EYP+b aC7/1IdRtHtfrg3XxFFgvqDI0Agko7pdOoXqGG8Phlv/yAyKR0bMn8S6vrCbXjTleQgICrmt+J7 SWhH1z6rBVU1mhxZ+g== X-Google-Smtp-Source: AGHT+IFdazoj6fte8hxpaNLnC8unSV/ywRTEJ0Kh0HqeY1Q7I7MuHrhSeeZdyfL7DZejliHH6GwWqA== X-Received: by 2002:a17:902:c947:b0:20c:9821:6998 with SMTP id d9443c01a7336-219e6e858b4mr607012025ad.10.1735869043451; Thu, 02 Jan 2025 17:50:43 -0800 (PST) Received: from saturn.. (c-67-188-127-15.hsd1.ca.comcast.net. [67.188.127.15]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-219dca04ce7sm228851505ad.283.2025.01.02.17.50.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 02 Jan 2025 17:50:42 -0800 (PST) From: JP Kobryn To: shakeel.butt@linux.dev, tj@kernel.org, mhocko@kernel.org, hannes@cmpxchg.org, yosryahmed@google.com, akpm@linux-foundation.org Cc: linux-mm@kvack.org, cgroups@vger.kernel.org Subject: [RFC PATCH 9/9 v2] cgroup: avoid allocating rstat when flush func not present Date: Thu, 2 Jan 2025 17:50:20 -0800 Message-ID: <20250103015020.78547-10-inwardvessel@gmail.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250103015020.78547-1-inwardvessel@gmail.com> References: <20250103015020.78547-1-inwardvessel@gmail.com> MIME-Version: 1.0 X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: D415A40009 X-Stat-Signature: krj38xt7f69hmr8zxqrzju38djy7hyua X-Rspam-User: X-HE-Tag: 1735868991-355137 X-HE-Meta: U2FsdGVkX19g1zsS5DpzVAP6S5OaSR8TWEw5afxV3PCLSmf8qhbeNrrGhVDuIapmQbFjFnutKHKO9ctJGheQlf+WtQ9zARVEFfgWCkmTsvLXsQJ+Rc+VZFPYJ4tSBAPe9G0ujcRP1nYscsxtpGzkzNS2MPfK6FnaqGYfO2hRLpH4Eh4fQwGg0n/G3Y7f/SJp2AZSFsr2vdZzNfBHxFm6Uebn42XOldwfmP+L8GcghUI+SamAJnjsTKUAMl2k9oXSAIk6KrEgnXnZTt0/2p/tCl4RxcKuf57S1rwpdHEZBGMxsNqWxXOw/qBkeEn2vjHTayrnU5g4M/1xOOfFn+GjzNQeTyN6kb6Time8+WF/Saur8tfYxmXnbvVU+I8kWQqUn1RQjiBJNQB9FGf9OH9wRORMy65M8MEadW9kH6IGK9HzAv/LX6OwzL9wyF/uOFWr/h0/G8oIEL2KMESNEb5iBJUmpmM6M6I0oQ4zOc0duJn6H0f/JndUu+QkOzfBYT5bD5o/8nAmyI0CaAelb9EPDQw+SNidotTqMfS+qlDX7k5Mv98VDmEvvrIdOFpemyaxL7M76LOS7qP1/lP2R0lamcBN6flpUaXaXBHiju3WUK+j125/iH5R1qlkcKoICLNcsv7WRQT4dYpqro4SMfJx+lF4yuz1WUAwUqhlE6nX03ayxfkYB3MewAyf2/KWhWVxoo3NKRdDI1P1uwgi+VoMreNJek+BQug1KUVAQ+AKkbHkG1aGd7QBHd31auz0S8At+epvAz5jNZl3NHVk7UYu7jH7fPhUMBJq2xTvE3pvxYPodPntaXeSn8SJJ765jwdGTG76P7teqXRy+J7+imxNyNL5HMNfEXBvTZ0VUz+Cn+Ld7R26LBRqq5L+PUKq3x55ndlrPkbdLx4ZGavJtcrW44ViDK3ddKbCBFgIqOXyM/eauGKGPIaiOkE/lNBqzKQpKbv0UkDVNjIfJ4IOFIf dYNYM/be Hd9D2MgCBVyhywRbQaWzUDSMsMvBJUcNl5rTKLya9iAF/B2MBFeumbS04vr8TOJC1r2UmtZA6OfNuIg6W77cTL+oz7JX/2Q03cqIWDAO3yBYjNtJjwlo5uczLlbEwvEYv9aINpLIkgIXtNDHR+zhwiuBSL7rckKOTSrc48CrIcUDEPMzuPe1D4UtjOOybEcyI4Y+SMmzDKCruJ6zxy2v1alj368yLmBG+e4DaUi5NUEfpuH7sJ1Y4cz6bOz02dqNcekwPgluuYb0gj17K3VXc6M3mPKJSP2BwMnrQDm4Yz10dO7e5+zHnKacs424nNtbbstRRqPZNjRqGhXAag6j8ynKVWACv/NRdxDcWAnryldH/mZbEDQCKPqsNMg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: If a given subsystem is not the base type and does not have a flush func, do not perform any rstat allocation. Signed-off-by: JP Kobryn --- kernel/cgroup/rstat.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/kernel/cgroup/rstat.c b/kernel/cgroup/rstat.c index 03effaaf09a4..4feefa37fa46 100644 --- a/kernel/cgroup/rstat.c +++ b/kernel/cgroup/rstat.c @@ -423,6 +423,9 @@ int cgroup_rstat_init(struct cgroup_subsys_state *css) { int cpu; + if (css->ss && !css->ss->css_rstat_flush) + return 0; + /* the root cgrp css has rstat_cpu preallocated */ if (!css->rstat_cpu) { css->rstat_cpu = alloc_percpu(struct cgroup_rstat_cpu); @@ -445,6 +448,9 @@ void cgroup_rstat_exit(struct cgroup_subsys_state *css) { int cpu; + if (css->ss && !css->ss->css_rstat_flush) + return; + cgroup_rstat_flush(css); /* sanity check */