From patchwork Tue Dec 24 01:13:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: JP Kobryn X-Patchwork-Id: 13919568 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2DE44E7718E for ; Tue, 24 Dec 2024 01:14:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7CB806B0082; Mon, 23 Dec 2024 20:14:18 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 7552C6B0083; Mon, 23 Dec 2024 20:14:18 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5CEC16B0085; Mon, 23 Dec 2024 20:14:18 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 3CD406B0082 for ; Mon, 23 Dec 2024 20:14:18 -0500 (EST) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id AC69C1A1AED for ; Tue, 24 Dec 2024 01:14:17 +0000 (UTC) X-FDA: 82928080620.24.AAE5D16 Received: from mail-pl1-f178.google.com (mail-pl1-f178.google.com [209.85.214.178]) by imf12.hostedemail.com (Postfix) with ESMTP id F147E4000B for ; Tue, 24 Dec 2024 01:13:58 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=aKI9BeuJ; spf=pass (imf12.hostedemail.com: domain of inwardvessel@gmail.com designates 209.85.214.178 as permitted sender) smtp.mailfrom=inwardvessel@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1735002811; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=zYtPz7aK9YIb0L/ZbBa0RixlCtZNyuJIN6JSdj1lrsk=; b=F3CXk+Oy+bFi0u1NA0uaZtPQJGyOal7Nchzfuy/EG3W6WXnKVQuVopu36aMGHGQ3gd0Mgs PeQbvqJQipIBEM8HSEcdDyLY8BxrrjZbZ1glRdPvzUDm7DlbTkjRlPUHgXDm/awMZjuIq1 k28KrP/WyE85dD5wNi6mYpQV6bEqB5Q= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1735002811; a=rsa-sha256; cv=none; b=nDYR9dvAjaA4bqLIRFwo2VrQ7eN5Xihz1SI2Yht/KABhi5iOtHu2T9eF4H4N/HERoyI6JN Gk3bnF+1t+jowPa83eV+A7v1MAZJa03Z+QVEnORp+c7xauxK4ydc5H5fTA95Uv0TIBU65h ZuAQq6rSxFl4Is6Yr/4/OFOw4kh4PR4= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=aKI9BeuJ; spf=pass (imf12.hostedemail.com: domain of inwardvessel@gmail.com designates 209.85.214.178 as permitted sender) smtp.mailfrom=inwardvessel@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-pl1-f178.google.com with SMTP id d9443c01a7336-21654fdd5daso44787095ad.1 for ; Mon, 23 Dec 2024 17:14:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1735002855; x=1735607655; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=zYtPz7aK9YIb0L/ZbBa0RixlCtZNyuJIN6JSdj1lrsk=; b=aKI9BeuJ4iambQmfTJceuTqn2rgyF5kK6adQzgk2dLwj7264PyqkoIqwHAdScKoGZv yZSxEIGPN0zyIg5w1ql4BESMbw9ccpwUwFJdLXiJt00v8JgHvARxYKTMxqjNyk3EIFTb YVh4fbcXu47JqPe5NluwzzNahmM0a6LkG0mzsR75K3R3X9iKIFxz+MADWCfRr0QazrxW rbcOJL2AnQyNDJErPtvlhmyRnqWDG5JVJaT3waXWKEGivetcoSTzguv1NPPeZhK7itlN apBTLpH2vgRfKBHsojZHPdyEJJzwPa3l0r1hp61lda7zHF84kEoVdMNe4+x3g7urtr+s BtXg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1735002855; x=1735607655; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=zYtPz7aK9YIb0L/ZbBa0RixlCtZNyuJIN6JSdj1lrsk=; b=uR5wlkhAtTUWOpai7kV8ju3yHhVr3+zI8qryRj8MbrVU7mpxqz/+LvfALN/pnV3qHv +c2S9JBENHsaOWg5W/JzWFabVFDdQgGdgSXiTmQ5YcbQyYTXEfGb4qlMeNuo3p5TKS4q 8AtQYN03bPZHEb59c/b6DfeUEiaW7LiONqtJ62J+d3Jhxrc7MaAV1HSCFOigEKMoeoQa EO+7/e3nBchAzn3bhgZmSHa3mbVbmtb4NiddIlWloQkttGZ5cw0QMk5juaZf5pjyRYAb ho3XQ7dEikj2KfIWLHbG1PMaxqaHZ9mmilQHOrHBb4BOR+hP7oC41eMgxOCFNCdnBkUd kg5Q== X-Gm-Message-State: AOJu0YyA5T5zZiYXVl09tqd4Bda3IWxijB3UghPsYEfYjC1TMZrazm6s wgOXdDD3sQLopD2wI3RCUAHUll4DUcgy0xWz9R4k2NLxG9wT2WGz X-Gm-Gg: ASbGnct30oRoF1IBIP6c6Kdp5PR870NWU3Voa3OloukIAFrrUx9clhj9/DPcBHUADmx 4kYnnqa5+BMMGVd7KJvtns7+CImlmlTbcYZsvGEvk8qqheX8mdMJ4+Fr7j2hysVorN6/fYahU8x 3nVu3qqadvCSxV3TIYgo2ehV7Qta9zqDWeHkZuWpq4Tf9lRLFkczBm77ddrxi9jihvw/NV0PUM+ NBzaeFh0YpjF0Wctsw8kXlpj2Rkp6XEsiFAuOsMHM5Nou520GCSbNO0Gp/jIaMLb4gSr8JhNTZj OQMVOPwEiAg7AzHeuQ== X-Google-Smtp-Source: AGHT+IHxt/8FXT3EeHlti8KBRnrfHz4fCj1K+PzJkO4GExu/s7GdqY6WQo4vKMy3RCnOcrAt/99eqA== X-Received: by 2002:a17:902:e5c7:b0:216:393b:23e0 with SMTP id d9443c01a7336-219e6f2e9d4mr214204245ad.36.1735002854611; Mon, 23 Dec 2024 17:14:14 -0800 (PST) Received: from saturn.. (c-67-188-127-15.hsd1.ca.comcast.net. [67.188.127.15]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-219dc970c84sm79541255ad.58.2024.12.23.17.14.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Dec 2024 17:14:14 -0800 (PST) From: JP Kobryn To: shakeel.butt@linux.dev, hannes@cmpxchg.org, yosryahmed@google.com, akpm@linux-foundation.org Cc: linux-mm@kvack.org, cgroups@vger.kernel.org Subject: [PATCH 1/9 RFC] change cgroup to css in rstat updated and flush api Date: Mon, 23 Dec 2024 17:13:54 -0800 Message-ID: <20241224011402.134009-2-inwardvessel@gmail.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20241224011402.134009-1-inwardvessel@gmail.com> References: <20241224011402.134009-1-inwardvessel@gmail.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: F147E4000B X-Rspam-User: X-Rspamd-Server: rspam07 X-Stat-Signature: 7i5s673tt4apwzx31z15sniwczmyny3x X-HE-Tag: 1735002838-225754 X-HE-Meta: U2FsdGVkX1+bqq+RI8qRmfRn0idwmf90iEXYm7FC0lvodxFmIageDbGpWjqa34iozlGQEtC2zC7kAOeKVql8LOUoDlxYRb08OQAo1wehbhlHvMKdrJcgoKhKi+qpy7//p8dJ52S0gjV/amhF6a15IAtd/OmFcXno+8nbuRjrBUI7LIep4nTPNg45f2Blcfkrdxpc6l23MTWgc9U/tWgoWtSneXUwtQVgYAFZiaFqw5TGP7Oewf9MEdXJ2wTJ5YV7oodd80tJllVCetZULUsvMUSaUyxI9uTSx++xDkFzW4HlclDqnY/POCMHvhauHIDjUmLgZJbYj5gf10M+VWl+lCILgCOa3AG49InU1R1KNEgCyuh1PM6IvpoVYciKsVFSBivEymeBSmKM7EOBGaMaiugKzuNEvkJlFob6DA62hcbEXTA8k4M2TMpfmfpY5BHTfj0duH+Sm/jWfsXmrXvwesmQI7ie17lvruxHzwFxbdP6EXcngPCfFeEVEe+hIZdsEnbetGmHvN0XxpArjGJpsQdTNCEBeMh4WFCMkNiNmPC82Byd64p1FUGvu7GcIFJYoLit+3qsv0sGmMQmuGvBZL9BzfXNkF4TtB57aQCVjkNIvvrA933l/quAKEXoevWKNozXuyZ3NtJqmyDdg4iM7E6Qo5klVT4XrpPiYY6g1IozADVCs1Pkw4tvqeuhoYQ/RyRV+gteUjE70IruudaEsnqaa8W5BnSJG9OAPfFC/+tKXpVcOINhuuICFAd6InK61GZJQ0yUZTbBayXIGArrHOcnF7hxV2Byo8toCc9E9nPvtZOXRUZVnCmc0qQYtip4sNphJvO0bfNvyr+TCDyGkXNFVTfbFm0wnFS21FGLTD4faEn7A55CWSHCj3L4sIhPZiekcE+/FzE5KlOvh5MCaymHouFgYFYfIvq1k6VSGZS3n2iH7LY1uehEQSYy/vMYCPilWmLTRTvlJKwTij3 og7/m/Eb XGNQKA9wkKe0MQHRlWAxzLxyRkckn7z930d3uqZnxNDltQ7wYTiXV51qDxu/7d25xHLs+kWE6IP+nyfyhv2aqQTaRf0moaNtkv4vP0v+22Uik1dF7QG9LhOVXyL+f/XBMRm7ZTsfx+2Y4TrbIKDclkN2pR+Q4KnW9HVHGxM7SaW8XDltkDROMqN/KN0wW46LqzCg0vaDNjZhuOHb8amYBYVVeipQ+p/zj4eWFPHuVWJ/2JGp+W9kH9CKsBzQC16kYMzz//pmcCdry9CLHsL9WFz9lLy1pSIJAZOUOb9fYJxEZTUEtx0rINeKVHuUNAzsx0ZqnbQDDImG0f2pQYhavkKR+VeM5IewriLpdBiDqcGqUvNko8mRCexOVuQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000153, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Change API calls to accept a cgroup_subsys_state instead of a cgroup. Signed-off-by: JP Kobryn --- block/blk-cgroup.c | 4 ++-- include/linux/cgroup.h | 8 ++++---- kernel/cgroup/cgroup.c | 4 ++-- kernel/cgroup/rstat.c | 28 +++++++++++++++++++--------- mm/memcontrol.c | 4 ++-- 5 files changed, 29 insertions(+), 19 deletions(-) diff --git a/block/blk-cgroup.c b/block/blk-cgroup.c index 45a395862fbc..98e586ae21ec 100644 --- a/block/blk-cgroup.c +++ b/block/blk-cgroup.c @@ -1200,7 +1200,7 @@ static int blkcg_print_stat(struct seq_file *sf, void *v) if (!seq_css(sf)->parent) blkcg_fill_root_iostats(); else - cgroup_rstat_flush(blkcg->css.cgroup); + cgroup_rstat_flush(&blkcg->css); rcu_read_lock(); hlist_for_each_entry_rcu(blkg, &blkcg->blkg_list, blkcg_node) { @@ -2183,7 +2183,7 @@ void blk_cgroup_bio_start(struct bio *bio) } u64_stats_update_end_irqrestore(&bis->sync, flags); - cgroup_rstat_updated(blkcg->css.cgroup, cpu); + cgroup_rstat_updated(&blkcg->css, cpu); put_cpu(); } diff --git a/include/linux/cgroup.h b/include/linux/cgroup.h index f8ef47f8a634..eec970622419 100644 --- a/include/linux/cgroup.h +++ b/include/linux/cgroup.h @@ -687,10 +687,10 @@ static inline void cgroup_path_from_kernfs_id(u64 id, char *buf, size_t buflen) /* * cgroup scalable recursive statistics. */ -void cgroup_rstat_updated(struct cgroup *cgrp, int cpu); -void cgroup_rstat_flush(struct cgroup *cgrp); -void cgroup_rstat_flush_hold(struct cgroup *cgrp); -void cgroup_rstat_flush_release(struct cgroup *cgrp); +void cgroup_rstat_updated(struct cgroup_subsys_state *css, int cpu); +void cgroup_rstat_flush(struct cgroup_subsys_state *css); +void cgroup_rstat_flush_hold(struct cgroup_subsys_state *css); +void cgroup_rstat_flush_release(struct cgroup_subsys_state *css); /* * Basic resource stats. diff --git a/kernel/cgroup/cgroup.c b/kernel/cgroup/cgroup.c index 43c949824814..fdddd5ec5f3c 100644 --- a/kernel/cgroup/cgroup.c +++ b/kernel/cgroup/cgroup.c @@ -5465,7 +5465,7 @@ static void css_release_work_fn(struct work_struct *work) /* css release path */ if (!list_empty(&css->rstat_css_node)) { - cgroup_rstat_flush(cgrp); + cgroup_rstat_flush(css); list_del_rcu(&css->rstat_css_node); } @@ -5493,7 +5493,7 @@ static void css_release_work_fn(struct work_struct *work) /* cgroup release path */ TRACE_CGROUP_PATH(release, cgrp); - cgroup_rstat_flush(cgrp); + cgroup_rstat_flush(css); spin_lock_irq(&css_set_lock); for (tcgrp = cgroup_parent(cgrp); tcgrp; diff --git a/kernel/cgroup/rstat.c b/kernel/cgroup/rstat.c index 5877974ece92..1ed0f3aab0d9 100644 --- a/kernel/cgroup/rstat.c +++ b/kernel/cgroup/rstat.c @@ -82,8 +82,9 @@ void _cgroup_rstat_cpu_unlock(raw_spinlock_t *cpu_lock, int cpu, * rstat_cpu->updated_children list. See the comment on top of * cgroup_rstat_cpu definition for details. */ -__bpf_kfunc void cgroup_rstat_updated(struct cgroup *cgrp, int cpu) +__bpf_kfunc void cgroup_rstat_updated(struct cgroup_subsys_state *css, int cpu) { + struct cgroup *cgrp = css->cgroup; raw_spinlock_t *cpu_lock = per_cpu_ptr(&cgroup_rstat_cpu_lock, cpu); unsigned long flags; @@ -346,8 +347,10 @@ static void cgroup_rstat_flush_locked(struct cgroup *cgrp) * * This function may block. */ -__bpf_kfunc void cgroup_rstat_flush(struct cgroup *cgrp) +__bpf_kfunc void cgroup_rstat_flush(struct cgroup_subsys_state *css) { + struct cgroup *cgrp = css->cgroup; + might_sleep(); __cgroup_rstat_lock(cgrp, -1); @@ -364,9 +367,11 @@ __bpf_kfunc void cgroup_rstat_flush(struct cgroup *cgrp) * * This function may block. */ -void cgroup_rstat_flush_hold(struct cgroup *cgrp) +void cgroup_rstat_flush_hold(struct cgroup_subsys_state *css) __acquires(&cgroup_rstat_lock) { + struct cgroup *cgrp = css->cgroup; + might_sleep(); __cgroup_rstat_lock(cgrp, -1); cgroup_rstat_flush_locked(cgrp); @@ -376,9 +381,11 @@ void cgroup_rstat_flush_hold(struct cgroup *cgrp) * cgroup_rstat_flush_release - release cgroup_rstat_flush_hold() * @cgrp: cgroup used by tracepoint */ -void cgroup_rstat_flush_release(struct cgroup *cgrp) +void cgroup_rstat_flush_release(struct cgroup_subsys_state *css) __releases(&cgroup_rstat_lock) { + struct cgroup *cgrp = css->cgroup; + __cgroup_rstat_unlock(cgrp, -1); } @@ -408,7 +415,7 @@ void cgroup_rstat_exit(struct cgroup *cgrp) { int cpu; - cgroup_rstat_flush(cgrp); + cgroup_rstat_flush(&cgrp->self); /* sanity check */ for_each_possible_cpu(cpu) { @@ -512,8 +519,10 @@ static void cgroup_base_stat_cputime_account_end(struct cgroup *cgrp, struct cgroup_rstat_cpu *rstatc, unsigned long flags) { + struct cgroup_subsys_state *css = &cgrp->self; + u64_stats_update_end_irqrestore(&rstatc->bsync, flags); - cgroup_rstat_updated(cgrp, smp_processor_id()); + cgroup_rstat_updated(css, smp_processor_id()); put_cpu_ptr(rstatc); } @@ -612,16 +621,17 @@ static void cgroup_force_idle_show(struct seq_file *seq, struct cgroup_base_stat void cgroup_base_stat_cputime_show(struct seq_file *seq) { - struct cgroup *cgrp = seq_css(seq)->cgroup; + struct cgroup_subsys_state *css = seq_css(seq); + struct cgroup *cgrp = css->cgroup; u64 usage, utime, stime, ntime; if (cgroup_parent(cgrp)) { - cgroup_rstat_flush_hold(cgrp); + cgroup_rstat_flush_hold(css); usage = cgrp->bstat.cputime.sum_exec_runtime; cputime_adjust(&cgrp->bstat.cputime, &cgrp->prev_cputime, &utime, &stime); ntime = cgrp->bstat.ntime; - cgroup_rstat_flush_release(cgrp); + cgroup_rstat_flush_release(css); } else { /* cgrp->bstat of root is not actually used, reuse it */ root_cgroup_cputime(&cgrp->bstat); diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 7b3503d12aaf..97552476b844 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -579,7 +579,7 @@ static inline void memcg_rstat_updated(struct mem_cgroup *memcg, int val) if (!val) return; - cgroup_rstat_updated(memcg->css.cgroup, cpu); + cgroup_rstat_updated(&memcg->css, cpu); statc = this_cpu_ptr(memcg->vmstats_percpu); for (; statc; statc = statc->parent) { stats_updates = READ_ONCE(statc->stats_updates) + abs(val); @@ -611,7 +611,7 @@ static void __mem_cgroup_flush_stats(struct mem_cgroup *memcg, bool force) if (mem_cgroup_is_root(memcg)) WRITE_ONCE(flush_last_time, jiffies_64); - cgroup_rstat_flush(memcg->css.cgroup); + cgroup_rstat_flush(&memcg->css); } /* From patchwork Tue Dec 24 01:13:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: JP Kobryn X-Patchwork-Id: 13919569 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id F2FADE7718B for ; Tue, 24 Dec 2024 01:14:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 819136B0083; Mon, 23 Dec 2024 20:14:19 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 7CB856B0085; Mon, 23 Dec 2024 20:14:19 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5F7416B0088; Mon, 23 Dec 2024 20:14:19 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 38CFA6B0083 for ; Mon, 23 Dec 2024 20:14:19 -0500 (EST) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id E6E171A1AEF for ; Tue, 24 Dec 2024 01:14:18 +0000 (UTC) X-FDA: 82928079906.03.48F223B Received: from mail-pl1-f176.google.com (mail-pl1-f176.google.com [209.85.214.176]) by imf13.hostedemail.com (Postfix) with ESMTP id D11DC20003 for ; Tue, 24 Dec 2024 01:13:39 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=KL8sdwvU; spf=pass (imf13.hostedemail.com: domain of inwardvessel@gmail.com designates 209.85.214.176 as permitted sender) smtp.mailfrom=inwardvessel@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1735002821; a=rsa-sha256; cv=none; b=oyfDoIK1n04wHu+J8XuHDo4A06EZ12/Fz0CHIBlvv8BR50oViLKqw7kiEVPM3YeaMKqvni PCYyq3ZaWsAHrQxLfxaUFHG+WAcUs8FmOAbZI05fDGHXhFT8CS0RzSUBIP4WjfuWBOtcwD 8UDgl/wYiLEg0K3RC7uxmTanOxzJPUU= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=KL8sdwvU; spf=pass (imf13.hostedemail.com: domain of inwardvessel@gmail.com designates 209.85.214.176 as permitted sender) smtp.mailfrom=inwardvessel@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1735002821; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=zWty/mMu5b89nU3pRLtKHkGxN7KspqZ/3bLWyjMFJcg=; b=Lc1Q/te8SfdsxibnCHieCYHYgora43ppFz9SEzWAheJ2aGPeKTFCOgwTFSxoar0a0YaSJP Y/UMDIZ7vZbHjtw16Z09gMrkelea2yZUr3F1C36UeLg61q34U43XV75UpiXjq+yJchSHWQ Z/PuhoBrzCaC9j80Q5JGxaHcTFflJHc= Received: by mail-pl1-f176.google.com with SMTP id d9443c01a7336-21644e6140cso47073845ad.1 for ; Mon, 23 Dec 2024 17:14:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1735002856; x=1735607656; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=zWty/mMu5b89nU3pRLtKHkGxN7KspqZ/3bLWyjMFJcg=; b=KL8sdwvUujhMMN8TXHjGiJKHtXIS/gJgGNJekn+RkjXoKO1O6P7yAiKGwDvSIHgy/Z sejh2Y360rOdpUj0Hl6hMe9mw9nUf0rXej0TcYV8BjtjdjejhgCWLsSu9K9VCbJs9Z3/ zRoPkaOY0qRKqWZU1T0Y/wvIF7Kb9WpyQ7Wv0EEeUtz9NH3gnORw5kOXPoqZe21T93nQ jS2QQRtIzTGYtS+L1bcIj+Gm8uE9EcC25fvpvbqNOOSQfUDeN8M1HB81YzsDzD2dKug3 AJAvu2GHxhF/t4GHsOmqdL9PWapcsus5CgO0E3WU/xMYuM2hQqjRr/XaK3ppLp0VpH4f 526Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1735002856; x=1735607656; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=zWty/mMu5b89nU3pRLtKHkGxN7KspqZ/3bLWyjMFJcg=; b=NO2gLI1ijbjj0E/4xzGdLdytjGWlmKVqgS0MsIZBbiBO5k9Xuu8obnRj6SBHTxPT7w YdRYv4im1lzE9CyPfOrvZSLIc/lygRboLln3jBvho9uRWzdiCSjauE79tIJgWw6SJfQ0 S05UtEzgumcUKCtLbVKD7QsioXMnnQQq9JPsR8OzLaB5VabMAXn/7c5JcXqXUCf1RN30 KmK86qivwxS4ahZIv6KI+lz9bcfJv2VroLdOS9m5v+TnbkH/1jMsASjhjYElXeh8ogof neL4c3HFP3rtRT+snC+xJtUVDYV+lczduHQpsB4LEoiGVbenIsuyirZEtj6CpP3Vf3vp 86XA== X-Gm-Message-State: AOJu0YyjNM0R4Sz5qwXAFod2Pp06EixY1x8XKlaEwlolnxiYF0t8V+ej Zm4zV19FYWUY6lroS7KE3kseoPAHnJ3KnmUZiAdTybiHkzwh7Sm0l2mTpQ== X-Gm-Gg: ASbGnct4QootAh2OnJ8ZcGsbd4LiZvICVrw1UWTuSXY1siMylE83cqDu93XFyS6bxW/ zUco45t5MqYcgE9TjCUy0JfIl7SftH84qm1SiN9OCWWOOrR0tlqUJIT7dqNOh38F9u+LP2DbOH0 EwjLxs6X6Pf1g3NQqzWvxkfttM/tWZLlj46mZWwq1negHyXFKCdOsapdei79FLssav4oKlCp735 pZUPAQUi8cQH/WiXR9AI/Pati+elnY4eTX+s35ot64QfFDxgUJHGem7L+at16mIFIIyY5sD3klQ tzzh5cNRE54rwUgVbQ== X-Google-Smtp-Source: AGHT+IGWsap+rf81uOwV/i5d8iA2KStdKUOzB7fbhzdvPn0+dAjQ8cTLglNwC2zy+of0DesGiNk7zA== X-Received: by 2002:a17:903:2444:b0:215:4a4e:9260 with SMTP id d9443c01a7336-219e6ea1c34mr187867835ad.14.1735002855905; Mon, 23 Dec 2024 17:14:15 -0800 (PST) Received: from saturn.. (c-67-188-127-15.hsd1.ca.comcast.net. [67.188.127.15]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-219dc970c84sm79541255ad.58.2024.12.23.17.14.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Dec 2024 17:14:15 -0800 (PST) From: JP Kobryn To: shakeel.butt@linux.dev, hannes@cmpxchg.org, yosryahmed@google.com, akpm@linux-foundation.org Cc: linux-mm@kvack.org, cgroups@vger.kernel.org Subject: [PATCH 2/9 RFC] cgroup: change cgroup to css in rstat internal flush and lock funcs Date: Mon, 23 Dec 2024 17:13:55 -0800 Message-ID: <20241224011402.134009-3-inwardvessel@gmail.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20241224011402.134009-1-inwardvessel@gmail.com> References: <20241224011402.134009-1-inwardvessel@gmail.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: D11DC20003 X-Stat-Signature: 8zmsz8ebpi3nky1my375ic4856k49jp5 X-Rspam-User: X-Rspamd-Server: rspam09 X-HE-Tag: 1735002819-710584 X-HE-Meta: U2FsdGVkX1+kOvrAouVaC5PN7fqumzxwAEWF1ppHPfpDSpqt/F76gx49mCRy9NcbPkgtD3pl8vYYucG277rICz539QO+RgZtL8nG11BwzWhdDNmq+0HJYZoQk/vs03XMXSmYqwDcVjwK91+zxyodJbUkvGWhM3bY+UG+DZKpuOXcM+dkwi8dKNwuxblwkPwDTYPdPYQPDVBFPxQpcAOroZ9az9sHU8b5zkpQK7QFM9T2PiJJZrUGdMjB2d8dLd4R1WLHsehLylhROUOrZGu8Dh1MtceP6rhOAIINYxETJ9A0m5DKOZ7QiEHeN+LZ4tIQlbYSGcDVtABc2gducdpxpxy8ahhn6dQV1/wSbGUcVeaXFO21iiWSe2VOhsx+Mlo6kncdWzz2Jc0Mzzpw02H8hNiAaapTSvEctNgYZoF9JlQMv4c51J2Mf4NMAYmomiGnpin+ODLnAc2XyEzquLgnGrm+T7IETRasXkFn76Vf4u6xt6d8S08z8r+YZKFtVR9JzvCMVHB4nmCLUn5UUNwtQeKXI0bicawyWuTtx817fDEPtoRo2rNuiMF78GeRl1dpMMm2N3qRaqrA/jS6bN2tdQQYcsrbxER1QBrEJqfL7Ltq/WB5Tg0Em8Zk3/7kp755zDBzsTpLPJUfIWH6yI5YOQ1p3Vqbf5KLclFZz6Wa6gOqkk6lnKLuA7/I6YJ0Nx139Yk8HVRe27G2fO5fjfSmKdiE4JPG2QpnQC81314tZJNEfnysMyMo91Ij4bk1b4bkYE2Fvp6nWORjmC7qOVU6FRniRe0uWKkMa2AmpIfolJRXEGxA2nk3RrOhvj6/Zt8VReQvz/Uhr1rdM358z6xGS0TrptJGAgto4Bs56JXn5JiNKhF2iBiPM0riOnEPtm/LANGWIaKALs42An3xD/LpVzCBEtIy8Q8ag7w9Txa3D7udoCAvbYN+o8SodT5SqyzXNfWWILENgaxq+AxSKSh 10Nmaf7H HWcI/kINutW4MnUxjFHJvbP1FoqVH23k7V24zxxkY+5S7mJXKIptnQyOypp7Kknr/DyabRSoOx/cDOI3Wqt/6NkQy7ZHmA3aOMA/LzA8bFbKGhdjCJtVGw8l0uubrKGtG6cxte6KF1GjMG9iYVSFNtQUg/PJ2gPrUQQ+FK2nze5AUt9u/AHpbTUHbC8xmrJ23aabkEdhWjJ3sZpgetEivW27ufK7Mbs1+k5wybCjgbZpBx0aF9uSy/Uqo9xrCZ73T1dDdZCZ11jxneSBPKt7Q1nKr6TDKOrOMoMM5KPyM2iZFZJK6LbnPcrKK3YQwfmBu3lwJ6NOpTxTyL90cTF4WKHduYIUNIDD16AA50vy7dI5Syb7ghQkjltAMbw== X-Bogosity: Unsure, tests=bogofilter, spamicity=0.493654, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Change select function calls to accept a cgroup_subsys_state instead of a cgroup. Signed-off-by: JP Kobryn --- kernel/cgroup/rstat.c | 45 ++++++++++++++++++++++--------------------- 1 file changed, 23 insertions(+), 22 deletions(-) diff --git a/kernel/cgroup/rstat.c b/kernel/cgroup/rstat.c index 1ed0f3aab0d9..1b7ef8690a09 100644 --- a/kernel/cgroup/rstat.c +++ b/kernel/cgroup/rstat.c @@ -201,8 +201,10 @@ static struct cgroup *cgroup_rstat_push_children(struct cgroup *head, * within the children list and terminated by the parent cgroup. An exception * here is the cgroup root whose updated_next can be self terminated. */ -static struct cgroup *cgroup_rstat_updated_list(struct cgroup *root, int cpu) +static struct cgroup *cgroup_rstat_updated_list(struct cgroup_subsys_state *root_css, + int cpu) { + struct cgroup *root = root_css->cgroup; raw_spinlock_t *cpu_lock = per_cpu_ptr(&cgroup_rstat_cpu_lock, cpu); struct cgroup_rstat_cpu *rstatc = cgroup_rstat_cpu(root, cpu); struct cgroup *head = NULL, *parent, *child; @@ -280,9 +282,11 @@ __bpf_hook_end(); * value -1 is used when obtaining the main lock else this is the CPU * number processed last. */ -static inline void __cgroup_rstat_lock(struct cgroup *cgrp, int cpu_in_loop) +static inline void __cgroup_rstat_lock(struct cgroup_subsys_state *css, + int cpu_in_loop) __acquires(&cgroup_rstat_lock) { + struct cgroup *cgrp = css->cgroup; bool contended; contended = !spin_trylock_irq(&cgroup_rstat_lock); @@ -293,15 +297,18 @@ static inline void __cgroup_rstat_lock(struct cgroup *cgrp, int cpu_in_loop) trace_cgroup_rstat_locked(cgrp, cpu_in_loop, contended); } -static inline void __cgroup_rstat_unlock(struct cgroup *cgrp, int cpu_in_loop) +static inline void __cgroup_rstat_unlock(struct cgroup_subsys_state *css, + int cpu_in_loop) __releases(&cgroup_rstat_lock) { + struct cgroup *cgrp = css->cgroup; + trace_cgroup_rstat_unlock(cgrp, cpu_in_loop, false); spin_unlock_irq(&cgroup_rstat_lock); } /* see cgroup_rstat_flush() */ -static void cgroup_rstat_flush_locked(struct cgroup *cgrp) +static void cgroup_rstat_flush_locked(struct cgroup_subsys_state *css) __releases(&cgroup_rstat_lock) __acquires(&cgroup_rstat_lock) { int cpu; @@ -309,27 +316,27 @@ static void cgroup_rstat_flush_locked(struct cgroup *cgrp) lockdep_assert_held(&cgroup_rstat_lock); for_each_possible_cpu(cpu) { - struct cgroup *pos = cgroup_rstat_updated_list(cgrp, cpu); + struct cgroup *pos = cgroup_rstat_updated_list(css, cpu); for (; pos; pos = pos->rstat_flush_next) { - struct cgroup_subsys_state *css; + struct cgroup_subsys_state *css_iter; cgroup_base_stat_flush(pos, cpu); bpf_rstat_flush(pos, cgroup_parent(pos), cpu); rcu_read_lock(); - list_for_each_entry_rcu(css, &pos->rstat_css_list, + list_for_each_entry_rcu(css_iter, &pos->rstat_css_list, rstat_css_node) - css->ss->css_rstat_flush(css, cpu); + css_iter->ss->css_rstat_flush(css_iter, cpu); rcu_read_unlock(); } /* play nice and yield if necessary */ if (need_resched() || spin_needbreak(&cgroup_rstat_lock)) { - __cgroup_rstat_unlock(cgrp, cpu); + __cgroup_rstat_unlock(css, cpu); if (!cond_resched()) cpu_relax(); - __cgroup_rstat_lock(cgrp, cpu); + __cgroup_rstat_lock(css, cpu); } } } @@ -349,13 +356,11 @@ static void cgroup_rstat_flush_locked(struct cgroup *cgrp) */ __bpf_kfunc void cgroup_rstat_flush(struct cgroup_subsys_state *css) { - struct cgroup *cgrp = css->cgroup; - might_sleep(); - __cgroup_rstat_lock(cgrp, -1); - cgroup_rstat_flush_locked(cgrp); - __cgroup_rstat_unlock(cgrp, -1); + __cgroup_rstat_lock(css, -1); + cgroup_rstat_flush_locked(css); + __cgroup_rstat_unlock(css, -1); } /** @@ -370,11 +375,9 @@ __bpf_kfunc void cgroup_rstat_flush(struct cgroup_subsys_state *css) void cgroup_rstat_flush_hold(struct cgroup_subsys_state *css) __acquires(&cgroup_rstat_lock) { - struct cgroup *cgrp = css->cgroup; - might_sleep(); - __cgroup_rstat_lock(cgrp, -1); - cgroup_rstat_flush_locked(cgrp); + __cgroup_rstat_lock(css, -1); + cgroup_rstat_flush_locked(css); } /** @@ -384,9 +387,7 @@ void cgroup_rstat_flush_hold(struct cgroup_subsys_state *css) void cgroup_rstat_flush_release(struct cgroup_subsys_state *css) __releases(&cgroup_rstat_lock) { - struct cgroup *cgrp = css->cgroup; - - __cgroup_rstat_unlock(cgrp, -1); + __cgroup_rstat_unlock(css, -1); } int cgroup_rstat_init(struct cgroup *cgrp) From patchwork Tue Dec 24 01:13:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: JP Kobryn X-Patchwork-Id: 13919570 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A8B45E7718D for ; Tue, 24 Dec 2024 01:14:24 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3FBD26B0088; Mon, 23 Dec 2024 20:14:21 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 3ABE66B0089; Mon, 23 Dec 2024 20:14:21 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1FF186B008A; Mon, 23 Dec 2024 20:14:21 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id EEFBD6B0088 for ; Mon, 23 Dec 2024 20:14:20 -0500 (EST) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 81AEF4667D for ; Tue, 24 Dec 2024 01:14:20 +0000 (UTC) X-FDA: 82928080956.15.D51C4B9 Received: from mail-pl1-f176.google.com (mail-pl1-f176.google.com [209.85.214.176]) by imf29.hostedemail.com (Postfix) with ESMTP id 48415120002 for ; Tue, 24 Dec 2024 01:13:24 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=Rn34CfIz; spf=pass (imf29.hostedemail.com: domain of inwardvessel@gmail.com designates 209.85.214.176 as permitted sender) smtp.mailfrom=inwardvessel@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1735002840; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=JL2jgd7riB8bL0/y/2MxSvAIl3x85Oo9zvXWGx0Vk6k=; b=PxvBDMPf9kcR7JVlEjKNHE9SJQeecBu4inO4LSzabi4exPheFs5HclT4WCu+UJvYtorVON ydzFj6zOK8/H39NS9ezlHjFsjzXEd/vRRryX1qZEMfSJ2Ue2ul6tOl0dHG/Dq8y+aImEVo x5qyHvSd2KpHgxo9cmHJui6tCQeQ7AE= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1735002840; a=rsa-sha256; cv=none; b=3vW0h4VqUV21wQylE4/npliHWmyLbDrfPNtvXSU3X3GKAPQpopYrI8OGOHVuo0sXGBY5er 7DcXo8dbC4OAktJCPme2qeQRxbRbOLqukLOrbRGKBKFJd6+QoU/BjVbeLaXNd9jMe13eo0 sO+m16h+d68AWgsjqy/9XYgN+Iesu08= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=Rn34CfIz; spf=pass (imf29.hostedemail.com: domain of inwardvessel@gmail.com designates 209.85.214.176 as permitted sender) smtp.mailfrom=inwardvessel@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-pl1-f176.google.com with SMTP id d9443c01a7336-21669fd5c7cso45795325ad.3 for ; Mon, 23 Dec 2024 17:14:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1735002857; x=1735607657; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=JL2jgd7riB8bL0/y/2MxSvAIl3x85Oo9zvXWGx0Vk6k=; b=Rn34CfIzKxGwPFjCMteVk/oFqFGzHuznxMksMz9Vg7Qzu8hiKTVjKuuBbqHV5ArkVl nj7fTxlLRTMfysfdfeQCpCm5DGkhgD2indVdCV3+z3Qmx/k3uGa8YxnZ0wv0Q/Hwif6M zxycO+Eo42/MlFRwcODX1fiz97s7EHH+p0bepUUC7NuLqOR+WA3DnURGsb8c76kC2XLP tUl7rs7NTNwX9enXiU5VBrDANc/avKnqe9PoaVdgRH0sVn++Qs12uUL3ASAoSLRrDhPF mb+UkCNvxx1cY3OabKxq0nAXjN5uHHphyfewVKTFfhyLPwy6uth5YNdmecLscsHitdpA r2Mw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1735002857; x=1735607657; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=JL2jgd7riB8bL0/y/2MxSvAIl3x85Oo9zvXWGx0Vk6k=; b=ZtVsXr7KvVThYxvWcUtqOJsRsnz2PLil+Y91a5dDtX6Yqt5Y+K4iUPVQJ/hB/+mFNv sqXoWdZ6o7mdVpcodKy+eh9XK7ZQde44PWajPSH/5NJLM4i9SbQ7Gl1iBd7H98H/xEHI c7R6sS8XCIC8BuWSxt7sy8J9ICAvXESHoUnksuitLbcfijRZ6QhXLHXqfxGSTjzca/Yv xXRAjFdgrj1LniTYb1C3xBbN+cQdIpP9C+ISVrKq3aM8uw4N4uTW2FUpDztzeQMZojKF 6hYa2CWUW6xe4BvObp2d0qlzGjp3qANZjeqh5QoslDK+plRzmoGNMgBKklLyQow71dkW d4Dw== X-Gm-Message-State: AOJu0YzH7ss38dGqFGdhZfe4PcAawdnW+7P59nzfuWpgTkZiqkilyYRG WNSII3EOLd7dWPLONL/0RMM9sbtqq5BJ8Oj4+Ejjccj79w9Eb8no X-Gm-Gg: ASbGncsBwEoIyBj62pnUlcZUdDmEsXLNzbIi0n5Xn3SLYHoolPucuV5MgH2PPnTyy2c r9QBVgyDjxJ+cTFWAUyLq/7Rdz7qbddGmgKisYvaDuFETNv9RUFzWEf2liHNs77p8EnJ7r2LdUt LT0d2C/dMpNACn+e+WW1LlYDEhqSjtgGpHAeOF2fD4MPuoM0SOAy4EhBh1bqQPyFQuTWlcGgyzp pT/I4QNhmv2gBr8G9SuI9KwyQmx9kXEXkzePMnPkssJBzen5i+AAnCGo91DG5zOPFJFVmGgvhLD 8od7Mbw5Iy2kdEfB0w== X-Google-Smtp-Source: AGHT+IE3QHLwg9ReNpugqbcBsAF2fSOEsH+CCNMaVbUB+N1MFpxZaa07QtuA97SwYcItxeQb83M4Xw== X-Received: by 2002:a17:902:c943:b0:216:6d48:9177 with SMTP id d9443c01a7336-219e6e8c7e8mr225018355ad.11.1735002857492; Mon, 23 Dec 2024 17:14:17 -0800 (PST) Received: from saturn.. (c-67-188-127-15.hsd1.ca.comcast.net. [67.188.127.15]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-219dc970c84sm79541255ad.58.2024.12.23.17.14.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Dec 2024 17:14:16 -0800 (PST) From: JP Kobryn To: shakeel.butt@linux.dev, hannes@cmpxchg.org, yosryahmed@google.com, akpm@linux-foundation.org Cc: linux-mm@kvack.org, cgroups@vger.kernel.org Subject: [PATCH 3/9 RFC] cgroup: change cgroup to css in rstat init and exit api Date: Mon, 23 Dec 2024 17:13:56 -0800 Message-ID: <20241224011402.134009-4-inwardvessel@gmail.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20241224011402.134009-1-inwardvessel@gmail.com> References: <20241224011402.134009-1-inwardvessel@gmail.com> MIME-Version: 1.0 X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 48415120002 X-Stat-Signature: ku1ch6e8p71knxiq8ts8tikj6dspb4a8 X-Rspam-User: X-HE-Tag: 1735002804-891006 X-HE-Meta: U2FsdGVkX1/DTYRpHYdCo9F51qJh16Ewfbxzwtq1mxNybkDb30acl+biN9lYUtNoajDssAAP5K5+I2/gDUNf3ZTGGMQ92hAKGj76vm4697EY2sW1t9ido2rYmjfcU7XSx0mvkXqvwIkNfUtVLJOw306Ib2wW5VRloG1M3IZs07iZbaK8scIdZWElVjM/YIaogB/5buWlu36YLFw9rgN8DDpHt4dxAEG/YeqXFsrmjpOAjQxlADGY5LhWNAGjInMn+Co8pd0QVKA/IsXJo5E+Y1g6obNOY37MdJae92jkabZD8k5wZsx0undD9URMDTCmrkSAy7IEChaMNLEYWwGwpN6EW/w01qu9jgCTCOekHePLNZZAYeMfe7AlTpC90DCWjzALCN5Tk6tSlYtOwGNQloSWe2vrWwsx8FvsnDBHUje55CIv6fjnDZGOYwfnmLq6b1Ae/K6MwUHOpDfrFQBF1ncouQ8ro60UAldK/+BI+FsmGWNSjefZt2mGMvnyVSya32LwdXdzUmH2lkE0698tHAn13l5dCIYT7BAUHoBkmPEEVg+olDUwMFmN2pJeru5vUQJAIts6ehApo4y7GmaXvYfrPMvQPLbSi2N/Gx3z4GmfbbCrGUXUTwrfXDNklY4QIgRk1QWbbCUz7q1jnMsCnGC0GGH9l/4bGagcyDyBK7HulbMzmtoTbbKBlNSiUd00md2PcIgR5jRmTxFp/PpubC+W3IZqj8ycwkokG3+g8X+93mqWd/trdReqDbYQGjKmcfp3A7cXbXKQUiOi+W255f6CRJwoM8gsJcMF47zqJie2y/Dadftg+rvKTvmLlGLFIpkfvh0Gkq3AqpwGLwg/rGGGEKCttBD/3XJ0IPmSg8bHhC5TYwPsLqaAy+bPa09xqIRNUDd31uelFabHE0B8bDdgnGKCNuqijrdgs+ZiO3auBznvq8cRbd9fdf3aI9JAaBakMBag1QWGDuxyej0 2HWaCVaO SXu58S353MZIO9S8FKY4O+8ik67YnlJuqLaSpXBMlQOOPErGMOz2LDQ88yNs+9dhYI7ig7hUAwQWD8SZ+B+3b2HipScsiwuf8auCK1+3ejtZAYzcboafaPK7K2h1n5oD6BO8d+Hq0lxRF4ZMxaxsrSlpgiZE5jn2jhkbAvoI2CB94Mqn2pLNXYxiDaXOdnYHC0X0Dgdz5hch6SgNXnUE/YepS0ULb74JXgyvpo8YpaLB+sdkmWPV+lpsGHAlnwTSeUgoxRpGdUm7ZTEB47+t+GooU3y0ZSO4OLkuT+A/DAU2e1M6GtvKewgXTMTPZIiuUnHb4V5jhdP77jC54ZbeyrC/zeQCHxYpymwSWytjJtHA2978NP8YgUDrYrQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.069124, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Change API calls to accept a cgroup_subsys_state instead of a cgroup. Signed-off-by: JP Kobryn --- kernel/cgroup/cgroup-internal.h | 4 ++-- kernel/cgroup/cgroup.c | 16 ++++++++++------ kernel/cgroup/rstat.c | 6 ++++-- 3 files changed, 16 insertions(+), 10 deletions(-) diff --git a/kernel/cgroup/cgroup-internal.h b/kernel/cgroup/cgroup-internal.h index c964dd7ff967..87d062baff90 100644 --- a/kernel/cgroup/cgroup-internal.h +++ b/kernel/cgroup/cgroup-internal.h @@ -269,8 +269,8 @@ int cgroup_task_count(const struct cgroup *cgrp); /* * rstat.c */ -int cgroup_rstat_init(struct cgroup *cgrp); -void cgroup_rstat_exit(struct cgroup *cgrp); +int cgroup_rstat_init(struct cgroup_subsys_state *css); +void cgroup_rstat_exit(struct cgroup_subsys_state *css); void cgroup_rstat_boot(void); void cgroup_base_stat_cputime_show(struct seq_file *seq); diff --git a/kernel/cgroup/cgroup.c b/kernel/cgroup/cgroup.c index fdddd5ec5f3c..848e09f433c0 100644 --- a/kernel/cgroup/cgroup.c +++ b/kernel/cgroup/cgroup.c @@ -1358,7 +1358,7 @@ static void cgroup_destroy_root(struct cgroup_root *root) cgroup_unlock(); - cgroup_rstat_exit(cgrp); + cgroup_rstat_exit(&cgrp->self); kernfs_destroy_root(root->kf_root); cgroup_free_root(root); } @@ -2132,7 +2132,7 @@ int cgroup_setup_root(struct cgroup_root *root, u16 ss_mask) if (ret) goto destroy_root; - ret = cgroup_rstat_init(root_cgrp); + ret = cgroup_rstat_init(&root_cgrp->self); if (ret) goto destroy_root; @@ -2174,7 +2174,7 @@ int cgroup_setup_root(struct cgroup_root *root, u16 ss_mask) goto out; exit_stats: - cgroup_rstat_exit(root_cgrp); + cgroup_rstat_exit(&root_cgrp->self); destroy_root: kernfs_destroy_root(root->kf_root); root->kf_root = NULL; @@ -5435,7 +5435,7 @@ static void css_free_rwork_fn(struct work_struct *work) cgroup_put(cgroup_parent(cgrp)); kernfs_put(cgrp->kn); psi_cgroup_free(cgrp); - cgroup_rstat_exit(cgrp); + cgroup_rstat_exit(css); kfree(cgrp); } else { /* @@ -5686,7 +5686,11 @@ static struct cgroup *cgroup_create(struct cgroup *parent, const char *name, if (ret) goto out_free_cgrp; - ret = cgroup_rstat_init(cgrp); + /* init self cgroup early so css->cgroup is valid within cgroup_rstat_init() + * note that this will go away in a subsequent patch in this series + */ + cgrp->self.cgroup = cgrp; + ret = cgroup_rstat_init(&cgrp->self); if (ret) goto out_cancel_ref; @@ -5779,7 +5783,7 @@ static struct cgroup *cgroup_create(struct cgroup *parent, const char *name, out_kernfs_remove: kernfs_remove(cgrp->kn); out_stat_exit: - cgroup_rstat_exit(cgrp); + cgroup_rstat_exit(&cgrp->self); out_cancel_ref: percpu_ref_exit(&cgrp->self.refcnt); out_free_cgrp: diff --git a/kernel/cgroup/rstat.c b/kernel/cgroup/rstat.c index 1b7ef8690a09..01a5c185b02a 100644 --- a/kernel/cgroup/rstat.c +++ b/kernel/cgroup/rstat.c @@ -390,8 +390,9 @@ void cgroup_rstat_flush_release(struct cgroup_subsys_state *css) __cgroup_rstat_unlock(css, -1); } -int cgroup_rstat_init(struct cgroup *cgrp) +int cgroup_rstat_init(struct cgroup_subsys_state *css) { + struct cgroup *cgrp = css->cgroup; int cpu; /* the root cgrp has rstat_cpu preallocated */ @@ -412,8 +413,9 @@ int cgroup_rstat_init(struct cgroup *cgrp) return 0; } -void cgroup_rstat_exit(struct cgroup *cgrp) +void cgroup_rstat_exit(struct cgroup_subsys_state *css) { + struct cgroup *cgrp = css->cgroup; int cpu; cgroup_rstat_flush(&cgrp->self); From patchwork Tue Dec 24 01:13:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: JP Kobryn X-Patchwork-Id: 13919571 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4016DE7718B for ; Tue, 24 Dec 2024 01:14:27 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9FC1C6B0089; Mon, 23 Dec 2024 20:14:22 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 99FBC6B008A; Mon, 23 Dec 2024 20:14:22 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7CD9B6B008C; Mon, 23 Dec 2024 20:14:22 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 5735C6B0089 for ; Mon, 23 Dec 2024 20:14:22 -0500 (EST) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 1DE7BB0EB2 for ; Tue, 24 Dec 2024 01:14:22 +0000 (UTC) X-FDA: 82928080368.27.6B8F4D7 Received: from mail-pl1-f173.google.com (mail-pl1-f173.google.com [209.85.214.173]) by imf08.hostedemail.com (Postfix) with ESMTP id 7FC9E160003 for ; Tue, 24 Dec 2024 01:13:54 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=KJiKan4b; spf=pass (imf08.hostedemail.com: domain of inwardvessel@gmail.com designates 209.85.214.173 as permitted sender) smtp.mailfrom=inwardvessel@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1735002831; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=13LzpkMgO0b7mnyCmlNhqmibgyHhCXjzFtjSKM6aNYI=; b=fZs8RMve6zJJIctruUJSc702JbmhPMra17pxX9ZU0Q4cjyUWSNKn9VEdQ0eSHcQxRTnbzn cZd2BNj4xZsWEcV1UJmYnw8vVyaIat2g6lfUfJY9ZUO6RRG8XDOydqoUJezmIpAPISW3AM DHK2w11S2YOWyvecJ93W0TIdMLgdgPI= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=KJiKan4b; spf=pass (imf08.hostedemail.com: domain of inwardvessel@gmail.com designates 209.85.214.173 as permitted sender) smtp.mailfrom=inwardvessel@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1735002831; a=rsa-sha256; cv=none; b=svyMl4gziypKQn8xuOZvg+5cppAkL9FVdtYWgVnHevLUFfb+EFFvysrWTjBc77ElAlwZoE ZG0gTLepYuR7qXc7S1I+/cDIdo7W5oSKbuDGap+xDuOQu4PFzk6YDW7EOYtsKUwoILWNMJ TVPHo2c8pNwj3r4i7uYfKFK9v0TETJw= Received: by mail-pl1-f173.google.com with SMTP id d9443c01a7336-2156e078563so42366785ad.2 for ; Mon, 23 Dec 2024 17:14:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1735002859; x=1735607659; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=13LzpkMgO0b7mnyCmlNhqmibgyHhCXjzFtjSKM6aNYI=; b=KJiKan4bQ1nHTQDoyMvBduRcRU8gSf48IzwrC78J7iunNZmGm0wsqo06eskrZU/TID X9l43vnbWrhCh7TAioQmVd1kO6wcz6n12GDP329pJRbYYezRtJu9yzpEwa1iBIIGlP0H b0QPuJphsm6W8nL7vW7V8Bde7k+j2/O4ld7CEuvgleaLPRw2b1tFyBZ0YjsyStLZAqh1 Wtb5XTDPIrMovOgy34Bo6McQYkXnMm7us/ZwYq1rrUWyQu9VeY7BUv1dhLlpL0hPBiqH zs6sBPdJlkAvUyvkPSwMDIRJysF0J+mB6tltxdDHsNPJw1GBkvGqJc0JTZ4IIIjsMNqZ 0dUg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1735002859; x=1735607659; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=13LzpkMgO0b7mnyCmlNhqmibgyHhCXjzFtjSKM6aNYI=; b=czLLpONsdHLNF8HctDf/4Dm69hYtrh80ih/eoJx9HHk8qKpmY3XlSOvDNhAaMUMOn2 vlrbREkJguszF1J+guiHHcqGcu1tcRts4MuFpMAIzIIAcv9+M8PcAORZacuq99kibhWE OvsJpRQsEoo1M7F2Y9qnReOKI4ez9YtmTbX1asu878BQ6uoAhMcU36yHOolUSaJzqqT3 g5QYJMLmtRd8sVVHXo5b4Few9YVWEmJFva8RtD3kZ4SFk3Ce+Tc5Wxwx4C9Hd+rRdQUS fUFOpY0bEGZFb8xenr/cY8c8q3Zi7tlWhzOR7u/IQYsgr5hBtapSuK1FIM+Wi2iEqmmL V/iQ== X-Gm-Message-State: AOJu0YxWGKszjPRsZ/u8A8R/9mjINWwvwJt0e6T1iUbSGLoo0mSeEbER KbxD7HpuC3oGLiOxsZK9HFSZISOvYlNmbOzUQhUkLVoM8z8Ppi74Snj3xQ== X-Gm-Gg: ASbGncupgpPsdrAhzSZ56i5AsZJxMd1BdIL4I9UGikZP1zK5JjizYqHo9lnlAardkL6 w/TE1/jSBeu1GOspCuxYZHR+qfhqt/YKbLfTonxH/Ivsgxfq45lgk4bkmla9/HqcHalFvjp8kYY gX+uaDMFtlKiEIF1K3LNiKp3LzkZOYoQfyc6KW7dYSOjwQMtmEK72zNJlQcpFg5+v52vKzmw4iY iSlM8zfr2x9CnzUBf+WC904dhKdNQTGAnKa/PDA9XK/IvAAYCrB/xe/M82g0WcEMydpZ7x/pKTm FTZuBwSe7HEE1rhlcg== X-Google-Smtp-Source: AGHT+IFl073fBGBueSkF2HB8ys5fQrX8Kxhz0xZtvlhkHeBvY8ScXCz8OIzgSfe3o9z9R/mbLdv+tQ== X-Received: by 2002:a17:903:41c8:b0:215:b058:289c with SMTP id d9443c01a7336-219e6e85beemr174581125ad.8.1735002858951; Mon, 23 Dec 2024 17:14:18 -0800 (PST) Received: from saturn.. (c-67-188-127-15.hsd1.ca.comcast.net. [67.188.127.15]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-219dc970c84sm79541255ad.58.2024.12.23.17.14.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Dec 2024 17:14:18 -0800 (PST) From: JP Kobryn To: shakeel.butt@linux.dev, hannes@cmpxchg.org, yosryahmed@google.com, akpm@linux-foundation.org Cc: linux-mm@kvack.org, cgroups@vger.kernel.org Subject: [PATCH 4/9 RFC] cgroup: split rstat from cgroup into separate css Date: Mon, 23 Dec 2024 17:13:57 -0800 Message-ID: <20241224011402.134009-5-inwardvessel@gmail.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20241224011402.134009-1-inwardvessel@gmail.com> References: <20241224011402.134009-1-inwardvessel@gmail.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 7FC9E160003 X-Rspamd-Server: rspam12 X-Stat-Signature: iyisu1khwepppezof6wigasis79pkc1p X-Rspam-User: X-HE-Tag: 1735002834-981497 X-HE-Meta: U2FsdGVkX18dcTJl0CsX1z9oDH08Of6nxQ7UlHBIaH3WhWNgg5jT/4nLfAolh1xwhiDBuIXTr22cMboiAxtuIJ525n7MhfC+wMiZC4rQj9fOnGEcq5I+iWt53h0CoA+LpjUTXqcpcWE/rMcDhDhyI3MrKQREM9jqedIs4glQ2vIXRgqcza5VxNTbEMLEHn+sCDvT5xcqbHLtyHPCvEcAQNiBF8eZ2CEbEdj/RaPiCnbMYMRZMo6miKhA8Y3LKVeOUuWJNEhv2Z7gmYwkKtRILIDsp8ZfFGEfxwk/R1mpdlK9Oyw6qERJ8apgkexiIUNm7MG21NjOcJBrHvDMSC/g8xr6R/24YudxBefb8ZfQu7vOblC7Urii6Pvkq/HB4iKcQ7aCB/MxcAFWhn88t2WhLtJRIa9o24vSIdK/zzAXD1y9aAgeBaVPWA3WXkZmc+2V0bSF+Tr++t6KqZyIYcUBiX3XiMbN4b2uDw6q6/hisXeFNcD1eIYRL/W94ufqrWV+kK1/xnrz/AbmR5Xed7SiS8OSAnlVpWAr9q0xtYo82LyXDe4IhOnqDKAhoo+5+T8Si2GVeLvBx76ZFLcTEuXMCbOtHzDhthdJD3Vy3JGKWR2bfXQqIvcS9rK2uPr9zoHhHMiEz64lmi4WwRjjuEglzBiMZu/aOFtrlVLRhjMrA74OeMkwuh7paq1UqJY9coAnH6RxNYbZtjJJSaX9bXXBrrp8yV2u0yq6vco0Na8vE98+FYXwu17E0YHbPaBGbo/VM8Tms1EdBe/MJELOgH7EX/3Y4QWIAbJM3tnLvuPT1Y7KN1MW2a6zNUL5sT9ANaYTBWj7AYScNceHz8ChLI85Ph5t+s1be01OX8gdDwJrAA5IzkXp8W7qY1+nBq+T8uzgIqHrVvpdsEifJcdwuNzIMGOL0225SbqcrD2rMoMLiTeyBObkeEvedmRbWXlgdgDn/TIbPxUTrsC8S6Onftx VzUYUo75 KKrbRfIBH9DL7CtWzvho45n6Wq/VVcHPCoyoKGNWh0/vP+qpZjHCmafkxXI0V4aYG+XOUN8S1ogdvZxDLOefoQ7s1by2oqHNP7IVWR9Ejy3/4q8kfy31lzYxbLYXVXBCKjqevejNJtWXtoHv15XuJ4SbSUuCSj6DlGIIXlx85oC3SLEQRTKbF7VVs4EO2nNoi23tQVpLM12HzjeVs9c6P7vMt2T/gR7/to8ADDKNaarGF6S14bheb4Id23sRT+qqXyX1Y3NJ9YW6zuhnzhWMXVueBZbgLpdD1fyj+J+QCYu8dTzsNF05J0aVnJpf0GED6/lg2Ol+Aewl1++VO/jM2q0O7YBH5EzxQq7Z+j+aGqdLzvGQFiu1qQf61jg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Move the rstat entities off of the cgroup struct and onto the cgroup_subsys_state struct. Adjust related code to reflect this new ownership. Signed-off-by: JP Kobryn --- include/linux/cgroup-defs.h | 40 ++++++++-------- kernel/cgroup/cgroup.c | 65 ++++++++++++++++++-------- kernel/cgroup/rstat.c | 92 ++++++++++++++++++------------------- 3 files changed, 111 insertions(+), 86 deletions(-) diff --git a/include/linux/cgroup-defs.h b/include/linux/cgroup-defs.h index 1b20d2d8ef7c..1932f8ae7995 100644 --- a/include/linux/cgroup-defs.h +++ b/include/linux/cgroup-defs.h @@ -180,6 +180,24 @@ struct cgroup_subsys_state { struct list_head sibling; struct list_head children; + /* per-cpu recursive resource statistics */ + struct cgroup_rstat_cpu __percpu *rstat_cpu; + struct list_head rstat_css_list; + + /* + * Add padding to separate the read mostly rstat_cpu and + * rstat_css_list into a different cacheline from the following + * rstat_flush_next and *bstat fields which can have frequent updates. + */ + CACHELINE_PADDING(_pad_); + + /* + * A singly-linked list of cgroup structures to be rstat flushed. + * This is a scratch field to be used exclusively by + * cgroup_rstat_flush_locked() and protected by cgroup_rstat_lock. + */ + struct cgroup_subsys_state *rstat_flush_next; + /* flush target list anchored at cgrp->rstat_css_list */ struct list_head rstat_css_node; @@ -389,8 +407,8 @@ struct cgroup_rstat_cpu { * * Protected by per-cpu cgroup_rstat_cpu_lock. */ - struct cgroup *updated_children; /* terminated by self cgroup */ - struct cgroup *updated_next; /* NULL iff not on the list */ + struct cgroup_subsys_state *updated_children; /* terminated by self cgroup */ + struct cgroup_subsys_state *updated_next; /* NULL iff not on the list */ }; struct cgroup_freezer_state { @@ -516,24 +534,6 @@ struct cgroup { struct cgroup *dom_cgrp; struct cgroup *old_dom_cgrp; /* used while enabling threaded */ - /* per-cpu recursive resource statistics */ - struct cgroup_rstat_cpu __percpu *rstat_cpu; - struct list_head rstat_css_list; - - /* - * Add padding to separate the read mostly rstat_cpu and - * rstat_css_list into a different cacheline from the following - * rstat_flush_next and *bstat fields which can have frequent updates. - */ - CACHELINE_PADDING(_pad_); - - /* - * A singly-linked list of cgroup structures to be rstat flushed. - * This is a scratch field to be used exclusively by - * cgroup_rstat_flush_locked() and protected by cgroup_rstat_lock. - */ - struct cgroup *rstat_flush_next; - /* cgroup basic resource statistics */ struct cgroup_base_stat last_bstat; struct cgroup_base_stat bstat; diff --git a/kernel/cgroup/cgroup.c b/kernel/cgroup/cgroup.c index 848e09f433c0..96a2d15fe5e9 100644 --- a/kernel/cgroup/cgroup.c +++ b/kernel/cgroup/cgroup.c @@ -164,7 +164,7 @@ static struct static_key_true *cgroup_subsys_on_dfl_key[] = { static DEFINE_PER_CPU(struct cgroup_rstat_cpu, cgrp_dfl_root_rstat_cpu); /* the default hierarchy */ -struct cgroup_root cgrp_dfl_root = { .cgrp.rstat_cpu = &cgrp_dfl_root_rstat_cpu }; +struct cgroup_root cgrp_dfl_root = { .cgrp.self.rstat_cpu = &cgrp_dfl_root_rstat_cpu }; EXPORT_SYMBOL_GPL(cgrp_dfl_root); /* @@ -1826,6 +1826,7 @@ int rebind_subsystems(struct cgroup_root *dst_root, u16 ss_mask) struct cgroup_root *src_root = ss->root; struct cgroup *scgrp = &src_root->cgrp; struct cgroup_subsys_state *css = cgroup_css(scgrp, ss); + struct cgroup_subsys_state *dcss = cgroup_css(dcgrp, ss); struct css_set *cset, *cset_pos; struct css_task_iter *it; @@ -1867,7 +1868,7 @@ int rebind_subsystems(struct cgroup_root *dst_root, u16 ss_mask) list_del_rcu(&css->rstat_css_node); synchronize_rcu(); list_add_rcu(&css->rstat_css_node, - &dcgrp->rstat_css_list); + &dcss->rstat_css_list); } /* default hierarchy doesn't enable controllers by default */ @@ -2052,7 +2053,6 @@ static void init_cgroup_housekeeping(struct cgroup *cgrp) cgrp->dom_cgrp = cgrp; cgrp->max_descendants = INT_MAX; cgrp->max_depth = INT_MAX; - INIT_LIST_HEAD(&cgrp->rstat_css_list); prev_cputime_init(&cgrp->prev_cputime); for_each_subsys(ss, ssid) @@ -2088,7 +2088,8 @@ int cgroup_setup_root(struct cgroup_root *root, u16 ss_mask) struct cgroup *root_cgrp = &root->cgrp; struct kernfs_syscall_ops *kf_sops; struct css_set *cset; - int i, ret; + struct cgroup_subsys *ss; + int i, ret, ssid; lockdep_assert_held(&cgroup_mutex); @@ -2132,10 +2133,6 @@ int cgroup_setup_root(struct cgroup_root *root, u16 ss_mask) if (ret) goto destroy_root; - ret = cgroup_rstat_init(&root_cgrp->self); - if (ret) - goto destroy_root; - ret = rebind_subsystems(root, ss_mask); if (ret) goto exit_stats; @@ -2174,7 +2171,10 @@ int cgroup_setup_root(struct cgroup_root *root, u16 ss_mask) goto out; exit_stats: - cgroup_rstat_exit(&root_cgrp->self); + for_each_subsys(ss, ssid) { + struct cgroup_subsys_state *css = init_css_set.subsys[ssid]; + cgroup_rstat_exit(css); + } destroy_root: kernfs_destroy_root(root->kf_root); root->kf_root = NULL; @@ -3229,6 +3229,10 @@ static int cgroup_apply_control_enable(struct cgroup *cgrp) int ssid, ret; cgroup_for_each_live_descendant_pre(dsct, d_css, cgrp) { + ret = cgroup_rstat_init(&dsct->self); + if (ret) + return ret; + for_each_subsys(ss, ssid) { struct cgroup_subsys_state *css = cgroup_css(dsct, ss); @@ -3239,6 +3243,10 @@ static int cgroup_apply_control_enable(struct cgroup *cgrp) css = css_create(dsct, ss); if (IS_ERR(css)) return PTR_ERR(css); + + ret = cgroup_rstat_init(css); + if (ret) + goto err_free_css; } WARN_ON_ONCE(percpu_ref_is_dying(&css->refcnt)); @@ -3252,6 +3260,20 @@ static int cgroup_apply_control_enable(struct cgroup *cgrp) } return 0; + +err_free_css: + cgroup_for_each_live_descendant_pre(dsct, d_css, cgrp) { + cgroup_rstat_exit(&dsct->self); + + for_each_subsys(ss, ssid) { + struct cgroup_subsys_state *css = cgroup_css(dsct, ss); + + if (css != &dsct->self) + cgroup_rstat_exit(css); + } + } + + return ret; } /** @@ -5403,6 +5425,7 @@ static void css_free_rwork_fn(struct work_struct *work) struct cgroup_subsys_state, destroy_rwork); struct cgroup_subsys *ss = css->ss; struct cgroup *cgrp = css->cgroup; + int ssid; percpu_ref_exit(&css->refcnt); @@ -5435,7 +5458,12 @@ static void css_free_rwork_fn(struct work_struct *work) cgroup_put(cgroup_parent(cgrp)); kernfs_put(cgrp->kn); psi_cgroup_free(cgrp); - cgroup_rstat_exit(css); + for_each_subsys(ss, ssid) { + struct cgroup_subsys_state *css = cgrp->subsys[ssid]; + + if (css) + cgroup_rstat_exit(css); + } kfree(cgrp); } else { /* @@ -5541,6 +5569,7 @@ static void init_and_link_css(struct cgroup_subsys_state *css, css->id = -1; INIT_LIST_HEAD(&css->sibling); INIT_LIST_HEAD(&css->children); + INIT_LIST_HEAD(&css->rstat_css_list); INIT_LIST_HEAD(&css->rstat_css_node); css->serial_nr = css_serial_nr_next++; atomic_set(&css->online_cnt, 0); @@ -5551,7 +5580,7 @@ static void init_and_link_css(struct cgroup_subsys_state *css, } if (ss->css_rstat_flush) - list_add_rcu(&css->rstat_css_node, &cgrp->rstat_css_list); + list_add_rcu(&css->rstat_css_node, &css->rstat_css_list); BUG_ON(cgroup_css(cgrp, ss)); } @@ -5686,14 +5715,6 @@ static struct cgroup *cgroup_create(struct cgroup *parent, const char *name, if (ret) goto out_free_cgrp; - /* init self cgroup early so css->cgroup is valid within cgroup_rstat_init() - * note that this will go away in a subsequent patch in this series - */ - cgrp->self.cgroup = cgrp; - ret = cgroup_rstat_init(&cgrp->self); - if (ret) - goto out_cancel_ref; - /* create the directory */ kn = kernfs_create_dir_ns(parent->kn, name, mode, current_fsuid(), current_fsgid(), @@ -5784,7 +5805,6 @@ static struct cgroup *cgroup_create(struct cgroup *parent, const char *name, kernfs_remove(cgrp->kn); out_stat_exit: cgroup_rstat_exit(&cgrp->self); -out_cancel_ref: percpu_ref_exit(&cgrp->self.refcnt); out_free_cgrp: kfree(cgrp); @@ -6189,6 +6209,8 @@ int __init cgroup_init(void) cgroup_unlock(); for_each_subsys(ss, ssid) { + struct cgroup_subsys_state *css; + if (ss->early_init) { struct cgroup_subsys_state *css = init_css_set.subsys[ss->id]; @@ -6200,6 +6222,9 @@ int __init cgroup_init(void) cgroup_init_subsys(ss, false); } + css = init_css_set.subsys[ss->id]; + BUG_ON(cgroup_rstat_init(css)); + list_add_tail(&init_css_set.e_cset_node[ssid], &cgrp_dfl_root.cgrp.e_csets[ssid]); diff --git a/kernel/cgroup/rstat.c b/kernel/cgroup/rstat.c index 01a5c185b02a..4381eb9ac426 100644 --- a/kernel/cgroup/rstat.c +++ b/kernel/cgroup/rstat.c @@ -14,9 +14,10 @@ static DEFINE_PER_CPU(raw_spinlock_t, cgroup_rstat_cpu_lock); static void cgroup_base_stat_flush(struct cgroup *cgrp, int cpu); -static struct cgroup_rstat_cpu *cgroup_rstat_cpu(struct cgroup *cgrp, int cpu) +static struct cgroup_rstat_cpu *css_rstat_cpu( + struct cgroup_subsys_state *css, int cpu) { - return per_cpu_ptr(cgrp->rstat_cpu, cpu); + return per_cpu_ptr(css->rstat_cpu, cpu); } /* @@ -96,15 +97,16 @@ __bpf_kfunc void cgroup_rstat_updated(struct cgroup_subsys_state *css, int cpu) * instead of NULL, we can tell whether @cgrp is on the list by * testing the next pointer for NULL. */ - if (data_race(cgroup_rstat_cpu(cgrp, cpu)->updated_next)) + if (data_race(css_rstat_cpu(css, cpu)->updated_next)) return; flags = _cgroup_rstat_cpu_lock(cpu_lock, cpu, cgrp, true); /* put @cgrp and all ancestors on the corresponding updated lists */ while (true) { - struct cgroup_rstat_cpu *rstatc = cgroup_rstat_cpu(cgrp, cpu); - struct cgroup *parent = cgroup_parent(cgrp); + struct cgroup_rstat_cpu *rstatc = css_rstat_cpu(css, cpu); + struct cgroup_subsys_state *parent = css->parent +; struct cgroup_rstat_cpu *prstatc; /* @@ -116,15 +118,15 @@ __bpf_kfunc void cgroup_rstat_updated(struct cgroup_subsys_state *css, int cpu) /* Root has no parent to link it to, but mark it busy */ if (!parent) { - rstatc->updated_next = cgrp; + rstatc->updated_next = css; break; } - prstatc = cgroup_rstat_cpu(parent, cpu); + prstatc = css_rstat_cpu(parent, cpu); rstatc->updated_next = prstatc->updated_children; - prstatc->updated_children = cgrp; + prstatc->updated_children = css; - cgrp = parent; + css = parent; } _cgroup_rstat_cpu_unlock(cpu_lock, cpu, cgrp, flags, true); @@ -142,12 +144,13 @@ __bpf_kfunc void cgroup_rstat_updated(struct cgroup_subsys_state *css, int cpu) * into a singly linked list built from the tail backward like "pushing" * cgroups into a stack. The root is pushed by the caller. */ -static struct cgroup *cgroup_rstat_push_children(struct cgroup *head, - struct cgroup *child, int cpu) +static struct cgroup_subsys_state *cgroup_rstat_push_children( + struct cgroup_subsys_state *head, + struct cgroup_subsys_state *child, int cpu) { - struct cgroup *chead = child; /* Head of child cgroup level */ - struct cgroup *ghead = NULL; /* Head of grandchild cgroup level */ - struct cgroup *parent, *grandchild; + struct cgroup_subsys_state *chead = child; /* Head of child cgroup level */ + struct cgroup_subsys_state *ghead = NULL; /* Head of grandchild cgroup level */ + struct cgroup_subsys_state *parent, *grandchild; struct cgroup_rstat_cpu *crstatc; child->rstat_flush_next = NULL; @@ -156,13 +159,13 @@ static struct cgroup *cgroup_rstat_push_children(struct cgroup *head, while (chead) { child = chead; chead = child->rstat_flush_next; - parent = cgroup_parent(child); + parent = child->parent; /* updated_next is parent cgroup terminated */ while (child != parent) { child->rstat_flush_next = head; head = child; - crstatc = cgroup_rstat_cpu(child, cpu); + crstatc = css_rstat_cpu(child, cpu); grandchild = crstatc->updated_children; if (grandchild != child) { /* Push the grand child to the next level */ @@ -201,16 +204,15 @@ static struct cgroup *cgroup_rstat_push_children(struct cgroup *head, * within the children list and terminated by the parent cgroup. An exception * here is the cgroup root whose updated_next can be self terminated. */ -static struct cgroup *cgroup_rstat_updated_list(struct cgroup_subsys_state *root_css, - int cpu) +static struct cgroup_subsys_state *cgroup_rstat_updated_list( + struct cgroup_subsys_state *root, int cpu) { - struct cgroup *root = root_css->cgroup; raw_spinlock_t *cpu_lock = per_cpu_ptr(&cgroup_rstat_cpu_lock, cpu); - struct cgroup_rstat_cpu *rstatc = cgroup_rstat_cpu(root, cpu); - struct cgroup *head = NULL, *parent, *child; + struct cgroup_rstat_cpu *rstatc = css_rstat_cpu(root, cpu); + struct cgroup_subsys_state *head = NULL, *parent, *child; unsigned long flags; - flags = _cgroup_rstat_cpu_lock(cpu_lock, cpu, root, false); + flags = _cgroup_rstat_cpu_lock(cpu_lock, cpu, root->cgroup, false); /* Return NULL if this subtree is not on-list */ if (!rstatc->updated_next) @@ -220,17 +222,17 @@ static struct cgroup *cgroup_rstat_updated_list(struct cgroup_subsys_state *root * Unlink @root from its parent. As the updated_children list is * singly linked, we have to walk it to find the removal point. */ - parent = cgroup_parent(root); + parent = root->parent; if (parent) { struct cgroup_rstat_cpu *prstatc; - struct cgroup **nextp; + struct cgroup_subsys_state **nextp; - prstatc = cgroup_rstat_cpu(parent, cpu); + prstatc = css_rstat_cpu(parent, cpu); nextp = &prstatc->updated_children; while (*nextp != root) { struct cgroup_rstat_cpu *nrstatc; - nrstatc = cgroup_rstat_cpu(*nextp, cpu); + nrstatc = css_rstat_cpu(*nextp, cpu); WARN_ON_ONCE(*nextp == parent); nextp = &nrstatc->updated_next; } @@ -247,7 +249,7 @@ static struct cgroup *cgroup_rstat_updated_list(struct cgroup_subsys_state *root if (child != root) head = cgroup_rstat_push_children(head, child, cpu); unlock_ret: - _cgroup_rstat_cpu_unlock(cpu_lock, cpu, root, flags, false); + _cgroup_rstat_cpu_unlock(cpu_lock, cpu, root->cgroup, flags, false); return head; } @@ -316,13 +318,13 @@ static void cgroup_rstat_flush_locked(struct cgroup_subsys_state *css) lockdep_assert_held(&cgroup_rstat_lock); for_each_possible_cpu(cpu) { - struct cgroup *pos = cgroup_rstat_updated_list(css, cpu); + struct cgroup_subsys_state *pos = cgroup_rstat_updated_list(css, cpu); for (; pos; pos = pos->rstat_flush_next) { struct cgroup_subsys_state *css_iter; - cgroup_base_stat_flush(pos, cpu); - bpf_rstat_flush(pos, cgroup_parent(pos), cpu); + cgroup_base_stat_flush(pos->cgroup, cpu); + bpf_rstat_flush(pos->cgroup, cgroup_parent(pos->cgroup), cpu); rcu_read_lock(); list_for_each_entry_rcu(css_iter, &pos->rstat_css_list, @@ -392,21 +394,20 @@ void cgroup_rstat_flush_release(struct cgroup_subsys_state *css) int cgroup_rstat_init(struct cgroup_subsys_state *css) { - struct cgroup *cgrp = css->cgroup; int cpu; - /* the root cgrp has rstat_cpu preallocated */ - if (!cgrp->rstat_cpu) { - cgrp->rstat_cpu = alloc_percpu(struct cgroup_rstat_cpu); - if (!cgrp->rstat_cpu) + /* the root cgrp css has rstat_cpu preallocated */ + if (!css->rstat_cpu) { + css->rstat_cpu = alloc_percpu(struct cgroup_rstat_cpu); + if (!css->rstat_cpu) return -ENOMEM; } /* ->updated_children list is self terminated */ for_each_possible_cpu(cpu) { - struct cgroup_rstat_cpu *rstatc = cgroup_rstat_cpu(cgrp, cpu); + struct cgroup_rstat_cpu *rstatc = css_rstat_cpu(css, cpu); - rstatc->updated_children = cgrp; + rstatc->updated_children = css; u64_stats_init(&rstatc->bsync); } @@ -415,22 +416,21 @@ int cgroup_rstat_init(struct cgroup_subsys_state *css) void cgroup_rstat_exit(struct cgroup_subsys_state *css) { - struct cgroup *cgrp = css->cgroup; int cpu; - cgroup_rstat_flush(&cgrp->self); + cgroup_rstat_flush(css); /* sanity check */ for_each_possible_cpu(cpu) { - struct cgroup_rstat_cpu *rstatc = cgroup_rstat_cpu(cgrp, cpu); + struct cgroup_rstat_cpu *rstatc = css_rstat_cpu(css, cpu); - if (WARN_ON_ONCE(rstatc->updated_children != cgrp) || + if (WARN_ON_ONCE(rstatc->updated_children != css) || WARN_ON_ONCE(rstatc->updated_next)) return; } - free_percpu(cgrp->rstat_cpu); - cgrp->rstat_cpu = NULL; + free_percpu(css->rstat_cpu); + css->rstat_cpu = NULL; } void __init cgroup_rstat_boot(void) @@ -471,7 +471,7 @@ static void cgroup_base_stat_sub(struct cgroup_base_stat *dst_bstat, static void cgroup_base_stat_flush(struct cgroup *cgrp, int cpu) { - struct cgroup_rstat_cpu *rstatc = cgroup_rstat_cpu(cgrp, cpu); + struct cgroup_rstat_cpu *rstatc = css_rstat_cpu(&cgrp->self, cpu); struct cgroup *parent = cgroup_parent(cgrp); struct cgroup_rstat_cpu *prstatc; struct cgroup_base_stat delta; @@ -501,7 +501,7 @@ static void cgroup_base_stat_flush(struct cgroup *cgrp, int cpu) cgroup_base_stat_add(&cgrp->last_bstat, &delta); delta = rstatc->subtree_bstat; - prstatc = cgroup_rstat_cpu(parent, cpu); + prstatc = css_rstat_cpu(&parent->self, cpu); cgroup_base_stat_sub(&delta, &rstatc->last_subtree_bstat); cgroup_base_stat_add(&prstatc->subtree_bstat, &delta); cgroup_base_stat_add(&rstatc->last_subtree_bstat, &delta); @@ -513,7 +513,7 @@ cgroup_base_stat_cputime_account_begin(struct cgroup *cgrp, unsigned long *flags { struct cgroup_rstat_cpu *rstatc; - rstatc = get_cpu_ptr(cgrp->rstat_cpu); + rstatc = get_cpu_ptr(cgrp->self.rstat_cpu); *flags = u64_stats_update_begin_irqsave(&rstatc->bsync); return rstatc; } From patchwork Tue Dec 24 01:13:58 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: JP Kobryn X-Patchwork-Id: 13919572 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 06A97E7718E for ; Tue, 24 Dec 2024 01:14:30 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 201356B008A; Mon, 23 Dec 2024 20:14:24 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 1BA556B0092; Mon, 23 Dec 2024 20:14:24 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EF87E6B0093; Mon, 23 Dec 2024 20:14:23 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id BDFFE6B008A for ; Mon, 23 Dec 2024 20:14:23 -0500 (EST) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 769D1C1A04 for ; Tue, 24 Dec 2024 01:14:23 +0000 (UTC) X-FDA: 82928080200.08.D88A72A Received: from mail-pl1-f180.google.com (mail-pl1-f180.google.com [209.85.214.180]) by imf08.hostedemail.com (Postfix) with ESMTP id DD7DD16000D for ; Tue, 24 Dec 2024 01:13:55 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=OkVQboGw; spf=pass (imf08.hostedemail.com: domain of inwardvessel@gmail.com designates 209.85.214.180 as permitted sender) smtp.mailfrom=inwardvessel@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1735002843; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=EpQSbn6C6AIKp7YKs7miT4jgNyhtNHCB16iSRmc/g1c=; b=yppP9ogyzO4GuZutDGjwJmUBHNjcJL3fiqUcKExFqWafiLqQBLGTkqijXwKpzKP7TvnlF1 BXHqS50FSc3/6poi14cx69XDjY73KCi8alBPSLaEefn7s/ouMoTVRGjRptVbkZu8LQrrdW AW6uTakDS8jtY6zSu/JONSq7BwN4I+s= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1735002843; a=rsa-sha256; cv=none; b=eo5jFb0YKYgKsiE4aIG8q/fW7roH7MCAFGytYCy1MWI2MNmZ81Zejb/bMnvo1LTSXSk5zu PPkG4ifX8JyNP8tuXBUnunHq3dWGzPXt4WOCkTWYyapA4jAxzUcG9FQnHpHJ+4bUW0TwQN rFsFKsw7XlnSNy4l//yQkJsk8u9yJrA= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=OkVQboGw; spf=pass (imf08.hostedemail.com: domain of inwardvessel@gmail.com designates 209.85.214.180 as permitted sender) smtp.mailfrom=inwardvessel@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-pl1-f180.google.com with SMTP id d9443c01a7336-21644aca3a0so52391655ad.3 for ; Mon, 23 Dec 2024 17:14:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1735002860; x=1735607660; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=EpQSbn6C6AIKp7YKs7miT4jgNyhtNHCB16iSRmc/g1c=; b=OkVQboGwVaJctbaw4hug58/+bEjAsiUfGs4t/m+aQsQwBjosOX3q2dUPVNH3X1KLew dZtHGlUHt/ZPvjNvIrIi5l76J+9wri8l2v4tWSJdSM+766HNleFzrGcWiZmC24PpRKYB 15myHRgFUtrt0vymZRDGWOu/++jGV4QDnRlApBFI+J8SFWCcvtllDjuGvqZKkOGVQ9Wh p/1dqo271C0wEtpyKQ2Tj//AYtvs5y+7Ht1cfVXJXklntqU5B0q4iEyZNk1LQQicIrbM Et52Zzdu6uYD/JO6kTl3N9h348fow8Fmr7sCumm4Pxa7FkcLsx9yTsRP+QV35Svqig5T 4N+w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1735002860; x=1735607660; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=EpQSbn6C6AIKp7YKs7miT4jgNyhtNHCB16iSRmc/g1c=; b=Qpd+672tX9bh32dhY2PiPqvaOfF3IVQ5pzwBRKxRlr/c+w01WMS9xW15HZAty+sCrF SXVct2wG7qEN989d5D+sTkyjJTnZxAMT7HEFCY90DFZGxKU+xJFL+O3OJAqINUL3Avh/ TVyAr6V6tsHf45Fq3zMovyuiVnssVKctpn1paKOzrN/KegByg1NJ+QYjJeGn2TePhQre vYMBfJ/hh4FeY/LZ0C0bFlr2+kP/OiwvX//ebuHCWzMXJKOAZKA9OfVIIsuDfKztFeAw +hYy1v4iKMPEmJGCSGcuYQtsJNIi5D6H6xbhliGI9hJmnYe8fn4jJ563dUtdPUZybMUB bbAA== X-Gm-Message-State: AOJu0YzS45h8hmXoMhsFHceH4dHHBwFIloSISQL562pyAC5OpejKDkQ7 IWDwi7FuhhKFGprVUFDlZ9R9EfCnK6ZGYTOjZYf2IXSFQQairGab X-Gm-Gg: ASbGnct5bb+WCkXymt+O5yXruvOTwHsPcTNGipqiywXbCPEQ4prwst73qNMBNjDbmTL 3ACdUXM9MBAjBZ3mcV+6vJ5bTeHCZNErKXU80Jh2yf920zMNh0FFyQzDTfSHI0WsT4TqUhM5Hzr VZoBFV/1JHUvQfWvVVSYMbEyvS38isbwfmVLub+sCG8lOiHHpK+YaxhIYhR4N89g0NaGjiLYXzi sE4XBw0Ta5q52ZvMg8C9uEpGi65ynscm6+nb9CDHf7fLdoHZgThNplQzy1uexnNrr9lbh9zdTfG w2nNxGtsaAhIHk2IuQ== X-Google-Smtp-Source: AGHT+IFwR7wXxhzrgfdQceACkEtKrEhPAYVpgAqlRJPF2+jNIw5EIHwhSSCWtl13waEhVD43uo5AMQ== X-Received: by 2002:a17:902:c943:b0:215:72aa:693f with SMTP id d9443c01a7336-219e6e8bb89mr229405165ad.9.1735002860313; Mon, 23 Dec 2024 17:14:20 -0800 (PST) Received: from saturn.. (c-67-188-127-15.hsd1.ca.comcast.net. [67.188.127.15]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-219dc970c84sm79541255ad.58.2024.12.23.17.14.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Dec 2024 17:14:19 -0800 (PST) From: JP Kobryn To: shakeel.butt@linux.dev, hannes@cmpxchg.org, yosryahmed@google.com, akpm@linux-foundation.org Cc: linux-mm@kvack.org, cgroups@vger.kernel.org Subject: [PATCH 5/9 RFC] cgroup: separate locking between base css and others Date: Mon, 23 Dec 2024 17:13:58 -0800 Message-ID: <20241224011402.134009-6-inwardvessel@gmail.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20241224011402.134009-1-inwardvessel@gmail.com> References: <20241224011402.134009-1-inwardvessel@gmail.com> MIME-Version: 1.0 X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: DD7DD16000D X-Stat-Signature: p54u5nhog3e8b8s6zwomipybnimdyzku X-Rspam-User: X-HE-Tag: 1735002835-383986 X-HE-Meta: U2FsdGVkX18jhDrR1Nbza5cKe+k5ujotSCzw6sRaT+y81fvupQdjnACHpQf8kXuO2DF1tuxwRK4rwhws+FGv1rgH9x2t0Aqsx3tP/HR7Iit3AkY2abHPztXSkCyb47RVBGQ1TDJb5fxITeqM1VwnW+dYGicZmEkPJ7N5WKyaRyQWqaWjdMdeApOdq9sdXI8N6KPeBS6WXqzVLDNXAhbdbK3U0WRm0sXmW+SeZSSmWK2MaEY4cuqYftK8S7xsvtGZza0TP1kK9JBewrtNY5gavT31yDVoqDvLgGNEzJHs641e+duijZllY92shCmaIgu+t0G7mjYX+s+yXoMVVbyUXl/dPi2j/TKLqo1mcyttcU/GuQ1/WtpuPu3HYriAhtBfbIXzLD2XrsSHlq4DNfBr6G3iu4ngP31V9S+Q5DRp0keL0dgBdkzkmcYTIu+LWtXzfJmlX6zS+BLEhI3f8aJiYL75WGhNwoMRAxGmhmDYVd+MRxI6ghppSgThBmTRyndqDIb4Gg2ASnBogyjeDPGVgyjG0qn/pD/SySnUkHqnhS9/fixHFB4YIU2BSVrAwcYPM/1/6gkfIr7/B5an9MMJy82QWE6lzWXYYuGFuk6lQ4OS0VRs0vdkTsCd0MA/cK5hqokeuLicuLTxzBI7wCmjp3zSwBT86MjysZx2/WiPZ5FalCglDVLPczsF6Q3+OdE8qY4mUhoMXrEooWXQNmFc384JvBEsx3pUb+axDfJa5el5KIU+XF+fD+IzZImMGsXJPHci/0hb4QyOrD+KrlITPlooNAc5MyRJSqOoULviVll4QJsIZa5CsofNC+GxX0fmyCfi6VjJyB+SAggFXbgrnuNo3hTrUItcAdSqB+kocIK7V+w9AxWZbyHgTx01sGCGHfxqlujTWn5KMHLWGQ9ReqL2j0bxyyhKB+cLqQtTsM4atc3Ejrn+RaEZ57zXNQkUnpytwRSKxpYF+bnVEEe Tm5PQO/6 468suCbIUq1MVp8+Pae4+5GI9LAfgm03o0q0L6n68eNNoEOuThQbkv3bG1ouphzxRmE5Z6N8Q3v1a4URLrW6UFKThdlMHKyGXOEjxgMhPMWyGc533BQfRp2gWjvUbbZ9Cb/ZIgeK0DX746qrMBHXhDuscGHfy7Y1ZNjxzHZl3Ko1HRlBbDABpdZKDRD6Gvtv5K9BoOuCZHKZnAMvlll92I+nxbNZFP58VM+dB47fMdPLyP/Vvga6bwH/6j4kckzSrcO952WIgWPL6BkrU4RnxPO+LeXgHvSP51XCDPquVI60frM+Zi8Fj4gG82y+KgtRYRQGE5OBNE7xY5E1XS7tfoEOC4O+5ufnBwM1CRAkjYO8AmNywPGe5138bDQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.032539, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Separate locks can be used to eliminate contention across subsystems that make use of rstat. The base stats also get their own lock. Where applicable, check for the existence of a subsystem pointer to determine if the given cgroup_subsys_state is the base css or not. Signed-off-by: JP Kobryn --- include/linux/cgroup-defs.h | 2 + kernel/cgroup/rstat.c | 92 +++++++++++++++++++++++++------------ 2 files changed, 65 insertions(+), 29 deletions(-) diff --git a/include/linux/cgroup-defs.h b/include/linux/cgroup-defs.h index 1932f8ae7995..4d87519ff023 100644 --- a/include/linux/cgroup-defs.h +++ b/include/linux/cgroup-defs.h @@ -790,6 +790,8 @@ struct cgroup_subsys { * specifies the mask of subsystems that this one depends on. */ unsigned int depends_on; + + spinlock_t rstat_lock; }; extern struct percpu_rw_semaphore cgroup_threadgroup_rwsem; diff --git a/kernel/cgroup/rstat.c b/kernel/cgroup/rstat.c index 4381eb9ac426..958bdccf0359 100644 --- a/kernel/cgroup/rstat.c +++ b/kernel/cgroup/rstat.c @@ -9,8 +9,9 @@ #include -static DEFINE_SPINLOCK(cgroup_rstat_lock); -static DEFINE_PER_CPU(raw_spinlock_t, cgroup_rstat_cpu_lock); +static DEFINE_SPINLOCK(cgroup_rstat_base_lock); +static DEFINE_PER_CPU(raw_spinlock_t, cgroup_rstat_base_cpu_lock); +static DEFINE_PER_CPU(raw_spinlock_t, cgroup_rstat_cpu_lock[CGROUP_SUBSYS_COUNT]); static void cgroup_base_stat_flush(struct cgroup *cgrp, int cpu); @@ -86,7 +87,7 @@ void _cgroup_rstat_cpu_unlock(raw_spinlock_t *cpu_lock, int cpu, __bpf_kfunc void cgroup_rstat_updated(struct cgroup_subsys_state *css, int cpu) { struct cgroup *cgrp = css->cgroup; - raw_spinlock_t *cpu_lock = per_cpu_ptr(&cgroup_rstat_cpu_lock, cpu); + raw_spinlock_t *cpu_lock; unsigned long flags; /* @@ -100,6 +101,11 @@ __bpf_kfunc void cgroup_rstat_updated(struct cgroup_subsys_state *css, int cpu) if (data_race(css_rstat_cpu(css, cpu)->updated_next)) return; + if (css->ss) + cpu_lock = per_cpu_ptr(&cgroup_rstat_cpu_lock[css->ss->id], cpu); + else + cpu_lock = per_cpu_ptr(&cgroup_rstat_base_cpu_lock, cpu); + flags = _cgroup_rstat_cpu_lock(cpu_lock, cpu, cgrp, true); /* put @cgrp and all ancestors on the corresponding updated lists */ @@ -207,11 +213,16 @@ static struct cgroup_subsys_state *cgroup_rstat_push_children( static struct cgroup_subsys_state *cgroup_rstat_updated_list( struct cgroup_subsys_state *root, int cpu) { - raw_spinlock_t *cpu_lock = per_cpu_ptr(&cgroup_rstat_cpu_lock, cpu); struct cgroup_rstat_cpu *rstatc = css_rstat_cpu(root, cpu); struct cgroup_subsys_state *head = NULL, *parent, *child; + raw_spinlock_t *cpu_lock; unsigned long flags; + if (root->ss) + cpu_lock = per_cpu_ptr(&cgroup_rstat_cpu_lock[root->ss->id], cpu); + else + cpu_lock = per_cpu_ptr(&cgroup_rstat_base_cpu_lock, cpu); + flags = _cgroup_rstat_cpu_lock(cpu_lock, cpu, root->cgroup, false); /* Return NULL if this subtree is not on-list */ @@ -285,37 +296,44 @@ __bpf_hook_end(); * number processed last. */ static inline void __cgroup_rstat_lock(struct cgroup_subsys_state *css, - int cpu_in_loop) - __acquires(&cgroup_rstat_lock) + spinlock_t *lock, int cpu_in_loop) + __acquires(lock) { struct cgroup *cgrp = css->cgroup; bool contended; - contended = !spin_trylock_irq(&cgroup_rstat_lock); + contended = !spin_trylock_irq(lock); if (contended) { trace_cgroup_rstat_lock_contended(cgrp, cpu_in_loop, contended); - spin_lock_irq(&cgroup_rstat_lock); + spin_lock_irq(lock); } trace_cgroup_rstat_locked(cgrp, cpu_in_loop, contended); } static inline void __cgroup_rstat_unlock(struct cgroup_subsys_state *css, - int cpu_in_loop) - __releases(&cgroup_rstat_lock) + spinlock_t *lock, int cpu_in_loop) + __releases(lock) { struct cgroup *cgrp = css->cgroup; trace_cgroup_rstat_unlock(cgrp, cpu_in_loop, false); - spin_unlock_irq(&cgroup_rstat_lock); + spin_unlock_irq(lock); } /* see cgroup_rstat_flush() */ static void cgroup_rstat_flush_locked(struct cgroup_subsys_state *css) - __releases(&cgroup_rstat_lock) __acquires(&cgroup_rstat_lock) + __releases(&css->ss->rstat_lock) __acquires(&css->ss->rstat_lock) { + spinlock_t *lock; int cpu; - lockdep_assert_held(&cgroup_rstat_lock); + if (!css->ss) { + pr_warn("cannot use generic flush on base subsystem\n"); + return; + } + + lock = &css->ss->rstat_lock; + lockdep_assert_held(lock); for_each_possible_cpu(cpu) { struct cgroup_subsys_state *pos = cgroup_rstat_updated_list(css, cpu); @@ -334,11 +352,11 @@ static void cgroup_rstat_flush_locked(struct cgroup_subsys_state *css) } /* play nice and yield if necessary */ - if (need_resched() || spin_needbreak(&cgroup_rstat_lock)) { - __cgroup_rstat_unlock(css, cpu); + if (need_resched() || spin_needbreak(lock)) { + __cgroup_rstat_unlock(css, lock, cpu); if (!cond_resched()) cpu_relax(); - __cgroup_rstat_lock(css, cpu); + __cgroup_rstat_lock(css, lock, cpu); } } } @@ -358,11 +376,22 @@ static void cgroup_rstat_flush_locked(struct cgroup_subsys_state *css) */ __bpf_kfunc void cgroup_rstat_flush(struct cgroup_subsys_state *css) { + spinlock_t *lock; + + if (!css->ss) { + int cpu; + + for_each_possible_cpu(cpu) + cgroup_base_stat_flush(css->cgroup, cpu); + return; + } + might_sleep(); - __cgroup_rstat_lock(css, -1); + lock = &css->ss->rstat_lock; + __cgroup_rstat_lock(css, lock, -1); cgroup_rstat_flush_locked(css); - __cgroup_rstat_unlock(css, -1); + __cgroup_rstat_unlock(css, lock, -1); } /** @@ -374,11 +403,11 @@ __bpf_kfunc void cgroup_rstat_flush(struct cgroup_subsys_state *css) * * This function may block. */ -void cgroup_rstat_flush_hold(struct cgroup_subsys_state *css) - __acquires(&cgroup_rstat_lock) +static void cgroup_rstat_base_flush_hold(struct cgroup_subsys_state *css) + __acquires(&cgroup_rstat_base_lock) { might_sleep(); - __cgroup_rstat_lock(css, -1); + __cgroup_rstat_lock(css, &cgroup_rstat_base_lock, -1); cgroup_rstat_flush_locked(css); } @@ -386,10 +415,10 @@ void cgroup_rstat_flush_hold(struct cgroup_subsys_state *css) * cgroup_rstat_flush_release - release cgroup_rstat_flush_hold() * @cgrp: cgroup used by tracepoint */ -void cgroup_rstat_flush_release(struct cgroup_subsys_state *css) - __releases(&cgroup_rstat_lock) +static void cgroup_rstat_base_flush_release(struct cgroup_subsys_state *css) + __releases(&cgroup_rstat_base_lock) { - __cgroup_rstat_unlock(css, -1); + __cgroup_rstat_unlock(css, &cgroup_rstat_base_lock, -1); } int cgroup_rstat_init(struct cgroup_subsys_state *css) @@ -435,10 +464,15 @@ void cgroup_rstat_exit(struct cgroup_subsys_state *css) void __init cgroup_rstat_boot(void) { - int cpu; + struct cgroup_subsys *ss; + int cpu, ssid; + + for_each_possible_cpu(cpu) { + raw_spin_lock_init(per_cpu_ptr(&cgroup_rstat_base_cpu_lock, cpu)); - for_each_possible_cpu(cpu) - raw_spin_lock_init(per_cpu_ptr(&cgroup_rstat_cpu_lock, cpu)); + for_each_subsys(ss, ssid) + raw_spin_lock_init(per_cpu_ptr(&cgroup_rstat_cpu_lock[ssid], cpu)); + } } /* @@ -629,12 +663,12 @@ void cgroup_base_stat_cputime_show(struct seq_file *seq) u64 usage, utime, stime, ntime; if (cgroup_parent(cgrp)) { - cgroup_rstat_flush_hold(css); + cgroup_rstat_base_flush_hold(css); usage = cgrp->bstat.cputime.sum_exec_runtime; cputime_adjust(&cgrp->bstat.cputime, &cgrp->prev_cputime, &utime, &stime); ntime = cgrp->bstat.ntime; - cgroup_rstat_flush_release(css); + cgroup_rstat_base_flush_release(css); } else { /* cgrp->bstat of root is not actually used, reuse it */ root_cgroup_cputime(&cgrp->bstat); From patchwork Tue Dec 24 01:13:59 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: JP Kobryn X-Patchwork-Id: 13919574 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8D257E7718B for ; Tue, 24 Dec 2024 01:14:34 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id ED40F6B0095; Mon, 23 Dec 2024 20:14:26 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id E5E896B0096; Mon, 23 Dec 2024 20:14:26 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C85386B0098; Mon, 23 Dec 2024 20:14:26 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id A81346B0095 for ; Mon, 23 Dec 2024 20:14:26 -0500 (EST) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 3B951121A11 for ; Tue, 24 Dec 2024 01:14:26 +0000 (UTC) X-FDA: 82928080242.01.6EBA587 Received: from mail-pj1-f52.google.com (mail-pj1-f52.google.com [209.85.216.52]) by imf24.hostedemail.com (Postfix) with ESMTP id 4E65C18000A for ; Tue, 24 Dec 2024 01:14:19 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=hGeao8Jz; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf24.hostedemail.com: domain of inwardvessel@gmail.com designates 209.85.216.52 as permitted sender) smtp.mailfrom=inwardvessel@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1735002828; a=rsa-sha256; cv=none; b=QU5XzUn+3xBbYehJXXQ6aRjWIaZM4N+fItRLyCOsvyDP5jhqBCQKij+16QOdlr2n+imyxf yDvuy+oiiJHDJQKvbEJGbIdIDoaEvzvC2MLrZNAlNI5JwBukbQxxkzAN8o8Kf1nrCm5k/j AK5wvWuG35bmVhEZd6le7KIq4M0nUWs= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=hGeao8Jz; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf24.hostedemail.com: domain of inwardvessel@gmail.com designates 209.85.216.52 as permitted sender) smtp.mailfrom=inwardvessel@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1735002828; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Ov79tpw4GuAmAWp+brDBEww96U26TW/4D2kYy+uPe/w=; b=6KUjhabN0y/WEJDOr38AVP4WFYRX7byPEVLWrvxINXP7SiV+54YH5J30ztARhb4deu/j5p q3crGcyBB5iL7H53p0UCITNh0w0hiL6Pb7juBnp55Ub4hxKVCyJgxmBLjnTAX2b6K3Q4/1 BsbgNhN+OZZWcimA0mozYk26N5hloFU= Received: by mail-pj1-f52.google.com with SMTP id 98e67ed59e1d1-2ee50ffcf14so4715973a91.0 for ; Mon, 23 Dec 2024 17:14:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1735002863; x=1735607663; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Ov79tpw4GuAmAWp+brDBEww96U26TW/4D2kYy+uPe/w=; b=hGeao8JzHgv+7IN8MD4e2I1zxj69Q7/5kg/1zbbCf1p7qAixSfiLmS3qWk74RM0hgm dg4EYIwc5IooxHbIPg4kzjMOyetlg2jkAPs2ehvu94YRVDlAs40IIjf1stnHO3MXFrRS vYtuv/FEHSgYvXHcs99J9WnOZFOTnQsT2nQWwTZOyhYUgUyRfK+Z/2V/S82epQZurhaS cxGvo8Xz2z6ky5a86K28QqsvALsXVYFLOOpL19m64FeGFU6Fgb4zmyeP2JKn0uH8iQ4s YqPBzCMAPo6kJIM1EyjFAFqwlKUS7GBmSOAQmEBcR2Z3mkP+ZdwuHb8vFXSgNh18FaEm JX3A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1735002863; x=1735607663; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Ov79tpw4GuAmAWp+brDBEww96U26TW/4D2kYy+uPe/w=; b=Im4inz6NXUGiLjTlp+X9xkazQut7BGs/apNJfBwgD9a8/PXS6A9NGyIGsYE46Ysr7c 7WBwEsPjZwCvKQ24efOqHnT4fBWJdCAUN1COjTrnmywzYZkTASpDiKjniF/LbXsWX+Hm wKiWEWUNuI1oUKM5UZ6258JTXMtaGOuCKQxB+n6CjyMUP4t85DpL2gHjExC2K7uiy66Q 9T+C6fxX0DtH88waI2GeDhWwpMIyp9Zczze9XuzFO02V5VVUCUp+Bqbh0gdH1Bk6sqQF xz2SN+N+zBzNPXnJGM4yuGyXO+thcDMWCU9L4v91H98jOxGneRTVKnb11vaVL1s6R6O8 Lunw== X-Gm-Message-State: AOJu0YzbEzGj3R++Wh9Uah2LC4DHKF4UqXBuy1YXbqX2PaQk+GSgYVOM ViIkN2ehflXpfBiJvMkri2pwz9qsfzCV/+9JHnZaRi8y93p9Tsr/6BmFcw== X-Gm-Gg: ASbGncs6enktv9LFenAAwGtFqptxJwmZq1GORQ/kNlQpSujGyzHv5Vns9Vwshn68UnD 0n3RwF5u9sc3Qh6FVh1dNAAEhqnfv3pwy+PRyIcPZK8MwxuTTiAITjEpAjlwpQRCbju9yDFjl5F oX3FFHFdaaO8XR+EvpWx77fKd7MFWDi3Oh8WweUQcSCKxk0uEl+cpdGRN8JEc942Dw3Fb0/Oveu HSq0BNA5lchkCecaHGWCNYg2svNq+9YxI6IRXjQ06vdyJXMOOeZU6JUqkPL2sZ7+cG8Qc5bQc2p S3DESfFO1J29xQWYqw== X-Google-Smtp-Source: AGHT+IEY86iGVikNx6UUKQVMCZN+fQ5IPsxtPbnuy3ffrZykI3u8eUI1nnnwBtZavbsacJHzUOJQqg== X-Received: by 2002:a17:90b:5251:b0:2f2:a90e:74ef with SMTP id 98e67ed59e1d1-2f44353f0b2mr28673142a91.1.1735002861637; Mon, 23 Dec 2024 17:14:21 -0800 (PST) Received: from saturn.. (c-67-188-127-15.hsd1.ca.comcast.net. [67.188.127.15]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-219dc970c84sm79541255ad.58.2024.12.23.17.14.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Dec 2024 17:14:21 -0800 (PST) From: JP Kobryn To: shakeel.butt@linux.dev, hannes@cmpxchg.org, yosryahmed@google.com, akpm@linux-foundation.org Cc: linux-mm@kvack.org, cgroups@vger.kernel.org Subject: [PATCH 6/9 RFC] cgroup: isolate base stat flush Date: Mon, 23 Dec 2024 17:13:59 -0800 Message-ID: <20241224011402.134009-7-inwardvessel@gmail.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20241224011402.134009-1-inwardvessel@gmail.com> References: <20241224011402.134009-1-inwardvessel@gmail.com> MIME-Version: 1.0 X-Stat-Signature: owfthxch556qz1rfpwasm5pz1nw1kzyn X-Rspam-User: X-Rspamd-Queue-Id: 4E65C18000A X-Rspamd-Server: rspam08 X-HE-Tag: 1735002859-74985 X-HE-Meta: U2FsdGVkX1+ersnec6vCpn1OY33zrWERxV4OhfTpflrd7lunztd4lScNGCG7ZrEjY5ZemwICOmhi1efuTbpILlRvhuac4sVNuPAKBH1KE2C62LiH3FaNj3ZPrNxhdJpMaZAG9sk4dBSagAcPjqvJc5RRNosqGd1ddXG3XI2x8DxqtRAEXSjgGfQQz41KtRJGOFEdbDsQ6be1qRUB+fwGmmqr262YkvD2MijlhlnUnSGj8nTUMRB9ga2RCiqxJpRZJSqrAUj0436FH90gErYUxlC9WF8/MK/n8UgXjBN6c7ZVbj/v+//Fdz5KvROiaJZbivnwWnACkj8eh/P0P6w/abEoio/EMJDxaNj1G6OgAqqxX0LyUaMNWS1vikmbo2ROK8bFL+uIyGrRyYbnDawLxYe8pck+ZY/gyLAmHVb7OH7YMopZSPSrUmJqdWyhQuVINX75GxkTlJd2HGpZyKLFR+cPQ9rDatJx9IcoOlRG5jUpvQ4y2CF1aQrMBK9LxMOPMzOBfChXyyNwPBtdK4WdhE95Uznk7RtlFSd0WgUKCksxD50bddbNDDkIZZWZqxyDJMwwfxl7+nHEtd0XXPpolnnR7/XwImeURuVAUjtTpEFj5A6WaONxQ9wFJXcvqr+JDi8ow5K3xsKFGLUSxHI7p+gnQvC1I4aWukqCntVwuD3cNEftbj7/aoI5ANKEl5lgL0lY4SAAbHGBagHC1iK7GvUUCYr7f46Zx8DK3hvsLyW+L92p0qUEUfS8f7zuscuv/egH9CrkXdVtMme90Qt6H66R0ePH82m995wX6OVtYX81tS3RZBSQGsUa4UplpjNGkYTx2UQMHBxfR6VeCAkcwdL2sEwLMUuuzULNHkhcApO150XIP0uoh5Pvkop246UosAdQhwaYdIbPvj8jQKxj5JEzvcsBgcB1JZr0EHULtXU7bQ1LR7hLS2gywGcRmI29g4f8LOQTLRP2uBt/Och dlKpAE7X Aon+8vJIRWEe1yrzjvVzi7eQ3jyeQZlAJaXjg83iQhPi8eP8fdRKUU+GDFq+Ba9vLRcpr6r0dHg0PEMSjks1PmnGOMuZRix/UBVYpHc4I5GDdhzDh4JBiRtjr7q2LZaGlgeeiQr4DdSQMd9l0fBD21bO3s4paXlYCL6n3X4mSVALsoxcc7rgKkV++790WzxzYHa8lUqDjsPMf/66tiwjJesZMHErLOUtutqFxMqbLh4cqSSBJXE0wn7A0R6eMsYtOdkRZe5LMCI/QuHNUb2v0o9O54EiA9uQTCNpg9ZM02HF6zFLIDwACXqz08M5BlpOv3/BHiVc9A8Zpatr1LlkR9qxHYQhcHvf/q6CVlSEqe62C7iuIafCF2K1bhw== X-Bogosity: Unsure, tests=bogofilter, spamicity=0.463905, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Remove the base stat flushing from the generic css flushing routine. Provide a direct way to flush the base stats and make use of it when the cpu stats are read. Signed-off-by: JP Kobryn --- kernel/cgroup/rstat.c | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/kernel/cgroup/rstat.c b/kernel/cgroup/rstat.c index 958bdccf0359..92a46b960be1 100644 --- a/kernel/cgroup/rstat.c +++ b/kernel/cgroup/rstat.c @@ -320,6 +320,14 @@ static inline void __cgroup_rstat_unlock(struct cgroup_subsys_state *css, spin_unlock_irq(lock); } +static void cgroup_rstat_base_flush_locked(struct cgroup *cgrp) +{ + int cpu; + + for_each_possible_cpu(cpu) + cgroup_base_stat_flush(cgrp, cpu); +} + /* see cgroup_rstat_flush() */ static void cgroup_rstat_flush_locked(struct cgroup_subsys_state *css) __releases(&css->ss->rstat_lock) __acquires(&css->ss->rstat_lock) @@ -341,7 +349,6 @@ static void cgroup_rstat_flush_locked(struct cgroup_subsys_state *css) for (; pos; pos = pos->rstat_flush_next) { struct cgroup_subsys_state *css_iter; - cgroup_base_stat_flush(pos->cgroup, cpu); bpf_rstat_flush(pos->cgroup, cgroup_parent(pos->cgroup), cpu); rcu_read_lock(); @@ -408,7 +415,7 @@ static void cgroup_rstat_base_flush_hold(struct cgroup_subsys_state *css) { might_sleep(); __cgroup_rstat_lock(css, &cgroup_rstat_base_lock, -1); - cgroup_rstat_flush_locked(css); + cgroup_rstat_base_flush_locked(css->cgroup); } /** From patchwork Tue Dec 24 01:14:00 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: JP Kobryn X-Patchwork-Id: 13919573 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 64247E7718D for ; Tue, 24 Dec 2024 01:14:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7A7EC6B0093; Mon, 23 Dec 2024 20:14:26 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 756276B0095; Mon, 23 Dec 2024 20:14:26 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5A8A96B0096; Mon, 23 Dec 2024 20:14:26 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 3BCED6B0093 for ; Mon, 23 Dec 2024 20:14:26 -0500 (EST) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 0123F1C7BAC for ; Tue, 24 Dec 2024 01:14:25 +0000 (UTC) X-FDA: 82928080956.24.57A9FC7 Received: from mail-pl1-f180.google.com (mail-pl1-f180.google.com [209.85.214.180]) by imf24.hostedemail.com (Postfix) with ESMTP id 0E311180007 for ; Tue, 24 Dec 2024 01:14:18 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=mG4H7Ytp; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf24.hostedemail.com: domain of inwardvessel@gmail.com designates 209.85.214.180 as permitted sender) smtp.mailfrom=inwardvessel@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1735002846; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=WZJllDXeQ8qx4y4BNkUJ894lZbTAuhDCQMxQBXIE4qE=; b=jLDWKIRg4JmAaPv8RO9zGLnrTmW3hzWK6xRbf+e/m+Ku6OBQdd9FJ2EEG4z4ZlDUErPFe+ LwlO91wjWW09xXB6o4Na/ydqBgLQgbsLuXOTVfzMm9C8gRoJkX4ySyRzfQbws1KXObC42I fgKOpPrpINqSorBBHYi8bIe6wR7cloo= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1735002846; a=rsa-sha256; cv=none; b=nA+9y7QXXQn7ebRf4hVHPrqRSmlElgQ1EhYxxOsXTN1A2cmGnbFfZmlu7M8+8eFeL0pnvJ Cc8vsy3heI26qxFGJ1VZdtbRDz93BAmG65MRZQlrLHwctAYEgHs+TjSTKRX7NEuUr3gExz AOqMxBFKrPECI5Tpj2b6Lxrj6fDw+Ig= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=mG4H7Ytp; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf24.hostedemail.com: domain of inwardvessel@gmail.com designates 209.85.214.180 as permitted sender) smtp.mailfrom=inwardvessel@gmail.com Received: by mail-pl1-f180.google.com with SMTP id d9443c01a7336-21670dce0a7so58505435ad.1 for ; Mon, 23 Dec 2024 17:14:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1735002863; x=1735607663; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=WZJllDXeQ8qx4y4BNkUJ894lZbTAuhDCQMxQBXIE4qE=; b=mG4H7YtpE4WYDyN4k2164ksiugFzBRl2F4Ru7zr25h0ytHYIoAlqtSlLjtg2a1T/wH vk0fbOzO51cIuu5yWngKkjEjc51VKfYcBxjnMxCk8sw3DNAywvm4qXhm136ZGvxP1WsZ AGe5XetILujFLXpAk7BFMLopz+9cyxvCh0ZKkOWZSn3J1eCVTgErQIIKdZbzcinTQyH/ 1/76F96BKSzPzjAASocJ9HSXID/FUNtxwEFrP+7ALlhaFKS4YMWWzffPi8UzY6xy6+7h 4sNau870HCQUI6TAYa8DNI8XKvwX+LbJK3kb1qku5F89PTu+3LsVoT/YAnWb8oL14MaS NAEg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1735002863; x=1735607663; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=WZJllDXeQ8qx4y4BNkUJ894lZbTAuhDCQMxQBXIE4qE=; b=QCRa2rX1xO7GZUDecodO65ygwaVKSOBh0LGxlIC+7bTrhNdnt3FV+kv/ijoKMNHzez uHVxiarTsQUMh8+rGfEdNMUBt6juJV561dVcXv3IX9m8CRPz9hZ1P5rghsMi4tNATNkB J6w/7fwHyR4w/47fNwMwQIep+o37zl9Tn+hJMgbIiV4aaVRe9Q24AUW/WFxmYgpGJ5vB stOhbDbGvEZwf3+/9f3usfUnuVjlNi8yIHpkxrtV8Wtfo1fymiV6F2jcSHvZRY5GSHQI mpFD5Aa16n4gdVrppYx9jE3ucqPkRNkKAih7eNJ/b67s4k3hs7UwwEqzGJ/nyLZwni1w aNxA== X-Gm-Message-State: AOJu0YzrZm6fTM3KDmvQbHv0FPjQDIYCpN5ANNxKF15WSC/pGvHrUtHw ILpeAIPve8JhOnXW5WsD+ia4rfN1sqxdp3pzo8C4gTp3eO4OrkUCmFPUXg== X-Gm-Gg: ASbGncvyYiJ8Sj8+9y6syXvvlDAoGgQHkbkcC1ArZE6qeq4vTiopMSgOs5EYPT6/CY+ ReV69qLD8ANxFrUVqJfFSkb3VlhhEadA7hch1wQ7mRK+bUgruV6EIsHrGiNvOCEa1fGp03Yl7OF o/f687D+T/Vs4QwYFe+UjVKbNNPe+mzIT0ZATWNrwKh7d0yrnDV9UFmM/miN+rg3MutbX/WeDzE wOwR8NMxXXMn9s643cSBX/PSJ45LjCn3WdAWfP27Q4xc3JK9G9dL42DYkB7wyUdf+H1tIjLAsQi YgHHgAmrcKAYfJuqmw== X-Google-Smtp-Source: AGHT+IGzsv3KE6G8tMJp0bteEaX1XLhMudfBrUqQxMi2wOnR+qelJ6wcB5nC0ZJLnxdCo6AT0qqnSQ== X-Received: by 2002:a17:902:f685:b0:212:615f:c1 with SMTP id d9443c01a7336-219e6e9f98amr173127195ad.14.1735002862898; Mon, 23 Dec 2024 17:14:22 -0800 (PST) Received: from saturn.. (c-67-188-127-15.hsd1.ca.comcast.net. [67.188.127.15]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-219dc970c84sm79541255ad.58.2024.12.23.17.14.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Dec 2024 17:14:22 -0800 (PST) From: JP Kobryn To: shakeel.butt@linux.dev, hannes@cmpxchg.org, yosryahmed@google.com, akpm@linux-foundation.org Cc: linux-mm@kvack.org, cgroups@vger.kernel.org Subject: [PATCH 7/9 RFC] cgroup: remove unneeded rcu list Date: Mon, 23 Dec 2024 17:14:00 -0800 Message-ID: <20241224011402.134009-8-inwardvessel@gmail.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20241224011402.134009-1-inwardvessel@gmail.com> References: <20241224011402.134009-1-inwardvessel@gmail.com> MIME-Version: 1.0 X-Stat-Signature: znppoq1syeyiabacyc1zfppewhi6qc69 X-Rspamd-Queue-Id: 0E311180007 X-Rspam-User: X-Rspamd-Server: rspam01 X-HE-Tag: 1735002858-370240 X-HE-Meta: U2FsdGVkX1+2bRagmYFcFeEKMQF7ZV2b5AZZvPp51M7IjyoJ+egQq6cCjO1RXt2cVKX9TF4ZVpPc23/bY2+PviPfqVJsTFSS0A3EwHm3pDay2Rrbwj30ZkgNBPaw+kNJ6cdbMAY6dLheGe53si4o+a1ODgw4GEFPbjDgqosb51Whvynr4loYGSSGyMK1E2xCpq8p6tqR6is5561ZISQ66D047vKrI+wvXoONXQAsgWW87kcSYcKGLbRFajJZv/A3KFMbTKSAVFR2Y9vmXyjDQxSQ7hKWvugW/zt7j2DzaXmZ7LhCRAMkbAJwOLamb9bA6rxRNIe92JoDzmXi7Hfxsb3WUpcWwtTI2gkDajImuI7U138QxUVRNI7oVztClUcrsO8DPU8N2Yso04D1Cba2XZDFV5YxQ8wlLIPqHo9LXm+PrV7FEiE2BPjD7YnZYoVhdDPrnxfLOR/EiTBRa4ndR/d9ihvM/DynsVj3Oqj0g60+6N6OnOiR56/V3muai3We7CZsqIu5rPaMs5hMXbzryVFnd54bhVRAiydVfrzmX39ZU8k4AVkODnQnFRci9/vhQfJAX16jh9AKeXD49w6fSqjhoERKkk7w7WDDeOWMZdXomFAONdEB84/axGqdyp3J2knYxuplOlNzHbDTz8uTw5JyCyMulM/MK+9kxz1mPXV6rTd0TCERxq0nZXEdsq5JbMyjf9mpjW4CUz1rNS6ZyLzUsVg0QyaRA9F4UHsqPlilqrw0qV8/i8UBHxUAxrqIYEEaUtmwPkj7T81uNvJVXdhwDOnOE5A21Y1N9RpCfX8VRP8l39X6RRlPXcl036pD9YZoC0GOxdtHnTRDsZm6TEMz7XiMu6mMEmuvs41CgrppyBxQ7H4vMSq5440/kiaMwctjTMWwp7+UoQJa0lFlhAswI/FmayA0S2PhBgCCniD43MZ5tqjbP/w2K6eUDiZmL4bF6o52VVhEyAufTfC UJUmrfq1 1zu2I9Nn7wzsf07Lx++ogwGbcmm6FnRSAEl2Y0HIZmnpJu2MMJRcVPrnTbVVue4PYyMTDl4eJ/optxWmJ+9T/mVHwIRsKgMh1ZiHbq594ZSx7lsidjd9L1nY4xrqgxXlyW5RZ0/S344olUq9L2+vSlKqyPYAMQylfoMrrgnf7LpScPQOwuuM1005ariyZ7Y9vmPcie+Vz9LZ1wdqXsqkI5si+DnrxkNOcghhrztCp1RjOKAjF0yUF/66begXIJpmhmwoP7en2KLw2jVtRmIoQm0kavGqSkbQGRg9+SZKgb/9KbuRxEBBR9mAeY12uKecIuSLmSDKPN2xX8NcLUHBbfGVQ/HTH9vXkHzkQect6rdEzC8WYJIG6A9FFaw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000001, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Since the cgroup_subsystem_state owns the rstat tree, the list management previously done by the cgroup is no longer needed. Signed-off-by: JP Kobryn --- include/linux/cgroup-defs.h | 11 ----------- kernel/cgroup/cgroup.c | 20 +------------------- kernel/cgroup/rstat.c | 9 +-------- 3 files changed, 2 insertions(+), 38 deletions(-) diff --git a/include/linux/cgroup-defs.h b/include/linux/cgroup-defs.h index 4d87519ff023..836260c422a0 100644 --- a/include/linux/cgroup-defs.h +++ b/include/linux/cgroup-defs.h @@ -182,14 +182,6 @@ struct cgroup_subsys_state { /* per-cpu recursive resource statistics */ struct cgroup_rstat_cpu __percpu *rstat_cpu; - struct list_head rstat_css_list; - - /* - * Add padding to separate the read mostly rstat_cpu and - * rstat_css_list into a different cacheline from the following - * rstat_flush_next and *bstat fields which can have frequent updates. - */ - CACHELINE_PADDING(_pad_); /* * A singly-linked list of cgroup structures to be rstat flushed. @@ -198,9 +190,6 @@ struct cgroup_subsys_state { */ struct cgroup_subsys_state *rstat_flush_next; - /* flush target list anchored at cgrp->rstat_css_list */ - struct list_head rstat_css_node; - /* * PI: Subsys-unique ID. 0 is unused and root is always 1. The * matching css can be looked up using css_from_id(). diff --git a/kernel/cgroup/cgroup.c b/kernel/cgroup/cgroup.c index 96a2d15fe5e9..a36ed3995c6f 100644 --- a/kernel/cgroup/cgroup.c +++ b/kernel/cgroup/cgroup.c @@ -1826,7 +1826,6 @@ int rebind_subsystems(struct cgroup_root *dst_root, u16 ss_mask) struct cgroup_root *src_root = ss->root; struct cgroup *scgrp = &src_root->cgrp; struct cgroup_subsys_state *css = cgroup_css(scgrp, ss); - struct cgroup_subsys_state *dcss = cgroup_css(dcgrp, ss); struct css_set *cset, *cset_pos; struct css_task_iter *it; @@ -1864,13 +1863,6 @@ int rebind_subsystems(struct cgroup_root *dst_root, u16 ss_mask) } spin_unlock_irq(&css_set_lock); - if (ss->css_rstat_flush) { - list_del_rcu(&css->rstat_css_node); - synchronize_rcu(); - list_add_rcu(&css->rstat_css_node, - &dcss->rstat_css_list); - } - /* default hierarchy doesn't enable controllers by default */ dst_root->subsys_mask |= 1 << ssid; if (dst_root == &cgrp_dfl_root) { @@ -5491,11 +5483,7 @@ static void css_release_work_fn(struct work_struct *work) if (ss) { struct cgroup *parent_cgrp; - /* css release path */ - if (!list_empty(&css->rstat_css_node)) { - cgroup_rstat_flush(css); - list_del_rcu(&css->rstat_css_node); - } + cgroup_rstat_flush(css); cgroup_idr_replace(&ss->css_idr, NULL, css->id); if (ss->css_released) @@ -5569,8 +5557,6 @@ static void init_and_link_css(struct cgroup_subsys_state *css, css->id = -1; INIT_LIST_HEAD(&css->sibling); INIT_LIST_HEAD(&css->children); - INIT_LIST_HEAD(&css->rstat_css_list); - INIT_LIST_HEAD(&css->rstat_css_node); css->serial_nr = css_serial_nr_next++; atomic_set(&css->online_cnt, 0); @@ -5579,9 +5565,6 @@ static void init_and_link_css(struct cgroup_subsys_state *css, css_get(css->parent); } - if (ss->css_rstat_flush) - list_add_rcu(&css->rstat_css_node, &css->rstat_css_list); - BUG_ON(cgroup_css(cgrp, ss)); } @@ -5687,7 +5670,6 @@ static struct cgroup_subsys_state *css_create(struct cgroup *cgrp, err_list_del: list_del_rcu(&css->sibling); err_free_css: - list_del_rcu(&css->rstat_css_node); INIT_RCU_WORK(&css->destroy_rwork, css_free_rwork_fn); queue_rcu_work(cgroup_destroy_wq, &css->destroy_rwork); return ERR_PTR(err); diff --git a/kernel/cgroup/rstat.c b/kernel/cgroup/rstat.c index 92a46b960be1..c52e8429c75d 100644 --- a/kernel/cgroup/rstat.c +++ b/kernel/cgroup/rstat.c @@ -347,15 +347,8 @@ static void cgroup_rstat_flush_locked(struct cgroup_subsys_state *css) struct cgroup_subsys_state *pos = cgroup_rstat_updated_list(css, cpu); for (; pos; pos = pos->rstat_flush_next) { - struct cgroup_subsys_state *css_iter; - bpf_rstat_flush(pos->cgroup, cgroup_parent(pos->cgroup), cpu); - - rcu_read_lock(); - list_for_each_entry_rcu(css_iter, &pos->rstat_css_list, - rstat_css_node) - css_iter->ss->css_rstat_flush(css_iter, cpu); - rcu_read_unlock(); + pos->ss->css_rstat_flush(pos, cpu); } /* play nice and yield if necessary */ From patchwork Tue Dec 24 01:14:01 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: JP Kobryn X-Patchwork-Id: 13919575 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 684BCE7718D for ; Tue, 24 Dec 2024 01:14:36 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A192D6B0096; Mon, 23 Dec 2024 20:14:27 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 99EC86B0098; Mon, 23 Dec 2024 20:14:27 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7F2946B0099; Mon, 23 Dec 2024 20:14:27 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 5CBCB6B0096 for ; Mon, 23 Dec 2024 20:14:27 -0500 (EST) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 25755140918 for ; Tue, 24 Dec 2024 01:14:27 +0000 (UTC) X-FDA: 82928080578.27.F070B8B Received: from mail-pj1-f48.google.com (mail-pj1-f48.google.com [209.85.216.48]) by imf18.hostedemail.com (Postfix) with ESMTP id C2D621C0005 for ; Tue, 24 Dec 2024 01:14:07 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=W6mhAfRU; spf=pass (imf18.hostedemail.com: domain of inwardvessel@gmail.com designates 209.85.216.48 as permitted sender) smtp.mailfrom=inwardvessel@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1735002847; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=JF5+lUY/lcy3tq2Y5kiTvVdLllXSJTZSMxUiRf304xw=; b=l5h1p5pdStFKrsPsrn2iinLvoJkTbXLziwiYA/Zs61CZ6XXtjRx9K+xUe4zJNudXU+SORQ MvyqqbckivkRxkU2gj2wnT9vYE3YnXJ4TWlWj6VN1xCaxKLVU7mghsCMVKfyDbNfq93kJQ qx7FF5skVQV8VCEvA7HSJj/cSFNs0fA= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=W6mhAfRU; spf=pass (imf18.hostedemail.com: domain of inwardvessel@gmail.com designates 209.85.216.48 as permitted sender) smtp.mailfrom=inwardvessel@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1735002847; a=rsa-sha256; cv=none; b=pppgpCOyMIKumV/0Jzgb/rnAoUj16NQQ8dWOVmRLkjGv1RHlc+h2KJQw2RomGEv3Ell0Ec ZadwY9SlY+L/eiqO03YCHusfBZaGsllJkpbrBaPWsR8SghuG2lkYTVBD9LW2J151Y0bugS FgR7F6vqW6NLIqe4oIPF0xYT5Bxko9k= Received: by mail-pj1-f48.google.com with SMTP id 98e67ed59e1d1-2ef6c56032eso3308353a91.2 for ; Mon, 23 Dec 2024 17:14:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1735002864; x=1735607664; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=JF5+lUY/lcy3tq2Y5kiTvVdLllXSJTZSMxUiRf304xw=; b=W6mhAfRUkrp2X0OHQde7RL8UxbkTo/8+SgVqpXaspl4Jo+YPHn9oP1/ZlX+SzrWwPW yFVuO6d5RJIt+TjtlcL8QbCg1P5ZKpkEmuSU3IHlFd6tTt213dpzEBwoew15dpT9luCV xCBpp4HykdW+LuOiSKSZLiCSFW5GOvVBHlxWz4lJlPG9FxGpzNCQKKC5+QLu7kG0ft/W OvHRAHhNRJwEBhAOL6jtTfhQFMPxSm0rP8qIGsWMhn4un8QiVPz7mAx9k5Bj3YVGYDv3 sNHtn7bIDG3oCtO2eQDaLbaDDHSmmZLLibhOveWZmfxDk1iCo7+5rxfKVlQ5XWFGfeeW VeiQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1735002864; x=1735607664; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=JF5+lUY/lcy3tq2Y5kiTvVdLllXSJTZSMxUiRf304xw=; b=RjP1O2wRGZIsN9w8Hn4X223Br/M57BQoEykx6oyGQel1CBBT1XHFwJpYQkOEz/kTMd 8xthA49rzHW5avL2/SsJPwz+EsZg6ZlwuqHJyiiVq9gd8+6eYHV2T815KDIzzH83KLvw U75d6UBmWDKxRlE152+bAp0ngUq8/b3g7AJ+0iK5UrSXajQY9/QHi+bI3h8Tp+GfbPdT B3oNqz/mhYsDDCRf4VSCdSw+MIDaj6svADOyfzTf6gdQRqKcGHnfx60cucTJPjv0nYFj NHzBKqdXuybSzJnY03hcKnNIjXcnzkV2rl2ElwPwPxTtJKKCBncBDSXEf5C4GicILsTY 9DEg== X-Gm-Message-State: AOJu0YxgFDnKeCTBX69GFnvRJjW9KGuUW/JFrI+5zG1pDI0d8nTbnMqS w3Fo7IhrjnKqR65VeaYHywCFrmvfh8PUQYuC2BimUSeKPktY0kMqo3v69Q== X-Gm-Gg: ASbGncsaub30uc7YlfrhKdmE/tqlgmap8wErMse0yHC7wJAXB6GV9pA05gyeThGzR0l 67sgOh8mLBF/uC1kFYgb8LDBoC9CQ6cMRyzb5EW0NzcpRnSJ67gd6mucPHC3gh9FX8b+UBHgvFp Ebjn5zdz4es+qI8W1CHreqrZ6cz2OHb41rYgurcba28PvgGPJLAaT8lItgDUIkYX/qw7WBP8Ozp evqVBEC8sVgCo1KwwBYj8knWoK+HAmgcbvOQPMLH98RraBTCcUdAeF9H4gDJaDahiq9fOAaApeQ 5P3umWWttPkL0LlKgQ== X-Google-Smtp-Source: AGHT+IESNbkbhfztvX6YzHV2B4TGbaIbNOhCOeua4C6yql386xi6jXCF50xnvVGI3HxJdfepwaKGzw== X-Received: by 2002:a17:90b:2545:b0:2ea:3f34:f18f with SMTP id 98e67ed59e1d1-2f452e49630mr22512064a91.19.1735002864134; Mon, 23 Dec 2024 17:14:24 -0800 (PST) Received: from saturn.. (c-67-188-127-15.hsd1.ca.comcast.net. [67.188.127.15]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-219dc970c84sm79541255ad.58.2024.12.23.17.14.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Dec 2024 17:14:23 -0800 (PST) From: JP Kobryn To: shakeel.butt@linux.dev, hannes@cmpxchg.org, yosryahmed@google.com, akpm@linux-foundation.org Cc: linux-mm@kvack.org, cgroups@vger.kernel.org Subject: [PATCH 8/9 RFC] cgroup: remove bpf rstat flush from css generic flush Date: Mon, 23 Dec 2024 17:14:01 -0800 Message-ID: <20241224011402.134009-9-inwardvessel@gmail.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20241224011402.134009-1-inwardvessel@gmail.com> References: <20241224011402.134009-1-inwardvessel@gmail.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: C2D621C0005 X-Stat-Signature: ugzmm5yzgnj4medkr69zo13uag4gxnyu X-Rspam-User: X-Rspamd-Server: rspam11 X-HE-Tag: 1735002847-591764 X-HE-Meta: U2FsdGVkX1/I45ztTTJJNrDXYyX2Chyk7DFHYGgQXU01nRRcvVul6wDSzwt8gurdSVTON9h0/CzoZYDea/f4f4MLsLMyYXFS0nu/JhntVoFNSAswZRMLUDL2jOJHjus1QUn/ctYt7UfAkprFEc7d9urPfexAAwsYpWPMdvL20HMlGoE6714YtiG4biyAMOIz3FVf7HYps78nosEPTtc0zuo4f27hk5BUfLpy3mKUi9T+v/a/vawfXFvidRpL8xdPzr3bwwFDIu1SxkRVy7uBJP/Fu9OOhUl5qvSjE8YqKD43ZA5zh3qMv9yNhuw+T+PTfLB3prti4GQIYXPKhgGNIzFcrgDlF6VMZw3sXdKTpFGy+vzupVcGD8FbJN/+T+3D7FvOR0e+4UG+moY7SVxu84xy/I73GZkHJ1cBqjvWvt1WpNVnN6HBrl7kbX1FAkaMZgdc31eRZntPZr2C4S9KS7fglGz4uhYDeHS4VNyjjCQjRrGCgFD00rMEOEHRJddPKBYOWr+1G7NJnR2pDFrs3CWoHPx3UiLUOIm7iDEB8ra8LKCOQ4u+f1zoPK43euP4u8fepqqe5G6fPGA/0IvLBh/yCg1spu8qWKHakKT/6Fu4Uz83a7mJVP+Nfa84o+SEnWDujRXcw0qb5qp0B61JFHOhb800cFELvIkmWd6ST7OG8bNs7+kSYW44ijcD/jl22JsvLnF+48zA6bB3bz5neW6T2ICHHlPEIxpq4eXqPbv0F4PwsStxHSek0ckNXduQvL5DUQNufIKokaTvItTNxoa43o7I7QFOXIvptjlf0P8JwT626NJpCfB+uyKNGQYabpcOh4jYRdXmVMJe4uKEQr41aBX1njPi9vdO2/MYl4vq83aiKA3CgMo/I5rlpVrRX5NFb1cFIhhH/+OM4CoeTZeNVeicZKzpH260RVMkqG2mRLKNfTE4GQgMH8oXeS2pZ93sfpQ6Q9c9DrAyUR2 w0kEdm6f YhBgza8B5/awSB7xQeRTILes4A8eTVLN7eEVoS2lRHGAYj8eaWeXjkcRaetq9Xju6l6Co4QWqA94Z9FNoAonS9o4Wd97+J/RV016C5wXRhzZmooPANIi2OV/gpmD51I+cLqzm9QBmS8FKNH7Ra6XJR1D+HalbvsAoI9di+VZHtItJmz7yLOmRWVusRGRrvdvuwNwn5K6tHsF9StkcXabPyTc8FVMdJdFsUvgRZl1AdrOWEb3wzj3GhAI9tDNKSMVoCe7aiBe/zL6g413oU4y/zXIzN46PDI5OXxsMc2Wc1jqItiBx2GNvqvywgqIBoyeTpVkGMX7cEHPPp1CIRv5lKpw+jgtvAzicdZMIb96d/0XUN9MWST2mp7+2yA== X-Bogosity: Unsure, tests=bogofilter, spamicity=0.477346, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Remove this call from the generic subsystem flush. Leave it up to bpf programs to manually flush any subsystems desired by using the kfunc cgroup_rstat_flush(). Signed-off-by: JP Kobryn --- kernel/cgroup/rstat.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/kernel/cgroup/rstat.c b/kernel/cgroup/rstat.c index c52e8429c75d..03effaaf09a4 100644 --- a/kernel/cgroup/rstat.c +++ b/kernel/cgroup/rstat.c @@ -346,10 +346,8 @@ static void cgroup_rstat_flush_locked(struct cgroup_subsys_state *css) for_each_possible_cpu(cpu) { struct cgroup_subsys_state *pos = cgroup_rstat_updated_list(css, cpu); - for (; pos; pos = pos->rstat_flush_next) { - bpf_rstat_flush(pos->cgroup, cgroup_parent(pos->cgroup), cpu); + for (; pos; pos = pos->rstat_flush_next) pos->ss->css_rstat_flush(pos, cpu); - } /* play nice and yield if necessary */ if (need_resched() || spin_needbreak(lock)) { From patchwork Tue Dec 24 01:14:02 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: JP Kobryn X-Patchwork-Id: 13919576 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 67F26E7718B for ; Tue, 24 Dec 2024 01:14:38 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3D9356B0099; Mon, 23 Dec 2024 20:14:29 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 385976B009A; Mon, 23 Dec 2024 20:14:29 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1FFB96B009B; Mon, 23 Dec 2024 20:14:29 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id E61B56B0099 for ; Mon, 23 Dec 2024 20:14:28 -0500 (EST) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 8B95514196C for ; Tue, 24 Dec 2024 01:14:28 +0000 (UTC) X-FDA: 82928080410.19.7E04A03 Received: from mail-pl1-f171.google.com (mail-pl1-f171.google.com [209.85.214.171]) by imf28.hostedemail.com (Postfix) with ESMTP id 9CF4FC0016 for ; Tue, 24 Dec 2024 01:13:44 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=X9LKSG2X; spf=pass (imf28.hostedemail.com: domain of inwardvessel@gmail.com designates 209.85.214.171 as permitted sender) smtp.mailfrom=inwardvessel@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1735002841; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Gm9yai/osEcIq2t3vZy97c5K46GB7iT4Bhg8cC3aZ6k=; b=BpniXAcnt0AKtUptZ71MW398Jzf+sQ2X+6yTHavaJLfZYd0YPHZhC3oOJ5eFdwbKW47bdm iTOUNtYPQZxlJNyQmMIZCY7rLpjEMGiH/W3CPjh/kdszwYwm23PrRxbaw62c72FQBwn244 zephq8ecEuwOtCEpNyBELB7mbDVepJQ= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1735002841; a=rsa-sha256; cv=none; b=L2gyE39zxGXKA74R4oYMNJHMjo8Zfj+QduTpa8q+4v8jypcO5B+UQco8yxw5xiPe2R9xP8 l2K8NATJXGN395HVIKy5Mlhwd0epW6kYAgcl63SwhSg+rRT+Fyy+JfAkl0BZIKRsBOaivH +/gziWxGEmi8IJX1bTY3jjAklqxB3fc= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=X9LKSG2X; spf=pass (imf28.hostedemail.com: domain of inwardvessel@gmail.com designates 209.85.214.171 as permitted sender) smtp.mailfrom=inwardvessel@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-pl1-f171.google.com with SMTP id d9443c01a7336-2167141dfa1so43808845ad.1 for ; Mon, 23 Dec 2024 17:14:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1735002865; x=1735607665; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Gm9yai/osEcIq2t3vZy97c5K46GB7iT4Bhg8cC3aZ6k=; b=X9LKSG2XV8XzulsQnQ/3R2m7iI4JaTQHw2CXgmiC/SGLvNh1s2YXQ6nCm/PIGzP7fs 1sHspt9zQYBWlK+PmJnR2wkDsKrg0n1L0/z64TWOHIw9OLcSTK8+yvRvGAfVeMJhLE2X CMAYrYNC4arUzoe6nVmmRJO5qI6/Kdp+vgQ808v6LoijeriSVkYcm5ggYiYpqgd1oLun W814F/aSv3HEGSgZUl6QVtHG1H22zkoMBU14EofClok5HHvj57TdN+85j6KIHi975YNC r87vPP9zJZ1XgBZf/eJsNnB+P/noOQm3gyIDnB6c+UhWxGtnCUGC8Lc8nHF6ofaipgOO t7Bg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1735002865; x=1735607665; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Gm9yai/osEcIq2t3vZy97c5K46GB7iT4Bhg8cC3aZ6k=; b=IKkQavUhUXYFt8IFm7BdblFGcIrA1L6ICasmhzaTBFNB5ySgh2bxBoQS3fMh9bcR2a +6tPaOuyzWNhnkJw3FBy+1dIIpYHpRLmVoTO99bUNGTwJqL5d+XWF9Allw0rtX9B7xpX uYkB8Y9mgBMtWVG9Xk6f7GOP4Xg/kZ2h93zHYqEp1E6fR9u7SJw+cgDcLNRHr779ZvaM pwmKkcrDSbrROUxgrzlk5cKxmHqWGR49pg5oHjrlaOQoRo3BQDbMNN08YGRqIwdFcS2G jZdntIgi1M1PbG7skDKuOvgrrgqNMrZJFaZYmKNoFqeEuFdNU582TYhQ41ecNOp3owvb aaTg== X-Gm-Message-State: AOJu0Yxuo/H2zT7+ZTyCzlUO+Cdo6/NIH3KENTvsZLlRgYeqQKFm+Jdc qA8TI8KI2qwfInNhWyTa0F8gYUsBMIqgg/Kp18BajyHyNf+ihUyo X-Gm-Gg: ASbGncspnaW68NaQfbifQ/Z0/TEzKLt6/zlcqJf4n88auo7ZnQWEfIuh6BYjSC8dZdr p0Bw6lNS8yrHQvOlUDTACUmxFHMcCht3vPsGubtokirjSxsAGLdLoYvDtL4kA5r/C5qT3RD7jTJ T6IiYVyf/GmlA+j2Ahz6Rzj7y1SqpY8Z8cP+TjppDcZ1e9WNF7Bfkx4o9/9vDZU/kAAD5qJHvkH 587+gjOkdf7pk3gJT2PaNCATnRZN52I2E1F9dRqTq0e+yDD+SFRmvKHpENhlE8U3Lk4pNkAui91 Gd9iOBhqY9I42wHZQQ== X-Google-Smtp-Source: AGHT+IEqRtFE3SjYN8vE3ZEp1iTgfgoCrR6J9QQ+lxWuXPA4f3+IfoIE2/oejmcZygYdV0QTvV6Ahw== X-Received: by 2002:a17:902:f646:b0:215:ba2b:cd55 with SMTP id d9443c01a7336-219e6ca6c61mr244012185ad.2.1735002865450; Mon, 23 Dec 2024 17:14:25 -0800 (PST) Received: from saturn.. (c-67-188-127-15.hsd1.ca.comcast.net. [67.188.127.15]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-219dc970c84sm79541255ad.58.2024.12.23.17.14.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Dec 2024 17:14:24 -0800 (PST) From: JP Kobryn To: shakeel.butt@linux.dev, hannes@cmpxchg.org, yosryahmed@google.com, akpm@linux-foundation.org Cc: linux-mm@kvack.org, cgroups@vger.kernel.org Subject: [PATCH 9/9 RFC] cgroup: avoid allocating rstat when flush func not present Date: Mon, 23 Dec 2024 17:14:02 -0800 Message-ID: <20241224011402.134009-10-inwardvessel@gmail.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20241224011402.134009-1-inwardvessel@gmail.com> References: <20241224011402.134009-1-inwardvessel@gmail.com> MIME-Version: 1.0 X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 9CF4FC0016 X-Stat-Signature: pt59ceeyow8anjsx1qwiyb5aaext43bx X-Rspam-User: X-HE-Tag: 1735002824-516925 X-HE-Meta: U2FsdGVkX1/20yfWkA+p3HCzc9pp+h74ZKtoSUGVLqyHTG1YcTY+pr83YjKSa6zwI7pW+ZAd9jG9ggZp82DWftGNCkRboBE0rYRIQFkUd80YdPbcRb3uM8OVPQP9nlVt9VzfLdv4ZsfJlpkAeUgGSwXZqC5TM0Udjh29t4Iaj1ixWxUWP44hX4tAycRGAkUDDQSLg1saND/OtEJ2dDCmLE0n5ODo0eju1XkMTI9VW32f+/uMiESqB3Q9NbrdPnWT/hJtpiicrtQLQAe6ldXzOkbS4kjKYNf8kczm2jg7Vi7mFJ6XXZa4hx3Upa2EUhsFbUEi3ZNaI91+1ISFQ6ppMjwc8fkaf3G0UUkSGYZbH29eWVUdLiQYRERc7WEnSV1SLxj7hC7oygFutDjA1Y+zLXYR/DoOSmjxnWhu1ibPS3SVOPBNftKVdTyongtSvLHyG+0cP20NocU6FZv7KSNKs8Yi4aduB3PxWZWwB+CE5hY7mVKXbtKF3iysC+IR3YwPEuWkwtBfbrOfdyt/HqmE6UHp+IV7vPjJ3S2w2chxOsS3Zs0wiA6WvUZR+PzFrpBV2YD3JS66YytSeXyTvA7RvvUp3YUsZZRBtNkR2C8dpmDPOeq6BI685mtnr0082InJ0TkHwEFqYW6O0m1UDC8uYliRpAUx+xKF1p7Ni3EjGguhXBKe/urq4IOu0Jwq59NejDBnwKVfR05nDlr1GY6zQfVXtBW5AANGVP/qbt4kmHHA66yrzEymSGPp5zHEJ0K0AUokEVKqRn3mJwU4fLjCbsJRuZag9GAgOVK0Qtbt9OCQ3Fts2aCGQRTk86P5pM3yaON7mysb/7+KGr/yJa43lh8ADBaSeLvy7kQwzIw7bptex9nX0+ga+SCT93h3aTkfdRmyVVN/WqNiChkRMnC4CgSJ+AtZjmpwXNPLNLZzaQKU85+easXhH38VPpkQg2qBa6SOt6rZJvTTh0S9Coo gQv/Fwff mNNMBBr1taEHKY3EdQ0mqMToYgT4ntRP4szKw9U/HTqF/IfE1Y8tCAucuIJbWEoii25y9VI15qqUz8eKFMoShsfWUmGjkwHFfKkdCfeHYyHi4/VVdmR3yL1qr84v4RXXJwVBMH0xnWe4uDEKcSIuCwFxwNQP0Pdt+x5K3Jb6jk2WA5HKdIJHcnxHDPnfOw3PxI2MH/RbtcXVCOIguhMqLleI6lxGnRc1K53acPCFwu8+4eHmVPTjEu69OsXeKYpasIBbcVkWKS5Rg/m7gUWo3x8CUuxRW8cfTvIfw5HR5B0VxfmK/HEyD75THnhuzy8812oBguqu05qxlG1y9Y8WaFTWMu034lJ8hGml5sWsOawQnKIsOLOL0DbhtfQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.147277, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: If a given subsystem is not the base type and does not have a flush func, do not allocate. Signed-off-by: JP Kobryn --- kernel/cgroup/rstat.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/kernel/cgroup/rstat.c b/kernel/cgroup/rstat.c index 03effaaf09a4..4feefa37fa46 100644 --- a/kernel/cgroup/rstat.c +++ b/kernel/cgroup/rstat.c @@ -423,6 +423,9 @@ int cgroup_rstat_init(struct cgroup_subsys_state *css) { int cpu; + if (css->ss && !css->ss->css_rstat_flush) + return 0; + /* the root cgrp css has rstat_cpu preallocated */ if (!css->rstat_cpu) { css->rstat_cpu = alloc_percpu(struct cgroup_rstat_cpu); @@ -445,6 +448,9 @@ void cgroup_rstat_exit(struct cgroup_subsys_state *css) { int cpu; + if (css->ss && !css->ss->css_rstat_flush) + return; + cgroup_rstat_flush(css); /* sanity check */