From patchwork Tue Sep 12 07:01:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuyi Zhou X-Patchwork-Id: 13380885 X-Patchwork-Delegate: bpf@iogearbox.net Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 194698F44 for ; Tue, 12 Sep 2023 07:02:04 +0000 (UTC) Received: from mail-pl1-x632.google.com (mail-pl1-x632.google.com [IPv6:2607:f8b0:4864:20::632]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8025110F1 for ; Tue, 12 Sep 2023 00:02:03 -0700 (PDT) Received: by mail-pl1-x632.google.com with SMTP id d9443c01a7336-1c0db66af1bso35316075ad.2 for ; Tue, 12 Sep 2023 00:02:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1694502123; x=1695106923; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=HDkGVzbyVwp5CP2Bj0/gBikBLcGAUp0LBh7IxC9dYSw=; b=FuKgP0rdrCNA8BL2+5RQShyltCoZSh6tDMZSCrIaPkjDxlemdkC06K0GeEsnkYypnY 1P5q0ytA/Wa9uX9rOokYy3bVL7QhC4/mnOg5lGyldv6U9E3UXW94xGBr1iTzzLPwCzNp vznLxsYaTmrIn2+UPKGpAB3ua88fhLAR4+0r37N84H7yhRbk2Y8X8wnRkRlL7Blnn+vG V8N3ePT6EE0isFWsHU6pdYes5qSfPgtFNHjU3wgHZAimBWV7WSApZ0byr5HZCSU1p/qN Hb4+qGeskPP6wsbsiPvxmJNhKV8BG1yop2cson0Os3mCblUf0qc6FplKx6Wq5kZbdgSI 40ww== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694502123; x=1695106923; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=HDkGVzbyVwp5CP2Bj0/gBikBLcGAUp0LBh7IxC9dYSw=; b=ruRA9b8UJ3rQkoa6/BbYcD16BDxKU1LpBlMK8w1OjzCije0RwCRm1JHdCkDO1pMcJ6 H0xseDB+yPvjk1Ombg4t+/bWpvQqJl0H+og6lJvZ91tvpIMgOHETH5IT7NyjbSqfW1aK f+w5XLoh8mVVQxXRe5iZ/77/Ojhr+Na0Dr8kRGfAwxGVHPyVoaeHtNUT+etj1VERb+Jf rHPTh05MVZQM7R9/6czrEhVYb5z33FjQJrwDzvcG6Woml23psqQElZqEzer/0oX25HwA gRfHQlDJU+zxY4HvgAQjYh11mdumgXcttcqRM6td9i2Ssn16M7d+ev03E95fzgSk0DsY QpVw== X-Gm-Message-State: AOJu0YxD1VYG7Fu1WQCOqgf66CKYScjKCmotaFB4ABMgnfX9VdcqEvzh XdNjQjJu+34EB2QaVbTFqkbe/y/PnlcRnV8ToTJGuA== X-Google-Smtp-Source: AGHT+IFckTbjOlPAIMiqK0BYx/GUeBDEJpj6eWIM+X+1r3sNOUrdk6v7pTIn0Cta0Ln/MF1jph7UfQ== X-Received: by 2002:a17:903:2443:b0:1c3:d864:d6b3 with SMTP id l3-20020a170903244300b001c3d864d6b3mr482903pls.57.1694502122788; Tue, 12 Sep 2023 00:02:02 -0700 (PDT) Received: from n37-019-243.byted.org ([180.184.84.173]) by smtp.gmail.com with ESMTPSA id b8-20020a170902d50800b001b8953365aesm7635401plg.22.2023.09.12.00.02.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Sep 2023 00:02:02 -0700 (PDT) From: Chuyi Zhou To: bpf@vger.kernel.org Cc: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, martin.lau@kernel.org, tj@kernel.org, linux-kernel@vger.kernel.org, Chuyi Zhou Subject: [PATCH bpf-next v2 1/6] cgroup: Prepare for using css_task_iter_*() in BPF Date: Tue, 12 Sep 2023 15:01:44 +0800 Message-Id: <20230912070149.969939-2-zhouchuyi@bytedance.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20230912070149.969939-1-zhouchuyi@bytedance.com> References: <20230912070149.969939-1-zhouchuyi@bytedance.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net This patch makes some preparations for using css_task_iter_*() in BPF Program. 1. Flags CSS_TASK_ITER_* are #define-s and it's not easy for bpf prog to use them. Convert them to enum so bpf prog can take them from vmlinux.h. 2. In the next patch we will add css_task_iter_*() in common kfuncs which is not safe. Since css_task_iter_*() does spin_unlock_irq() which might screw up irq flags depending on the context where bpf prog is running. So we should use irqsave/irqrestore here and the switching is harmless. Signed-off-by: Chuyi Zhou Acked-by: Tejun Heo --- include/linux/cgroup.h | 12 +++++------- kernel/cgroup/cgroup.c | 18 ++++++++++++------ 2 files changed, 17 insertions(+), 13 deletions(-) diff --git a/include/linux/cgroup.h b/include/linux/cgroup.h index b307013b9c6c..0ef0af66080e 100644 --- a/include/linux/cgroup.h +++ b/include/linux/cgroup.h @@ -40,13 +40,11 @@ struct kernel_clone_args; #define CGROUP_WEIGHT_DFL 100 #define CGROUP_WEIGHT_MAX 10000 -/* walk only threadgroup leaders */ -#define CSS_TASK_ITER_PROCS (1U << 0) -/* walk all threaded css_sets in the domain */ -#define CSS_TASK_ITER_THREADED (1U << 1) - -/* internal flags */ -#define CSS_TASK_ITER_SKIPPED (1U << 16) +enum { + CSS_TASK_ITER_PROCS = (1U << 0), /* walk only threadgroup leaders */ + CSS_TASK_ITER_THREADED = (1U << 1), /* walk all threaded css_sets in the domain */ + CSS_TASK_ITER_SKIPPED = (1U << 16), /* internal flags */ +}; /* a css_task_iter should be treated as an opaque object */ struct css_task_iter { diff --git a/kernel/cgroup/cgroup.c b/kernel/cgroup/cgroup.c index 1fb7f562289d..b6d64f3b8888 100644 --- a/kernel/cgroup/cgroup.c +++ b/kernel/cgroup/cgroup.c @@ -4917,9 +4917,11 @@ static void css_task_iter_advance(struct css_task_iter *it) void css_task_iter_start(struct cgroup_subsys_state *css, unsigned int flags, struct css_task_iter *it) { + unsigned long irqflags; + memset(it, 0, sizeof(*it)); - spin_lock_irq(&css_set_lock); + spin_lock_irqsave(&css_set_lock, irqflags); it->ss = css->ss; it->flags = flags; @@ -4933,7 +4935,7 @@ void css_task_iter_start(struct cgroup_subsys_state *css, unsigned int flags, css_task_iter_advance(it); - spin_unlock_irq(&css_set_lock); + spin_unlock_irqrestore(&css_set_lock, irqflags); } /** @@ -4946,12 +4948,14 @@ void css_task_iter_start(struct cgroup_subsys_state *css, unsigned int flags, */ struct task_struct *css_task_iter_next(struct css_task_iter *it) { + unsigned long irqflags; + if (it->cur_task) { put_task_struct(it->cur_task); it->cur_task = NULL; } - spin_lock_irq(&css_set_lock); + spin_lock_irqsave(&css_set_lock, irqflags); /* @it may be half-advanced by skips, finish advancing */ if (it->flags & CSS_TASK_ITER_SKIPPED) @@ -4964,7 +4968,7 @@ struct task_struct *css_task_iter_next(struct css_task_iter *it) css_task_iter_advance(it); } - spin_unlock_irq(&css_set_lock); + spin_unlock_irqrestore(&css_set_lock, irqflags); return it->cur_task; } @@ -4977,11 +4981,13 @@ struct task_struct *css_task_iter_next(struct css_task_iter *it) */ void css_task_iter_end(struct css_task_iter *it) { + unsigned long irqflags; + if (it->cur_cset) { - spin_lock_irq(&css_set_lock); + spin_lock_irqsave(&css_set_lock, irqflags); list_del(&it->iters_node); put_css_set_locked(it->cur_cset); - spin_unlock_irq(&css_set_lock); + spin_unlock_irqrestore(&css_set_lock, irqflags); } if (it->cur_dcset) From patchwork Tue Sep 12 07:01:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuyi Zhou X-Patchwork-Id: 13380886 X-Patchwork-Delegate: bpf@iogearbox.net Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 914898F44 for ; Tue, 12 Sep 2023 07:02:07 +0000 (UTC) Received: from mail-pf1-x429.google.com (mail-pf1-x429.google.com [IPv6:2607:f8b0:4864:20::429]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DBBECE78 for ; Tue, 12 Sep 2023 00:02:06 -0700 (PDT) Received: by mail-pf1-x429.google.com with SMTP id d2e1a72fcca58-68fba57030fso1746987b3a.3 for ; Tue, 12 Sep 2023 00:02:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1694502126; x=1695106926; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=FNQpGQe4u79p1mkZVJD58U2KIJEFL4MZuGQeXTwOQ8w=; b=Dow5IJyKwkOy28TWDR9DDnjAogp9ctHCGYgTIiopVm61CJBX4KtSrMGSU1tt3U8dZX t8jMpD60rg5/hDwiqUJeq9z4WKLQwBXfV7ALggUQeh7PqPCrti/rKseXZU4/2/ivg1/M t5mdw24PCiXIwotA7LSzjM5CpbpQpODL/Dn/30HR64J/9e4rJhIpvl9l4hPrgOJbqsfJ RkROP7y53WJoqvfKDUMZYjz0QnDf7EWX/eM2mjGuGXMgnfMuOkyDw0Xe/nQTbeKaVznR bLX1GuNfRU/ADul/SXbTD7ZuEl25jbAoJdvDAO7ZRl2/CLRyqKPi+dt4vKy/NLOY/Y8K a6lA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694502126; x=1695106926; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=FNQpGQe4u79p1mkZVJD58U2KIJEFL4MZuGQeXTwOQ8w=; b=MdJ8P76xkvOQQ6CECkhsihNtgz2m1XIW7DlktwkCEWLDIAEM9GS3yhWZ176iXxuqDP vHDhFP7I7I5xsLsNxgFEwKixJeMKQT3eKMY6vAiIx/eRPblTyNzDo4SrUSLsmabrMknu HQZitGP0uX/cQoS0KNOCAETaRgJDF2ZaLHS/gv5afU4mpUeFq9wDr7WaJIvisTSwXjAZ br6I0aL/+Wk5M9m7zyJBYbbPT3Q3fGYhk1K/UUW2zsX53OnpfxpQUyBDq49OlyTh5dqZ oQAQWg4/D8Dw1ASgla1pOP8V8QuZoxQ0JUetZ7xPb98S0ZjgzWV0MifOtzSQMuII6NHl TJFw== X-Gm-Message-State: AOJu0YyjJ2YVDHzAFpHizjJWn1dxBrxUlu2bhrPHQM8k5eLd4GbLPKOt HrWV68Trp/wylbkrmnxOh6Nc0LrIKMxdOFRElx4CnQ== X-Google-Smtp-Source: AGHT+IFrGkPdm7n165iwKopet18YYKmVo1jjoqtm9QF2jj2jLPZw8NkCsAYe/iI/yLg8hGzAh6dENw== X-Received: by 2002:a05:6a20:ce9f:b0:153:c9fa:4ed6 with SMTP id if31-20020a056a20ce9f00b00153c9fa4ed6mr8870801pzb.10.1694502126142; Tue, 12 Sep 2023 00:02:06 -0700 (PDT) Received: from n37-019-243.byted.org ([180.184.84.173]) by smtp.gmail.com with ESMTPSA id b8-20020a170902d50800b001b8953365aesm7635401plg.22.2023.09.12.00.02.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Sep 2023 00:02:05 -0700 (PDT) From: Chuyi Zhou To: bpf@vger.kernel.org Cc: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, martin.lau@kernel.org, tj@kernel.org, linux-kernel@vger.kernel.org, Chuyi Zhou Subject: [PATCH bpf-next v2 2/6] bpf: Introduce css_task open-coded iterator kfuncs Date: Tue, 12 Sep 2023 15:01:45 +0800 Message-Id: <20230912070149.969939-3-zhouchuyi@bytedance.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20230912070149.969939-1-zhouchuyi@bytedance.com> References: <20230912070149.969939-1-zhouchuyi@bytedance.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net This patch adds kfuncs bpf_iter_css_task_{new,next,destroy} which allow creation and manipulation of struct bpf_iter_css_task in open-coded iterator style. These kfuncs actually wrapps css_task_iter_{start,next, end}. BPF programs can use these kfuncs through bpf_for_each macro for iteration of all tasks under a css. css_task_iter_*() would try to get the global spin-lock *css_set_lock*, so the bpf side has to be careful in where it allows to use this iter. Currently we only allow it in bpf_lsm and bpf iter-s. Signed-off-by: Chuyi Zhou --- include/uapi/linux/bpf.h | 4 +++ kernel/bpf/helpers.c | 3 +++ kernel/bpf/task_iter.c | 48 ++++++++++++++++++++++++++++++++++ kernel/bpf/verifier.c | 23 ++++++++++++++++ tools/include/uapi/linux/bpf.h | 4 +++ tools/lib/bpf/bpf_helpers.h | 7 +++++ 6 files changed, 89 insertions(+) diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index 73b155e52204..de02c0971428 100644 --- a/include/uapi/linux/bpf.h +++ b/include/uapi/linux/bpf.h @@ -7318,4 +7318,8 @@ struct bpf_iter_num { __u64 __opaque[1]; } __attribute__((aligned(8))); +struct bpf_iter_css_task { + __u64 __opaque[1]; +} __attribute__((aligned(8))); + #endif /* _UAPI__LINUX_BPF_H__ */ diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c index b0a9834f1051..d6a16becfbb9 100644 --- a/kernel/bpf/helpers.c +++ b/kernel/bpf/helpers.c @@ -2504,6 +2504,9 @@ BTF_ID_FLAGS(func, bpf_dynptr_slice_rdwr, KF_RET_NULL) BTF_ID_FLAGS(func, bpf_iter_num_new, KF_ITER_NEW) BTF_ID_FLAGS(func, bpf_iter_num_next, KF_ITER_NEXT | KF_RET_NULL) BTF_ID_FLAGS(func, bpf_iter_num_destroy, KF_ITER_DESTROY) +BTF_ID_FLAGS(func, bpf_iter_css_task_new, KF_ITER_NEW) +BTF_ID_FLAGS(func, bpf_iter_css_task_next, KF_ITER_NEXT | KF_RET_NULL) +BTF_ID_FLAGS(func, bpf_iter_css_task_destroy, KF_ITER_DESTROY) BTF_ID_FLAGS(func, bpf_dynptr_adjust) BTF_ID_FLAGS(func, bpf_dynptr_is_null) BTF_ID_FLAGS(func, bpf_dynptr_is_rdonly) diff --git a/kernel/bpf/task_iter.c b/kernel/bpf/task_iter.c index 7473068ed313..d8539cc05ffd 100644 --- a/kernel/bpf/task_iter.c +++ b/kernel/bpf/task_iter.c @@ -803,6 +803,54 @@ const struct bpf_func_proto bpf_find_vma_proto = { .arg5_type = ARG_ANYTHING, }; +struct bpf_iter_css_task_kern { + struct css_task_iter *css_it; +} __attribute__((aligned(8))); + +__bpf_kfunc int bpf_iter_css_task_new(struct bpf_iter_css_task *it, + struct cgroup_subsys_state *css, unsigned int flags) +{ + struct bpf_iter_css_task_kern *kit = (void *)it; + + BUILD_BUG_ON(sizeof(struct bpf_iter_css_task_kern) != sizeof(struct bpf_iter_css_task)); + BUILD_BUG_ON(__alignof__(struct bpf_iter_css_task_kern) != + __alignof__(struct bpf_iter_css_task)); + + switch (flags) { + case CSS_TASK_ITER_PROCS | CSS_TASK_ITER_THREADED: + case CSS_TASK_ITER_PROCS: + case 0: + break; + default: + return -EINVAL; + } + + kit->css_it = kzalloc(sizeof(struct css_task_iter), GFP_KERNEL); + if (!kit->css_it) + return -ENOMEM; + css_task_iter_start(css, flags, kit->css_it); + return 0; +} + +__bpf_kfunc struct task_struct *bpf_iter_css_task_next(struct bpf_iter_css_task *it) +{ + struct bpf_iter_css_task_kern *kit = (void *)it; + + if (!kit->css_it) + return NULL; + return css_task_iter_next(kit->css_it); +} + +__bpf_kfunc void bpf_iter_css_task_destroy(struct bpf_iter_css_task *it) +{ + struct bpf_iter_css_task_kern *kit = (void *)it; + + if (!kit->css_it) + return; + css_task_iter_end(kit->css_it); + kfree(kit->css_it); +} + DEFINE_PER_CPU(struct mmap_unlock_irq_work, mmap_unlock_work); static void do_mmap_read_unlock(struct irq_work *entry) diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index dbba2b806017..2367483bf4c2 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -10332,6 +10332,7 @@ enum special_kfunc_type { KF_bpf_dynptr_clone, KF_bpf_percpu_obj_new_impl, KF_bpf_percpu_obj_drop_impl, + KF_bpf_iter_css_task_new, }; BTF_SET_START(special_kfunc_set) @@ -10354,6 +10355,7 @@ BTF_ID(func, bpf_dynptr_slice_rdwr) BTF_ID(func, bpf_dynptr_clone) BTF_ID(func, bpf_percpu_obj_new_impl) BTF_ID(func, bpf_percpu_obj_drop_impl) +BTF_ID(func, bpf_iter_css_task_new) BTF_SET_END(special_kfunc_set) BTF_ID_LIST(special_kfunc_list) @@ -10378,6 +10380,7 @@ BTF_ID(func, bpf_dynptr_slice_rdwr) BTF_ID(func, bpf_dynptr_clone) BTF_ID(func, bpf_percpu_obj_new_impl) BTF_ID(func, bpf_percpu_obj_drop_impl) +BTF_ID(func, bpf_iter_css_task_new) static bool is_kfunc_ret_null(struct bpf_kfunc_call_arg_meta *meta) { @@ -10902,6 +10905,20 @@ static int process_kf_arg_ptr_to_rbtree_node(struct bpf_verifier_env *env, &meta->arg_rbtree_root.field); } +static bool check_css_task_iter_allowlist(struct bpf_verifier_env *env) +{ + enum bpf_prog_type prog_type = resolve_prog_type(env->prog); + + switch (prog_type) { + case BPF_PROG_TYPE_LSM: + return true; + case BPF_TRACE_ITER: + return env->prog->aux->sleepable; + default: + return false; + } +} + static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_arg_meta *meta, int insn_idx) { @@ -11152,6 +11169,12 @@ static int check_kfunc_args(struct bpf_verifier_env *env, struct bpf_kfunc_call_ break; } case KF_ARG_PTR_TO_ITER: + if (meta->func_id == special_kfunc_list[KF_bpf_iter_css_task_new]) { + if (!check_css_task_iter_allowlist(env)) { + verbose(env, "css_task_iter is only allowed in bpf_lsm and bpf iter-s\n"); + return -EINVAL; + } + } ret = process_iter_arg(env, regno, insn_idx, meta); if (ret < 0) return ret; diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h index 73b155e52204..de02c0971428 100644 --- a/tools/include/uapi/linux/bpf.h +++ b/tools/include/uapi/linux/bpf.h @@ -7318,4 +7318,8 @@ struct bpf_iter_num { __u64 __opaque[1]; } __attribute__((aligned(8))); +struct bpf_iter_css_task { + __u64 __opaque[1]; +} __attribute__((aligned(8))); + #endif /* _UAPI__LINUX_BPF_H__ */ diff --git a/tools/lib/bpf/bpf_helpers.h b/tools/lib/bpf/bpf_helpers.h index 77ceea575dc7..f48723c6c593 100644 --- a/tools/lib/bpf/bpf_helpers.h +++ b/tools/lib/bpf/bpf_helpers.h @@ -303,6 +303,13 @@ extern int bpf_iter_num_new(struct bpf_iter_num *it, int start, int end) __weak extern int *bpf_iter_num_next(struct bpf_iter_num *it) __weak __ksym; extern void bpf_iter_num_destroy(struct bpf_iter_num *it) __weak __ksym; +struct bpf_iter_css_task; +struct cgroup_subsys_state; +extern int bpf_iter_css_task_new(struct bpf_iter_css_task *it, + struct cgroup_subsys_state *css, unsigned int flags) __weak __ksym; +extern struct task_struct *bpf_iter_css_task_next(struct bpf_iter_css_task *it) __weak __ksym; +extern void bpf_iter_css_task_destroy(struct bpf_iter_css_task *it) __weak __ksym; + #ifndef bpf_for_each /* bpf_for_each(iter_type, cur_elem, args...) provides generic construct for * using BPF open-coded iterators without having to write mundane explicit From patchwork Tue Sep 12 07:01:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuyi Zhou X-Patchwork-Id: 13380887 X-Patchwork-Delegate: bpf@iogearbox.net Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D37878F44 for ; Tue, 12 Sep 2023 07:02:10 +0000 (UTC) Received: from mail-pl1-x62a.google.com (mail-pl1-x62a.google.com [IPv6:2607:f8b0:4864:20::62a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2DF83E7F for ; Tue, 12 Sep 2023 00:02:10 -0700 (PDT) Received: by mail-pl1-x62a.google.com with SMTP id d9443c01a7336-1c3d8fb23d9so1044365ad.0 for ; Tue, 12 Sep 2023 00:02:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1694502129; x=1695106929; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=+2pL38sdR1zkqvND06hEOkj6aCyDQM/YMk3Q7fUlrh4=; b=M2IuHdH8p6VL/T+r4KhufKVPTFU9Qdd3Vj8fRX16Ps/8Jex8HpP2UTkOUzS3bE7/oW p0lcRAwO7+0E3dt+h7uqMRwB2GqCI5g1PuSu6FZ9Ct0E7OnEX2gq5qyVLx3IIsoPkP6Q /2dD5jdLkzIsjH5bpUst9xbVHww97Jsxzgzlp5UJ2M4/+FHtUSdVd/tN+dxIcr/kq/bS GO2r9XE86jJyy7IkIzNpEanGMne3qAOb/KEHS2MlvGdiAEYv9VkI7cR7SuirsS019hcM DYFOZjhfY7g19waibSAbfQyTU+hR1nhYAnO2Bmms4oPTE+dMd4ikYAN0QIGgiINaHiuJ Vnzw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694502129; x=1695106929; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=+2pL38sdR1zkqvND06hEOkj6aCyDQM/YMk3Q7fUlrh4=; b=PhPBUsbJ/75c7nwnDJBmk4rfBhq36x9VRRyQM/2IsX+eD6x219n/enuDkvNz2QI1oC XQ3kKRR2cMu8w0Pub7chf+SxyDZs4HZZV/bKoPWS8/fexpoj/HbjYtIFTAZMsPCEp/oG f7+hqk5eBNAy1ZpoBK2OLAf0WWDKOg+LgCnxFcyMvMM/SSP1wxA/dda5DewbN8N8dEtq ifzs/5uy0K1IM0kRPukeYFLKb66WV4iExi/GyRk8AdvFW6ZnjSow7LPaf8mdyGQgXZN9 SbZrRVYtFCcjg+Bnt/K0/42hMeWat1rTTqcQjHTLy394W6xcAbGGEerollWehBUGG/qF t6cw== X-Gm-Message-State: AOJu0YxOdWg8NYxmiIEcfuO5BPxmO+oFoyDtXaZhlBtAvfqHFphpoT/X ze2iwZ8O0SLxX3SrBEmC8c2U+YlhgXe6CM3PpZ27QA== X-Google-Smtp-Source: AGHT+IFLHzQBh59ISNd1Ew9UXrAaH+IQXBWi+Di1oQTdwgCpmr79ELmZXckxp25ZL61NyEOj/fQqLg== X-Received: by 2002:a17:903:2302:b0:1bb:e74b:39ff with SMTP id d2-20020a170903230200b001bbe74b39ffmr12134008plh.0.1694502129564; Tue, 12 Sep 2023 00:02:09 -0700 (PDT) Received: from n37-019-243.byted.org ([180.184.84.173]) by smtp.gmail.com with ESMTPSA id b8-20020a170902d50800b001b8953365aesm7635401plg.22.2023.09.12.00.02.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Sep 2023 00:02:09 -0700 (PDT) From: Chuyi Zhou To: bpf@vger.kernel.org Cc: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, martin.lau@kernel.org, tj@kernel.org, linux-kernel@vger.kernel.org, Chuyi Zhou Subject: [PATCH bpf-next v2 3/6] bpf: Introduce process open coded iterator kfuncs Date: Tue, 12 Sep 2023 15:01:46 +0800 Message-Id: <20230912070149.969939-4-zhouchuyi@bytedance.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20230912070149.969939-1-zhouchuyi@bytedance.com> References: <20230912070149.969939-1-zhouchuyi@bytedance.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net This patch adds kfuncs bpf_iter_process_{new,next,destroy} which allow creation and manipulation of struct bpf_iter_process in open-coded iterator style. BPF programs can use these kfuncs or through bpf_for_each macro to iterate all processes in the system. Signed-off-by: Chuyi Zhou --- include/uapi/linux/bpf.h | 4 ++++ kernel/bpf/helpers.c | 3 +++ kernel/bpf/task_iter.c | 29 +++++++++++++++++++++++++++++ tools/include/uapi/linux/bpf.h | 4 ++++ tools/lib/bpf/bpf_helpers.h | 5 +++++ 5 files changed, 45 insertions(+) diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index de02c0971428..befa55b52e29 100644 --- a/include/uapi/linux/bpf.h +++ b/include/uapi/linux/bpf.h @@ -7322,4 +7322,8 @@ struct bpf_iter_css_task { __u64 __opaque[1]; } __attribute__((aligned(8))); +struct bpf_iter_process { + __u64 __opaque[1]; +} __attribute__((aligned(8))); + #endif /* _UAPI__LINUX_BPF_H__ */ diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c index d6a16becfbb9..9b7d2c6f99d1 100644 --- a/kernel/bpf/helpers.c +++ b/kernel/bpf/helpers.c @@ -2507,6 +2507,9 @@ BTF_ID_FLAGS(func, bpf_iter_num_destroy, KF_ITER_DESTROY) BTF_ID_FLAGS(func, bpf_iter_css_task_new, KF_ITER_NEW) BTF_ID_FLAGS(func, bpf_iter_css_task_next, KF_ITER_NEXT | KF_RET_NULL) BTF_ID_FLAGS(func, bpf_iter_css_task_destroy, KF_ITER_DESTROY) +BTF_ID_FLAGS(func, bpf_iter_process_new, KF_ITER_NEW) +BTF_ID_FLAGS(func, bpf_iter_process_next, KF_ITER_NEXT | KF_RET_NULL) +BTF_ID_FLAGS(func, bpf_iter_process_destroy, KF_ITER_DESTROY) BTF_ID_FLAGS(func, bpf_dynptr_adjust) BTF_ID_FLAGS(func, bpf_dynptr_is_null) BTF_ID_FLAGS(func, bpf_dynptr_is_rdonly) diff --git a/kernel/bpf/task_iter.c b/kernel/bpf/task_iter.c index d8539cc05ffd..9d1927dc3a06 100644 --- a/kernel/bpf/task_iter.c +++ b/kernel/bpf/task_iter.c @@ -851,6 +851,35 @@ __bpf_kfunc void bpf_iter_css_task_destroy(struct bpf_iter_css_task *it) kfree(kit->css_it); } +struct bpf_iter_process_kern { + struct task_struct *tsk; +} __attribute__((aligned(8))); + +__bpf_kfunc int bpf_iter_process_new(struct bpf_iter_process *it) +{ + struct bpf_iter_process_kern *kit = (void *)it; + + BUILD_BUG_ON(sizeof(struct bpf_iter_process_kern) != sizeof(struct bpf_iter_process)); + BUILD_BUG_ON(__alignof__(struct bpf_iter_process_kern) != + __alignof__(struct bpf_iter_process)); + + kit->tsk = &init_task; + return 0; +} + +__bpf_kfunc struct task_struct *bpf_iter_process_next(struct bpf_iter_process *it) +{ + struct bpf_iter_process_kern *kit = (void *)it; + + kit->tsk = next_task(kit->tsk); + + return kit->tsk == &init_task ? NULL : kit->tsk; +} + +__bpf_kfunc void bpf_iter_process_destroy(struct bpf_iter_process *it) +{ +} + DEFINE_PER_CPU(struct mmap_unlock_irq_work, mmap_unlock_work); static void do_mmap_read_unlock(struct irq_work *entry) diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h index de02c0971428..befa55b52e29 100644 --- a/tools/include/uapi/linux/bpf.h +++ b/tools/include/uapi/linux/bpf.h @@ -7322,4 +7322,8 @@ struct bpf_iter_css_task { __u64 __opaque[1]; } __attribute__((aligned(8))); +struct bpf_iter_process { + __u64 __opaque[1]; +} __attribute__((aligned(8))); + #endif /* _UAPI__LINUX_BPF_H__ */ diff --git a/tools/lib/bpf/bpf_helpers.h b/tools/lib/bpf/bpf_helpers.h index f48723c6c593..858252c2641c 100644 --- a/tools/lib/bpf/bpf_helpers.h +++ b/tools/lib/bpf/bpf_helpers.h @@ -310,6 +310,11 @@ extern int bpf_iter_css_task_new(struct bpf_iter_css_task *it, extern struct task_struct *bpf_iter_css_task_next(struct bpf_iter_css_task *it) __weak __ksym; extern void bpf_iter_css_task_destroy(struct bpf_iter_css_task *it) __weak __ksym; +struct bpf_iter_process; +extern int bpf_iter_process_new(struct bpf_iter_process *it) __weak __ksym; +extern struct task_struct *bpf_iter_process_next(struct bpf_iter_process *it) __weak __ksym; +extern void bpf_iter_process_destroy(struct bpf_iter_process *it) __weak __ksym; + #ifndef bpf_for_each /* bpf_for_each(iter_type, cur_elem, args...) provides generic construct for * using BPF open-coded iterators without having to write mundane explicit From patchwork Tue Sep 12 07:01:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuyi Zhou X-Patchwork-Id: 13380888 X-Patchwork-Delegate: bpf@iogearbox.net Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1B9B28F44 for ; Tue, 12 Sep 2023 07:02:14 +0000 (UTC) Received: from mail-pl1-x636.google.com (mail-pl1-x636.google.com [IPv6:2607:f8b0:4864:20::636]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8F876E78 for ; Tue, 12 Sep 2023 00:02:13 -0700 (PDT) Received: by mail-pl1-x636.google.com with SMTP id d9443c01a7336-1c1f8aaab9aso45900165ad.1 for ; Tue, 12 Sep 2023 00:02:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1694502133; x=1695106933; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=zbzquHAnwok7U8VH5ASiuRWtN2QtgmEjZHX4A6sEdQ0=; b=EgE/SEvhM6y7H0wOdoJWkDzV29CKcPkS3iWL8GssBkcc0O9ymUk5IhMnFKhvYVz7M0 OhAQpdsl5k1naQB1oRrSa/OXFxDDPGjdfgZG/ECrAQa2+x71LlfB7pXbFUe4Ox/t9F77 cLIcHTy2RaBomVkoR/Kljy5ollN4wNcDVT6/jAE5Xi/aZtEhd7wzp8WKSN2XV4qi4hSs 7+fs9Ifk/Us6R8GOVpjdC57+grmarGBjpPdJIKcUIgqzTPIyM5tMTYV4ZqmplsSyMVsH bQC6P6th9E06JuHAEDCV018vo8zzU5d4N6htKV8QC2a9LDXHUL8vF3u7yKRamWZKRcZT S5GA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694502133; x=1695106933; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=zbzquHAnwok7U8VH5ASiuRWtN2QtgmEjZHX4A6sEdQ0=; b=rv6e/WnRwJ08KE5ujpwiUDkQYn3OD2IVrihy8Ga6ZmqjqwnrtR01Iaj2+k1QF13lIp 2Ay34GzPVpWk35GvAtJCBL8mtBtU06Jq8sVazMunCnoWMVjoEytiUX+9UYAjfNs0lmRP 2PdnX4/Q5p0i12pbhJCHn8BB6mM/Kl3nm3QeLgNVCstkpvFiAznl30ZfyVgLPqAB5hKP AziE9VUImlYIYYzy9ksaN4vK5IgdKnYNOX4B64mmRsD33guZVdpJDZ44a1+B2rlJQEb2 LGoHjERRr222u7VD6Ighg87t9qP2HQQe6q6Ajip5PehlqPo5x33WQ1fn2jSIkDENEzTf EXUA== X-Gm-Message-State: AOJu0YxZRQqzCBp37ZOnpC6MpAJKzEdWSmcchjhadBLBtehvPvSvdaeY cwCGJboo/YMx4oK+IWlNUAh7iOptwV5kevsoy2PKlg== X-Google-Smtp-Source: AGHT+IH3nN6SdkGHHenKlH89H8T3xLnbtavEibYfCQL2fxVa7I7VNXrgt1h0+smga0DjUw8E5wv8fA== X-Received: by 2002:a17:902:8693:b0:1b8:66f6:87a3 with SMTP id g19-20020a170902869300b001b866f687a3mr10957357plo.52.1694502132925; Tue, 12 Sep 2023 00:02:12 -0700 (PDT) Received: from n37-019-243.byted.org ([180.184.84.173]) by smtp.gmail.com with ESMTPSA id b8-20020a170902d50800b001b8953365aesm7635401plg.22.2023.09.12.00.02.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Sep 2023 00:02:12 -0700 (PDT) From: Chuyi Zhou To: bpf@vger.kernel.org Cc: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, martin.lau@kernel.org, tj@kernel.org, linux-kernel@vger.kernel.org, Chuyi Zhou Subject: [PATCH bpf-next v2 4/6] bpf: Introduce css_descendant open-coded iterator kfuncs Date: Tue, 12 Sep 2023 15:01:47 +0800 Message-Id: <20230912070149.969939-5-zhouchuyi@bytedance.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20230912070149.969939-1-zhouchuyi@bytedance.com> References: <20230912070149.969939-1-zhouchuyi@bytedance.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net This Patch adds kfuncs bpf_iter_css_{pre,post}_{new,next,destroy} which allow creation and manipulation of struct bpf_iter_css in open-coded iterator style. These kfuncs actually wrapps css_next_descendant_{pre, post}. BPF programs can use these kfuncs through bpf_for_each macro for iteration of all descendant css under a root css. Signed-off-by: Chuyi Zhou --- include/uapi/linux/bpf.h | 8 +++++ kernel/bpf/helpers.c | 6 ++++ kernel/bpf/task_iter.c | 53 ++++++++++++++++++++++++++++++++++ tools/include/uapi/linux/bpf.h | 8 +++++ tools/lib/bpf/bpf_helpers.h | 12 ++++++++ 5 files changed, 87 insertions(+) diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index befa55b52e29..57760afc13d0 100644 --- a/include/uapi/linux/bpf.h +++ b/include/uapi/linux/bpf.h @@ -7326,4 +7326,12 @@ struct bpf_iter_process { __u64 __opaque[1]; } __attribute__((aligned(8))); +struct bpf_iter_css_pre { + __u64 __opaque[2]; +} __attribute__((aligned(8))); + +struct bpf_iter_css_post { + __u64 __opaque[2]; +} __attribute__((aligned(8))); + #endif /* _UAPI__LINUX_BPF_H__ */ diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c index 9b7d2c6f99d1..ca1f6404af9e 100644 --- a/kernel/bpf/helpers.c +++ b/kernel/bpf/helpers.c @@ -2510,6 +2510,12 @@ BTF_ID_FLAGS(func, bpf_iter_css_task_destroy, KF_ITER_DESTROY) BTF_ID_FLAGS(func, bpf_iter_process_new, KF_ITER_NEW) BTF_ID_FLAGS(func, bpf_iter_process_next, KF_ITER_NEXT | KF_RET_NULL) BTF_ID_FLAGS(func, bpf_iter_process_destroy, KF_ITER_DESTROY) +BTF_ID_FLAGS(func, bpf_iter_css_pre_new, KF_ITER_NEW) +BTF_ID_FLAGS(func, bpf_iter_css_pre_next, KF_ITER_NEXT | KF_RET_NULL) +BTF_ID_FLAGS(func, bpf_iter_css_pre_destroy, KF_ITER_DESTROY) +BTF_ID_FLAGS(func, bpf_iter_css_post_new, KF_ITER_NEW) +BTF_ID_FLAGS(func, bpf_iter_css_post_next, KF_ITER_NEXT | KF_RET_NULL) +BTF_ID_FLAGS(func, bpf_iter_css_post_destroy, KF_ITER_DESTROY) BTF_ID_FLAGS(func, bpf_dynptr_adjust) BTF_ID_FLAGS(func, bpf_dynptr_is_null) BTF_ID_FLAGS(func, bpf_dynptr_is_rdonly) diff --git a/kernel/bpf/task_iter.c b/kernel/bpf/task_iter.c index 9d1927dc3a06..8963fc779b87 100644 --- a/kernel/bpf/task_iter.c +++ b/kernel/bpf/task_iter.c @@ -880,6 +880,59 @@ __bpf_kfunc void bpf_iter_process_destroy(struct bpf_iter_process *it) { } +struct bpf_iter_css_kern { + struct cgroup_subsys_state *root; + struct cgroup_subsys_state *pos; +} __attribute__((aligned(8))); + +__bpf_kfunc int bpf_iter_css_pre_new(struct bpf_iter_css_pre *it, + struct cgroup_subsys_state *root) +{ + struct bpf_iter_css_kern *kit = (void *)it; + + BUILD_BUG_ON(sizeof(struct bpf_iter_css_kern) != sizeof(struct bpf_iter_css_pre)); + BUILD_BUG_ON(__alignof__(struct bpf_iter_css_kern) != __alignof__(struct bpf_iter_css_pre)); + kit->root = root; + kit->pos = NULL; + return 0; +} + +__bpf_kfunc struct cgroup_subsys_state *bpf_iter_css_pre_next(struct bpf_iter_css_pre *it) +{ + struct bpf_iter_css_kern *kit = (void *)it; + + kit->pos = css_next_descendant_pre(kit->pos, kit->root); + return kit->pos; +} + +__bpf_kfunc void bpf_iter_css_pre_destroy(struct bpf_iter_css_pre *it) +{ +} + +__bpf_kfunc int bpf_iter_css_post_new(struct bpf_iter_css_post *it, + struct cgroup_subsys_state *root) +{ + struct bpf_iter_css_kern *kit = (void *)it; + + BUILD_BUG_ON(sizeof(struct bpf_iter_css_kern) != sizeof(struct bpf_iter_css_post)); + BUILD_BUG_ON(__alignof__(struct bpf_iter_css_kern) != __alignof__(struct bpf_iter_css_post)); + kit->root = root; + kit->pos = NULL; + return 0; +} + +__bpf_kfunc struct cgroup_subsys_state *bpf_iter_css_post_next(struct bpf_iter_css_post *it) +{ + struct bpf_iter_css_kern *kit = (void *)it; + + kit->pos = css_next_descendant_post(kit->pos, kit->root); + return kit->pos; +} + +__bpf_kfunc void bpf_iter_css_post_destroy(struct bpf_iter_css_post *it) +{ +} + DEFINE_PER_CPU(struct mmap_unlock_irq_work, mmap_unlock_work); static void do_mmap_read_unlock(struct irq_work *entry) diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h index befa55b52e29..57760afc13d0 100644 --- a/tools/include/uapi/linux/bpf.h +++ b/tools/include/uapi/linux/bpf.h @@ -7326,4 +7326,12 @@ struct bpf_iter_process { __u64 __opaque[1]; } __attribute__((aligned(8))); +struct bpf_iter_css_pre { + __u64 __opaque[2]; +} __attribute__((aligned(8))); + +struct bpf_iter_css_post { + __u64 __opaque[2]; +} __attribute__((aligned(8))); + #endif /* _UAPI__LINUX_BPF_H__ */ diff --git a/tools/lib/bpf/bpf_helpers.h b/tools/lib/bpf/bpf_helpers.h index 858252c2641c..6e5bd9ef14d6 100644 --- a/tools/lib/bpf/bpf_helpers.h +++ b/tools/lib/bpf/bpf_helpers.h @@ -315,6 +315,18 @@ extern int bpf_iter_process_new(struct bpf_iter_process *it) __weak __ksym; extern struct task_struct *bpf_iter_process_next(struct bpf_iter_process *it) __weak __ksym; extern void bpf_iter_process_destroy(struct bpf_iter_process *it) __weak __ksym; +struct bpf_iter_css_pre; +extern int bpf_iter_css_pre_new(struct bpf_iter_css_pre *it, + struct cgroup_subsys_state *root) __weak __ksym; +extern struct cgroup_subsys_state *bpf_iter_css_pre_next(struct bpf_iter_css_pre *it) __weak __ksym; +extern void bpf_iter_css_pre_destroy(struct bpf_iter_css_pre *it) __weak __ksym; + +struct bpf_iter_css_post; +extern int bpf_iter_css_post_new(struct bpf_iter_css_post *it, + struct cgroup_subsys_state *root) __weak __ksym; +extern struct cgroup_subsys_state *bpf_iter_css_post_next(struct bpf_iter_css_post *it) __weak __ksym; +extern void bpf_iter_css_post_destroy(struct bpf_iter_css_post *it) __weak __ksym; + #ifndef bpf_for_each /* bpf_for_each(iter_type, cur_elem, args...) provides generic construct for * using BPF open-coded iterators without having to write mundane explicit From patchwork Tue Sep 12 07:01:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuyi Zhou X-Patchwork-Id: 13380889 X-Patchwork-Delegate: bpf@iogearbox.net Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3F4568F44 for ; Tue, 12 Sep 2023 07:02:17 +0000 (UTC) Received: from mail-pl1-x631.google.com (mail-pl1-x631.google.com [IPv6:2607:f8b0:4864:20::631]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DF7A4E79 for ; Tue, 12 Sep 2023 00:02:16 -0700 (PDT) Received: by mail-pl1-x631.google.com with SMTP id d9443c01a7336-1bf55a81eeaso35459565ad.0 for ; Tue, 12 Sep 2023 00:02:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1694502136; x=1695106936; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=zZ7auXMwt2HG2eO79EZb79CGVaa+fy/IcLCkXgQTU2E=; b=gJntY8cV6wapICpdZfEIKXOai/sUbNM/O2PEKnDVSxn2gbWVIXA5t6cb7ba/zTwtwZ WPKEc9Sb6BUUld9t3MOlZYSn36CNY7tArLtbbMwnTUTx3Mf5iaTfiZltVlVMV5xD4xQY X0FW9FZVCyndsjXUA8mh1czh/TU1HNC0/w83YQ7H09Et5h9x0ZmpeyHEFeji9O+0Vmxl xlJ0cfto8eS7a4We7zUxCHvM91RjLFelbmxD1yMP5jZN/ZD7K0XFwIEDw+SW7VfCMNOT Jep7/hQOIfpGe4m+m5bjhWqY0GvnE9Fwk1FcBmB1GqSj2HWUlw3/QSPGHW5JjFwi6oqI xDyg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694502136; x=1695106936; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=zZ7auXMwt2HG2eO79EZb79CGVaa+fy/IcLCkXgQTU2E=; b=Rs2XSHnY+IMCElIusJ3xuze0yfEoyczfo3SafY69hwKw/dpJK5nMQHutkQ60kfGBqy TVVM51IfhJSKZsTNkFMi5sxLfGvMMq8Hi5Nz+PDrMvDut3WDEL2fIgIQIBh1KbmSDu8q U1uAvstEaTMUR/27c5mtjsdvNvv0DEJOGtFMLG7cXXtDHGXNYtSMSeuJXLozz2+ySjJF gTuDFUEwop5XIySoXojf9HGPao63pZ6Crip/RHu2ShxgWnzUnu1NpqUNNd/DaXcc1hYZ Swm+6x6azTNta3N3O5gK/JnktWBAMuWxkO2G8iRb1RLJdKZq0E9nGUYdjgpA+0xRUnKh o14Q== X-Gm-Message-State: AOJu0Yws4job5gVDueRzfRpIndLgE4YeE1X6xbPM9EZ/jqyQbkhlWgCW 3Q68y7KLj74EJpJG+7XOoSAVViQGdMNmct37Mqa6YQ== X-Google-Smtp-Source: AGHT+IF1L3zszd0Qt7OImIjEsl7I2HCtreNc7f6XF9vPUvB7JGyRpTNFMRSxt9GUYpLv2X335et5TA== X-Received: by 2002:a17:902:e883:b0:1bc:506a:58f2 with SMTP id w3-20020a170902e88300b001bc506a58f2mr10371670plg.46.1694502136153; Tue, 12 Sep 2023 00:02:16 -0700 (PDT) Received: from n37-019-243.byted.org ([180.184.84.173]) by smtp.gmail.com with ESMTPSA id b8-20020a170902d50800b001b8953365aesm7635401plg.22.2023.09.12.00.02.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Sep 2023 00:02:15 -0700 (PDT) From: Chuyi Zhou To: bpf@vger.kernel.org Cc: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, martin.lau@kernel.org, tj@kernel.org, linux-kernel@vger.kernel.org, Chuyi Zhou Subject: [PATCH bpf-next v2 5/6] bpf: teach the verifier to enforce css_iter and process_iter in RCU CS Date: Tue, 12 Sep 2023 15:01:48 +0800 Message-Id: <20230912070149.969939-6-zhouchuyi@bytedance.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20230912070149.969939-1-zhouchuyi@bytedance.com> References: <20230912070149.969939-1-zhouchuyi@bytedance.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net css_iter and process_iter should be used in rcu section. Specifically, in sleepable progs explicit bpf_rcu_read_lock() is needed before use these iters. In normal bpf progs that have implicit rcu_read_lock(), it's OK to use them directly. This patch checks whether we are in rcu cs before we want to invoke bpf_iter_process_new and bpf_iter_css_{pre, post}_new in mark_stack_slots_iter(). If the rcu protection is guaranteed, we would let st->type = PTR_TO_STACK | MEM_RCU. is_iter_reg_valid_init() will reject if reg->type is UNTRUSTED. Signed-off-by: Chuyi Zhou --- kernel/bpf/verifier.c | 30 ++++++++++++++++++++++++++++-- 1 file changed, 28 insertions(+), 2 deletions(-) diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 2367483bf4c2..6a6827ba7a18 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -1172,7 +1172,13 @@ static bool is_dynptr_type_expected(struct bpf_verifier_env *env, struct bpf_reg static void __mark_reg_known_zero(struct bpf_reg_state *reg); +static bool in_rcu_cs(struct bpf_verifier_env *env); + +/* check whether we are using bpf_iter_process_*() or bpf_iter_css_*() */ +static bool is_iter_need_rcu(struct bpf_kfunc_call_arg_meta *meta); + static int mark_stack_slots_iter(struct bpf_verifier_env *env, + struct bpf_kfunc_call_arg_meta *meta, struct bpf_reg_state *reg, int insn_idx, struct btf *btf, u32 btf_id, int nr_slots) { @@ -1193,6 +1199,12 @@ static int mark_stack_slots_iter(struct bpf_verifier_env *env, __mark_reg_known_zero(st); st->type = PTR_TO_STACK; /* we don't have dedicated reg type */ + if (is_iter_need_rcu(meta)) { + if (in_rcu_cs(env)) + st->type |= MEM_RCU; + else + st->type |= PTR_UNTRUSTED; + } st->live |= REG_LIVE_WRITTEN; st->ref_obj_id = i == 0 ? id : 0; st->iter.btf = btf; @@ -1281,6 +1293,8 @@ static bool is_iter_reg_valid_init(struct bpf_verifier_env *env, struct bpf_reg_ struct bpf_stack_state *slot = &state->stack[spi - i]; struct bpf_reg_state *st = &slot->spilled_ptr; + if (st->type & PTR_UNTRUSTED) + return false; /* only main (first) slot has ref_obj_id set */ if (i == 0 && !st->ref_obj_id) return false; @@ -7503,13 +7517,13 @@ static int process_iter_arg(struct bpf_verifier_env *env, int regno, int insn_id return err; } - err = mark_stack_slots_iter(env, reg, insn_idx, meta->btf, btf_id, nr_slots); + err = mark_stack_slots_iter(env, meta, reg, insn_idx, meta->btf, btf_id, nr_slots); if (err) return err; } else { /* iter_next() or iter_destroy() expect initialized iter state*/ if (!is_iter_reg_valid_init(env, reg, meta->btf, btf_id, nr_slots)) { - verbose(env, "expected an initialized iter_%s as arg #%d\n", + verbose(env, "expected an initialized iter_%s as arg #%d or without bpf_rcu_read_lock()\n", iter_type_str(meta->btf, btf_id), regno); return -EINVAL; } @@ -10382,6 +10396,18 @@ BTF_ID(func, bpf_percpu_obj_new_impl) BTF_ID(func, bpf_percpu_obj_drop_impl) BTF_ID(func, bpf_iter_css_task_new) +BTF_SET_START(rcu_protect_kfuns_set) +BTF_ID(func, bpf_iter_process_new) +BTF_ID(func, bpf_iter_css_pre_new) +BTF_ID(func, bpf_iter_css_post_new) +BTF_SET_END(rcu_protect_kfuns_set) + +static inline bool is_iter_need_rcu(struct bpf_kfunc_call_arg_meta *meta) +{ + return btf_id_set_contains(&rcu_protect_kfuns_set, meta->func_id); +} + + static bool is_kfunc_ret_null(struct bpf_kfunc_call_arg_meta *meta) { if (meta->func_id == special_kfunc_list[KF_bpf_refcount_acquire_impl] && From patchwork Tue Sep 12 07:01:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuyi Zhou X-Patchwork-Id: 13380890 X-Patchwork-Delegate: bpf@iogearbox.net Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 54C7E8F44 for ; Tue, 12 Sep 2023 07:02:20 +0000 (UTC) Received: from mail-pl1-x62f.google.com (mail-pl1-x62f.google.com [IPv6:2607:f8b0:4864:20::62f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F1A9010D0 for ; Tue, 12 Sep 2023 00:02:19 -0700 (PDT) Received: by mail-pl1-x62f.google.com with SMTP id d9443c01a7336-1c1f8aaab9aso45901155ad.1 for ; Tue, 12 Sep 2023 00:02:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1694502139; x=1695106939; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=WHvChKGnnZi2VbMoC/1ghgca6iZmmMi6EKlv8J2G2fg=; b=Pz94yrZEm1gAewdNBZNyHYGilUQYcv52N/6CM7ofREcH9Azf91rO5OrplltOLjm/vK 2IjyqTcsuiHZR2OZlAWJlsf3XCAkc07jw5hrHF3czY017l5o1m+shrQ07R8ALKrHbDsM sqOozaHKk1//v8czq1/Et6jCeIAuhLqU8nfj+qGH4NUWsHn0unUwp4tQAmEIH0PA7Oyg F4tDd+v3U65B5PQXAz+S65m2l8CQVvAKbW20jgwvksYdCtctjwy30kapQemgdGTTH4Bl e/siybWVV/ldUfRAqjdF0+W9DLCPMZtZ++C9BLwYUypljfkYj9xhPuaPoz0TD54DAI2V 4p5Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694502139; x=1695106939; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=WHvChKGnnZi2VbMoC/1ghgca6iZmmMi6EKlv8J2G2fg=; b=iY0jetgbmutkKGyBBC6+Qd8V+U0SzDwnqanbtpbXilYfMi01KRJOFggaz0JEWmrrz2 MGSQUuw54Nx9/+5Fnru6brdJUprH7TOfGuIPyp5YkCIA2bpdCN5kYtZLfRdkApM3Cd0p qVeunT6MisG2HQhzt0CGg5/gyuW78MnHagz+O5Cjb6v5hVhO7KfkUItFsffuEXxCC+E6 JSa48HVY6bTgP5G/eOTZCoecwUgqlV0RdRcPcHjYBQgQg+SjqDObHv4V/Dh3SaX9AXen 64qYXXgjUnz1URhDoGxoo+E7FxLvlagnQpDbtIFFB3rnIkDtguUOlVAWylDsMx/Egr8m utKQ== X-Gm-Message-State: AOJu0Yx2bYD3HC0ApzLzxY/0rV8bCg1ghXD24E6VhSpXYMirAanLETmg LpS9n2WMv3uORDSrSXmQhoM7oHg6N0nA+AoPZVnAtg== X-Google-Smtp-Source: AGHT+IGAzzXLvSkqyFtsQ81mmEO/NsorZtgGazng17R/SZuKMut3kM4aZbbVuYOInENRSv12IHsRgQ== X-Received: by 2002:a17:902:b287:b0:1c3:7628:fcbb with SMTP id u7-20020a170902b28700b001c37628fcbbmr10807814plr.43.1694502139227; Tue, 12 Sep 2023 00:02:19 -0700 (PDT) Received: from n37-019-243.byted.org ([180.184.84.173]) by smtp.gmail.com with ESMTPSA id b8-20020a170902d50800b001b8953365aesm7635401plg.22.2023.09.12.00.02.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Sep 2023 00:02:19 -0700 (PDT) From: Chuyi Zhou To: bpf@vger.kernel.org Cc: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, martin.lau@kernel.org, tj@kernel.org, linux-kernel@vger.kernel.org, Chuyi Zhou Subject: [PATCH bpf-next v2 6/6] selftests/bpf: Add tests for open-coded task and css iter Date: Tue, 12 Sep 2023 15:01:49 +0800 Message-Id: <20230912070149.969939-7-zhouchuyi@bytedance.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20230912070149.969939-1-zhouchuyi@bytedance.com> References: <20230912070149.969939-1-zhouchuyi@bytedance.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net This patch adds three tests to demonstrate these patterns and validating correctness. test1: We use bpf_for_each(process, task) to iterate all process in the system and search for the current process with a given pid. test2: We create a cgroup and add the current task to the cgroup. In the BPF program, we would use bpf_for_each(css_task, task, css) to iterate all tasks under the cgroup. As expected, we would find the current process. test3: We create a cgroup tree. In the BPF program, we use bpf_for_each(css_{pre,post}, pos, root) to iterate all descendant under the root with pre and post order. As expected, we would find all descendant and the last iterating cgroup in post-order is root cgroup, the first iterating cgroup in pre-order is root cgroup. Signed-off-by: Chuyi Zhou --- .../testing/selftests/bpf/prog_tests/iters.c | 138 ++++++++++++++++++ .../testing/selftests/bpf/progs/iters_task.c | 104 +++++++++++++ 2 files changed, 242 insertions(+) create mode 100644 tools/testing/selftests/bpf/progs/iters_task.c diff --git a/tools/testing/selftests/bpf/prog_tests/iters.c b/tools/testing/selftests/bpf/prog_tests/iters.c index 10804ae5ae97..f4e69a506509 100644 --- a/tools/testing/selftests/bpf/prog_tests/iters.c +++ b/tools/testing/selftests/bpf/prog_tests/iters.c @@ -1,13 +1,21 @@ // SPDX-License-Identifier: GPL-2.0 /* Copyright (c) 2023 Meta Platforms, Inc. and affiliates. */ +#include +#include +#include +#include +#include +#include #include +#include "cgroup_helpers.h" #include "iters.skel.h" #include "iters_state_safety.skel.h" #include "iters_looping.skel.h" #include "iters_num.skel.h" #include "iters_testmod_seq.skel.h" +#include "iters_task.skel.h" static void subtest_num_iters(void) { @@ -90,6 +98,130 @@ static void subtest_testmod_seq_iters(void) iters_testmod_seq__destroy(skel); } +static void subtest_process_iters(void) +{ + struct iters_task *skel; + int err; + + skel = iters_task__open(); + if (!ASSERT_OK_PTR(skel, "skel_open")) + goto cleanup; + bpf_program__set_autoload(skel->progs.iter_task_for_each_sleep, true); + err = iters_task__load(skel); + if (!ASSERT_OK(err, "skel_load")) + goto cleanup; + skel->bss->target_pid = getpid(); + err = iters_task__attach(skel); + if (!ASSERT_OK(err, "iters_task__attach")) + goto cleanup; + syscall(SYS_getpgid); + iters_task__detach(skel); + ASSERT_EQ(skel->bss->process_cnt, 1, "process_cnt"); + +cleanup: + iters_task__destroy(skel); +} + +extern int stack_mprotect(void); + +static void subtest_css_task_iters(void) +{ + struct iters_task *skel; + int err, cg_fd, cg_id; + const char *cgrp_path = "/cg1"; + + err = setup_cgroup_environment(); + if (!ASSERT_OK(err, "setup_cgroup_environment")) + goto cleanup; + cg_fd = create_and_get_cgroup(cgrp_path); + if (!ASSERT_GE(cg_fd, 0, "cg_create")) + goto cleanup; + cg_id = get_cgroup_id(cgrp_path); + err = join_cgroup(cgrp_path); + if (!ASSERT_OK(err, "setup_cgroup_environment")) + goto cleanup; + + skel = iters_task__open(); + if (!ASSERT_OK_PTR(skel, "skel_open")) + goto cleanup; + + bpf_program__set_autoload(skel->progs.iter_css_task_for_each, true); + err = iters_task__load(skel); + if (!ASSERT_OK(err, "skel_load")) + goto cleanup; + + skel->bss->target_pid = getpid(); + skel->bss->cg_id = cg_id; + err = iters_task__attach(skel); + + err = stack_mprotect(); + if (!ASSERT_OK(err, "iters_task__attach")) + goto cleanup; + + iters_task__detach(skel); + ASSERT_EQ(skel->bss->css_task_cnt, 1, "css_task_cnt"); + +cleanup: + cleanup_cgroup_environment(); + iters_task__destroy(skel); +} + +static void subtest_css_dec_iters(void) +{ + struct iters_task *skel; + struct { + const char *path; + int fd; + } cgs[] = { + { "/cg1" }, + { "/cg1/cg2" }, + { "/cg1/cg2/cg3" }, + { "/cg1/cg2/cg3/cg4" }, + }; + int err, cg_nr = ARRAY_SIZE(cgs); + int i; + + err = setup_cgroup_environment(); + if (!ASSERT_OK(err, "setup_cgroup_environment")) + goto cleanup; + for (i = 0; i < cg_nr; i++) { + cgs[i].fd = create_and_get_cgroup(cgs[i].path); + if (!ASSERT_GE(cgs[i].fd, 0, "cg_create")) + goto cleanup; + } + + skel = iters_task__open(); + if (!ASSERT_OK_PTR(skel, "skel_open")) + goto cleanup; + bpf_program__set_autoload(skel->progs.iter_css_dec_for_each, true); + err = iters_task__load(skel); + if (!ASSERT_OK(err, "skel_load")) + goto cleanup; + + skel->bss->target_pid = getpid(); + skel->bss->cg_id = get_cgroup_id(cgs[0].path); + + err = iters_task__attach(skel); + + if (!ASSERT_OK(err, "iters_task__attach")) + goto cleanup; + + syscall(SYS_getpgid); + ASSERT_EQ(skel->bss->css_dec_cnt, cg_nr, "pre order search dec count"); + ASSERT_EQ(skel->bss->first_cg_id, get_cgroup_id(cgs[0].path), + "pre order search first cgroup id"); + skel->bss->css_dec_cnt = 0; + skel->bss->is_post_order = 1; + syscall(SYS_getpgid); + ASSERT_EQ(skel->bss->css_dec_cnt, cg_nr, "post order search dec count"); + ASSERT_EQ(skel->bss->last_cg_id, get_cgroup_id(cgs[0].path), + "post order search last cgroup id"); + iters_task__detach(skel); +cleanup: + cleanup_cgroup_environment(); + iters_task__destroy(skel); +} + void test_iters(void) { RUN_TESTS(iters_state_safety); @@ -103,4 +235,10 @@ void test_iters(void) subtest_num_iters(); if (test__start_subtest("testmod_seq")) subtest_testmod_seq_iters(); + if (test__start_subtest("process")) + subtest_process_iters(); + if (test__start_subtest("css_task")) + subtest_css_task_iters(); + if (test__start_subtest("css_dec")) + subtest_css_dec_iters(); } diff --git a/tools/testing/selftests/bpf/progs/iters_task.c b/tools/testing/selftests/bpf/progs/iters_task.c new file mode 100644 index 000000000000..cf24a3b177f2 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/iters_task.c @@ -0,0 +1,104 @@ +// SPDX-License-Identifier: GPL-2.0 +#include "vmlinux.h" +#include +#include +#include "bpf_misc.h" +char _license[] SEC("license") = "GPL"; + +pid_t target_pid = 0; +int process_cnt = 0; +int css_task_cnt = 0; +int css_dec_cnt = 0; + +char is_post_order; +u64 cg_id; +u64 last_cg_id; +u64 first_cg_id; + +struct cgroup *bpf_cgroup_from_id(u64 cgid) __ksym; +struct cgroup *bpf_cgroup_acquire(struct cgroup *cgrp) __ksym; +void bpf_cgroup_release(struct cgroup *p) __ksym; +void bpf_rcu_read_lock(void) __ksym; +void bpf_rcu_read_unlock(void) __ksym; + +SEC("?fentry.s/" SYS_PREFIX "sys_getpgid") +int iter_task_for_each_sleep(void *ctx) +{ + struct task_struct *task; + struct task_struct *cur_task = bpf_get_current_task_btf(); + + if (cur_task->pid != target_pid) + return 0; + bpf_rcu_read_lock(); + bpf_for_each(process, task) { + if (task->pid == target_pid) + process_cnt += 1; + } + bpf_rcu_read_unlock(); + return 0; +} + + +SEC("?lsm/file_mprotect") +int BPF_PROG(iter_css_task_for_each) +{ + struct task_struct *task; + struct task_struct *cur_task = bpf_get_current_task_btf(); + + if (cur_task->pid != target_pid) + return 0; + + struct cgroup *cgrp = bpf_cgroup_from_id(cg_id); + + if (cgrp == NULL) + return 0; + struct cgroup_subsys_state *css = &cgrp->self; + + bpf_for_each(css_task, task, css, CSS_TASK_ITER_PROCS) { + if (!task) + continue; + if (task->pid == target_pid) + css_task_cnt += 1; + } + bpf_cgroup_release(cgrp); + return 0; +} + +SEC("?fentry.s/" SYS_PREFIX "sys_getpgid") +int iter_css_dec_for_each(const void *ctx) +{ + struct task_struct *cur_task = bpf_get_current_task_btf(); + + if (cur_task->pid != target_pid) + return 0; + + struct cgroup *cgrp = bpf_cgroup_from_id(cg_id); + + if (cgrp == NULL) + return 0; + struct cgroup_subsys_state *root = &cgrp->self; + struct cgroup_subsys_state *pos = NULL; + + bpf_rcu_read_lock(); + if (is_post_order) { + bpf_for_each(css_post, pos, root) { + struct cgroup *cur_cgrp = pos->cgroup; + + css_dec_cnt += 1; + if (cur_cgrp) + last_cg_id = cur_cgrp->kn->id; + } + } else { + bpf_for_each(css_pre, pos, root) { + struct cgroup *cur_cgrp = pos->cgroup; + + css_dec_cnt += 1; + if (cur_cgrp && !first_cg_id) + first_cg_id = cur_cgrp->kn->id; + } + } + bpf_rcu_read_unlock(); + bpf_cgroup_release(cgrp); + return 0; +} +