From patchwork Thu Oct 10 23:25:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Namhyung Kim X-Patchwork-Id: 13831301 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 16FC3D2445D for ; Thu, 10 Oct 2024 23:25:12 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8B3186B0088; Thu, 10 Oct 2024 19:25:11 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 865BB6B008A; Thu, 10 Oct 2024 19:25:11 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6E6FA6B0088; Thu, 10 Oct 2024 19:25:11 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 5116D6B0088 for ; Thu, 10 Oct 2024 19:25:11 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 3AA4540CA8 for ; Thu, 10 Oct 2024 23:25:08 +0000 (UTC) X-FDA: 82659275580.06.643567A Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf09.hostedemail.com (Postfix) with ESMTP id 65DAB14001A for ; Thu, 10 Oct 2024 23:25:07 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=diCbxJ1V; spf=pass (imf09.hostedemail.com: domain of namhyung@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=namhyung@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1728602571; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=KqXB5q/NyOlpNhsJmf6mwELvm2zk8XMc6NrpFn4Ohwk=; b=jn3NV/tfqzVQrW7CtmUISDF8RQKUMN5IS16QUW4DULxdrGHg9a1AfsvGhttzfkmlqRGFxH RA7L0LXPXM5OOPHmrTY9Xw8xNTVpEQYqh8E1pJQpY3wwcbybes9wBeqa1bZ+FalalQlDoX UQe9OM7P8IuxQdCJO2OOme0s9kjh0JU= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1728602571; a=rsa-sha256; cv=none; b=kXINK2wz7DvXY5iKQN3P4BjcmIVv10Qf0GoUcjRJ1kJqbJOtYcSBzOxZYJ55b6ZslZWkTw dCIw2H7g8ZHbnabh9EYEdMmCJ9iNpqK8AaP+UwqiZlMwsMCU8G+Wu7miAzaTK+1XmtjFR9 ieVzfgtfITcOr5jTrBApUOuSoVECYHY= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=diCbxJ1V; spf=pass (imf09.hostedemail.com: domain of namhyung@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=namhyung@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id D97355C6009; Thu, 10 Oct 2024 23:25:03 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2A5BDC4CED1; Thu, 10 Oct 2024 23:25:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1728602708; bh=qwr7dD1YMBw5Ph7vaygJ/ovjODa5iDddCD/l90SK30g=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=diCbxJ1VXjr7niALk+JJ9UkjhhtqbN2rskMmlYxFM/dazCKZtrvQLMHnp2MtoCxWM xfPLdR8d3QYjj2YVpEf1v3jrOY9sv6YV/Bf1LAf58Qap8JNuQEokJX0BuJMf1D0H5H xptQ7M2FOVg/6mPUkWJzrg32HcdwAdSlFYq62rIPL2621ftJ/0/EgDYgiRq6sNTXZ5 HLQrj654wA68XLE3Lv4Z8zXeOFrF22J63Wn//nIaI6pFnXQB7KpA/O8VKaNWWWQfHF bE/hkHGZf+waYurr4ytGDdcGzZZ2uNzaPCWhyyCN9IdcRfTTH3i3x1X3Fw96akpxmF yVkwUmSAotEiQ== From: Namhyung Kim To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: Martin KaFai Lau , Eduard Zingerman , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Jiri Olsa , LKML , bpf@vger.kernel.org, Andrew Morton , Christoph Lameter , Pekka Enberg , David Rientjes , Joonsoo Kim , Vlastimil Babka , Roman Gushchin , Hyeonggon Yoo <42.hyeyoo@gmail.com>, linux-mm@kvack.org, Arnaldo Carvalho de Melo , Kees Cook , "Paul E. McKenney" Subject: [PATCH v5 bpf-next 1/3] bpf: Add kmem_cache iterator Date: Thu, 10 Oct 2024 16:25:03 -0700 Message-ID: <20241010232505.1339892-2-namhyung@kernel.org> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog In-Reply-To: <20241010232505.1339892-1-namhyung@kernel.org> References: <20241010232505.1339892-1-namhyung@kernel.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 65DAB14001A X-Stat-Signature: pw664pf4sd14eboi139fwojjh9ccck6m X-HE-Tag: 1728602707-797196 X-HE-Meta: U2FsdGVkX19NcGGa+JhKuti/kaQj454zMqgvXerov+4vZLJzDqplXDecG772cUt3MUlDo5db1HEe12DDUS4IrukALAC8J0BDu8D2SmGbbsKr8XVSeVMVf/Nh6uUNj3LhOZZlZ5hfn5WYhZwi1p/+gZ5QgcCKTUpDAAEeqX+i78aPKXbG5OO2F5y44vXeKGadZJUTWiUe+gdeK0O1CmiL6utjArt7vGnDVcwjESh+XsY+Evx3e/yEcKd/+IE2jZO2MHtZXJOo7mbsX855/aMnxwLf25jNOdWXUBUdAqC2uXo2QwW/XeYpe5E4VhhjTFnlUQX+7k58M3vGgmDCMYUbqKZmAt7D6qKa7ED1ZA8seFHK5XoFk7rQlmv0cPalUahGNiuKwRI1BqHFDTocLaKXMjjmILSg7Ed3qQTRsKwV7GpE3xnwzUfqZJ1S8doipKar5AaFzPcrmU1CqeqSEZC3IsJHrEhvfq3sA+bdYuu9mW+oB2h9z3jgri8REc1/t82yEW6dVLX2uw3iH7VptCVlIQ8KxrwQUcFf9ZSI12xQebhuRhzkp7096IqlHDEcp3h+s8UTuXIY867PyJHHFJqhofudTwryV9xAGbgHDG8dhpie4Cj3D3cg4ozI1QEWQAJm0s5qrfMNvIKZcYZr5yOG3QA1Htpygl2HhstJnyH9ddPY6bJg3mvEmpY7mJOr7O6KovvZdTI34NHg54UElZGuO417mUO9zMrTJVjir9RVGhxfGExiCLIgMub+ozuIOZrvjJRJx+hUpsbhfNS3exVMA/g8md1oZErHnyKGdpNaeNUmEOqMp5a92nsVZN9mrWKIJJskmjSKdJiLDEHjfSorPG0xLm5g++gsfSvwS/R78IQHcMHoH0v2inpq+MIBwwZh0LB5OjjXicNQ1sSvUGbdlDc/0HE+mXc1BvF1fs6cxRNAza6nHkaB2ljX+w6AbKa5sRwWhpHhVQJ5uaaFckE eCYFW4t6 6wB+B3QNrmnKojv02cGcAF0J3aquOoizUEnC+8Rwmcw2gJM8R+EcOrI2D7TtWVasykW/QbWCV+9u980/aqL1L4p8LQIfnlnB/ZxnTOUO/PmPure7t4tbcwS81xrcI4E9X/CaGWyorDacwESVrKkFLXSV0c+oBIhasKLdtdFVNLE/2HpfrTsjkELGDHoTzQSq/qWtp3IJJoavFx6+Ngnt1tJXNyZjheM3w8BRO1mL9YvPYVh2x5ygacslhLi7oy6eshdSLqCICNQ5GrEQq0PKOB8TwLA4/GXhdJ6Sz0SbGGbb4EoMYGuLY5pay3FTjWa3qd9efnlWOLqK0kRmh145E0pBCK66ZL+SjdgRAL7wuRJhj1bU= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The new "kmem_cache" iterator will traverse the list of slab caches and call attached BPF programs for each entry. It should check the argument (ctx.s) if it's NULL before using it. Now the iteration grabs the slab_mutex only if it traverse the list and releases the mutex when it runs the BPF program. The kmem_cache entry is protected by a refcount during the execution. It includes the internal "mm/slab.h" header to access kmem_cache, slab_caches and slab_mutex. Hope it's ok to mm folks. Signed-off-by: Namhyung Kim Acked-by: Vlastimil Babka #slab --- include/linux/btf_ids.h | 1 + kernel/bpf/Makefile | 1 + kernel/bpf/kmem_cache_iter.c | 175 +++++++++++++++++++++++++++++++++++ 3 files changed, 177 insertions(+) create mode 100644 kernel/bpf/kmem_cache_iter.c diff --git a/include/linux/btf_ids.h b/include/linux/btf_ids.h index c0e3e1426a82f5c4..139bdececdcfaefb 100644 --- a/include/linux/btf_ids.h +++ b/include/linux/btf_ids.h @@ -283,5 +283,6 @@ extern u32 btf_tracing_ids[]; extern u32 bpf_cgroup_btf_id[]; extern u32 bpf_local_storage_map_btf_id[]; extern u32 btf_bpf_map_id[]; +extern u32 bpf_kmem_cache_btf_id[]; #endif diff --git a/kernel/bpf/Makefile b/kernel/bpf/Makefile index 9b9c151b5c826b31..105328f0b9c04e37 100644 --- a/kernel/bpf/Makefile +++ b/kernel/bpf/Makefile @@ -52,3 +52,4 @@ obj-$(CONFIG_BPF_PRELOAD) += preload/ obj-$(CONFIG_BPF_SYSCALL) += relo_core.o obj-$(CONFIG_BPF_SYSCALL) += btf_iter.o obj-$(CONFIG_BPF_SYSCALL) += btf_relocate.o +obj-$(CONFIG_BPF_SYSCALL) += kmem_cache_iter.o diff --git a/kernel/bpf/kmem_cache_iter.c b/kernel/bpf/kmem_cache_iter.c new file mode 100644 index 0000000000000000..2de0682c6d4c773f --- /dev/null +++ b/kernel/bpf/kmem_cache_iter.c @@ -0,0 +1,175 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* Copyright (c) 2024 Google */ +#include +#include +#include +#include +#include + +#include "../../mm/slab.h" /* kmem_cache, slab_caches and slab_mutex */ + +struct bpf_iter__kmem_cache { + __bpf_md_ptr(struct bpf_iter_meta *, meta); + __bpf_md_ptr(struct kmem_cache *, s); +}; + +static void *kmem_cache_iter_seq_start(struct seq_file *seq, loff_t *pos) +{ + loff_t cnt = 0; + bool found = false; + struct kmem_cache *s; + + mutex_lock(&slab_mutex); + + /* Find an entry at the given position in the slab_caches list instead + * of keeping a reference (of the last visited entry, if any) out of + * slab_mutex. It might miss something if one is deleted in the middle + * while it releases the lock. But it should be rare and there's not + * much we can do about it. + */ + list_for_each_entry(s, &slab_caches, list) { + if (cnt == *pos) { + /* Make sure this entry remains in the list by getting + * a new reference count. Note that boot_cache entries + * have a negative refcount, so don't touch them. + */ + if (s->refcount > 0) + s->refcount++; + found = true; + break; + } + cnt++; + } + mutex_unlock(&slab_mutex); + + if (!found) + return NULL; + + return s; +} + +static void kmem_cache_iter_seq_stop(struct seq_file *seq, void *v) +{ + struct bpf_iter_meta meta; + struct bpf_iter__kmem_cache ctx = { + .meta = &meta, + .s = v, + }; + struct bpf_prog *prog; + bool destroy = false; + + meta.seq = seq; + prog = bpf_iter_get_info(&meta, true); + if (prog && !ctx.s) + bpf_iter_run_prog(prog, &ctx); + + if (ctx.s == NULL) + return; + + mutex_lock(&slab_mutex); + + /* Skip kmem_cache_destroy() for active entries */ + if (ctx.s->refcount > 1) + ctx.s->refcount--; + else if (ctx.s->refcount == 1) + destroy = true; + + mutex_unlock(&slab_mutex); + + if (destroy) + kmem_cache_destroy(ctx.s); +} + +static void *kmem_cache_iter_seq_next(struct seq_file *seq, void *v, loff_t *pos) +{ + struct kmem_cache *s = v; + struct kmem_cache *next = NULL; + bool destroy = false; + + ++*pos; + + mutex_lock(&slab_mutex); + + if (list_last_entry(&slab_caches, struct kmem_cache, list) != s) { + next = list_next_entry(s, list); + + WARN_ON_ONCE(next->refcount == 0); + + /* boot_caches have negative refcount, don't touch them */ + if (next->refcount > 0) + next->refcount++; + } + + /* Skip kmem_cache_destroy() for active entries */ + if (s->refcount > 1) + s->refcount--; + else if (s->refcount == 1) + destroy = true; + + mutex_unlock(&slab_mutex); + + if (destroy) + kmem_cache_destroy(s); + + return next; +} + +static int kmem_cache_iter_seq_show(struct seq_file *seq, void *v) +{ + struct bpf_iter_meta meta; + struct bpf_iter__kmem_cache ctx = { + .meta = &meta, + .s = v, + }; + struct bpf_prog *prog; + int ret = 0; + + meta.seq = seq; + prog = bpf_iter_get_info(&meta, false); + if (prog) + ret = bpf_iter_run_prog(prog, &ctx); + + return ret; +} + +static const struct seq_operations kmem_cache_iter_seq_ops = { + .start = kmem_cache_iter_seq_start, + .next = kmem_cache_iter_seq_next, + .stop = kmem_cache_iter_seq_stop, + .show = kmem_cache_iter_seq_show, +}; + +BTF_ID_LIST_GLOBAL_SINGLE(bpf_kmem_cache_btf_id, struct, kmem_cache) + +static const struct bpf_iter_seq_info kmem_cache_iter_seq_info = { + .seq_ops = &kmem_cache_iter_seq_ops, +}; + +static void bpf_iter_kmem_cache_show_fdinfo(const struct bpf_iter_aux_info *aux, + struct seq_file *seq) +{ + seq_puts(seq, "kmem_cache iter\n"); +} + +DEFINE_BPF_ITER_FUNC(kmem_cache, struct bpf_iter_meta *meta, + struct kmem_cache *s) + +static struct bpf_iter_reg bpf_kmem_cache_reg_info = { + .target = "kmem_cache", + .feature = BPF_ITER_RESCHED, + .show_fdinfo = bpf_iter_kmem_cache_show_fdinfo, + .ctx_arg_info_size = 1, + .ctx_arg_info = { + { offsetof(struct bpf_iter__kmem_cache, s), + PTR_TO_BTF_ID_OR_NULL | PTR_UNTRUSTED }, + }, + .seq_info = &kmem_cache_iter_seq_info, +}; + +static int __init bpf_kmem_cache_iter_init(void) +{ + bpf_kmem_cache_reg_info.ctx_arg_info[0].btf_id = bpf_kmem_cache_btf_id[0]; + return bpf_iter_reg_target(&bpf_kmem_cache_reg_info); +} + +late_initcall(bpf_kmem_cache_iter_init); From patchwork Thu Oct 10 23:25:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Namhyung Kim X-Patchwork-Id: 13831302 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 60985D2444F for ; Thu, 10 Oct 2024 23:25:14 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3AA836B0089; Thu, 10 Oct 2024 19:25:12 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 30CE66B008A; Thu, 10 Oct 2024 19:25:12 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F3F626B008C; Thu, 10 Oct 2024 19:25:11 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id D99186B0089 for ; Thu, 10 Oct 2024 19:25:11 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 733591C6964 for ; Thu, 10 Oct 2024 23:25:07 +0000 (UTC) X-FDA: 82659275580.12.B3608A8 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf10.hostedemail.com (Postfix) with ESMTP id 11DB5C0006 for ; Thu, 10 Oct 2024 23:25:08 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=LpjMyrxb; spf=pass (imf10.hostedemail.com: domain of namhyung@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=namhyung@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1728602572; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=PJEKwEWnujsI7OvXqbMa/wPYsumJ28JqxOqu+XT5ozs=; b=V1jOCX4DB9X2L62m6i1h7kYIkZtcIQtLSJSIAvta8fAN8WjgbRC9QF9BX4kV4qujMximvM qEre1see9IuQU/Tvu2UrN0HnZ5PR5MZcW+dusCqLiOIdpWvbUyf9s3KwuTItD/OSB052qK XjFkq8Jo3XwRcKVg5OGEfQlJhLFmE9w= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1728602572; a=rsa-sha256; cv=none; b=vMIEAwDibt4OFs8dV1t1nKKD1w0er4Lel6BPVDBbgmW9Sp4FL6VheQelsPP51DcBLabjNQ QvdDXiyEahzw+GuqCupcn7XhgteXfpJ5tjiH1nk+MmtBizFwXndaFUmyE3SONLu6ypHpk+ ptGZbQKIsz+NnTxV6nFiQ2CPNrOphPo= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=LpjMyrxb; spf=pass (imf10.hostedemail.com: domain of namhyung@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=namhyung@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id E58655C6013; Thu, 10 Oct 2024 23:25:04 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2D8B8C4CECC; Thu, 10 Oct 2024 23:25:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1728602709; bh=oQNc8mViY1+Zpg9pTngTgTZBRAhT3aWMoYlhdEdPulk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=LpjMyrxbIkU8p8Pos+uewoY7l6slYSFawh+sAhm4DKth60xZfVibOZUz7Q6yJfaS9 3BieWcZ1AB31oU4P2cdt6jVckI/XGUIuNal+JYlW6EC5bjS5GyD2crZULXYCqB2AvS pAP9K0fAZg0LENhrccEFmhwVCDT33GDgLUwbdYYl90qz//WMQXC6O6DBYyBtfdZ4ob AfzoKOLuZH8Wi8V4Aejb9+FIDypglbDiCktzmpuiuVcWORss5rG7yW1GO2Si5l022x tNa6F5diiFsS75Ryvb3II3cIsRUVM9gOgqJNWNvqiIreIfJMsGK78mMda+iZFqGik2 LR0aLVQRbFOEA== From: Namhyung Kim To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: Martin KaFai Lau , Eduard Zingerman , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Jiri Olsa , LKML , bpf@vger.kernel.org, Andrew Morton , Christoph Lameter , Pekka Enberg , David Rientjes , Joonsoo Kim , Vlastimil Babka , Roman Gushchin , Hyeonggon Yoo <42.hyeyoo@gmail.com>, linux-mm@kvack.org, Arnaldo Carvalho de Melo , Kees Cook , "Paul E. McKenney" Subject: [PATCH v5 bpf-next 2/3] mm/bpf: Add bpf_get_kmem_cache() kfunc Date: Thu, 10 Oct 2024 16:25:04 -0700 Message-ID: <20241010232505.1339892-3-namhyung@kernel.org> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog In-Reply-To: <20241010232505.1339892-1-namhyung@kernel.org> References: <20241010232505.1339892-1-namhyung@kernel.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 11DB5C0006 X-Stat-Signature: mb85j7wq41fajfxqgi4ami9qpj7ms677 X-HE-Tag: 1728602708-981692 X-HE-Meta: U2FsdGVkX1/yqdsfBTeuGkwmDP+uK2o2F942FahSpq8Imp4rBBteXjEw1HSAIsTddq4jHrE5WdN12YadImwLWz4QY9PPm+JeeOkb1eqyc6q+N1yJUnq5NSNBfg72UE6nfW9FLj8lNEg6NTjwTU6bzzg5YsbfPHZZUGV2YbmDswmELoM5gExZorj85ITtIT9u24QFMg03uODaT+yD8qY8+P7UMzhcoBT9Mme4/6w9NV+xcpkG1sQtK19xfSF7YrOdHSe++smtnw1ntlHKIjuhXCYzG3CRwJKw+bq0Vezp3z1Rn41Hpex4ON7IliuzvDU2xhgWwZxfXX/PyoKtfkrQ+T5b2+a6ZjfzmX5jvigmnpkJbl3spSFRWCjtuFA77UBg6oJEN6ubMgVy3z3IYa7K8Ti/KMYUSNZJaBhenou4TfrhZkGXpxIEGYv7eTJf8iImAhSufaPX32yY5CUeYkeb6B8eI8zzY33py1z/1Blc5XdLxKlp1DRF4KhWVcGptU9imjRYETSvg5zahnxzYOYjPgmw55zGiNg6gZUiDOM5weFIW5gCIwHxYfHfaZD/Ov7KBz+c6MF5Ae62b0SSthBDJ9VZM8VGBL6mZxrypk2OgEn6VtyaG93PBjTJ/4HPcWuOkfTpNxJ3gtKaTLPMTyrImtS5I2TluYiYTQ9NcvO8c503MsjelhbYNLMvbMYN3nSXi5r0ORHetfQCyitzka6i0F7hbDn3Dvs8Q7UgP5wMnA7pSWeVNotpfmFl7o+iEk06Nuq2n5ofEEmlEUI+5yXAp0kPFHGo8jL9kJ857Icr+LTlOI/NtEWHZteSL7Ep1sESvlW1KJMSyjKxS+dPblThs9eMlxquwLL+SpumRfcaGLdfgC+KOjb9SrDF93RaPoIRNgYSCtk2sgPMyaYO+gBY1YscVAXEqEIuz4w5OpJrLvYOJC9dB6ayxo7RsMv7XqHtWU+UqkUKMk+CvGOxwzp B9EDn1M3 hSyaNeEXMieGOUoxaB8A8uXqHMFyhXBc+Lt85wPSyg6PQX/zQ7xKrZxMqnGqFnyPu4rX2XSwhuc3IhyxCLpqQjPK3cxk0XPBBskO6dXE73xnSiklh2JA0uE9NtrO/I+D1DzLqLoJg3Z31zXu2UtVOHkhTlxUJBpTW2bIWPWUsuwbSGiXDH1inBpBcJcyxfhg4ydiK2B9y7kntFXuLxh+WylFVIgT1Rocg/U+rfUyl5Y9eUA1Ay2K65G414zrsFGUdZTDGJEOnhs0q7uVfiNLjN4/sRi9kL09BHFam2sjuGopvDKjcU9EUVpmJA1Ti/6t64QjXtjF/Zm9fx+66eKvBXq8PGWrBOSKkTHh045PN5K5r4YZw/cb/SzDWP75kTCf5QQ1nkIYlI6ZvyfE= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The bpf_get_kmem_cache() is to get a slab cache information from a virtual address like virt_to_cache(). If the address is a pointer to a slab object, it'd return a valid kmem_cache pointer, otherwise NULL is returned. It doesn't grab a reference count of the kmem_cache so the caller is responsible to manage the access. The returned point is marked as PTR_UNTRUSTED. And the kfunc has KF_RCU_PROTECTED as the slab object might be protected by RCU. The intended use case for now is to symbolize locks in slab objects from the lock contention tracepoints. Suggested-by: Vlastimil Babka Acked-by: Roman Gushchin (mm/*) Acked-by: Vlastimil Babka #mm/slab Signed-off-by: Namhyung Kim --- kernel/bpf/helpers.c | 1 + kernel/bpf/verifier.c | 5 +++++ mm/slab_common.c | 19 +++++++++++++++++++ 3 files changed, 25 insertions(+) diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c index 4053f279ed4cc7ab..7bfef9378ab21267 100644 --- a/kernel/bpf/helpers.c +++ b/kernel/bpf/helpers.c @@ -3090,6 +3090,7 @@ BTF_ID_FLAGS(func, bpf_iter_bits_new, KF_ITER_NEW) BTF_ID_FLAGS(func, bpf_iter_bits_next, KF_ITER_NEXT | KF_RET_NULL) BTF_ID_FLAGS(func, bpf_iter_bits_destroy, KF_ITER_DESTROY) BTF_ID_FLAGS(func, bpf_copy_from_user_str, KF_SLEEPABLE) +BTF_ID_FLAGS(func, bpf_get_kmem_cache, KF_RCU_PROTECTED) BTF_KFUNCS_END(common_btf_ids) static const struct btf_kfunc_id_set common_kfunc_set = { diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index cfc62e0776bff2c8..f514247ba8ba8a57 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -11259,6 +11259,7 @@ enum special_kfunc_type { KF_bpf_preempt_enable, KF_bpf_iter_css_task_new, KF_bpf_session_cookie, + KF_bpf_get_kmem_cache, }; BTF_SET_START(special_kfunc_set) @@ -11324,6 +11325,7 @@ BTF_ID(func, bpf_session_cookie) #else BTF_ID_UNUSED #endif +BTF_ID(func, bpf_get_kmem_cache) static bool is_kfunc_ret_null(struct bpf_kfunc_call_arg_meta *meta) { @@ -12834,6 +12836,9 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn, regs[BPF_REG_0].type = PTR_TO_BTF_ID; regs[BPF_REG_0].btf_id = ptr_type_id; + if (meta.func_id == special_kfunc_list[KF_bpf_get_kmem_cache]) + regs[BPF_REG_0].type |= PTR_UNTRUSTED; + if (is_iter_next_kfunc(&meta)) { struct bpf_reg_state *cur_iter; diff --git a/mm/slab_common.c b/mm/slab_common.c index 7443244656150325..5484e1cd812f698e 100644 --- a/mm/slab_common.c +++ b/mm/slab_common.c @@ -1322,6 +1322,25 @@ size_t ksize(const void *objp) } EXPORT_SYMBOL(ksize); +#ifdef CONFIG_BPF_SYSCALL +#include + +__bpf_kfunc_start_defs(); + +__bpf_kfunc struct kmem_cache *bpf_get_kmem_cache(u64 addr) +{ + struct slab *slab; + + if (!virt_addr_valid(addr)) + return NULL; + + slab = virt_to_slab((void *)(long)addr); + return slab ? slab->slab_cache : NULL; +} + +__bpf_kfunc_end_defs(); +#endif /* CONFIG_BPF_SYSCALL */ + /* Tracepoints definitions. */ EXPORT_TRACEPOINT_SYMBOL(kmalloc); EXPORT_TRACEPOINT_SYMBOL(kmem_cache_alloc); From patchwork Thu Oct 10 23:25:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Namhyung Kim X-Patchwork-Id: 13831303 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DD9E2D2445D for ; Thu, 10 Oct 2024 23:25:16 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3FA7A6B008A; Thu, 10 Oct 2024 19:25:13 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 35A196B008C; Thu, 10 Oct 2024 19:25:13 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 188F56B0093; Thu, 10 Oct 2024 19:25:13 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id E43136B008A for ; Thu, 10 Oct 2024 19:25:12 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 6C741C08AF for ; Thu, 10 Oct 2024 23:25:08 +0000 (UTC) X-FDA: 82659275664.11.8D24E10 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf19.hostedemail.com (Postfix) with ESMTP id D2A0C1A0005 for ; Thu, 10 Oct 2024 23:25:08 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=QHpcbVx7; spf=pass (imf19.hostedemail.com: domain of namhyung@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=namhyung@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1728602666; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=iMx/c3Pk3dJhd0ShTmpeWu8SQK5CcSHEJL4yO87fhIc=; b=3QSr8HDd93Jv0pWHnyl33hAu1jc80J0QPw10gZ+YvR9KnGHTfcz03A4Eh51LQ1nXekZwm2 tiTjsoorXTnYpZZjIMsAt86ISOslfMFqWke8r+V5Fj5iOfZ3gphV3Xm2XM6tQ8rklgjAhd zTwVYfno2+VqWSwTA45dsBDKczHyHng= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=QHpcbVx7; spf=pass (imf19.hostedemail.com: domain of namhyung@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=namhyung@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1728602666; a=rsa-sha256; cv=none; b=6EbOfLo0/meqrm+R00qQ7bn0aw1jpQyRpCpoZho41oEKbHqBdY5kb9RofNSRn19FTAwYMF XaJO0dycRx1scAr4Z4Y06DUTMRNmf5f8+a5ssc8HQx/wlmztSuRkl1Z2Fjrw6j51Xl3Rjm ppWnies1YRjnMVZC5x7NGTPqb8IgZfg= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id E14395C600F; Thu, 10 Oct 2024 23:25:05 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 32FDEC4CECD; Thu, 10 Oct 2024 23:25:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1728602710; bh=7yJR3lSy+PJG3f5Th9Zl71ZDLgIU1wQOEfOZlZTpl6A=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=QHpcbVx7KlyHyNWV8tvabSN0gHqEVUyi1DUCy8I3uD0ebc++xqJTwF2xkjqB9RKeR bC2Gnrp/Wk4moBDDnqhiWejPrMC9uspwRv/0qVXbX/j7KJP7b5Z6IFWlRD49quVJV4 TiAFnjE/6U3aEOmpjUUPf0Ksy54llXg0N/So/YsDcRFh9MlMVaGx+XE08ynsMihBvg h3TToYrEr757LszZ/2Zg2KOehFRRJaLJ9AxSCg+cPJqUWy3Sied2tka2i0QAWSB2ql zNQNa5ncObsz5PPvtWF9Pgwx/ANuamQZB4ZW+GLnCHROCY/LlvLEIodS5SNwNl4N0u xVRdxRzRlpdqw== From: Namhyung Kim To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: Martin KaFai Lau , Eduard Zingerman , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Jiri Olsa , LKML , bpf@vger.kernel.org, Andrew Morton , Christoph Lameter , Pekka Enberg , David Rientjes , Joonsoo Kim , Vlastimil Babka , Roman Gushchin , Hyeonggon Yoo <42.hyeyoo@gmail.com>, linux-mm@kvack.org, Arnaldo Carvalho de Melo , Kees Cook , "Paul E. McKenney" Subject: [PATCH v5 bpf-next 3/3] selftests/bpf: Add a test for kmem_cache_iter Date: Thu, 10 Oct 2024 16:25:05 -0700 Message-ID: <20241010232505.1339892-4-namhyung@kernel.org> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog In-Reply-To: <20241010232505.1339892-1-namhyung@kernel.org> References: <20241010232505.1339892-1-namhyung@kernel.org> MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: mhhxmyn8ia7rjp76w9mdfpsw85ti9dz1 X-Rspamd-Queue-Id: D2A0C1A0005 X-Rspamd-Server: rspam11 X-HE-Tag: 1728602708-355061 X-HE-Meta: U2FsdGVkX19QMtavm+BLjS4wGBdpXGQXW1iv0h2HBnLsIQOI0acMsC/4/02+mxaVDZu5AKmiPvQbpNxUIjUnQyZAUHg8ue0t6eXtVvzjqSdFuHj/i54nsTsNO6H41UBPuyErmn1RVOjfdh95xl23P3VN3+Ge2xzpWo3TxGtnQFvlju5Y+nTYbqLaFOdMQcwESKQsar/5Yau3LMm1d2BcabCnTeIS97cysD1ESW3epRr+jgHWJNkoK6AEiAIbEGY7oBtD0vnWHJYijQs6n3L5mxBWnw0GLhAo5LomPGxmYanK+fUN8DejBV2eRY5P4J78czkhlV/NfFhUe5J+HwhOkScDIn7aGZ2U5MkzcWJi5k0pP5bPBNEjJXNLw2CotnfqWxovWhMmBKWmpuRJMkL6AuPorWsBueYPY62Zue9K4dtHHV1Ifo5cznLygVynhTzxT3A1RgN0yttvNxBPXZ65D9wRUL/Q0zICCxpNcH8oaz6B2RtPeZG8McoBBpnic91NyYk9Goy4WFBhp3NK/BZg6xAfCnmtRWxvRSilB4/R5H8ncySu3g8GRFFLopdyvQeAR1g2zBDGVMgmIbtIjrbRVH7vNrUxw2QuMeI/lz1G/+p2+1MqGBGeHIw/6zlvqk9FIaLbBIv/voIps18k8yIWhJ9VCB96sxj5eJfFV78XXjHbH5qlAdv+uOS58xksG4vCzuy3qAPLcX/b1GC4Y91NWdZM2vcb7P7a3aJXcFVbjUKBq8/QlN/dYlgtmQ2vqE3iWxq0F7ZArSQ9YtmjVtln0ltRwdoWB4RKJwswzULWE6hO4eKEvyHGeNe0TII46cLz19IPyI5mL61lZLPwQ9lmab/okJPEhtuvtQtIyo+XeqwmZ+kgllvaknt9GJ96aCWrdr7LM5HdxFpX7RR0aiG/BjLGM9xh8UL9v+G0i4iBRAz9cZ1xD9wORuSI592+wEWWeNSYwLivqPntEsc+TT2 5/v4nJxH 5S8Z4hxCUOw6m/zCKeiykN1YDKOHE2a269JM2y6/K3DSw702XpDQctJTLa8yyv60jiiWu+cpcjH+X2hUjhhanPHBIvwjZBGXSGZWdygLM1ema/2cwV/1N7OfEALU+90ueLARptxLfxRxc0i/Usq18V3R2reL8fvGKtc8IOHY7utOpkeaHMvWswShb+z5gJpfywid9Sq92QKE/HUVKQ7LULrSbegqGE+58jkhMFOSnUEp/GWWuKrxl3zNsxfG+yD1LqqA8AGsAAmdO0yBOzDwMzBaH/mEhKHfSEpMNuC4ZApfuFnz3KhmmMteGORWt1yD/0ioteKsSuRbfd8PDi0TC4sh74WtuC1wYDzPF X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The test traverses all slab caches using the kmem_cache_iter and save the data into slab_result array map. And check if current task's pointer is from "task_struct" slab cache using bpf_get_kmem_cache(). Also compare the result array with /proc/slabinfo if available (when CONFIG_SLUB_DEBUG is on). Note that many of the fields in the slabinfo are transient, so it only compares the name and objsize fields. Signed-off-by: Namhyung Kim --- .../bpf/prog_tests/kmem_cache_iter.c | 115 ++++++++++++++++++ tools/testing/selftests/bpf/progs/bpf_iter.h | 7 ++ .../selftests/bpf/progs/kmem_cache_iter.c | 95 +++++++++++++++ 3 files changed, 217 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/kmem_cache_iter.c create mode 100644 tools/testing/selftests/bpf/progs/kmem_cache_iter.c diff --git a/tools/testing/selftests/bpf/prog_tests/kmem_cache_iter.c b/tools/testing/selftests/bpf/prog_tests/kmem_cache_iter.c new file mode 100644 index 0000000000000000..848d8fc9171fae45 --- /dev/null +++ b/tools/testing/selftests/bpf/prog_tests/kmem_cache_iter.c @@ -0,0 +1,115 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2024 Google */ + +#include +#include +#include +#include "kmem_cache_iter.skel.h" + +#define SLAB_NAME_MAX 32 + +struct kmem_cache_result { + char name[SLAB_NAME_MAX]; + long obj_size; +}; + +static void subtest_kmem_cache_iter_check_task_struct(struct kmem_cache_iter *skel) +{ + LIBBPF_OPTS(bpf_test_run_opts, opts, + .flags = 0, /* Run it with the current task */ + ); + int prog_fd = bpf_program__fd(skel->progs.check_task_struct); + + /* Get task_struct and check it if's from a slab cache */ + ASSERT_OK(bpf_prog_test_run_opts(prog_fd, &opts), "prog_test_run"); + + /* The BPF program should set 'found' variable */ + ASSERT_EQ(skel->bss->task_struct_found, 1, "task_struct_found"); +} + +static void subtest_kmem_cache_iter_check_slabinfo(struct kmem_cache_iter *skel) +{ + FILE *fp; + int map_fd; + char name[SLAB_NAME_MAX]; + unsigned long objsize; + char rest_of_line[1000]; + struct kmem_cache_result r; + int seen = 0; + + fp = fopen("/proc/slabinfo", "r"); + if (fp == NULL) { + /* CONFIG_SLUB_DEBUG is not enabled */ + return; + } + + map_fd = bpf_map__fd(skel->maps.slab_result); + + /* Ignore first two lines for header */ + fscanf(fp, "slabinfo - version: %*d.%*d\n"); + fscanf(fp, "# %*s %*s %*s %*s %*s %*s : %[^\n]\n", rest_of_line); + + /* Compare name and objsize only - others can be changes frequently */ + while (fscanf(fp, "%s %*u %*u %lu %*u %*u : %[^\n]\n", + name, &objsize, rest_of_line) == 3) { + int ret = bpf_map_lookup_elem(map_fd, &seen, &r); + + if (!ASSERT_OK(ret, "kmem_cache_lookup")) + break; + + ASSERT_STREQ(r.name, name, "kmem_cache_name"); + ASSERT_EQ(r.obj_size, objsize, "kmem_cache_objsize"); + + seen++; + } + + ASSERT_EQ(skel->bss->kmem_cache_seen, seen, "kmem_cache_seen_eq"); + + fclose(fp); +} + +void test_kmem_cache_iter(void) +{ + DECLARE_LIBBPF_OPTS(bpf_iter_attach_opts, opts); + struct kmem_cache_iter *skel = NULL; + union bpf_iter_link_info linfo = {}; + struct bpf_link *link; + char buf[256]; + int iter_fd; + + skel = kmem_cache_iter__open_and_load(); + if (!ASSERT_OK_PTR(skel, "kmem_cache_iter__open_and_load")) + return; + + opts.link_info = &linfo; + opts.link_info_len = sizeof(linfo); + + link = bpf_program__attach_iter(skel->progs.slab_info_collector, &opts); + if (!ASSERT_OK_PTR(link, "attach_iter")) + goto destroy; + + iter_fd = bpf_iter_create(bpf_link__fd(link)); + if (!ASSERT_GE(iter_fd, 0, "iter_create")) + goto free_link; + + memset(buf, 0, sizeof(buf)); + while (read(iter_fd, buf, sizeof(buf) > 0)) { + /* Read out all contents */ + printf("%s", buf); + } + + /* Next reads should return 0 */ + ASSERT_EQ(read(iter_fd, buf, sizeof(buf)), 0, "read"); + + if (test__start_subtest("check_task_struct")) + subtest_kmem_cache_iter_check_task_struct(skel); + if (test__start_subtest("check_slabinfo")) + subtest_kmem_cache_iter_check_slabinfo(skel); + + close(iter_fd); + +free_link: + bpf_link__destroy(link); +destroy: + kmem_cache_iter__destroy(skel); +} diff --git a/tools/testing/selftests/bpf/progs/bpf_iter.h b/tools/testing/selftests/bpf/progs/bpf_iter.h index c41ee80533ca219a..3305dc3a74b32481 100644 --- a/tools/testing/selftests/bpf/progs/bpf_iter.h +++ b/tools/testing/selftests/bpf/progs/bpf_iter.h @@ -24,6 +24,7 @@ #define BTF_F_PTR_RAW BTF_F_PTR_RAW___not_used #define BTF_F_ZERO BTF_F_ZERO___not_used #define bpf_iter__ksym bpf_iter__ksym___not_used +#define bpf_iter__kmem_cache bpf_iter__kmem_cache___not_used #include "vmlinux.h" #undef bpf_iter_meta #undef bpf_iter__bpf_map @@ -48,6 +49,7 @@ #undef BTF_F_PTR_RAW #undef BTF_F_ZERO #undef bpf_iter__ksym +#undef bpf_iter__kmem_cache struct bpf_iter_meta { struct seq_file *seq; @@ -165,3 +167,8 @@ struct bpf_iter__ksym { struct bpf_iter_meta *meta; struct kallsym_iter *ksym; }; + +struct bpf_iter__kmem_cache { + struct bpf_iter_meta *meta; + struct kmem_cache *s; +} __attribute__((preserve_access_index)); diff --git a/tools/testing/selftests/bpf/progs/kmem_cache_iter.c b/tools/testing/selftests/bpf/progs/kmem_cache_iter.c new file mode 100644 index 0000000000000000..1cff8c7772683caf --- /dev/null +++ b/tools/testing/selftests/bpf/progs/kmem_cache_iter.c @@ -0,0 +1,95 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2024 Google */ + +#include "bpf_iter.h" +#include +#include + +char _license[] SEC("license") = "GPL"; + +#define SLAB_NAME_MAX 32 + +struct kmem_cache_result { + char name[SLAB_NAME_MAX]; + long obj_size; +}; + +struct { + __uint(type, BPF_MAP_TYPE_HASH); + __uint(key_size, sizeof(void *)); + __uint(value_size, SLAB_NAME_MAX); + __uint(max_entries, 1); +} slab_hash SEC(".maps"); + +struct { + __uint(type, BPF_MAP_TYPE_ARRAY); + __uint(key_size, sizeof(int)); + __uint(value_size, sizeof(struct kmem_cache_result)); + __uint(max_entries, 1024); +} slab_result SEC(".maps"); + +extern void bpf_rcu_read_lock(void) __ksym; +extern void bpf_rcu_read_unlock(void) __ksym; +extern struct kmem_cache *bpf_get_kmem_cache(u64 addr) __ksym; + +/* Result, will be checked by userspace */ +int task_struct_found; +int kmem_cache_seen; + +SEC("iter/kmem_cache") +int slab_info_collector(struct bpf_iter__kmem_cache *ctx) +{ + struct seq_file *seq = ctx->meta->seq; + struct kmem_cache *s = ctx->s; + struct kmem_cache_result *r; + int idx; + + if (s) { + /* To make sure if the slab_iter implements the seq interface + * properly and it's also useful for debugging. + */ + BPF_SEQ_PRINTF(seq, "%s: %u\n", s->name, s->size); + + idx = kmem_cache_seen; + r = bpf_map_lookup_elem(&slab_result, &idx); + if (r == NULL) + return 0; + + kmem_cache_seen++; + + /* Save name and size to match /proc/slabinfo */ + bpf_probe_read_kernel_str(r->name, sizeof(r->name), s->name); + r->obj_size = s->size; + + if (!bpf_strncmp(r->name, 11, "task_struct")) + bpf_map_update_elem(&slab_hash, &s, r->name, BPF_NOEXIST); + } + + return 0; +} + +SEC("raw_tp/bpf_test_finish") +int BPF_PROG(check_task_struct) +{ + u64 curr = bpf_get_current_task(); + struct kmem_cache *s; + char *name; + + bpf_rcu_read_lock(); + + s = bpf_get_kmem_cache(curr); + if (s == NULL) { + task_struct_found = -1; + bpf_rcu_read_unlock(); + return 0; + } + + name = bpf_map_lookup_elem(&slab_hash, &s); + if (name && !bpf_strncmp(name, 11, "task_struct")) + task_struct_found = 1; + else + task_struct_found = -2; + + bpf_rcu_read_unlock(); + return 0; +}