From patchwork Fri Sep 27 18:41:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Namhyung Kim X-Patchwork-Id: 13814485 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B3A8BCDD1D2 for ; Fri, 27 Sep 2024 18:41:43 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6E7176B0128; Fri, 27 Sep 2024 14:41:41 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 696606B012A; Fri, 27 Sep 2024 14:41:41 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 536A26B012B; Fri, 27 Sep 2024 14:41:41 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 348E56B0128 for ; Fri, 27 Sep 2024 14:41:41 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id D64061A13F1 for ; Fri, 27 Sep 2024 18:41:40 +0000 (UTC) X-FDA: 82611386760.11.A9CE6A9 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf14.hostedemail.com (Postfix) with ESMTP id 2F5FE100008 for ; Fri, 27 Sep 2024 18:41:38 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b="mpN/lLFl"; dmarc=pass (policy=quarantine) header.from=kernel.org; spf=pass (imf14.hostedemail.com: domain of namhyung@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=namhyung@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1727462480; a=rsa-sha256; cv=none; b=EHO7OcZdNsQ2UzMF9IOCL0edG9QSaC2SW44HtydFrG5a1j0zwBefvSYPs8QOSBHXWHELlX M+VgFUlpWgNrbpIT6RiPVt4EZFhMEuefMD7R2Bi6Z1DDsu928N4HOjmEnBbJIVPpWTC9Ot 6IF6qwNaNvB8/0Xekfa6EhP7qFC/3lA= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b="mpN/lLFl"; dmarc=pass (policy=quarantine) header.from=kernel.org; spf=pass (imf14.hostedemail.com: domain of namhyung@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=namhyung@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1727462480; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=/fudKJsChFaU2ONbY5+4NGPqNDMKtJmMIBOKIHSnlRY=; b=pBKaZHBnKt4f8XDWRD47dBH3shrLWdYZ5WEbylinM3o3R0WbkAu5dPvqgAvGq0JvG1rUSe hxQsz/NRvuRkJaCw7UT9wGbmlytf+UF3xUPYPHIRJwzlglCpMYEwf9faS7PpKhNj7UZpGw 46NLeBcdg9+i9KbsZ15LJdzducU0LDk= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 295BE5C5A25; Fri, 27 Sep 2024 18:41:33 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id C852AC4CED0; Fri, 27 Sep 2024 18:41:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1727462496; bh=kkigyrBYeiWFl3tgn2/scYgLndEN9wtat/LKpQKHAbM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=mpN/lLFlB1Vb1cprdBgZpMgvY7xKvVW4ERpyob0A4VD5ETjBfYgXi5tR6/8EAF+Jb 1EcA81jZSJ7KmCVBP9+6vc4aJqOy0eKwdD33fngUqmVFPrTLgpSkQBX2RIByycMYrN 5RgnokD2ZNhl5Xm/6BhI1KIMqHOa2Uwa7EAyGs1C6KwXVpJRC0tK4z+gDiUrguKvv2 iu1kuFDjSJ3ZhR8zEhurS21hnyht5jvxNxGc0Xy5KrRNGCJ38g8ll1FATQLjDFHUt+ r/SzBZ81vAKbFt3VbIB1dg77XaUGQrKBFfP6uXPYmU7m+/B5ecq6z7mVufBXloKQuj nnTrTYVsUKbWw== From: Namhyung Kim To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: Martin KaFai Lau , Eduard Zingerman , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Jiri Olsa , LKML , bpf@vger.kernel.org, Andrew Morton , Christoph Lameter , Pekka Enberg , David Rientjes , Joonsoo Kim , Vlastimil Babka , Roman Gushchin , Hyeonggon Yoo <42.hyeyoo@gmail.com>, linux-mm@kvack.org, Arnaldo Carvalho de Melo Subject: [RFC/PATCH bpf-next 1/3] bpf: Add kmem_cache iterator Date: Fri, 27 Sep 2024 11:41:31 -0700 Message-ID: <20240927184133.968283-2-namhyung@kernel.org> X-Mailer: git-send-email 2.46.1.824.gd892dcdcdd-goog In-Reply-To: <20240927184133.968283-1-namhyung@kernel.org> References: <20240927184133.968283-1-namhyung@kernel.org> MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: 94h8te6gnfoff6unfsw31mu9fg6384yi X-Rspamd-Queue-Id: 2F5FE100008 X-Rspamd-Server: rspam02 X-HE-Tag: 1727462498-741067 X-HE-Meta: U2FsdGVkX18EDIdY7Fp4VWZX57Jl5QZG5bxP5y5G6a1YhcbZJcZ0oUqDoID4RA4HRFRU/gOpzjUdyJEQIxhWRMH8UCjg103RZxiYVY9aK30nbkcscE+tMWfsHgrBW5NxpHpYQAXqb9Z71IMbzvaaz24hOnljDmIZMF31a8Azair3rvmy6XCQqj+vwoP5gAzsdeYvMQ0SuBjsJiQMWg+lsxkrBrLPsacnWXtBiSZosP3KeQakvzybOwXtHLe4imPXlgFLZ8dFZ/mfUcy4U3VLHzY4L4MfZvEgvlcSyFXRxDOR1ATnF1879EltTGxxPcTjsfj5KJZepGKplQpKoH8iC1g/Yv+Xu2DjJ2DHA2QjjZsJHpU12EAQoqCMgLscwAPemrc1R3xVG4Z80zeeRZPUMQomHtZvx0P3i7uuTq4dRaaQ7DT+0IbJSmVcanMJINFlSRF8XxukpvWfFY7JDZRWJ/66SKIIHrid8I9HTgbaIIapjCxLffWQRnQCPknEQ7UI8o8MaX4zbDgHklNV1aJ9Jk+SG9B7tMh5dQlWrU2Hlwf4R8NnDL0AZq0aIer9I7VYh7JpsP7de5ahcDnV8GmexLmPHPuUd8E/gyZS3vVrABFHCEPdGM3RTSqro1ZDp2N5jveGaIsdbqL0M8OuJGcEUkR+TKoKgWtsPdiXvzou5FGDN89uKS3JW+kuTQU63+PIXhTN9LZvnIG3Jd+yzVqpHEma97mjUtqBzquVSZUCESJksY7JDloiMu0yDfAtIfQUjMoUPjjwi/HGsbIZug6/RE17i7qlhhKcM5GrV55kA35SSFaam+Q07l566ZBG6pafsZ+lAinRCV77JAx0XXJtTrU+wO4JJM/0BrCNdGaZZHswCWAuLXPeezA7Z5fwY71IzRCsuTiFeVsDvDdiKvjiFw/6c0ovzvzQFSJJK2GBQlNLTs7r0yzgGu44ao0SWZFTRVq7EngaB3j0/qoVwDR gjcHjHjf 6fejG+MRDmKZHaa2ZwolsqreRz2/aaDCTFCAFTP0jccfYA/LTTreAeVNA2BrUMJfG81c5NxjgEq1vLquTRBQ5nXUxXxGchShj++r4rOc1sus2zRKD2ojn7LisfGNu4isHMdaGt2KkFOUBdoM5fiSJNs30+gU6Np1MV5xO/+caOzo4s2gKKgRaK3wywbF8edG7TjqiGi4IOEyZaQ29R3+jhJ/ITSbZ6ypFaxeKojPbg9RGk8ddnaGTrppeqzbkq09Ksi9LK/UfhSMjyN+Q9DN6Y769Nfu+xZvAeEzFvbMmxu/uxrYNRSf0Mb8aNtdWixj/ZEqHBD/PckFd/FHCQHJBisucw5QApxxlnaw5PNTHOywRQQlZsbef+hYTJZqASmHqWWJ7RaELmywM+rY= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The new "kmem_cache" iterator will traverse the list of slab caches and call attached BPF programs for each entry. It should check the argument (ctx.s) if it's NULL before using it. The iteration will be done with slab_mutex held but it'd break and return to user if the BPF program emits data to seq buffer more than the buffer size given by the user. IOW the whole iteration would be protected by the slab_mutex as long as it won't emit anything. It includes the internal "mm/slab.h" header to access kmem_cache, slab_caches and slab_mutex. Hope it's ok to mm folks. Acked-by: Roman Gushchin (mm/*) Acked-by: Vlastimil Babka #mm/slab Signed-off-by: Namhyung Kim --- include/linux/btf_ids.h | 1 + kernel/bpf/Makefile | 1 + kernel/bpf/kmem_cache_iter.c | 131 +++++++++++++++++++++++++++++++++++ 3 files changed, 133 insertions(+) create mode 100644 kernel/bpf/kmem_cache_iter.c diff --git a/include/linux/btf_ids.h b/include/linux/btf_ids.h index c0e3e1426a82f5c4..139bdececdcfaefb 100644 --- a/include/linux/btf_ids.h +++ b/include/linux/btf_ids.h @@ -283,5 +283,6 @@ extern u32 btf_tracing_ids[]; extern u32 bpf_cgroup_btf_id[]; extern u32 bpf_local_storage_map_btf_id[]; extern u32 btf_bpf_map_id[]; +extern u32 bpf_kmem_cache_btf_id[]; #endif diff --git a/kernel/bpf/Makefile b/kernel/bpf/Makefile index 9b9c151b5c826b31..105328f0b9c04e37 100644 --- a/kernel/bpf/Makefile +++ b/kernel/bpf/Makefile @@ -52,3 +52,4 @@ obj-$(CONFIG_BPF_PRELOAD) += preload/ obj-$(CONFIG_BPF_SYSCALL) += relo_core.o obj-$(CONFIG_BPF_SYSCALL) += btf_iter.o obj-$(CONFIG_BPF_SYSCALL) += btf_relocate.o +obj-$(CONFIG_BPF_SYSCALL) += kmem_cache_iter.o diff --git a/kernel/bpf/kmem_cache_iter.c b/kernel/bpf/kmem_cache_iter.c new file mode 100644 index 0000000000000000..5f7436b52f2e6b06 --- /dev/null +++ b/kernel/bpf/kmem_cache_iter.c @@ -0,0 +1,131 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* Copyright (c) 2024 Google */ +#include +#include +#include +#include +#include + +#include "../../mm/slab.h" /* kmem_cache, slab_caches and slab_mutex */ + +struct bpf_iter__kmem_cache { + __bpf_md_ptr(struct bpf_iter_meta *, meta); + __bpf_md_ptr(struct kmem_cache *, s); +}; + +static void *kmem_cache_iter_seq_start(struct seq_file *seq, loff_t *pos) +{ + loff_t cnt = 0; + struct kmem_cache *s = NULL; + + mutex_lock(&slab_mutex); + + /* + * Find an entry at the given position in the slab_caches list instead + * of keeping a reference (of the last visited entry, if any) out of + * slab_mutex. It might miss something if one is deleted in the middle + * while it releases the lock. But it should be rare and there's not + * much we can do about it. + */ + list_for_each_entry(s, &slab_caches, list) { + if (cnt == *pos) + break; + + cnt++; + } + + if (cnt != *pos) + return NULL; + + ++*pos; + return s; +} + +static void kmem_cache_iter_seq_stop(struct seq_file *seq, void *v) +{ + struct bpf_iter_meta meta; + struct bpf_iter__kmem_cache ctx = { + .meta = &meta, + .s = v, + }; + struct bpf_prog *prog; + + meta.seq = seq; + prog = bpf_iter_get_info(&meta, true); + if (prog) + bpf_iter_run_prog(prog, &ctx); + + mutex_unlock(&slab_mutex); +} + +static void *kmem_cache_iter_seq_next(struct seq_file *seq, void *v, loff_t *pos) +{ + struct kmem_cache *s = v; + + ++*pos; + + if (list_last_entry(&slab_caches, struct kmem_cache, list) == s) + return NULL; + + return list_next_entry(s, list); +} + +static int kmem_cache_iter_seq_show(struct seq_file *seq, void *v) +{ + struct bpf_iter_meta meta; + struct bpf_iter__kmem_cache ctx = { + .meta = &meta, + .s = v, + }; + struct bpf_prog *prog; + int ret = 0; + + meta.seq = seq; + prog = bpf_iter_get_info(&meta, false); + if (prog) + ret = bpf_iter_run_prog(prog, &ctx); + + return ret; +} + +static const struct seq_operations kmem_cache_iter_seq_ops = { + .start = kmem_cache_iter_seq_start, + .next = kmem_cache_iter_seq_next, + .stop = kmem_cache_iter_seq_stop, + .show = kmem_cache_iter_seq_show, +}; + +BTF_ID_LIST_GLOBAL_SINGLE(bpf_kmem_cache_btf_id, struct, kmem_cache) + +static const struct bpf_iter_seq_info kmem_cache_iter_seq_info = { + .seq_ops = &kmem_cache_iter_seq_ops, +}; + +static void bpf_iter_kmem_cache_show_fdinfo(const struct bpf_iter_aux_info *aux, + struct seq_file *seq) +{ + seq_puts(seq, "kmem_cache iter\n"); +} + +DEFINE_BPF_ITER_FUNC(kmem_cache, struct bpf_iter_meta *meta, + struct kmem_cache *s) + +static struct bpf_iter_reg bpf_kmem_cache_reg_info = { + .target = "kmem_cache", + .feature = BPF_ITER_RESCHED, + .show_fdinfo = bpf_iter_kmem_cache_show_fdinfo, + .ctx_arg_info_size = 1, + .ctx_arg_info = { + { offsetof(struct bpf_iter__kmem_cache, s), + PTR_TO_BTF_ID_OR_NULL | PTR_TRUSTED }, + }, + .seq_info = &kmem_cache_iter_seq_info, +}; + +static int __init bpf_kmem_cache_iter_init(void) +{ + bpf_kmem_cache_reg_info.ctx_arg_info[0].btf_id = bpf_kmem_cache_btf_id[0]; + return bpf_iter_reg_target(&bpf_kmem_cache_reg_info); +} + +late_initcall(bpf_kmem_cache_iter_init); From patchwork Fri Sep 27 18:41:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Namhyung Kim X-Patchwork-Id: 13814484 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6EA88CDD1D4 for ; Fri, 27 Sep 2024 18:41:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E39886B0127; Fri, 27 Sep 2024 14:41:40 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id DE85F6B0128; Fri, 27 Sep 2024 14:41:40 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C3D616B0129; Fri, 27 Sep 2024 14:41:40 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id A43906B0127 for ; Fri, 27 Sep 2024 14:41:40 -0400 (EDT) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 5B49DC1395 for ; Fri, 27 Sep 2024 18:41:40 +0000 (UTC) X-FDA: 82611386760.28.6D3B226 Received: from nyc.source.kernel.org (nyc.source.kernel.org [147.75.193.91]) by imf05.hostedemail.com (Postfix) with ESMTP id C1A43100008 for ; Fri, 27 Sep 2024 18:41:38 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b="h/xmwgNC"; dmarc=pass (policy=quarantine) header.from=kernel.org; spf=pass (imf05.hostedemail.com: domain of namhyung@kernel.org designates 147.75.193.91 as permitted sender) smtp.mailfrom=namhyung@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1727462461; a=rsa-sha256; cv=none; b=ioUfjR+1VNKMLrh9Jmy4329mrrRP7rZnPNPA1FhHzLhEnjIkSscOLkyNPilW2JNLC3XtG0 6n2NniLQB9aTc6XqgKCA7tGcwJ/7o+Hm2y6ir+LeOxn3tZKjiMD6VNGnsG2aqQHvJD//T0 J/WnYKUt2JMjgA4e2po+qAegRoc77DQ= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b="h/xmwgNC"; dmarc=pass (policy=quarantine) header.from=kernel.org; spf=pass (imf05.hostedemail.com: domain of namhyung@kernel.org designates 147.75.193.91 as permitted sender) smtp.mailfrom=namhyung@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1727462461; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=viKyFMixliHnI2HInYy9r5fic0KTNf5AsKWSi3DHVDs=; b=1rrXVASjOxylDeKbgqy2uE05XDuiNet0Ewf26ud8Q64wbAjzzPNIxc82GKgWp0gHw4B9eL Cby3SFZtZywPYY9vgOrNCW9A103PQeEK0KklIth+qW5yQ7x39doFiD2qouvrk9Jgau1UDz u3ie5snwJxpAdkKplo9IO47ElDmFD3U= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by nyc.source.kernel.org (Postfix) with ESMTP id D38B8A459D8; Fri, 27 Sep 2024 18:41:29 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id F1612C4CED9; Fri, 27 Sep 2024 18:41:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1727462497; bh=aGXxmsEp/uzql1BOkNU+QldkkvWzmazSesv6pu9JlCg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=h/xmwgNC9gCQMLx3EopU51gjZjDEszjSx7m0YzwM9dxl9jGiQIfZfu7fXfcf2beZz d/SmetuGMSmmNIkLuEdEl53xnFkV5+JkvmJD79VTWBih+Z720cMEeOFnbFQ21+Gb5M ZbMqLldLREXQzUUeKjYRGs5aEqw86o+UBP2TIqbLSO80yAV65emdDN7Tiy3gkHXRcP fRxziCdTpjXwWpzBaEGpvJiZt+d5v3MoGa6drDAMezJzaQg9D8YiepRoq7StCxX3ZO +2ATJyOcO4sK/7+bL5IhOebBEP3qkQwwSS7I8sIMgi9HAlDmMxpDg5ZdIRXCzsrOEi pYWhKpTE5LbEA== From: Namhyung Kim To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: Martin KaFai Lau , Eduard Zingerman , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Jiri Olsa , LKML , bpf@vger.kernel.org, Andrew Morton , Christoph Lameter , Pekka Enberg , David Rientjes , Joonsoo Kim , Vlastimil Babka , Roman Gushchin , Hyeonggon Yoo <42.hyeyoo@gmail.com>, linux-mm@kvack.org, Arnaldo Carvalho de Melo Subject: [RFC/PATCH bpf-next 2/3] mm/bpf: Add bpf_get_kmem_cache() kfunc Date: Fri, 27 Sep 2024 11:41:32 -0700 Message-ID: <20240927184133.968283-3-namhyung@kernel.org> X-Mailer: git-send-email 2.46.1.824.gd892dcdcdd-goog In-Reply-To: <20240927184133.968283-1-namhyung@kernel.org> References: <20240927184133.968283-1-namhyung@kernel.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Queue-Id: C1A43100008 X-Rspamd-Server: rspam01 X-Stat-Signature: hwom1oroj539k6yrhakzj3b8ixazedit X-HE-Tag: 1727462498-760129 X-HE-Meta: U2FsdGVkX1+HSWd9JHuBWlCxdc2/UlmB17mUC6JMBkRv7fOSCKIvIpuQEu2MTB3NDGALw5pa0xuuCogh0ze63R4E1ALGL/UuFrAqWv6TUvRtgCbCjkK+tRtV7FwGojFWvQpP9G70Cc6Cwiu4UhTuhTj5+S1qKkE84bgXtZXnuU51pp4Mke+eqb0Q1wrQUQ50u5PlNL5PX288h44eHZ93nQQ9nWWcPtNLazQOXvHTHrNVQyJ+rUdLLWGBDQRwd6MHFX5FObb9cEO75VKUCSqv633A1RitKcIJVFK/BUiVU/D2Bbjza6mhod9Bm3RwQiZtjISy8TtpuvQ4v2BEYrCSUcw5llaHr2TwurSqF+kZrFAJQnq+kjd5TPB0dAxGnj3DyG6UjAONQmH+wrU27ebN/M/yRVFcPdDobNOIfS0imQu+RdA9HAombnslFTm/AQ3W7EHsNGNvRPqj68IBJRcNRk555vI+CYFT5GbyzVPopxFTgpIJoAi2F9zIWeeXqRwYzXzuav+dqEqRFHUC3TsV73qcLFVqGJ1JccG2mMD9vUDkHUzYQ2fHdPTKyFwEyh3+3bZvCr/km41USw377q2XlhEsBzPFt46qwi1v0Q4KWO1AHcwBVAoX9rDPbk4xpto3FfQglMxdbE0+UYpZQY+t+oQs6IY2pEqmgANqPIUjgc+Zy87tv1/n+lJliIF/rZSwB4qzFjm/8RmYG4eFnqRMV9ihCykP0Ps2vmqKzcWfeSCFVjBhpMmyrsYR+1ByHndjgOQbUpaTbaOrwyOa9fRnDeYFQhSYfxXAvMgNwVhIzI1Qb/XZKeBD6BMauqSeYJ+tZhK2IvONhncGbUKTjH6fi1FGIq7ifOmQ7YAu63eSZcJ8WjgKTOhckppOaYWloDF2SL32JdF1r1RWIP8ilKjaXt4DisRKm/L7+toMftsnBDEdkIXg+suE+TwTOUpENS/+Ht3jvLwqDiE2/+2VOut Fp5MHvFM qxGK7q6N1Bc20zC1RB2kDvchFlxq0OYsx0vJP9ctgIFe/iNXmyiVARtmhqA3dgRqtndOJ5aNJndvNoTovsNNk6OgT7KNRnpDFzW87yjlyi3oWjYpInuXuFQXIeYy/TMYzqz85u/exYUhO79gq5QLEmhZkTwaOatEsY32Y2jufgQ1nAyu9CQkhDzO8QCl9D31DFFDyocCahe/+afBG2J4P3MBjMgiYKunoOwLlYdYoobijkgODfUVNfjSSQXqBZ1fo8+ZosNHpeSCHHD5iVpJlKmq6mmZuzoj9DFAIFD1cskX6pTQcDUNGUAaODuwpkKAQy/2xXJ4aHu5hQZDlBbmtgeKc9uYdyi2FTOFF3Gap01zWrQPiqNDcavJs+r1qYzrwQaCW X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The bpf_get_kmem_cache() is to get a slab cache information from a virtual address like virt_to_cache(). If the address is a pointer to a slab object, it'd return a valid kmem_cache pointer, otherwise NULL is returned. It doesn't grab a reference count of the kmem_cache so the caller is responsible to manage the access. The intended use case for now is to symbolize locks in slab objects from the lock contention tracepoints. Suggested-by: Vlastimil Babka Acked-by: Roman Gushchin (mm/*) Acked-by: Vlastimil Babka #mm/slab Signed-off-by: Namhyung Kim --- kernel/bpf/helpers.c | 1 + mm/slab_common.c | 16 ++++++++++++++++ 2 files changed, 17 insertions(+) diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c index 1a43d06eab286c26..bbc5800ec3afc899 100644 --- a/kernel/bpf/helpers.c +++ b/kernel/bpf/helpers.c @@ -3090,6 +3090,7 @@ BTF_ID_FLAGS(func, bpf_iter_bits_new, KF_ITER_NEW) BTF_ID_FLAGS(func, bpf_iter_bits_next, KF_ITER_NEXT | KF_RET_NULL) BTF_ID_FLAGS(func, bpf_iter_bits_destroy, KF_ITER_DESTROY) BTF_ID_FLAGS(func, bpf_copy_from_user_str, KF_SLEEPABLE) +BTF_ID_FLAGS(func, bpf_get_kmem_cache, KF_RET_NULL) BTF_KFUNCS_END(common_btf_ids) static const struct btf_kfunc_id_set common_kfunc_set = { diff --git a/mm/slab_common.c b/mm/slab_common.c index 7443244656150325..e648b05a635b94bf 100644 --- a/mm/slab_common.c +++ b/mm/slab_common.c @@ -1322,6 +1322,22 @@ size_t ksize(const void *objp) } EXPORT_SYMBOL(ksize); +#ifdef CONFIG_BPF_SYSCALL +#include + +__bpf_kfunc_start_defs(); + +__bpf_kfunc struct kmem_cache *bpf_get_kmem_cache(u64 addr) +{ + struct slab *slab; + + slab = virt_to_slab((void *)(long)addr); + return slab ? slab->slab_cache : NULL; +} + +__bpf_kfunc_end_defs(); +#endif /* CONFIG_BPF_SYSCALL */ + /* Tracepoints definitions. */ EXPORT_TRACEPOINT_SYMBOL(kmalloc); EXPORT_TRACEPOINT_SYMBOL(kmem_cache_alloc); From patchwork Fri Sep 27 18:41:33 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Namhyung Kim X-Patchwork-Id: 13814486 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 47F69CDD1D3 for ; Fri, 27 Sep 2024 18:41:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EFCE26B012A; Fri, 27 Sep 2024 14:41:41 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id EAFCC6B012B; Fri, 27 Sep 2024 14:41:41 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D249A6B012C; Fri, 27 Sep 2024 14:41:41 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id B719F6B012A for ; Fri, 27 Sep 2024 14:41:41 -0400 (EDT) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 4C33E161470 for ; Fri, 27 Sep 2024 18:41:41 +0000 (UTC) X-FDA: 82611386802.05.2FDDEC0 Received: from nyc.source.kernel.org (nyc.source.kernel.org [147.75.193.91]) by imf15.hostedemail.com (Postfix) with ESMTP id AF3F3A000D for ; Fri, 27 Sep 2024 18:41:39 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=O0V0JRTC; dmarc=pass (policy=quarantine) header.from=kernel.org; spf=pass (imf15.hostedemail.com: domain of namhyung@kernel.org designates 147.75.193.91 as permitted sender) smtp.mailfrom=namhyung@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1727462482; a=rsa-sha256; cv=none; b=xCD2eFNyVDLElvfdr3VrQTw4mhNIBJDtl3FDKFcFgYGgggoEM3r7U8BUgoXvWfbLcDZZ1n vdyzRAJDs0WWGdpMJFk4bBRdTlVVIGIQ0n6to/hjYlt/1HQyCioVnIAg1CZcBCgfBjUReU 9b2t9iQ9ZuZuJ2slrjwkDmW2IiVgFXo= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=O0V0JRTC; dmarc=pass (policy=quarantine) header.from=kernel.org; spf=pass (imf15.hostedemail.com: domain of namhyung@kernel.org designates 147.75.193.91 as permitted sender) smtp.mailfrom=namhyung@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1727462482; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=no3Y9dfUjM0/B/e2JZI2W+mflRPOSUm9cJ7aDFbhLNI=; b=376dhYnsWbjsiO2VIjgbiizsNMsW3GlX9Os31NaYsejHAuswpIOZ/3C3lBCWEPkIoub3Rm jiX9AaU0j4+xxfSQg//JBwnzXSF+Bji3d7VR0JwROHgr383iKtyAznMuhYONbURnp3qv/Z YBjJIiSWAYZTng7kjVegdFqFvObQUdA= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by nyc.source.kernel.org (Postfix) with ESMTP id DE80BA459E2; Fri, 27 Sep 2024 18:41:30 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id E5A2EC4CEDD; Fri, 27 Sep 2024 18:41:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1727462498; bh=gSjsCO+yJeYz9Q0JxuHX0Iwg1Cw1ZvOjdo01YYnkwIo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=O0V0JRTCDe/wi56R1NflEXujgQBdOWrfznE7+vluy+tlpBux8OuX2VjTmiEIcWbEH TRE7muqFK7JcqDcg3FiisHUhxZdmBs6pIBLFv6tuVez+1psKp+nO2K0bk858ed5fo8 x4P8gaXsb4MnULmS8VvqfzKY9QPkn79ARxez5G6YdPtIr0kQboxI1U0E+1PhQRAvym Haicdh30Re2Dhap5VA9MGj9//kxcpDFifjilW46lTnD3b+PNWbbJVVJDg3p4FnFe3c ddg9JeOWBUGpkKaghbPKrpVUm3BiIjfFVr/m1E8qhHf7hwyJM9pRNLSYX/ifChdowt fCYi3t+ccwu7w== From: Namhyung Kim To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: Martin KaFai Lau , Eduard Zingerman , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Jiri Olsa , LKML , bpf@vger.kernel.org, Andrew Morton , Christoph Lameter , Pekka Enberg , David Rientjes , Joonsoo Kim , Vlastimil Babka , Roman Gushchin , Hyeonggon Yoo <42.hyeyoo@gmail.com>, linux-mm@kvack.org, Arnaldo Carvalho de Melo Subject: [RFC/PATCH bpf-next 3/3] selftests/bpf: Add a test for kmem_cache_iter Date: Fri, 27 Sep 2024 11:41:33 -0700 Message-ID: <20240927184133.968283-4-namhyung@kernel.org> X-Mailer: git-send-email 2.46.1.824.gd892dcdcdd-goog In-Reply-To: <20240927184133.968283-1-namhyung@kernel.org> References: <20240927184133.968283-1-namhyung@kernel.org> MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: s7g81sfng5ni5tzzajkx9fb98hp4es7n X-Rspamd-Queue-Id: AF3F3A000D X-Rspamd-Server: rspam02 X-HE-Tag: 1727462499-611285 X-HE-Meta: U2FsdGVkX1/b3aKPSSye85NQ75ZREQ0FsgnoN5csH/tpZB3NR46oisIaEvYJSXUFkyjtnFnJnA/W6+fqoCppMG8T1wGInZ7NY8bkSWM+19aht18WzS95aHh3cU5UDKlFiFZinw20slzg18KxP6gEmTTvnYsg9ZDh83X1MXdk1aPkOjo8V90vPnrMI+IBh+PUTgjNQMhgMm330JMIKjtGDWIJJv6G5oGuPcUm+xI7UVdg+UWCwfn2FEHiRv4U+ocnh1lpMaPWwrjjzsCaBeOY9IdpN0uql82nuT7JQuFMg0sl1ZR6oc4TOTkZMoynuvS8RlD8zp20U55kuCI4ZXxo3Y4aY/RhXVSHZfI1iBCvtfAEBkwQMeoJ7vDonrOmfTvLvu4v9G35njUGcpqLraxqsUw7R9Si639cXW2kTvw0VfT9yKfjwXrkcLHqPXl4LE3iqeDf5dk01fhMa0DHWqShy4fXrk93BGRZ6z006wddrSyrQLSdwlpvlc/0p3al/XMGJoNDVXfys+AwiRyScNa/1XJQoe4WOVkgZqsxRvb/ouQrbh0MV75hc+bIgC3z66R5D6o/g63eFOj9Nvw4trn0wrOWK161+Hzsk24DcIuMUYlRUa5PQpzQdVXwkF7OrWKi0Jjet+BI74ZUew00BCRNMe8RKncF3wIkW/bLW1IvJqQgiIFpyN0EiZ/Ei45uSLzIWCqJEWyR4YnJokHjP8l1/Sw2tEeagR2oFnLmTb21sboKgN1AvkbhlRIm2Lyzl8M/ObyZvNIN/2D8tuZfwEDJB+XqQSjnne5zHNCFruVZ2Xiq2qiSTR9PKmC1NV3tdZvsTZpMX+1PqC/DmYDwvo2d2IcFpdFVypevvR4rdvdlq1Jvxa4Ne6nOQaj+OYV+r+gRsaPvcg68BYggaMRhV8V/hBzc56rADYwZ2W74phYoDFhyn/S8EISve3nX+I+8BCUhNGV5yGBs5ODTOlv9nY7 8NlwbNVI KWur/ZGNfqoaLuAJwBJ7DgXZKwdkvwXiohOl14qUiiqLHIkH7J1JgtbujHEexC75tqkTilM9YrVTOlGdFgzb++3My6/C48yLMBxtiZ29vkLFRFcRFfK6MvBRer96J1tR9YtYmy3bXAB1DJdhBmeMopyfKZ5EIkMOibp8fVKhcPNfj3q1a86gmtx0/ZhW+nCnr43YUD+ptduoZzHCEYa83mh5HkGpGkitJMpJBjJukrWgroP12d2uFIJm3q92EdRSBRsVrMySMgwarC/gdyIJZud5TeU/NaaI0HQ1iN6KZ7TSHL3AXyA3WZkNsUOwAyKyPQC3JoN0azX58praZEmGokrWVJjRymZ/m3p38F1RS2MS+JjPDe3vEih2xw4hzvUlOVWswXEpyqqrp/Tg= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The test traverses all slab caches using the kmem_cache_iter and check if current task's pointer is from "task_struct" slab cache. Signed-off-by: Namhyung Kim --- .../bpf/prog_tests/kmem_cache_iter.c | 64 ++++++++++++++++++ tools/testing/selftests/bpf/progs/bpf_iter.h | 7 ++ .../selftests/bpf/progs/kmem_cache_iter.c | 66 +++++++++++++++++++ 3 files changed, 137 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/kmem_cache_iter.c create mode 100644 tools/testing/selftests/bpf/progs/kmem_cache_iter.c diff --git a/tools/testing/selftests/bpf/prog_tests/kmem_cache_iter.c b/tools/testing/selftests/bpf/prog_tests/kmem_cache_iter.c new file mode 100644 index 0000000000000000..814bcc453e9f3ccd --- /dev/null +++ b/tools/testing/selftests/bpf/prog_tests/kmem_cache_iter.c @@ -0,0 +1,64 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2024 Google */ + +#include +#include +#include +#include "kmem_cache_iter.skel.h" + +static void test_kmem_cache_iter_check_task(struct kmem_cache_iter *skel) +{ + LIBBPF_OPTS(bpf_test_run_opts, opts, + .flags = BPF_F_TEST_RUN_ON_CPU, + ); + int prog_fd = bpf_program__fd(skel->progs.check_task_struct); + + /* get task_struct and check it if's from a slab cache */ + bpf_prog_test_run_opts(prog_fd, &opts); + + /* the BPF program should set 'found' variable */ + ASSERT_EQ(skel->bss->found, 1, "found task_struct"); +} + +void test_kmem_cache_iter(void) +{ + DECLARE_LIBBPF_OPTS(bpf_iter_attach_opts, opts); + struct kmem_cache_iter *skel = NULL; + union bpf_iter_link_info linfo = {}; + struct bpf_link *link; + char buf[1024]; + int iter_fd; + + skel = kmem_cache_iter__open_and_load(); + if (!ASSERT_OK_PTR(skel, "kmem_cache_iter__open_and_load")) + return; + + opts.link_info = &linfo; + opts.link_info_len = sizeof(linfo); + + link = bpf_program__attach_iter(skel->progs.slab_info_collector, &opts); + if (!ASSERT_OK_PTR(link, "attach_iter")) + goto destroy; + + iter_fd = bpf_iter_create(bpf_link__fd(link)); + if (!ASSERT_GE(iter_fd, 0, "iter_create")) + goto free_link; + + memset(buf, 0, sizeof(buf)); + while (read(iter_fd, buf, sizeof(buf) > 0)) { + /* read out all contents */ + printf("%s", buf); + } + + /* next reads should return 0 */ + ASSERT_EQ(read(iter_fd, buf, sizeof(buf)), 0, "read"); + + test_kmem_cache_iter_check_task(skel); + + close(iter_fd); + +free_link: + bpf_link__destroy(link); +destroy: + kmem_cache_iter__destroy(skel); +} diff --git a/tools/testing/selftests/bpf/progs/bpf_iter.h b/tools/testing/selftests/bpf/progs/bpf_iter.h index c41ee80533ca219a..3305dc3a74b32481 100644 --- a/tools/testing/selftests/bpf/progs/bpf_iter.h +++ b/tools/testing/selftests/bpf/progs/bpf_iter.h @@ -24,6 +24,7 @@ #define BTF_F_PTR_RAW BTF_F_PTR_RAW___not_used #define BTF_F_ZERO BTF_F_ZERO___not_used #define bpf_iter__ksym bpf_iter__ksym___not_used +#define bpf_iter__kmem_cache bpf_iter__kmem_cache___not_used #include "vmlinux.h" #undef bpf_iter_meta #undef bpf_iter__bpf_map @@ -48,6 +49,7 @@ #undef BTF_F_PTR_RAW #undef BTF_F_ZERO #undef bpf_iter__ksym +#undef bpf_iter__kmem_cache struct bpf_iter_meta { struct seq_file *seq; @@ -165,3 +167,8 @@ struct bpf_iter__ksym { struct bpf_iter_meta *meta; struct kallsym_iter *ksym; }; + +struct bpf_iter__kmem_cache { + struct bpf_iter_meta *meta; + struct kmem_cache *s; +} __attribute__((preserve_access_index)); diff --git a/tools/testing/selftests/bpf/progs/kmem_cache_iter.c b/tools/testing/selftests/bpf/progs/kmem_cache_iter.c new file mode 100644 index 0000000000000000..3f6ec15a1bf6344c --- /dev/null +++ b/tools/testing/selftests/bpf/progs/kmem_cache_iter.c @@ -0,0 +1,66 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2024 Google */ + +#include "bpf_iter.h" +#include +#include + +char _license[] SEC("license") = "GPL"; + +#define SLAB_NAME_MAX 256 + +struct { + __uint(type, BPF_MAP_TYPE_HASH); + __uint(key_size, sizeof(void *)); + __uint(value_size, SLAB_NAME_MAX); + __uint(max_entries, 1024); +} slab_hash SEC(".maps"); + +extern struct kmem_cache *bpf_get_kmem_cache(__u64 addr) __ksym; + +/* result, will be checked by userspace */ +int found; + +SEC("iter/kmem_cache") +int slab_info_collector(struct bpf_iter__kmem_cache *ctx) +{ + struct seq_file *seq = ctx->meta->seq; + struct kmem_cache *s = ctx->s; + + if (s) { + char name[SLAB_NAME_MAX]; + + /* + * To make sure if the slab_iter implements the seq interface + * properly and it's also useful for debugging. + */ + BPF_SEQ_PRINTF(seq, "%s: %u\n", s->name, s->object_size); + + bpf_probe_read_kernel_str(name, sizeof(name), s->name); + bpf_map_update_elem(&slab_hash, &s, name, BPF_NOEXIST); + } + + return 0; +} + +SEC("raw_tp/bpf_test_finish") +int BPF_PROG(check_task_struct) +{ + __u64 curr = bpf_get_current_task(); + struct kmem_cache *s; + char *name; + + s = bpf_get_kmem_cache(curr); + if (s == NULL) { + found = -1; + return 0; + } + + name = bpf_map_lookup_elem(&slab_hash, &s); + if (name && !bpf_strncmp(name, 11, "task_struct")) + found = 1; + else + found = -2; + + return 0; +}