From patchwork Wed Oct 2 18:09:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Namhyung Kim X-Patchwork-Id: 13820195 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 67A5FCEB2E2 for ; Wed, 2 Oct 2024 18:10:03 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D8D1D6B02A6; Wed, 2 Oct 2024 14:10:02 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D43EA6B02AF; Wed, 2 Oct 2024 14:10:02 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B3DB46B02A6; Wed, 2 Oct 2024 14:10:02 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 8510C6B011E for ; Wed, 2 Oct 2024 14:10:02 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 26F8A140E01 for ; Wed, 2 Oct 2024 18:10:02 +0000 (UTC) X-FDA: 82629451044.22.8CBA6BE Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf08.hostedemail.com (Postfix) with ESMTP id 6CD6616001C for ; Wed, 2 Oct 2024 18:10:00 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=cFiiyiX1; spf=pass (imf08.hostedemail.com: domain of namhyung@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=namhyung@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1727892560; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=+HNPaEwOEv30YqIsbGZrnqdb+NT3GStBU1g8u3WoX4I=; b=aWTpLTJPGmq42xBM4zRHMX1ruX+RyTeyYYn+4Yd/oQxJK0e9mpvQftlqsZURyXaYGUOGMl 5C4x/tvG9K5k30xwXwE+gEerkp6bLhassNbqqoR2iP6NICcNgY0YVb9vtGL3VduH5A2R+r uII20SX4lCD+fOk73F2eSsHTrp7GcN0= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=cFiiyiX1; spf=pass (imf08.hostedemail.com: domain of namhyung@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=namhyung@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1727892560; a=rsa-sha256; cv=none; b=ONoWesnqeU6nqKjJ7Ui2KKSqBOvX3JzOC2lYQQPY5T3rUYhUAPshLTaMbE1MXXGznGNmfK scOwvb60ajxGR7i3f6WM7aNi1fJYVnJU06yHxgiqeoy0YyIEonmrK9VoePwL7v+C+U12oB W402Ekm+5Ev5lVGhNPpKuxVDp7bSDAg= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id BF4125C365B; Wed, 2 Oct 2024 18:09:54 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id E3F97C4CED3; Wed, 2 Oct 2024 18:09:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1727892598; bh=OspFckh9hcfNpQfmco31YwmJgcBtSOdPnDKdNDPKD04=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=cFiiyiX1K3U6QE/qWPMNufn3ufF9eAQ7QU8VWg84F7VQkXI9sYF+Dqgx0QhzilOkB yKEGYNBagzW52NB8dTcBOGGkaTx1puduS09f3KE4sAdGUdNtSXCIUWmib1raJAZyjs yxSYo9a3zhUKsvt7TSFzKe5/+1EbgsEy1cO36RLt1LGeZRwgtstndGxlAya9ZGoi4a xz62UNP+TEuOIQvLMVI3y7sYrIScLSIdirJIX2XXnwew/9qXIJrQ8VPDzH9x7b+nJh YI5pWUHU5FY8r5qJIdNL7XImWGm8MKS+5tNxxwiemvHN9G4khA8pQPFUMqZBQP3PnK 10Hbz3ixnpkXg== From: Namhyung Kim To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: Martin KaFai Lau , Eduard Zingerman , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Jiri Olsa , LKML , bpf@vger.kernel.org, Andrew Morton , Christoph Lameter , Pekka Enberg , David Rientjes , Joonsoo Kim , Vlastimil Babka , Roman Gushchin , Hyeonggon Yoo <42.hyeyoo@gmail.com>, linux-mm@kvack.org, Arnaldo Carvalho de Melo , Kees Cook Subject: [PATCH v4 bpf-next 1/3] bpf: Add kmem_cache iterator Date: Wed, 2 Oct 2024 11:09:54 -0700 Message-ID: <20241002180956.1781008-2-namhyung@kernel.org> X-Mailer: git-send-email 2.46.1.824.gd892dcdcdd-goog In-Reply-To: <20241002180956.1781008-1-namhyung@kernel.org> References: <20241002180956.1781008-1-namhyung@kernel.org> MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: o3xz186k1w7k6ud4uqk5s1tgepiu8jyq X-Rspamd-Queue-Id: 6CD6616001C X-Rspamd-Server: rspam11 X-HE-Tag: 1727892600-633463 X-HE-Meta: U2FsdGVkX1+J5pXJkyuceH83zzstXk63Rxjv6a4P+tTfWJsvYG+b7MuewlIY0DucxAQ8kwdDmp7ZlE6A2Y3AH6ohA70fhFannlzcpEtYmm5CtslZ+zt7vlGTw/2pWbMr6HAIcF4bPGqA4fMmI8uVoV3gmpJ8t2KgxNiFw4Lz2L2LteEeWpqgb72ytAw2akxTRzPx2xKhchD8WDur9vMC+zidDK3OCxcnS5gJspdHUh+23V8aC+lI101hDmj857KgSjfeMvoVqzfIkpMAsmhkHyU7X3DHw18hvjkon0LOIv5n60snZcsFz4/zGakZ9oB63wq1CiWRRmS758h7muQYrtT1mUIVkHiygy07Ikp5h1RVgl4JoOTdkXVNHnJykrqvJtALw95QuBEs8O7g2x/eIjBiKS+3Bg8F7nf4pn08cf7y7xSLwA8vznu8jC9Pc9VATLRbNiuJ/f3/jvJXYID7xcANr7dl7DjcUZWmJ5n2yOvHVtT3x7VUxZkgbWVQL2CvPRW7ckjkqVSqNXKUiGVS4lD/tCJjCfEUbkoybWtTnWbjHXB3J8QK4PQQaOA+Hwy7WI2VDeI/j/i4t+SyAZrZU94DOVJhCqhHoDpGNoezYTCCqualtvUBI8l1dF1kcqa0H4uIDNzP1chiFoPI2ddOoNJPSIvkyyZiTN7WnwHqAk9WgRHTupAtRdLQh3DYoWHuw6GpyBxSvhFmqwoozh6QkoomTXd3xT29ht65X8/0+RwZY4HEXv8NFDJsaGm4A4aS0b/6UUs8jUNrzV06sshRRKJ+EdztRbkznmqcGt1Vi3DmWWZiOi5+1RtncFCEYZafTE8fW7jeUqJakbdJ/3Ws1N+C29wPJJUgMmGaj00v4vJm24Zg9dzcA3a1nP7SJuJ2GAQRc2jYraRPj75wYllgjC5zd31BFbk0Q3z+1M2h3utADOvId+GY/Agh4TwR1zpb2tgrzZHWiOY8O1ZTxkt BXEqGTBH tbYmFqTiKBlQMCm/QoEjJ3N3o9TToFAezZ8RyUG88Woztzglicl85Jk4GjqgHGZBy+3ISmNp2dIFw52UPC/ShstZ3iD/KZ7AsusCSyChSzcqfr4BhRJ+5+m7e6IIwO46ghRiYlnd7FF65Wd4bGVt99atTJBhAV9gtT6vA2/J4al2Y7NRAWH/DMCIN7bpRT3Cteypj6BnOs89YFxsiGgtPId47L2JeWfpKSDchtUi5nOHEHxcFOJobuT0sKm4wT34prBXSod9G/tm3IHAxJEMlpILEEzRA+Cl4Z+uaN0+Sdn32IR7GPqhX06OS9YQFPr2TjJUAX/9KElsdvFVAS06+2ONzrgZ2GHCwjW1T X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The new "kmem_cache" iterator will traverse the list of slab caches and call attached BPF programs for each entry. It should check the argument (ctx.s) if it's NULL before using it. Now the iteration grabs the slab_mutex only if it traverse the list and releases the mutex when it runs the BPF program. The kmem_cache entry is protected by a refcount during the execution. It includes the internal "mm/slab.h" header to access kmem_cache, slab_caches and slab_mutex. Hope it's ok to mm folks. Signed-off-by: Namhyung Kim Acked-by: Vlastimil Babka #mm/slab --- I've removed the Acked-by's from Roman and Vlastimil since it's changed not to hold the slab_mutex and to manage the refcount. Please review this change again! include/linux/btf_ids.h | 1 + kernel/bpf/Makefile | 1 + kernel/bpf/kmem_cache_iter.c | 174 +++++++++++++++++++++++++++++++++++ 3 files changed, 176 insertions(+) create mode 100644 kernel/bpf/kmem_cache_iter.c diff --git a/include/linux/btf_ids.h b/include/linux/btf_ids.h index c0e3e1426a82f5c4..139bdececdcfaefb 100644 --- a/include/linux/btf_ids.h +++ b/include/linux/btf_ids.h @@ -283,5 +283,6 @@ extern u32 btf_tracing_ids[]; extern u32 bpf_cgroup_btf_id[]; extern u32 bpf_local_storage_map_btf_id[]; extern u32 btf_bpf_map_id[]; +extern u32 bpf_kmem_cache_btf_id[]; #endif diff --git a/kernel/bpf/Makefile b/kernel/bpf/Makefile index 9b9c151b5c826b31..105328f0b9c04e37 100644 --- a/kernel/bpf/Makefile +++ b/kernel/bpf/Makefile @@ -52,3 +52,4 @@ obj-$(CONFIG_BPF_PRELOAD) += preload/ obj-$(CONFIG_BPF_SYSCALL) += relo_core.o obj-$(CONFIG_BPF_SYSCALL) += btf_iter.o obj-$(CONFIG_BPF_SYSCALL) += btf_relocate.o +obj-$(CONFIG_BPF_SYSCALL) += kmem_cache_iter.o diff --git a/kernel/bpf/kmem_cache_iter.c b/kernel/bpf/kmem_cache_iter.c new file mode 100644 index 0000000000000000..e103d25175126ab0 --- /dev/null +++ b/kernel/bpf/kmem_cache_iter.c @@ -0,0 +1,174 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* Copyright (c) 2024 Google */ +#include +#include +#include +#include +#include + +#include "../../mm/slab.h" /* kmem_cache, slab_caches and slab_mutex */ + +struct bpf_iter__kmem_cache { + __bpf_md_ptr(struct bpf_iter_meta *, meta); + __bpf_md_ptr(struct kmem_cache *, s); +}; + +static void *kmem_cache_iter_seq_start(struct seq_file *seq, loff_t *pos) +{ + loff_t cnt = 0; + bool found = false; + struct kmem_cache *s; + + mutex_lock(&slab_mutex); + + /* + * Find an entry at the given position in the slab_caches list instead + * of keeping a reference (of the last visited entry, if any) out of + * slab_mutex. It might miss something if one is deleted in the middle + * while it releases the lock. But it should be rare and there's not + * much we can do about it. + */ + list_for_each_entry(s, &slab_caches, list) { + if (cnt == *pos) { + /* + * Make sure this entry remains in the list by getting + * a new reference count. Note that boot_cache entries + * have a negative refcount, so don't touch them. + */ + if (s->refcount > 0) + s->refcount++; + found = true; + break; + } + cnt++; + } + mutex_unlock(&slab_mutex); + + if (!found) + return NULL; + + ++*pos; + return s; +} + +static void kmem_cache_iter_seq_stop(struct seq_file *seq, void *v) +{ + struct bpf_iter_meta meta; + struct bpf_iter__kmem_cache ctx = { + .meta = &meta, + .s = v, + }; + struct bpf_prog *prog; + bool destroy = false; + + meta.seq = seq; + prog = bpf_iter_get_info(&meta, true); + if (prog) + bpf_iter_run_prog(prog, &ctx); + + if (ctx.s == NULL) + return; + + mutex_lock(&slab_mutex); + + /* Skip kmem_cache_destroy() for active entries */ + if (ctx.s->refcount > 1) + ctx.s->refcount--; + else if (ctx.s->refcount == 1) + destroy = true; + + mutex_unlock(&slab_mutex); + + if (destroy) + kmem_cache_destroy(ctx.s); +} + +static void *kmem_cache_iter_seq_next(struct seq_file *seq, void *v, loff_t *pos) +{ + struct kmem_cache *s = v; + struct kmem_cache *next = NULL; + bool destroy = false; + + ++*pos; + + mutex_lock(&slab_mutex); + + if (list_last_entry(&slab_caches, struct kmem_cache, list) != s) { + next = list_next_entry(s, list); + if (next->refcount > 0) + next->refcount++; + } + + /* Skip kmem_cache_destroy() for active entries */ + if (s->refcount > 1) + s->refcount--; + else if (s->refcount == 1) + destroy = true; + + mutex_unlock(&slab_mutex); + + if (destroy) + kmem_cache_destroy(s); + + return next; +} + +static int kmem_cache_iter_seq_show(struct seq_file *seq, void *v) +{ + struct bpf_iter_meta meta; + struct bpf_iter__kmem_cache ctx = { + .meta = &meta, + .s = v, + }; + struct bpf_prog *prog; + int ret = 0; + + meta.seq = seq; + prog = bpf_iter_get_info(&meta, false); + if (prog) + ret = bpf_iter_run_prog(prog, &ctx); + + return ret; +} + +static const struct seq_operations kmem_cache_iter_seq_ops = { + .start = kmem_cache_iter_seq_start, + .next = kmem_cache_iter_seq_next, + .stop = kmem_cache_iter_seq_stop, + .show = kmem_cache_iter_seq_show, +}; + +BTF_ID_LIST_GLOBAL_SINGLE(bpf_kmem_cache_btf_id, struct, kmem_cache) + +static const struct bpf_iter_seq_info kmem_cache_iter_seq_info = { + .seq_ops = &kmem_cache_iter_seq_ops, +}; + +static void bpf_iter_kmem_cache_show_fdinfo(const struct bpf_iter_aux_info *aux, + struct seq_file *seq) +{ + seq_puts(seq, "kmem_cache iter\n"); +} + +DEFINE_BPF_ITER_FUNC(kmem_cache, struct bpf_iter_meta *meta, + struct kmem_cache *s) + +static struct bpf_iter_reg bpf_kmem_cache_reg_info = { + .target = "kmem_cache", + .feature = BPF_ITER_RESCHED, + .show_fdinfo = bpf_iter_kmem_cache_show_fdinfo, + .ctx_arg_info_size = 1, + .ctx_arg_info = { + { offsetof(struct bpf_iter__kmem_cache, s), + PTR_TO_BTF_ID_OR_NULL | PTR_TRUSTED }, + }, + .seq_info = &kmem_cache_iter_seq_info, +}; + +static int __init bpf_kmem_cache_iter_init(void) +{ + bpf_kmem_cache_reg_info.ctx_arg_info[0].btf_id = bpf_kmem_cache_btf_id[0]; + return bpf_iter_reg_target(&bpf_kmem_cache_reg_info); +} + +late_initcall(bpf_kmem_cache_iter_init); From patchwork Wed Oct 2 18:09:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Namhyung Kim X-Patchwork-Id: 13820196 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7D9AECEACF3 for ; Wed, 2 Oct 2024 18:10:05 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 39FB46B02AF; Wed, 2 Oct 2024 14:10:03 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3297C6B02B7; Wed, 2 Oct 2024 14:10:03 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 153A36B02B0; Wed, 2 Oct 2024 14:10:03 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id D0DED6B02AE for ; Wed, 2 Oct 2024 14:10:02 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 731B4140D65 for ; Wed, 2 Oct 2024 18:10:02 +0000 (UTC) X-FDA: 82629451044.06.634BF06 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf27.hostedemail.com (Postfix) with ESMTP id C196F40019 for ; Wed, 2 Oct 2024 18:10:00 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=C1nN9bS1; spf=pass (imf27.hostedemail.com: domain of namhyung@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=namhyung@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1727892473; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=/gFwJL66H9iO1jq0RcGmk7K6IL+6CFpwiDSLj/EXxAo=; b=RwtNb7nr5PpL/Da/RHKFyf8RVixtMNPynTVjqcQcvpnmm1mKMP9tifL4SoFf+WlZPZ1eb6 DLHq+iGbJ9y8XNjSqLlpaALV66Hslb6iV+AAH7NqIq+xsQJvXyrOnm2Wvhflt1GMHK6FU0 CWlWb6XtVY58Y/oDXhqVcHgluXh9bB4= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1727892473; a=rsa-sha256; cv=none; b=UsE/z9QssCOts2csEgkwouluG1C5QOPFWdy02HC/9G1OLi/KZHI+1ZSNukDe2HmeuX+lUS 7ofcBqYvcl7carGDYLEcVZdsuojHML/wQ7tzqkXYB0eCfcQwdE+ljt83UcC4o74m2Bb8aG kw0stQc0+L1qGzR7JRLw4pjwtXCtmxU= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=C1nN9bS1; spf=pass (imf27.hostedemail.com: domain of namhyung@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=namhyung@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id BD3505C3E58; Wed, 2 Oct 2024 18:09:55 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id D3F17C4CED6; Wed, 2 Oct 2024 18:09:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1727892599; bh=CZ1VN+wISqfF7U/NBBkytViSmRb/vkGLNUY76gXrLxU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=C1nN9bS1juVNiiTLMsoIIP2DGHBhXblmGXHhsKBD+j6XPJwI5tpUtFSZYdSkMTJVU PBXFgrsz8B8ZV+VT0cgqgK6qrFs2w4+ycrTH0yA9gyW/m5l7V02/XBNfHa+8kFJAxc 4lDiS2LMUucmUhiFgs7PefmznRoDDt92VSgPcL4FERTeuDorLGV9p9f336azJI+4UE OA/Xcg20450+n41GGYuAvcr+sjyEwfRG4b5943hnrFLr1Jc36oc3uHQDVlH90Kgvc4 otDwmKd42es+dgexigXc+VKAxyPI3gA5HxIY2GFemE/uxwLmPRtnBaR2WSVkvv29E2 eedyeP/3Sdorw== From: Namhyung Kim To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: Martin KaFai Lau , Eduard Zingerman , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Jiri Olsa , LKML , bpf@vger.kernel.org, Andrew Morton , Christoph Lameter , Pekka Enberg , David Rientjes , Joonsoo Kim , Vlastimil Babka , Roman Gushchin , Hyeonggon Yoo <42.hyeyoo@gmail.com>, linux-mm@kvack.org, Arnaldo Carvalho de Melo , Kees Cook Subject: [PATCH v4 bpf-next 2/3] mm/bpf: Add bpf_get_kmem_cache() kfunc Date: Wed, 2 Oct 2024 11:09:55 -0700 Message-ID: <20241002180956.1781008-3-namhyung@kernel.org> X-Mailer: git-send-email 2.46.1.824.gd892dcdcdd-goog In-Reply-To: <20241002180956.1781008-1-namhyung@kernel.org> References: <20241002180956.1781008-1-namhyung@kernel.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: C196F40019 X-Stat-Signature: ogmjw4z9aiss3utzp8hgddojm86s3ckm X-Rspamd-Server: rspam09 X-Rspam-User: X-HE-Tag: 1727892600-696600 X-HE-Meta: U2FsdGVkX1/30dgTIDKInSRtjRrdlB5NNn3xuvz0t1ey+WZyBQcN268Eb2ZIptyrRetf14Cr4vM5WlttOSyOjbcYm78ViBIlKeQKxqr1xygrdmJD8hTAiRrkVDXWynK6YmgxrpsvWq9LWmBFL8GwYbPObhI/nbBrvKK6Di2y44OTQlsAeILz/OHh5dnuWfSiVfeCt/OVlIkkJgN/npssV3D5OOCvY7orunK/YK2b462TGdOifZ2V9n0Cqpu5SStZs4X6M96R+/qx+0eRtWEWDWmC679z3RnWwnzx2I6P8kroKA3NxsrwsGWfOLn/BmUxWkWQ08wtcWrDnHf8x986C49Svl9qz1zdruKigs+IIqmumX3unFyv5Mw7zCJW+e9jinZpwL5PTdQPbQbNdZIyhuiO3F9t8jIExyWAt65vacOby4UFO5W8FJep1Pv6iY7Q+UWzm3gej9St5rQ2boRQfK9DxDQ3m3uMFTiFvOJfCAcl+vk658GORM+Ks81R2WAhL1i6KDEr4cNCF29Ts3S2Kqt7tAtpXJYuKbrmmbspFcS32MavJzoNTRbXk5zfRfw7yKGZv6l8kPdkQ4zdYX/eNFtFLSkUhYRvlJwXcchoBBXOrgqfJ7au/gYUKhk2+1M2I5zmli7eZD9xOMbYbAM01jZTDRiPqInoo/uzJC5OCzuZsOkwZnRJyLp0OYa8utCfjEXJ9ycAP4uUaajvdjboBD8mHnlNrUvHEroUDgYzWkIEscFh+tPNmKJlZVT0zDt0L+SKcij2HAt5198APKA8sWf7jGrxU+DKkYsPcMEe2aol4tGSD0KIUIqAY2P0506yGdapHoDzqEuqvZvvfg4mpwmwFvKmkUDD1NA8AJRZ74eWdiXrTRcWRbp1xHMSnyVyZyOVHo42TNxjwnhElaoBKkzCQxA2D4U+vhSjjOVuYUzKwREfsb+IUT+BoH6V7k3IdYkvKGWwHUMt5LExFTf iP6VhU9s Uy8nyueuhAaiekmT7IQSzMWQoqs7s2AtPPZ+5kjIpPuABX+oP8hgy1zZ/nJJHzklZrQiGdJBj6+HTEW9h/N0EYBEVoCArbu0Nl6tsuJf/vJZQp6rn+80CXpzheW6ohSF4rMrio/kpZTS/PQTTIuktO9NIim0rrnLqFqomS3M5Vb66HRcz6bnIJ7HCLHjWLXl87tKtA1AuBrbeSpMo53deCUofj15N8inPTLXlBTozFwHSGM9q1pvuSeik0hqGgeeke78nAiHrNR6RVaWGrMuFb9idTEeOW86+oWHHVE/LlGoSBSQVN3occuQz8HS3UCV73NqfdbAEr7MKKRqIg6yRWARDybP2uSx1bxATdlRw8zwoW5slBSkwHezNAI1M81KsB13zNLrJaNBInz8= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The bpf_get_kmem_cache() is to get a slab cache information from a virtual address like virt_to_cache(). If the address is a pointer to a slab object, it'd return a valid kmem_cache pointer, otherwise NULL is returned. It doesn't grab a reference count of the kmem_cache so the caller is responsible to manage the access. The intended use case for now is to symbolize locks in slab objects from the lock contention tracepoints. Suggested-by: Vlastimil Babka Acked-by: Roman Gushchin (mm/*) Acked-by: Vlastimil Babka #mm/slab Signed-off-by: Namhyung Kim --- kernel/bpf/helpers.c | 1 + mm/slab_common.c | 19 +++++++++++++++++++ 2 files changed, 20 insertions(+) diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c index 4053f279ed4cc7ab..3709fb14288105c6 100644 --- a/kernel/bpf/helpers.c +++ b/kernel/bpf/helpers.c @@ -3090,6 +3090,7 @@ BTF_ID_FLAGS(func, bpf_iter_bits_new, KF_ITER_NEW) BTF_ID_FLAGS(func, bpf_iter_bits_next, KF_ITER_NEXT | KF_RET_NULL) BTF_ID_FLAGS(func, bpf_iter_bits_destroy, KF_ITER_DESTROY) BTF_ID_FLAGS(func, bpf_copy_from_user_str, KF_SLEEPABLE) +BTF_ID_FLAGS(func, bpf_get_kmem_cache, KF_RET_NULL) BTF_KFUNCS_END(common_btf_ids) static const struct btf_kfunc_id_set common_kfunc_set = { diff --git a/mm/slab_common.c b/mm/slab_common.c index 7443244656150325..5484e1cd812f698e 100644 --- a/mm/slab_common.c +++ b/mm/slab_common.c @@ -1322,6 +1322,25 @@ size_t ksize(const void *objp) } EXPORT_SYMBOL(ksize); +#ifdef CONFIG_BPF_SYSCALL +#include + +__bpf_kfunc_start_defs(); + +__bpf_kfunc struct kmem_cache *bpf_get_kmem_cache(u64 addr) +{ + struct slab *slab; + + if (!virt_addr_valid(addr)) + return NULL; + + slab = virt_to_slab((void *)(long)addr); + return slab ? slab->slab_cache : NULL; +} + +__bpf_kfunc_end_defs(); +#endif /* CONFIG_BPF_SYSCALL */ + /* Tracepoints definitions. */ EXPORT_TRACEPOINT_SYMBOL(kmalloc); EXPORT_TRACEPOINT_SYMBOL(kmem_cache_alloc); From patchwork Wed Oct 2 18:09:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Namhyung Kim X-Patchwork-Id: 13820197 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2386ECEB2E2 for ; Wed, 2 Oct 2024 18:10:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7E9DA6B0425; Wed, 2 Oct 2024 14:10:04 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 793E66B0427; Wed, 2 Oct 2024 14:10:04 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5E8366B0428; Wed, 2 Oct 2024 14:10:04 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 2F9B56B0425 for ; Wed, 2 Oct 2024 14:10:04 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 8008F40D4A for ; Wed, 2 Oct 2024 18:10:03 +0000 (UTC) X-FDA: 82629451086.14.A42E246 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf25.hostedemail.com (Postfix) with ESMTP id C602DA0011 for ; Wed, 2 Oct 2024 18:10:01 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=Wi4ubqCt; spf=pass (imf25.hostedemail.com: domain of namhyung@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=namhyung@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1727892536; a=rsa-sha256; cv=none; b=MUJv3aPbkY9gsSLvZtqrW6mM3kiF3xawsEYV3aYiB2FSfSAz28jr4MavknuRwSe7TcDJHp qsZFYPzHfLbSIzxSpnW5Nltz3zfpTia1U4QRQRCZ8olmlLhNFdz4l4QpLi/deqPSq6SlSt IwP7zYmgf0yZ+FlV0zJu6+EQOVxMgSs= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=Wi4ubqCt; spf=pass (imf25.hostedemail.com: domain of namhyung@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=namhyung@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1727892536; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=f0m9MJsP55KrK3+a3K8sFov9FGQ4S5Wxs3ZBzUE7qQg=; b=rrsvOlfEpC9dQFkvWr3u/U70Wxif+BtyU40qKNMowbutw3t6Ims1EPFN5YDZBteaEfK9v1 zjd9oS48zjyv2mghf0cGdsx9+sa1kC7Q0a08cO1bmcr0iiNFw+byZSlD7xNeYNLNqsYa80 q7jXvwAqq8R1yl4BUj5r5cKJpUejdno= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id AE3B05C3CAC; Wed, 2 Oct 2024 18:09:56 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id C8FEFC4CECF; Wed, 2 Oct 2024 18:09:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1727892600; bh=ePjQez0zpwMreMUAuz+KvIM2b9Vii7HFddbSYrTFhpc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Wi4ubqCtkGXIKFj8UkgIyBOAnxzZZ4rGMyF8fzGhCZlvRlhRcuVtq1zaxLxBO7D2B YIa0YE0VwzDvdvCx2E3L6+tfbVRSvYbgESP5vjC6VnYWMjTuDxAfbUFiOmyX+qE/yJ SpcOCRE2lFI1WeyjcJstBSx4pl1hcNBlwZyNKXuaoMXHRs/k4ITT1ouVY4jjtQ/3fY veNrgPyqOU7aoSqrAeVzyW2FUSVOZPt26DGJo/mQSzl5cYlwtPURE15RSR1BKw5goE SIhtC9kgrdwp+knOGWJ90EpG+dEiimHqjCMj+zAIgpL5ofo2WV3gGqVJFdiz48pyJA yr1h7z8v5Uj3w== From: Namhyung Kim To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: Martin KaFai Lau , Eduard Zingerman , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Jiri Olsa , LKML , bpf@vger.kernel.org, Andrew Morton , Christoph Lameter , Pekka Enberg , David Rientjes , Joonsoo Kim , Vlastimil Babka , Roman Gushchin , Hyeonggon Yoo <42.hyeyoo@gmail.com>, linux-mm@kvack.org, Arnaldo Carvalho de Melo , Kees Cook Subject: [PATCH v4 bpf-next 3/3] selftests/bpf: Add a test for kmem_cache_iter Date: Wed, 2 Oct 2024 11:09:56 -0700 Message-ID: <20241002180956.1781008-4-namhyung@kernel.org> X-Mailer: git-send-email 2.46.1.824.gd892dcdcdd-goog In-Reply-To: <20241002180956.1781008-1-namhyung@kernel.org> References: <20241002180956.1781008-1-namhyung@kernel.org> MIME-Version: 1.0 X-Stat-Signature: 6r8spdqmy48eqqnxrxfzh647ez78rnmx X-Rspamd-Queue-Id: C602DA0011 X-Rspam-User: X-Rspamd-Server: rspam10 X-HE-Tag: 1727892601-157220 X-HE-Meta: U2FsdGVkX1/cxYMag5wxMVwWVtFTaNhumlk7vWQ9i+hv3dWSj+oOUm855KOqqtA3xZLHkzVDb4kpe0VLhdT45f0omnTsIktlOutaAZkIGgS4LHI1vzyMUcOicNu3Wht2pyaBGtclYDnEpt0LJOsgJgZWQHVt1YoeTN0FnEDgbUJOf/Fgg8cX4zL+AhKqb/J8VnrbAL/og3LdtJ0XWPsTnDEFnZMpONR0+/MDzJQHDbo7EvNJNCFKtF/6djP1cohnGsroCt5ZT+1d9DmbS4AXxWlzKAL67nWThxwNV/tUszFtubDIoV4aoHoDeRNJIvR+qt2OOkm5i0vNdbaAh44DktdXZZv5nfMh6V+x5WFtx6P094W4H6pN4YVMfXwoLbbJc7V8fDznqqsms1jykHp/ouN1KfRm5sh26GTjuhkFz/QzTfaRpldrf4pJKBNq3C46+POtmZh1vmjQ+sWAVx027yhW3qcJ8vgoQlCv2dl1oqEtrOzdXM1ZKgNugyj8viUvugIxndy90vcnYS+bM0PfZtH6lcTylM0na03VU3xZtTagMzOgINJNDf4bDKXsA1MKPDc1D6E+DzCk9CWelOd8uKnRIR/CgdonQ15hZ+5VjOthIn8ACWxAbBxzUn+/79aE45sv3833TYUnXODu/fu8+zyRZrtOICM8Clrsbmg6X0kqN0lDcX3INjgzD6XW9TcXs1SupTN5K3JOnTeIIoOwX/Oh1ROKL7Hx1EP+ProtIrz/N2NB89viErCrMnMvtsJJ2E7JLt+2xOTz+NesMVSwZ91bGiSlsZF5QMe4naeq9SqNmzYODOmO/PoJMYztbLcVgRpaoFPhCr3+4mUq5TDn6TfbmJj9OrTSrubrhaS5d7l1gEqW+UYHaR1iRX/2Nd7yLyXrEf/edCU4NcKWiaylb4Olr7z/vmT+lKsPRlyZHrvgmBSUiY51TXOsJBK3G5KVp3l16lFhukecm6nePPL 1eZm7YFa MuOW2Ar5ZjGAg4tNUxCXYBG0I99Xk9aeqVZjFW7v7B+5KOlmcRje68ylJmUxMTdDwna8CJm4JNO2QY5bCUawjmUoU8glqMhpTmK7FamNYnVf5FvH3BhR3CQBGsBqRSYzYz6vxkprBR4cTNxmlZRK6rQQYqCmTr/1hJLiHMgttCKlg45UUz3rGLtNaW+5AUurUhAhsBUwu4KWQBlCyz6koeDc955zF/c63zf6ksxRvdk+VBTL+CycpKrqvo06yJN9K3hIJuE6kPAXjX1+uJ+LQa/T2tw0YP/F82C/umvcqIbwPP3hIKT4ar03VjIeuI7EFPHSTT1WnsvNL6fmrVDCAiMP1aAbGFAJtq0dIgbOFZ1FxxOlhUXdZcrPCEnY8j1m/hBxIRgkuwhsU048= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The test traverses all slab caches using the kmem_cache_iter and check if current task's pointer is from "task_struct" slab cache. Signed-off-by: Namhyung Kim --- .../bpf/prog_tests/kmem_cache_iter.c | 64 ++++++++++++++++++ tools/testing/selftests/bpf/progs/bpf_iter.h | 7 ++ .../selftests/bpf/progs/kmem_cache_iter.c | 66 +++++++++++++++++++ 3 files changed, 137 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/kmem_cache_iter.c create mode 100644 tools/testing/selftests/bpf/progs/kmem_cache_iter.c diff --git a/tools/testing/selftests/bpf/prog_tests/kmem_cache_iter.c b/tools/testing/selftests/bpf/prog_tests/kmem_cache_iter.c new file mode 100644 index 0000000000000000..3965e2924ac82d91 --- /dev/null +++ b/tools/testing/selftests/bpf/prog_tests/kmem_cache_iter.c @@ -0,0 +1,64 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2024 Google */ + +#include +#include +#include +#include "kmem_cache_iter.skel.h" + +static void test_kmem_cache_iter_check_task(struct kmem_cache_iter *skel) +{ + LIBBPF_OPTS(bpf_test_run_opts, opts, + .flags = 0, /* run it with the current task */ + ); + int prog_fd = bpf_program__fd(skel->progs.check_task_struct); + + /* get task_struct and check it if's from a slab cache */ + bpf_prog_test_run_opts(prog_fd, &opts); + + /* the BPF program should set 'found' variable */ + ASSERT_EQ(skel->bss->found, 1, "found task_struct"); +} + +void test_kmem_cache_iter(void) +{ + DECLARE_LIBBPF_OPTS(bpf_iter_attach_opts, opts); + struct kmem_cache_iter *skel = NULL; + union bpf_iter_link_info linfo = {}; + struct bpf_link *link; + char buf[1024]; + int iter_fd; + + skel = kmem_cache_iter__open_and_load(); + if (!ASSERT_OK_PTR(skel, "kmem_cache_iter__open_and_load")) + return; + + opts.link_info = &linfo; + opts.link_info_len = sizeof(linfo); + + link = bpf_program__attach_iter(skel->progs.slab_info_collector, &opts); + if (!ASSERT_OK_PTR(link, "attach_iter")) + goto destroy; + + iter_fd = bpf_iter_create(bpf_link__fd(link)); + if (!ASSERT_GE(iter_fd, 0, "iter_create")) + goto free_link; + + memset(buf, 0, sizeof(buf)); + while (read(iter_fd, buf, sizeof(buf) > 0)) { + /* read out all contents */ + printf("%s", buf); + } + + /* next reads should return 0 */ + ASSERT_EQ(read(iter_fd, buf, sizeof(buf)), 0, "read"); + + test_kmem_cache_iter_check_task(skel); + + close(iter_fd); + +free_link: + bpf_link__destroy(link); +destroy: + kmem_cache_iter__destroy(skel); +} diff --git a/tools/testing/selftests/bpf/progs/bpf_iter.h b/tools/testing/selftests/bpf/progs/bpf_iter.h index c41ee80533ca219a..3305dc3a74b32481 100644 --- a/tools/testing/selftests/bpf/progs/bpf_iter.h +++ b/tools/testing/selftests/bpf/progs/bpf_iter.h @@ -24,6 +24,7 @@ #define BTF_F_PTR_RAW BTF_F_PTR_RAW___not_used #define BTF_F_ZERO BTF_F_ZERO___not_used #define bpf_iter__ksym bpf_iter__ksym___not_used +#define bpf_iter__kmem_cache bpf_iter__kmem_cache___not_used #include "vmlinux.h" #undef bpf_iter_meta #undef bpf_iter__bpf_map @@ -48,6 +49,7 @@ #undef BTF_F_PTR_RAW #undef BTF_F_ZERO #undef bpf_iter__ksym +#undef bpf_iter__kmem_cache struct bpf_iter_meta { struct seq_file *seq; @@ -165,3 +167,8 @@ struct bpf_iter__ksym { struct bpf_iter_meta *meta; struct kallsym_iter *ksym; }; + +struct bpf_iter__kmem_cache { + struct bpf_iter_meta *meta; + struct kmem_cache *s; +} __attribute__((preserve_access_index)); diff --git a/tools/testing/selftests/bpf/progs/kmem_cache_iter.c b/tools/testing/selftests/bpf/progs/kmem_cache_iter.c new file mode 100644 index 0000000000000000..3f6ec15a1bf6344c --- /dev/null +++ b/tools/testing/selftests/bpf/progs/kmem_cache_iter.c @@ -0,0 +1,66 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2024 Google */ + +#include "bpf_iter.h" +#include +#include + +char _license[] SEC("license") = "GPL"; + +#define SLAB_NAME_MAX 256 + +struct { + __uint(type, BPF_MAP_TYPE_HASH); + __uint(key_size, sizeof(void *)); + __uint(value_size, SLAB_NAME_MAX); + __uint(max_entries, 1024); +} slab_hash SEC(".maps"); + +extern struct kmem_cache *bpf_get_kmem_cache(__u64 addr) __ksym; + +/* result, will be checked by userspace */ +int found; + +SEC("iter/kmem_cache") +int slab_info_collector(struct bpf_iter__kmem_cache *ctx) +{ + struct seq_file *seq = ctx->meta->seq; + struct kmem_cache *s = ctx->s; + + if (s) { + char name[SLAB_NAME_MAX]; + + /* + * To make sure if the slab_iter implements the seq interface + * properly and it's also useful for debugging. + */ + BPF_SEQ_PRINTF(seq, "%s: %u\n", s->name, s->object_size); + + bpf_probe_read_kernel_str(name, sizeof(name), s->name); + bpf_map_update_elem(&slab_hash, &s, name, BPF_NOEXIST); + } + + return 0; +} + +SEC("raw_tp/bpf_test_finish") +int BPF_PROG(check_task_struct) +{ + __u64 curr = bpf_get_current_task(); + struct kmem_cache *s; + char *name; + + s = bpf_get_kmem_cache(curr); + if (s == NULL) { + found = -1; + return 0; + } + + name = bpf_map_lookup_elem(&slab_hash, &s); + if (name && !bpf_strncmp(name, 11, "task_struct")) + found = 1; + else + found = -2; + + return 0; +}