From patchwork Fri Dec 22 12:21:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Philo Lu X-Patchwork-Id: 13503294 X-Patchwork-Delegate: bpf@iogearbox.net Received: from out30-133.freemail.mail.aliyun.com (out30-133.freemail.mail.aliyun.com [115.124.30.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7077218636; Fri, 22 Dec 2023 12:21:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.alibaba.com X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R131e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018046051;MF=lulie@linux.alibaba.com;NM=1;PH=DS;RN=22;SR=0;TI=SMTPD_---0Vz-sd5I_1703247708; Received: from localhost(mailfrom:lulie@linux.alibaba.com fp:SMTPD_---0Vz-sd5I_1703247708) by smtp.aliyun-inc.com; Fri, 22 Dec 2023 20:21:49 +0800 From: Philo Lu To: bpf@vger.kernel.org Cc: ast@kernel.org, daniel@iogearbox.net, john.fastabend@gmail.com, andrii@kernel.org, martin.lau@linux.dev, song@kernel.org, yonghong.song@linux.dev, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, jolsa@kernel.org, rostedt@goodmis.org, mhiramat@kernel.org, mathieu.desnoyers@efficios.com, linux-trace-kernel@vger.kernel.org, xuanzhuo@linux.alibaba.com, dust.li@linux.alibaba.com, alibuda@linux.alibaba.com, guwen@linux.alibaba.com, hengqi@linux.alibaba.com, shung-hsi.yu@suse.com Subject: [PATCH bpf-next 1/3] bpf: implement relay map basis Date: Fri, 22 Dec 2023 20:21:44 +0800 Message-Id: <20231222122146.65519-2-lulie@linux.alibaba.com> X-Mailer: git-send-email 2.32.0.3.g01195cf9f In-Reply-To: <20231222122146.65519-1-lulie@linux.alibaba.com> References: <20231222122146.65519-1-lulie@linux.alibaba.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net BPF_MAP_TYPE_RELAY is implemented based on relay interface, which creates per-cpu buffer to transfer data. Each buffer is essentially a list of fix-sized sub-buffers, and is exposed to user space as files in debugfs. attr->max_entries is used as subbuf size and attr->map_extra is used as subbuf num. Currently, the default value of subbuf num is 8. The data can be accessed by read or mmap via these files. For example, if there are 2 cpus, files could be `/sys/kernel/debug/mydir/my_rmap0` and `/sys/kernel/debug/mydir/my_rmap1`. Buffer-only mode is used to create the relay map, which just allocates the buffer without creating user-space files. Then user can setup the files with map_update_elem, thus allowing user to define the directory name in debugfs. map_update_elem is implemented in the following patch. A new map flag named BPF_F_OVERWRITE is introduced to set overwrite mode of relay map. Signed-off-by: Philo Lu --- include/linux/bpf_types.h | 3 + include/uapi/linux/bpf.h | 7 ++ kernel/bpf/Makefile | 3 + kernel/bpf/relaymap.c | 157 ++++++++++++++++++++++++++++++++++++++ kernel/bpf/syscall.c | 1 + 5 files changed, 171 insertions(+) create mode 100644 kernel/bpf/relaymap.c diff --git a/include/linux/bpf_types.h b/include/linux/bpf_types.h index fc0d6f32c687..c122d7b494c5 100644 --- a/include/linux/bpf_types.h +++ b/include/linux/bpf_types.h @@ -132,6 +132,9 @@ BPF_MAP_TYPE(BPF_MAP_TYPE_STRUCT_OPS, bpf_struct_ops_map_ops) BPF_MAP_TYPE(BPF_MAP_TYPE_RINGBUF, ringbuf_map_ops) BPF_MAP_TYPE(BPF_MAP_TYPE_BLOOM_FILTER, bloom_filter_map_ops) BPF_MAP_TYPE(BPF_MAP_TYPE_USER_RINGBUF, user_ringbuf_map_ops) +#ifdef CONFIG_RELAY +BPF_MAP_TYPE(BPF_MAP_TYPE_RELAY, relay_map_ops) +#endif BPF_LINK_TYPE(BPF_LINK_TYPE_RAW_TRACEPOINT, raw_tracepoint) BPF_LINK_TYPE(BPF_LINK_TYPE_TRACING, tracing) diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index 754e68ca8744..143b75676bd3 100644 --- a/include/uapi/linux/bpf.h +++ b/include/uapi/linux/bpf.h @@ -951,6 +951,7 @@ enum bpf_map_type { BPF_MAP_TYPE_BLOOM_FILTER, BPF_MAP_TYPE_USER_RINGBUF, BPF_MAP_TYPE_CGRP_STORAGE, + BPF_MAP_TYPE_RELAY, }; /* Note that tracing related programs such as @@ -1330,6 +1331,9 @@ enum { /* Get path from provided FD in BPF_OBJ_PIN/BPF_OBJ_GET commands */ BPF_F_PATH_FD = (1U << 14), + +/* Enable overwrite for relay map */ + BPF_F_OVERWRITE = (1U << 15), }; /* Flags for BPF_PROG_QUERY. */ @@ -1401,6 +1405,9 @@ union bpf_attr { * BPF_MAP_TYPE_BLOOM_FILTER - the lowest 4 bits indicate the * number of hash functions (if 0, the bloom filter will default * to using 5 hash functions). + * + * BPF_MAP_TYPE_RELAY - the lowest 32 bits indicate the number of + * relay subbufs (if 0, the number will be set to 8 by default). */ __u64 map_extra; }; diff --git a/kernel/bpf/Makefile b/kernel/bpf/Makefile index f526b7573e97..45b35bb0e572 100644 --- a/kernel/bpf/Makefile +++ b/kernel/bpf/Makefile @@ -10,6 +10,9 @@ obj-$(CONFIG_BPF_SYSCALL) += syscall.o verifier.o inode.o helpers.o tnum.o log.o obj-$(CONFIG_BPF_SYSCALL) += bpf_iter.o map_iter.o task_iter.o prog_iter.o link_iter.o obj-$(CONFIG_BPF_SYSCALL) += hashtab.o arraymap.o percpu_freelist.o bpf_lru_list.o lpm_trie.o map_in_map.o bloom_filter.o obj-$(CONFIG_BPF_SYSCALL) += local_storage.o queue_stack_maps.o ringbuf.o +ifeq ($(CONFIG_RELAY),y) +obj-$(CONFIG_BPF_SYSCALL) += relaymap.o +endif obj-$(CONFIG_BPF_SYSCALL) += bpf_local_storage.o bpf_task_storage.o obj-${CONFIG_BPF_LSM} += bpf_inode_storage.o obj-$(CONFIG_BPF_SYSCALL) += disasm.o mprog.o diff --git a/kernel/bpf/relaymap.c b/kernel/bpf/relaymap.c new file mode 100644 index 000000000000..d0adc7f67758 --- /dev/null +++ b/kernel/bpf/relaymap.c @@ -0,0 +1,157 @@ +#include +#include +#include +#include +#include +#include +#include + +#define RELAY_CREATE_FLAG_MASK (BPF_F_OVERWRITE) + +struct bpf_relay_map { + struct bpf_map map; + struct rchan *relay_chan; + struct rchan_callbacks relay_cb; +}; + +static struct dentry *create_buf_file_handler(const char *filename, + struct dentry *parent, umode_t mode, + struct rchan_buf *buf, int *is_global) +{ + /* Because we do relay_late_setup_files(), create_buf_file(NULL, NULL, ...) + * will be called by relay_open. + */ + if (!filename) + return NULL; + + return debugfs_create_file(filename, mode, parent, buf, + &relay_file_operations); +} + +static int remove_buf_file_handler(struct dentry *dentry) +{ + debugfs_remove(dentry); + return 0; +} + +/* For non-overwrite, use default subbuf_start cb */ +static int subbuf_start_overwrite(struct rchan_buf *buf, void *subbuf, + void *prev_subbuf, size_t prev_padding) +{ + return 1; +} + +/* bpf_attr is used as follows: + * - key size: must be 0 + * - value size: value will be used as directory name by map_update_elem + * (to create relay files). If passed as 0, it will be set to NAME_MAX as + * default + * + * - max_entries: subbuf size + * - map_extra: subbuf num, default as 8 + * + * When alloc, we do not set up relay files considering dir_name conflicts. + * Instead we use relay_late_setup_files() in map_update_elem(), and thus the + * value is used as dir_name, and map->name is used as base_filename. + */ +static struct bpf_map *relay_map_alloc(union bpf_attr *attr) +{ + struct bpf_relay_map *rmap; + + if (unlikely(attr->map_flags & ~RELAY_CREATE_FLAG_MASK)) + return ERR_PTR(-EINVAL); + + /* key size must be 0 in relay map */ + if (unlikely(attr->key_size)) + return ERR_PTR(-EINVAL); + + if (unlikely(attr->value_size > NAME_MAX)) { + pr_warn("value_size should be no more than %d\n", NAME_MAX); + return ERR_PTR(-EINVAL); + } else if (attr->value_size == 0) + attr->value_size = NAME_MAX; + + /* set default subbuf num */ + attr->map_extra = attr->map_extra & UINT_MAX; + if (!attr->map_extra) + attr->map_extra = 8; + + if (!attr->map_name || strlen(attr->map_name) == 0) + return ERR_PTR(-EINVAL); + + rmap = bpf_map_area_alloc(sizeof(*rmap), NUMA_NO_NODE); + if (!rmap) + return ERR_PTR(-ENOMEM); + + bpf_map_init_from_attr(&rmap->map, attr); + + rmap->relay_cb.create_buf_file = create_buf_file_handler; + rmap->relay_cb.remove_buf_file = remove_buf_file_handler; + if (attr->map_flags & BPF_F_OVERWRITE) + rmap->relay_cb.subbuf_start = subbuf_start_overwrite; + + rmap->relay_chan = relay_open(NULL, NULL, + attr->max_entries, attr->map_extra, + &rmap->relay_cb, NULL); + if (!rmap->relay_chan) + return ERR_PTR(-EINVAL); + + return &rmap->map; +} + +static void relay_map_free(struct bpf_map *map) +{ + struct bpf_relay_map *rmap; + + rmap = container_of(map, struct bpf_relay_map, map); + relay_close(rmap->relay_chan); + debugfs_remove_recursive(rmap->relay_chan->parent); + kfree(rmap); +} + +static void *relay_map_lookup_elem(struct bpf_map *map, void *key) +{ + return ERR_PTR(-EOPNOTSUPP); +} + +static long relay_map_update_elem(struct bpf_map *map, void *key, void *value, + u64 flags) +{ + return -EOPNOTSUPP; +} + +static long relay_map_delete_elem(struct bpf_map *map, void *key) +{ + return -EOPNOTSUPP; +} + +static int relay_map_get_next_key(struct bpf_map *map, void *key, + void *next_key) +{ + return -EOPNOTSUPP; +} + +static u64 relay_map_mem_usage(const struct bpf_map *map) +{ + struct bpf_relay_map *rmap; + u64 usage = sizeof(struct bpf_relay_map); + + rmap = container_of(map, struct bpf_relay_map, map); + usage += sizeof(struct rchan); + usage += (sizeof(struct rchan_buf) + rmap->relay_chan->alloc_size) + * num_online_cpus(); + return usage; +} + +BTF_ID_LIST_SINGLE(relay_map_btf_ids, struct, bpf_relay_map) +const struct bpf_map_ops relay_map_ops = { + .map_meta_equal = bpf_map_meta_equal, + .map_alloc = relay_map_alloc, + .map_free = relay_map_free, + .map_lookup_elem = relay_map_lookup_elem, + .map_update_elem = relay_map_update_elem, + .map_delete_elem = relay_map_delete_elem, + .map_get_next_key = relay_map_get_next_key, + .map_mem_usage = relay_map_mem_usage, + .map_btf_id = &relay_map_btf_ids[0], +}; diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index 1bf9805ee185..35ae54ac6736 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -1147,6 +1147,7 @@ static int map_create(union bpf_attr *attr) } if (attr->map_type != BPF_MAP_TYPE_BLOOM_FILTER && + attr->map_type != BPF_MAP_TYPE_RELAY && attr->map_extra != 0) return -EINVAL; From patchwork Fri Dec 22 12:21:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Philo Lu X-Patchwork-Id: 13503295 X-Patchwork-Delegate: bpf@iogearbox.net Received: from out30-118.freemail.mail.aliyun.com (out30-118.freemail.mail.aliyun.com [115.124.30.118]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 15A5418631; Fri, 22 Dec 2023 12:21:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.alibaba.com X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R161e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018046060;MF=lulie@linux.alibaba.com;NM=1;PH=DS;RN=22;SR=0;TI=SMTPD_---0Vz-mQOA_1703247709; Received: from localhost(mailfrom:lulie@linux.alibaba.com fp:SMTPD_---0Vz-mQOA_1703247709) by smtp.aliyun-inc.com; Fri, 22 Dec 2023 20:21:51 +0800 From: Philo Lu To: bpf@vger.kernel.org Cc: ast@kernel.org, daniel@iogearbox.net, john.fastabend@gmail.com, andrii@kernel.org, martin.lau@linux.dev, song@kernel.org, yonghong.song@linux.dev, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, jolsa@kernel.org, rostedt@goodmis.org, mhiramat@kernel.org, mathieu.desnoyers@efficios.com, linux-trace-kernel@vger.kernel.org, xuanzhuo@linux.alibaba.com, dust.li@linux.alibaba.com, alibuda@linux.alibaba.com, guwen@linux.alibaba.com, hengqi@linux.alibaba.com, shung-hsi.yu@suse.com Subject: [PATCH bpf-next 2/3] bpf: implement map_update_elem to init relay file Date: Fri, 22 Dec 2023 20:21:45 +0800 Message-Id: <20231222122146.65519-3-lulie@linux.alibaba.com> X-Mailer: git-send-email 2.32.0.3.g01195cf9f In-Reply-To: <20231222122146.65519-1-lulie@linux.alibaba.com> References: <20231222122146.65519-1-lulie@linux.alibaba.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net map_update_elem is used to create relay files and bind them with the relay channel, which is created with BPF_MAP_CREATE. This allows users to set a custom directory name. It must be used with key=NULL and flag=0. Here is an example: ``` struct { __uint(type, BPF_MAP_TYPE_RELAY); __uint(max_entries, 4096); } my_relay SEC(".maps"); ... char dir_name[] = "relay_test"; bpf_map_update_elem(map_fd, NULL, dir_name, 0); ``` Then, directory `/sys/kerenl/debug/relay_test` will be created, which includes files of my_relay0...my_relay[#cpu]. Each represents a per-cpu buffer with size 8 * 4096 B (there are 8 subbufs by default, each with size 4096B). Signed-off-by: Philo Lu --- kernel/bpf/relaymap.c | 32 +++++++++++++++++++++++++++++++- 1 file changed, 31 insertions(+), 1 deletion(-) diff --git a/kernel/bpf/relaymap.c b/kernel/bpf/relaymap.c index d0adc7f67758..588c8de0a4bd 100644 --- a/kernel/bpf/relaymap.c +++ b/kernel/bpf/relaymap.c @@ -117,7 +117,37 @@ static void *relay_map_lookup_elem(struct bpf_map *map, void *key) static long relay_map_update_elem(struct bpf_map *map, void *key, void *value, u64 flags) { - return -EOPNOTSUPP; + struct bpf_relay_map *rmap; + struct dentry *parent; + int err; + + if (unlikely(flags)) + return -EINVAL; + + if (unlikely(key)) + return -EINVAL; + + rmap = container_of(map, struct bpf_relay_map, map); + + /* The directory already exists */ + if (rmap->relay_chan->has_base_filename) + return -EEXIST; + + /* Setup relay files. Note that the directory name passed as value should + * not be longer than map->value_size, including the '\0' at the end. + */ + ((char *)value)[map->value_size - 1] = '\0'; + parent = debugfs_create_dir(value, NULL); + if (IS_ERR_OR_NULL(parent)) + return PTR_ERR(parent); + + err = relay_late_setup_files(rmap->relay_chan, map->name, parent); + if (err) { + debugfs_remove_recursive(parent); + return err; + } + + return 0; } static long relay_map_delete_elem(struct bpf_map *map, void *key) From patchwork Fri Dec 22 12:21:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Philo Lu X-Patchwork-Id: 13503297 X-Patchwork-Delegate: bpf@iogearbox.net Received: from out30-97.freemail.mail.aliyun.com (out30-97.freemail.mail.aliyun.com [115.124.30.97]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CDC70179A1; Fri, 22 Dec 2023 12:22:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.alibaba.com X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R121e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018046049;MF=lulie@linux.alibaba.com;NM=1;PH=DS;RN=22;SR=0;TI=SMTPD_---0Vz-qTK6_1703247711; Received: from localhost(mailfrom:lulie@linux.alibaba.com fp:SMTPD_---0Vz-qTK6_1703247711) by smtp.aliyun-inc.com; Fri, 22 Dec 2023 20:21:52 +0800 From: Philo Lu To: bpf@vger.kernel.org Cc: ast@kernel.org, daniel@iogearbox.net, john.fastabend@gmail.com, andrii@kernel.org, martin.lau@linux.dev, song@kernel.org, yonghong.song@linux.dev, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, jolsa@kernel.org, rostedt@goodmis.org, mhiramat@kernel.org, mathieu.desnoyers@efficios.com, linux-trace-kernel@vger.kernel.org, xuanzhuo@linux.alibaba.com, dust.li@linux.alibaba.com, alibuda@linux.alibaba.com, guwen@linux.alibaba.com, hengqi@linux.alibaba.com, shung-hsi.yu@suse.com Subject: [PATCH bpf-next 3/3] bpf: introduce bpf_relay_output helper Date: Fri, 22 Dec 2023 20:21:46 +0800 Message-Id: <20231222122146.65519-4-lulie@linux.alibaba.com> X-Mailer: git-send-email 2.32.0.3.g01195cf9f In-Reply-To: <20231222122146.65519-1-lulie@linux.alibaba.com> References: <20231222122146.65519-1-lulie@linux.alibaba.com> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net Like perfbuf/ringbuf, a helper is needed to write into the buffer, named bpf_relay_output, whose usage is same as ringbuf. Note that it works only after relay files are set, i.e., after calling map_update_elem for the created relay map. Signed-off-by: Philo Lu --- include/linux/bpf.h | 1 + include/uapi/linux/bpf.h | 10 ++++++++++ kernel/bpf/helpers.c | 4 ++++ kernel/bpf/relaymap.c | 26 ++++++++++++++++++++++++++ kernel/bpf/verifier.c | 8 ++++++++ kernel/trace/bpf_trace.c | 4 ++++ 6 files changed, 53 insertions(+) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index 7671530d6e4e..b177122369e6 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -3095,6 +3095,7 @@ extern const struct bpf_func_proto bpf_get_retval_proto; extern const struct bpf_func_proto bpf_user_ringbuf_drain_proto; extern const struct bpf_func_proto bpf_cgrp_storage_get_proto; extern const struct bpf_func_proto bpf_cgrp_storage_delete_proto; +extern const struct bpf_func_proto bpf_relay_output_proto; const struct bpf_func_proto *tracing_prog_func_proto( enum bpf_func_id func_id, const struct bpf_prog *prog); diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index 143b75676bd3..03c0c1953ba1 100644 --- a/include/uapi/linux/bpf.h +++ b/include/uapi/linux/bpf.h @@ -5686,6 +5686,15 @@ union bpf_attr { * 0 on success. * * **-ENOENT** if the bpf_local_storage cannot be found. + * + * long bpf_relay_output(void *map, void *data, u64 size, u64 flags) + * Description + * Copy *size* bytes from *data* into *map* of type BPF_MAP_TYPE_RELAY. + * Currently, the *flags* must be 0. + * Return + * 0 on success. + * + * **-ENOENT** if the relay base_file in debugfs cannot be found. */ #define ___BPF_FUNC_MAPPER(FN, ctx...) \ FN(unspec, 0, ##ctx) \ @@ -5900,6 +5909,7 @@ union bpf_attr { FN(user_ringbuf_drain, 209, ##ctx) \ FN(cgrp_storage_get, 210, ##ctx) \ FN(cgrp_storage_delete, 211, ##ctx) \ + FN(relay_output, 212, ##ctx) \ /* */ /* backwards-compatibility macros for users of __BPF_FUNC_MAPPER that don't diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c index be72824f32b2..0c26e87ce729 100644 --- a/kernel/bpf/helpers.c +++ b/kernel/bpf/helpers.c @@ -1720,6 +1720,10 @@ bpf_base_func_proto(enum bpf_func_id func_id) return &bpf_ringbuf_discard_proto; case BPF_FUNC_ringbuf_query: return &bpf_ringbuf_query_proto; +#ifdef CONFIG_RELAY + case BPF_FUNC_relay_output: + return &bpf_relay_output_proto; +#endif case BPF_FUNC_strncmp: return &bpf_strncmp_proto; case BPF_FUNC_strtol: diff --git a/kernel/bpf/relaymap.c b/kernel/bpf/relaymap.c index 588c8de0a4bd..f9e2e4a780df 100644 --- a/kernel/bpf/relaymap.c +++ b/kernel/bpf/relaymap.c @@ -173,6 +173,32 @@ static u64 relay_map_mem_usage(const struct bpf_map *map) return usage; } +BPF_CALL_4(bpf_relay_output, struct bpf_map *, map, void *, data, u64, size, + u64, flags) +{ + struct bpf_relay_map *rmap; + + /* not support any flag now */ + if (unlikely(flags)) + return -EINVAL; + + rmap = container_of(map, struct bpf_relay_map, map); + if (!rmap->relay_chan->has_base_filename) + return -ENOENT; + + relay_write(rmap->relay_chan, data, size); + return 0; +} + +const struct bpf_func_proto bpf_relay_output_proto = { + .func = bpf_relay_output, + .ret_type = RET_INTEGER, + .arg1_type = ARG_CONST_MAP_PTR, + .arg2_type = ARG_PTR_TO_MEM | MEM_RDONLY, + .arg3_type = ARG_CONST_SIZE_OR_ZERO, + .arg4_type = ARG_ANYTHING, +}; + BTF_ID_LIST_SINGLE(relay_map_btf_ids, struct, bpf_relay_map) const struct bpf_map_ops relay_map_ops = { .map_meta_equal = bpf_map_meta_equal, diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index f13008d27f35..8c8287d6ae18 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -8800,6 +8800,10 @@ static int check_map_func_compatibility(struct bpf_verifier_env *env, if (func_id != BPF_FUNC_user_ringbuf_drain) goto error; break; + case BPF_MAP_TYPE_RELAY: + if (func_id != BPF_FUNC_relay_output) + goto error; + break; case BPF_MAP_TYPE_STACK_TRACE: if (func_id != BPF_FUNC_get_stackid) goto error; @@ -8932,6 +8936,10 @@ static int check_map_func_compatibility(struct bpf_verifier_env *env, if (map->map_type != BPF_MAP_TYPE_USER_RINGBUF) goto error; break; + case BPF_FUNC_relay_output: + if (map->map_type != BPF_MAP_TYPE_RELAY) + goto error; + break; case BPF_FUNC_get_stackid: if (map->map_type != BPF_MAP_TYPE_STACK_TRACE) goto error; diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c index 7ac6c52b25eb..5b13553c6232 100644 --- a/kernel/trace/bpf_trace.c +++ b/kernel/trace/bpf_trace.c @@ -1594,6 +1594,10 @@ bpf_tracing_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog) return &bpf_ringbuf_discard_proto; case BPF_FUNC_ringbuf_query: return &bpf_ringbuf_query_proto; +#ifdef CONFIG_RELAY + case BPF_FUNC_relay_output: + return &bpf_relay_output_proto; +#endif case BPF_FUNC_jiffies64: return &bpf_jiffies64_proto; case BPF_FUNC_get_task_stack: