diff mbox series

[bpf-next,1/2] bpf: Allow ringbuf memory to be used as map key

Message ID 20220912101106.2765921-1-davemarchevsky@fb.com (mailing list archive)
State Superseded
Delegated to: BPF
Headers show
Series [bpf-next,1/2] bpf: Allow ringbuf memory to be used as map key | expand

Checks

Context Check Description
netdev/tree_selection success Clearly marked for bpf-next, async
netdev/fixes_present success Fixes tag not required for -next series
netdev/subject_prefix success Link
netdev/cover_letter success Single patches do not need cover letters
netdev/patch_count success Link
netdev/header_inline success No static functions without inline keyword in header files
netdev/build_32bit success Errors and warnings before: 10 this patch: 10
netdev/cc_maintainers warning 13 maintainers not CCed: shuah@kernel.org john.fastabend@gmail.com linux-kselftest@vger.kernel.org jolsa@kernel.org song@kernel.org yhs@fb.com naveen.n.rao@linux.vnet.ibm.com haoluo@google.com martin.lau@linux.dev mykolal@fb.com memxor@gmail.com kpsingh@kernel.org sdf@google.com
netdev/build_clang success Errors and warnings before: 5 this patch: 5
netdev/module_param success Was 0 now: 0
netdev/verify_signedoff success Signed-off-by tag matches author and committer
netdev/check_selftest success No net selftest shell script
netdev/verify_fixes success No Fixes tag
netdev/build_allmodconfig_warn success Errors and warnings before: 10 this patch: 10
netdev/checkpatch warning WARNING: added, moved or deleted file(s), does MAINTAINERS need updating? WARNING: line length of 88 exceeds 80 columns
netdev/kdoc success Errors and warnings before: 0 this patch: 0
netdev/source_inline success Was 0 now: 0
bpf/vmtest-bpf-next-PR fail PR summary
bpf/vmtest-bpf-next-VM_Test-1 success Logs for build for s390x with gcc
bpf/vmtest-bpf-next-VM_Test-2 success Logs for build for x86_64 with gcc
bpf/vmtest-bpf-next-VM_Test-3 success Logs for build for x86_64 with llvm-16
bpf/vmtest-bpf-next-VM_Test-4 success Logs for llvm-toolchain
bpf/vmtest-bpf-next-VM_Test-5 success Logs for set-matrix
bpf/vmtest-bpf-next-VM_Test-6 success Logs for test_maps on s390x with gcc
bpf/vmtest-bpf-next-VM_Test-7 success Logs for test_maps on x86_64 with gcc
bpf/vmtest-bpf-next-VM_Test-8 success Logs for test_maps on x86_64 with llvm-16
bpf/vmtest-bpf-next-VM_Test-9 fail Logs for test_progs on s390x with gcc
bpf/vmtest-bpf-next-VM_Test-10 success Logs for test_progs on x86_64 with gcc
bpf/vmtest-bpf-next-VM_Test-11 success Logs for test_progs on x86_64 with llvm-16
bpf/vmtest-bpf-next-VM_Test-12 fail Logs for test_progs_no_alu32 on s390x with gcc
bpf/vmtest-bpf-next-VM_Test-13 success Logs for test_progs_no_alu32 on x86_64 with gcc
bpf/vmtest-bpf-next-VM_Test-14 success Logs for test_progs_no_alu32 on x86_64 with llvm-16
bpf/vmtest-bpf-next-VM_Test-15 success Logs for test_verifier on s390x with gcc
bpf/vmtest-bpf-next-VM_Test-16 success Logs for test_verifier on x86_64 with gcc
bpf/vmtest-bpf-next-VM_Test-17 success Logs for test_verifier on x86_64 with llvm-16

Commit Message

Dave Marchevsky Sept. 12, 2022, 10:11 a.m. UTC 
This patch adds support for the following pattern:

  struct some_data *data = bpf_ringbuf_reserve(&ringbuf, sizeof(struct some_data, 0));
  bpf_map_lookup_elem(&another_map, &data->some_field);
  bpf_ringbuf_submit(data);

Currently the verifier does not consider bpf_ringbuf_reserve's
PTR_TO_MEM ret type a valid key input to bpf_map_lookup_elem. Since
PTR_TO_MEM is by definition a valid region of memory, it is safe to use
it as a key for lookups.

Signed-off-by: Dave Marchevsky <davemarchevsky@fb.com>
---
 kernel/bpf/verifier.c                         |  2 +
 tools/testing/selftests/bpf/Makefile          |  8 ++-
 .../selftests/bpf/prog_tests/ringbuf.c        | 10 +++
 .../bpf/progs/test_ringbuf_map_key.c          | 69 +++++++++++++++++++
 4 files changed, 86 insertions(+), 3 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/progs/test_ringbuf_map_key.c

Comments

Yonghong Song Sept. 13, 2022, 10:25 a.m. UTC | #1 
On 9/12/22 11:11 AM, Dave Marchevsky wrote:
> This patch adds support for the following pattern:
> 
>    struct some_data *data = bpf_ringbuf_reserve(&ringbuf, sizeof(struct some_data, 0));

Maybe add:
	if (!data)
		return;
in the example code? Otherwise, the code looks invalid.

>    bpf_map_lookup_elem(&another_map, &data->some_field);
>    bpf_ringbuf_submit(data);
> 
> Currently the verifier does not consider bpf_ringbuf_reserve's
> PTR_TO_MEM ret type a valid key input to bpf_map_lookup_elem. Since
> PTR_TO_MEM is by definition a valid region of memory, it is safe to use
> it as a key for lookups.

bpf_ringbuf_reserve return types also has MEM_ALLOC. Maybe should
mention in the above commit message.

> 
> Signed-off-by: Dave Marchevsky <davemarchevsky@fb.com>
> ---
>   kernel/bpf/verifier.c                         |  2 +
>   tools/testing/selftests/bpf/Makefile          |  8 ++-
>   .../selftests/bpf/prog_tests/ringbuf.c        | 10 +++
>   .../bpf/progs/test_ringbuf_map_key.c          | 69 +++++++++++++++++++
>   4 files changed, 86 insertions(+), 3 deletions(-)
>   create mode 100644 tools/testing/selftests/bpf/progs/test_ringbuf_map_key.c
> 
> diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
> index c259d734f863..d093618aed99 100644
> --- a/kernel/bpf/verifier.c
> +++ b/kernel/bpf/verifier.c
> @@ -5626,6 +5626,8 @@ static const struct bpf_reg_types map_key_value_types = {
>   		PTR_TO_PACKET_META,
>   		PTR_TO_MAP_KEY,
>   		PTR_TO_MAP_VALUE,
> +		PTR_TO_MEM,

PTR_TO_MEM is okay. But bpf_ringbuf_reserve() will trigger
PTR_TO_MEM | MEM_ALLOC. So suggest to add PTR_TO_MEM unless
you find a use case for it.

> +		PTR_TO_MEM | MEM_ALLOC,
>   	},
>   };
>   
> diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
> index 6cd327f1f216..231d9c1364c9 100644
> --- a/tools/testing/selftests/bpf/Makefile
> +++ b/tools/testing/selftests/bpf/Makefile
> @@ -351,9 +351,11 @@ LINKED_SKELS := test_static_linked.skel.h linked_funcs.skel.h		\
>   		test_subskeleton.skel.h test_subskeleton_lib.skel.h	\
>   		test_usdt.skel.h
>   
> -LSKELS := fentry_test.c fexit_test.c fexit_sleep.c \
> -	test_ringbuf.c atomics.c trace_printk.c trace_vprintk.c \
> -	map_ptr_kern.c core_kern.c core_kern_overflow.c
> +LSKELS := fentry_test.c fexit_test.c fexit_sleep.c atomics.c 		\
> +	trace_printk.c trace_vprintk.c map_ptr_kern.c 			\
> +	core_kern.c core_kern_overflow.c test_ringbuf.c			\
> +	test_ringbuf_map_key.c

Maybe put the selftest in a separate patch?

> +
>   # Generate both light skeleton and libbpf skeleton for these
>   LSKELS_EXTRA := test_ksyms_module.c test_ksyms_weak.c kfunc_call_test.c \
>   	kfunc_call_test_subprog.c
> diff --git a/tools/testing/selftests/bpf/prog_tests/ringbuf.c b/tools/testing/selftests/bpf/prog_tests/ringbuf.c
> index 9a80fe8a6427..1cf458d1a179 100644
> --- a/tools/testing/selftests/bpf/prog_tests/ringbuf.c
> +++ b/tools/testing/selftests/bpf/prog_tests/ringbuf.c
> @@ -13,6 +13,7 @@
>   #include <linux/perf_event.h>
>   #include <linux/ring_buffer.h>
>   #include "test_ringbuf.lskel.h"
> +#include "test_ringbuf_map_key.lskel.h"
>   
>   #define EDONE 7777
>   
> @@ -297,3 +298,12 @@ void test_ringbuf(void)
>   	ring_buffer__free(ringbuf);
>   	test_ringbuf_lskel__destroy(skel);
>   }
> +
> +void test_ringbuf_map_key(void)
> +{
> +	struct test_ringbuf_map_key_lskel *skel_map_key;
> +
> +	skel_map_key = test_ringbuf_map_key_lskel__open_and_load();
> +	ASSERT_OK_PTR(skel_map_key, "test_ringbuf_map_key_lskel__open_and_load failed");
> +	test_ringbuf_map_key_lskel__destroy(skel_map_key);
> +}

This adds another *top* test in ringbuf.c.
Should we add test_ringbuf_map_key as a subtest
for test_ringbuf test so in the future more subtests
could be added in the umbrella of test_ringbuf test?

> diff --git a/tools/testing/selftests/bpf/progs/test_ringbuf_map_key.c b/tools/testing/selftests/bpf/progs/test_ringbuf_map_key.c
> new file mode 100644
> index 000000000000..96a791a9762e
> --- /dev/null
> +++ b/tools/testing/selftests/bpf/progs/test_ringbuf_map_key.c
> @@ -0,0 +1,69 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/* Copyright (c) 2022 Meta Platforms, Inc. and affiliates. */
> +
> +#include <linux/bpf.h>
> +#include <bpf/bpf_helpers.h>
> +#include "bpf_misc.h"
> +
> +char _license[] SEC("license") = "GPL";
> +
> +struct sample {
> +	int pid;
> +	int seq;
> +	long value;
> +	char comm[16];
> +};
> +
> +struct {
> +	__uint(type, BPF_MAP_TYPE_RINGBUF);
> +	__uint(max_entries, 4096);
> +} ringbuf SEC(".maps");
> +
> +struct {
> +	__uint(type, BPF_MAP_TYPE_HASH);
> +	__uint(max_entries, 1000);
> +	__type(key, struct sample);
> +	__type(value, int);
> +} hash_map SEC(".maps");
> +
> +/* inputs */
> +int pid = 0;
> +long value = 0;
> +long flags = 0;

I didn't see these values are assigned in user space.
So you can simplify the code by remove 'flags'
and manually inline the 'value'.

> +
> +/* outputs */
> +long total = 0;
> +long dropped = 0;

'total' is not used. 'dropped' is not checked by user
space application, so it can be dropped too.

> +
> +/* inner state */
> +long seq = 0;
> +
> +SEC("fentry/" SYS_PREFIX "sys_getpgid")
> +int test_ringbuf_mem_map_key(void *ctx)
> +{
> +	int cur_pid = bpf_get_current_pid_tgid() >> 32;
> +	struct sample *sample;
> +	int *lookup_val;
> +	int zero = 0;
> +
> +	if (cur_pid != pid)
> +		return 0;
> +
> +	sample = bpf_ringbuf_reserve(&ringbuf, sizeof(*sample), 0);
> +	if (!sample) {
> +		__sync_fetch_and_add(&dropped, 1);
> +		return 0;
> +	}
> +
> +	sample->pid = pid;
> +	bpf_get_current_comm(sample->comm, sizeof(sample->comm));
> +	sample->value = value;
> +	sample->seq = seq++;
> +
> +	/* This prog is never run, successful load w/ below use of sample mem
> +	 * as map key is considered success
> +	 */
> +	lookup_val = (int *)bpf_map_lookup_elem(&hash_map, sample);

Maybe add code for bpf_map_update_elem() as well since that is also
common use case?
This way, you can check the result in user space after 
bpf_map_update_elem().

> +	bpf_ringbuf_submit(sample, 0);
> +	return 0;
> +}
diff mbox series

Patch

diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index c259d734f863..d093618aed99 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -5626,6 +5626,8 @@  static const struct bpf_reg_types map_key_value_types = {
 		PTR_TO_PACKET_META,
 		PTR_TO_MAP_KEY,
 		PTR_TO_MAP_VALUE,
+		PTR_TO_MEM,
+		PTR_TO_MEM | MEM_ALLOC,
 	},
 };
 
diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
index 6cd327f1f216..231d9c1364c9 100644
--- a/tools/testing/selftests/bpf/Makefile
+++ b/tools/testing/selftests/bpf/Makefile
@@ -351,9 +351,11 @@  LINKED_SKELS := test_static_linked.skel.h linked_funcs.skel.h		\
 		test_subskeleton.skel.h test_subskeleton_lib.skel.h	\
 		test_usdt.skel.h
 
-LSKELS := fentry_test.c fexit_test.c fexit_sleep.c \
-	test_ringbuf.c atomics.c trace_printk.c trace_vprintk.c \
-	map_ptr_kern.c core_kern.c core_kern_overflow.c
+LSKELS := fentry_test.c fexit_test.c fexit_sleep.c atomics.c 		\
+	trace_printk.c trace_vprintk.c map_ptr_kern.c 			\
+	core_kern.c core_kern_overflow.c test_ringbuf.c			\
+	test_ringbuf_map_key.c
+
 # Generate both light skeleton and libbpf skeleton for these
 LSKELS_EXTRA := test_ksyms_module.c test_ksyms_weak.c kfunc_call_test.c \
 	kfunc_call_test_subprog.c
diff --git a/tools/testing/selftests/bpf/prog_tests/ringbuf.c b/tools/testing/selftests/bpf/prog_tests/ringbuf.c
index 9a80fe8a6427..1cf458d1a179 100644
--- a/tools/testing/selftests/bpf/prog_tests/ringbuf.c
+++ b/tools/testing/selftests/bpf/prog_tests/ringbuf.c
@@ -13,6 +13,7 @@ 
 #include <linux/perf_event.h>
 #include <linux/ring_buffer.h>
 #include "test_ringbuf.lskel.h"
+#include "test_ringbuf_map_key.lskel.h"
 
 #define EDONE 7777
 
@@ -297,3 +298,12 @@  void test_ringbuf(void)
 	ring_buffer__free(ringbuf);
 	test_ringbuf_lskel__destroy(skel);
 }
+
+void test_ringbuf_map_key(void)
+{
+	struct test_ringbuf_map_key_lskel *skel_map_key;
+
+	skel_map_key = test_ringbuf_map_key_lskel__open_and_load();
+	ASSERT_OK_PTR(skel_map_key, "test_ringbuf_map_key_lskel__open_and_load failed");
+	test_ringbuf_map_key_lskel__destroy(skel_map_key);
+}
diff --git a/tools/testing/selftests/bpf/progs/test_ringbuf_map_key.c b/tools/testing/selftests/bpf/progs/test_ringbuf_map_key.c
new file mode 100644
index 000000000000..96a791a9762e
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/test_ringbuf_map_key.c
@@ -0,0 +1,69 @@ 
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2022 Meta Platforms, Inc. and affiliates. */
+
+#include <linux/bpf.h>
+#include <bpf/bpf_helpers.h>
+#include "bpf_misc.h"
+
+char _license[] SEC("license") = "GPL";
+
+struct sample {
+	int pid;
+	int seq;
+	long value;
+	char comm[16];
+};
+
+struct {
+	__uint(type, BPF_MAP_TYPE_RINGBUF);
+	__uint(max_entries, 4096);
+} ringbuf SEC(".maps");
+
+struct {
+	__uint(type, BPF_MAP_TYPE_HASH);
+	__uint(max_entries, 1000);
+	__type(key, struct sample);
+	__type(value, int);
+} hash_map SEC(".maps");
+
+/* inputs */
+int pid = 0;
+long value = 0;
+long flags = 0;
+
+/* outputs */
+long total = 0;
+long dropped = 0;
+
+/* inner state */
+long seq = 0;
+
+SEC("fentry/" SYS_PREFIX "sys_getpgid")
+int test_ringbuf_mem_map_key(void *ctx)
+{
+	int cur_pid = bpf_get_current_pid_tgid() >> 32;
+	struct sample *sample;
+	int *lookup_val;
+	int zero = 0;
+
+	if (cur_pid != pid)
+		return 0;
+
+	sample = bpf_ringbuf_reserve(&ringbuf, sizeof(*sample), 0);
+	if (!sample) {
+		__sync_fetch_and_add(&dropped, 1);
+		return 0;
+	}
+
+	sample->pid = pid;
+	bpf_get_current_comm(sample->comm, sizeof(sample->comm));
+	sample->value = value;
+	sample->seq = seq++;
+
+	/* This prog is never run, successful load w/ below use of sample mem
+	 * as map key is considered success
+	 */
+	lookup_val = (int *)bpf_map_lookup_elem(&hash_map, sample);
+	bpf_ringbuf_submit(sample, 0);
+	return 0;
+}