| Message ID | 20250106145328.399610-1-charalampos.stylianopoulos@gmail.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | expose number of map entries to userspace | expand |
On 1/6/25 3:53 PM, Charalampos Stylianopoulos wrote: > This patch series provides an easy way for userspace applications to > query the number of entries currently present in a map. > > Currently, the number of entries in a map is accessible only from kernel space > and eBPF programs. A userspace program that wants to track map utilization has to > create and attach an eBPF program solely for that purpose. > > This series makes the number of entries in a map easily accessible, by extending the > main bpf syscall with a new command. The command supports only maps that already > track utilization, namely hash maps, LPM maps and queue/stack maps. An earlier attempt to directly expose it to user space can be found here [0], which eventually led to [1] to only expose it via kfunc for BPF programs in order to avoid extending UAPI. Perhaps instead add a small libbpf helper (e.g. bpf_map__current_entries to complement bpf_map__max_entries) which does all the work to extract that info via [1] underneath? Thanks, Daniel [0] https://lore.kernel.org/bpf/20230531110511.64612-1-aspsk@isovalent.com/ [1] https://lore.kernel.org/bpf/20230705160139.19967-1-aspsk@isovalent.com/ https://lore.kernel.org/bpf/20230719092952.41202-1-aspsk@isovalent.com/ > Charalampos Stylianopoulos (4): > bpf: Add map_num_entries map op > bpf: Add bpf command to get number of map entries > libbpf: Add support for MAP_GET_NUM_ENTRIES command > selftests/bpf: Add tests for bpf_map_get_num_entries > > include/linux/bpf.h | 3 ++ > include/linux/bpf_local_storage.h | 1 + > include/uapi/linux/bpf.h | 17 +++++++++ > kernel/bpf/devmap.c | 14 ++++++++ > kernel/bpf/hashtab.c | 10 ++++++ > kernel/bpf/lpm_trie.c | 8 +++++ > kernel/bpf/queue_stack_maps.c | 11 +++++- > kernel/bpf/syscall.c | 32 +++++++++++++++++ > tools/include/uapi/linux/bpf.h | 17 +++++++++ > tools/lib/bpf/bpf.c | 16 +++++++++ > tools/lib/bpf/bpf.h | 2 ++ > tools/lib/bpf/libbpf.map | 1 + > .../bpf/map_tests/lpm_trie_map_basic_ops.c | 5 +++ > tools/testing/selftests/bpf/test_maps.c | 35 +++++++++++++++++++ > 14 files changed, 171 insertions(+), 1 deletion(-) >
On 25/01/06 05:19PM, Daniel Borkmann wrote: > On 1/6/25 3:53 PM, Charalampos Stylianopoulos wrote: > > This patch series provides an easy way for userspace applications to > > query the number of entries currently present in a map. > > > > Currently, the number of entries in a map is accessible only from kernel space > > and eBPF programs. A userspace program that wants to track map utilization has to > > create and attach an eBPF program solely for that purpose. > > > > This series makes the number of entries in a map easily accessible, by extending the > > main bpf syscall with a new command. The command supports only maps that already > > track utilization, namely hash maps, LPM maps and queue/stack maps. > > An earlier attempt to directly expose it to user space can be found here [0], which > eventually led to [1] to only expose it via kfunc for BPF programs in order to avoid > extending UAPI. Yes, see the bpf_map_sum_elem_count() kfunc in the kernel/bpf/map_iter.c file, and the links posted by Daniel. > Perhaps instead add a small libbpf helper (e.g. bpf_map__current_entries to complement > bpf_map__max_entries) which does all the work to extract that info via [1] underneath? > > Thanks, > Daniel > > [0] https://lore.kernel.org/bpf/20230531110511.64612-1-aspsk@isovalent.com/ > [1] https://lore.kernel.org/bpf/20230705160139.19967-1-aspsk@isovalent.com/ > https://lore.kernel.org/bpf/20230719092952.41202-1-aspsk@isovalent.com/ > > > Charalampos Stylianopoulos (4): > > bpf: Add map_num_entries map op > > bpf: Add bpf command to get number of map entries > > libbpf: Add support for MAP_GET_NUM_ENTRIES command > > selftests/bpf: Add tests for bpf_map_get_num_entries > > > > include/linux/bpf.h | 3 ++ > > include/linux/bpf_local_storage.h | 1 + > > include/uapi/linux/bpf.h | 17 +++++++++ > > kernel/bpf/devmap.c | 14 ++++++++ > > kernel/bpf/hashtab.c | 10 ++++++ > > kernel/bpf/lpm_trie.c | 8 +++++ > > kernel/bpf/queue_stack_maps.c | 11 +++++- > > kernel/bpf/syscall.c | 32 +++++++++++++++++ > > tools/include/uapi/linux/bpf.h | 17 +++++++++ > > tools/lib/bpf/bpf.c | 16 +++++++++ > > tools/lib/bpf/bpf.h | 2 ++ > > tools/lib/bpf/libbpf.map | 1 + > > .../bpf/map_tests/lpm_trie_map_basic_ops.c | 5 +++ > > tools/testing/selftests/bpf/test_maps.c | 35 +++++++++++++++++++ > > 14 files changed, 171 insertions(+), 1 deletion(-) > > >
On 25/01/06 05:19PM, Daniel Borkmann wrote: > On 1/6/25 3:53 PM, Charalampos Stylianopoulos wrote: > > This patch series provides an easy way for userspace applications to > > query the number of entries currently present in a map. > > > > Currently, the number of entries in a map is accessible only from kernel space > > and eBPF programs. A userspace program that wants to track map utilization has to > > create and attach an eBPF program solely for that purpose. > > > > This series makes the number of entries in a map easily accessible, by extending the > > main bpf syscall with a new command. The command supports only maps that already > > track utilization, namely hash maps, LPM maps and queue/stack maps. > > An earlier attempt to directly expose it to user space can be found here [0], which > eventually led to [1] to only expose it via kfunc for BPF programs in order to avoid > extending UAPI. > > Perhaps instead add a small libbpf helper (e.g. bpf_map__current_entries to complement > bpf_map__max_entries) which does all the work to extract that info via [1] underneath? One small thingy here is that bpf_map_sum_elem_count() is only available from the map iterator. Which means that to get the bpf_map_sum_elem_count() for one map only, one have to iterate through the whole set of maps (and filter out all but one). I wanted to follow up my series by either adding the result of calling bpf_map_sum_elem_count() to map_info as u32 or to add possibility to provide a map_fd/map_id when creating an iterator (so that it is only called for one map). But so far I haven't a real use case for getting the number of elements for one map only. > Thanks, > Daniel > > [0] https://lore.kernel.org/bpf/20230531110511.64612-1-aspsk@isovalent.com/ > [1] https://lore.kernel.org/bpf/20230705160139.19967-1-aspsk@isovalent.com/ > https://lore.kernel.org/bpf/20230719092952.41202-1-aspsk@isovalent.com/ > > > Charalampos Stylianopoulos (4): > > bpf: Add map_num_entries map op > > bpf: Add bpf command to get number of map entries > > libbpf: Add support for MAP_GET_NUM_ENTRIES command > > selftests/bpf: Add tests for bpf_map_get_num_entries > > > > include/linux/bpf.h | 3 ++ > > include/linux/bpf_local_storage.h | 1 + > > include/uapi/linux/bpf.h | 17 +++++++++ > > kernel/bpf/devmap.c | 14 ++++++++ > > kernel/bpf/hashtab.c | 10 ++++++ > > kernel/bpf/lpm_trie.c | 8 +++++ > > kernel/bpf/queue_stack_maps.c | 11 +++++- > > kernel/bpf/syscall.c | 32 +++++++++++++++++ > > tools/include/uapi/linux/bpf.h | 17 +++++++++ > > tools/lib/bpf/bpf.c | 16 +++++++++ > > tools/lib/bpf/bpf.h | 2 ++ > > tools/lib/bpf/libbpf.map | 1 + > > .../bpf/map_tests/lpm_trie_map_basic_ops.c | 5 +++ > > tools/testing/selftests/bpf/test_maps.c | 35 +++++++++++++++++++ > > 14 files changed, 171 insertions(+), 1 deletion(-) > > >
(sorry for double posting, this time in plain text) Thanks a lot for the feedback! So, to double check, the suggestion is to only extend the libbpf API with a new helper that does pretty much what get_cur_elements() does in tools/testing/selftests/bpf/map_tests/map_percpu_stats.c ? > On Tue, 7 Jan 2025 at 08:44, Anton Protopopov <aspsk@isovalent.com> wrote: >> >> On 25/01/06 05:19PM, Daniel Borkmann wrote: >> > On 1/6/25 3:53 PM, Charalampos Stylianopoulos wrote: >> > > This patch series provides an easy way for userspace applications to >> > > query the number of entries currently present in a map. >> > > >> > > Currently, the number of entries in a map is accessible only from kernel space >> > > and eBPF programs. A userspace program that wants to track map utilization has to >> > > create and attach an eBPF program solely for that purpose. >> > > >> > > This series makes the number of entries in a map easily accessible, by extending the >> > > main bpf syscall with a new command. The command supports only maps that already >> > > track utilization, namely hash maps, LPM maps and queue/stack maps. >> > >> > An earlier attempt to directly expose it to user space can be found here [0], which >> > eventually led to [1] to only expose it via kfunc for BPF programs in order to avoid >> > extending UAPI. >> > >> > Perhaps instead add a small libbpf helper (e.g. bpf_map__current_entries to complement >> > bpf_map__max_entries) which does all the work to extract that info via [1] underneath? >> >> One small thingy here is that bpf_map_sum_elem_count() is only >> available from the map iterator. Which means that to get the >> bpf_map_sum_elem_count() for one map only, one have to iterate >> through the whole set of maps (and filter out all but one). >> >> I wanted to follow up my series by either adding the result of >> calling bpf_map_sum_elem_count() to map_info as u32 or to add >> possibility to provide a map_fd/map_id when creating an iterator >> (so that it is only called for one map). But so far I haven't >> a real use case for getting the number of elements for one map only. >> >> > Thanks, >> > Daniel >> > >> > [0] https://lore.kernel.org/bpf/20230531110511.64612-1-aspsk@isovalent.com/ >> > [1] https://lore.kernel.org/bpf/20230705160139.19967-1-aspsk@isovalent.com/ >> > https://lore.kernel.org/bpf/20230719092952.41202-1-aspsk@isovalent.com/ >> > >> > > Charalampos Stylianopoulos (4): >> > > bpf: Add map_num_entries map op >> > > bpf: Add bpf command to get number of map entries >> > > libbpf: Add support for MAP_GET_NUM_ENTRIES command >> > > selftests/bpf: Add tests for bpf_map_get_num_entries >> > > >> > > include/linux/bpf.h | 3 ++ >> > > include/linux/bpf_local_storage.h | 1 + >> > > include/uapi/linux/bpf.h | 17 +++++++++ >> > > kernel/bpf/devmap.c | 14 ++++++++ >> > > kernel/bpf/hashtab.c | 10 ++++++ >> > > kernel/bpf/lpm_trie.c | 8 +++++ >> > > kernel/bpf/queue_stack_maps.c | 11 +++++- >> > > kernel/bpf/syscall.c | 32 +++++++++++++++++ >> > > tools/include/uapi/linux/bpf.h | 17 +++++++++ >> > > tools/lib/bpf/bpf.c | 16 +++++++++ >> > > tools/lib/bpf/bpf.h | 2 ++ >> > > tools/lib/bpf/libbpf.map | 1 + >> > > .../bpf/map_tests/lpm_trie_map_basic_ops.c | 5 +++ >> > > tools/testing/selftests/bpf/test_maps.c | 35 +++++++++++++++++++ >> > > 14 files changed, 171 insertions(+), 1 deletion(-) >> > > >> >