| Message ID | 20230825195328.92126-1-yonghong.song@linux.dev (mailing list archive) |
|---|---|
| Headers | show |
| Series | bpf: Add support for local percpu kptr | expand |
On 8/25/23 12:53 PM, Yonghong Song wrote: > Patch set [1] implemented cgroup local storage BPF_MAP_TYPE_CGRP_STORAGE > similar to sk/task/inode local storage and old BPF_MAP_TYPE_CGROUP_STORAGE > map is marked as deprecated since old BPF_MAP_TYPE_CGROUP_STORAGE map can > only work with current cgroup. > > Similarly, the existing BPF_MAP_TYPE_PERCPU_CGROUP_STORAGE map > is a percpu version of BPF_MAP_TYPE_CGROUP_STORAGE and only works > with current cgroup. But there is no replacement which can work > with arbitrary cgroup. > > This patch set solved this problem but adding support for local > percpu kptr. The map value can have a percpu kptr field which holds > a bpf prog allocated percpu data. The below is an example, > > struct percpu_val_t { > ... fields ... > } > > struct map_value_t { > struct percpu_val_t __percpu *percpu_data_ptr; > } > > In the above, 'map_value_t' is the map value type for a > BPF_MAP_TYPE_CGRP_STORAGE map. User can access 'percpu_data_ptr' > and then read/write percpu data. This covers BPF_MAP_TYPE_PERCPU_CGROUP_STORAGE > and more. So BPF_MAP_TYPE_PERCPU_CGROUP_STORAGE map type > is marked as deprecated. > > In additional, local percpu kptr supports the same map type > as other kptrs including hash, lru_hash, array, sk/inode/task/cgrp > local storage. Currently, percpu data structure does not support > non-scalars or special fields (e.g., bpf_spin_lock, bpf_rb_root, etc.). > They can be supported in the future if there exist use cases. > > Please for individual patches for details. > > [1] https://lore.kernel.org/all/20221026042835.672317-1-yhs@fb.com/ > > Changelog: > v1 -> v2: > - does not support special fields in percpu data structure. > - rename __percpu attr to __percpu_kptr attr. > - rename BPF_KPTR_PERCPU_REF to BPF_KPTR_PERCPU. > - better code to handle bpf_{this,per}_cpu_ptr() helpers. > - add more negative tests. > - fix a bpftool related test failure. > > Yonghong Song (13): > bpf: Add support for non-fix-size percpu mem allocation > bpf: Add BPF_KPTR_PERCPU as a field type > bpf: Add alloc/xchg/direct_access support for local percpu kptr > bpf: Add bpf_this_cpu_ptr/bpf_per_cpu_ptr support for allocated percpu > obj > selftests/bpf: Update error message in negative linked_list test > libbpf: Add __percpu_kptr macro definition > selftests/bpf: Add bpf_percpu_obj_{new,drop}() macro in > bpf_experimental.h > selftests/bpf: Add tests for array map with local percpu kptr > bpf: Mark OBJ_RELEASE argument as MEM_RCU when possible > selftests/bpf: Remove unnecessary direct read of local percpu kptr > selftests/bpf: Add tests for cgrp_local_storage with local percpu kptr > selftests/bpf: Add some negative tests > bpf: Mark BPF_MAP_TYPE_PERCPU_CGROUP_STORAGE deprecated > > include/linux/bpf.h | 22 +- > include/linux/bpf_verifier.h | 1 + > include/uapi/linux/bpf.h | 9 +- > kernel/bpf/btf.c | 5 + > kernel/bpf/core.c | 8 +- > kernel/bpf/helpers.c | 16 ++ > kernel/bpf/memalloc.c | 14 +- > kernel/bpf/syscall.c | 4 + > kernel/bpf/verifier.c | 191 +++++++++++++++--- > tools/include/uapi/linux/bpf.h | 9 +- > tools/lib/bpf/bpf_helpers.h | 1 + > .../testing/selftests/bpf/bpf_experimental.h | 31 +++ > .../selftests/bpf/prog_tests/linked_list.c | 4 +- > .../selftests/bpf/prog_tests/percpu_alloc.c | 125 ++++++++++++ > .../selftests/bpf/progs/percpu_alloc_array.c | 183 +++++++++++++++++ > .../progs/percpu_alloc_cgrp_local_storage.c | 105 ++++++++++ > .../selftests/bpf/progs/percpu_alloc_fail.c | 164 +++++++++++++++ > .../selftests/bpf/test_bpftool_synctypes.py | 9 + > 18 files changed, 848 insertions(+), 53 deletions(-) > create mode 100644 tools/testing/selftests/bpf/prog_tests/percpu_alloc.c > create mode 100644 tools/testing/selftests/bpf/progs/percpu_alloc_array.c > create mode 100644 tools/testing/selftests/bpf/progs/percpu_alloc_cgrp_local_storage.c > create mode 100644 tools/testing/selftests/bpf/progs/percpu_alloc_fail.c There is a test failure for libbpf_str/bpf_map_type_str reported by CI, I will fix it in the next revision.
Patch set [1] implemented cgroup local storage BPF_MAP_TYPE_CGRP_STORAGE similar to sk/task/inode local storage and old BPF_MAP_TYPE_CGROUP_STORAGE map is marked as deprecated since old BPF_MAP_TYPE_CGROUP_STORAGE map can only work with current cgroup. Similarly, the existing BPF_MAP_TYPE_PERCPU_CGROUP_STORAGE map is a percpu version of BPF_MAP_TYPE_CGROUP_STORAGE and only works with current cgroup. But there is no replacement which can work with arbitrary cgroup. This patch set solved this problem but adding support for local percpu kptr. The map value can have a percpu kptr field which holds a bpf prog allocated percpu data. The below is an example, struct percpu_val_t { ... fields ... } struct map_value_t { struct percpu_val_t __percpu *percpu_data_ptr; } In the above, 'map_value_t' is the map value type for a BPF_MAP_TYPE_CGRP_STORAGE map. User can access 'percpu_data_ptr' and then read/write percpu data. This covers BPF_MAP_TYPE_PERCPU_CGROUP_STORAGE and more. So BPF_MAP_TYPE_PERCPU_CGROUP_STORAGE map type is marked as deprecated. In additional, local percpu kptr supports the same map type as other kptrs including hash, lru_hash, array, sk/inode/task/cgrp local storage. Currently, percpu data structure does not support non-scalars or special fields (e.g., bpf_spin_lock, bpf_rb_root, etc.). They can be supported in the future if there exist use cases. Please for individual patches for details. [1] https://lore.kernel.org/all/20221026042835.672317-1-yhs@fb.com/ Changelog: v1 -> v2: - does not support special fields in percpu data structure. - rename __percpu attr to __percpu_kptr attr. - rename BPF_KPTR_PERCPU_REF to BPF_KPTR_PERCPU. - better code to handle bpf_{this,per}_cpu_ptr() helpers. - add more negative tests. - fix a bpftool related test failure. Yonghong Song (13): bpf: Add support for non-fix-size percpu mem allocation bpf: Add BPF_KPTR_PERCPU as a field type bpf: Add alloc/xchg/direct_access support for local percpu kptr bpf: Add bpf_this_cpu_ptr/bpf_per_cpu_ptr support for allocated percpu obj selftests/bpf: Update error message in negative linked_list test libbpf: Add __percpu_kptr macro definition selftests/bpf: Add bpf_percpu_obj_{new,drop}() macro in bpf_experimental.h selftests/bpf: Add tests for array map with local percpu kptr bpf: Mark OBJ_RELEASE argument as MEM_RCU when possible selftests/bpf: Remove unnecessary direct read of local percpu kptr selftests/bpf: Add tests for cgrp_local_storage with local percpu kptr selftests/bpf: Add some negative tests bpf: Mark BPF_MAP_TYPE_PERCPU_CGROUP_STORAGE deprecated include/linux/bpf.h | 22 +- include/linux/bpf_verifier.h | 1 + include/uapi/linux/bpf.h | 9 +- kernel/bpf/btf.c | 5 + kernel/bpf/core.c | 8 +- kernel/bpf/helpers.c | 16 ++ kernel/bpf/memalloc.c | 14 +- kernel/bpf/syscall.c | 4 + kernel/bpf/verifier.c | 191 +++++++++++++++--- tools/include/uapi/linux/bpf.h | 9 +- tools/lib/bpf/bpf_helpers.h | 1 + .../testing/selftests/bpf/bpf_experimental.h | 31 +++ .../selftests/bpf/prog_tests/linked_list.c | 4 +- .../selftests/bpf/prog_tests/percpu_alloc.c | 125 ++++++++++++ .../selftests/bpf/progs/percpu_alloc_array.c | 183 +++++++++++++++++ .../progs/percpu_alloc_cgrp_local_storage.c | 105 ++++++++++ .../selftests/bpf/progs/percpu_alloc_fail.c | 164 +++++++++++++++ .../selftests/bpf/test_bpftool_synctypes.py | 9 + 18 files changed, 848 insertions(+), 53 deletions(-) create mode 100644 tools/testing/selftests/bpf/prog_tests/percpu_alloc.c create mode 100644 tools/testing/selftests/bpf/progs/percpu_alloc_array.c create mode 100644 tools/testing/selftests/bpf/progs/percpu_alloc_cgrp_local_storage.c create mode 100644 tools/testing/selftests/bpf/progs/percpu_alloc_fail.c