mbox series

[bpf-next,v4,00/15] libbpf: type suffixes and autocreate flag for struct_ops maps

Message ID 20240306104529.6453-1-eddyz87@gmail.com (mailing list archive)
Headers show
Series libbpf: type suffixes and autocreate flag for struct_ops maps | expand

Message

Eduard Zingerman March 6, 2024, 10:45 a.m. UTC 
Tweak struct_ops related APIs to allow the following features:
- specify version suffixes for stuct_ops map types;
- share same BPF program between several map definitions with
  different local BTF types, assuming only maps with same
  kernel BTF type would be selected for load;
- toggle autocreate flag for struct_ops maps;
- automatically toggle autoload for struct_ops programs referenced
  from struct_ops maps, depending on autocreate status of the
  corresponding map;
- use SEC("?.struct_ops") and SEC("?.struct_ops.link")
  to define struct_ops maps with autocreate == false after object open.

This would allow loading programs like below:

    SEC("struct_ops/foo") int BPF_PROG(foo) { ... }
    SEC("struct_ops/bar") int BPF_PROG(bar) { ... }

    struct bpf_testmod_ops___v1 {
        int (*foo)(void);
    };

    struct bpf_testmod_ops___v2 {
        int (*foo)(void);
        int (*bar)(void);
    };

    /* Assume kernel type name to be 'test_ops' */
    SEC(".struct_ops.link")
    struct test_ops___v1 map_v1 = {
        /* Program 'foo' shared by maps with
         * different local BTF type
         */
        .foo = (void *)foo
    };

    SEC(".struct_ops.link")
    struct test_ops___v2 map_v2 = {
        .foo = (void *)foo,
        .bar = (void *)bar
    };

Assuming the following tweaks are done before loading:

    /* to load v1 */
    bpf_map__set_autocreate(skel->maps.map_v1, true);
    bpf_map__set_autocreate(skel->maps.map_v2, false);

    /* to load v2 */
    bpf_map__set_autocreate(skel->maps.map_v1, false);
    bpf_map__set_autocreate(skel->maps.map_v2, true);

Patch #8 ties autocreate and autoload flags for struct_ops maps and
programs.

Changelog:
- v3 [3] -> v4:
  - changes for multiple styling suggestions from Andrii;
  - patch #5: libbpf log capture now happens for LIBBPF_INFO and
    LIBBPF_WARN messages and does not depend on verbosity flags
    (Andrii);
  - patch #6: fixed runtime crash caused by conflict with newly added
    test case struct_ops_multi_pages;
  - patch #7: fixed free of possibly uninitialized pointer (Daniel)
  - patch #8: simpler algorithm to detect which programs to autoload
    (Andrii);
  - patch #9: added assertions for autoload flag after object load
    (Andrii);
  - patch #12: DATASEC name rewrite in libbpf is now done inplace, no
    new strings added to BTF (Andrii);
  - patch #14: allow any printable characters in DATASEC names when
    kernel validates BTF (Andrii)
- v2 [2] -> v3:
  - moved patch #8 logic to be fully done on load
    (requested by Andrii in offlist discussion);
  - in patch #9 added test case for shadow vars and
    autocreate/autoload interaction.
- v1 [1] -> v2:
  - fixed memory leak in patch #1 (Kui-Feng);
  - improved error messages in patch #2 (Martin, Andrii);
  - in bad_struct_ops selftest from patch #6 added .test_2
    map member setup (David);
  - added utility functions to capture libbpf log from selftests (David)
  - in selftests replaced usage of ...__open_and_load by separate
    calls to ..._open() and ..._load() (Andrii);
  - removed serial_... in selftest definitions (Andrii);
  - improved comments in selftest struct_ops_autocreate
    from patch #7 (David);
  - removed autoload toggling logic incompatible with shadow variables
    from bpf_map__set_autocreate(), instead struct_ops programs
    autoload property is computed at struct_ops maps load phase,
    see patch #8 (Kui-Feng, Martin, Andrii);
  - added support for SEC("?.struct_ops") and SEC("?.struct_ops.link")
    (Andrii).

[1] https://lore.kernel.org/bpf/20240227204556.17524-1-eddyz87@gmail.com/
[2] https://lore.kernel.org/bpf/20240302011920.15302-1-eddyz87@gmail.com/
[3] https://lore.kernel.org/bpf/20240304225156.24765-1-eddyz87@gmail.com/

Eduard Zingerman (15):
  libbpf: allow version suffixes (___smth) for struct_ops types
  libbpf: tie struct_ops programs to kernel BTF ids, not to local ids
  libbpf: honor autocreate flag for struct_ops maps
  selftests/bpf: test struct_ops map definition with type suffix
  selftests/bpf: utility functions to capture libbpf log in test_progs
  selftests/bpf: bad_struct_ops test
  selftests/bpf: test autocreate behavior for struct_ops maps
  libbpf: sync progs autoload with maps autocreate for struct_ops maps
  selftests/bpf: verify struct_ops autoload/autocreate sync
  libbpf: replace elf_state->st_ops_* fields with SEC_ST_OPS sec_type
  libbpf: struct_ops in SEC("?.struct_ops") / SEC("?.struct_ops.link")
  libbpf: rewrite btf datasec names starting from '?'
  selftests/bpf: test case for SEC("?.struct_ops")
  bpf: allow all printable characters in BTF DATASEC names
  selftests/bpf: test cases for '?' in BTF names

 kernel/bpf/btf.c                              |  16 +-
 tools/lib/bpf/features.c                      |  22 ++
 tools/lib/bpf/libbpf.c                        | 209 +++++++++++++-----
 tools/lib/bpf/libbpf_internal.h               |   2 +
 .../selftests/bpf/bpf_testmod/bpf_testmod.c   |  26 +++
 .../selftests/bpf/bpf_testmod/bpf_testmod.h   |   4 +
 .../selftests/bpf/prog_tests/bad_struct_ops.c |  67 ++++++
 tools/testing/selftests/bpf/prog_tests/btf.c  |  29 +++
 .../bpf/prog_tests/struct_ops_autocreate.c    | 159 +++++++++++++
 .../bpf/prog_tests/test_struct_ops_module.c   |  33 ++-
 .../selftests/bpf/progs/bad_struct_ops.c      |  25 +++
 .../selftests/bpf/progs/bad_struct_ops2.c     |  14 ++
 .../bpf/progs/struct_ops_autocreate.c         |  52 +++++
 .../bpf/progs/struct_ops_autocreate2.c        |  32 +++
 .../selftests/bpf/progs/struct_ops_module.c   |  21 +-
 tools/testing/selftests/bpf/test_progs.c      |  59 +++++
 tools/testing/selftests/bpf/test_progs.h      |   3 +
 17 files changed, 704 insertions(+), 69 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/bad_struct_ops.c
 create mode 100644 tools/testing/selftests/bpf/prog_tests/struct_ops_autocreate.c
 create mode 100644 tools/testing/selftests/bpf/progs/bad_struct_ops.c
 create mode 100644 tools/testing/selftests/bpf/progs/bad_struct_ops2.c
 create mode 100644 tools/testing/selftests/bpf/progs/struct_ops_autocreate.c
 create mode 100644 tools/testing/selftests/bpf/progs/struct_ops_autocreate2.c

Comments

patchwork-bot+netdevbpf@kernel.org March 6, 2024, 7:30 p.m. UTC | #1 
Hello:

This series was applied to bpf/bpf-next.git (master)
by Andrii Nakryiko <andrii@kernel.org>:

On Wed,  6 Mar 2024 12:45:14 +0200 you wrote:
> Tweak struct_ops related APIs to allow the following features:
> - specify version suffixes for stuct_ops map types;
> - share same BPF program between several map definitions with
>   different local BTF types, assuming only maps with same
>   kernel BTF type would be selected for load;
> - toggle autocreate flag for struct_ops maps;
> - automatically toggle autoload for struct_ops programs referenced
>   from struct_ops maps, depending on autocreate status of the
>   corresponding map;
> - use SEC("?.struct_ops") and SEC("?.struct_ops.link")
>   to define struct_ops maps with autocreate == false after object open.
> 
> [...]

Here is the summary with links:
  - [bpf-next,v4,01/15] libbpf: allow version suffixes (___smth) for struct_ops types
    https://git.kernel.org/bpf/bpf-next/c/88972acee1f9
  - [bpf-next,v4,02/15] libbpf: tie struct_ops programs to kernel BTF ids, not to local ids
    https://git.kernel.org/bpf/bpf-next/c/1fcdcca90bf2
  - [bpf-next,v4,03/15] libbpf: honor autocreate flag for struct_ops maps
    https://git.kernel.org/bpf/bpf-next/c/08c5dde48dd0
  - [bpf-next,v4,04/15] selftests/bpf: test struct_ops map definition with type suffix
    https://git.kernel.org/bpf/bpf-next/c/901a9c536f26
  - [bpf-next,v4,05/15] selftests/bpf: utility functions to capture libbpf log in test_progs
    https://git.kernel.org/bpf/bpf-next/c/4597f853d5bc
  - [bpf-next,v4,06/15] selftests/bpf: bad_struct_ops test
    https://git.kernel.org/bpf/bpf-next/c/9c82929ce1bc
  - [bpf-next,v4,07/15] selftests/bpf: test autocreate behavior for struct_ops maps
    https://git.kernel.org/bpf/bpf-next/c/08dfc1369554
  - [bpf-next,v4,08/15] libbpf: sync progs autoload with maps autocreate for struct_ops maps
    https://git.kernel.org/bpf/bpf-next/c/607a50ff2802
  - [bpf-next,v4,09/15] selftests/bpf: verify struct_ops autoload/autocreate sync
    https://git.kernel.org/bpf/bpf-next/c/404855a8c3be
  - [bpf-next,v4,10/15] libbpf: replace elf_state->st_ops_* fields with SEC_ST_OPS sec_type
    https://git.kernel.org/bpf/bpf-next/c/24d9fe302af5
  - [bpf-next,v4,11/15] libbpf: struct_ops in SEC("?.struct_ops") / SEC("?.struct_ops.link")
    https://git.kernel.org/bpf/bpf-next/c/cea716b7850b
  - [bpf-next,v4,12/15] libbpf: rewrite btf datasec names starting from '?'
    https://git.kernel.org/bpf/bpf-next/c/43525abbdb0d
  - [bpf-next,v4,13/15] selftests/bpf: test case for SEC("?.struct_ops")
    https://git.kernel.org/bpf/bpf-next/c/df32d591bf63
  - [bpf-next,v4,14/15] bpf: allow all printable characters in BTF DATASEC names
    https://git.kernel.org/bpf/bpf-next/c/503012aad7f3
  - [bpf-next,v4,15/15] selftests/bpf: test cases for '?' in BTF names
    https://git.kernel.org/bpf/bpf-next/c/bc22f9a13840

You are awesome, thank you!