[bpf-next,v5,0/9] Introduce unstable CT lookup helpers

Message

Kumar Kartikeya Dwivedi Dec. 30, 2021, 2:36 a.m. UTC

This series adds unstable conntrack lookup helpers using BPF kfunc support.  The
patch adding the lookup helper is based off of Maxim's recent patch to aid in
rebasing their series on top of this, all adjusted to work with module kfuncs [0].

  [0]: https://lore.kernel.org/bpf/[e-mail]

To enable returning a reference to struct nf_conn, the verifier is extended to
support reference tracking for PTR_TO_BTF_ID, and kfunc is extended with support
for working as acquire/release functions, similar to existing BPF helpers. kfunc
returning pointer (limited to PTR_TO_BTF_ID in the kernel) can also return a
PTR_TO_BTF_ID_OR_NULL now, typically needed when acquiring a resource can fail.
kfunc can also receive PTR_TO_CTX and PTR_TO_MEM (with some limitations) as
arguments now. There is also support for passing a mem, len pair as argument
to kfunc now. In such cases, passing pointer to unsized type (void) is also
permitted.

Please see individual commits for details.

Changelog:
----------
v4 -> v5:
v4: https://lore.kernel.org/bpf/[e-mail]

 * Move nf_conntrack helpers code to its own separate file (Toke, Pablo)
 * Remove verifier callbacks, put btf_id_sets in struct btf (Alexei)
  * Convert the in-kernel users away from the old API
 * Change len__ prefix convention to __sz suffix (Alexei)
 * Drop parent_ref_obj_id patch (Alexei)

v3 -> v4:
v3: https://lore.kernel.org/bpf/[e-mail]

 * Guard unstable CT helpers with CONFIG_DEBUG_INFO_BTF_MODULES
 * Move addition of prog_test test kfuncs to selftest commit
 * Move negative kfunc tests to test_verifier suite
 * Limit struct nesting depth to 4, which should be enough for now

v2 -> v3:
v2: https://lore.kernel.org/bpf/[e-mail]

 * Fix build error for !CONFIG_BPF_SYSCALL (Patchwork)

RFC v1 -> v2:
v1: https://lore.kernel.org/bpf/[e-mail]

 * Limit PTR_TO_MEM support to pointer to scalar, or struct with scalars (Alexei)
 * Use btf_id_set for checking acquire, release, ret type null (Alexei)
 * Introduce opts struct for CT helpers, move int err parameter to it
 * Add l4proto as parameter to CT helper's opts, remove separate tcp/udp helpers
 * Add support for mem, len argument pair to kfunc
 * Allow void * as pointer type for mem, len argument pair
 * Extend selftests to cover new additions to kfuncs
 * Copy ref_obj_id to PTR_TO_BTF_ID dst_reg on btf_struct_access, test it
 * Fix other misc nits, bugs, and expand commit messages

Kumar Kartikeya Dwivedi (9):
  kernel: Add kallsyms_on_each_symbol variant for single module
  bpf: Prepare kfunc BTF ID sets when parsing kernel BTF
  bpf: Remove check_kfunc_call callback and old kfunc BTF ID API
  bpf: Introduce mem, size argument pair support for kfunc
  bpf: Add reference tracking support to kfunc
  net/netfilter: Add unstable CT lookup helpers for XDP and TC-BPF
  selftests/bpf: Add test for unstable CT lookup API
  selftests/bpf: Add test_verifier support to fixup kfunc call insns
  selftests/bpf: Extend kfunc selftests

 include/linux/bpf.h                           |   8 -
 include/linux/bpf_verifier.h                  |   7 +
 include/linux/btf.h                           |  63 +--
 include/linux/btf_ids.h                       |  20 +-
 include/linux/kallsyms.h                      |  11 +-
 include/linux/module.h                        |  37 +-
 kernel/bpf/btf.c                              | 401 +++++++++++++++---
 kernel/bpf/verifier.c                         | 196 ++++++---
 kernel/kallsyms.c                             |   4 +-
 kernel/livepatch/core.c                       |   2 +-
 kernel/module.c                               |  62 ++-
 net/bpf/test_run.c                            | 121 +++++-
 net/core/filter.c                             |   1 -
 net/core/net_namespace.c                      |   1 +
 net/ipv4/bpf_tcp_ca.c                         |  12 +-
 net/ipv4/tcp_bbr.c                            |  15 +-
 net/ipv4/tcp_cubic.c                          |  15 +-
 net/ipv4/tcp_dctcp.c                          |  15 +-
 net/netfilter/Makefile                        |   5 +
 net/netfilter/nf_conntrack_bpf.c              | 253 +++++++++++
 .../selftests/bpf/bpf_testmod/bpf_testmod.c   |  15 +-
 tools/testing/selftests/bpf/config            |   4 +
 .../testing/selftests/bpf/prog_tests/bpf_nf.c |  48 +++
 .../selftests/bpf/prog_tests/kfunc_call.c     |   6 +
 .../selftests/bpf/progs/kfunc_call_test.c     |  52 ++-
 .../testing/selftests/bpf/progs/test_bpf_nf.c | 105 +++++
 tools/testing/selftests/bpf/test_verifier.c   |  28 ++
 tools/testing/selftests/bpf/verifier/calls.c  |  75 ++++
 28 files changed, 1314 insertions(+), 268 deletions(-)
 create mode 100644 net/netfilter/nf_conntrack_bpf.c
 create mode 100644 tools/testing/selftests/bpf/prog_tests/bpf_nf.c
 create mode 100644 tools/testing/selftests/bpf/progs/test_bpf_nf.c

Comments

Kumar Kartikeya Dwivedi Dec. 30, 2021, 3:04 a.m. UTC | #1

On Thu, Dec 30, 2021 at 08:06:56AM IST, Kumar Kartikeya Dwivedi wrote:
> This series adds unstable conntrack lookup helpers using BPF kfunc support.  The
> patch adding the lookup helper is based off of Maxim's recent patch to aid in
> rebasing their series on top of this, all adjusted to work with module kfuncs [0].
>
>   [0]: https://lore.kernel.org/bpf/[e-mail]
>
> To enable returning a reference to struct nf_conn, the verifier is extended to
> support reference tracking for PTR_TO_BTF_ID, and kfunc is extended with support
> for working as acquire/release functions, similar to existing BPF helpers. kfunc
> returning pointer (limited to PTR_TO_BTF_ID in the kernel) can also return a
> PTR_TO_BTF_ID_OR_NULL now, typically needed when acquiring a resource can fail.
> kfunc can also receive PTR_TO_CTX and PTR_TO_MEM (with some limitations) as
> arguments now. There is also support for passing a mem, len pair as argument
> to kfunc now. In such cases, passing pointer to unsized type (void) is also
> permitted.
>
> Please see individual commits for details.
>
> Changelog:
> ----------

PR [0] and [1] are needed to make BPF CI green for this set.

[0]: https://github.com/kernel-patches/vmtest/pull/53
[1]: https://github.com/libbpf/libbpf/pull/429

> [...]

--
Kartikeya