[bpf-next,0/8] Teach verifier to determine necessary log buffer size

Message

Andrii Nakryiko March 30, 2023, 4:16 a.m. UTC

My imagination is failing me on how to succinctly name this feature and patch
set, but the point here is to perform internal accounting of what should be
the necessary size of user-supplied log buffer such as to fit entire log
contents without truncation, thus avoiding -ENOSPC.

This is a long-standing limitation, which causes users and BPF loader library
writers to guess and pre-size log buffer, often allocating unnecessary extra
memory for this or doing extra program verifications just to size logs better,
ultimately wasting resources. This was requested most recently by Go BPF
library maintainers ([0]). 

Note, this patch set is based on top of not yet landed BPF verifier rotating
mode patch set ([1]), as those changes make it easier to implement this for
both fixed and rotating mode. This patch set is split from [1], as [1] is
pretty much ready to go in, and this one is more centered around UAPI aspects
and would probably require few iterations to finalize the UAPI. Regardless,
getting this out early to get feedback and see if this is useful for users.

Patches #1-#4 are some preliminary clean ups, fixed, improvements. Patch #5
implements internal logic and changes how fixed log mode operates, see that
patch for details. Patch #6 exposes log->len_max through UAPI. Patch #7 wires
this feature in libbpf API. Patch #8 adds selftests for this feature, both for
BPF_LOG_FIXED and default rotating log modes.

  [0] https://lore.kernel.org/bpf/[e-mail]/
  [1] https://patchwork.kernel.org/project/netdevbpf/list/?series=734791&state=*

Andrii Nakryiko (8):
  bpf: ignore verifier log reset in BPF_LOG_KERNEL mode
  bpf: fix missing -EFAULT return on user log buf error in btf_parse()
  bpf: avoid incorrect -EFAULT error in BPF_LOG_KERNEL mode
  bpf: simplify logging related error conditions handling
  bpf: keep track of total log content size in both fixed and rolling
    modes
  bpf: add log_size_actual output field to return log contents size
  libbpf: wire through log_size_actual value returned from kernel
  selftests/bpf: add tests to validate log_size_actual feature

 include/linux/bpf.h                           |  2 +-
 include/linux/bpf_verifier.h                  | 12 +--
 include/linux/btf.h                           |  2 +-
 include/uapi/linux/bpf.h                      | 10 ++
 kernel/bpf/btf.c                              | 38 +++++---
 kernel/bpf/log.c                              | 68 ++++++++++----
 kernel/bpf/syscall.c                          | 16 ++--
 kernel/bpf/verifier.c                         | 12 ++-
 tools/include/uapi/linux/bpf.h                | 12 ++-
 tools/lib/bpf/bpf.c                           |  7 +-
 tools/lib/bpf/bpf.h                           | 11 ++-
 .../selftests/bpf/prog_tests/verifier_log.c   | 92 +++++++++++++++----
 12 files changed, 203 insertions(+), 79 deletions(-)