[bpf-next,v1,0/8] Introduce load-acquire and store-release BPF instructions

Message

Peilin Ye Jan. 25, 2025, 2:16 a.m. UTC

Hi all!

This patchset adds kernel support for BPF load-acquire and store-release
instructions (for background, please see [1]), including core/verifier,
arm64 JIT compiler, and Documentation/ changes.  x86-64 and riscv64 are
also planned to be supported.  The corresponding LLVM changes can be
found at:
  https://github.com/llvm/llvm-project/pull/108636

Following RFC 9669, 7.3. Adding Instructions [2], this patchset adds two
conformance groups for the new instructions:

  * atomic32v2: includes "atomic32", plus the new 8-, 16- and 32-bit
                atomic load-acquire and store-release instructions

  * atomic64v2: includes "atomic64" and "atomic32v2", plus the new
                64-bit atomic load-acquire and store-release
                instructions

See patch 8/8 for details; please suggest if you believe the new
instructions should be grouped differently.

RFC v1: https://lore.kernel.org/all/[e-mail]

This patchset has been reorganized based on comments and suggestions
from Xu and Eduard.  Notable changes since RFC v1:

  o 1-2/8: minor verifier.c refactoring patches
  o   3/8: core/verifier changes
         * (Eduard) handle load-acquire properly in backtrack_insn()
         * (Eduard) avoid skipping checks (e.g.,
                    bpf_jit_supports_insn()) for load-acquires
         * track the value stored by store-releases, just like how
           non-atomic STX instructions are handled [3]
         * (Eduard) add missing link in commit message
         * (Eduard) always print 'r' for disasm.c changes
  o   4/8: arm64/insn: avoid treating load_acq/store_rel as
           load_ex/store_ex
  o   5/8: arm64/insn: add load_acq/store_rel
         * (Xu) include Should-Be-One (SBO) bits in "mask" and "value",
                to avoid setting fixed bits during runtime (JIT-compile
                time)
  o   6/8: arm64 JIT compiler changes
         * (Xu) use emit_a64_add_i() for "pointer + offset" to optimize
                code emission
  o   7/8: selftests
         * (Eduard) avoid adding new tests to the 'test_verifier' runner
         * add more tests, e.g., checking mark_precise logic
  o   8/8: instruction-set.rst changes

Please refer to individual kernel patches (and LLVM commits) for
details.  Any feedback would be much appreciated!

[1] https://lore.kernel.org/all/[e-mail]/
[2] https://www.rfc-editor.org/rfc/rfc9669.html#section-7.3
[3] Specifically, for store-releases, make sure we do that
    check_mem_access(..., BPF_WRITE, ...) call with @value_regno equals
    'src_reg' instead of -1.

Thanks,
Peilin Ye (8):
  bpf/verifier: Factor out atomic_ptr_type_ok()
  bpf/verifier: Factor out check_atomic_rmw()
  bpf: Introduce load-acquire and store-release instructions
  arm64: insn: Add BIT(23) to {load,store}_ex's mask
  arm64: insn: Add load-acquire and store-release instructions
  bpf, arm64: Support load-acquire and store-release instructions
  selftests/bpf: Add selftests for load-acquire and store-release
    instructions
  bpf, docs: Update instruction-set.rst for load-acquire and
    store-release instructions

 .../bpf/standardization/instruction-set.rst   | 114 ++++++++++--
 arch/arm64/include/asm/insn.h                 |  12 +-
 arch/arm64/lib/insn.c                         |  28 +++
 arch/arm64/net/bpf_jit.h                      |  20 +++
 arch/arm64/net/bpf_jit_comp.c                 |  92 +++++++++-
 include/linux/filter.h                        |   2 +
 include/uapi/linux/bpf.h                      |  13 ++
 kernel/bpf/core.c                             |  41 ++++-
 kernel/bpf/disasm.c                           |  12 ++
 kernel/bpf/verifier.c                         | 165 +++++++++++++++---
 tools/include/uapi/linux/bpf.h                |  13 ++
 .../selftests/bpf/prog_tests/arena_atomics.c  |  61 ++++++-
 .../selftests/bpf/prog_tests/atomics.c        |  57 +++++-
 .../selftests/bpf/prog_tests/verifier.c       |   4 +
 .../selftests/bpf/progs/arena_atomics.c       |  62 ++++++-
 tools/testing/selftests/bpf/progs/atomics.c   |  62 ++++++-
 .../bpf/progs/verifier_load_acquire.c         |  92 ++++++++++
 .../selftests/bpf/progs/verifier_precision.c  |  39 +++++
 .../bpf/progs/verifier_store_release.c        | 153 ++++++++++++++++
 19 files changed, 988 insertions(+), 54 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/progs/verifier_load_acquire.c
 create mode 100644 tools/testing/selftests/bpf/progs/verifier_store_release.c