[bpf-next,v2,0/8] bpf: Add bpf_rcu_read_lock() support

Message

Yonghong Song Nov. 8, 2022, 7:40 a.m. UTC

Currently, without rcu attribute info in BTF, the verifier treats
rcu tagged pointer as a normal pointer. This might be a problem
for sleepable program where rcu_read_lock()/unlock() is not available.
For example, for a sleepable fentry program, if rcu protected memory
access is interleaved with a sleepable helper/kfunc, it is possible
the memory access after the sleepable helper/kfunc might be invalid
since the object might have been freed then. Even without
a sleepable helper/kfunc, without rcu_read_lock() protection,
it is possible that the rcu protected object might be release
in the middle of bpf program execution which may cause incorrect
result.

To prevent above cases, enable btf_type_tag("rcu") attributes,
introduce new bpf_rcu_read_lock/unlock() kfuncs and add verifier support.

In the rest of patch set, Patch 1 enabled btf_type_tag for __rcu
attribute. Patches 2 and 3 are refactoring patches. Patch 4 added new
bpf_rcu_read_lock/unlock() kfuncs. Patch 5 added verifier support
and Patch 6 enabled sleepable program support for cgrp local storage.
Patch 7 added some tests for new helpers and verifier support and
Patch 8 added new test to the deny list for s390x arch.

Changelogs:
  v1 -> v2:
    . use kfunc instead of helper for bpf_rcu_read_lock/unlock.
    . not use MEM_RCU bpf_type_flag, instead use active_rcu_lock
      in reg state to identify rcu ptr's.
    . Add more self tests.
    . add new test to s390x deny list.

Yonghong Song (8):
  compiler_types: Define __rcu as __attribute__((btf_type_tag("rcu")))
  bpf: Refactor btf_struct_access callback interface
  bpf: Abstract out functions to check sleepable helpers
  bpf: Add kfunc bpf_rcu_read_lock/unlock()
  bpf: Add bpf_rcu_read_lock() verifier support
  bpf: Enable sleeptable support for cgrp local storage
  selftests/bpf: Add tests for bpf_rcu_read_lock()
  selftests/bpf: Add rcu_read_lock test to s390x deny list

 include/linux/bpf.h                           |  15 +-
 include/linux/bpf_lsm.h                       |   6 +
 include/linux/bpf_verifier.h                  |   7 +
 include/linux/btf.h                           |   2 +
 include/linux/compiler_types.h                |   3 +-
 include/linux/filter.h                        |   4 +-
 include/linux/trace_events.h                  |   8 +
 kernel/bpf/bpf_lsm.c                          |  20 +-
 kernel/bpf/btf.c                              |  65 +++-
 kernel/bpf/helpers.c                          |  25 +-
 kernel/bpf/verifier.c                         | 111 +++++-
 kernel/trace/bpf_trace.c                      |  22 +-
 net/bpf/bpf_dummy_struct_ops.c                |   6 +-
 net/core/filter.c                             |  20 +-
 net/ipv4/bpf_tcp_ca.c                         |   6 +-
 net/netfilter/nf_conntrack_bpf.c              |   3 +-
 tools/testing/selftests/bpf/DENYLIST.s390x    |   1 +
 .../selftests/bpf/prog_tests/rcu_read_lock.c  | 127 +++++++
 .../selftests/bpf/progs/rcu_read_lock.c       | 353 ++++++++++++++++++
 19 files changed, 733 insertions(+), 71 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/rcu_read_lock.c
 create mode 100644 tools/testing/selftests/bpf/progs/rcu_read_lock.c