[bpf-next,v3,0/3] libbpf: Resolve unambigous forward declarations

Message

Eduard Zingerman Nov. 6, 2022, 8:29 p.m. UTC

The patch-set is consists of the following parts:
- A refactoring of the libbpf's hashmap interface to use `long`
  instead of `void*` for keys and values. The reasoning behind the
  refactoring is that integer keys / values are used in libbpf more
  often then pointer keys / values. Thus the refactoring reduces the
  number of awkward looking casts like `(void *)(long)off`.
  The `long` is used with an implicit assumption that
  `sizeof(long) == sizeof(void *)` on all supported platforms.
  This includes changes in `libbpf` and `perf` source code. 

- A change to `lib/bpf/btf.c:btf__dedup` that adds a new pass named
  "Resolve unambiguous forward declaration". This pass builds a
  hashmap `name_off -> uniquely named struct or union` and uses it to
  replace FWD types by structs or unions. This is necessary for corner
  cases when FWD is not used as a part of some struct or union
  definition de-duplicated by `btf_dedup_struct_types`.

The goal of the patch-set is to resolve forward declarations that
don't take part in type graphs comparisons if declaration name is
unambiguous.

Example:

CU #1:

struct foo;              // standalone forward declaration
struct foo *some_global;

CU #2:

struct foo { int x; };
struct foo *another_global;

Currently the de-duplicated BTF for this example looks as follows:

[1] STRUCT 'foo' size=4 vlen=1 ...
[2] INT 'int' size=4 ...
[3] PTR '(anon)' type_id=1
[4] FWD 'foo' fwd_kind=struct
[5] PTR '(anon)' type_id=4

The goal of this patch-set is to simplify it as follows:

[1] STRUCT 'foo' size=4 vlen=1
	'x' type_id=2 bits_offset=0
[2] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED
[3] PTR '(anon)' type_id=1

For defconfig kernel with BTF enabled this removes 63 forward
declarations.

For allmodconfig kernel with BTF enabled this removes ~5K out of ~21K
forward declarations in ko objects. This unlocks some additional
de-duplication in ko objects, but impact is tiny: ~13K less BTF ids
out of ~2M.

Changelog:
 v1 -> v2
 - Style fixes in btf_dedup_resolve_fwd and btf_dedup_resolve_fwds as
   suggested by Alan.
 v2 -> v3
 Changes suggested by Andrii:
 - perf's util/hashtable.{c,h} are synchronized with libbpf
   implementation, perf's source code updated accordingly;
 - changes to libbpf, bpf selftests and perf are combined in a single
   patch to simplify bisecting;
 - hashtable interface updated to be long -> long instead of
   uintptr_t -> uintptr_t;
 - btf_dedup_resolve_fwds updated to correctly use IS_ERR / PTR_ERR
   macro;
 - test cases for btf_dedup_resolve_fwds are updated for better
   clarity.

[v1] https://lore.kernel.org/bpf/[e-mail]/
[v2] https://lore.kernel.org/bpf/[e-mail]/

Eduard Zingerman (3):
  libbpf: hashmap interface update to long -> long
  libbpf: Resolve unambigous forward declarations
  selftests/bpf: Tests for btf_dedup_resolve_fwds

 tools/bpf/bpftool/btf.c                       |  25 +--
 tools/bpf/bpftool/common.c                    |  10 +-
 tools/bpf/bpftool/gen.c                       |  19 +-
 tools/bpf/bpftool/link.c                      |   8 +-
 tools/bpf/bpftool/main.h                      |  14 +-
 tools/bpf/bpftool/map.c                       |   8 +-
 tools/bpf/bpftool/pids.c                      |  16 +-
 tools/bpf/bpftool/prog.c                      |   8 +-
 tools/lib/bpf/btf.c                           | 184 +++++++++++++++---
 tools/lib/bpf/btf_dump.c                      |  16 +-
 tools/lib/bpf/hashmap.c                       |  16 +-
 tools/lib/bpf/hashmap.h                       |  34 ++--
 tools/lib/bpf/libbpf.c                        |  18 +-
 tools/lib/bpf/strset.c                        |  18 +-
 tools/lib/bpf/usdt.c                          |  31 ++-
 tools/perf/tests/expr.c                       |  40 ++--
 tools/perf/tests/pmu-events.c                 |   6 +-
 tools/perf/util/bpf-loader.c                  |  23 ++-
 tools/perf/util/expr.c                        |  32 ++-
 tools/perf/util/hashmap.c                     |  16 +-
 tools/perf/util/hashmap.h                     |  34 ++--
 tools/perf/util/metricgroup.c                 |  12 +-
 tools/perf/util/stat.c                        |   9 +-
 tools/testing/selftests/bpf/prog_tests/btf.c  | 176 +++++++++++++++++
 .../bpf/prog_tests/btf_dedup_split.c          |  45 +++--
 .../selftests/bpf/prog_tests/hashmap.c        | 102 +++++-----
 .../bpf/prog_tests/kprobe_multi_test.c        |   6 +-
 27 files changed, 602 insertions(+), 324 deletions(-)