mbox series

[dwarves,v3,0/4] Parallelize BTF type info generating of pahole

Message ID 20220126040509.1862767-1-kuifeng@fb.com (mailing list archive)
Headers show
Series Parallelize BTF type info generating of pahole | expand

Message

Kui-Feng Lee Jan. 26, 2022, 4:05 a.m. UTC 
Creating an instance of btf for each worker thread allows
steal-function provided by pahole to add type info on multiple threads
without a lock.  The main thread merges the results of worker threads
to the primary instance.

Copying data from per-thread btf instances to the primary instance is
expensive now.  However, there is a patch landed at the bpf-next
repository. [1] With the patch for bpf-next and this patch, they drop
total runtime to 5.4s from 6.0s with "-j4" on my device to generate
BTF for Linux.

V3 includes following changes.

 - Merge types collected by workers threads at thread_exit and
   simplify the code creating btf_encoder.

 - Update libbpf and fix uses of deprecated APIs.

[1] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git/commit/?id=d81283d27266
[v1] https://lore.kernel.org/dwarves/20220120010817.2803482-1-kuifeng@fb.com/
[v2] https://lore.kernel.org/dwarves/20220124191858.1601255-1-kuifeng@fb.com/

Kui-Feng Lee (4):
  dwarf_loader: Receive per-thread data on worker threads.
  dwarf_loader: Prepare and pass per-thread data to worker threads.
  pahole: Use per-thread btf instances to avoid mutex locking.
  libbpf: Update libbpf to a new revision.

 btf_encoder.c  |  25 ++++++----
 btf_encoder.h  |   2 +
 btf_loader.c   |   4 +-
 ctf_loader.c   |   2 +-
 dwarf_loader.c |  58 +++++++++++++++++------
 dwarves.h      |   9 +++-
 lib/bpf        |   2 +-
 pahole.c       | 124 +++++++++++++++++++++++++++++++++++++++++++++----
 pdwtags.c      |   3 +-
 pfunct.c       |   4 +-
 10 files changed, 193 insertions(+), 40 deletions(-)