mbox series

[v3,bpf-next,0/7] libbpf: add support for kernel module BTF CO-RE relocations

Message ID 20201201034709.2918694-1-andrii@kernel.org (mailing list archive)
Headers show
Series libbpf: add support for kernel module BTF CO-RE relocations | expand

Message

Andrii Nakryiko Dec. 1, 2020, 3:47 a.m. UTC 
Implement libbpf support for performing CO-RE relocations against types in
kernel module BTFs, in addition to existing vmlinux BTF support.

This is a first step towards fully supporting kernel module BTFs. Subsequent
patch sets will expand kernel and libbpf sides to allow using other
BTF-powered capabilities (fentry/fexit, struct_ops, ksym externs, etc). For
CO-RE relocations support, though, no extra kernel changes are necessary.

This patch set also sets up a convenient and fully-controlled custom kernel
module (called "bpf_testmod"), that is a predictable playground for all the
BPF selftests, that rely on module BTFs.

v2->v3:
  - fix subtle uninitialized variable use in BTF ID iteration code;

v1->v2:
  - module_put() inside preempt_disable() region (Alexei);
  - bpf_sidecar -> bpf_testmod rename (Alexei);
  - test_progs more relaxed handling of bpf_testmod;
  - test_progs marks skipped sub-tests properly as SKIP now.


Andrii Nakryiko (7):
  bpf: fix bpf_put_raw_tracepoint()'s use of __module_address()
  libbpf: add internal helper to load BTF data by FD
  libbpf: refactor CO-RE relocs to not assume a single BTF object
  libbpf: add kernel module BTF support for CO-RE relocations
  selftests/bpf: add bpf_testmod kernel module for testing
  selftests/bpf: add support for marking sub-tests as skipped
  selftests/bpf: add CO-RE relocs selftest relying on kernel module BTF

 kernel/trace/bpf_trace.c                      |   8 +-
 tools/lib/bpf/btf.c                           |  61 ++--
 tools/lib/bpf/libbpf.c                        | 345 ++++++++++++++----
 tools/lib/bpf/libbpf_internal.h               |   1 +
 tools/testing/selftests/bpf/.gitignore        |   1 +
 tools/testing/selftests/bpf/Makefile          |  12 +-
 .../selftests/bpf/bpf_testmod/.gitignore      |   6 +
 .../selftests/bpf/bpf_testmod/Makefile        |  20 +
 .../bpf/bpf_testmod/bpf_testmod-events.h      |  36 ++
 .../selftests/bpf/bpf_testmod/bpf_testmod.c   |  51 +++
 .../selftests/bpf/bpf_testmod/bpf_testmod.h   |  14 +
 .../selftests/bpf/prog_tests/core_reloc.c     |  79 +++-
 .../selftests/bpf/progs/core_reloc_types.h    |  17 +
 .../bpf/progs/test_core_reloc_module.c        |  66 ++++
 tools/testing/selftests/bpf/test_progs.c      |  65 +++-
 tools/testing/selftests/bpf/test_progs.h      |   1 +
 16 files changed, 661 insertions(+), 122 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/bpf_testmod/.gitignore
 create mode 100644 tools/testing/selftests/bpf/bpf_testmod/Makefile
 create mode 100644 tools/testing/selftests/bpf/bpf_testmod/bpf_testmod-events.h
 create mode 100644 tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.c
 create mode 100644 tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.h
 create mode 100644 tools/testing/selftests/bpf/progs/test_core_reloc_module.c

Comments

Alexei Starovoitov Dec. 1, 2020, 9:29 p.m. UTC | #1 
On Mon, Nov 30, 2020 at 7:49 PM Andrii Nakryiko <andrii@kernel.org> wrote:
>
> Implement libbpf support for performing CO-RE relocations against types in
> kernel module BTFs, in addition to existing vmlinux BTF support.
>
> This is a first step towards fully supporting kernel module BTFs. Subsequent
> patch sets will expand kernel and libbpf sides to allow using other
> BTF-powered capabilities (fentry/fexit, struct_ops, ksym externs, etc). For
> CO-RE relocations support, though, no extra kernel changes are necessary.
>
> This patch set also sets up a convenient and fully-controlled custom kernel
> module (called "bpf_testmod"), that is a predictable playground for all the
> BPF selftests, that rely on module BTFs.
>
> v2->v3:
>   - fix subtle uninitialized variable use in BTF ID iteration code;

While testing this patch I've hit this:
#111 test_bpffs:OK
[  222.418122] 9pnet_virtio: no channels available for device hostshare0
[  222.688972] sysfs: cannot create duplicate filename '/kernel/btf/bpf_preload'
[  222.689811] CPU: 0 PID: 2485 Comm: modprobe Tainted: G           O
    5.10.0-rc3-00874-gfe521960f79f #3078
[  222.690813] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996),
BIOS 1.11.0-2.el7 04/01/2014
[  222.691681] Call Trace:
[  222.691937]  dump_stack+0x9a/0xcc
[  222.692303]  sysfs_warn_dup.cold+0x17/0x24
[  222.692714]  sysfs_add_file_mode_ns+0x38c/0x560
[  222.693200]  sysfs_create_bin_file+0xf8/0x150
[  222.693635]  ? sysfs_add_file_to_group+0x150/0x150
[  222.694140]  ? __kasan_kmalloc.constprop.0+0xc2/0xd0
[  222.694648]  btf_module_notify+0x6ac/0xd30
[  222.695111]  notifier_call_chain_robust+0xa6/0x1a0
[  222.695658]  blocking_notifier_call_chain_robust+0x61/0x90
[  222.696251]  load_module+0x62da/0x9f40
[  222.696627]  ? rcu_read_lock_sched_held+0x81/0xb0
[  222.697156]  ? module_frob_arch_sections+0x20/0x20
[  222.697632]  ? ima_read_file+0x140/0x140
[  222.698027]  ? security_kernel_post_read_file+0x68/0xb0
[  222.698643]  ? __do_sys_finit_module+0xf7/0x150
[  222.699126]  __do_sys_finit_module+0xf7/0x150
[  222.699588]  ? __ia32_sys_init_module+0xa0/0xa0
[  222.700049]  ? bpf_prog_32007c34f7726d29_bpf_prog1+0x198/0x7ec
[  222.700690]  ? rcu_read_lock_bh_held+0x90/0x90
[  222.701161]  ? __bpf_trace_sys_enter+0x4f/0x60
[  222.701601]  ? syscall_trace_enter.isra.0+0x174/0x230
[  222.702123]  do_syscall_64+0x2d/0x40
[  222.702483]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
[  222.703002] RIP: 0033:0x7fd909d727f9
[  222.703385] Code: 00 f3 c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40
00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24
08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 57 76 2b 00 f7 d8 64 89
01 48
[  222.705258] RSP: 002b:00007ffeecdb8e28 EFLAGS: 00000206 ORIG_RAX:
0000000000000139
[  222.706010] RAX: ffffffffffffffda RBX: 0000000000625380 RCX: 00007fd909d727f9
[  222.706778] RDX: 0000000000000000 RSI: 000000000041a213 RDI: 0000000000000000
[  222.707522] RBP: 000000000041a213 R08: 0000000000000000 R09: 00000000006242a0
[  222.708241] R10: 0000000000000000 R11: 0000000000000206 R12: 0000000000000000
[  222.708932] R13: 0000000000625530 R14: 0000000000040000 R15: 0000000000625380
[  222.709715] failed to register module [bpf_preload] BTF in sysfs: -17
[  222.892778] sysfs: cannot create duplicate filename '/kernel/btf/bpf_testmod'
[  222.893609] CPU: 2 PID: 2493 Comm: test_progs Tainted: G
O      5.10.0-rc3-00874-gfe521960f79f #3078
[  222.894622] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996),
BIOS 1.11.0-2.el7 04/01/2014
[  222.895498] Call Trace:
[  222.895768]  dump_stack+0x9a/0xcc
[  222.896128]  sysfs_warn_dup.cold+0x17/0x24
[  222.896573]  sysfs_add_file_mode_ns+0x38c/0x560
[  222.897047]  sysfs_create_bin_file+0xf8/0x150
[  222.897506]  ? sysfs_add_file_to_group+0x150/0x150
[  222.898004]  ? __kasan_kmalloc.constprop.0+0xc2/0xd0
[  222.898524]  btf_module_notify+0x6ac/0xd30
[  222.898968]  notifier_call_chain_robust+0xa6/0x1a0
[  222.899477]  blocking_notifier_call_chain_robust+0x61/0x90
[  222.900040]  load_module+0x62da/0x9f40
[  222.900434]  ? rcu_read_lock_sched_held+0x81/0xb0
[  222.900951]  ? module_frob_arch_sections+0x20/0x20
[  222.901474]  ? ima_read_file+0x140/0x140
[  222.901881]  ? security_kernel_post_read_file+0x68/0xb0
[  222.902450]  ? __do_sys_finit_module+0xf7/0x150
[  222.902937]  __do_sys_finit_module+0xf7/0x150
[  222.903401]  ? __ia32_sys_init_module+0xa0/0xa0
[  222.903867]  ? bpf_prog_32007c34f7726d29_bpf_prog1+0x198/0x7ec
[  222.904480]  ? rcu_read_lock_bh_held+0x90/0x90
[  222.904935]  ? __bpf_trace_sys_enter+0x4f/0x60
[  222.905401]  ? syscall_trace_enter.isra.0+0x174/0x230
[  222.905925]  do_syscall_64+0x2d/0x40
[  222.906304]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
[  222.906818] RIP: 0033:0x7fd58327e7f9
[  222.907193] Code: 00 f3 c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40
00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24
08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 57 76 2b 00 f7 d8 64 89
01 48
[  222.909049] RSP: 002b:00007ffce1e96fe8 EFLAGS: 00000246 ORIG_RAX:
0000000000000139
[  222.909823] RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007fd58327e7f9
[  222.910541] RDX: 0000000000000000 RSI: 000000000165481f RDI: 0000000000000004
[  222.911260] RBP: 0000000000000004 R08: 00007ffce1e97230 R09: 0000000000000000
[  222.911995] R10: 0000000000002011 R11: 0000000000000246 R12: 0000000000000000
[  222.912710] R13: 00007ffce1e97230 R14: 0000000000000000 R15: 0000000000000000
[  222.913503] failed to register module [bpf_testmod] BTF in sysfs: -17
Andrii Nakryiko Dec. 1, 2020, 9:49 p.m. UTC | #2 
On Tue, Dec 1, 2020 at 1:30 PM Alexei Starovoitov
<alexei.starovoitov@gmail.com> wrote:
>
> On Mon, Nov 30, 2020 at 7:49 PM Andrii Nakryiko <andrii@kernel.org> wrote:
> >
> > Implement libbpf support for performing CO-RE relocations against types in
> > kernel module BTFs, in addition to existing vmlinux BTF support.
> >
> > This is a first step towards fully supporting kernel module BTFs. Subsequent
> > patch sets will expand kernel and libbpf sides to allow using other
> > BTF-powered capabilities (fentry/fexit, struct_ops, ksym externs, etc). For
> > CO-RE relocations support, though, no extra kernel changes are necessary.
> >
> > This patch set also sets up a convenient and fully-controlled custom kernel
> > module (called "bpf_testmod"), that is a predictable playground for all the
> > BPF selftests, that rely on module BTFs.
> >
> > v2->v3:
> >   - fix subtle uninitialized variable use in BTF ID iteration code;
>
> While testing this patch I've hit this:

Right, I ran into that while testing the second patch set
(fexit/fentry one), and fixed in patch "bpf: keep module's
btf_data_size intact after load". But I've mistakenly added it to the
second patch set, not to this one, my bad. I'll move it into this one.
Or maybe I should just combine those two now for easier logistics?

[...]
Alexei Starovoitov Dec. 1, 2020, 10:22 p.m. UTC | #3 
On 12/1/20 1:49 PM, Andrii Nakryiko wrote:
> On Tue, Dec 1, 2020 at 1:30 PM Alexei Starovoitov
> <alexei.starovoitov@gmail.com> wrote:
>>
>> On Mon, Nov 30, 2020 at 7:49 PM Andrii Nakryiko <andrii@kernel.org> wrote:
>>>
>>> Implement libbpf support for performing CO-RE relocations against types in
>>> kernel module BTFs, in addition to existing vmlinux BTF support.
>>>
>>> This is a first step towards fully supporting kernel module BTFs. Subsequent
>>> patch sets will expand kernel and libbpf sides to allow using other
>>> BTF-powered capabilities (fentry/fexit, struct_ops, ksym externs, etc). For
>>> CO-RE relocations support, though, no extra kernel changes are necessary.
>>>
>>> This patch set also sets up a convenient and fully-controlled custom kernel
>>> module (called "bpf_testmod"), that is a predictable playground for all the
>>> BPF selftests, that rely on module BTFs.
>>>
>>> v2->v3:
>>>    - fix subtle uninitialized variable use in BTF ID iteration code;
>>
>> While testing this patch I've hit this:
> 
> Right, I ran into that while testing the second patch set
> (fexit/fentry one), and fixed in patch "bpf: keep module's
> btf_data_size intact after load". But I've mistakenly added it to the
> second patch set, not to this one, my bad. I'll move it into this one.
> Or maybe I should just combine those two now for easier logistics?

whichever way. Combining is fine by me, since I've looked at both sets 
already.