[bpf-next,0/5] bpf: make tracing program support multi-attach

Message

梦龙董 Feb. 20, 2024, 3:51 a.m. UTC

For now, the BPF program of type BPF_PROG_TYPE_TRACING is not allowed to
be attached to multiple hooks, and we have to create a BPF program for
each kernel function, for which we want to trace, even through all the
program have the same (or similar) logic. This can consume extra memory,
and make the program loading slow if we have plenty of kernel function to
trace.

In the commit 4a1e7c0c63e0 ("bpf: Support attaching freplace programs to
multiple attach points"), the freplace BPF program is made to support
attach to multiple attach points. And in this series, we extend it to
fentry/fexit/raw_tp/...

In the 1st patch, we add the support to record index of the accessed
function args of the target for tracing program. Meanwhile, we add the
function btf_check_func_part_match() to compare the accessed function args
of two function prototype. This function will be used in the next commit.

In the 2nd patch, we do some adjust to bpf_tracing_prog_attach() to make
it support multiple attaching.

In the 3rd patch, we allow to set bpf cookie in bpf_link_create() even if
target_btf_id is set, as we are allowed to attach the tracing program to
new target.

In the 4th patch, we introduce the function libbpf_find_kernel_btf_id() to
libbpf to find the btf type id of the kernel function, and this function
will be used in the next commit.

In the 5th patch, we add the testcases for this series.

Menglong Dong (5):
  bpf: tracing: add support to record and check the accessed args
  bpf: tracing: support to attach program to multi hooks
  libbpf: allow to set coookie when target_btf_id is set in
    bpf_link_create
  libbpf: add the function libbpf_find_kernel_btf_id()
  selftests/bpf: add test cases for multiple attach of tracing program

 include/linux/bpf.h                           |   6 +
 include/uapi/linux/bpf.h                      |   1 +
 kernel/bpf/btf.c                              | 121 ++++++++++++++
 kernel/bpf/syscall.c                          | 118 +++++++++++---
 tools/lib/bpf/bpf.c                           |  17 +-
 tools/lib/bpf/libbpf.c                        |  83 ++++++++++
 tools/lib/bpf/libbpf.h                        |   3 +
 tools/lib/bpf/libbpf.map                      |   1 +
 .../selftests/bpf/bpf_testmod/bpf_testmod.c   |  49 ++++++
 .../bpf/prog_tests/tracing_multi_attach.c     | 153 ++++++++++++++++++
 .../selftests/bpf/progs/tracing_multi_test.c  |  66 ++++++++
 11 files changed, 583 insertions(+), 35 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/tracing_multi_attach.c
 create mode 100644 tools/testing/selftests/bpf/progs/tracing_multi_test.c

Comments

Alexei Starovoitov Feb. 21, 2024, 1:24 a.m. UTC | #1

On Mon, Feb 19, 2024 at 7:51 PM Menglong Dong
<[e-mail]> wrote:
>
> For now, the BPF program of type BPF_PROG_TYPE_TRACING is not allowed to
> be attached to multiple hooks, and we have to create a BPF program for
> each kernel function, for which we want to trace, even through all the
> program have the same (or similar) logic. This can consume extra memory,
> and make the program loading slow if we have plenty of kernel function to
> trace.

Should this be combined with multi link ?
(As was recently done for kprobe_multi and uprobe_multi).
Loading fentry prog once and attaching it through many bpf_links
to multiple places is a nice addition,
but we should probably add a multi link right away too.

梦龙董 Feb. 21, 2024, 2:35 a.m. UTC | #2

Hello,

On Wed, Feb 21, 2024 at 9:24 AM Alexei Starovoitov
<[e-mail]> wrote:
>
> On Mon, Feb 19, 2024 at 7:51 PM Menglong Dong
> <[e-mail]> wrote:
> >
> > For now, the BPF program of type BPF_PROG_TYPE_TRACING is not allowed to
> > be attached to multiple hooks, and we have to create a BPF program for
> > each kernel function, for which we want to trace, even through all the
> > program have the same (or similar) logic. This can consume extra memory,
> > and make the program loading slow if we have plenty of kernel function to
> > trace.
>
> Should this be combined with multi link ?
> (As was recently done for kprobe_multi and uprobe_multi).
> Loading fentry prog once and attaching it through many bpf_links
> to multiple places is a nice addition,
> but we should probably add a multi link right away too.

I was planning to implement the multi link for tracing after this
series in another series. I can do it together with this series
if you prefer.

Thanks!
Menglong Dong

梦龙董 Feb. 21, 2024, 2:45 a.m. UTC | #3

On Wed, Feb 21, 2024 at 10:35 AM 梦龙董 <[e-mail]> wrote:
>
> Hello,
>
> On Wed, Feb 21, 2024 at 9:24 AM Alexei Starovoitov
> <[e-mail]> wrote:
> >
> > On Mon, Feb 19, 2024 at 7:51 PM Menglong Dong
> > <[e-mail]> wrote:
> > >
> > > For now, the BPF program of type BPF_PROG_TYPE_TRACING is not allowed to
> > > be attached to multiple hooks, and we have to create a BPF program for
> > > each kernel function, for which we want to trace, even through all the
> > > program have the same (or similar) logic. This can consume extra memory,
> > > and make the program loading slow if we have plenty of kernel function to
> > > trace.
> >
> > Should this be combined with multi link ?
> > (As was recently done for kprobe_multi and uprobe_multi).
> > Loading fentry prog once and attaching it through many bpf_links
> > to multiple places is a nice addition,
> > but we should probably add a multi link right away too.
>
> I was planning to implement the multi link for tracing after this
> series in another series. I can do it together with this series
> if you prefer.
>

Should I introduce the multi link for tracing first, then this series?
(Furthermore, this series seems not necessary.)

> Thanks!
> Menglong Dong

Alexei Starovoitov Feb. 21, 2024, 3:02 a.m. UTC | #4

On Tue, Feb 20, 2024 at 6:45 PM 梦龙董 <[e-mail]> wrote:
>
> On Wed, Feb 21, 2024 at 10:35 AM 梦龙董 <[e-mail]> wrote:
> >
> > Hello,
> >
> > On Wed, Feb 21, 2024 at 9:24 AM Alexei Starovoitov
> > <[e-mail]> wrote:
> > >
> > > On Mon, Feb 19, 2024 at 7:51 PM Menglong Dong
> > > <[e-mail]> wrote:
> > > >
> > > > For now, the BPF program of type BPF_PROG_TYPE_TRACING is not allowed to
> > > > be attached to multiple hooks, and we have to create a BPF program for
> > > > each kernel function, for which we want to trace, even through all the
> > > > program have the same (or similar) logic. This can consume extra memory,
> > > > and make the program loading slow if we have plenty of kernel function to
> > > > trace.
> > >
> > > Should this be combined with multi link ?
> > > (As was recently done for kprobe_multi and uprobe_multi).
> > > Loading fentry prog once and attaching it through many bpf_links
> > > to multiple places is a nice addition,
> > > but we should probably add a multi link right away too.
> >
> > I was planning to implement the multi link for tracing after this
> > series in another series. I can do it together with this series
> > if you prefer.
> >
>
> Should I introduce the multi link for tracing first, then this series?
> (Furthermore, this series seems not necessary.)

What do you mean "not necessary" ?
Don't you want to still check that bpf prog access only N args
and BTF for these args matches across all attach points ?

梦龙董 Feb. 21, 2024, 3:06 a.m. UTC | #5

On Wed, Feb 21, 2024 at 11:02 AM Alexei Starovoitov
<[e-mail]> wrote:
>
> On Tue, Feb 20, 2024 at 6:45 PM 梦龙董 <[e-mail]> wrote:
> >
> > On Wed, Feb 21, 2024 at 10:35 AM 梦龙董 <[e-mail]> wrote:
> > >
> > > Hello,
> > >
> > > On Wed, Feb 21, 2024 at 9:24 AM Alexei Starovoitov
> > > <[e-mail]> wrote:
> > > >
> > > > On Mon, Feb 19, 2024 at 7:51 PM Menglong Dong
> > > > <[e-mail]> wrote:
> > > > >
> > > > > For now, the BPF program of type BPF_PROG_TYPE_TRACING is not allowed to
> > > > > be attached to multiple hooks, and we have to create a BPF program for
> > > > > each kernel function, for which we want to trace, even through all the
> > > > > program have the same (or similar) logic. This can consume extra memory,
> > > > > and make the program loading slow if we have plenty of kernel function to
> > > > > trace.
> > > >
> > > > Should this be combined with multi link ?
> > > > (As was recently done for kprobe_multi and uprobe_multi).
> > > > Loading fentry prog once and attaching it through many bpf_links
> > > > to multiple places is a nice addition,
> > > > but we should probably add a multi link right away too.
> > >
> > > I was planning to implement the multi link for tracing after this
> > > series in another series. I can do it together with this series
> > > if you prefer.
> > >
> >
> > Should I introduce the multi link for tracing first, then this series?
> > (Furthermore, this series seems not necessary.)
>
> What do you mean "not necessary" ?
> Don't you want to still check that bpf prog access only N args
> and BTF for these args matches across all attach points ?

No, I means that if we should keep the

"Loading fentry prog once and attaching it through many bpf_links to
multiple places"

and only keep the multi link.

Both methods need to check the accessed args of the target.

Alexei Starovoitov Feb. 21, 2024, 3:18 a.m. UTC | #6

On Tue, Feb 20, 2024 at 7:06 PM 梦龙董 <[e-mail]> wrote:
>
> On Wed, Feb 21, 2024 at 11:02 AM Alexei Starovoitov
> <[e-mail]> wrote:
> >
> > On Tue, Feb 20, 2024 at 6:45 PM 梦龙董 <[e-mail]> wrote:
> > >
> > > On Wed, Feb 21, 2024 at 10:35 AM 梦龙董 <[e-mail]> wrote:
> > > >
> > > > Hello,
> > > >
> > > > On Wed, Feb 21, 2024 at 9:24 AM Alexei Starovoitov
> > > > <[e-mail]> wrote:
> > > > >
> > > > > On Mon, Feb 19, 2024 at 7:51 PM Menglong Dong
> > > > > <[e-mail]> wrote:
> > > > > >
> > > > > > For now, the BPF program of type BPF_PROG_TYPE_TRACING is not allowed to
> > > > > > be attached to multiple hooks, and we have to create a BPF program for
> > > > > > each kernel function, for which we want to trace, even through all the
> > > > > > program have the same (or similar) logic. This can consume extra memory,
> > > > > > and make the program loading slow if we have plenty of kernel function to
> > > > > > trace.
> > > > >
> > > > > Should this be combined with multi link ?
> > > > > (As was recently done for kprobe_multi and uprobe_multi).
> > > > > Loading fentry prog once and attaching it through many bpf_links
> > > > > to multiple places is a nice addition,
> > > > > but we should probably add a multi link right away too.
> > > >
> > > > I was planning to implement the multi link for tracing after this
> > > > series in another series. I can do it together with this series
> > > > if you prefer.
> > > >
> > >
> > > Should I introduce the multi link for tracing first, then this series?
> > > (Furthermore, this series seems not necessary.)
> >
> > What do you mean "not necessary" ?
> > Don't you want to still check that bpf prog access only N args
> > and BTF for these args matches across all attach points ?
>
> No, I means that if we should keep the
>
> "Loading fentry prog once and attaching it through many bpf_links to
> multiple places"
>
> and only keep the multi link.

I suspect supporting multi link only is better,
since the amount of kernel code to maintain will be less.

梦龙董 Feb. 21, 2024, 3:57 a.m. UTC | #7

On Wed, Feb 21, 2024 at 11:18 AM Alexei Starovoitov
<[e-mail]> wrote:
>
> On Tue, Feb 20, 2024 at 7:06 PM 梦龙董 <[e-mail]> wrote:
> >
> > On Wed, Feb 21, 2024 at 11:02 AM Alexei Starovoitov
> > <[e-mail]> wrote:
> > >
> > > On Tue, Feb 20, 2024 at 6:45 PM 梦龙董 <[e-mail]> wrote:
> > > >
> > > > On Wed, Feb 21, 2024 at 10:35 AM 梦龙董 <[e-mail]> wrote:
> > > > >
> > > > > Hello,
> > > > >
> > > > > On Wed, Feb 21, 2024 at 9:24 AM Alexei Starovoitov
> > > > > <[e-mail]> wrote:
> > > > > >
> > > > > > On Mon, Feb 19, 2024 at 7:51 PM Menglong Dong
> > > > > > <[e-mail]> wrote:
> > > > > > >
> > > > > > > For now, the BPF program of type BPF_PROG_TYPE_TRACING is not allowed to
> > > > > > > be attached to multiple hooks, and we have to create a BPF program for
> > > > > > > each kernel function, for which we want to trace, even through all the
> > > > > > > program have the same (or similar) logic. This can consume extra memory,
> > > > > > > and make the program loading slow if we have plenty of kernel function to
> > > > > > > trace.
> > > > > >
> > > > > > Should this be combined with multi link ?
> > > > > > (As was recently done for kprobe_multi and uprobe_multi).
> > > > > > Loading fentry prog once and attaching it through many bpf_links
> > > > > > to multiple places is a nice addition,
> > > > > > but we should probably add a multi link right away too.
> > > > >
> > > > > I was planning to implement the multi link for tracing after this
> > > > > series in another series. I can do it together with this series
> > > > > if you prefer.
> > > > >
> > > >
> > > > Should I introduce the multi link for tracing first, then this series?
> > > > (Furthermore, this series seems not necessary.)
> > >
> > > What do you mean "not necessary" ?
> > > Don't you want to still check that bpf prog access only N args
> > > and BTF for these args matches across all attach points ?
> >
> > No, I means that if we should keep the
> >
> > "Loading fentry prog once and attaching it through many bpf_links to
> > multiple places"
> >
> > and only keep the multi link.
>
> I suspect supporting multi link only is better,
> since the amount of kernel code to maintain will be less.

Okay!