mbox series

[bpf-next,00/12] bpf: Add missed stats for kprobes

Message ID 20230828075537.194192-1-jolsa@kernel.org (mailing list archive)
Headers show
Series bpf: Add missed stats for kprobes | expand

Message

Jiri Olsa Aug. 28, 2023, 7:55 a.m. UTC
hi,
at the moment we can't retrieve the number of missed kprobe
executions and subsequent execution of BPF programs.

This patchset adds:
  - counting of missed execution on attach layer for:
    . kprobes attached through perf link (kprobe/ftrace)
    . kprobes attached through kprobe.multi link (fprobe)
  - counting of recursion_misses for BPF kprobe programs
  - counting runtime stats (kernel.bpf_stats_enabled=1) for BPF programs
    executed through bpf_prog_run_array - kprobes, perf events, trace
    syscall probes


It's still technically possible to create kprobe without perf link (using
SET_BPF perf ioctl) in which case we don't have a way to retrieve the kprobe's
'missed' count. However both libbpf and cilium/ebpf libraries use perf link
if it's available, and for old kernels without perf link support we can use
BPF program to retrieve the kprobe missed count.

Also available at:
  https://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git
  bpf/missed_stats

thanks,
jirka


---
Jiri Olsa (12):
      bpf: Move update_prog_stats to syscall object
      bpf: Move bpf_prog_start_time to linux/filter.h
      bpf: Count stats for kprobe_multi programs
      bpf: Add missed value to kprobe_multi link info
      bpf: Add missed value to kprobe perf link info
      bpf: Count missed stats in trace_call_bpf
      bpf: Move bpf_prog_run_array down in the header file
      bpf: Count run stats in bpf_prog_run_array
      bpftool: Display missed count for kprobe_multi link
      bpftool: Display missed count for kprobe perf link
      selftests/bpf: Add test missed counts of perf event link kprobe
      elftests/bpf: Add test recursion stats of perf event link kprobe

 include/linux/bpf.h                                         | 106 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++--------------------------
 include/linux/trace_events.h                                |   6 ++++--
 include/uapi/linux/bpf.h                                    |   2 ++
 kernel/bpf/syscall.c                                        |  36 +++++++++++++++++++++++++------
 kernel/bpf/trampoline.c                                     |  45 +++++----------------------------------
 kernel/trace/bpf_trace.c                                    |  17 ++++++++++++---
 kernel/trace/trace_kprobe.c                                 |   5 ++++-
 tools/bpf/bpftool/link.c                                    |   8 ++++++-
 tools/include/uapi/linux/bpf.h                              |   2 ++
 tools/testing/selftests/bpf/DENYLIST.aarch64                |   1 +
 tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.c       |   5 +++++
 tools/testing/selftests/bpf/bpf_testmod/bpf_testmod_kfunc.h |   2 ++
 tools/testing/selftests/bpf/prog_tests/missed.c             |  97 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 tools/testing/selftests/bpf/progs/missed_kprobe.c           |  30 ++++++++++++++++++++++++++
 tools/testing/selftests/bpf/progs/missed_kprobe_recursion.c |  48 +++++++++++++++++++++++++++++++++++++++++
 15 files changed, 327 insertions(+), 83 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/missed.c
 create mode 100644 tools/testing/selftests/bpf/progs/missed_kprobe.c
 create mode 100644 tools/testing/selftests/bpf/progs/missed_kprobe_recursion.c

Comments

Hou Tao Sept. 5, 2023, 3:30 a.m. UTC | #1
Hi,

On 8/28/2023 3:55 PM, Jiri Olsa wrote:
> hi,
> at the moment we can't retrieve the number of missed kprobe
> executions and subsequent execution of BPF programs.
>
> This patchset adds:
>   - counting of missed execution on attach layer for:
>     . kprobes attached through perf link (kprobe/ftrace)
>     . kprobes attached through kprobe.multi link (fprobe)
>   - counting of recursion_misses for BPF kprobe programs

Because trace_call_bpf() is used for both kprobe and trace-point bpf
program, so I think it is better to add one selftest for missed counter
for trace-point program as well.
>   - counting runtime stats (kernel.bpf_stats_enabled=1) for BPF programs
>     executed through bpf_prog_run_array - kprobes, perf events, trace
>     syscall probes
>
>
> It's still technically possible to create kprobe without perf link (using
> SET_BPF perf ioctl) in which case we don't have a way to retrieve the kprobe's
> 'missed' count. However both libbpf and cilium/ebpf libraries use perf link
> if it's available, and for old kernels without perf link support we can use
> BPF program to retrieve the kprobe missed count.
>
> Also available at:
>   https://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git
>   bpf/missed_stats
>
> thanks,
> jirka
>
>
> ---
> Jiri Olsa (12):
>       bpf: Move update_prog_stats to syscall object
>       bpf: Move bpf_prog_start_time to linux/filter.h
>       bpf: Count stats for kprobe_multi programs
>       bpf: Add missed value to kprobe_multi link info
>       bpf: Add missed value to kprobe perf link info
>       bpf: Count missed stats in trace_call_bpf
>       bpf: Move bpf_prog_run_array down in the header file
>       bpf: Count run stats in bpf_prog_run_array
>       bpftool: Display missed count for kprobe_multi link
>       bpftool: Display missed count for kprobe perf link
>       selftests/bpf: Add test missed counts of perf event link kprobe
>       elftests/bpf: Add test recursion stats of perf event link kprobe
>
>  include/linux/bpf.h                                         | 106 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++--------------------------
>  include/linux/trace_events.h                                |   6 ++++--
>  include/uapi/linux/bpf.h                                    |   2 ++
>  kernel/bpf/syscall.c                                        |  36 +++++++++++++++++++++++++------
>  kernel/bpf/trampoline.c                                     |  45 +++++----------------------------------
>  kernel/trace/bpf_trace.c                                    |  17 ++++++++++++---
>  kernel/trace/trace_kprobe.c                                 |   5 ++++-
>  tools/bpf/bpftool/link.c                                    |   8 ++++++-
>  tools/include/uapi/linux/bpf.h                              |   2 ++
>  tools/testing/selftests/bpf/DENYLIST.aarch64                |   1 +
>  tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.c       |   5 +++++
>  tools/testing/selftests/bpf/bpf_testmod/bpf_testmod_kfunc.h |   2 ++
>  tools/testing/selftests/bpf/prog_tests/missed.c             |  97 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>  tools/testing/selftests/bpf/progs/missed_kprobe.c           |  30 ++++++++++++++++++++++++++
>  tools/testing/selftests/bpf/progs/missed_kprobe_recursion.c |  48 +++++++++++++++++++++++++++++++++++++++++
>  15 files changed, 327 insertions(+), 83 deletions(-)
>  create mode 100644 tools/testing/selftests/bpf/prog_tests/missed.c
>  create mode 100644 tools/testing/selftests/bpf/progs/missed_kprobe.c
>  create mode 100644 tools/testing/selftests/bpf/progs/missed_kprobe_recursion.c
>
> .
Jiri Olsa Sept. 5, 2023, 7:20 a.m. UTC | #2
On Tue, Sep 05, 2023 at 11:30:54AM +0800, Hou Tao wrote:
> Hi,
> 
> On 8/28/2023 3:55 PM, Jiri Olsa wrote:
> > hi,
> > at the moment we can't retrieve the number of missed kprobe
> > executions and subsequent execution of BPF programs.
> >
> > This patchset adds:
> >   - counting of missed execution on attach layer for:
> >     . kprobes attached through perf link (kprobe/ftrace)
> >     . kprobes attached through kprobe.multi link (fprobe)
> >   - counting of recursion_misses for BPF kprobe programs
> 
> Because trace_call_bpf() is used for both kprobe and trace-point bpf
> program, so I think it is better to add one selftest for missed counter
> for trace-point program as well.

ok, will try to add some

thanks,
jirka

> >   - counting runtime stats (kernel.bpf_stats_enabled=1) for BPF programs
> >     executed through bpf_prog_run_array - kprobes, perf events, trace
> >     syscall probes
> >
> >
> > It's still technically possible to create kprobe without perf link (using
> > SET_BPF perf ioctl) in which case we don't have a way to retrieve the kprobe's
> > 'missed' count. However both libbpf and cilium/ebpf libraries use perf link
> > if it's available, and for old kernels without perf link support we can use
> > BPF program to retrieve the kprobe missed count.
> >
> > Also available at:
> >   https://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git
> >   bpf/missed_stats
> >
> > thanks,
> > jirka
> >
> >
> > ---
> > Jiri Olsa (12):
> >       bpf: Move update_prog_stats to syscall object
> >       bpf: Move bpf_prog_start_time to linux/filter.h
> >       bpf: Count stats for kprobe_multi programs
> >       bpf: Add missed value to kprobe_multi link info
> >       bpf: Add missed value to kprobe perf link info
> >       bpf: Count missed stats in trace_call_bpf
> >       bpf: Move bpf_prog_run_array down in the header file
> >       bpf: Count run stats in bpf_prog_run_array
> >       bpftool: Display missed count for kprobe_multi link
> >       bpftool: Display missed count for kprobe perf link
> >       selftests/bpf: Add test missed counts of perf event link kprobe
> >       elftests/bpf: Add test recursion stats of perf event link kprobe
> >
> >  include/linux/bpf.h                                         | 106 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++--------------------------
> >  include/linux/trace_events.h                                |   6 ++++--
> >  include/uapi/linux/bpf.h                                    |   2 ++
> >  kernel/bpf/syscall.c                                        |  36 +++++++++++++++++++++++++------
> >  kernel/bpf/trampoline.c                                     |  45 +++++----------------------------------
> >  kernel/trace/bpf_trace.c                                    |  17 ++++++++++++---
> >  kernel/trace/trace_kprobe.c                                 |   5 ++++-
> >  tools/bpf/bpftool/link.c                                    |   8 ++++++-
> >  tools/include/uapi/linux/bpf.h                              |   2 ++
> >  tools/testing/selftests/bpf/DENYLIST.aarch64                |   1 +
> >  tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.c       |   5 +++++
> >  tools/testing/selftests/bpf/bpf_testmod/bpf_testmod_kfunc.h |   2 ++
> >  tools/testing/selftests/bpf/prog_tests/missed.c             |  97 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> >  tools/testing/selftests/bpf/progs/missed_kprobe.c           |  30 ++++++++++++++++++++++++++
> >  tools/testing/selftests/bpf/progs/missed_kprobe_recursion.c |  48 +++++++++++++++++++++++++++++++++++++++++
> >  15 files changed, 327 insertions(+), 83 deletions(-)
> >  create mode 100644 tools/testing/selftests/bpf/prog_tests/missed.c
> >  create mode 100644 tools/testing/selftests/bpf/progs/missed_kprobe.c
> >  create mode 100644 tools/testing/selftests/bpf/progs/missed_kprobe_recursion.c
> >
> > .
>