mbox series

[PATCHSET,0/5] perf lock contention: Add cgroup support (v1)

Message ID 20230830230126.260508-1-namhyung@kernel.org (mailing list archive)
Headers show
Series perf lock contention: Add cgroup support (v1) | expand

Message

Namhyung Kim Aug. 30, 2023, 11:01 p.m. UTC
Hello,

The cgroup support comes with two flavors.  One is to aggregate the
result by cgroups and the other is to filter result for the given
cgroups.  For now, it only works in BPF mode.

The first one is -g/--lock-cgroup option to show lock stats by cgroups
like below.  The cgroup names were shortened for brevity:

  $ sudo perf lock con -abg perf bench sched messaging
   contended   total wait     max wait     avg wait   cgroup
  
        1052      3.34 ms     84.71 us      3.17 us   /app-org.gnome.Terminal.slice/vte-spawn-52221fb8-b33f-4a52-b5c3-e35d1e6fc0e0.scope
          13    106.60 us     11.48 us      8.20 us   /session.slice/org.gnome.Shell@x11.service
          12     21.20 us      4.93 us      1.77 us   /
           3     12.10 us      8.80 us      4.03 us   /session-4.scope
           2     10.98 us      7.50 us      5.49 us   /app-gnome-firefox\x2desr-34054.scope
           2      6.04 us      4.88 us      3.02 us   /app-gnome-google\x2dchrome-6442.scope
           1      5.63 us      5.63 us      5.63 us   /app-org.gnome.Terminal.slice/gnome-terminal-server.service
           1      3.51 us      3.51 us      3.51 us   /pipewire.service
           1      2.15 us      2.15 us      2.15 us   /pipewire-pulse.service
           1       742 ns       742 ns       742 ns   /dbus.service

The other is -G/--cgroup-filter option to show lock stats only from the
given cgroups.  It doesn't support cgroup hierarchy and regex matching.

  $ sudo perf lock con -abt -G / perf bench sched messaging
   contended   total wait     max wait     avg wait          pid   comm
  
           2     10.58 us      8.39 us      5.29 us       257552   kworker/4:1
           2      9.76 us      7.96 us      4.88 us            0   swapper
           4      5.36 us      2.09 us      1.34 us       255462   kworker/0:2
           3      3.33 us      1.48 us      1.11 us       257680   kworker/3:1
           2      2.59 us      1.46 us      1.29 us       257478   kworker/2:2
           1      1.50 us      1.50 us      1.50 us           15   rcu_preempt

You can also use these two options together. :)

The two more test cases were added to the existing lock contention test.

The code is available at 'perf/lock-cgroup-v1' branch in the tree below.

  git://git.kernel.org/pub/scm/linux/kernel/git/namhyung/linux-perf.git


Thanks,
Namhyung


Namhyung Kim (5):
  perf tools: Add read_all_cgroups() and __cgroup_find()
  perf lock contention: Prepare to handle cgroups
  perf lock contention: Add -g/--lock-cgroup option
  perf lock contention: Add -G/--cgroup-filter option
  perf test: Improve perf lock contention test

 tools/perf/Documentation/perf-lock.txt        |  8 ++
 tools/perf/builtin-lock.c                     | 99 ++++++++++++++++++-
 tools/perf/tests/shell/lock_contention.sh     | 45 +++++++++
 tools/perf/util/bpf_lock_contention.c         | 51 +++++++++-
 .../perf/util/bpf_skel/lock_contention.bpf.c  | 48 ++++++++-
 tools/perf/util/bpf_skel/lock_data.h          |  3 +-
 tools/perf/util/cgroup.c                      | 63 ++++++++++--
 tools/perf/util/cgroup.h                      |  5 +
 tools/perf/util/lock-contention.h             | 10 +-
 9 files changed, 312 insertions(+), 20 deletions(-)


base-commit: d2045f87154bf67a50ebefe28d2ca0e1e3f8eef1

Comments

Ian Rogers Sept. 4, 2023, 3:28 a.m. UTC | #1
On Wed, Aug 30, 2023 at 4:01 PM Namhyung Kim <namhyung@kernel.org> wrote:
>
> Hello,
>
> The cgroup support comes with two flavors.  One is to aggregate the
> result by cgroups and the other is to filter result for the given
> cgroups.  For now, it only works in BPF mode.
>
> The first one is -g/--lock-cgroup option to show lock stats by cgroups
> like below.  The cgroup names were shortened for brevity:
>
>   $ sudo perf lock con -abg perf bench sched messaging
>    contended   total wait     max wait     avg wait   cgroup
>
>         1052      3.34 ms     84.71 us      3.17 us   /app-org.gnome.Terminal.slice/vte-spawn-52221fb8-b33f-4a52-b5c3-e35d1e6fc0e0.scope
>           13    106.60 us     11.48 us      8.20 us   /session.slice/org.gnome.Shell@x11.service
>           12     21.20 us      4.93 us      1.77 us   /
>            3     12.10 us      8.80 us      4.03 us   /session-4.scope
>            2     10.98 us      7.50 us      5.49 us   /app-gnome-firefox\x2desr-34054.scope
>            2      6.04 us      4.88 us      3.02 us   /app-gnome-google\x2dchrome-6442.scope
>            1      5.63 us      5.63 us      5.63 us   /app-org.gnome.Terminal.slice/gnome-terminal-server.service
>            1      3.51 us      3.51 us      3.51 us   /pipewire.service
>            1      2.15 us      2.15 us      2.15 us   /pipewire-pulse.service
>            1       742 ns       742 ns       742 ns   /dbus.service
>
> The other is -G/--cgroup-filter option to show lock stats only from the
> given cgroups.  It doesn't support cgroup hierarchy and regex matching.
>
>   $ sudo perf lock con -abt -G / perf bench sched messaging
>    contended   total wait     max wait     avg wait          pid   comm
>
>            2     10.58 us      8.39 us      5.29 us       257552   kworker/4:1
>            2      9.76 us      7.96 us      4.88 us            0   swapper
>            4      5.36 us      2.09 us      1.34 us       255462   kworker/0:2
>            3      3.33 us      1.48 us      1.11 us       257680   kworker/3:1
>            2      2.59 us      1.46 us      1.29 us       257478   kworker/2:2
>            1      1.50 us      1.50 us      1.50 us           15   rcu_preempt
>
> You can also use these two options together. :)
>
> The two more test cases were added to the existing lock contention test.
>
> The code is available at 'perf/lock-cgroup-v1' branch in the tree below.
>
>   git://git.kernel.org/pub/scm/linux/kernel/git/namhyung/linux-perf.git
>
>
> Thanks,
> Namhyung
>
>
> Namhyung Kim (5):
>   perf tools: Add read_all_cgroups() and __cgroup_find()
>   perf lock contention: Prepare to handle cgroups
>   perf lock contention: Add -g/--lock-cgroup option
>   perf lock contention: Add -G/--cgroup-filter option
>   perf test: Improve perf lock contention test

Series:
Reviewed-by: Ian Rogers <irogers@google.com>

Thanks,
Ian

>  tools/perf/Documentation/perf-lock.txt        |  8 ++
>  tools/perf/builtin-lock.c                     | 99 ++++++++++++++++++-
>  tools/perf/tests/shell/lock_contention.sh     | 45 +++++++++
>  tools/perf/util/bpf_lock_contention.c         | 51 +++++++++-
>  .../perf/util/bpf_skel/lock_contention.bpf.c  | 48 ++++++++-
>  tools/perf/util/bpf_skel/lock_data.h          |  3 +-
>  tools/perf/util/cgroup.c                      | 63 ++++++++++--
>  tools/perf/util/cgroup.h                      |  5 +
>  tools/perf/util/lock-contention.h             | 10 +-
>  9 files changed, 312 insertions(+), 20 deletions(-)
>
>
> base-commit: d2045f87154bf67a50ebefe28d2ca0e1e3f8eef1
> --
> 2.42.0.283.g2d96d420d3-goog
>