mbox series

[v3,0/8] tracing/boot: Add new options for tracing specific period

Message ID 159972809455.428528.4737752126800169128.stgit@devnote2 (mailing list archive)
Headers show
Series tracing/boot: Add new options for tracing specific period | expand

Message

Masami Hiramatsu (Google) Sept. 10, 2020, 8:54 a.m. UTC
Hi,

Here is the 3rd version of the series to improve the boot-time tracing to
support kretprobe and tracing_on option. Previous version is here:

 https://lkml.kernel.org/r/159894698993.1478826.2813843560314595660.stgit@devnote2

This version adds uprobe %return suffix support ([5/8]) and the testcases
([8/8]), and update kprobe %suffix support([4/8]) and the uprobe event
document([6/8]).


The combination of tracing_on and kretprobe allows us to trace events
while a specific function call period. For example, the below bootconfig
will make a function callgraph in the pci_proc_init() function at boot
time.

ftrace {
	tracing_on = 0  # off at start
	tracer = function_graph
	event.kprobes {
		start_event {
			probes = "pci_proc_init"
			actions = "traceon"
		}
		end_event {
			probes = "pci_proc_init%return"
			actions = "traceoff"
		}
	}
}

Here is the example output;

# tracer: function_graph
#
# CPU  DURATION                  FUNCTION CALLS
# |     |   |                     |   |   |   |
 0)               |  pci_proc_init() {
 0)               |    proc_mkdir() {
 0)               |      proc_mkdir_data() {
 0)               |        __proc_create() {
 0)               |          _raw_read_lock() {
 0)   0.179 us    |            preempt_count_add();
 0)   0.203 us    |            do_raw_read_lock();
 0)   1.210 us    |          }
 0)               |          __xlate_proc_name() {
 0)   0.449 us    |            pde_subdir_find();
 0)   0.913 us    |          }
 0)               |          _raw_read_unlock() {
 0)   0.169 us    |            do_raw_read_unlock();
 0)   0.175 us    |            preempt_count_sub();
 0)   0.841 us    |          }
 0)               |          kmem_cache_alloc() {
 0)               |            fs_reclaim_acquire() {
 0)   0.154 us    |              __need_fs_reclaim();
 0)   0.240 us    |              fs_reclaim_acquire.part.0();
 0)   0.889 us    |            }
 0)               |            fs_reclaim_release() {
 0)   0.174 us    |              __need_fs_reclaim();
 0)   0.516 us    |            }
 0)   0.157 us    |            should_failslab();
 0)               |            rcu_read_lock_sched_held() {
 0)               |              rcu_read_lock_held_common() {
 0)   0.156 us    |                rcu_is_watching();
 0)   0.158 us    |                rcu_lockdep_current_cpu_online();
 0)   0.735 us    |              }
 0)   1.054 us    |            }
 0)   3.407 us    |          }
 0)   0.168 us    |          __raw_spin_lock_init();
 0)   7.575 us    |        }
 0)               |        proc_register() {
 0)               |          _raw_spin_lock_irqsave() {
 0)   0.187 us    |            preempt_count_add();
...


Thank you,

---

Masami Hiramatsu (8):
      kprobes: tracing/kprobes: Fix to kill kprobes on initmem after boot
      tracing/boot: Add per-instance tracing_on option support
      Documentation: tracing: Add tracing_on option to boot-time tracer
      tracing/kprobes: Support perf-style return probe
      tracing/uprobes: Support perf-style return probe
      Documentation: tracing: Add %return suffix description
      Documentation: tracing: boot: Add an example of tracing function-calls
      selftests/ftrace: Add %return suffix tests


 Documentation/trace/boottime-trace.rst             |   24 ++++++++++++++++++++
 Documentation/trace/kprobetrace.rst                |    2 ++
 Documentation/trace/uprobetracer.rst               |    2 ++
 include/linux/kprobes.h                            |    5 ++++
 init/main.c                                        |    2 ++
 kernel/kprobes.c                                   |   22 ++++++++++++++++++
 kernel/trace/trace.c                               |    4 ++-
 kernel/trace/trace_boot.c                          |   10 ++++++++
 kernel/trace/trace_kprobe.c                        |   18 ++++++++++++++-
 kernel/trace/trace_probe.h                         |    1 +
 kernel/trace/trace_uprobe.c                        |   15 ++++++++++++-
 .../ftrace/test.d/kprobe/kprobe_syntax_errors.tc   |    6 +++++
 .../test.d/kprobe/kretprobe_return_suffix.tc       |   21 ++++++++++++++++++
 .../ftrace/test.d/kprobe/uprobe_syntax_errors.tc   |    6 +++++
 14 files changed, 134 insertions(+), 4 deletions(-)
 create mode 100644 tools/testing/selftests/ftrace/test.d/kprobe/kretprobe_return_suffix.tc

--
Masami Hiramatsu (Linaro) <mhiramat@kernel.org>

Comments

Steven Rostedt Sept. 10, 2020, 11:34 p.m. UTC | #1
On Thu, 10 Sep 2020 17:54:54 +0900
Masami Hiramatsu <mhiramat@kernel.org> wrote:

> Hi,
> 
> Here is the 3rd version of the series to improve the boot-time tracing to
> support kretprobe and tracing_on option. Previous version is here:
> 
>  https://lkml.kernel.org/r/159894698993.1478826.2813843560314595660.stgit@devnote2
> 
> This version adds uprobe %return suffix support ([5/8]) and the testcases
> ([8/8]), and update kprobe %suffix support([4/8]) and the uprobe event
> document([6/8]).
> 
> 
> The combination of tracing_on and kretprobe allows us to trace events
> while a specific function call period. For example, the below bootconfig
> will make a function callgraph in the pci_proc_init() function at boot
> time.
> 
> ftrace {
> 	tracing_on = 0  # off at start
> 	tracer = function_graph
> 	event.kprobes {
> 		start_event {
> 			probes = "pci_proc_init"
> 			actions = "traceon"
> 		}
> 		end_event {
> 			probes = "pci_proc_init%return"
> 			actions = "traceoff"
> 		}
> 	}
> }
> 
> Here is the example output;
> 

[..]

Hi Masami,

This looks really great! I just got back from a 10 day holiday, and I'm
drowning in "catch-up".  I plan on looking at all this relatively soon
(in a week or two?). I just don't want you to think I'm ignoring this.

-- Steve
Masami Hiramatsu (Google) Sept. 11, 2020, 1:27 a.m. UTC | #2
On Thu, 10 Sep 2020 19:34:33 -0400
Steven Rostedt <rostedt@goodmis.org> wrote:

> On Thu, 10 Sep 2020 17:54:54 +0900
> Masami Hiramatsu <mhiramat@kernel.org> wrote:
> 
> > Hi,
> > 
> > Here is the 3rd version of the series to improve the boot-time tracing to
> > support kretprobe and tracing_on option. Previous version is here:
> > 
> >  https://lkml.kernel.org/r/159894698993.1478826.2813843560314595660.stgit@devnote2
> > 
> > This version adds uprobe %return suffix support ([5/8]) and the testcases
> > ([8/8]), and update kprobe %suffix support([4/8]) and the uprobe event
> > document([6/8]).
> > 
> > 
> > The combination of tracing_on and kretprobe allows us to trace events
> > while a specific function call period. For example, the below bootconfig
> > will make a function callgraph in the pci_proc_init() function at boot
> > time.
> > 
> > ftrace {
> > 	tracing_on = 0  # off at start
> > 	tracer = function_graph
> > 	event.kprobes {
> > 		start_event {
> > 			probes = "pci_proc_init"
> > 			actions = "traceon"
> > 		}
> > 		end_event {
> > 			probes = "pci_proc_init%return"
> > 			actions = "traceoff"
> > 		}
> > 	}
> > }
> > 
> > Here is the example output;
> > 
> 
> [..]
> 
> Hi Masami,
> 
> This looks really great! I just got back from a 10 day holiday, and I'm
> drowning in "catch-up".  I plan on looking at all this relatively soon
> (in a week or two?). I just don't want you to think I'm ignoring this.

Hi Steve,

Thanks, I hope you enjoyed the holiday!
BTW, this series includes some document update and testcases.
Since we can not decouple these updates, I think it is better you to
pull the series.

Regards,
Shuah Khan Sept. 11, 2020, 1:34 a.m. UTC | #3
On 9/10/20 7:27 PM, Masami Hiramatsu wrote:
> On Thu, 10 Sep 2020 19:34:33 -0400
> Steven Rostedt <rostedt@goodmis.org> wrote:
> 
>> On Thu, 10 Sep 2020 17:54:54 +0900
>> Masami Hiramatsu <mhiramat@kernel.org> wrote:
>>
>>> Hi,
>>>
>>> Here is the 3rd version of the series to improve the boot-time tracing to
>>> support kretprobe and tracing_on option. Previous version is here:
>>>
>>>   https://lkml.kernel.org/r/159894698993.1478826.2813843560314595660.stgit@devnote2
>>>
>>> This version adds uprobe %return suffix support ([5/8]) and the testcases
>>> ([8/8]), and update kprobe %suffix support([4/8]) and the uprobe event
>>> document([6/8]).
>>>
>>>
>>> The combination of tracing_on and kretprobe allows us to trace events
>>> while a specific function call period. For example, the below bootconfig
>>> will make a function callgraph in the pci_proc_init() function at boot
>>> time.
>>>
>>> ftrace {
>>> 	tracing_on = 0  # off at start
>>> 	tracer = function_graph
>>> 	event.kprobes {
>>> 		start_event {
>>> 			probes = "pci_proc_init"
>>> 			actions = "traceon"
>>> 		}
>>> 		end_event {
>>> 			probes = "pci_proc_init%return"
>>> 			actions = "traceoff"
>>> 		}
>>> 	}
>>> }
>>>
>>> Here is the example output;
>>>
>>
>> [..]
>>
>> Hi Masami,
>>
>> This looks really great! I just got back from a 10 day holiday, and I'm
>> drowning in "catch-up".  I plan on looking at all this relatively soon
>> (in a week or two?). I just don't want you to think I'm ignoring this.
> 
> Hi Steve,
> 
> Thanks, I hope you enjoyed the holiday!
> BTW, this series includes some document update and testcases.
> Since we can not decouple these updates, I think it is better you to
> pull the series.
> 

Hi Steve and Masami,

selftests patch 8/8 in this series looks good to me. My ack for
it to go through Steve's ftrace tree

Acked-by: Shuah Khan <skhan@linuxfoundation.org>

thanks,
-- Shuah