diff mbox

[v2,1/2] trace: Default to using trace_global_clock if sched_clock is unstable

Message ID 20180330150132.16903-1-chris@chris-wilson.co.uk (mailing list archive)
State New, archived
Headers show

Commit Message

Chris Wilson March 30, 2018, 3:01 p.m. UTC
Across suspend, we may see a very large drift in timestamps if the sched
clock is unstable, prompting the global trace's ringbuffer code to warn
and suggest switching to the global clock. Preempt this request by
detecting when the sched clock is unstable (determined during
late_initcall) and automatically switching the default clock over to
trace_global_clock.

This should prevent requiring user interaction to resolve warnings such
as:

    Delta way too big! 18446743856563626466 ts=18446744054496180323 write stamp = 197932553857
    If you just came from a suspend/resume,
    please switch to the trace global clock:
    echo global > /sys/kernel/debug/tracing/trace_clock

Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Steven Rostedt (VMware) <rostedt@goodmis.org>
---
v2: Tell the user what's happening and what they can do to correct it.
---
 kernel/trace/trace.c | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)

Comments

Steven Rostedt April 2, 2018, 3:17 p.m. UTC | #1
On Fri, 30 Mar 2018 16:01:31 +0100
Chris Wilson <chris@chris-wilson.co.uk> wrote:

> Across suspend, we may see a very large drift in timestamps if the sched
> clock is unstable, prompting the global trace's ringbuffer code to warn
> and suggest switching to the global clock. Preempt this request by
> detecting when the sched clock is unstable (determined during
> late_initcall) and automatically switching the default clock over to
> trace_global_clock.
> 
> This should prevent requiring user interaction to resolve warnings such
> as:
> 
>     Delta way too big! 18446743856563626466 ts=18446744054496180323 write stamp = 197932553857
>     If you just came from a suspend/resume,
>     please switch to the trace global clock:
>     echo global > /sys/kernel/debug/tracing/trace_clock
> 
> Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
> Cc: Steven Rostedt (VMware) <rostedt@goodmis.org>
> ---
> v2: Tell the user what's happening and what they can do to correct it.
>

Thanks, I pulled them in. I'll start testing them before I post to git.

-- Steve
Chris Wilson April 3, 2018, 12:07 p.m. UTC | #2
Quoting Steven Rostedt (2018-04-02 16:17:36)
> On Fri, 30 Mar 2018 16:01:31 +0100
> Chris Wilson <chris@chris-wilson.co.uk> wrote:
> 
> > Across suspend, we may see a very large drift in timestamps if the sched
> > clock is unstable, prompting the global trace's ringbuffer code to warn
> > and suggest switching to the global clock. Preempt this request by
> > detecting when the sched clock is unstable (determined during
> > late_initcall) and automatically switching the default clock over to
> > trace_global_clock.
> > 
> > This should prevent requiring user interaction to resolve warnings such
> > as:
> > 
> >     Delta way too big! 18446743856563626466 ts=18446744054496180323 write stamp = 197932553857
> >     If you just came from a suspend/resume,
> >     please switch to the trace global clock:
> >     echo global > /sys/kernel/debug/tracing/trace_clock
> > 
> > Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
> > Cc: Steven Rostedt (VMware) <rostedt@goodmis.org>
> > ---
> > v2: Tell the user what's happening and what they can do to correct it.
> >
> 
> Thanks, I pulled them in. I'll start testing them before I post to git.

Ta, I've pushed them onto our local CI branch to be superseded when they
land upstream. If you see any problems, just holler.
-Chris
Chris Wilson April 4, 2018, 8:46 p.m. UTC | #3
Quoting Chris Wilson (2018-03-30 16:01:31)
> Across suspend, we may see a very large drift in timestamps if the sched
> clock is unstable, prompting the global trace's ringbuffer code to warn
> and suggest switching to the global clock. Preempt this request by
> detecting when the sched clock is unstable (determined during
> late_initcall) and automatically switching the default clock over to
> trace_global_clock.
> 
> This should prevent requiring user interaction to resolve warnings such
> as:
> 
>     Delta way too big! 18446743856563626466 ts=18446744054496180323 write stamp = 197932553857
>     If you just came from a suspend/resume,
>     please switch to the trace global clock:
>     echo global > /sys/kernel/debug/tracing/trace_clock
> 
> Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
> Cc: Steven Rostedt (VMware) <rostedt@goodmis.org>
> ---
> v2: Tell the user what's happening and what they can do to correct it.
> ---
>  kernel/trace/trace.c | 19 +++++++++++++++++++
>  1 file changed, 19 insertions(+)
> 
> diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c
> index 13baf85b27d8..7c155fa879e1 100644
> --- a/kernel/trace/trace.c
> +++ b/kernel/trace/trace.c
> @@ -41,6 +41,7 @@
>  #include <linux/nmi.h>
>  #include <linux/fs.h>
>  #include <linux/trace.h>
> +#include <linux/sched/clock.h>
>  #include <linux/sched/rt.h>
>  
>  #include "trace.h"
> @@ -8505,3 +8506,21 @@ __init static int clear_boot_tracer(void)
>  
>  fs_initcall(tracer_init_tracefs);
>  late_initcall_sync(clear_boot_tracer);
> +
> +#ifdef CONFIG_HAVE_UNSTABLE_SCHED_CLOCK
> +__init static int tracing_set_default_clock(void)
> +{
> +       /* sched_clock_stable() is determined in late_initcall */
> +       if (trace_boot_clock || sched_clock_stable()) {

Oh flip, I reversed the logic from v1.

I was going to do an early return:

	if (trace_boot_clock || sched_clock_stable())
		return;

to shift the message to the left slightly, at left it half way.

(I was wondering why CI didn't show the same results as v1!)
-Chris
diff mbox

Patch

diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c
index 13baf85b27d8..7c155fa879e1 100644
--- a/kernel/trace/trace.c
+++ b/kernel/trace/trace.c
@@ -41,6 +41,7 @@ 
 #include <linux/nmi.h>
 #include <linux/fs.h>
 #include <linux/trace.h>
+#include <linux/sched/clock.h>
 #include <linux/sched/rt.h>
 
 #include "trace.h"
@@ -8505,3 +8506,21 @@  __init static int clear_boot_tracer(void)
 
 fs_initcall(tracer_init_tracefs);
 late_initcall_sync(clear_boot_tracer);
+
+#ifdef CONFIG_HAVE_UNSTABLE_SCHED_CLOCK
+__init static int tracing_set_default_clock(void)
+{
+	/* sched_clock_stable() is determined in late_initcall */
+	if (trace_boot_clock || sched_clock_stable()) {
+		printk(KERN_WARNING
+		       "Unstable clock detected, switching default tracing clock to \"global\"\n"
+		       "If you want to keep using the local clock, then add:\n"
+		       "  \"trace_clock=local\"\n"
+		       "on the kernel command line\n");
+		tracing_set_clock(&global_trace, "global");
+	}
+
+	return 0;
+}
+late_initcall_sync(tracing_set_default_clock);
+#endif