[v5,02/11] timekeeping: Add function to convert realtime to base clock

Commit Message

D, Lakshmi Sowjanya March 19, 2024, 1:05 p.m. UTC

From: Lakshmi Sowjanya D <[e-mail]>

PPS(Pulse Per Second) generates signals in realtime, but Timed IO
hardware understands time in base clock reference. Add an interface,
ktime_real_to_base_clock() to convert realtime to base clock.

Convert the base clock to the system clock using convert_base_to_cs() in
get_device_system_crosststamp().

Add the helper function timekeeping_clocksource_has_base(), to check
whether the current clocksource has the same base clock. This will be
used by Timed IO device to check if the base clock is X86_ART(Always
Running Timer).

Co-developed-by: Thomas Gleixner <[e-mail]>
Signed-off-by: Thomas Gleixner <[e-mail]>
Co-developed-by: Christopher S. Hall <[e-mail]>
Signed-off-by: Christopher S. Hall <[e-mail]>
Signed-off-by: Lakshmi Sowjanya D <[e-mail]>
---
 include/linux/timekeeping.h |   6 +++
 kernel/time/timekeeping.c   | 105 +++++++++++++++++++++++++++++++++++-
 2 files changed, 109 insertions(+), 2 deletions(-)

Comments

John Stultz March 19, 2024, 10:29 p.m. UTC | #1

On Tue, Mar 19, 2024 at 6:06 AM <[e-mail]> wrote:
>
> From: Lakshmi Sowjanya D <[e-mail]>
>
> PPS(Pulse Per Second) generates signals in realtime, but Timed IO
> hardware understands time in base clock reference. Add an interface,
> ktime_real_to_base_clock() to convert realtime to base clock.
>
> Convert the base clock to the system clock using convert_base_to_cs() in
> get_device_system_crosststamp().
>
> Add the helper function timekeeping_clocksource_has_base(), to check
> whether the current clocksource has the same base clock. This will be
> used by Timed IO device to check if the base clock is X86_ART(Always
> Running Timer).
>
> Co-developed-by: Thomas Gleixner <[e-mail]>
> Signed-off-by: Thomas Gleixner <[e-mail]>
> Co-developed-by: Christopher S. Hall <[e-mail]>
> Signed-off-by: Christopher S. Hall <[e-mail]>
> Signed-off-by: Lakshmi Sowjanya D <[e-mail]>
> ---
>  include/linux/timekeeping.h |   6 +++
>  kernel/time/timekeeping.c   | 105 +++++++++++++++++++++++++++++++++++-
>  2 files changed, 109 insertions(+), 2 deletions(-)
>
> diff --git a/include/linux/timekeeping.h b/include/linux/timekeeping.h
> index 7e50cbd97f86..1b2a4a37bf93 100644
> --- a/include/linux/timekeeping.h
> +++ b/include/linux/timekeeping.h
> @@ -275,12 +275,18 @@ struct system_device_crosststamp {
>   *             timekeeping code to verify comparability of two cycle values.
>   *             The default ID, CSID_GENERIC, does not identify a specific
>   *             clocksource.
> + * @nsecs:     @cycles is in nanoseconds.
>   */
>  struct system_counterval_t {
>         u64                     cycles;
>         enum clocksource_ids    cs_id;
> +       bool                    nsecs;

Apologies, this is a bit of an annoying bikeshed request, but maybe
use_nsecs here?
There are plenty of places where nsecs fields hold actual nanoseconds,
so what you have might be easy to misread in the future.

Also, at least in this series, I'm not sure I see where this nsecs
value gets set? Maybe something to split out and add in a separate
patch, where its use is more clear?

> +bool timekeeping_clocksource_has_base(enum clocksource_ids id)
> +{
> +       unsigned int seq;
> +       bool ret;
> +
> +       do {
> +               seq = read_seqcount_begin(&tk_core.seq);
> +               ret = tk_core.timekeeper.tkr_mono.clock->base ?
> +               tk_core.timekeeper.tkr_mono.clock->base->id == id : false;

Again, bikeshed nit: I know folks like ternaries for conciseness, but
once you've crossed a single line, I'd often prefer to read an if
statement.

thanks
-john

Thomas Gleixner March 20, 2024, 11:29 a.m. UTC | #2

On Tue, Mar 19 2024 at 18:35, [e-mail] wrote:
> From: Lakshmi Sowjanya D <[e-mail]>
>
> PPS(Pulse Per Second) generates signals in realtime, but Timed IO
> hardware understands time in base clock reference. Add an interface,
> ktime_real_to_base_clock() to convert realtime to base clock.
>
> Convert the base clock to the system clock using convert_base_to_cs() in
> get_device_system_crosststamp().

This really is doing two unrelated things. ktime_real_to_base_clock()
has absolutely nothing to do with the clocksource_base related changes
to get_device_system_crosststamp()

Thomas Gleixner March 21, 2024, 2:53 p.m. UTC | #3

On Tue, Mar 19 2024 at 18:35, [e-mail] wrote:
> +bool ktime_real_to_base_clock(ktime_t treal, enum clocksource_ids base_id, u64 *cycles)
> +{
> +	struct timekeeper *tk = &tk_core.timekeeper;
> +	unsigned int seq;
> +	u64 delta;
> +
> +	do {
> +		seq = read_seqcount_begin(&tk_core.seq);
> +		delta = (u64)treal - tk->tkr_mono.base_real;
> +		if (delta > tk->tkr_mono.clock->max_idle_ns)
> +			return false;

I don't think this cutoff is valid. There is no guarantee that this is
linear unless:

       Treal[last timekeeper update] <= treal < Treal[next timekeeper update]

Look at the dance in get_device_system_crosststamp() and
adjust_historical_crosststamp() to see why.

> +		*cycles = tk->tkr_mono.cycle_last + convert_ns_to_cs(delta);
> +		if (!convert_cs_to_base(cycles, base_id))
> +			return false;
> +	} while (read_seqcount_retry(&tk_core.seq, seq));
> +
> +	return true;
> +}
> +EXPORT_SYMBOL_GPL(ktime_real_to_base_clock);

Looking at the usage site:

> +static bool pps_generate_next_pulse(struct pps_tio *tio, ktime_t expires)
> +{
> +	u64 art;
> +
> +	if (!ktime_real_to_base_clock(expires, CSID_X86_ART, &art)) {
> +		pps_tio_disable(tio);

I'm pretty sure this can happen when there is sufficient delay between
the check for (now - expires < SAFE_TIME_NS) and the delta computation
in ktime_real_to_base_clock() if there is a timerkeeper update
interleaving which brings tkr_mono.base_real in front of expires.

Is that intentional and correct?

If so, then it's inconsistent with the behaviour of the hrtimer
callback:

> +		return false;
> +	}
> +
> +	pps_compv_write(tio, art - ART_HW_DELAY_CYCLES);
> +	return true;
> +}
> +
> +static enum hrtimer_restart hrtimer_callback(struct hrtimer *timer)
> +{
> +	struct pps_tio *tio = container_of(timer, struct pps_tio, timer);
> +	ktime_t expires, now;
> +
> +	guard(spinlock)(&tio->lock);
> +
> +	expires = hrtimer_get_expires(timer);
> +	now = ktime_get_real();
> +
> +	if (now - expires < SAFE_TIME_NS) {
> +		if (!pps_generate_next_pulse(tio, expires + SAFE_TIME_NS))
> +			return HRTIMER_NORESTART;
> +	}

This safe guard does not care about time being set. I'm not familiar
with the PPS logic, but is it expected that the pulse pattern will be
like this:

         

    ---|-----|-----|-----|----------------->
       P     P  ^        P
                |
        clock_settime(CLOCK_REALTIME, now - 2 seconds)         
        
        Obviously the pulse gap will be as big as the time is set
        backwards, which might be way more than 2 seconds.
        

    ---|-----|-----|-----|----------------->
       P     P  ^  P     P
                |
        clock_settime(CLOCK_REALTIME, now + 2 seconds)         

I don't see anything in this code which cares about CLOCK_REALTIME being
set via clock_settime() or adjtimex().

Aside of that I have a question about how the TIO hardware treats this
case:

   ktime_real_to_base_clock(expires, &art);

-> GAP which makes @art get into the past

   pps_compv_write(tio, art - ART_HW_DELAY_CYCLES);

Will the hardware ignore that already expired value or just emit a pulse
immediately? In the latter case the pulse will be at a random point in
time, which does not sound correct.

Thanks,

        tglx

D, Lakshmi Sowjanya April 1, 2024, 1:07 p.m. UTC | #4

> -----Original Message-----
> From: Thomas Gleixner <[e-mail]>
> Sent: Thursday, March 21, 2024 8:23 PM
> To: D, Lakshmi Sowjanya <[e-mail]>;
> [e-mail]; [e-mail]; [e-mail]; linux-
> [e-mail]
> Cc: [e-mail]; [e-mail]; [e-mail]; intel-
> [e-mail]; [e-mail]; Dong, Eddie
> <[e-mail]>; Hall, Christopher S <[e-mail]>;
> Brandeburg, Jesse <[e-mail]>; [e-mail];
> [e-mail]; [e-mail];
> [e-mail]; [e-mail]; [e-mail];
> Nguyen, Anthony L <[e-mail]>;
> [e-mail]; N, Pandith <[e-mail]>;
> Sangannavar, Mallikarjunappa <[e-mail]>;
> Mohan, Subramanian <[e-mail]>; Goudar, Basavaraj
> <[e-mail]>; T R, Thejesh Reddy
> <[e-mail]>; D, Lakshmi Sowjanya
> <[e-mail]>
> Subject: Re: [PATCH v5 02/11] timekeeping: Add function to convert realtime to
> base clock
> 
> On Tue, Mar 19 2024 at 18:35, [e-mail] wrote:
> > +bool ktime_real_to_base_clock(ktime_t treal, enum clocksource_ids
> > +base_id, u64 *cycles) {
> > +	struct timekeeper *tk = &tk_core.timekeeper;
> > +	unsigned int seq;
> > +	u64 delta;
> > +
> > +	do {
> > +		seq = read_seqcount_begin(&tk_core.seq);
> > +		delta = (u64)treal - tk->tkr_mono.base_real;
> > +		if (delta > tk->tkr_mono.clock->max_idle_ns)
> > +			return false;
> 
> I don't think this cutoff is valid. There is no guarantee that this is linear unless:
> 
>        Treal[last timekeeper update] <= treal < Treal[next timekeeper update]
> 
> Look at the dance in get_device_system_crosststamp() and
> adjust_historical_crosststamp() to see why.
> 
> > +		*cycles = tk->tkr_mono.cycle_last + convert_ns_to_cs(delta);
> > +		if (!convert_cs_to_base(cycles, base_id))
> > +			return false;
> > +	} while (read_seqcount_retry(&tk_core.seq, seq));
> > +
> > +	return true;
> > +}
> > +EXPORT_SYMBOL_GPL(ktime_real_to_base_clock);
> 
> Looking at the usage site:
> 
> > +static bool pps_generate_next_pulse(struct pps_tio *tio, ktime_t
> > +expires) {
> > +	u64 art;
> > +
> > +	if (!ktime_real_to_base_clock(expires, CSID_X86_ART, &art)) {
> > +		pps_tio_disable(tio);
> 
> I'm pretty sure this can happen when there is sufficient delay between the check
> for (now - expires < SAFE_TIME_NS) and the delta computation in
> ktime_real_to_base_clock() if there is a timerkeeper update interleaving which
> brings tkr_mono.base_real in front of expires.
> 
> Is that intentional and correct?
> 
> If so, then it's inconsistent with the behaviour of the hrtimer
> callback:
> 
> > +		return false;
> > +	}
> > +
> > +	pps_compv_write(tio, art - ART_HW_DELAY_CYCLES);
> > +	return true;
> > +}
> > +
> > +static enum hrtimer_restart hrtimer_callback(struct hrtimer *timer) {
> > +	struct pps_tio *tio = container_of(timer, struct pps_tio, timer);
> > +	ktime_t expires, now;
> > +
> > +	guard(spinlock)(&tio->lock);
> > +
> > +	expires = hrtimer_get_expires(timer);
> > +	now = ktime_get_real();
> > +
> > +	if (now - expires < SAFE_TIME_NS) {
> > +		if (!pps_generate_next_pulse(tio, expires + SAFE_TIME_NS))
> > +			return HRTIMER_NORESTART;
> > +	}
> 
> This safe guard does not care about time being set. I'm not familiar with the PPS
> logic, but is it expected that the pulse pattern will be like this:
> 
> 
> 
>     ---|-----|-----|-----|----------------->
>        P     P  ^        P
>                 |
>         clock_settime(CLOCK_REALTIME, now - 2 seconds)
> 
>         Obviously the pulse gap will be as big as the time is set
>         backwards, which might be way more than 2 seconds.
> 
> 
>     ---|-----|-----|-----|----------------->
>        P     P  ^  P     P
>                 |
>         clock_settime(CLOCK_REALTIME, now + 2 seconds)
> 
> I don't see anything in this code which cares about CLOCK_REALTIME being set
> via clock_settime() or adjtimex().
> 
> Aside of that I have a question about how the TIO hardware treats this
> case:
> 
>    ktime_real_to_base_clock(expires, &art);
> 
> -> GAP which makes @art get into the past
> 
>    pps_compv_write(tio, art - ART_HW_DELAY_CYCLES);
> 
> Will the hardware ignore that already expired value or just emit a pulse
> immediately? In the latter case the pulse will be at a random point in time, which
> does not sound correct.

Thanks for the review Thomas,

If an expired time is programmed to hardware, it ignores and no pulse is generated.
There is an event counter feature in hardware, this increments on every pulse sent. In next version of driver, we will be monitoring this counter to detect any missed pulse and disable the hardware.

Regards,
Sowjanya

> 
> Thanks,
> 
>         tglx

Patch

diff --git a/include/linux/timekeeping.h b/include/linux/timekeeping.h
index 7e50cbd97f86..1b2a4a37bf93 100644
--- a/include/linux/timekeeping.h
+++ b/include/linux/timekeeping.h
@@ -275,12 +275,18 @@  struct system_device_crosststamp {
  *		timekeeping code to verify comparability of two cycle values.
  *		The default ID, CSID_GENERIC, does not identify a specific
  *		clocksource.
+ * @nsecs:	@cycles is in nanoseconds.
  */
 struct system_counterval_t {
 	u64			cycles;
 	enum clocksource_ids	cs_id;
+	bool			nsecs;
 };
 
+extern bool ktime_real_to_base_clock(ktime_t treal,
+				     enum clocksource_ids base_id, u64 *cycles);
+extern bool timekeeping_clocksource_has_base(enum clocksource_ids id);
+
 /*
  * Get cross timestamp between system clock and device clock
  */
diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c
index b58dffc58a8f..78836c7712dd 100644
--- a/kernel/time/timekeeping.c
+++ b/kernel/time/timekeeping.c
@@ -1193,6 +1193,82 @@  static bool timestamp_in_interval(u64 start, u64 end, u64 ts)
 	return false;
 }
 
+static bool convert_clock(u64 *val, u32 numerator, u32 denominator)
+{
+	u64 rem, res;
+
+	if (!numerator || !denominator)
+		return false;
+
+	res = div64_u64_rem(*val, denominator, &rem) * numerator;
+	*val = res + div_u64(rem * numerator, denominator);
+	return true;
+}
+
+static bool convert_base_to_cs(struct system_counterval_t *scv)
+{
+	struct clocksource *cs = tk_core.timekeeper.tkr_mono.clock;
+	struct clocksource_base *base = cs->base;
+	u32 num, den;
+
+	/* The timestamp was taken from the time keeper clock source */
+	if (cs->id == scv->cs_id)
+		return true;
+
+	/* Check whether cs_id matches the base clock */
+	if (!base || base->id != scv->cs_id)
+		return false;
+
+	num = scv->nsecs ? cs->freq_khz : base->numerator;
+	den = scv->nsecs ? USEC_PER_SEC : base->denominator;
+
+	convert_clock(&scv->cycles, num, den);
+	scv->cycles += base->offset;
+	return true;
+}
+
+static bool convert_cs_to_base(u64 *cycles, enum clocksource_ids base_id)
+{
+	struct clocksource *cs = tk_core.timekeeper.tkr_mono.clock;
+	struct clocksource_base *base = cs->base;
+
+	/* Check whether base_id matches the base clock */
+	if (!base || base->id != base_id)
+		return false;
+
+	*cycles -= base->offset;
+	if (!convert_clock(cycles, base->denominator, base->numerator))
+		return false;
+	return true;
+}
+
+static u64 convert_ns_to_cs(u64 delta)
+{
+	struct tk_read_base *tkr = &tk_core.timekeeper.tkr_mono;
+
+	return div_u64((delta << tkr->shift) - tkr->xtime_nsec, tkr->mult);
+}
+
+bool ktime_real_to_base_clock(ktime_t treal, enum clocksource_ids base_id, u64 *cycles)
+{
+	struct timekeeper *tk = &tk_core.timekeeper;
+	unsigned int seq;
+	u64 delta;
+
+	do {
+		seq = read_seqcount_begin(&tk_core.seq);
+		delta = (u64)treal - tk->tkr_mono.base_real;
+		if (delta > tk->tkr_mono.clock->max_idle_ns)
+			return false;
+		*cycles = tk->tkr_mono.cycle_last + convert_ns_to_cs(delta);
+		if (!convert_cs_to_base(cycles, base_id))
+			return false;
+	} while (read_seqcount_retry(&tk_core.seq, seq));
+
+	return true;
+}
+EXPORT_SYMBOL_GPL(ktime_real_to_base_clock);
+
 /**
  * get_device_system_crosststamp - Synchronously capture system/device timestamp
  * @get_time_fn:	Callback to get simultaneous device time and
@@ -1238,8 +1314,7 @@  int get_device_system_crosststamp(int (*get_time_fn)
 		 * system counter value is the same as for the currently
 		 * installed timekeeper clocksource
 		 */
-		if (system_counterval.cs_id == CSID_GENERIC ||
-		    tk->tkr_mono.clock->id != system_counterval.cs_id)
+		if (!convert_base_to_cs(&system_counterval))
 			return -ENODEV;
 		cycles = system_counterval.cycles;
 
@@ -1304,6 +1379,32 @@  int get_device_system_crosststamp(int (*get_time_fn)
 }
 EXPORT_SYMBOL_GPL(get_device_system_crosststamp);
 
+/**
+ * timekeeping_clocksource_has_base - Check whether the current clocksource
+ *	has a base clock
+ * @id:		The clocksource ID to check for
+ *
+ * Note:	The return value is a snapshot which can become invalid right
+ *	after the function returns.
+ *
+ * Return:	true if the timekeeper clocksource has a base clock with @id,
+ *	false otherwise
+ */
+bool timekeeping_clocksource_has_base(enum clocksource_ids id)
+{
+	unsigned int seq;
+	bool ret;
+
+	do {
+		seq = read_seqcount_begin(&tk_core.seq);
+		ret = tk_core.timekeeper.tkr_mono.clock->base ?
+		tk_core.timekeeper.tkr_mono.clock->base->id == id : false;
+	} while (read_seqcount_retry(&tk_core.seq, seq));
+
+	return ret;
+}
+EXPORT_SYMBOL_GPL(timekeeping_clocksource_has_base);
+
 /**
  * do_settimeofday64 - Sets the time of day.
  * @ts:     pointer to the timespec64 variable containing the new time