| Message ID | 20210323074326.28302-3-tony@atomide.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | Fixes for for dra7 timer wrap errata i940 | expand |
On Tue, Mar 23, 2021 at 09:43:26AM +0200, Tony Lindgren wrote: > There is a timer wrap issue on dra7 for the ARM architected timer. > In a typical clock configuration the timer fails to wrap after 388 days. > > To work around the issue, we need to use timer-ti-dm percpu timers instead. > > Let's configure dmtimer3 and 4 as percpu timers by default, and warn about > the issue if the dtb is not configured properly. Hi Tony, This causes a conflict for IPU2 which is using timer 3 and 4. From arch/arm/boot/dts/dra7-ipu-dsp-common.dtsi: &ipu2 { mboxes = <&mailbox6 &mbox_ipu2_ipc3x>; ti,timers = <&timer3>; ti,watchdog-timers = <&timer4>, <&timer9>; }; I noticed an error ("could not get timer platform device") when booting mainline on a BeagleBoard X15 (AM578): omap-rproc 55020000.ipu: assigned reserved memory node ipu2-memory@95800000 remoteproc remoteproc1: 55020000.ipu is available remoteproc remoteproc1: powering up 55020000.ipu remoteproc remoteproc1: Booting fw image dra7-ipu2-fw.xem4, size 3747220 omap-rproc 55020000.ipu: could not get timer platform device omap-rproc 55020000.ipu: omap_rproc_enable_timers failed: -19 remoteproc remoteproc1: can't start rproc 55020000.ipu: -19 I switched this errata fix to use timer 15 and 16 instead which resolves the error. Do you think that is an acceptable solution? If so, I will post the patch to the list. Thanks, Drew
Hi, * Drew Fustini <dfustini@baylibre.com> [220129 21:05]: > On Tue, Mar 23, 2021 at 09:43:26AM +0200, Tony Lindgren wrote: > > There is a timer wrap issue on dra7 for the ARM architected timer. > > In a typical clock configuration the timer fails to wrap after 388 days. > > > > To work around the issue, we need to use timer-ti-dm percpu timers instead. > > > > Let's configure dmtimer3 and 4 as percpu timers by default, and warn about > > the issue if the dtb is not configured properly. > > Hi Tony, > > This causes a conflict for IPU2 which is using timer 3 and 4. > > From arch/arm/boot/dts/dra7-ipu-dsp-common.dtsi: > > &ipu2 { > mboxes = <&mailbox6 &mbox_ipu2_ipc3x>; > ti,timers = <&timer3>; > ti,watchdog-timers = <&timer4>, <&timer9>; > }; OK, sorry I missed that part. > I noticed an error ("could not get timer platform device") when booting > mainline on a BeagleBoard X15 (AM578): > > omap-rproc 55020000.ipu: assigned reserved memory node ipu2-memory@95800000 > remoteproc remoteproc1: 55020000.ipu is available > remoteproc remoteproc1: powering up 55020000.ipu > remoteproc remoteproc1: Booting fw image dra7-ipu2-fw.xem4, size 3747220 > omap-rproc 55020000.ipu: could not get timer platform device > omap-rproc 55020000.ipu: omap_rproc_enable_timers failed: -19 > remoteproc remoteproc1: can't start rproc 55020000.ipu: -19 > > I switched this errata fix to use timer 15 and 16 instead which resolves > the error. Do you think that is an acceptable solution? I think the only difference is that timers 15 and 16 are in l4_per3 instead of l4_per1. I doubt that matters as they are pretty much always clocked in this case. If you want to check you can run cyclictest :) > If so, I will post the patch to the list. OK thanks, Tony
On Mon, Jan 31, 2022 at 12:16:06PM +0200, Tony Lindgren wrote: > Hi, > > * Drew Fustini <dfustini@baylibre.com> [220129 21:05]: > > On Tue, Mar 23, 2021 at 09:43:26AM +0200, Tony Lindgren wrote: > > > There is a timer wrap issue on dra7 for the ARM architected timer. > > > In a typical clock configuration the timer fails to wrap after 388 days. > > > > > > To work around the issue, we need to use timer-ti-dm percpu timers instead. > > > > > > Let's configure dmtimer3 and 4 as percpu timers by default, and warn about > > > the issue if the dtb is not configured properly. > > > > Hi Tony, > > > > This causes a conflict for IPU2 which is using timer 3 and 4. > > > > From arch/arm/boot/dts/dra7-ipu-dsp-common.dtsi: > > > > &ipu2 { > > mboxes = <&mailbox6 &mbox_ipu2_ipc3x>; > > ti,timers = <&timer3>; > > ti,watchdog-timers = <&timer4>, <&timer9>; > > }; > > OK, sorry I missed that part. > > > I noticed an error ("could not get timer platform device") when booting > > mainline on a BeagleBoard X15 (AM578): > > > > omap-rproc 55020000.ipu: assigned reserved memory node ipu2-memory@95800000 > > remoteproc remoteproc1: 55020000.ipu is available > > remoteproc remoteproc1: powering up 55020000.ipu > > remoteproc remoteproc1: Booting fw image dra7-ipu2-fw.xem4, size 3747220 > > omap-rproc 55020000.ipu: could not get timer platform device > > omap-rproc 55020000.ipu: omap_rproc_enable_timers failed: -19 > > remoteproc remoteproc1: can't start rproc 55020000.ipu: -19 > > > > I switched this errata fix to use timer 15 and 16 instead which resolves > > the error. Do you think that is an acceptable solution? > > I think the only difference is that timers 15 and 16 are in l4_per3 instead > of l4_per1. I doubt that matters as they are pretty much always clocked in > this case. If you want to check you can run cyclictest :) I ran this with existing errata fix with dmtimer 3 and 4: root@am57xx-evm:~# cyclictest --mlockall --smp --priority=80 --interval=200 --distance=0 # /dev/cpu_dma_latency set to 0us policy: fifo: loadavg: 0.02 0.03 0.05 T: 0 ( 1449) P:80 I:200 C: 800368 Min: 0 Act: 32 Avg: 22 Max: 128 T: 1 ( 1450) P:80 I:200 C: 800301 Min: 0 Act: 12 Avg: 23 Max: 70 And the results after my switch to dmtimer 15 and 16: root@am57xx-evm:~# cyclictest --mlockall --smp --priority=80 --interval=200 --distance=0 # /dev/cpu_dma_latency set to 0us policy: fifo: loadavg: 0.36 0.19 0.07 T: 0 ( 1711) P:80 I:200 C: 759599 Min: 0 Act: 6 Avg: 22 Max: 108 T: 1 ( 1712) P:80 I:200 C: 759539 Min: 0 Act: 19 Avg: 23 Max: 79 This doesn't appear to show any latency regression. Any other options for cyclictest that that you'd recommend trying? Thank you, Drew
diff --git a/arch/arm/boot/dts/dra7-l4.dtsi b/arch/arm/boot/dts/dra7-l4.dtsi --- a/arch/arm/boot/dts/dra7-l4.dtsi +++ b/arch/arm/boot/dts/dra7-l4.dtsi @@ -1168,7 +1168,7 @@ timer2: timer@0 { }; }; - target-module@34000 { /* 0x48034000, ap 7 46.0 */ + timer3_target: target-module@34000 { /* 0x48034000, ap 7 46.0 */ compatible = "ti,sysc-omap4-timer", "ti,sysc"; reg = <0x34000 0x4>, <0x34010 0x4>; @@ -1195,7 +1195,7 @@ timer3: timer@0 { }; }; - target-module@36000 { /* 0x48036000, ap 9 4e.0 */ + timer4_target: target-module@36000 { /* 0x48036000, ap 9 4e.0 */ compatible = "ti,sysc-omap4-timer", "ti,sysc"; reg = <0x36000 0x4>, <0x36010 0x4>; diff --git a/arch/arm/boot/dts/dra7.dtsi b/arch/arm/boot/dts/dra7.dtsi --- a/arch/arm/boot/dts/dra7.dtsi +++ b/arch/arm/boot/dts/dra7.dtsi @@ -46,6 +46,7 @@ aliases { timer { compatible = "arm,armv7-timer"; + status = "disabled"; /* See ARM architected timer wrap erratum i940 */ interrupts = <GIC_PPI 13 (GIC_CPU_MASK_SIMPLE(2) | IRQ_TYPE_LEVEL_LOW)>, <GIC_PPI 14 (GIC_CPU_MASK_SIMPLE(2) | IRQ_TYPE_LEVEL_LOW)>, <GIC_PPI 11 (GIC_CPU_MASK_SIMPLE(2) | IRQ_TYPE_LEVEL_LOW)>, @@ -1241,3 +1242,22 @@ timer@0 { assigned-clock-parents = <&sys_32k_ck>; }; }; + +/* Local timers, see ARM architected timer wrap erratum i940 */ +&timer3_target { + ti,no-reset-on-init; + ti,no-idle; + timer@0 { + assigned-clocks = <&l4per_clkctrl DRA7_L4PER_TIMER3_CLKCTRL 24>; + assigned-clock-parents = <&timer_sys_clk_div>; + }; +}; + +&timer4_target { + ti,no-reset-on-init; + ti,no-idle; + timer@0 { + assigned-clocks = <&l4per_clkctrl DRA7_L4PER_TIMER4_CLKCTRL 24>; + assigned-clock-parents = <&timer_sys_clk_div>; + }; +}; diff --git a/drivers/clocksource/timer-ti-dm-systimer.c b/drivers/clocksource/timer-ti-dm-systimer.c --- a/drivers/clocksource/timer-ti-dm-systimer.c +++ b/drivers/clocksource/timer-ti-dm-systimer.c @@ -2,6 +2,7 @@ #include <linux/clk.h> #include <linux/clocksource.h> #include <linux/clockchips.h> +#include <linux/cpuhotplug.h> #include <linux/interrupt.h> #include <linux/io.h> #include <linux/iopoll.h> @@ -630,6 +631,78 @@ static int __init dmtimer_clockevent_init(struct device_node *np) return error; } +/* Dmtimer as percpu timer. See dra7 ARM architected timer wrap erratum i940 */ +static DEFINE_PER_CPU(struct dmtimer_clockevent, dmtimer_percpu_timer); + +static int __init dmtimer_percpu_timer_init(struct device_node *np, int cpu) +{ + struct dmtimer_clockevent *clkevt; + int error; + + if (!cpu_possible(cpu)) + return -EINVAL; + + if (!of_property_read_bool(np->parent, "ti,no-reset-on-init") || + !of_property_read_bool(np->parent, "ti,no-idle")) + pr_warn("Incomplete dtb for percpu dmtimer %pOF\n", np->parent); + + clkevt = per_cpu_ptr(&dmtimer_percpu_timer, cpu); + + error = dmtimer_clkevt_init_common(clkevt, np, CLOCK_EVT_FEAT_ONESHOT, + cpumask_of(cpu), "percpu-dmtimer", + 500); + if (error) + return error; + + return 0; +} + +/* See TRM for timer internal resynch latency */ +static int omap_dmtimer_starting_cpu(unsigned int cpu) +{ + struct dmtimer_clockevent *clkevt = per_cpu_ptr(&dmtimer_percpu_timer, cpu); + struct clock_event_device *dev = &clkevt->dev; + struct dmtimer_systimer *t = &clkevt->t; + + clockevents_config_and_register(dev, t->rate, 3, ULONG_MAX); + irq_force_affinity(dev->irq, cpumask_of(cpu)); + + return 0; +} + +static int __init dmtimer_percpu_timer_startup(void) +{ + struct dmtimer_clockevent *clkevt = per_cpu_ptr(&dmtimer_percpu_timer, 0); + struct dmtimer_systimer *t = &clkevt->t; + + if (t->sysc) { + cpuhp_setup_state(CPUHP_AP_TI_GP_TIMER_STARTING, + "clockevents/omap/gptimer:starting", + omap_dmtimer_starting_cpu, NULL); + } + + return 0; +} +subsys_initcall(dmtimer_percpu_timer_startup); + +static int __init dmtimer_percpu_quirk_init(struct device_node *np, u32 pa) +{ + struct device_node *arm_timer; + + arm_timer = of_find_compatible_node(NULL, NULL, "arm,armv7-timer"); + if (of_device_is_available(arm_timer)) { + pr_warn_once("ARM architected timer wrap issue i940 detected\n"); + return 0; + } + + if (pa == 0x48034000) /* dra7 dmtimer3 */ + return dmtimer_percpu_timer_init(np, 0); + else if (pa == 0x48036000) /* dra7 dmtimer4 */ + return dmtimer_percpu_timer_init(np, 1); + + return 0; +} + /* Clocksource */ static struct dmtimer_clocksource * to_dmtimer_clocksource(struct clocksource *cs) @@ -763,6 +836,9 @@ static int __init dmtimer_systimer_init(struct device_node *np) if (clockevent == pa) return dmtimer_clockevent_init(np); + if (of_machine_is_compatible("ti,dra7")) + return dmtimer_percpu_quirk_init(np, pa); + return 0; } diff --git a/include/linux/cpuhotplug.h b/include/linux/cpuhotplug.h --- a/include/linux/cpuhotplug.h +++ b/include/linux/cpuhotplug.h @@ -135,6 +135,7 @@ enum cpuhp_state { CPUHP_AP_RISCV_TIMER_STARTING, CPUHP_AP_CLINT_TIMER_STARTING, CPUHP_AP_CSKY_TIMER_STARTING, + CPUHP_AP_TI_GP_TIMER_STARTING, CPUHP_AP_HYPERV_TIMER_STARTING, CPUHP_AP_KVM_STARTING, CPUHP_AP_KVM_ARM_VGIC_INIT_STARTING,
There is a timer wrap issue on dra7 for the ARM architected timer. In a typical clock configuration the timer fails to wrap after 388 days. To work around the issue, we need to use timer-ti-dm percpu timers instead. Let's configure dmtimer3 and 4 as percpu timers by default, and warn about the issue if the dtb is not configured properly. Let's do this as a single patch so it can be backported to v5.8 and later kernels easily. Note that this patch depends on earlier timer-ti-dm systimer posted mode fixes, and a preparatory clockevent patch "clocksource/drivers/timer-ti-dm: Prepare to handle dra7 timer wrap issue". For more information, please see the errata for "AM572x Sitara Processors Silicon Revisions 1.1, 2.0": https://www.ti.com/lit/er/sprz429m/sprz429m.pdf The concept is based on earlier reference patches done by Tero Kristo and Keerthy. Cc: Keerthy <j-keerthy@ti.com> Cc: Tero Kristo <kristo@kernel.org> Signed-off-by: Tony Lindgren <tony@atomide.com> --- arch/arm/boot/dts/dra7-l4.dtsi | 4 +- arch/arm/boot/dts/dra7.dtsi | 20 ++++++ drivers/clocksource/timer-ti-dm-systimer.c | 76 ++++++++++++++++++++++ include/linux/cpuhotplug.h | 1 + 4 files changed, 99 insertions(+), 2 deletions(-)