| Message ID | 1448027861-21472-1-git-send-email-grygorii.strashko@ti.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
+ Thomas, Marc On 11/20/2015 5:57 AM, Grygorii Strashko wrote: > Now the System stall is observed on TI AM437x based board > (am437x-gp-evm) during resuming from System suspend when ARM Global > timer is selected as clocksource device - SysRq are working, but > nothing else. The reason of stall is that ARM Global timer loses its > contexts. > > The reason of stall is that ARM Global timer loses its contexts during > System suspend: > GT_CONTROL.TIMER_ENABLE = 0 (unbanked) > GT_COUNTERx = 0 > > Hence, update ARM Global timer driver to reflect above behaviour > - re-enable ARM Global timer on resume GT_CONTROL.TIMER_ENABLE = 1 > - ensure clocksource and clockevent devices have coresponding flags > (CLOCK_SOURCE_SUSPEND_NONSTOP and CLOCK_EVT_FEAT_C3STOP) set > depending on presence of "always-on" DT property. > Something which loses context in low power states can't be called "always-on" Also if the clock-soucre can't tick in the low power states then that device shouldn't be used as a clock-source. > CC: Arnd Bergmann <arnd@arndb.de> > Cc: John Stultz <john.stultz@linaro.org> > Cc: Felipe Balbi <balbi@ti.com> > Cc: Tony Lindgren <tony@atomide.com> > Cc: Santosh Shilimkar <ssantosh@kernel.org> > Signed-off-by: Grygorii Strashko <grygorii.strashko@ti.com> > --- > Changes in v2: > - suspend/resume simplified: nothing is stored any more and > ARM GT just re-enabled > Link on v1: > https://lkml.org/lkml/2015/11/13/456 > > drivers/clocksource/arm_global_timer.c | 16 ++++++++++++++++ > 1 file changed, 16 insertions(+) > > diff --git a/drivers/clocksource/arm_global_timer.c b/drivers/clocksource/arm_global_timer.c > index a2cb6fa..867e546 100644 > --- a/drivers/clocksource/arm_global_timer.c > +++ b/drivers/clocksource/arm_global_timer.c > @@ -51,6 +51,7 @@ static void __iomem *gt_base; > static unsigned long gt_clk_rate; > static int gt_ppi; > static struct clock_event_device __percpu *gt_evt; > +static bool gt_always_on; > > /* > * To get the value from the Global Timer Counter register proceed as follows: > @@ -168,6 +169,9 @@ static int gt_clockevents_init(struct clock_event_device *clk) > { > int cpu = smp_processor_id(); > > + if (!gt_always_on) > + clk->features |= CLOCK_EVT_FEAT_C3STOP; > + > clk->name = "arm_global_timer"; > clk->features = CLOCK_EVT_FEAT_PERIODIC | CLOCK_EVT_FEAT_ONESHOT | > CLOCK_EVT_FEAT_PERCPU; > @@ -195,12 +199,19 @@ static cycle_t gt_clocksource_read(struct clocksource *cs) > return gt_counter_read(); > } > > +static void gt_resume(struct clocksource *cs) > +{ > + /* re-enable timer on resume */ > + writel(GT_CONTROL_TIMER_ENABLE, gt_base + GT_CONTROL); > +} > + > static struct clocksource gt_clocksource = { > .name = "arm_global_timer", > .rating = 300, > .read = gt_clocksource_read, > .mask = CLOCKSOURCE_MASK(64), > .flags = CLOCK_SOURCE_IS_CONTINUOUS, > + .resume = gt_resume, > }; > > #ifdef CONFIG_CLKSRC_ARM_GLOBAL_TIMER_SCHED_CLOCK > @@ -218,6 +229,9 @@ static void __init gt_clocksource_init(void) > /* enables timer on all the cores */ > writel(GT_CONTROL_TIMER_ENABLE, gt_base + GT_CONTROL); > > + if (gt_always_on) > + gt_clocksource.flags |= CLOCK_SOURCE_SUSPEND_NONSTOP; > + > #ifdef CONFIG_CLKSRC_ARM_GLOBAL_TIMER_SCHED_CLOCK > sched_clock_register(gt_sched_clock_read, 64, gt_clk_rate); > #endif > @@ -289,6 +303,8 @@ static void __init global_timer_of_register(struct device_node *np) > goto out_clk; > } > > + gt_always_on = of_property_read_bool(np, "always-on"); > + > err = request_percpu_irq(gt_ppi, gt_clockevent_interrupt, > "gt", gt_evt); > if (err) { > Am really confused with this patch. Because 'C3STOP' is a clock-event flag which we use for cases where we have back-up broadcast timer which continue to tick in low power states. If the ARM global timer is considered as that device which actually doesn't tick then we are doomed. Clocksource device must be *CONTINUOUS* and if it is not then its not a viable device to be used as clock-source. May be am missing the context but this whole patch doesn't make sense to me. Regards, Santosh
Hi Santosh, On 11/20/2015 07:23 PM, santosh shilimkar wrote: > + Thomas, Marc > > On 11/20/2015 5:57 AM, Grygorii Strashko wrote: >> Now the System stall is observed on TI AM437x based board >> (am437x-gp-evm) during resuming from System suspend when ARM Global >> timer is selected as clocksource device - SysRq are working, but >> nothing else. The reason of stall is that ARM Global timer loses its >> contexts. >> >> The reason of stall is that ARM Global timer loses its contexts during >> System suspend: >> GT_CONTROL.TIMER_ENABLE = 0 (unbanked) >> GT_COUNTERx = 0 >> >> Hence, update ARM Global timer driver to reflect above behaviour >> - re-enable ARM Global timer on resume GT_CONTROL.TIMER_ENABLE = 1 >> - ensure clocksource and clockevent devices have coresponding flags >> (CLOCK_SOURCE_SUSPEND_NONSTOP and CLOCK_EVT_FEAT_C3STOP) set >> depending on presence of "always-on" DT property. >> > Something which loses context in low power states can't be > called "always-on" Sry, it's kinda new area for me and I could make mistakes. While working on this patch I've: - re-used implementation from ARM arch timer commit 82a5619410d4c4df65c04272db198eca5a867c18 Author: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> Date: Tue Apr 8 10:04:32 2014 +0100 clocksource: arch_arm_timer: Fix age-old arch timer C3STOP detection issue - and followed timekeeping.txt: "Typically the clock source is a monotonic, atomic counter which will provide n bits which count from 0 to 2^(n-1) and then wraps around to 0 and start over. It will ideally NEVER stop ticking as long as the system is running. It may stop during system suspend." ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ And that exactly my use-case: I'd like to use ARM GT as clocksource and with CPUIdle = n. But System suspend has to be allowed. > > Also if the clock-soucre can't tick in the low power states > then that device shouldn't be used as a clock-source. Agree. clocksource can't (except with suspend). Have I missed something? > >> CC: Arnd Bergmann <arnd@arndb.de> >> Cc: John Stultz <john.stultz@linaro.org> >> Cc: Felipe Balbi <balbi@ti.com> >> Cc: Tony Lindgren <tony@atomide.com> >> Cc: Santosh Shilimkar <ssantosh@kernel.org> >> Signed-off-by: Grygorii Strashko <grygorii.strashko@ti.com> >> --- >> Changes in v2: >> - suspend/resume simplified: nothing is stored any more and >> ARM GT just re-enabled >> Link on v1: >> https://lkml.org/lkml/2015/11/13/456 >> >> drivers/clocksource/arm_global_timer.c | 16 ++++++++++++++++ >> 1 file changed, 16 insertions(+) >> >> diff --git a/drivers/clocksource/arm_global_timer.c >> b/drivers/clocksource/arm_global_timer.c >> index a2cb6fa..867e546 100644 >> --- a/drivers/clocksource/arm_global_timer.c >> +++ b/drivers/clocksource/arm_global_timer.c >> @@ -51,6 +51,7 @@ static void __iomem *gt_base; >> static unsigned long gt_clk_rate; >> static int gt_ppi; >> static struct clock_event_device __percpu *gt_evt; >> +static bool gt_always_on; >> >> /* >> * To get the value from the Global Timer Counter register proceed >> as follows: >> @@ -168,6 +169,9 @@ static int gt_clockevents_init(struct >> clock_event_device *clk) >> { >> int cpu = smp_processor_id(); >> >> + if (!gt_always_on) >> + clk->features |= CLOCK_EVT_FEAT_C3STOP; >> + I can drop ^ >> clk->name = "arm_global_timer"; >> clk->features = CLOCK_EVT_FEAT_PERIODIC | CLOCK_EVT_FEAT_ONESHOT | >> CLOCK_EVT_FEAT_PERCPU; and change this as clk->features = CLOCK_EVT_FEAT_PERIODIC | CLOCK_EVT_FEAT_ONESHOT | CLOCK_EVT_FEAT_PERCPU | CLOCK_EVT_FEAT_C3STOP; >> @@ -195,12 +199,19 @@ static cycle_t gt_clocksource_read(struct >> clocksource *cs) >> return gt_counter_read(); >> } >> >> +static void gt_resume(struct clocksource *cs) >> +{ >> + /* re-enable timer on resume */ >> + writel(GT_CONTROL_TIMER_ENABLE, gt_base + GT_CONTROL); >> +} >> + >> static struct clocksource gt_clocksource = { >> .name = "arm_global_timer", >> .rating = 300, >> .read = gt_clocksource_read, >> .mask = CLOCKSOURCE_MASK(64), >> .flags = CLOCK_SOURCE_IS_CONTINUOUS, >> + .resume = gt_resume, >> }; >> >> #ifdef CONFIG_CLKSRC_ARM_GLOBAL_TIMER_SCHED_CLOCK >> @@ -218,6 +229,9 @@ static void __init gt_clocksource_init(void) >> /* enables timer on all the cores */ >> writel(GT_CONTROL_TIMER_ENABLE, gt_base + GT_CONTROL); >> >> + if (gt_always_on) >> + gt_clocksource.flags |= CLOCK_SOURCE_SUSPEND_NONSTOP; >> + Or, may just drop only this? ^ >> #ifdef CONFIG_CLKSRC_ARM_GLOBAL_TIMER_SCHED_CLOCK >> sched_clock_register(gt_sched_clock_read, 64, gt_clk_rate); >> #endif >> @@ -289,6 +303,8 @@ static void __init global_timer_of_register(struct >> device_node *np) >> goto out_clk; >> } >> >> + gt_always_on = of_property_read_bool(np, "always-on"); >> + >> err = request_percpu_irq(gt_ppi, gt_clockevent_interrupt, >> "gt", gt_evt); >> if (err) { >> > Am really confused with this patch. Because 'C3STOP' is a clock-event > flag which we use for cases where we have back-up broadcast timer which > continue to tick in low power states. Yes. And it will work with CPUIDLE only together with OMAP GP timer, which will be used as broadcast timer (usual configuration for OMAP4 where TWD is used as clockevent, OMAP GP timer as broadcast device and OMAP 32ksynctimer/counter_32 as clocksource). Internally I had to fix cpuidle driver to make it work properly with multiplatform/muliSoC build - https://git.ti.com/~gragst/ti-linux-kernel/gragsts-ti-linux-kernel/commit/143572633ff418548390827265d270a0cc915a81 The issue with ARM GT was discovered when It was enabled om AM437x SoC (which is UP). And, again, my target is CPUIDLE=n and SUSPEND=y. > > If the ARM global timer is considered as that device which actually > doesn't tick then we are doomed. Clocksource device must be *CONTINUOUS* > and if it is not then its not a viable device to be used as clock-source. > > May be am missing the context but this whole patch doesn't make sense > to me. If it's better, I can drop "always-on" related code and just add CLOCK_EVT_FEAT_C3STOP to clock_event_device gt_evt by default (as it's done in arch/arm/kernel/smp_twd.c).
On 20/11/15 18:35, Grygorii Strashko wrote: > Hi Santosh, > > On 11/20/2015 07:23 PM, santosh shilimkar wrote: >> + Thomas, Marc >> >> On 11/20/2015 5:57 AM, Grygorii Strashko wrote: >>> Now the System stall is observed on TI AM437x based board >>> (am437x-gp-evm) during resuming from System suspend when ARM Global >>> timer is selected as clocksource device - SysRq are working, but >>> nothing else. The reason of stall is that ARM Global timer loses its >>> contexts. >>> >>> The reason of stall is that ARM Global timer loses its contexts during >>> System suspend: >>> GT_CONTROL.TIMER_ENABLE = 0 (unbanked) >>> GT_COUNTERx = 0 >>> >>> Hence, update ARM Global timer driver to reflect above behaviour >>> - re-enable ARM Global timer on resume GT_CONTROL.TIMER_ENABLE = 1 >>> - ensure clocksource and clockevent devices have coresponding flags >>> (CLOCK_SOURCE_SUSPEND_NONSTOP and CLOCK_EVT_FEAT_C3STOP) set >>> depending on presence of "always-on" DT property. >>> >> Something which loses context in low power states can't be >> called "always-on" > > Sry, it's kinda new area for me and I could make mistakes. > > While working on this patch I've: > - re-used implementation from ARM arch timer > commit 82a5619410d4c4df65c04272db198eca5a867c18 > Author: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> > Date: Tue Apr 8 10:04:32 2014 +0100 > > clocksource: arch_arm_timer: Fix age-old arch timer C3STOP detection issue [...] This patch has a very specific purpose: instructing the core code that this timer will never stop ticking, ever. It is really targeted at virtual machines, whose timer is backed by the host timer, even when the VM is not running. Using it on actual hardware is may not be the best idea, specially in the presence of PM. Thanks, M.
On 11/20/2015 10:46 AM, Marc Zyngier wrote: > On 20/11/15 18:35, Grygorii Strashko wrote: >> Hi Santosh, >> >> On 11/20/2015 07:23 PM, santosh shilimkar wrote: >>> + Thomas, Marc >>> >>> On 11/20/2015 5:57 AM, Grygorii Strashko wrote: >>>> Now the System stall is observed on TI AM437x based board >>>> (am437x-gp-evm) during resuming from System suspend when ARM Global >>>> timer is selected as clocksource device - SysRq are working, but >>>> nothing else. The reason of stall is that ARM Global timer loses its >>>> contexts. >>>> >>>> The reason of stall is that ARM Global timer loses its contexts during >>>> System suspend: >>>> GT_CONTROL.TIMER_ENABLE = 0 (unbanked) >>>> GT_COUNTERx = 0 >>>> >>>> Hence, update ARM Global timer driver to reflect above behaviour >>>> - re-enable ARM Global timer on resume GT_CONTROL.TIMER_ENABLE = 1 >>>> - ensure clocksource and clockevent devices have coresponding flags >>>> (CLOCK_SOURCE_SUSPEND_NONSTOP and CLOCK_EVT_FEAT_C3STOP) set >>>> depending on presence of "always-on" DT property. >>>> >>> Something which loses context in low power states can't be >>> called "always-on" >> >> Sry, it's kinda new area for me and I could make mistakes. >> >> While working on this patch I've: >> - re-used implementation from ARM arch timer >> commit 82a5619410d4c4df65c04272db198eca5a867c18 >> Author: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> >> Date: Tue Apr 8 10:04:32 2014 +0100 >> >> clocksource: arch_arm_timer: Fix age-old arch timer C3STOP detection issue > > [...] > > This patch has a very specific purpose: instructing the core code that > this timer will never stop ticking, ever. It is really targeted at > virtual machines, whose timer is backed by the host timer, even when the > VM is not running. > > Using it on actual hardware is may not be the best idea, specially in > the presence of PM. > Exactly. Thanks for clarifying the commit Marc. Regards, Santosh
On Fri, Nov 20, 2015 at 10:35 AM, Grygorii Strashko <grygorii.strashko@ti.com> wrote: > Hi Santosh, > > On 11/20/2015 07:23 PM, santosh shilimkar wrote: >> + Thomas, Marc >> >> On 11/20/2015 5:57 AM, Grygorii Strashko wrote: >>> Now the System stall is observed on TI AM437x based board >>> (am437x-gp-evm) during resuming from System suspend when ARM Global >>> timer is selected as clocksource device - SysRq are working, but >>> nothing else. The reason of stall is that ARM Global timer loses its >>> contexts. >>> >>> The reason of stall is that ARM Global timer loses its contexts during >>> System suspend: >>> GT_CONTROL.TIMER_ENABLE = 0 (unbanked) >>> GT_COUNTERx = 0 >>> >>> Hence, update ARM Global timer driver to reflect above behaviour >>> - re-enable ARM Global timer on resume GT_CONTROL.TIMER_ENABLE = 1 >>> - ensure clocksource and clockevent devices have coresponding flags >>> (CLOCK_SOURCE_SUSPEND_NONSTOP and CLOCK_EVT_FEAT_C3STOP) set >>> depending on presence of "always-on" DT property. >>> >> Something which loses context in low power states can't be >> called "always-on" > > Sry, it's kinda new area for me and I could make mistakes. > > While working on this patch I've: > - re-used implementation from ARM arch timer > commit 82a5619410d4c4df65c04272db198eca5a867c18 > Author: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> > Date: Tue Apr 8 10:04:32 2014 +0100 > > clocksource: arch_arm_timer: Fix age-old arch timer C3STOP detection issue > > > - and followed timekeeping.txt: > "Typically the clock source is a monotonic, atomic counter which will provide > n bits which count from 0 to 2^(n-1) and then wraps around to 0 and start over. > It will ideally NEVER stop ticking as long as the system is running. It > may stop during system suspend." > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > > And that exactly my use-case: I'd like to use ARM GT as clocksource > and with CPUIdle = n. But System suspend has to be allowed. > > >> >> Also if the clock-soucre can't tick in the low power states >> then that device shouldn't be used as a clock-source. > > Agree. clocksource can't (except with suspend). Have I missed something? I think the point Stantosh is making is that if the clocksource stops in suspend (which is allowed) you should not be setting CLOCK_SOURCE_SUSPEND_NONSTOP (which promises the clocksource doesn't stop in suspend, so it can be used for suspend timing as well). The contradictory part in your patch is that you're also setting the clockevent logic as CLOCK_EVT_FEAT_C3STOP, which flags that the clockevent hardware might stop in low-power idle states (ie: not suspend, but while the system is running). Usually hardware that halts in low-power mode (like the apic on some x86 machines) is not also used for timekeeping (instead we use the hpet/acpi there). So its unlikely that the hardware both stays running through suspend, but also might halt in idle. That would be "unique". thanks -john
On Fri, Nov 20, 2015 at 10:46 AM, Marc Zyngier <marc.zyngier@arm.com> wrote: > > This patch has a very specific purpose: instructing the core code that > this timer will never stop ticking, ever. It is really targeted at > virtual machines, whose timer is backed by the host timer, even when the > VM is not running. > > Using it on actual hardware is may not be the best idea, specially in > the presence of PM. And actually, the CLOCK_SOURCE_SUSPEND_NONSTOP flag was added to support real hardware, so its not just limited to VMs. But that hardware does have to keep the lights on for the counter backing the clocksource, and I'm not sure how common it ever became. thanks -john
On 11/20/2015 08:52 PM, santosh shilimkar wrote: > On 11/20/2015 10:46 AM, Marc Zyngier wrote: >> On 20/11/15 18:35, Grygorii Strashko wrote: >>> Hi Santosh, >>> >>> On 11/20/2015 07:23 PM, santosh shilimkar wrote: >>>> + Thomas, Marc >>>> >>>> On 11/20/2015 5:57 AM, Grygorii Strashko wrote: >>>>> Now the System stall is observed on TI AM437x based board >>>>> (am437x-gp-evm) during resuming from System suspend when ARM Global >>>>> timer is selected as clocksource device - SysRq are working, but >>>>> nothing else. The reason of stall is that ARM Global timer loses its >>>>> contexts. >>>>> >>>>> The reason of stall is that ARM Global timer loses its contexts during >>>>> System suspend: >>>>> GT_CONTROL.TIMER_ENABLE = 0 (unbanked) >>>>> GT_COUNTERx = 0 >>>>> >>>>> Hence, update ARM Global timer driver to reflect above behaviour >>>>> - re-enable ARM Global timer on resume GT_CONTROL.TIMER_ENABLE = 1 >>>>> - ensure clocksource and clockevent devices have coresponding flags >>>>> (CLOCK_SOURCE_SUSPEND_NONSTOP and CLOCK_EVT_FEAT_C3STOP) set >>>>> depending on presence of "always-on" DT property. >>>>> >>>> Something which loses context in low power states can't be >>>> called "always-on" >>> >>> Sry, it's kinda new area for me and I could make mistakes. >>> >>> While working on this patch I've: >>> - re-used implementation from ARM arch timer >>> commit 82a5619410d4c4df65c04272db198eca5a867c18 >>> Author: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> >>> Date: Tue Apr 8 10:04:32 2014 +0100 >>> >>> clocksource: arch_arm_timer: Fix age-old arch timer C3STOP >>> detection issue >> >> [...] >> >> This patch has a very specific purpose: instructing the core code that >> this timer will never stop ticking, ever. It is really targeted at >> virtual machines, whose timer is backed by the host timer, even when the >> VM is not running. >> >> Using it on actual hardware is may not be the best idea, specially in >> the presence of PM. >> > Exactly. Thanks for clarifying the commit Marc. Thanks for your comments, but could you clarify pls - Can i use ARM GT for my use case CPUIDLE=n and SUSPEND=y, taking into account that System time can be corrected by RTC core during resume? (it will be used as clocksource only). And would it be reasonable to resend new version if i will drop all "always-on" related code?
On Fri, 20 Nov 2015, John Stultz wrote: > So its unlikely that the hardware both stays running through suspend, > but also might halt in idle. That would be "unique". The amount of creativity put into the next variants of differently broken timers is amazing. So I wouldn't be too surprised if such a thing actually surfaces. Thanks, tglx
On 11/20/2015 09:09 PM, John Stultz wrote: > On Fri, Nov 20, 2015 at 10:35 AM, Grygorii Strashko > <grygorii.strashko@ti.com> wrote: >> Hi Santosh, >> >> On 11/20/2015 07:23 PM, santosh shilimkar wrote: >>> + Thomas, Marc >>> >>> On 11/20/2015 5:57 AM, Grygorii Strashko wrote: >>>> Now the System stall is observed on TI AM437x based board >>>> (am437x-gp-evm) during resuming from System suspend when ARM Global >>>> timer is selected as clocksource device - SysRq are working, but >>>> nothing else. The reason of stall is that ARM Global timer loses its >>>> contexts. >>>> >>>> The reason of stall is that ARM Global timer loses its contexts during >>>> System suspend: >>>> GT_CONTROL.TIMER_ENABLE = 0 (unbanked) >>>> GT_COUNTERx = 0 >>>> >>>> Hence, update ARM Global timer driver to reflect above behaviour >>>> - re-enable ARM Global timer on resume GT_CONTROL.TIMER_ENABLE = 1 >>>> - ensure clocksource and clockevent devices have coresponding flags >>>> (CLOCK_SOURCE_SUSPEND_NONSTOP and CLOCK_EVT_FEAT_C3STOP) set >>>> depending on presence of "always-on" DT property. >>>> >>> Something which loses context in low power states can't be >>> called "always-on" >> >> Sry, it's kinda new area for me and I could make mistakes. >> >> While working on this patch I've: >> - re-used implementation from ARM arch timer >> commit 82a5619410d4c4df65c04272db198eca5a867c18 >> Author: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> >> Date: Tue Apr 8 10:04:32 2014 +0100 >> >> clocksource: arch_arm_timer: Fix age-old arch timer C3STOP detection issue >> >> >> - and followed timekeeping.txt: >> "Typically the clock source is a monotonic, atomic counter which will provide >> n bits which count from 0 to 2^(n-1) and then wraps around to 0 and start over. >> It will ideally NEVER stop ticking as long as the system is running. It >> may stop during system suspend." >> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ >> >> And that exactly my use-case: I'd like to use ARM GT as clocksource >> and with CPUIdle = n. But System suspend has to be allowed. >> >> >>> >>> Also if the clock-soucre can't tick in the low power states >>> then that device shouldn't be used as a clock-source. >> >> Agree. clocksource can't (except with suspend). Have I missed something? > > I think the point Stantosh is making is that if the clocksource stops > in suspend (which is allowed) you should not be setting > CLOCK_SOURCE_SUSPEND_NONSTOP (which promises the clocksource doesn't > stop in suspend, so it can be used for suspend timing as well). > Ok. Thanks. I got it now. Adding CLOCK_SOURCE_SUSPEND_NONSTOP is mistake. > The contradictory part in your patch is that you're also setting the > clockevent logic as CLOCK_EVT_FEAT_C3STOP, which flags that the > clockevent hardware might stop in low-power idle states (ie: not > suspend, but while the system is running). Usually hardware that > halts in low-power mode (like the apic on some x86 machines) is not > also used for timekeeping (instead we use the hpet/acpi there). Sry, I've set CLOCK_EVT_FEAT_C3STOP if "always-on" = *false* ^ this, I thought, might be valid because it will stop in low-power idle states and "always-on" was expected to indicate case when CPUIdle = n (no CPU PM), same way as Lorenzo did. Would it be correct to just set it always? and CLOCK_SOURCE_SUSPEND_NONSTOP if "always-on" = *true* ^ this part is mistake > > So its unlikely that the hardware both stays running through suspend, > but also might halt in idle. That would be "unique". > Thanks a lot.
On Fri, Nov 20, 2015 at 11:28 AM, Grygorii Strashko <grygorii.strashko@ti.com> wrote: > On 11/20/2015 09:09 PM, John Stultz wrote: >> On Fri, Nov 20, 2015 at 10:35 AM, Grygorii Strashko >> <grygorii.strashko@ti.com> wrote: >>> Hi Santosh, >>> >>> On 11/20/2015 07:23 PM, santosh shilimkar wrote: >>>> + Thomas, Marc >>>> >>>> On 11/20/2015 5:57 AM, Grygorii Strashko wrote: >>>>> Now the System stall is observed on TI AM437x based board >>>>> (am437x-gp-evm) during resuming from System suspend when ARM Global >>>>> timer is selected as clocksource device - SysRq are working, but >>>>> nothing else. The reason of stall is that ARM Global timer loses its >>>>> contexts. >>>>> >>>>> The reason of stall is that ARM Global timer loses its contexts during >>>>> System suspend: >>>>> GT_CONTROL.TIMER_ENABLE = 0 (unbanked) >>>>> GT_COUNTERx = 0 >>>>> >>>>> Hence, update ARM Global timer driver to reflect above behaviour >>>>> - re-enable ARM Global timer on resume GT_CONTROL.TIMER_ENABLE = 1 >>>>> - ensure clocksource and clockevent devices have coresponding flags >>>>> (CLOCK_SOURCE_SUSPEND_NONSTOP and CLOCK_EVT_FEAT_C3STOP) set >>>>> depending on presence of "always-on" DT property. >>>>> >>>> Something which loses context in low power states can't be >>>> called "always-on" >>> >>> Sry, it's kinda new area for me and I could make mistakes. >>> >>> While working on this patch I've: >>> - re-used implementation from ARM arch timer >>> commit 82a5619410d4c4df65c04272db198eca5a867c18 >>> Author: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> >>> Date: Tue Apr 8 10:04:32 2014 +0100 >>> >>> clocksource: arch_arm_timer: Fix age-old arch timer C3STOP detection issue >>> >>> >>> - and followed timekeeping.txt: >>> "Typically the clock source is a monotonic, atomic counter which will provide >>> n bits which count from 0 to 2^(n-1) and then wraps around to 0 and start over. >>> It will ideally NEVER stop ticking as long as the system is running. It >>> may stop during system suspend." >>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ >>> >>> And that exactly my use-case: I'd like to use ARM GT as clocksource >>> and with CPUIdle = n. But System suspend has to be allowed. >>> >>> >>>> >>>> Also if the clock-soucre can't tick in the low power states >>>> then that device shouldn't be used as a clock-source. >>> >>> Agree. clocksource can't (except with suspend). Have I missed something? >> >> I think the point Stantosh is making is that if the clocksource stops >> in suspend (which is allowed) you should not be setting >> CLOCK_SOURCE_SUSPEND_NONSTOP (which promises the clocksource doesn't >> stop in suspend, so it can be used for suspend timing as well). >> > > Ok. Thanks. I got it now. Adding CLOCK_SOURCE_SUSPEND_NONSTOP is mistake. > >> The contradictory part in your patch is that you're also setting the >> clockevent logic as CLOCK_EVT_FEAT_C3STOP, which flags that the >> clockevent hardware might stop in low-power idle states (ie: not >> suspend, but while the system is running). Usually hardware that >> halts in low-power mode (like the apic on some x86 machines) is not >> also used for timekeeping (instead we use the hpet/acpi there). > > Sry, I've set CLOCK_EVT_FEAT_C3STOP if "always-on" = *false* You might also consider renaming that value from always_on to something more descriptive, given the subtlety of the different states here. Maybe instead use a flag called halts_in_idle or something? thanks -john
On 11/20/2015 09:32 PM, John Stultz wrote: > On Fri, Nov 20, 2015 at 11:28 AM, Grygorii Strashko > <grygorii.strashko@ti.com> wrote: >> On 11/20/2015 09:09 PM, John Stultz wrote: >>> On Fri, Nov 20, 2015 at 10:35 AM, Grygorii Strashko >>> <grygorii.strashko@ti.com> wrote: >>>> Hi Santosh, >>>> >>>> On 11/20/2015 07:23 PM, santosh shilimkar wrote: >>>>> + Thomas, Marc >>>>> >>>>> On 11/20/2015 5:57 AM, Grygorii Strashko wrote: >>>>>> Now the System stall is observed on TI AM437x based board >>>>>> (am437x-gp-evm) during resuming from System suspend when ARM Global >>>>>> timer is selected as clocksource device - SysRq are working, but >>>>>> nothing else. The reason of stall is that ARM Global timer loses its >>>>>> contexts. >>>>>> >>>>>> The reason of stall is that ARM Global timer loses its contexts during >>>>>> System suspend: >>>>>> GT_CONTROL.TIMER_ENABLE = 0 (unbanked) >>>>>> GT_COUNTERx = 0 >>>>>> >>>>>> Hence, update ARM Global timer driver to reflect above behaviour >>>>>> - re-enable ARM Global timer on resume GT_CONTROL.TIMER_ENABLE = 1 >>>>>> - ensure clocksource and clockevent devices have coresponding flags >>>>>> (CLOCK_SOURCE_SUSPEND_NONSTOP and CLOCK_EVT_FEAT_C3STOP) set >>>>>> depending on presence of "always-on" DT property. >>>>>> >>>>> Something which loses context in low power states can't be >>>>> called "always-on" >>>> >>>> Sry, it's kinda new area for me and I could make mistakes. >>>> >>>> While working on this patch I've: >>>> - re-used implementation from ARM arch timer >>>> commit 82a5619410d4c4df65c04272db198eca5a867c18 >>>> Author: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> >>>> Date: Tue Apr 8 10:04:32 2014 +0100 >>>> >>>> clocksource: arch_arm_timer: Fix age-old arch timer C3STOP detection issue >>>> >>>> >>>> - and followed timekeeping.txt: >>>> "Typically the clock source is a monotonic, atomic counter which will provide >>>> n bits which count from 0 to 2^(n-1) and then wraps around to 0 and start over. >>>> It will ideally NEVER stop ticking as long as the system is running. It >>>> may stop during system suspend." >>>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ >>>> >>>> And that exactly my use-case: I'd like to use ARM GT as clocksource >>>> and with CPUIdle = n. But System suspend has to be allowed. >>>> >>>> >>>>> >>>>> Also if the clock-soucre can't tick in the low power states >>>>> then that device shouldn't be used as a clock-source. >>>> >>>> Agree. clocksource can't (except with suspend). Have I missed something? >>> >>> I think the point Stantosh is making is that if the clocksource stops >>> in suspend (which is allowed) you should not be setting >>> CLOCK_SOURCE_SUSPEND_NONSTOP (which promises the clocksource doesn't >>> stop in suspend, so it can be used for suspend timing as well). >>> >> >> Ok. Thanks. I got it now. Adding CLOCK_SOURCE_SUSPEND_NONSTOP is mistake. >> >>> The contradictory part in your patch is that you're also setting the >>> clockevent logic as CLOCK_EVT_FEAT_C3STOP, which flags that the >>> clockevent hardware might stop in low-power idle states (ie: not >>> suspend, but while the system is running). Usually hardware that >>> halts in low-power mode (like the apic on some x86 machines) is not >>> also used for timekeeping (instead we use the hpet/acpi there). >> >> Sry, I've set CLOCK_EVT_FEAT_C3STOP if "always-on" = *false* > > You might also consider renaming that value from always_on to > something more descriptive, given the subtlety of the different states > here. Maybe instead use a flag called halts_in_idle or something? > I'd better just drop it for now hence I'm still not sure what is more reasonable continue with DT property or just add this flag always.
diff --git a/drivers/clocksource/arm_global_timer.c b/drivers/clocksource/arm_global_timer.c index a2cb6fa..867e546 100644 --- a/drivers/clocksource/arm_global_timer.c +++ b/drivers/clocksource/arm_global_timer.c @@ -51,6 +51,7 @@ static void __iomem *gt_base; static unsigned long gt_clk_rate; static int gt_ppi; static struct clock_event_device __percpu *gt_evt; +static bool gt_always_on; /* * To get the value from the Global Timer Counter register proceed as follows: @@ -168,6 +169,9 @@ static int gt_clockevents_init(struct clock_event_device *clk) { int cpu = smp_processor_id(); + if (!gt_always_on) + clk->features |= CLOCK_EVT_FEAT_C3STOP; + clk->name = "arm_global_timer"; clk->features = CLOCK_EVT_FEAT_PERIODIC | CLOCK_EVT_FEAT_ONESHOT | CLOCK_EVT_FEAT_PERCPU; @@ -195,12 +199,19 @@ static cycle_t gt_clocksource_read(struct clocksource *cs) return gt_counter_read(); } +static void gt_resume(struct clocksource *cs) +{ + /* re-enable timer on resume */ + writel(GT_CONTROL_TIMER_ENABLE, gt_base + GT_CONTROL); +} + static struct clocksource gt_clocksource = { .name = "arm_global_timer", .rating = 300, .read = gt_clocksource_read, .mask = CLOCKSOURCE_MASK(64), .flags = CLOCK_SOURCE_IS_CONTINUOUS, + .resume = gt_resume, }; #ifdef CONFIG_CLKSRC_ARM_GLOBAL_TIMER_SCHED_CLOCK @@ -218,6 +229,9 @@ static void __init gt_clocksource_init(void) /* enables timer on all the cores */ writel(GT_CONTROL_TIMER_ENABLE, gt_base + GT_CONTROL); + if (gt_always_on) + gt_clocksource.flags |= CLOCK_SOURCE_SUSPEND_NONSTOP; + #ifdef CONFIG_CLKSRC_ARM_GLOBAL_TIMER_SCHED_CLOCK sched_clock_register(gt_sched_clock_read, 64, gt_clk_rate); #endif @@ -289,6 +303,8 @@ static void __init global_timer_of_register(struct device_node *np) goto out_clk; } + gt_always_on = of_property_read_bool(np, "always-on"); + err = request_percpu_irq(gt_ppi, gt_clockevent_interrupt, "gt", gt_evt); if (err) {
Now the System stall is observed on TI AM437x based board (am437x-gp-evm) during resuming from System suspend when ARM Global timer is selected as clocksource device - SysRq are working, but nothing else. The reason of stall is that ARM Global timer loses its contexts. The reason of stall is that ARM Global timer loses its contexts during System suspend: GT_CONTROL.TIMER_ENABLE = 0 (unbanked) GT_COUNTERx = 0 Hence, update ARM Global timer driver to reflect above behaviour - re-enable ARM Global timer on resume GT_CONTROL.TIMER_ENABLE = 1 - ensure clocksource and clockevent devices have coresponding flags (CLOCK_SOURCE_SUSPEND_NONSTOP and CLOCK_EVT_FEAT_C3STOP) set depending on presence of "always-on" DT property. CC: Arnd Bergmann <arnd@arndb.de> Cc: John Stultz <john.stultz@linaro.org> Cc: Felipe Balbi <balbi@ti.com> Cc: Tony Lindgren <tony@atomide.com> Cc: Santosh Shilimkar <ssantosh@kernel.org> Signed-off-by: Grygorii Strashko <grygorii.strashko@ti.com> --- Changes in v2: - suspend/resume simplified: nothing is stored any more and ARM GT just re-enabled Link on v1: https://lkml.org/lkml/2015/11/13/456 drivers/clocksource/arm_global_timer.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+)