ARM: timer: Shutdown clock event device when stopping local timer
diff mbox

Message ID 1364549049-29278-1-git-send-email-ning.n.jiang@gmail.com
State New, archived
Headers show

Commit Message

ning.n.jiang@gmail.com March 29, 2013, 9:24 a.m. UTC
From: Ning Jiang <ning.n.jiang@gmail.com>

Currently there are two problems when we try to stop local timer.
First, it calls set_mode function directly so mode state is not
updated for the clock event device. Second, it makes the device
unused instead of shutdown.

A subtle error will happen because of it. When a cpu is plugged out
it will stop the local timer. It will call tick_nohz_idle_enter()
in idle thread afterwards. It will cancel the sched timer and try
to reprogram the next event. This is wrong since the local timer
is supposed to be stopped.

The right way to stop the local timer is to shutdown it by calling
clockevents_set_mode(). Thus when we try to reprogram the clock
event device, it will return directly without doing anything since
the clock mode is CLOCK_EVT_MODE_SHUTDOWN.

Signed-off-by: Ning Jiang <ning.n.jiang@gmail.com>
---
 arch/arm/kernel/smp_twd.c                |    2 +-
 arch/arm/mach-exynos/mct.c               |    2 +-
 arch/arm/mach-msm/timer.c                |    2 +-
 drivers/clocksource/arm_arch_timer.c     |    2 +-
 drivers/clocksource/time-armada-370-xp.c |    2 +-
 5 files changed, 5 insertions(+), 5 deletions(-)

Comments

Stephen Boyd March 29, 2013, 6:32 p.m. UTC | #1
On 03/29/13 02:24, ning.n.jiang@gmail.com wrote:
> From: Ning Jiang <ning.n.jiang@gmail.com>
>
> Currently there are two problems when we try to stop local timer.
> First, it calls set_mode function directly so mode state is not
> updated for the clock event device. Second, it makes the device
> unused instead of shutdown.

What device is this a problem on? I believe this only matters to drivers
which enable their timer in their set_next_event() callback? But even
then, does anything actually happen because the interrupt should have
been disabled in the local timer stop callback.

>
> A subtle error will happen because of it. When a cpu is plugged out
> it will stop the local timer. It will call tick_nohz_idle_enter()
> in idle thread afterwards. It will cancel the sched timer and try
> to reprogram the next event. This is wrong since the local timer
> is supposed to be stopped.
>
> The right way to stop the local timer is to shutdown it by calling
> clockevents_set_mode(). Thus when we try to reprogram the clock
> event device, it will return directly without doing anything since
> the clock mode is CLOCK_EVT_MODE_SHUTDOWN.

While this prevents the set_next_event() callback from being called on a
dying CPU, wouldn't it be better to fix this problem in the core code
once instead of fixing it many times in each local timer driver? It
doesn't seem to make much sense to program an event on a CPU that is
about to die, so why do we do that?
Daniel Lezcano March 31, 2013, 11:41 p.m. UTC | #2
On 03/29/2013 10:24 AM, ning.n.jiang@gmail.com wrote:
> From: Ning Jiang <ning.n.jiang@gmail.com>
> 
> Currently there are two problems when we try to stop local timer.
> First, it calls set_mode function directly so mode state is not
> updated for the clock event device. Second, it makes the device
> unused instead of shutdown.
> 
> A subtle error will happen because of it. When a cpu is plugged out
> it will stop the local timer. It will call tick_nohz_idle_enter()
> in idle thread afterwards. It will cancel the sched timer and try
> to reprogram the next event. This is wrong since the local timer
> is supposed to be stopped.
> 
> The right way to stop the local timer is to shutdown it by calling
> clockevents_set_mode(). Thus when we try to reprogram the clock
> event device, it will return directly without doing anything since
> the clock mode is CLOCK_EVT_MODE_SHUTDOWN.
> 
> Signed-off-by: Ning Jiang <ning.n.jiang@gmail.com>
> ---
>  arch/arm/kernel/smp_twd.c                |    2 +-
>  arch/arm/mach-exynos/mct.c               |    2 +-
>  arch/arm/mach-msm/timer.c                |    2 +-
>  drivers/clocksource/arm_arch_timer.c     |    2 +-
>  drivers/clocksource/time-armada-370-xp.c |    2 +-
>  5 files changed, 5 insertions(+), 5 deletions(-)
> 
> diff --git a/arch/arm/kernel/smp_twd.c b/arch/arm/kernel/smp_twd.c
> index 3f25650..c1d4ab4 100644
> --- a/arch/arm/kernel/smp_twd.c
> +++ b/arch/arm/kernel/smp_twd.c
> @@ -92,7 +92,7 @@ static int twd_timer_ack(void)
>  
>  static void twd_timer_stop(struct clock_event_device *clk)
>  {
> -	twd_set_mode(CLOCK_EVT_MODE_UNUSED, clk);
> +	clockevents_set_mode(clk, CLOCK_EVT_MODE_SHUTDOWN);
>  	disable_percpu_irq(clk->irq);

Wouldn't be clockevents_shutdown more adequate here ? The next event
will be also set.
Daniel Lezcano April 1, 2013, 12:06 a.m. UTC | #3
On 03/29/2013 10:24 AM, ning.n.jiang@gmail.com wrote:
> From: Ning Jiang <ning.n.jiang@gmail.com>
> 
> Currently there are two problems when we try to stop local timer.
> First, it calls set_mode function directly so mode state is not
> updated for the clock event device. Second, it makes the device
> unused instead of shutdown.
> 
> A subtle error will happen because of it. When a cpu is plugged out
> it will stop the local timer. It will call tick_nohz_idle_enter()
> in idle thread afterwards. It will cancel the sched timer and try
> to reprogram the next event. This is wrong since the local timer
> is supposed to be stopped.
> 
> The right way to stop the local timer is to shutdown it by calling
> clockevents_set_mode(). Thus when we try to reprogram the clock
> event device, it will return directly without doing anything since
> the clock mode is CLOCK_EVT_MODE_SHUTDOWN.
> 
> Signed-off-by: Ning Jiang <ning.n.jiang@gmail.com>
> ---


Don't you see a kernel BUG with this patch ?

In the clockevents code, there is:

/**
 * clockevents_notify - notification about relevant events
 */
void clockevents_notify(unsigned long reason, void *arg)
{
	...
	case CLOCK_EVT_NOTIFY_CPU_DEAD:
	...
                cpu = *((int *)arg);
                list_for_each_entry_safe(dev, tmp, &clockevent_devices,
					list) {
                       if (cpumask_test_cpu(cpu, dev->cpumask) &&
                            cpumask_weight(dev->cpumask) == 1 &&
                            !tick_is_broadcast_device(dev)) {
                                BUG_ON(dev->mode !=
					CLOCK_EVT_MODE_UNUSED);

				^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

                                list_del(&dev->list);
                        }
                }
                break;
	...
}

This is called triggered from hrtimer_cpu_notify with the CPU_DEAD event.
ning.n.jiang@gmail.com April 1, 2013, 1:56 a.m. UTC | #4
2013/4/1 Daniel Lezcano <daniel.lezcano@linaro.org>:
> On 03/29/2013 10:24 AM, ning.n.jiang@gmail.com wrote:
>> From: Ning Jiang <ning.n.jiang@gmail.com>
>>
>> Currently there are two problems when we try to stop local timer.
>> First, it calls set_mode function directly so mode state is not
>> updated for the clock event device. Second, it makes the device
>> unused instead of shutdown.
>>
>> A subtle error will happen because of it. When a cpu is plugged out
>> it will stop the local timer. It will call tick_nohz_idle_enter()
>> in idle thread afterwards. It will cancel the sched timer and try
>> to reprogram the next event. This is wrong since the local timer
>> is supposed to be stopped.
>>
>> The right way to stop the local timer is to shutdown it by calling
>> clockevents_set_mode(). Thus when we try to reprogram the clock
>> event device, it will return directly without doing anything since
>> the clock mode is CLOCK_EVT_MODE_SHUTDOWN.
>>
>> Signed-off-by: Ning Jiang <ning.n.jiang@gmail.com>
>> ---
>>  arch/arm/kernel/smp_twd.c                |    2 +-
>>  arch/arm/mach-exynos/mct.c               |    2 +-
>>  arch/arm/mach-msm/timer.c                |    2 +-
>>  drivers/clocksource/arm_arch_timer.c     |    2 +-
>>  drivers/clocksource/time-armada-370-xp.c |    2 +-
>>  5 files changed, 5 insertions(+), 5 deletions(-)
>>
>> diff --git a/arch/arm/kernel/smp_twd.c b/arch/arm/kernel/smp_twd.c
>> index 3f25650..c1d4ab4 100644
>> --- a/arch/arm/kernel/smp_twd.c
>> +++ b/arch/arm/kernel/smp_twd.c
>> @@ -92,7 +92,7 @@ static int twd_timer_ack(void)
>>
>>  static void twd_timer_stop(struct clock_event_device *clk)
>>  {
>> -     twd_set_mode(CLOCK_EVT_MODE_UNUSED, clk);
>> +     clockevents_set_mode(clk, CLOCK_EVT_MODE_SHUTDOWN);
>>       disable_percpu_irq(clk->irq);
>
> Wouldn't be clockevents_shutdown more adequate here ? The next event
> will be also set.

You're right. clockevents_shutdown seems more appropriate here. I'll
submit a revised patch for it.

> --
>  <http://www.linaro.org/> Linaro.org ? Open source software for ARM SoCs
>
> Follow Linaro:  <http://www.facebook.com/pages/Linaro> Facebook |
> <http://twitter.com/#!/linaroorg> Twitter |
> <http://www.linaro.org/linaro-blog/> Blog
>
--
To unsubscribe from this list: send the line "unsubscribe linux-samsung-soc" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
ning.n.jiang@gmail.com April 1, 2013, 2:05 a.m. UTC | #5
2013/4/1 Daniel Lezcano <daniel.lezcano@linaro.org>:
> On 03/29/2013 10:24 AM, ning.n.jiang@gmail.com wrote:
>> From: Ning Jiang <ning.n.jiang@gmail.com>
>>
>> Currently there are two problems when we try to stop local timer.
>> First, it calls set_mode function directly so mode state is not
>> updated for the clock event device. Second, it makes the device
>> unused instead of shutdown.
>>
>> A subtle error will happen because of it. When a cpu is plugged out
>> it will stop the local timer. It will call tick_nohz_idle_enter()
>> in idle thread afterwards. It will cancel the sched timer and try
>> to reprogram the next event. This is wrong since the local timer
>> is supposed to be stopped.
>>
>> The right way to stop the local timer is to shutdown it by calling
>> clockevents_set_mode(). Thus when we try to reprogram the clock
>> event device, it will return directly without doing anything since
>> the clock mode is CLOCK_EVT_MODE_SHUTDOWN.
>>
>> Signed-off-by: Ning Jiang <ning.n.jiang@gmail.com>
>> ---
>
>
> Don't you see a kernel BUG with this patch ?
>
> In the clockevents code, there is:
>
> /**
>  * clockevents_notify - notification about relevant events
>  */
> void clockevents_notify(unsigned long reason, void *arg)
> {
>         ...
>         case CLOCK_EVT_NOTIFY_CPU_DEAD:
>         ...
>                 cpu = *((int *)arg);
>                 list_for_each_entry_safe(dev, tmp, &clockevent_devices,
>                                         list) {
>                        if (cpumask_test_cpu(cpu, dev->cpumask) &&
>                             cpumask_weight(dev->cpumask) == 1 &&
>                             !tick_is_broadcast_device(dev)) {
>                                 BUG_ON(dev->mode !=
>                                         CLOCK_EVT_MODE_UNUSED);
>
>                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>
>                                 list_del(&dev->list);
>                         }
>                 }
>                 break;
>         ...
> }
>
> This is called triggered from hrtimer_cpu_notify with the CPU_DEAD event.
>

The clockevents_do_notify() in clockevents_notify() will call
tick_notify() which will call tick_shutdown() in turn. tick_shutdown()
will set clock event device mode to UNUSED. So no panic afterwards.

void clockevents_notify(unsigned long reason, void *arg)
{
        ...
        clockevents_do_notify(reason, arg);

        switch (reason) {
        case CLOCK_EVT_NOTIFY_CPU_DEAD:
                ...
                list_for_each_entry_safe(dev, tmp, &clockevent_devices, list) {
                        if (cpumask_test_cpu(cpu, dev->cpumask) &&
                            cpumask_weight(dev->cpumask) == 1 &&
                            !tick_is_broadcast_device(dev)) {
                                BUG_ON(dev->mode != CLOCK_EVT_MODE_UNUSED);
                                list_del(&dev->list);
                        }
                }
                ....
        }
        raw_spin_unlock_irqrestore(&clockevents_lock, flags);
}
--
To unsubscribe from this list: send the line "unsubscribe linux-samsung-soc" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
ning.n.jiang@gmail.com April 2, 2013, 12:01 p.m. UTC | #6
2013/4/1 Ning Jiang <ning.n.jiang@gmail.com>:
> 2013/4/1 Daniel Lezcano <daniel.lezcano@linaro.org>:
>> On 03/29/2013 10:24 AM, ning.n.jiang@gmail.com wrote:
>>> From: Ning Jiang <ning.n.jiang@gmail.com>
>>>
>>> Currently there are two problems when we try to stop local timer.
>>> First, it calls set_mode function directly so mode state is not
>>> updated for the clock event device. Second, it makes the device
>>> unused instead of shutdown.
>>>
>>> A subtle error will happen because of it. When a cpu is plugged out
>>> it will stop the local timer. It will call tick_nohz_idle_enter()
>>> in idle thread afterwards. It will cancel the sched timer and try
>>> to reprogram the next event. This is wrong since the local timer
>>> is supposed to be stopped.
>>>
>>> The right way to stop the local timer is to shutdown it by calling
>>> clockevents_set_mode(). Thus when we try to reprogram the clock
>>> event device, it will return directly without doing anything since
>>> the clock mode is CLOCK_EVT_MODE_SHUTDOWN.
>>>
>>> Signed-off-by: Ning Jiang <ning.n.jiang@gmail.com>
>>> ---
>>>  arch/arm/kernel/smp_twd.c                |    2 +-
>>>  arch/arm/mach-exynos/mct.c               |    2 +-
>>>  arch/arm/mach-msm/timer.c                |    2 +-
>>>  drivers/clocksource/arm_arch_timer.c     |    2 +-
>>>  drivers/clocksource/time-armada-370-xp.c |    2 +-
>>>  5 files changed, 5 insertions(+), 5 deletions(-)
>>>
>>> diff --git a/arch/arm/kernel/smp_twd.c b/arch/arm/kernel/smp_twd.c
>>> index 3f25650..c1d4ab4 100644
>>> --- a/arch/arm/kernel/smp_twd.c
>>> +++ b/arch/arm/kernel/smp_twd.c
>>> @@ -92,7 +92,7 @@ static int twd_timer_ack(void)
>>>
>>>  static void twd_timer_stop(struct clock_event_device *clk)
>>>  {
>>> -     twd_set_mode(CLOCK_EVT_MODE_UNUSED, clk);
>>> +     clockevents_set_mode(clk, CLOCK_EVT_MODE_SHUTDOWN);
>>>       disable_percpu_irq(clk->irq);
>>
>> Wouldn't be clockevents_shutdown more adequate here ? The next event
>> will be also set.
>
> You're right. clockevents_shutdown seems more appropriate here. I'll
> submit a revised patch for it.
>

Here attached the revised patch. Any more comments?

Patch
diff mbox

diff --git a/arch/arm/kernel/smp_twd.c b/arch/arm/kernel/smp_twd.c
index 3f25650..c1d4ab4 100644
--- a/arch/arm/kernel/smp_twd.c
+++ b/arch/arm/kernel/smp_twd.c
@@ -92,7 +92,7 @@  static int twd_timer_ack(void)
 
 static void twd_timer_stop(struct clock_event_device *clk)
 {
-	twd_set_mode(CLOCK_EVT_MODE_UNUSED, clk);
+	clockevents_set_mode(clk, CLOCK_EVT_MODE_SHUTDOWN);
 	disable_percpu_irq(clk->irq);
 }
 
diff --git a/arch/arm/mach-exynos/mct.c b/arch/arm/mach-exynos/mct.c
index c9d6650..b2d0d46 100644
--- a/arch/arm/mach-exynos/mct.c
+++ b/arch/arm/mach-exynos/mct.c
@@ -429,7 +429,7 @@  static int __cpuinit exynos4_local_timer_setup(struct clock_event_device *evt)
 static void exynos4_local_timer_stop(struct clock_event_device *evt)
 {
 	unsigned int cpu = smp_processor_id();
-	evt->set_mode(CLOCK_EVT_MODE_UNUSED, evt);
+	clockevents_set_mode(evt, CLOCK_EVT_MODE_SHUTDOWN);
 	if (mct_int_type == MCT_INT_SPI)
 		if (cpu == 0)
 			remove_irq(evt->irq, &mct_tick0_event_irq);
diff --git a/arch/arm/mach-msm/timer.c b/arch/arm/mach-msm/timer.c
index 2969027..9b0cc66 100644
--- a/arch/arm/mach-msm/timer.c
+++ b/arch/arm/mach-msm/timer.c
@@ -152,7 +152,7 @@  static int __cpuinit msm_local_timer_setup(struct clock_event_device *evt)
 
 static void msm_local_timer_stop(struct clock_event_device *evt)
 {
-	evt->set_mode(CLOCK_EVT_MODE_UNUSED, evt);
+	clockevents_set_mode(evt, CLOCK_EVT_MODE_SHUTDOWN);
 	disable_percpu_irq(evt->irq);
 }
 
diff --git a/drivers/clocksource/arm_arch_timer.c b/drivers/clocksource/arm_arch_timer.c
index d7ad425..beecb26 100644
--- a/drivers/clocksource/arm_arch_timer.c
+++ b/drivers/clocksource/arm_arch_timer.c
@@ -242,7 +242,7 @@  static void __cpuinit arch_timer_stop(struct clock_event_device *clk)
 			disable_percpu_irq(arch_timer_ppi[PHYS_NONSECURE_PPI]);
 	}
 
-	clk->set_mode(CLOCK_EVT_MODE_UNUSED, clk);
+	clockevents_set_mode(clk, CLOCK_EVT_MODE_SHUTDOWN);
 }
 
 static int __cpuinit arch_timer_cpu_notify(struct notifier_block *self,
diff --git a/drivers/clocksource/time-armada-370-xp.c b/drivers/clocksource/time-armada-370-xp.c
index 47a6730..bfb2971 100644
--- a/drivers/clocksource/time-armada-370-xp.c
+++ b/drivers/clocksource/time-armada-370-xp.c
@@ -201,7 +201,7 @@  static int __cpuinit armada_370_xp_timer_setup(struct clock_event_device *evt)
 
 static void  armada_370_xp_timer_stop(struct clock_event_device *evt)
 {
-	evt->set_mode(CLOCK_EVT_MODE_UNUSED, evt);
+	clockevents_set_mode(evt, CLOCK_EVT_MODE_SHUTDOWN);
 	disable_percpu_irq(evt->irq);
 }