[v2] ARM: avoid Cortex-A9 livelock on tight dmb loops
diff mbox series

Message ID E1gn8dp-0004Ai-AW@rmk-PC.armlinux.org.uk
State New
Headers show
Series
  • [v2] ARM: avoid Cortex-A9 livelock on tight dmb loops
Related show

Commit Message

Russell King Jan. 25, 2019, 9:03 p.m. UTC
Executing loops such as:

	while (1)
		cpu_relax();

with interrupts disabled results in a livelock of the entire system,
as other CPUs are prevented making progress.  This is most noticable
as a failure of crashdump kexec, which stops just after issuing:

	Loading crashdump kernel...

to the system console.  A workaround for this is to use 10 nops in
cpu_relax().

We also use wfe() in while (1) loops to avoid burning cycles in a
tight loop, giving the CPU a hint that we're not doing anything
useful.

Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>
---
It's been a while since this was posted, Will's suggestion was to use
10 nops in cpu_relax() last time around.  I still prefer wfe() in
these infinite-not-doing-anything-ever loops.

 arch/arm/include/asm/barrier.h   | 2 ++
 arch/arm/include/asm/processor.h | 6 +++++-
 arch/arm/kernel/machine_kexec.c  | 5 ++++-
 arch/arm/kernel/smp.c            | 4 +++-
 arch/arm/mach-omap2/prm_common.c | 4 +++-
 5 files changed, 17 insertions(+), 4 deletions(-)

Comments

Tony Lindgren Jan. 25, 2019, 11:20 p.m. UTC | #1
* Russell King <rmk+kernel@armlinux.org.uk> [190125 21:04]:
> Executing loops such as:
> 
> 	while (1)
> 		cpu_relax();
> 
> with interrupts disabled results in a livelock of the entire system,
> as other CPUs are prevented making progress.  This is most noticable
> as a failure of crashdump kexec, which stops just after issuing:
> 
> 	Loading crashdump kernel...
> 
> to the system console.  A workaround for this is to use 10 nops in
> cpu_relax().
> 
> We also use wfe() in while (1) loops to avoid burning cycles in a
> tight loop, giving the CPU a hint that we're not doing anything
> useful.
> 
> Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>
> ---
> It's been a while since this was posted, Will's suggestion was to use
> 10 nops in cpu_relax() last time around.  I still prefer wfe() in
> these infinite-not-doing-anything-ever loops.

Works for me:

Tested-by: Tony Lindgren <tony@atomide.com>
Paul Walmsley Jan. 26, 2019, 9 p.m. UTC | #2
On Fri, 25 Jan 2019, Tony Lindgren wrote:

> * Russell King <rmk+kernel@armlinux.org.uk> [190125 21:04]:
> > Executing loops such as:
> > 
> > 	while (1)
> > 		cpu_relax();
> > 
> > with interrupts disabled results in a livelock of the entire system,
> > as other CPUs are prevented making progress.  This is most noticable
> > as a failure of crashdump kexec, which stops just after issuing:
> > 
> > 	Loading crashdump kernel...
> > 
> > to the system console.  A workaround for this is to use 10 nops in
> > cpu_relax().
> > 
> > We also use wfe() in while (1) loops to avoid burning cycles in a
> > tight loop, giving the CPU a hint that we're not doing anything
> > useful.
> > 
> > Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>
> > ---
> > It's been a while since this was posted, Will's suggestion was to use
> > 10 nops in cpu_relax() last time around.  I still prefer wfe() in
> > these infinite-not-doing-anything-ever loops.
> 
> Works for me:
> 
> Tested-by: Tony Lindgren <tony@atomide.com>

There was some concern in the past that WFE, like WFI, might cause the 
core to assert an external signal that might cause the SoC integration to 
place the core into a low-power mode from which it might not be able to 
wake up.  This could happen on OMAP, for example, with WFI.

I don't recall the outcome of those discussions.  Was a conclusion ever 
reached?


- Paul
Russell King - ARM Linux admin Jan. 26, 2019, 11:51 p.m. UTC | #3
On Sat, Jan 26, 2019 at 09:00:03PM +0000, Paul Walmsley wrote:
> On Fri, 25 Jan 2019, Tony Lindgren wrote:
> 
> > * Russell King <rmk+kernel@armlinux.org.uk> [190125 21:04]:
> > > Executing loops such as:
> > > 
> > > 	while (1)
> > > 		cpu_relax();
> > > 
> > > with interrupts disabled results in a livelock of the entire system,
> > > as other CPUs are prevented making progress.  This is most noticable
> > > as a failure of crashdump kexec, which stops just after issuing:
> > > 
> > > 	Loading crashdump kernel...
> > > 
> > > to the system console.  A workaround for this is to use 10 nops in
> > > cpu_relax().
> > > 
> > > We also use wfe() in while (1) loops to avoid burning cycles in a
> > > tight loop, giving the CPU a hint that we're not doing anything
> > > useful.
> > > 
> > > Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>
> > > ---
> > > It's been a while since this was posted, Will's suggestion was to use
> > > 10 nops in cpu_relax() last time around.  I still prefer wfe() in
> > > these infinite-not-doing-anything-ever loops.
> > 
> > Works for me:
> > 
> > Tested-by: Tony Lindgren <tony@atomide.com>
> 
> There was some concern in the past that WFE, like WFI, might cause the 
> core to assert an external signal that might cause the SoC integration to 
> place the core into a low-power mode from which it might not be able to 
> wake up.  This could happen on OMAP, for example, with WFI.
> 
> I don't recall the outcome of those discussions.  Was a conclusion ever 
> reached?

First, we use WFE in spinlocks.  If WFE were to place the CPU in a
low power state that it may not be able to wake up from, all our
spinlocks would be unsafe.

Next, in all of the situations in this patch, we're executing an
infinite loop.  If it were to cause the core to go into a low power
mode, surely that's a good thing, rather than the core endlessly
executing NOPs?  The only way out of that is for the core to receive
a reset _anyway_.
Paul Walmsley Jan. 27, 2019, 1:15 a.m. UTC | #4
On Sat, 26 Jan 2019, Russell King - ARM Linux admin wrote:

> On Sat, Jan 26, 2019 at 09:00:03PM +0000, Paul Walmsley wrote:
> > On Fri, 25 Jan 2019, Tony Lindgren wrote:
> > 
> > > * Russell King <rmk+kernel@armlinux.org.uk> [190125 21:04]:
> > > > Executing loops such as:
> > > > 
> > > > 	while (1)
> > > > 		cpu_relax();
> > > > 
> > > > with interrupts disabled results in a livelock of the entire system,
> > > > as other CPUs are prevented making progress.  This is most noticable
> > > > as a failure of crashdump kexec, which stops just after issuing:
> > > > 
> > > > 	Loading crashdump kernel...
> > > > 
> > > > to the system console.  A workaround for this is to use 10 nops in
> > > > cpu_relax().
> > > > 
> > > > We also use wfe() in while (1) loops to avoid burning cycles in a
> > > > tight loop, giving the CPU a hint that we're not doing anything
> > > > useful.
> > > > 
> > > > Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>
> > > > ---
> > > > It's been a while since this was posted, Will's suggestion was to use
> > > > 10 nops in cpu_relax() last time around.  I still prefer wfe() in
> > > > these infinite-not-doing-anything-ever loops.
> > > 
> > > Works for me:
> > > 
> > > Tested-by: Tony Lindgren <tony@atomide.com>
> > 
> > There was some concern in the past that WFE, like WFI, might cause the 
> > core to assert an external signal that might cause the SoC integration to 
> > place the core into a low-power mode from which it might not be able to 
> > wake up.  This could happen on OMAP, for example, with WFI.
> > 
> > I don't recall the outcome of those discussions.  Was a conclusion ever 
> > reached?
> 
> First, we use WFE in spinlocks.  If WFE were to place the CPU in a
> low power state that it may not be able to wake up from, all our
> spinlocks would be unsafe.

Good point.  WFE must not assert the external signal that indicates 
that the core is inactive.

> Next, in all of the situations in this patch, we're executing an
> infinite loop.  If it were to cause the core to go into a low power
> mode, surely that's a good thing, rather than the core endlessly
> executing NOPs?  The only way out of that is for the core to receive
> a reset _anyway_.

Makes sense.  

Do you recall what Will's reasoning was for preferring 10 NOPs to a WFE?


- Paul
Russell King - ARM Linux admin Jan. 27, 2019, 3:28 p.m. UTC | #5
On Sun, Jan 27, 2019 at 01:15:31AM +0000, Paul Walmsley wrote:
> On Sat, 26 Jan 2019, Russell King - ARM Linux admin wrote:
> 
> > On Sat, Jan 26, 2019 at 09:00:03PM +0000, Paul Walmsley wrote:
> > > On Fri, 25 Jan 2019, Tony Lindgren wrote:
> > > 
> > > > * Russell King <rmk+kernel@armlinux.org.uk> [190125 21:04]:
> > > > > Executing loops such as:
> > > > > 
> > > > > 	while (1)
> > > > > 		cpu_relax();
> > > > > 
> > > > > with interrupts disabled results in a livelock of the entire system,
> > > > > as other CPUs are prevented making progress.  This is most noticable
> > > > > as a failure of crashdump kexec, which stops just after issuing:
> > > > > 
> > > > > 	Loading crashdump kernel...
> > > > > 
> > > > > to the system console.  A workaround for this is to use 10 nops in
> > > > > cpu_relax().
> > > > > 
> > > > > We also use wfe() in while (1) loops to avoid burning cycles in a
> > > > > tight loop, giving the CPU a hint that we're not doing anything
> > > > > useful.
> > > > > 
> > > > > Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>
> > > > > ---
> > > > > It's been a while since this was posted, Will's suggestion was to use
> > > > > 10 nops in cpu_relax() last time around.  I still prefer wfe() in
> > > > > these infinite-not-doing-anything-ever loops.
> > > > 
> > > > Works for me:
> > > > 
> > > > Tested-by: Tony Lindgren <tony@atomide.com>
> > > 
> > > There was some concern in the past that WFE, like WFI, might cause the 
> > > core to assert an external signal that might cause the SoC integration to 
> > > place the core into a low-power mode from which it might not be able to 
> > > wake up.  This could happen on OMAP, for example, with WFI.
> > > 
> > > I don't recall the outcome of those discussions.  Was a conclusion ever 
> > > reached?
> > 
> > First, we use WFE in spinlocks.  If WFE were to place the CPU in a
> > low power state that it may not be able to wake up from, all our
> > spinlocks would be unsafe.
> 
> Good point.  WFE must not assert the external signal that indicates 
> that the core is inactive.
> 
> > Next, in all of the situations in this patch, we're executing an
> > infinite loop.  If it were to cause the core to go into a low power
> > mode, surely that's a good thing, rather than the core endlessly
> > executing NOPs?  The only way out of that is for the core to receive
> > a reset _anyway_.
> 
> Makes sense.  
> 
> Do you recall what Will's reasoning was for preferring 10 NOPs to a WFE?

I think there may be an erratum for this which specifies 10 NOPs as
its workaround, but I don't have its number.
Will Deacon Jan. 31, 2019, 1:58 p.m. UTC | #6
Hi Russell, Paul,

On Sun, Jan 27, 2019 at 03:28:50PM +0000, Russell King - ARM Linux admin wrote:
> On Sun, Jan 27, 2019 at 01:15:31AM +0000, Paul Walmsley wrote:
> > On Sat, 26 Jan 2019, Russell King - ARM Linux admin wrote:
> > > On Sat, Jan 26, 2019 at 09:00:03PM +0000, Paul Walmsley wrote:
> > > > There was some concern in the past that WFE, like WFI, might cause the 
> > > > core to assert an external signal that might cause the SoC integration to 
> > > > place the core into a low-power mode from which it might not be able to 
> > > > wake up.  This could happen on OMAP, for example, with WFI.
> > > > 
> > > > I don't recall the outcome of those discussions.  Was a conclusion ever 
> > > > reached?
> > > 
> > > First, we use WFE in spinlocks.  If WFE were to place the CPU in a
> > > low power state that it may not be able to wake up from, all our
> > > spinlocks would be unsafe.
> > 
> > Good point.  WFE must not assert the external signal that indicates 
> > that the core is inactive.
> > 
> > > Next, in all of the situations in this patch, we're executing an
> > > infinite loop.  If it were to cause the core to go into a low power
> > > mode, surely that's a good thing, rather than the core endlessly
> > > executing NOPs?  The only way out of that is for the core to receive
> > > a reset _anyway_.
> > 
> > Makes sense.  
> > 
> > Do you recall what Will's reasoning was for preferring 10 NOPs to a WFE?
> 
> I think there may be an erratum for this which specifies 10 NOPs as
> its workaround, but I don't have its number.

The erratum hits because cpu_relax() is a DMB instruction due to erratum
754327. That then triggers erratum 794072 because a tight loop of DMB
instructions can cause a denial of service. One of the conditions for that
to occur is:

  | * No more than 10 instructions other than the DMB are executed between
  |   each DMB

Digging up the workaround:

  |  This erratum can be worked round by setting bit[4] of the undocumented
  |  Diagnostic Control Register to 1. This register is encoded as
  |  CP15 c15 0 c0 1. This bit can be written in Secure state only, with the
  |  following Read/Modify/Write code sequence:
  |
  |	MRC p15,0,rt,c15,c0,1
  |	ORR rt,rt,#0x10
  |	MCR p15,0,rt,c15,c0,1
  |
  |  When it is set, this bit causes the DMB instruction to be decoded and
  |  executed like a DSB. Using this software workaround is not expected to
  |  have any impact on the overall performance of the processor on a typical
  |  code base.
  |
  |  Other workarounds are also available for this erratum, to either prevent
  |  or interrupt the continuous stream of DMB instructions that causes the
  |  deadlock.
  |
  |  For example:
  |	* Inserting a non-conditional Load or Store instruction in the loop
  |	  between each DMB
  |	* Inserting additional instructions in the loop, such as NOPs, to
  |       avoid the processor seeing back to back DMB instructions.
  |	* Making the processor executing the short loop take regular
  |	  interrupts.

So the reason I prefer the NOPs is because that's guaranteed by the h/w folks
to do the trick, whereas they say nothing about WFE. It should be dead easy to
use NOPs instead, so I'm not sure why we're not just following the workaround
here. We could even use NOPs + WFE if you like!

Will "archaeologist" Deacon
Russell King - ARM Linux admin Jan. 31, 2019, 10:58 p.m. UTC | #7
On Thu, Jan 31, 2019 at 01:58:05PM +0000, Will Deacon wrote:
> Hi Russell, Paul,
> 
> On Sun, Jan 27, 2019 at 03:28:50PM +0000, Russell King - ARM Linux admin wrote:
> > On Sun, Jan 27, 2019 at 01:15:31AM +0000, Paul Walmsley wrote:
> > > On Sat, 26 Jan 2019, Russell King - ARM Linux admin wrote:
> > > > On Sat, Jan 26, 2019 at 09:00:03PM +0000, Paul Walmsley wrote:
> > > > > There was some concern in the past that WFE, like WFI, might cause the 
> > > > > core to assert an external signal that might cause the SoC integration to 
> > > > > place the core into a low-power mode from which it might not be able to 
> > > > > wake up.  This could happen on OMAP, for example, with WFI.
> > > > > 
> > > > > I don't recall the outcome of those discussions.  Was a conclusion ever 
> > > > > reached?
> > > > 
> > > > First, we use WFE in spinlocks.  If WFE were to place the CPU in a
> > > > low power state that it may not be able to wake up from, all our
> > > > spinlocks would be unsafe.
> > > 
> > > Good point.  WFE must not assert the external signal that indicates 
> > > that the core is inactive.
> > > 
> > > > Next, in all of the situations in this patch, we're executing an
> > > > infinite loop.  If it were to cause the core to go into a low power
> > > > mode, surely that's a good thing, rather than the core endlessly
> > > > executing NOPs?  The only way out of that is for the core to receive
> > > > a reset _anyway_.
> > > 
> > > Makes sense.  
> > > 
> > > Do you recall what Will's reasoning was for preferring 10 NOPs to a WFE?
> > 
> > I think there may be an erratum for this which specifies 10 NOPs as
> > its workaround, but I don't have its number.
> 
> The erratum hits because cpu_relax() is a DMB instruction due to erratum
> 754327. That then triggers erratum 794072 because a tight loop of DMB
> instructions can cause a denial of service. One of the conditions for that
> to occur is:
> 
>   | * No more than 10 instructions other than the DMB are executed between
>   |   each DMB
> 
> Digging up the workaround:
> 
>   |  This erratum can be worked round by setting bit[4] of the undocumented
>   |  Diagnostic Control Register to 1. This register is encoded as
>   |  CP15 c15 0 c0 1. This bit can be written in Secure state only, with the
>   |  following Read/Modify/Write code sequence:
>   |
>   |	MRC p15,0,rt,c15,c0,1
>   |	ORR rt,rt,#0x10
>   |	MCR p15,0,rt,c15,c0,1
>   |
>   |  When it is set, this bit causes the DMB instruction to be decoded and
>   |  executed like a DSB. Using this software workaround is not expected to
>   |  have any impact on the overall performance of the processor on a typical
>   |  code base.
>   |
>   |  Other workarounds are also available for this erratum, to either prevent
>   |  or interrupt the continuous stream of DMB instructions that causes the
>   |  deadlock.
>   |
>   |  For example:
>   |	* Inserting a non-conditional Load or Store instruction in the loop
>   |	  between each DMB
>   |	* Inserting additional instructions in the loop, such as NOPs, to
>   |       avoid the processor seeing back to back DMB instructions.
>   |	* Making the processor executing the short loop take regular
>   |	  interrupts.
> 
> So the reason I prefer the NOPs is because that's guaranteed by the h/w folks
> to do the trick, whereas they say nothing about WFE. It should be dead easy to
> use NOPs instead, so I'm not sure why we're not just following the workaround
> here. We could even use NOPs + WFE if you like!

Okay, let's start off at the beginning:

machine_crash_nonpanic_core() does this:

	while (1)
		cpu_relax();

because the kernel has crashed, and we have no known safe way to deal
with the CPU.  So, we place the CPU into an infinite loop which we
expect it to _never_ exit - at least not until the system as a whole is
reset by some method.

In the absence of 754327, this code assembles to:

	b	.

In other words, an infinite loop.  When 754327 is enabled, this
becomes:

1:	dmb
	b	1b

Now, it's been so long ago (the commit says April 2018), that I don't
remember _which_ of these triggered the problem in OMAP4 where, if a
crash is triggered, the system tries to kexec into the panic kernel,
but fails after taking the secondary CPU down - placing it into one
of these loops.

The test as working solution I came up with was to add wfe() to
these infinite loops thusly:

	while (1) {
		cpu_relax();
		wfe();
	}

which, without 754327 builds to:

1:	wfe
	b	1b

or with 754327 is enabled:

1:	dmb
	wfe
	b	1b

Adding "wfe" does two things - where we're running on bare metal, and
the processor implements "wfe", it stops us spinning endlessly in a
loop where we're never going to do any useful work.

If we hit one of these loops in a VM, it allows the CPU to be given
back to the hypervisor and rescheduled for other purposes (maybe a
different VM) rather than wasting CPU cycles inside a crashed VM.

However, in light of 794072, you decided you wanted to see 10 nops
as well - which is reasonable to cover the case where we have 754327
enabled _and_ we have a processor that doesn't implement the wfe
hint.

So, we now end up with:

1:	wfe
	b	1b

when 754327 is disabled, or:

1:	dmb
	nop
	nop
	nop
	nop
	nop
	nop
	nop
	nop
	nop
	nop
	wfe
	b	1b

when 754327 is enabled.  We also get the dmb + 10 nop sequence
elsewhere in the kernel, in terminating loops.

IMHO, this is reasonable - it means we get the workaround for 794072
when 754327 is enabled, but still relinquish the dead processor -
either by placing it in a lower power mode when wfe is implemented as
such or by returning it to the hypervisior, or in the case where wfe
is a no-op, we use the workaround specified in 794072 to avoid the
problem.

I personally see these as two entirely orthogonal problems - the 10
nops addresses 794072, and the wfe is an optimisation that makes the
system more efficient when crashed either in terms of power consumption
or by allowing the host/other VMs to make use of the CPU.

I don't see any reason not to use kexec() inside a VM - it has the
potential to provide automated recovery from a failure of the VMs
kernel with the opportunity for saving a crashdump of the failure.
A panic() with a reboot timeout won't do that, and reading the
libvirt documentation, setting on_reboot to "preserve" won't either
(the documentation states "The preserve action for an on_reboot event
is treated as a destroy".)  Surely it has to be a good thing to
avoiding having CPUs spinning inside a VM that is doing no useful
work.
Will Deacon Feb. 1, 2019, 10:19 a.m. UTC | #8
Hi Russell,

On Fri, Jan 25, 2019 at 09:03:57PM +0000, Russell King wrote:
> Executing loops such as:
> 
> 	while (1)
> 		cpu_relax();
> 
> with interrupts disabled results in a livelock of the entire system,
> as other CPUs are prevented making progress.  This is most noticable
> as a failure of crashdump kexec, which stops just after issuing:
> 
> 	Loading crashdump kernel...
> 
> to the system console.  A workaround for this is to use 10 nops in
> cpu_relax().
> 
> We also use wfe() in while (1) loops to avoid burning cycles in a
> tight loop, giving the CPU a hint that we're not doing anything
> useful.
> 
> Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>
> ---
> It's been a while since this was posted, Will's suggestion was to use
> 10 nops in cpu_relax() last time around.  I still prefer wfe() in
> these infinite-not-doing-anything-ever loops.
> 
>  arch/arm/include/asm/barrier.h   | 2 ++
>  arch/arm/include/asm/processor.h | 6 +++++-
>  arch/arm/kernel/machine_kexec.c  | 5 ++++-
>  arch/arm/kernel/smp.c            | 4 +++-
>  arch/arm/mach-omap2/prm_common.c | 4 +++-
>  5 files changed, 17 insertions(+), 4 deletions(-)

Thanks, this looks good to me and your explanation later in the thread makes
a lot of sense:

Acked-by: Will Deacon <will.deacon@arm.com>

Feel free to put some of the erratum writeup that I shared in the commit
message, if you like.

Will
Russell King - ARM Linux admin Feb. 1, 2019, 9:20 p.m. UTC | #9
Hi Will,

On Fri, Feb 01, 2019 at 10:19:19AM +0000, Will Deacon wrote:
> Hi Russell,
> 
> On Fri, Jan 25, 2019 at 09:03:57PM +0000, Russell King wrote:
> > Executing loops such as:
> > 
> > 	while (1)
> > 		cpu_relax();
> > 
> > with interrupts disabled results in a livelock of the entire system,
> > as other CPUs are prevented making progress.  This is most noticable
> > as a failure of crashdump kexec, which stops just after issuing:
> > 
> > 	Loading crashdump kernel...
> > 
> > to the system console.  A workaround for this is to use 10 nops in
> > cpu_relax().
> > 
> > We also use wfe() in while (1) loops to avoid burning cycles in a
> > tight loop, giving the CPU a hint that we're not doing anything
> > useful.
> > 
> > Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>
> > ---
> > It's been a while since this was posted, Will's suggestion was to use
> > 10 nops in cpu_relax() last time around.  I still prefer wfe() in
> > these infinite-not-doing-anything-ever loops.
> > 
> >  arch/arm/include/asm/barrier.h   | 2 ++
> >  arch/arm/include/asm/processor.h | 6 +++++-
> >  arch/arm/kernel/machine_kexec.c  | 5 ++++-
> >  arch/arm/kernel/smp.c            | 4 +++-
> >  arch/arm/mach-omap2/prm_common.c | 4 +++-
> >  5 files changed, 17 insertions(+), 4 deletions(-)
> 
> Thanks, this looks good to me and your explanation later in the thread makes
> a lot of sense:
> 
> Acked-by: Will Deacon <will.deacon@arm.com>
> 
> Feel free to put some of the erratum writeup that I shared in the commit
> message, if you like.

I think it may make more sense to use my writeup as a basis for a
better commit log that explains why we're doing what we're doing.

Patch
diff mbox series

diff --git a/arch/arm/include/asm/barrier.h b/arch/arm/include/asm/barrier.h
index 69772e742a0a..83ae97c049d9 100644
--- a/arch/arm/include/asm/barrier.h
+++ b/arch/arm/include/asm/barrier.h
@@ -11,6 +11,8 @@ 
 #define sev()	__asm__ __volatile__ ("sev" : : : "memory")
 #define wfe()	__asm__ __volatile__ ("wfe" : : : "memory")
 #define wfi()	__asm__ __volatile__ ("wfi" : : : "memory")
+#else
+#define wfe()	do { } while (0)
 #endif
 
 #if __LINUX_ARM_ARCH__ >= 7
diff --git a/arch/arm/include/asm/processor.h b/arch/arm/include/asm/processor.h
index 120f4c9bbfde..57fe73ea0f72 100644
--- a/arch/arm/include/asm/processor.h
+++ b/arch/arm/include/asm/processor.h
@@ -89,7 +89,11 @@  extern void release_thread(struct task_struct *);
 unsigned long get_wchan(struct task_struct *p);
 
 #if __LINUX_ARM_ARCH__ == 6 || defined(CONFIG_ARM_ERRATA_754327)
-#define cpu_relax()			smp_mb()
+#define cpu_relax()						\
+	do {							\
+		smp_mb();					\
+		__asm__ __volatile__("nop; nop; nop; nop; nop; nop; nop; nop; nop; nop;");	\
+	} while (0)
 #else
 #define cpu_relax()			barrier()
 #endif
diff --git a/arch/arm/kernel/machine_kexec.c b/arch/arm/kernel/machine_kexec.c
index dd2eb5f76b9f..76300f3813e8 100644
--- a/arch/arm/kernel/machine_kexec.c
+++ b/arch/arm/kernel/machine_kexec.c
@@ -91,8 +91,11 @@  void machine_crash_nonpanic_core(void *unused)
 
 	set_cpu_online(smp_processor_id(), false);
 	atomic_dec(&waiting_for_crash_ipi);
-	while (1)
+
+	while (1) {
 		cpu_relax();
+		wfe();
+	}
 }
 
 void crash_smp_send_stop(void)
diff --git a/arch/arm/kernel/smp.c b/arch/arm/kernel/smp.c
index ebac63fe458b..4e785d025771 100644
--- a/arch/arm/kernel/smp.c
+++ b/arch/arm/kernel/smp.c
@@ -595,8 +595,10 @@  static void ipi_cpu_stop(unsigned int cpu)
 	local_fiq_disable();
 	local_irq_disable();
 
-	while (1)
+	while (1) {
 		cpu_relax();
+		wfe();
+	}
 }
 
 static DEFINE_PER_CPU(struct completion *, cpu_completion);
diff --git a/arch/arm/mach-omap2/prm_common.c b/arch/arm/mach-omap2/prm_common.c
index 058a37e6d11c..fd6e0671f957 100644
--- a/arch/arm/mach-omap2/prm_common.c
+++ b/arch/arm/mach-omap2/prm_common.c
@@ -523,8 +523,10 @@  void omap_prm_reset_system(void)
 
 	prm_ll_data->reset_system();
 
-	while (1)
+	while (1) {
 		cpu_relax();
+		wfe();
+	}
 }
 
 /**