diff mbox series

[stable,v4.9,v2] arm64: entry: Place an SB sequence following an ERET instruction

Message ID 20200709195034.15185-1-f.fainelli@gmail.com (mailing list archive)
State New, archived
Headers show
Series [stable,v4.9,v2] arm64: entry: Place an SB sequence following an ERET instruction | expand

Commit Message

Florian Fainelli July 9, 2020, 7:50 p.m. UTC
From: Will Deacon <will.deacon@arm.com>

commit 679db70801da9fda91d26caf13bf5b5ccc74e8e8 upstream

Some CPUs can speculate past an ERET instruction and potentially perform
speculative accesses to memory before processing the exception return.
Since the register state is often controlled by a lower privilege level
at the point of an ERET, this could potentially be used as part of a
side-channel attack.

This patch emits an SB sequence after each ERET so that speculation is
held up on exception return.

Signed-off-by: Will Deacon <will.deacon@arm.com>
[florian: Adjust hyp-entry.S to account for the label
 added change to hyp/entry.S]
Signed-off-by: Florian Fainelli <f.fainelli@gmail.com>
---
Changes in v2:

- added missing hunk in hyp/entry.S per Will's feedback

 arch/arm64/kernel/entry.S      | 2 ++
 arch/arm64/kvm/hyp/entry.S     | 2 ++
 arch/arm64/kvm/hyp/hyp-entry.S | 4 ++++
 3 files changed, 8 insertions(+)

Comments

Sasha Levin July 11, 2020, 12:28 a.m. UTC | #1
On Thu, Jul 09, 2020 at 12:50:23PM -0700, Florian Fainelli wrote:
>From: Will Deacon <will.deacon@arm.com>
>
>commit 679db70801da9fda91d26caf13bf5b5ccc74e8e8 upstream
>
>Some CPUs can speculate past an ERET instruction and potentially perform
>speculative accesses to memory before processing the exception return.
>Since the register state is often controlled by a lower privilege level
>at the point of an ERET, this could potentially be used as part of a
>side-channel attack.
>
>This patch emits an SB sequence after each ERET so that speculation is
>held up on exception return.
>
>Signed-off-by: Will Deacon <will.deacon@arm.com>
>[florian: Adjust hyp-entry.S to account for the label
> added change to hyp/entry.S]
>Signed-off-by: Florian Fainelli <f.fainelli@gmail.com>

I've queued it up, thanks!
Greg Kroah-Hartman July 20, 2020, 1:04 p.m. UTC | #2
On Thu, Jul 09, 2020 at 12:50:23PM -0700, Florian Fainelli wrote:
> From: Will Deacon <will.deacon@arm.com>
> 
> commit 679db70801da9fda91d26caf13bf5b5ccc74e8e8 upstream
> 
> Some CPUs can speculate past an ERET instruction and potentially perform
> speculative accesses to memory before processing the exception return.
> Since the register state is often controlled by a lower privilege level
> at the point of an ERET, this could potentially be used as part of a
> side-channel attack.
> 
> This patch emits an SB sequence after each ERET so that speculation is
> held up on exception return.
> 
> Signed-off-by: Will Deacon <will.deacon@arm.com>
> [florian: Adjust hyp-entry.S to account for the label
>  added change to hyp/entry.S]
> Signed-off-by: Florian Fainelli <f.fainelli@gmail.com>
> ---
> Changes in v2:
> 
> - added missing hunk in hyp/entry.S per Will's feedback

What about 4.19.y and 4.14.y trees?  I can't take something for 4.9.y
and then have a regression if someone moves to a newer release, right?

thanks,

greg k-h
Florian Fainelli July 20, 2020, 6:26 p.m. UTC | #3
On 7/20/20 6:04 AM, Greg KH wrote:
> On Thu, Jul 09, 2020 at 12:50:23PM -0700, Florian Fainelli wrote:
>> From: Will Deacon <will.deacon@arm.com>
>>
>> commit 679db70801da9fda91d26caf13bf5b5ccc74e8e8 upstream
>>
>> Some CPUs can speculate past an ERET instruction and potentially perform
>> speculative accesses to memory before processing the exception return.
>> Since the register state is often controlled by a lower privilege level
>> at the point of an ERET, this could potentially be used as part of a
>> side-channel attack.
>>
>> This patch emits an SB sequence after each ERET so that speculation is
>> held up on exception return.
>>
>> Signed-off-by: Will Deacon <will.deacon@arm.com>
>> [florian: Adjust hyp-entry.S to account for the label
>>  added change to hyp/entry.S]
>> Signed-off-by: Florian Fainelli <f.fainelli@gmail.com>
>> ---
>> Changes in v2:
>>
>> - added missing hunk in hyp/entry.S per Will's feedback
> 
> What about 4.19.y and 4.14.y trees?  I can't take something for 4.9.y
> and then have a regression if someone moves to a newer release, right?

Sure, send you candidates for 4.14 and 4.19.
Florian Fainelli Aug. 6, 2020, 8 p.m. UTC | #4
On 7/20/2020 11:26 AM, Florian Fainelli wrote:
> On 7/20/20 6:04 AM, Greg KH wrote:
>> On Thu, Jul 09, 2020 at 12:50:23PM -0700, Florian Fainelli wrote:
>>> From: Will Deacon <will.deacon@arm.com>
>>>
>>> commit 679db70801da9fda91d26caf13bf5b5ccc74e8e8 upstream
>>>
>>> Some CPUs can speculate past an ERET instruction and potentially perform
>>> speculative accesses to memory before processing the exception return.
>>> Since the register state is often controlled by a lower privilege level
>>> at the point of an ERET, this could potentially be used as part of a
>>> side-channel attack.
>>>
>>> This patch emits an SB sequence after each ERET so that speculation is
>>> held up on exception return.
>>>
>>> Signed-off-by: Will Deacon <will.deacon@arm.com>
>>> [florian: Adjust hyp-entry.S to account for the label
>>>  added change to hyp/entry.S]
>>> Signed-off-by: Florian Fainelli <f.fainelli@gmail.com>
>>> ---
>>> Changes in v2:
>>>
>>> - added missing hunk in hyp/entry.S per Will's feedback
>>
>> What about 4.19.y and 4.14.y trees?  I can't take something for 4.9.y
>> and then have a regression if someone moves to a newer release, right?
> 
> Sure, send you candidates for 4.14 and 4.19.

Greg, did you have a chance to queue those changes for 4.9, 4.14 and 4.19?

https://lore.kernel.org/linux-arm-kernel/20200720182538.13304-1-f.fainelli@gmail.com/
https://lore.kernel.org/linux-arm-kernel/20200720182937.14099-1-f.fainelli@gmail.com/
https://lore.kernel.org/linux-arm-kernel/20200709195034.15185-1-f.fainelli@gmail.com/

Thanks
Greg Kroah-Hartman Aug. 7, 2020, 1:14 p.m. UTC | #5
On Thu, Aug 06, 2020 at 01:00:54PM -0700, Florian Fainelli wrote:
> 
> 
> On 7/20/2020 11:26 AM, Florian Fainelli wrote:
> > On 7/20/20 6:04 AM, Greg KH wrote:
> >> On Thu, Jul 09, 2020 at 12:50:23PM -0700, Florian Fainelli wrote:
> >>> From: Will Deacon <will.deacon@arm.com>
> >>>
> >>> commit 679db70801da9fda91d26caf13bf5b5ccc74e8e8 upstream
> >>>
> >>> Some CPUs can speculate past an ERET instruction and potentially perform
> >>> speculative accesses to memory before processing the exception return.
> >>> Since the register state is often controlled by a lower privilege level
> >>> at the point of an ERET, this could potentially be used as part of a
> >>> side-channel attack.
> >>>
> >>> This patch emits an SB sequence after each ERET so that speculation is
> >>> held up on exception return.
> >>>
> >>> Signed-off-by: Will Deacon <will.deacon@arm.com>
> >>> [florian: Adjust hyp-entry.S to account for the label
> >>>  added change to hyp/entry.S]
> >>> Signed-off-by: Florian Fainelli <f.fainelli@gmail.com>
> >>> ---
> >>> Changes in v2:
> >>>
> >>> - added missing hunk in hyp/entry.S per Will's feedback
> >>
> >> What about 4.19.y and 4.14.y trees?  I can't take something for 4.9.y
> >> and then have a regression if someone moves to a newer release, right?
> > 
> > Sure, send you candidates for 4.14 and 4.19.
> 
> Greg, did you have a chance to queue those changes for 4.9, 4.14 and 4.19?
> 
> https://lore.kernel.org/linux-arm-kernel/20200720182538.13304-1-f.fainelli@gmail.com/
> https://lore.kernel.org/linux-arm-kernel/20200720182937.14099-1-f.fainelli@gmail.com/
> https://lore.kernel.org/linux-arm-kernel/20200709195034.15185-1-f.fainelli@gmail.com/

Nope, I was waiting for Will's "ack" for these.

thanks,

greg k-h
Florian Fainelli Aug. 7, 2020, 6:17 p.m. UTC | #6
On 8/7/2020 6:14 AM, Greg KH wrote:
> On Thu, Aug 06, 2020 at 01:00:54PM -0700, Florian Fainelli wrote:
>>
>>
>> On 7/20/2020 11:26 AM, Florian Fainelli wrote:
>>> On 7/20/20 6:04 AM, Greg KH wrote:
>>>> On Thu, Jul 09, 2020 at 12:50:23PM -0700, Florian Fainelli wrote:
>>>>> From: Will Deacon <will.deacon@arm.com>
>>>>>
>>>>> commit 679db70801da9fda91d26caf13bf5b5ccc74e8e8 upstream
>>>>>
>>>>> Some CPUs can speculate past an ERET instruction and potentially perform
>>>>> speculative accesses to memory before processing the exception return.
>>>>> Since the register state is often controlled by a lower privilege level
>>>>> at the point of an ERET, this could potentially be used as part of a
>>>>> side-channel attack.
>>>>>
>>>>> This patch emits an SB sequence after each ERET so that speculation is
>>>>> held up on exception return.
>>>>>
>>>>> Signed-off-by: Will Deacon <will.deacon@arm.com>
>>>>> [florian: Adjust hyp-entry.S to account for the label
>>>>>  added change to hyp/entry.S]
>>>>> Signed-off-by: Florian Fainelli <f.fainelli@gmail.com>
>>>>> ---
>>>>> Changes in v2:
>>>>>
>>>>> - added missing hunk in hyp/entry.S per Will's feedback
>>>>
>>>> What about 4.19.y and 4.14.y trees?  I can't take something for 4.9.y
>>>> and then have a regression if someone moves to a newer release, right?
>>>
>>> Sure, send you candidates for 4.14 and 4.19.
>>
>> Greg, did you have a chance to queue those changes for 4.9, 4.14 and 4.19?
>>
>> https://lore.kernel.org/linux-arm-kernel/20200720182538.13304-1-f.fainelli@gmail.com/
>> https://lore.kernel.org/linux-arm-kernel/20200720182937.14099-1-f.fainelli@gmail.com/
>> https://lore.kernel.org/linux-arm-kernel/20200709195034.15185-1-f.fainelli@gmail.com/
> 
> Nope, I was waiting for Will's "ack" for these.

OK, Will, can you review those? Thanks
Florian Fainelli Aug. 13, 2020, 8:52 p.m. UTC | #7
On 8/7/2020 11:17 AM, Florian Fainelli wrote:
> 
> 
> On 8/7/2020 6:14 AM, Greg KH wrote:
>> On Thu, Aug 06, 2020 at 01:00:54PM -0700, Florian Fainelli wrote:
>>>
>>>
>>> On 7/20/2020 11:26 AM, Florian Fainelli wrote:
>>>> On 7/20/20 6:04 AM, Greg KH wrote:
>>>>> On Thu, Jul 09, 2020 at 12:50:23PM -0700, Florian Fainelli wrote:
>>>>>> From: Will Deacon <will.deacon@arm.com>
>>>>>>
>>>>>> commit 679db70801da9fda91d26caf13bf5b5ccc74e8e8 upstream
>>>>>>
>>>>>> Some CPUs can speculate past an ERET instruction and potentially perform
>>>>>> speculative accesses to memory before processing the exception return.
>>>>>> Since the register state is often controlled by a lower privilege level
>>>>>> at the point of an ERET, this could potentially be used as part of a
>>>>>> side-channel attack.
>>>>>>
>>>>>> This patch emits an SB sequence after each ERET so that speculation is
>>>>>> held up on exception return.
>>>>>>
>>>>>> Signed-off-by: Will Deacon <will.deacon@arm.com>
>>>>>> [florian: Adjust hyp-entry.S to account for the label
>>>>>>   added change to hyp/entry.S]
>>>>>> Signed-off-by: Florian Fainelli <f.fainelli@gmail.com>
>>>>>> ---
>>>>>> Changes in v2:
>>>>>>
>>>>>> - added missing hunk in hyp/entry.S per Will's feedback
>>>>>
>>>>> What about 4.19.y and 4.14.y trees?  I can't take something for 4.9.y
>>>>> and then have a regression if someone moves to a newer release, right?
>>>>
>>>> Sure, send you candidates for 4.14 and 4.19.
>>>
>>> Greg, did you have a chance to queue those changes for 4.9, 4.14 and 4.19?
>>>
>>> https://lore.kernel.org/linux-arm-kernel/20200720182538.13304-1-f.fainelli@gmail.com/
>>> https://lore.kernel.org/linux-arm-kernel/20200720182937.14099-1-f.fainelli@gmail.com/
>>> https://lore.kernel.org/linux-arm-kernel/20200709195034.15185-1-f.fainelli@gmail.com/
>>
>> Nope, I was waiting for Will's "ack" for these.
> 
> OK, Will, can you review those? Thanks

Will, can you please review those patches?
Will Deacon Aug. 21, 2020, 4:03 p.m. UTC | #8
On Fri, Aug 07, 2020 at 03:14:29PM +0200, Greg KH wrote:
> On Thu, Aug 06, 2020 at 01:00:54PM -0700, Florian Fainelli wrote:
> > 
> > 
> > On 7/20/2020 11:26 AM, Florian Fainelli wrote:
> > > On 7/20/20 6:04 AM, Greg KH wrote:
> > >> On Thu, Jul 09, 2020 at 12:50:23PM -0700, Florian Fainelli wrote:
> > >>> From: Will Deacon <will.deacon@arm.com>
> > >>>
> > >>> commit 679db70801da9fda91d26caf13bf5b5ccc74e8e8 upstream
> > >>>
> > >>> Some CPUs can speculate past an ERET instruction and potentially perform
> > >>> speculative accesses to memory before processing the exception return.
> > >>> Since the register state is often controlled by a lower privilege level
> > >>> at the point of an ERET, this could potentially be used as part of a
> > >>> side-channel attack.
> > >>>
> > >>> This patch emits an SB sequence after each ERET so that speculation is
> > >>> held up on exception return.
> > >>>
> > >>> Signed-off-by: Will Deacon <will.deacon@arm.com>
> > >>> [florian: Adjust hyp-entry.S to account for the label
> > >>>  added change to hyp/entry.S]
> > >>> Signed-off-by: Florian Fainelli <f.fainelli@gmail.com>
> > >>> ---
> > >>> Changes in v2:
> > >>>
> > >>> - added missing hunk in hyp/entry.S per Will's feedback
> > >>
> > >> What about 4.19.y and 4.14.y trees?  I can't take something for 4.9.y
> > >> and then have a regression if someone moves to a newer release, right?
> > > 
> > > Sure, send you candidates for 4.14 and 4.19.
> > 
> > Greg, did you have a chance to queue those changes for 4.9, 4.14 and 4.19?
> > 
> > https://lore.kernel.org/linux-arm-kernel/20200720182538.13304-1-f.fainelli@gmail.com/
> > https://lore.kernel.org/linux-arm-kernel/20200720182937.14099-1-f.fainelli@gmail.com/
> > https://lore.kernel.org/linux-arm-kernel/20200709195034.15185-1-f.fainelli@gmail.com/
> 
> Nope, I was waiting for Will's "ack" for these.

This patch doesn't even build for me (the 'sb' macro is not defined in 4.9),
and I really wonder why we bother backporting it at all. Nobody's ever shown
it to be a problem in practice, and it's clear that this is just being
submitted to tick a box rather than anything else (otherwise it would build,
right?).

So I'm not going to Ack any of them. As with a lot of this side-channel
stuff the cure is far worse than the disease.

Will
Florian Fainelli Aug. 21, 2020, 5:16 p.m. UTC | #9
On 8/21/20 9:03 AM, Will Deacon wrote:
> On Fri, Aug 07, 2020 at 03:14:29PM +0200, Greg KH wrote:
>> On Thu, Aug 06, 2020 at 01:00:54PM -0700, Florian Fainelli wrote:
>>>
>>>
>>> On 7/20/2020 11:26 AM, Florian Fainelli wrote:
>>>> On 7/20/20 6:04 AM, Greg KH wrote:
>>>>> On Thu, Jul 09, 2020 at 12:50:23PM -0700, Florian Fainelli wrote:
>>>>>> From: Will Deacon <will.deacon@arm.com>
>>>>>>
>>>>>> commit 679db70801da9fda91d26caf13bf5b5ccc74e8e8 upstream
>>>>>>
>>>>>> Some CPUs can speculate past an ERET instruction and potentially perform
>>>>>> speculative accesses to memory before processing the exception return.
>>>>>> Since the register state is often controlled by a lower privilege level
>>>>>> at the point of an ERET, this could potentially be used as part of a
>>>>>> side-channel attack.
>>>>>>
>>>>>> This patch emits an SB sequence after each ERET so that speculation is
>>>>>> held up on exception return.
>>>>>>
>>>>>> Signed-off-by: Will Deacon <will.deacon@arm.com>
>>>>>> [florian: Adjust hyp-entry.S to account for the label
>>>>>>  added change to hyp/entry.S]
>>>>>> Signed-off-by: Florian Fainelli <f.fainelli@gmail.com>
>>>>>> ---
>>>>>> Changes in v2:
>>>>>>
>>>>>> - added missing hunk in hyp/entry.S per Will's feedback
>>>>>
>>>>> What about 4.19.y and 4.14.y trees?  I can't take something for 4.9.y
>>>>> and then have a regression if someone moves to a newer release, right?
>>>>
>>>> Sure, send you candidates for 4.14 and 4.19.
>>>
>>> Greg, did you have a chance to queue those changes for 4.9, 4.14 and 4.19?
>>>
>>> https://lore.kernel.org/linux-arm-kernel/20200720182538.13304-1-f.fainelli@gmail.com/
>>> https://lore.kernel.org/linux-arm-kernel/20200720182937.14099-1-f.fainelli@gmail.com/
>>> https://lore.kernel.org/linux-arm-kernel/20200709195034.15185-1-f.fainelli@gmail.com/
>>
>> Nope, I was waiting for Will's "ack" for these.
> 
> This patch doesn't even build for me (the 'sb' macro is not defined in 4.9),
> and I really wonder why we bother backporting it at all. Nobody's ever shown
> it to be a problem in practice, and it's clear that this is just being
> submitted to tick a box rather than anything else (otherwise it would build,
> right?).

Doh, I completely missed submitting the patch this depended on that's
why I did not notice the build failure locally, sorry about that, what a
shame.

Would not be the same "tick a box" argument be used against your
original submission then? Sure, I have not been able to demonstrate in
real life this was a problem, however the same can be said about a lot
security related fixes.

What if it becomes exploitable in the future, would not it be nice to
have it in a 6 year LTS kernel?

> 
> So I'm not going to Ack any of them. As with a lot of this side-channel
> stuff the cure is far worse than the disease.
Assuming that my v3 does build correctly, which it will, would you be
keen on changing your position?
Will Deacon Aug. 24, 2020, 4:32 p.m. UTC | #10
Hi Florian,

On Fri, Aug 21, 2020 at 10:16:23AM -0700, Florian Fainelli wrote:
> On 8/21/20 9:03 AM, Will Deacon wrote:
> > On Fri, Aug 07, 2020 at 03:14:29PM +0200, Greg KH wrote:
> >> On Thu, Aug 06, 2020 at 01:00:54PM -0700, Florian Fainelli wrote:
> >>> Greg, did you have a chance to queue those changes for 4.9, 4.14 and 4.19?
> >>>
> >>> https://lore.kernel.org/linux-arm-kernel/20200720182538.13304-1-f.fainelli@gmail.com/
> >>> https://lore.kernel.org/linux-arm-kernel/20200720182937.14099-1-f.fainelli@gmail.com/
> >>> https://lore.kernel.org/linux-arm-kernel/20200709195034.15185-1-f.fainelli@gmail.com/
> >>
> >> Nope, I was waiting for Will's "ack" for these.
> > 
> > This patch doesn't even build for me (the 'sb' macro is not defined in 4.9),
> > and I really wonder why we bother backporting it at all. Nobody's ever shown
> > it to be a problem in practice, and it's clear that this is just being
> > submitted to tick a box rather than anything else (otherwise it would build,
> > right?).
> 
> Doh, I completely missed submitting the patch this depended on that's
> why I did not notice the build failure locally, sorry about that, what a
> shame.
> 
> Would not be the same "tick a box" argument be used against your
> original submission then? Sure, I have not been able to demonstrate in
> real life this was a problem, however the same can be said about a lot
> security related fixes.

Sort of, although I wrote the original patch because it was dead easy to do
and saved having to think too much about the problem, whereas the complexity
of backporting largerly diminishes that imo.

> What if it becomes exploitable in the future, would not it be nice to
> have it in a 6 year LTS kernel?

Even if people are stuck on an old LTS, they should still be taking the
regular updates for it, and we would obviously need to backport the fix if
it turned out to be exploitable (and hey, we could even test it then!).

> > So I'm not going to Ack any of them. As with a lot of this side-channel
> > stuff the cure is far worse than the disease.
> Assuming that my v3 does build correctly, which it will, would you be
> keen on changing your position?

Note that I'm not trying to block this patch from going in, I'm just saying
that I'm not supportive of it. Perhaps somebody from Arm can review it if
they think it's worth the effort.

Will
Florian Fainelli Aug. 24, 2020, 4:42 p.m. UTC | #11
On 8/24/2020 9:32 AM, Will Deacon wrote:
> Hi Florian,
> 
> On Fri, Aug 21, 2020 at 10:16:23AM -0700, Florian Fainelli wrote:
>> On 8/21/20 9:03 AM, Will Deacon wrote:
>>> On Fri, Aug 07, 2020 at 03:14:29PM +0200, Greg KH wrote:
>>>> On Thu, Aug 06, 2020 at 01:00:54PM -0700, Florian Fainelli wrote:
>>>>> Greg, did you have a chance to queue those changes for 4.9, 4.14 and 4.19?
>>>>>
>>>>> https://lore.kernel.org/linux-arm-kernel/20200720182538.13304-1-f.fainelli@gmail.com/
>>>>> https://lore.kernel.org/linux-arm-kernel/20200720182937.14099-1-f.fainelli@gmail.com/
>>>>> https://lore.kernel.org/linux-arm-kernel/20200709195034.15185-1-f.fainelli@gmail.com/
>>>>
>>>> Nope, I was waiting for Will's "ack" for these.
>>>
>>> This patch doesn't even build for me (the 'sb' macro is not defined in 4.9),
>>> and I really wonder why we bother backporting it at all. Nobody's ever shown
>>> it to be a problem in practice, and it's clear that this is just being
>>> submitted to tick a box rather than anything else (otherwise it would build,
>>> right?).
>>
>> Doh, I completely missed submitting the patch this depended on that's
>> why I did not notice the build failure locally, sorry about that, what a
>> shame.
>>
>> Would not be the same "tick a box" argument be used against your
>> original submission then? Sure, I have not been able to demonstrate in
>> real life this was a problem, however the same can be said about a lot
>> security related fixes.
> 
> Sort of, although I wrote the original patch because it was dead easy to do
> and saved having to think too much about the problem, whereas the complexity
> of backporting largerly diminishes that imo.
> 
>> What if it becomes exploitable in the future, would not it be nice to
>> have it in a 6 year LTS kernel?
> 
> Even if people are stuck on an old LTS, they should still be taking the
> regular updates for it, and we would obviously need to backport the fix if
> it turned out to be exploitable (and hey, we could even test it then!).
> 
>>> So I'm not going to Ack any of them. As with a lot of this side-channel
>>> stuff the cure is far worse than the disease.
>> Assuming that my v3 does build correctly, which it will, would you be
>> keen on changing your position?
> 
> Note that I'm not trying to block this patch from going in, I'm just saying
> that I'm not supportive of it. Perhaps somebody from Arm can review it if
> they think it's worth the effort.

How about I submit the actual full series (two patches) and we take the 
discussion from there?

Thanks for responding!
diff mbox series

Patch

diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S
index ca978d7d98eb..3408c782702c 100644
--- a/arch/arm64/kernel/entry.S
+++ b/arch/arm64/kernel/entry.S
@@ -255,6 +255,7 @@  alternative_insn eret, nop, ARM64_UNMAP_KERNEL_AT_EL0
 	.else
 	eret
 	.endif
+	sb
 	.endm
 
 	.macro	get_thread_info, rd
@@ -945,6 +946,7 @@  __ni_sys_trace:
 	mrs	x30, far_el1
 	.endif
 	eret
+	sb
 	.endm
 
 	.align	11
diff --git a/arch/arm64/kvm/hyp/entry.S b/arch/arm64/kvm/hyp/entry.S
index a360ac6e89e9..93704e6894d2 100644
--- a/arch/arm64/kvm/hyp/entry.S
+++ b/arch/arm64/kvm/hyp/entry.S
@@ -83,6 +83,7 @@  ENTRY(__guest_enter)
 
 	// Do not touch any register after this!
 	eret
+	sb
 ENDPROC(__guest_enter)
 
 ENTRY(__guest_exit)
@@ -195,4 +196,5 @@  alternative_endif
 	ldp	x0, x1, [sp], #16
 
 	eret
+	sb
 ENDPROC(__fpsimd_guest_restore)
diff --git a/arch/arm64/kvm/hyp/hyp-entry.S b/arch/arm64/kvm/hyp/hyp-entry.S
index bf4988f9dae8..3675e7f0ab72 100644
--- a/arch/arm64/kvm/hyp/hyp-entry.S
+++ b/arch/arm64/kvm/hyp/hyp-entry.S
@@ -97,6 +97,7 @@  el1_sync:				// Guest trapped into EL2
 	do_el2_call
 
 2:	eret
+	sb
 
 el1_hvc_guest:
 	/*
@@ -147,6 +148,7 @@  wa_epilogue:
 	mov	x0, xzr
 	add	sp, sp, #16
 	eret
+	sb
 
 el1_trap:
 	get_vcpu_ptr	x1, x0
@@ -198,6 +200,7 @@  el2_error:
 	b.ne	__hyp_panic
 	mov	x0, #(1 << ARM_EXIT_WITH_SERROR_BIT)
 	eret
+	sb
 
 ENTRY(__hyp_do_panic)
 	mov	lr, #(PSR_F_BIT | PSR_I_BIT | PSR_A_BIT | PSR_D_BIT |\
@@ -206,6 +209,7 @@  ENTRY(__hyp_do_panic)
 	ldr	lr, =panic
 	msr	elr_el2, lr
 	eret
+	sb
 ENDPROC(__hyp_do_panic)
 
 ENTRY(__hyp_panic)