diff mbox

[14/18] arm64: erratum: Work around Falkor erratum #E1003 in trampoline code

Message ID 1510942921-12564-15-git-send-email-will.deacon@arm.com (mailing list archive)
State New, archived
Headers show

Commit Message

Will Deacon Nov. 17, 2017, 6:21 p.m. UTC
We rely on an atomic swizzling of TTBR1 when transitioning from the entry
trampoline to the kernel proper on an exception. We can't rely on this
atomicity in the face of Falkor erratum #E1003, so on affected cores we
can issue a TLB invalidation prior to jumping into the kernel. There is
still the possibility of a TLB conflict here due to conflicting walk
cache entries, but this doesn't appear to be the case on these CPUs in
practice.

Signed-off-by: Will Deacon <will.deacon@arm.com>
---
 arch/arm64/Kconfig        | 17 +++++------------
 arch/arm64/kernel/entry.S |  8 ++++++++
 2 files changed, 13 insertions(+), 12 deletions(-)

Comments

Stephen Boyd Nov. 18, 2017, 12:27 a.m. UTC | #1
On 11/17, Will Deacon wrote:
> We rely on an atomic swizzling of TTBR1 when transitioning from the entry
> trampoline to the kernel proper on an exception. We can't rely on this
> atomicity in the face of Falkor erratum #E1003, so on affected cores we
> can issue a TLB invalidation prior to jumping into the kernel. There is
> still the possibility of a TLB conflict here due to conflicting walk
> cache entries, but this doesn't appear to be the case on these CPUs in
> practice.
> 
> Signed-off-by: Will Deacon <will.deacon@arm.com>
> ---
>  arch/arm64/Kconfig        | 17 +++++------------
>  arch/arm64/kernel/entry.S |  8 ++++++++
>  2 files changed, 13 insertions(+), 12 deletions(-)
> 
> diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
> index 0df64a6a56d4..f0fcbfc2262e 100644
> --- a/arch/arm64/Kconfig
> +++ b/arch/arm64/Kconfig
> @@ -504,20 +504,13 @@ config CAVIUM_ERRATUM_30115
>  config QCOM_FALKOR_ERRATUM_1003
>  	bool "Falkor E1003: Incorrect translation due to ASID change"
>  	default y
> -	select ARM64_PAN if ARM64_SW_TTBR0_PAN

Cool, this sort of complicates the backport of the Kryo MIDR
update of this errata to stable trees though.

> diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S
> index a839b94bba05..a600879939ce 100644
> --- a/arch/arm64/kernel/entry.S
> +++ b/arch/arm64/kernel/entry.S
> @@ -941,6 +941,14 @@ __ni_sys_trace:
>  	sub	\tmp, \tmp, #(SWAPPER_DIR_SIZE + RESERVED_TTBR0_SIZE)
>  	bic	\tmp, \tmp, #USER_ASID_FLAG
>  	msr	ttbr1_el1, \tmp
> +alternative_if ARM64_WORKAROUND_QCOM_FALKOR_E1003

Shouldn't we put this inside an #ifdef QCOM_FALKOR_ERRATUM_1003
so that we don't even emit nops in case we have the errata
disabled? Or did I miss something in the alternatives assembly
code?
Will Deacon Nov. 20, 2017, 6:05 p.m. UTC | #2
On Fri, Nov 17, 2017 at 04:27:14PM -0800, Stephen Boyd wrote:
> On 11/17, Will Deacon wrote:
> > We rely on an atomic swizzling of TTBR1 when transitioning from the entry
> > trampoline to the kernel proper on an exception. We can't rely on this
> > atomicity in the face of Falkor erratum #E1003, so on affected cores we
> > can issue a TLB invalidation prior to jumping into the kernel. There is
> > still the possibility of a TLB conflict here due to conflicting walk
> > cache entries, but this doesn't appear to be the case on these CPUs in
> > practice.
> > 
> > Signed-off-by: Will Deacon <will.deacon@arm.com>
> > ---
> >  arch/arm64/Kconfig        | 17 +++++------------
> >  arch/arm64/kernel/entry.S |  8 ++++++++
> >  2 files changed, 13 insertions(+), 12 deletions(-)
> > 
> > diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
> > index 0df64a6a56d4..f0fcbfc2262e 100644
> > --- a/arch/arm64/Kconfig
> > +++ b/arch/arm64/Kconfig
> > @@ -504,20 +504,13 @@ config CAVIUM_ERRATUM_30115
> >  config QCOM_FALKOR_ERRATUM_1003
> >  	bool "Falkor E1003: Incorrect translation due to ASID change"
> >  	default y
> > -	select ARM64_PAN if ARM64_SW_TTBR0_PAN
> 
> Cool, this sort of complicates the backport of the Kryo MIDR
> update of this errata to stable trees though.

Yeah, you may have to do a separate version for -stable if you don't
want to backport parts of this series.

> > diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S
> > index a839b94bba05..a600879939ce 100644
> > --- a/arch/arm64/kernel/entry.S
> > +++ b/arch/arm64/kernel/entry.S
> > @@ -941,6 +941,14 @@ __ni_sys_trace:
> >  	sub	\tmp, \tmp, #(SWAPPER_DIR_SIZE + RESERVED_TTBR0_SIZE)
> >  	bic	\tmp, \tmp, #USER_ASID_FLAG
> >  	msr	ttbr1_el1, \tmp
> > +alternative_if ARM64_WORKAROUND_QCOM_FALKOR_E1003
> 
> Shouldn't we put this inside an #ifdef QCOM_FALKOR_ERRATUM_1003
> so that we don't even emit nops in case we have the errata
> disabled? Or did I miss something in the alternatives assembly
> code?

Yes, you're right. Thanks!

Will
diff mbox

Patch

diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index 0df64a6a56d4..f0fcbfc2262e 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -504,20 +504,13 @@  config CAVIUM_ERRATUM_30115
 config QCOM_FALKOR_ERRATUM_1003
 	bool "Falkor E1003: Incorrect translation due to ASID change"
 	default y
-	select ARM64_PAN if ARM64_SW_TTBR0_PAN
 	help
 	  On Falkor v1, an incorrect ASID may be cached in the TLB when ASID
-	  and BADDR are changed together in TTBRx_EL1. The workaround for this
-	  issue is to use a reserved ASID in cpu_do_switch_mm() before
-	  switching to the new ASID. Saying Y here selects ARM64_PAN if
-	  ARM64_SW_TTBR0_PAN is selected. This is done because implementing and
-	  maintaining the E1003 workaround in the software PAN emulation code
-	  would be an unnecessary complication. The affected Falkor v1 CPU
-	  implements ARMv8.1 hardware PAN support and using hardware PAN
-	  support versus software PAN emulation is mutually exclusive at
-	  runtime.
-
-	  If unsure, say Y.
+	  and BADDR are changed together in TTBRx_EL1. Since we keep the ASID
+	  in TTBR1_EL1, this situation only occurs in the entry trampoline and
+	  then only for entries in the walk cache, since the leaf translation
+	  is unchanged. Work around the erratum by invalidating the walk cache
+	  entries for the trampoline before entering the kernel proper.
 
 config QCOM_FALKOR_ERRATUM_1009
 	bool "Falkor E1009: Prematurely complete a DSB after a TLBI"
diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S
index a839b94bba05..a600879939ce 100644
--- a/arch/arm64/kernel/entry.S
+++ b/arch/arm64/kernel/entry.S
@@ -941,6 +941,14 @@  __ni_sys_trace:
 	sub	\tmp, \tmp, #(SWAPPER_DIR_SIZE + RESERVED_TTBR0_SIZE)
 	bic	\tmp, \tmp, #USER_ASID_FLAG
 	msr	ttbr1_el1, \tmp
+alternative_if ARM64_WORKAROUND_QCOM_FALKOR_E1003
+	isb
+	movk	\tmp, #:abs_g2_nc:(TRAMP_VALIAS >> 12)
+	movk	\tmp, #:abs_g1_nc:(TRAMP_VALIAS >> 12)
+	movk	\tmp, #:abs_g0_nc:(TRAMP_VALIAS >> 12)
+	tlbi	vae1, \tmp
+	dsb	nsh
+alternative_else_nop_endif
 	.endm
 
 	.macro tramp_unmap_kernel, tmp