[v2,1/2] arm64: Combine workarounds for speculative AT errata
diff mbox series

Message ID 20191113114118.2427-2-steven.price@arm.com
State New
Headers show
Series
  • arm64: Workaround for Cortex-A55 erratum 1530923
Related show

Commit Message

Steven Price Nov. 13, 2019, 11:41 a.m. UTC
Cortex-A57/A72 have a similar errata to Cortex-A76 regarding speculation
of the AT instruction. Since the workaround for A57/A72 doesn't require
VHE, the restriction enforcing VHE for A76 can be removed by combining
the workaround flag for both errata.

So combine WORKAROUND_1165522 and WORKAROUND_1319367 into
WORKAROUND_SPECULATIVE_AT. The majority of code is contained within VHE
or NVHE specific functions, for the cases where the code is shared extra
checks against has_vhe().

This also paves the way for adding a similar erratum for Cortex-A55.

Signed-off-by: Steven Price <steven.price@arm.com>
---
 arch/arm64/Kconfig                |  5 +++++
 arch/arm64/include/asm/cpucaps.h  |  5 ++---
 arch/arm64/include/asm/kvm_host.h |  4 ----
 arch/arm64/include/asm/kvm_hyp.h  |  3 ++-
 arch/arm64/kernel/cpu_errata.c    | 19 ++++++++++++++-----
 arch/arm64/kvm/hyp/switch.c       |  6 +++---
 arch/arm64/kvm/hyp/sysreg-sr.c    |  6 ++++--
 arch/arm64/kvm/hyp/tlb.c          |  8 ++++----
 8 files changed, 34 insertions(+), 22 deletions(-)

Comments

Marc Zyngier Nov. 13, 2019, 12:44 p.m. UTC | #1
On 2019-11-13 12:50, Steven Price wrote:
> Cortex-A57/A72 have a similar errata to Cortex-A76 regarding 
> speculation
> of the AT instruction. Since the workaround for A57/A72 doesn't 
> require
> VHE, the restriction enforcing VHE for A76 can be removed by 
> combining
> the workaround flag for both errata.

Are we sure that A76 behaves the same as A57/A72 when it comes to not
caching any of the EPD* bits in the TLB? Because the 1319367 workaround
has a lot of the A72 microarch implicit to it, and I'm not sure this
works as is on A76 or A55...

The patch itself looks OK, but I'd like some reassurance about the
above.

          M.
Suzuki Kuruppassery Poulose Nov. 13, 2019, 2:01 p.m. UTC | #2
On 13/11/2019 11:41, Steven Price wrote:
> Cortex-A57/A72 have a similar errata to Cortex-A76 regarding speculation
> of the AT instruction. Since the workaround for A57/A72 doesn't require
> VHE, the restriction enforcing VHE for A76 can be removed by combining
> the workaround flag for both errata.
> 
> So combine WORKAROUND_1165522 and WORKAROUND_1319367 into
> WORKAROUND_SPECULATIVE_AT. The majority of code is contained within VHE
> or NVHE specific functions, for the cases where the code is shared extra
> checks against has_vhe().
> 
> This also paves the way for adding a similar erratum for Cortex-A55.
> 
> Signed-off-by: Steven Price <steven.price@arm.com>

> diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c
> index 4f8187a4fc46..b801f8e832aa 100644
> --- a/arch/arm64/kernel/cpu_errata.c
> +++ b/arch/arm64/kernel/cpu_errata.c
> @@ -744,6 +744,16 @@ static const struct midr_range erratum_1418040_list[] = {
>   };
>   #endif
>   
> +#ifdef CONFIG_ARM64_WORKAROUND_SPECULATIVE_AT
> +static const struct midr_range erratum_speculative_at_list[] = {
> +#ifdef CONFIG_ARM64_ERRATUM_1165522
> +	/* Cortex A76 r0p0 to r2p0 */
> +	MIDR_RANGE(MIDR_CORTEX_A76, 0, 0, 2, 0),
> +#endif
> +	{},
> +};
> +#endif
> +
>   const struct arm64_cpu_capabilities arm64_errata[] = {
>   #ifdef CONFIG_ARM64_WORKAROUND_CLEAN_CACHE
>   	{
> @@ -868,12 +878,11 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
>   		ERRATA_MIDR_RANGE_LIST(erratum_1418040_list),
>   	},
>   #endif
> -#ifdef CONFIG_ARM64_ERRATUM_1165522
> +#ifdef CONFIG_ARM64_WORKAROUND_SPECULATIVE_AT
>   	{
> -		/* Cortex-A76 r0p0 to r2p0 */
>   		.desc = "ARM erratum 1165522",
> -		.capability = ARM64_WORKAROUND_1165522,
> -		ERRATA_MIDR_RANGE(MIDR_CORTEX_A76, 0, 0, 2, 0),
> +		.capability = ARM64_WORKAROUND_SPECULATIVE_AT,
> +		ERRATA_MIDR_RANGE_LIST(erratum_speculative_at_list),
>   	},
>   #endif
>   #ifdef CONFIG_ARM64_ERRATUM_1463225
> @@ -910,7 +919,7 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
>   #ifdef CONFIG_ARM64_ERRATUM_1319367
>   	{
>   		.desc = "ARM erratum 1319367",
> -		.capability = ARM64_WORKAROUND_1319367,
> +		.capability = ARM64_WORKAROUND_SPECULATIVE_AT,
>   		ERRATA_MIDR_RANGE_LIST(ca57_a72),
>   	},
>   #endif

Have you tested this patch with both the errata CONFIGs turned on ?
Having multiple entries for the same capability should trigger a WARNING at
boot with init_cpu_hwcaps_indirect_list_from_array().
You could simply add the MIDRs to the midr_list and update the description
to include all the Errata numbers.

Suzuki
Steven Price Nov. 13, 2019, 2:29 p.m. UTC | #3
On 13/11/2019 12:44, Marc Zyngier wrote:
> On 2019-11-13 12:50, Steven Price wrote:
>> Cortex-A57/A72 have a similar errata to Cortex-A76 regarding speculation
>> of the AT instruction. Since the workaround for A57/A72 doesn't require
>> VHE, the restriction enforcing VHE for A76 can be removed by combining
>> the workaround flag for both errata.
> 
> Are we sure that A76 behaves the same as A57/A72 when it comes to not
> caching any of the EPD* bits in the TLB? Because the 1319367 workaround
> has a lot of the A72 microarch implicit to it, and I'm not sure this
> works as is on A76 or A55...

Hmm, well I was going purely on the errata documents which have
basically the same text for all the errata. I have to admit I do not
understand the microarch details here. Perhaps it would be better to
leave the VHE and NVHE cases separated then?

Steven

> The patch itself looks OK, but I'd like some reassurance about the
> above.
> 
>          M.
Steven Price Nov. 13, 2019, 2:43 p.m. UTC | #4
On 13/11/2019 14:01, Suzuki K Poulose wrote:
> 
> On 13/11/2019 11:41, Steven Price wrote:
>> Cortex-A57/A72 have a similar errata to Cortex-A76 regarding speculation
>> of the AT instruction. Since the workaround for A57/A72 doesn't require
>> VHE, the restriction enforcing VHE for A76 can be removed by combining
>> the workaround flag for both errata.
>>
>> So combine WORKAROUND_1165522 and WORKAROUND_1319367 into
>> WORKAROUND_SPECULATIVE_AT. The majority of code is contained within VHE
>> or NVHE specific functions, for the cases where the code is shared extra
>> checks against has_vhe().
>>
>> This also paves the way for adding a similar erratum for Cortex-A55.
>>
>> Signed-off-by: Steven Price <steven.price@arm.com>
> 
>> diff --git a/arch/arm64/kernel/cpu_errata.c
>> b/arch/arm64/kernel/cpu_errata.c
>> index 4f8187a4fc46..b801f8e832aa 100644
>> --- a/arch/arm64/kernel/cpu_errata.c
>> +++ b/arch/arm64/kernel/cpu_errata.c
>> @@ -744,6 +744,16 @@ static const struct midr_range
>> erratum_1418040_list[] = {
>>   };
>>   #endif
>>   +#ifdef CONFIG_ARM64_WORKAROUND_SPECULATIVE_AT
>> +static const struct midr_range erratum_speculative_at_list[] = {
>> +#ifdef CONFIG_ARM64_ERRATUM_1165522
>> +    /* Cortex A76 r0p0 to r2p0 */
>> +    MIDR_RANGE(MIDR_CORTEX_A76, 0, 0, 2, 0),
>> +#endif
>> +    {},
>> +};
>> +#endif
>> +
>>   const struct arm64_cpu_capabilities arm64_errata[] = {
>>   #ifdef CONFIG_ARM64_WORKAROUND_CLEAN_CACHE
>>       {
>> @@ -868,12 +878,11 @@ const struct arm64_cpu_capabilities
>> arm64_errata[] = {
>>           ERRATA_MIDR_RANGE_LIST(erratum_1418040_list),
>>       },
>>   #endif
>> -#ifdef CONFIG_ARM64_ERRATUM_1165522
>> +#ifdef CONFIG_ARM64_WORKAROUND_SPECULATIVE_AT
>>       {
>> -        /* Cortex-A76 r0p0 to r2p0 */
>>           .desc = "ARM erratum 1165522",
>> -        .capability = ARM64_WORKAROUND_1165522,
>> -        ERRATA_MIDR_RANGE(MIDR_CORTEX_A76, 0, 0, 2, 0),
>> +        .capability = ARM64_WORKAROUND_SPECULATIVE_AT,
>> +        ERRATA_MIDR_RANGE_LIST(erratum_speculative_at_list),
>>       },
>>   #endif
>>   #ifdef CONFIG_ARM64_ERRATUM_1463225
>> @@ -910,7 +919,7 @@ const struct arm64_cpu_capabilities arm64_errata[]
>> = {
>>   #ifdef CONFIG_ARM64_ERRATUM_1319367
>>       {
>>           .desc = "ARM erratum 1319367",
>> -        .capability = ARM64_WORKAROUND_1319367,
>> +        .capability = ARM64_WORKAROUND_SPECULATIVE_AT,
>>           ERRATA_MIDR_RANGE_LIST(ca57_a72),
>>       },
>>   #endif
> 
> Have you tested this patch with both the errata CONFIGs turned on ?
> Having multiple entries for the same capability should trigger a WARNING at
> boot with init_cpu_hwcaps_indirect_list_from_array().
> You could simply add the MIDRs to the midr_list and update the description
> to include all the Errata numbers.

Ha! You of course are right - I had 'tested' the combination but
apparently not looked carefully enough - there is indeed a WARNING in
the boot.

I might well be keeping the two entries anyway due to Marc's concerns
about the micro-architectural details of the 1319367 workaround.

Steve

Patch
diff mbox series

diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index 3e53441ee067..5a235e724fbb 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -516,9 +516,13 @@  config ARM64_ERRATUM_1418040
 
 	  If unsure, say Y.
 
+config ARM64_WORKAROUND_SPECULATIVE_AT
+	bool
+
 config ARM64_ERRATUM_1165522
 	bool "Cortex-A76: Speculative AT instruction using out-of-context translation regime could cause subsequent request to generate an incorrect translation"
 	default y
+	select ARM64_WORKAROUND_SPECULATIVE_AT
 	help
 	  This option adds a workaround for ARM Cortex-A76 erratum 1165522.
 
@@ -547,6 +551,7 @@  config ARM64_ERRATUM_1286807
 config ARM64_ERRATUM_1319367
 	bool "Cortex-A57/A72: Speculative AT instruction using out-of-context translation regime could cause subsequent request to generate an incorrect translation"
 	default y
+	select ARM64_WORKAROUND_SPECULATIVE_AT
 	help
 	  This option adds work arounds for ARM Cortex-A57 erratum 1319537
 	  and A72 erratum 1319367
diff --git a/arch/arm64/include/asm/cpucaps.h b/arch/arm64/include/asm/cpucaps.h
index b92683871119..b57821e10a8f 100644
--- a/arch/arm64/include/asm/cpucaps.h
+++ b/arch/arm64/include/asm/cpucaps.h
@@ -44,7 +44,7 @@ 
 #define ARM64_SSBS				34
 #define ARM64_WORKAROUND_1418040		35
 #define ARM64_HAS_SB				36
-#define ARM64_WORKAROUND_1165522		37
+#define ARM64_WORKAROUND_SPECULATIVE_AT		37
 #define ARM64_HAS_ADDRESS_AUTH_ARCH		38
 #define ARM64_HAS_ADDRESS_AUTH_IMP_DEF		39
 #define ARM64_HAS_GENERIC_AUTH_ARCH		40
@@ -55,8 +55,7 @@ 
 #define ARM64_WORKAROUND_CAVIUM_TX2_219_TVM	45
 #define ARM64_WORKAROUND_CAVIUM_TX2_219_PRFM	46
 #define ARM64_WORKAROUND_1542419		47
-#define ARM64_WORKAROUND_1319367		48
 
-#define ARM64_NCAPS				49
+#define ARM64_NCAPS				48
 
 #endif /* __ASM_CPUCAPS_H */
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index 5ecb091c8576..583ada60d5d9 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -533,10 +533,6 @@  static inline bool kvm_arch_requires_vhe(void)
 	if (system_supports_sve())
 		return true;
 
-	/* Some implementations have defects that confine them to VHE */
-	if (cpus_have_cap(ARM64_WORKAROUND_1165522))
-		return true;
-
 	return false;
 }
 
diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h
index 97f21cc66657..965a9b4e5454 100644
--- a/arch/arm64/include/asm/kvm_hyp.h
+++ b/arch/arm64/include/asm/kvm_hyp.h
@@ -95,7 +95,8 @@  static __always_inline void __hyp_text __load_guest_stage2(struct kvm *kvm)
 	 * before we can switch to the EL1/EL0 translation regime used by
 	 * the guest.
 	 */
-	asm(ALTERNATIVE("nop", "isb", ARM64_WORKAROUND_1165522));
+	if (!has_vhe())
+		asm(ALTERNATIVE("nop", "isb", ARM64_WORKAROUND_SPECULATIVE_AT));
 }
 
 #endif /* __ARM64_KVM_HYP_H__ */
diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c
index 4f8187a4fc46..b801f8e832aa 100644
--- a/arch/arm64/kernel/cpu_errata.c
+++ b/arch/arm64/kernel/cpu_errata.c
@@ -744,6 +744,16 @@  static const struct midr_range erratum_1418040_list[] = {
 };
 #endif
 
+#ifdef CONFIG_ARM64_WORKAROUND_SPECULATIVE_AT
+static const struct midr_range erratum_speculative_at_list[] = {
+#ifdef CONFIG_ARM64_ERRATUM_1165522
+	/* Cortex A76 r0p0 to r2p0 */
+	MIDR_RANGE(MIDR_CORTEX_A76, 0, 0, 2, 0),
+#endif
+	{},
+};
+#endif
+
 const struct arm64_cpu_capabilities arm64_errata[] = {
 #ifdef CONFIG_ARM64_WORKAROUND_CLEAN_CACHE
 	{
@@ -868,12 +878,11 @@  const struct arm64_cpu_capabilities arm64_errata[] = {
 		ERRATA_MIDR_RANGE_LIST(erratum_1418040_list),
 	},
 #endif
-#ifdef CONFIG_ARM64_ERRATUM_1165522
+#ifdef CONFIG_ARM64_WORKAROUND_SPECULATIVE_AT
 	{
-		/* Cortex-A76 r0p0 to r2p0 */
 		.desc = "ARM erratum 1165522",
-		.capability = ARM64_WORKAROUND_1165522,
-		ERRATA_MIDR_RANGE(MIDR_CORTEX_A76, 0, 0, 2, 0),
+		.capability = ARM64_WORKAROUND_SPECULATIVE_AT,
+		ERRATA_MIDR_RANGE_LIST(erratum_speculative_at_list),
 	},
 #endif
 #ifdef CONFIG_ARM64_ERRATUM_1463225
@@ -910,7 +919,7 @@  const struct arm64_cpu_capabilities arm64_errata[] = {
 #ifdef CONFIG_ARM64_ERRATUM_1319367
 	{
 		.desc = "ARM erratum 1319367",
-		.capability = ARM64_WORKAROUND_1319367,
+		.capability = ARM64_WORKAROUND_SPECULATIVE_AT,
 		ERRATA_MIDR_RANGE_LIST(ca57_a72),
 	},
 #endif
diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
index 72fbbd86eb5e..140f467b1f16 100644
--- a/arch/arm64/kvm/hyp/switch.c
+++ b/arch/arm64/kvm/hyp/switch.c
@@ -119,7 +119,7 @@  static void __hyp_text __activate_traps_nvhe(struct kvm_vcpu *vcpu)
 
 	write_sysreg(val, cptr_el2);
 
-	if (cpus_have_const_cap(ARM64_WORKAROUND_1319367)) {
+	if (cpus_have_const_cap(ARM64_WORKAROUND_SPECULATIVE_AT)) {
 		struct kvm_cpu_context *ctxt = &vcpu->arch.ctxt;
 
 		isb();
@@ -162,7 +162,7 @@  static void deactivate_traps_vhe(void)
 	 * before we can switch to the EL2/EL0 translation regime used by
 	 * the host.
 	 */
-	asm(ALTERNATIVE("nop", "isb", ARM64_WORKAROUND_1165522));
+	asm(ALTERNATIVE("nop", "isb", ARM64_WORKAROUND_SPECULATIVE_AT));
 
 	write_sysreg(CPACR_EL1_DEFAULT, cpacr_el1);
 	write_sysreg(vectors, vbar_el1);
@@ -173,7 +173,7 @@  static void __hyp_text __deactivate_traps_nvhe(void)
 {
 	u64 mdcr_el2 = read_sysreg(mdcr_el2);
 
-	if (cpus_have_const_cap(ARM64_WORKAROUND_1319367)) {
+	if (cpus_have_const_cap(ARM64_WORKAROUND_SPECULATIVE_AT)) {
 		u64 val;
 
 		/*
diff --git a/arch/arm64/kvm/hyp/sysreg-sr.c b/arch/arm64/kvm/hyp/sysreg-sr.c
index 22b8128d19f6..f023f56397b3 100644
--- a/arch/arm64/kvm/hyp/sysreg-sr.c
+++ b/arch/arm64/kvm/hyp/sysreg-sr.c
@@ -118,7 +118,8 @@  static void __hyp_text __sysreg_restore_el1_state(struct kvm_cpu_context *ctxt)
 	write_sysreg(ctxt->sys_regs[MPIDR_EL1],		vmpidr_el2);
 	write_sysreg(ctxt->sys_regs[CSSELR_EL1],	csselr_el1);
 
-	if (!cpus_have_const_cap(ARM64_WORKAROUND_1319367)) {
+	if (!cpus_have_const_cap(ARM64_WORKAROUND_SPECULATIVE_AT) ||
+	    has_vhe()) {
 		write_sysreg_el1(ctxt->sys_regs[SCTLR_EL1],	SYS_SCTLR);
 		write_sysreg_el1(ctxt->sys_regs[TCR_EL1],	SYS_TCR);
 	} else	if (!ctxt->__hyp_running_vcpu) {
@@ -149,7 +150,8 @@  static void __hyp_text __sysreg_restore_el1_state(struct kvm_cpu_context *ctxt)
 	write_sysreg(ctxt->sys_regs[PAR_EL1],		par_el1);
 	write_sysreg(ctxt->sys_regs[TPIDR_EL1],		tpidr_el1);
 
-	if (cpus_have_const_cap(ARM64_WORKAROUND_1319367) &&
+	if (cpus_have_const_cap(ARM64_WORKAROUND_SPECULATIVE_AT) &&
+	    !has_vhe() &&
 	    ctxt->__hyp_running_vcpu) {
 		/*
 		 * Must only be done for host registers, hence the context
diff --git a/arch/arm64/kvm/hyp/tlb.c b/arch/arm64/kvm/hyp/tlb.c
index c2bc17ca6430..338df7f9149a 100644
--- a/arch/arm64/kvm/hyp/tlb.c
+++ b/arch/arm64/kvm/hyp/tlb.c
@@ -23,7 +23,7 @@  static void __hyp_text __tlb_switch_to_guest_vhe(struct kvm *kvm,
 
 	local_irq_save(cxt->flags);
 
-	if (cpus_have_const_cap(ARM64_WORKAROUND_1165522)) {
+	if (cpus_have_const_cap(ARM64_WORKAROUND_SPECULATIVE_AT)) {
 		/*
 		 * For CPUs that are affected by ARM erratum 1165522, we
 		 * cannot trust stage-1 to be in a correct state at that
@@ -63,7 +63,7 @@  static void __hyp_text __tlb_switch_to_guest_vhe(struct kvm *kvm,
 static void __hyp_text __tlb_switch_to_guest_nvhe(struct kvm *kvm,
 						  struct tlb_inv_context *cxt)
 {
-	if (cpus_have_const_cap(ARM64_WORKAROUND_1319367)) {
+	if (cpus_have_const_cap(ARM64_WORKAROUND_SPECULATIVE_AT)) {
 		u64 val;
 
 		/*
@@ -103,7 +103,7 @@  static void __hyp_text __tlb_switch_to_host_vhe(struct kvm *kvm,
 	write_sysreg(HCR_HOST_VHE_FLAGS, hcr_el2);
 	isb();
 
-	if (cpus_have_const_cap(ARM64_WORKAROUND_1165522)) {
+	if (cpus_have_const_cap(ARM64_WORKAROUND_SPECULATIVE_AT)) {
 		/* Restore the registers to what they were */
 		write_sysreg_el1(cxt->tcr, SYS_TCR);
 		write_sysreg_el1(cxt->sctlr, SYS_SCTLR);
@@ -117,7 +117,7 @@  static void __hyp_text __tlb_switch_to_host_nvhe(struct kvm *kvm,
 {
 	write_sysreg(0, vttbr_el2);
 
-	if (cpus_have_const_cap(ARM64_WORKAROUND_1319367)) {
+	if (cpus_have_const_cap(ARM64_WORKAROUND_SPECULATIVE_AT)) {
 		/* Ensure write of the host VMID */
 		isb();
 		/* Restore the host's TCR_EL1 */