| Message ID | 20191113114118.2427-2-steven.price@arm.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | arm64: Workaround for Cortex-A55 erratum 1530923 | expand |
On 2019-11-13 12:50, Steven Price wrote: > Cortex-A57/A72 have a similar errata to Cortex-A76 regarding > speculation > of the AT instruction. Since the workaround for A57/A72 doesn't > require > VHE, the restriction enforcing VHE for A76 can be removed by > combining > the workaround flag for both errata. Are we sure that A76 behaves the same as A57/A72 when it comes to not caching any of the EPD* bits in the TLB? Because the 1319367 workaround has a lot of the A72 microarch implicit to it, and I'm not sure this works as is on A76 or A55... The patch itself looks OK, but I'd like some reassurance about the above. M.
On 13/11/2019 11:41, Steven Price wrote: > Cortex-A57/A72 have a similar errata to Cortex-A76 regarding speculation > of the AT instruction. Since the workaround for A57/A72 doesn't require > VHE, the restriction enforcing VHE for A76 can be removed by combining > the workaround flag for both errata. > > So combine WORKAROUND_1165522 and WORKAROUND_1319367 into > WORKAROUND_SPECULATIVE_AT. The majority of code is contained within VHE > or NVHE specific functions, for the cases where the code is shared extra > checks against has_vhe(). > > This also paves the way for adding a similar erratum for Cortex-A55. > > Signed-off-by: Steven Price <steven.price@arm.com> > diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c > index 4f8187a4fc46..b801f8e832aa 100644 > --- a/arch/arm64/kernel/cpu_errata.c > +++ b/arch/arm64/kernel/cpu_errata.c > @@ -744,6 +744,16 @@ static const struct midr_range erratum_1418040_list[] = { > }; > #endif > > +#ifdef CONFIG_ARM64_WORKAROUND_SPECULATIVE_AT > +static const struct midr_range erratum_speculative_at_list[] = { > +#ifdef CONFIG_ARM64_ERRATUM_1165522 > + /* Cortex A76 r0p0 to r2p0 */ > + MIDR_RANGE(MIDR_CORTEX_A76, 0, 0, 2, 0), > +#endif > + {}, > +}; > +#endif > + > const struct arm64_cpu_capabilities arm64_errata[] = { > #ifdef CONFIG_ARM64_WORKAROUND_CLEAN_CACHE > { > @@ -868,12 +878,11 @@ const struct arm64_cpu_capabilities arm64_errata[] = { > ERRATA_MIDR_RANGE_LIST(erratum_1418040_list), > }, > #endif > -#ifdef CONFIG_ARM64_ERRATUM_1165522 > +#ifdef CONFIG_ARM64_WORKAROUND_SPECULATIVE_AT > { > - /* Cortex-A76 r0p0 to r2p0 */ > .desc = "ARM erratum 1165522", > - .capability = ARM64_WORKAROUND_1165522, > - ERRATA_MIDR_RANGE(MIDR_CORTEX_A76, 0, 0, 2, 0), > + .capability = ARM64_WORKAROUND_SPECULATIVE_AT, > + ERRATA_MIDR_RANGE_LIST(erratum_speculative_at_list), > }, > #endif > #ifdef CONFIG_ARM64_ERRATUM_1463225 > @@ -910,7 +919,7 @@ const struct arm64_cpu_capabilities arm64_errata[] = { > #ifdef CONFIG_ARM64_ERRATUM_1319367 > { > .desc = "ARM erratum 1319367", > - .capability = ARM64_WORKAROUND_1319367, > + .capability = ARM64_WORKAROUND_SPECULATIVE_AT, > ERRATA_MIDR_RANGE_LIST(ca57_a72), > }, > #endif Have you tested this patch with both the errata CONFIGs turned on ? Having multiple entries for the same capability should trigger a WARNING at boot with init_cpu_hwcaps_indirect_list_from_array(). You could simply add the MIDRs to the midr_list and update the description to include all the Errata numbers. Suzuki
On 13/11/2019 12:44, Marc Zyngier wrote: > On 2019-11-13 12:50, Steven Price wrote: >> Cortex-A57/A72 have a similar errata to Cortex-A76 regarding speculation >> of the AT instruction. Since the workaround for A57/A72 doesn't require >> VHE, the restriction enforcing VHE for A76 can be removed by combining >> the workaround flag for both errata. > > Are we sure that A76 behaves the same as A57/A72 when it comes to not > caching any of the EPD* bits in the TLB? Because the 1319367 workaround > has a lot of the A72 microarch implicit to it, and I'm not sure this > works as is on A76 or A55... Hmm, well I was going purely on the errata documents which have basically the same text for all the errata. I have to admit I do not understand the microarch details here. Perhaps it would be better to leave the VHE and NVHE cases separated then? Steven > The patch itself looks OK, but I'd like some reassurance about the > above. > > M.
On 13/11/2019 14:01, Suzuki K Poulose wrote: > > On 13/11/2019 11:41, Steven Price wrote: >> Cortex-A57/A72 have a similar errata to Cortex-A76 regarding speculation >> of the AT instruction. Since the workaround for A57/A72 doesn't require >> VHE, the restriction enforcing VHE for A76 can be removed by combining >> the workaround flag for both errata. >> >> So combine WORKAROUND_1165522 and WORKAROUND_1319367 into >> WORKAROUND_SPECULATIVE_AT. The majority of code is contained within VHE >> or NVHE specific functions, for the cases where the code is shared extra >> checks against has_vhe(). >> >> This also paves the way for adding a similar erratum for Cortex-A55. >> >> Signed-off-by: Steven Price <steven.price@arm.com> > >> diff --git a/arch/arm64/kernel/cpu_errata.c >> b/arch/arm64/kernel/cpu_errata.c >> index 4f8187a4fc46..b801f8e832aa 100644 >> --- a/arch/arm64/kernel/cpu_errata.c >> +++ b/arch/arm64/kernel/cpu_errata.c >> @@ -744,6 +744,16 @@ static const struct midr_range >> erratum_1418040_list[] = { >> }; >> #endif >> +#ifdef CONFIG_ARM64_WORKAROUND_SPECULATIVE_AT >> +static const struct midr_range erratum_speculative_at_list[] = { >> +#ifdef CONFIG_ARM64_ERRATUM_1165522 >> + /* Cortex A76 r0p0 to r2p0 */ >> + MIDR_RANGE(MIDR_CORTEX_A76, 0, 0, 2, 0), >> +#endif >> + {}, >> +}; >> +#endif >> + >> const struct arm64_cpu_capabilities arm64_errata[] = { >> #ifdef CONFIG_ARM64_WORKAROUND_CLEAN_CACHE >> { >> @@ -868,12 +878,11 @@ const struct arm64_cpu_capabilities >> arm64_errata[] = { >> ERRATA_MIDR_RANGE_LIST(erratum_1418040_list), >> }, >> #endif >> -#ifdef CONFIG_ARM64_ERRATUM_1165522 >> +#ifdef CONFIG_ARM64_WORKAROUND_SPECULATIVE_AT >> { >> - /* Cortex-A76 r0p0 to r2p0 */ >> .desc = "ARM erratum 1165522", >> - .capability = ARM64_WORKAROUND_1165522, >> - ERRATA_MIDR_RANGE(MIDR_CORTEX_A76, 0, 0, 2, 0), >> + .capability = ARM64_WORKAROUND_SPECULATIVE_AT, >> + ERRATA_MIDR_RANGE_LIST(erratum_speculative_at_list), >> }, >> #endif >> #ifdef CONFIG_ARM64_ERRATUM_1463225 >> @@ -910,7 +919,7 @@ const struct arm64_cpu_capabilities arm64_errata[] >> = { >> #ifdef CONFIG_ARM64_ERRATUM_1319367 >> { >> .desc = "ARM erratum 1319367", >> - .capability = ARM64_WORKAROUND_1319367, >> + .capability = ARM64_WORKAROUND_SPECULATIVE_AT, >> ERRATA_MIDR_RANGE_LIST(ca57_a72), >> }, >> #endif > > Have you tested this patch with both the errata CONFIGs turned on ? > Having multiple entries for the same capability should trigger a WARNING at > boot with init_cpu_hwcaps_indirect_list_from_array(). > You could simply add the MIDRs to the midr_list and update the description > to include all the Errata numbers. Ha! You of course are right - I had 'tested' the combination but apparently not looked carefully enough - there is indeed a WARNING in the boot. I might well be keeping the two entries anyway due to Marc's concerns about the micro-architectural details of the 1319367 workaround. Steve
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 3e53441ee067..5a235e724fbb 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -516,9 +516,13 @@ config ARM64_ERRATUM_1418040 If unsure, say Y. +config ARM64_WORKAROUND_SPECULATIVE_AT + bool + config ARM64_ERRATUM_1165522 bool "Cortex-A76: Speculative AT instruction using out-of-context translation regime could cause subsequent request to generate an incorrect translation" default y + select ARM64_WORKAROUND_SPECULATIVE_AT help This option adds a workaround for ARM Cortex-A76 erratum 1165522. @@ -547,6 +551,7 @@ config ARM64_ERRATUM_1286807 config ARM64_ERRATUM_1319367 bool "Cortex-A57/A72: Speculative AT instruction using out-of-context translation regime could cause subsequent request to generate an incorrect translation" default y + select ARM64_WORKAROUND_SPECULATIVE_AT help This option adds work arounds for ARM Cortex-A57 erratum 1319537 and A72 erratum 1319367 diff --git a/arch/arm64/include/asm/cpucaps.h b/arch/arm64/include/asm/cpucaps.h index b92683871119..b57821e10a8f 100644 --- a/arch/arm64/include/asm/cpucaps.h +++ b/arch/arm64/include/asm/cpucaps.h @@ -44,7 +44,7 @@ #define ARM64_SSBS 34 #define ARM64_WORKAROUND_1418040 35 #define ARM64_HAS_SB 36 -#define ARM64_WORKAROUND_1165522 37 +#define ARM64_WORKAROUND_SPECULATIVE_AT 37 #define ARM64_HAS_ADDRESS_AUTH_ARCH 38 #define ARM64_HAS_ADDRESS_AUTH_IMP_DEF 39 #define ARM64_HAS_GENERIC_AUTH_ARCH 40 @@ -55,8 +55,7 @@ #define ARM64_WORKAROUND_CAVIUM_TX2_219_TVM 45 #define ARM64_WORKAROUND_CAVIUM_TX2_219_PRFM 46 #define ARM64_WORKAROUND_1542419 47 -#define ARM64_WORKAROUND_1319367 48 -#define ARM64_NCAPS 49 +#define ARM64_NCAPS 48 #endif /* __ASM_CPUCAPS_H */ diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index 5ecb091c8576..583ada60d5d9 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -533,10 +533,6 @@ static inline bool kvm_arch_requires_vhe(void) if (system_supports_sve()) return true; - /* Some implementations have defects that confine them to VHE */ - if (cpus_have_cap(ARM64_WORKAROUND_1165522)) - return true; - return false; } diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h index 97f21cc66657..965a9b4e5454 100644 --- a/arch/arm64/include/asm/kvm_hyp.h +++ b/arch/arm64/include/asm/kvm_hyp.h @@ -95,7 +95,8 @@ static __always_inline void __hyp_text __load_guest_stage2(struct kvm *kvm) * before we can switch to the EL1/EL0 translation regime used by * the guest. */ - asm(ALTERNATIVE("nop", "isb", ARM64_WORKAROUND_1165522)); + if (!has_vhe()) + asm(ALTERNATIVE("nop", "isb", ARM64_WORKAROUND_SPECULATIVE_AT)); } #endif /* __ARM64_KVM_HYP_H__ */ diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c index 4f8187a4fc46..b801f8e832aa 100644 --- a/arch/arm64/kernel/cpu_errata.c +++ b/arch/arm64/kernel/cpu_errata.c @@ -744,6 +744,16 @@ static const struct midr_range erratum_1418040_list[] = { }; #endif +#ifdef CONFIG_ARM64_WORKAROUND_SPECULATIVE_AT +static const struct midr_range erratum_speculative_at_list[] = { +#ifdef CONFIG_ARM64_ERRATUM_1165522 + /* Cortex A76 r0p0 to r2p0 */ + MIDR_RANGE(MIDR_CORTEX_A76, 0, 0, 2, 0), +#endif + {}, +}; +#endif + const struct arm64_cpu_capabilities arm64_errata[] = { #ifdef CONFIG_ARM64_WORKAROUND_CLEAN_CACHE { @@ -868,12 +878,11 @@ const struct arm64_cpu_capabilities arm64_errata[] = { ERRATA_MIDR_RANGE_LIST(erratum_1418040_list), }, #endif -#ifdef CONFIG_ARM64_ERRATUM_1165522 +#ifdef CONFIG_ARM64_WORKAROUND_SPECULATIVE_AT { - /* Cortex-A76 r0p0 to r2p0 */ .desc = "ARM erratum 1165522", - .capability = ARM64_WORKAROUND_1165522, - ERRATA_MIDR_RANGE(MIDR_CORTEX_A76, 0, 0, 2, 0), + .capability = ARM64_WORKAROUND_SPECULATIVE_AT, + ERRATA_MIDR_RANGE_LIST(erratum_speculative_at_list), }, #endif #ifdef CONFIG_ARM64_ERRATUM_1463225 @@ -910,7 +919,7 @@ const struct arm64_cpu_capabilities arm64_errata[] = { #ifdef CONFIG_ARM64_ERRATUM_1319367 { .desc = "ARM erratum 1319367", - .capability = ARM64_WORKAROUND_1319367, + .capability = ARM64_WORKAROUND_SPECULATIVE_AT, ERRATA_MIDR_RANGE_LIST(ca57_a72), }, #endif diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c index 72fbbd86eb5e..140f467b1f16 100644 --- a/arch/arm64/kvm/hyp/switch.c +++ b/arch/arm64/kvm/hyp/switch.c @@ -119,7 +119,7 @@ static void __hyp_text __activate_traps_nvhe(struct kvm_vcpu *vcpu) write_sysreg(val, cptr_el2); - if (cpus_have_const_cap(ARM64_WORKAROUND_1319367)) { + if (cpus_have_const_cap(ARM64_WORKAROUND_SPECULATIVE_AT)) { struct kvm_cpu_context *ctxt = &vcpu->arch.ctxt; isb(); @@ -162,7 +162,7 @@ static void deactivate_traps_vhe(void) * before we can switch to the EL2/EL0 translation regime used by * the host. */ - asm(ALTERNATIVE("nop", "isb", ARM64_WORKAROUND_1165522)); + asm(ALTERNATIVE("nop", "isb", ARM64_WORKAROUND_SPECULATIVE_AT)); write_sysreg(CPACR_EL1_DEFAULT, cpacr_el1); write_sysreg(vectors, vbar_el1); @@ -173,7 +173,7 @@ static void __hyp_text __deactivate_traps_nvhe(void) { u64 mdcr_el2 = read_sysreg(mdcr_el2); - if (cpus_have_const_cap(ARM64_WORKAROUND_1319367)) { + if (cpus_have_const_cap(ARM64_WORKAROUND_SPECULATIVE_AT)) { u64 val; /* diff --git a/arch/arm64/kvm/hyp/sysreg-sr.c b/arch/arm64/kvm/hyp/sysreg-sr.c index 22b8128d19f6..f023f56397b3 100644 --- a/arch/arm64/kvm/hyp/sysreg-sr.c +++ b/arch/arm64/kvm/hyp/sysreg-sr.c @@ -118,7 +118,8 @@ static void __hyp_text __sysreg_restore_el1_state(struct kvm_cpu_context *ctxt) write_sysreg(ctxt->sys_regs[MPIDR_EL1], vmpidr_el2); write_sysreg(ctxt->sys_regs[CSSELR_EL1], csselr_el1); - if (!cpus_have_const_cap(ARM64_WORKAROUND_1319367)) { + if (!cpus_have_const_cap(ARM64_WORKAROUND_SPECULATIVE_AT) || + has_vhe()) { write_sysreg_el1(ctxt->sys_regs[SCTLR_EL1], SYS_SCTLR); write_sysreg_el1(ctxt->sys_regs[TCR_EL1], SYS_TCR); } else if (!ctxt->__hyp_running_vcpu) { @@ -149,7 +150,8 @@ static void __hyp_text __sysreg_restore_el1_state(struct kvm_cpu_context *ctxt) write_sysreg(ctxt->sys_regs[PAR_EL1], par_el1); write_sysreg(ctxt->sys_regs[TPIDR_EL1], tpidr_el1); - if (cpus_have_const_cap(ARM64_WORKAROUND_1319367) && + if (cpus_have_const_cap(ARM64_WORKAROUND_SPECULATIVE_AT) && + !has_vhe() && ctxt->__hyp_running_vcpu) { /* * Must only be done for host registers, hence the context diff --git a/arch/arm64/kvm/hyp/tlb.c b/arch/arm64/kvm/hyp/tlb.c index c2bc17ca6430..338df7f9149a 100644 --- a/arch/arm64/kvm/hyp/tlb.c +++ b/arch/arm64/kvm/hyp/tlb.c @@ -23,7 +23,7 @@ static void __hyp_text __tlb_switch_to_guest_vhe(struct kvm *kvm, local_irq_save(cxt->flags); - if (cpus_have_const_cap(ARM64_WORKAROUND_1165522)) { + if (cpus_have_const_cap(ARM64_WORKAROUND_SPECULATIVE_AT)) { /* * For CPUs that are affected by ARM erratum 1165522, we * cannot trust stage-1 to be in a correct state at that @@ -63,7 +63,7 @@ static void __hyp_text __tlb_switch_to_guest_vhe(struct kvm *kvm, static void __hyp_text __tlb_switch_to_guest_nvhe(struct kvm *kvm, struct tlb_inv_context *cxt) { - if (cpus_have_const_cap(ARM64_WORKAROUND_1319367)) { + if (cpus_have_const_cap(ARM64_WORKAROUND_SPECULATIVE_AT)) { u64 val; /* @@ -103,7 +103,7 @@ static void __hyp_text __tlb_switch_to_host_vhe(struct kvm *kvm, write_sysreg(HCR_HOST_VHE_FLAGS, hcr_el2); isb(); - if (cpus_have_const_cap(ARM64_WORKAROUND_1165522)) { + if (cpus_have_const_cap(ARM64_WORKAROUND_SPECULATIVE_AT)) { /* Restore the registers to what they were */ write_sysreg_el1(cxt->tcr, SYS_TCR); write_sysreg_el1(cxt->sctlr, SYS_SCTLR); @@ -117,7 +117,7 @@ static void __hyp_text __tlb_switch_to_host_nvhe(struct kvm *kvm, { write_sysreg(0, vttbr_el2); - if (cpus_have_const_cap(ARM64_WORKAROUND_1319367)) { + if (cpus_have_const_cap(ARM64_WORKAROUND_SPECULATIVE_AT)) { /* Ensure write of the host VMID */ isb(); /* Restore the host's TCR_EL1 */
Cortex-A57/A72 have a similar errata to Cortex-A76 regarding speculation of the AT instruction. Since the workaround for A57/A72 doesn't require VHE, the restriction enforcing VHE for A76 can be removed by combining the workaround flag for both errata. So combine WORKAROUND_1165522 and WORKAROUND_1319367 into WORKAROUND_SPECULATIVE_AT. The majority of code is contained within VHE or NVHE specific functions, for the cases where the code is shared extra checks against has_vhe(). This also paves the way for adding a similar erratum for Cortex-A55. Signed-off-by: Steven Price <steven.price@arm.com> --- arch/arm64/Kconfig | 5 +++++ arch/arm64/include/asm/cpucaps.h | 5 ++--- arch/arm64/include/asm/kvm_host.h | 4 ---- arch/arm64/include/asm/kvm_hyp.h | 3 ++- arch/arm64/kernel/cpu_errata.c | 19 ++++++++++++++----- arch/arm64/kvm/hyp/switch.c | 6 +++--- arch/arm64/kvm/hyp/sysreg-sr.c | 6 ++++-- arch/arm64/kvm/hyp/tlb.c | 8 ++++---- 8 files changed, 34 insertions(+), 22 deletions(-)