| Message ID | 20191002094935.48848-2-james.morse@arm.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | arm64: errata: Workaround Neoverse-N1 #1542419 | expand |
Hi James, On 02/10/2019 10:49, James Morse wrote: > Cores affected by Neoverse-N1 #1542419 could execute a stale instruction > when a branch is updated to point to freshly generated instructions. > > To workaround this issue we need user-space to issue unnecessary > icache maintenance that we can trap. Start by hiding CTR_EL0.DIC. > > Signed-off-by: James Morse <james.morse@arm.com> > --- > arch/arm64/Kconfig | 16 ++++++++++++++++ > arch/arm64/include/asm/cpucaps.h | 3 ++- > arch/arm64/kernel/cpu_errata.c | 30 ++++++++++++++++++++++++++++++ > arch/arm64/kernel/traps.c | 3 +++ > 4 files changed, 51 insertions(+), 1 deletion(-) > > diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig > index 41a9b4257b72..f2e1965d2461 100644 > --- a/arch/arm64/Kconfig > +++ b/arch/arm64/Kconfig > @@ -559,6 +559,22 @@ config ARM64_ERRATUM_1463225 > > If unsure, say Y. > > +config ARM64_ERRATUM_1542419 > + bool "Neoverse-N1: workaround mis-ordering of instruction fetches" > + default y > + help > + This option adds a workaround for ARM Neoverse-N1 erratum > + 1542419. > + > + Affected Neoverse-N1 cores could execute a stale instruction when > + modified by another CPU. The workaround depends on a firmware > + counterpart. > + > + Workaround the issue by hiding the DIC feature from EL0. This > + forces user-space to perform cache maintenance. > + > + If unsure, say Y. > + > config CAVIUM_ERRATUM_22375 > bool "Cavium erratum 22375, 24313" > default y > diff --git a/arch/arm64/include/asm/cpucaps.h b/arch/arm64/include/asm/cpucaps.h > index f19fe4b9acc4..f05afaec18cd 100644 > --- a/arch/arm64/include/asm/cpucaps.h > +++ b/arch/arm64/include/asm/cpucaps.h > @@ -52,7 +52,8 @@ > #define ARM64_HAS_IRQ_PRIO_MASKING 42 > #define ARM64_HAS_DCPODP 43 > #define ARM64_WORKAROUND_1463225 44 > +#define ARM64_WORKAROUND_1542419 45 > > -#define ARM64_NCAPS 45 > +#define ARM64_NCAPS 46 > > #endif /* __ASM_CPUCAPS_H */ > diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c > index 1e43ba5c79b7..a7de0d5dde9a 100644 > --- a/arch/arm64/kernel/cpu_errata.c > +++ b/arch/arm64/kernel/cpu_errata.c > @@ -90,10 +90,18 @@ static void > cpu_enable_trap_ctr_access(const struct arm64_cpu_capabilities *__unused) > { > u64 mask = arm64_ftr_reg_ctrel0.strict_mask; > + bool enable_uct_trap = false; > > /* Trap CTR_EL0 access on this CPU, only if it has a mismatch */ > if ((read_cpuid_cachetype() & mask) != > (arm64_ftr_reg_ctrel0.sys_val & mask)) > + enable_uct_trap = true; > + > + /* ... or if this CPU is affected by an errata */ > + if (this_cpu_has_cap(ARM64_WORKAROUND_1542419)) > + enable_uct_trap = true; I think we need to trap the CTR accesses on all the CPUs if at least one of the CPU is affected by the Errata. i.e, even if both the LITTLE and the big CPU has DIC, we must trap this on both types to make sure the application doesn't use a cached value of the DIC read on the CPU that is not affected. So we could change this enable() routine a bit to : cpu_enable_trap_ctr_access(const struct arm64_cpu_capabilities *cap) { .... existing checks .... if (cap->capability == ARM64_WORKAROUND_1542419) enable_uct_trap = true; } Rest looks good to me. Cheers Suuzki
Hi Suzuki, On 07/10/2019 10:38, Suzuki K Poulose wrote: > On 02/10/2019 10:49, James Morse wrote: >> Cores affected by Neoverse-N1 #1542419 could execute a stale instruction >> when a branch is updated to point to freshly generated instructions. >> >> To workaround this issue we need user-space to issue unnecessary >> icache maintenance that we can trap. Start by hiding CTR_EL0.DIC. >> diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c >> index 1e43ba5c79b7..a7de0d5dde9a 100644 >> --- a/arch/arm64/kernel/cpu_errata.c >> +++ b/arch/arm64/kernel/cpu_errata.c >> @@ -90,10 +90,18 @@ static void >> cpu_enable_trap_ctr_access(const struct arm64_cpu_capabilities *__unused) >> { >> u64 mask = arm64_ftr_reg_ctrel0.strict_mask; >> + bool enable_uct_trap = false; >> /* Trap CTR_EL0 access on this CPU, only if it has a mismatch */ >> if ((read_cpuid_cachetype() & mask) != >> (arm64_ftr_reg_ctrel0.sys_val & mask)) >> + enable_uct_trap = true; >> + >> + /* ... or if this CPU is affected by an errata */ >> + if (this_cpu_has_cap(ARM64_WORKAROUND_1542419)) >> + enable_uct_trap = true; > > I think we need to trap the CTR accesses on all the CPUs if at least one > of the CPU is affected by the Errata. i.e, even if both the LITTLE and the > big CPU has DIC, we must trap this on both types to make sure the application > doesn't use a cached value of the DIC read on the CPU that is not affected. Good point! ... I hadn't considered big-little. Thanks, James
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 41a9b4257b72..f2e1965d2461 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -559,6 +559,22 @@ config ARM64_ERRATUM_1463225 If unsure, say Y. +config ARM64_ERRATUM_1542419 + bool "Neoverse-N1: workaround mis-ordering of instruction fetches" + default y + help + This option adds a workaround for ARM Neoverse-N1 erratum + 1542419. + + Affected Neoverse-N1 cores could execute a stale instruction when + modified by another CPU. The workaround depends on a firmware + counterpart. + + Workaround the issue by hiding the DIC feature from EL0. This + forces user-space to perform cache maintenance. + + If unsure, say Y. + config CAVIUM_ERRATUM_22375 bool "Cavium erratum 22375, 24313" default y diff --git a/arch/arm64/include/asm/cpucaps.h b/arch/arm64/include/asm/cpucaps.h index f19fe4b9acc4..f05afaec18cd 100644 --- a/arch/arm64/include/asm/cpucaps.h +++ b/arch/arm64/include/asm/cpucaps.h @@ -52,7 +52,8 @@ #define ARM64_HAS_IRQ_PRIO_MASKING 42 #define ARM64_HAS_DCPODP 43 #define ARM64_WORKAROUND_1463225 44 +#define ARM64_WORKAROUND_1542419 45 -#define ARM64_NCAPS 45 +#define ARM64_NCAPS 46 #endif /* __ASM_CPUCAPS_H */ diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c index 1e43ba5c79b7..a7de0d5dde9a 100644 --- a/arch/arm64/kernel/cpu_errata.c +++ b/arch/arm64/kernel/cpu_errata.c @@ -90,10 +90,18 @@ static void cpu_enable_trap_ctr_access(const struct arm64_cpu_capabilities *__unused) { u64 mask = arm64_ftr_reg_ctrel0.strict_mask; + bool enable_uct_trap = false; /* Trap CTR_EL0 access on this CPU, only if it has a mismatch */ if ((read_cpuid_cachetype() & mask) != (arm64_ftr_reg_ctrel0.sys_val & mask)) + enable_uct_trap = true; + + /* ... or if this CPU is affected by an errata */ + if (this_cpu_has_cap(ARM64_WORKAROUND_1542419)) + enable_uct_trap = true; + + if (enable_uct_trap) sysreg_clear_set(sctlr_el1, SCTLR_EL1_UCT, 0); } @@ -623,6 +631,18 @@ check_branch_predictor(const struct arm64_cpu_capabilities *entry, int scope) return (need_wa > 0); } +static bool __maybe_unused +has_neoverse_n1_erratum_1542419(const struct arm64_cpu_capabilities *entry, + int scope) +{ + u32 midr = read_cpuid_id(); + bool has_dic = read_cpuid_cachetype() & BIT(CTR_DIC_SHIFT); + const struct midr_range range = MIDR_ALL_VERSIONS(MIDR_NEOVERSE_N1); + + WARN_ON(scope != SCOPE_LOCAL_CPU || preemptible()); + return is_midr_in_range(midr, &range) && has_dic; +} + #ifdef CONFIG_HARDEN_EL2_VECTORS static const struct midr_range arm64_harden_el2_vectors[] = { @@ -851,6 +871,16 @@ const struct arm64_cpu_capabilities arm64_errata[] = { .type = ARM64_CPUCAP_LOCAL_CPU_ERRATUM, .matches = has_cortex_a76_erratum_1463225, }, +#endif +#ifdef CONFIG_ARM64_ERRATUM_1542419 + { + /* we depend on the firmware portion for correctness */ + .desc = "ARM erratum 1542419 (kernel portion)", + .capability = ARM64_WORKAROUND_1542419, + .type = ARM64_CPUCAP_LOCAL_CPU_ERRATUM, + .matches = has_neoverse_n1_erratum_1542419, + .cpu_enable = cpu_enable_trap_ctr_access, + }, #endif { } diff --git a/arch/arm64/kernel/traps.c b/arch/arm64/kernel/traps.c index 34739e80211b..465f0a0f8f0a 100644 --- a/arch/arm64/kernel/traps.c +++ b/arch/arm64/kernel/traps.c @@ -470,6 +470,9 @@ static void ctr_read_handler(unsigned int esr, struct pt_regs *regs) int rt = ESR_ELx_SYS64_ISS_RT(esr); unsigned long val = arm64_ftr_reg_user_value(&arm64_ftr_reg_ctrel0); + if (cpus_have_const_cap(ARM64_WORKAROUND_1542419)) + val &= ~BIT(CTR_DIC_SHIFT); + pt_regs_write_reg(regs, rt, val); arm64_skip_faulting_instruction(regs, AARCH64_INSN_SIZE);
Cores affected by Neoverse-N1 #1542419 could execute a stale instruction when a branch is updated to point to freshly generated instructions. To workaround this issue we need user-space to issue unnecessary icache maintenance that we can trap. Start by hiding CTR_EL0.DIC. Signed-off-by: James Morse <james.morse@arm.com> --- arch/arm64/Kconfig | 16 ++++++++++++++++ arch/arm64/include/asm/cpucaps.h | 3 ++- arch/arm64/kernel/cpu_errata.c | 30 ++++++++++++++++++++++++++++++ arch/arm64/kernel/traps.c | 3 +++ 4 files changed, 51 insertions(+), 1 deletion(-)