[v2,0/3] arm64: errata: Workaround Neoverse-N1 #1542419

Message

James Morse Oct. 17, 2019, 5:42 p.m. UTC

Hello,

---
Changes since v1:
 * Rebased on rc3 to disappear patch4, now merged as dd8a1f134884
   This series is aimed for v5.5

 * Update silicon-errata.txt
 * Consider big-little, to avoid shearing DIC.
 * Added CTR_IMINLINE_MASK masks and comment

Changes are also noted in the tear-off of each patch.

---

Neoverse-N1 cores with the 'COHERENT_ICACHE' feature may fetch stale
instructions when software depends on prefetch-speculation-protection
instead of explicit synchronization. [0]

This concerns self modifying executables, like a JIT ... and the kernel.
The ARM-ARM has some fairly heavy rules for software that modifies
instructions that may be currently executing on another CPU.
There are some easier requirements if the instructions being modified
are just between Branches and NOP, (and a few others).

(See B2.2.5 "Concurrent modification and execution of instructions"
of DDI0487E.a for details).


A JIT can use all this to avoid synchronisation between threads. It
can generate new instructions at some new location, then update a
branch in the executable instructions to point at the new location.

Prefetch-speculation-protection guarantees that if another CPU sees
the new branch, it also sees the new instructions that were written
there.

On affected Neoverse-N1 cores, this can go wrong, fortunately its rare.

The workaround is to trap I-Cache maintenance and issue an
inner-shareable TLBI.

The affected cores have a Coherent I-Cache, so the I-Cache maintenance
isn't necessary. The core tells user-space it can skip it with
CTR_EL0.DIC. We also have to trap this register to hide the bit forcing
DIC-aware user-space to perform the maintenance.

Because the cache-maintenance wasn't needed, we can do the TLBI instead.
In fact, the I-Cache line-size isn't relevant anymore, we can reduce
the number of traps by producing a fake value.

Unfortunately the bulk of user-space is not DIC-aware, it blindly does
the D-side and I-side cache maintenance. To make matters worse, the
kernel can only trap all cache maintenance from EL0 with SCTLR_EL1.UCI.
The normal-world can't trap Data/Instruction cache maintenance
independently, but EL3 firmware can.

To avoid trapping all cache-maintenance, this workaround depends on
a firmware component that only traps I-cache maintenance from EL0 and
performs the workaround.


For user-space, the kernel's work is now to trap CTR_EL0 to hide DIC,
and produce a fake IminLine. EL3 traps the now-necessary I-Cache
maintenance and performs the inner-shareable-TLBI that makes everything
better.


We can't detect whether EL3 has the workaround for any particular
erratum. We lamely print '(kernel portion)' as part of the CPU-feature
text.

While the kernel has some JIT like features, they don't rely on
prefetch-speculation-protection. In particular the module-loader
ends up calling kick_all_cpus_sync() as part of the 'heavy' rules
above.

commit dd8a1f134884 ("arm64: ftrace: Ensure synchronisation in PLT setup
for Neoverse-N1 #1542419") lists the cases the kernel modifies itself.
Only one of these depends on prefetch-speculation-protection, it was easy
enough to remove it.

... questions welcome ...


Thanks,

James

James Morse (3):
  arm64: errata: Hide CTR_EL0.DIC on systems affected by Neoverse-N1
    #1542419
  arm64: Fake the IminLine size on systems affected by Neoverse-N1
    #1542419
  arm64: compat: Workaround Neoverse-N1 #1542419 for compat user-space

 Documentation/arm64/silicon-errata.rst |  2 ++
 arch/arm64/Kconfig                     | 16 +++++++++++++
 arch/arm64/include/asm/cache.h         |  3 ++-
 arch/arm64/include/asm/cpucaps.h       |  3 ++-
 arch/arm64/kernel/cpu_errata.c         | 32 +++++++++++++++++++++++++-
 arch/arm64/kernel/sys_compat.c         | 11 +++++++++
 arch/arm64/kernel/traps.c              |  9 ++++++++
 7 files changed, 73 insertions(+), 3 deletions(-)

Comments

Catalin Marinas Oct. 25, 2019, 4:46 p.m. UTC | #1

On Thu, Oct 17, 2019 at 06:42:57PM +0100, James Morse wrote:
> James Morse (3):
>   arm64: errata: Hide CTR_EL0.DIC on systems affected by Neoverse-N1
>     #1542419
>   arm64: Fake the IminLine size on systems affected by Neoverse-N1
>     #1542419
>   arm64: compat: Workaround Neoverse-N1 #1542419 for compat user-space

Queued for 5.5 (with the fix suggested by Suzuki). Thanks.