[v2,0/3] ARM: v7: get rid of boot time mini stack

Message

Ard Biesheuvel Feb. 10, 2021, 6:55 p.m. UTC

The v7 boot code uses a small chunk of BSS to preserve some register
contents across a call to v7_invalidate_l1 that occurs with the MMU and
caches disabled. Memory accesses in such cases are tricky on v7+, given
that the architecture permits some unintuitive behaviors (it is
implementation defined whether accesses done with the MMU and caches off
may hit in the caches). Also, cache invalidation is not safe under
virtualization if the intent is to retain stores issued directly to DRAM,
as the hypervisor may upgrade invalidate operations to clean+invalidate,
resulting in DRAM contents to be overwritte by the dirty cachelines that
we were trying to evict in the first place.

So let's address this issue, by removing the need for this stack to
exist in the first place: v7_invalidate_l1 can be rewritten to use fewer
registers, which means fewer registers need to be preserved, and we have
enough spare registers available.

Patch #1 adds a missing ISB. This patch is included separately so it can
be backported if desired.

Patch #2 rewrites v7_invalidate_l1 so it only uses 5 registers (not
counting lr which it must preserve as well)

Patch #3 updates the callers to use spare registers instead of the mini
stack to stash the values that need to be preserved across the calls to
v7_invalidate_l1.

Changes since v1:
- use correct stop condition in outer loop (cc not mi)
- bring back ENDPROC() in patch #3
- add Nico's ack

This version of the series was build/boot tested here:
https://kernelci.org/build/ardb/branch/for-kernelci/kernel/v5.11-rc7-3-gafc656039a99/
https://kernelci.org/test/job/ardb/branch/for-kernelci/kernel/v5.11-rc7-3-gafc656039a99/plan/baseline/

Cc: Marc Zyngier <[e-mail]>
Cc: Russell King <[e-mail]>
Cc: Linus Walleij <[e-mail]>
Cc: Nicolas Pitre <[e-mail]>

Ard Biesheuvel (3):
  ARM: cache-v7: add missing ISB after cache level selection
  ARM: cache-v7: refactor v7_invalidate_l1 to avoid clobbering r5/r6
  ARM: cache-v7: get rid of mini-stack

 arch/arm/include/asm/memory.h | 15 -----
 arch/arm/mm/cache-v7.S        | 58 ++++++++++----------
 arch/arm/mm/proc-v7.S         | 39 ++++++-------
 3 files changed, 47 insertions(+), 65 deletions(-)

Comments

Linus Walleij March 1, 2021, 2:38 p.m. UTC | #1

On Wed, Feb 10, 2021 at 7:55 PM Ard Biesheuvel <[e-mail]> wrote:

> The v7 boot code uses a small chunk of BSS to preserve some register
> contents across a call to v7_invalidate_l1 that occurs with the MMU and
> caches disabled. Memory accesses in such cases are tricky on v7+, given
> that the architecture permits some unintuitive behaviors (it is
> implementation defined whether accesses done with the MMU and caches off
> may hit in the caches). Also, cache invalidation is not safe under
> virtualization if the intent is to retain stores issued directly to DRAM,
> as the hypervisor may upgrade invalidate operations to clean+invalidate,
> resulting in DRAM contents to be overwritte by the dirty cachelines that
> we were trying to evict in the first place.
>
> So let's address this issue, by removing the need for this stack to
> exist in the first place: v7_invalidate_l1 can be rewritten to use fewer
> registers, which means fewer registers need to be preserved, and we have
> enough spare registers available.
>
> Patch #1 adds a missing ISB. This patch is included separately so it can
> be backported if desired.
>
> Patch #2 rewrites v7_invalidate_l1 so it only uses 5 registers (not
> counting lr which it must preserve as well)
>
> Patch #3 updates the callers to use spare registers instead of the mini
> stack to stash the values that need to be preserved across the calls to
> v7_invalidate_l1.
>
> Changes since v1:
> - use correct stop condition in outer loop (cc not mi)
> - bring back ENDPROC() in patch #3
> - add Nico's ack

This looks good to me, so FWIW:
Acked-by: Linus Walleij <[e-mail]>

Yours,
Linus Walleij