[RFC,v2,0/8] arm64/sve: First steps towards optimizing syscalls

Message

Julien Grall June 13, 2019, 4:16 p.m. UTC

Hi all,

This is a first attempt to optimize the syscall path when the user
application uses SVE. The patch series is based on v5.2-rc4.

Per the syscall ABI, SVE registers will be unknown after a syscall. In
practice, the kernel will disable SVE and the registers will be zeroed
(except the first 128-bits of each vector) on the next SVE instruction.
In a workload mixing SVE and syscalls, this will result to 2 entry/exit
to the kernel per syscall.

This series aims to avoid the second entry/exit by zeroing the SVE
registers on syscall return with a twist when the task will get
rescheduled.

This implementation will have an impact on application using SVE
only once. SVE will now be turned on until the application terminates
(unless disabling it via ptrace). Cleverer strategies for choosing
between SVE and FPSIMD context switching are possible (see [1]), but
it is difficult to assess the benefit right now. We could improve the
behaviour in the future as a selection of mature hardware platform
emerges that we can benchmark.

It is also possible to optimize the case when the SVE vector-length
is 128-bit (i.e the same size as the FPSIMD vectors). This could be
explored in the future respin of the series.

While developing the series, I have added a series of tracepoint in
the SVE code. They may not be suitable for upstreaming and hence not
included in the series. I can provide them if anyone is interested.

Note that the last patch for the series is is not here to optimize syscall
but SVE trap access by directly converting in hardware the FPSIMD state
to SVE state. If there are an interest to have this optimization earlier,
I can reshuffle the patches in the series.

Comments are welcomed.

Best regards,

[1] https://git.sphere.ly/dtc/kernel_moto_falcon/commit/acc207616a91a413a50fdd8847a747c4a7324167

Julien Grall (8):
  arm64/fpsimd: Update documentation of do_sve_acc
  arm64/signal: Update the comment in preserve_sve_context
  arm64/fpsimdmacros: Allow the macro "for" to be used in more cases
  arm64/fpsimdmacros: Introduce a macro to update ZCR_EL1.LEN
  arm64/sve: Implement an helper to flush SVE registers
  arm64/sve: Implement an helper to load SVE registers from FPSIMD state
  arm64/sve: Don't disable SVE on syscalls return
  arm64/sve: Rework SVE trap access to use TIF_SVE_NEEDS_FLUSH

 arch/arm64/include/asm/fpsimd.h       |  8 +++++
 arch/arm64/include/asm/fpsimdmacros.h | 48 +++++++++++++++++++------
 arch/arm64/include/asm/thread_info.h  |  5 ++-
 arch/arm64/kernel/entry-fpsimd.S      | 29 +++++++++++++++
 arch/arm64/kernel/fpsimd.c            | 67 ++++++++++++++++++++++++++++-------
 arch/arm64/kernel/process.c           |  1 +
 arch/arm64/kernel/ptrace.c            |  7 ++++
 arch/arm64/kernel/signal.c            | 17 +++++++--
 arch/arm64/kernel/syscall.c           | 13 +++----
 9 files changed, 162 insertions(+), 33 deletions(-)

Comments

Dave Martin June 21, 2019, 3:32 p.m. UTC | #1

On Thu, Jun 13, 2019 at 05:16:48PM +0100, Julien Grall wrote:
> Hi all,
> 
> This is a first attempt to optimize the syscall path when the user
> application uses SVE. The patch series is based on v5.2-rc4.
> 
> Per the syscall ABI, SVE registers will be unknown after a syscall. In
> practice, the kernel will disable SVE and the registers will be zeroed
> (except the first 128-bits of each vector) on the next SVE instruction.
> In a workload mixing SVE and syscalls, this will result to 2 entry/exit
> to the kernel per syscall.
> 
> This series aims to avoid the second entry/exit by zeroing the SVE
> registers on syscall return with a twist when the task will get
> rescheduled.
> 
> This implementation will have an impact on application using SVE
> only once. SVE will now be turned on until the application terminates
> (unless disabling it via ptrace). Cleverer strategies for choosing
> between SVE and FPSIMD context switching are possible (see [1]), but
> it is difficult to assess the benefit right now. We could improve the
> behaviour in the future as a selection of mature hardware platform
> emerges that we can benchmark.

I'm wondering whether we ought to do something about this such as
turning SVE back off after the nth syscall.  Given the complexity of
this code though, let's stabilise the series as-is first.

I probably ask dumb questions in some places, since I'm trying to
refresh my memory of the subtleties of this code as I go...

> It is also possible to optimize the case when the SVE vector-length
> is 128-bit (i.e the same size as the FPSIMD vectors). This could be
> explored in the future respin of the series.
> 
> While developing the series, I have added a series of tracepoint in
> the SVE code. They may not be suitable for upstreaming and hence not
> included in the series. I can provide them if anyone is interested.
> 
> Note that the last patch for the series is is not here to optimize syscall
> but SVE trap access by directly converting in hardware the FPSIMD state
> to SVE state. If there are an interest to have this optimization earlier,
> I can reshuffle the patches in the series.

I think this could make sense.  Maybe see what Will and Catalin think
about it.

[...]

Cheers
---Dave