mbox series

[v7,0/7] running kernel mode SIMD with softirqs disabled

Message ID 20210827070342.218276-1-ardb@kernel.org (mailing list archive)
Headers show
Series running kernel mode SIMD with softirqs disabled | expand

Message

Ard Biesheuvel Aug. 27, 2021, 7:03 a.m. UTC
This is a follow-up to [0], but given that the arm64 architectural
pieces have been merged for arm64, the only remaining changes are crypto
specific. Therefore, the audience has been reduced to those people who
are somewhat more likely to care about these specifics.

The AEAD and skcipher APIs may only be called from task or softirq
context. This permits the arm64 AEAD and skcipher code to get rid of all
scalar fallbacks, given that on this architecture, softirqs are now no
longer served while the SIMD unit is being used in kernel mode, which
means that the scalar fallbacks are never needed. These are removed in
this series.

Changes since v6:
- add patch to yield the NEON every 4k of input when processing the AAD
- add some more acks from Eric

Changes since v5:
- add Eric's R-b to patches #1 to #3
- split CCM changes into 3 separate patches

Changes since v4:
- drop skcipher_walk layer change to deal with zero sized walks
- drop aead/skcipher layer sanity checks on invocations from hardirq
  context
- add patch to clean up CCM a bit more after removing the SIMD code path

Changes since v3:
- clarify the nature of the issue addressed by patch #1, and apply the
  same fix to the skcipher walker
- update patches #2 and #3 so that the failures can be observed by the
  crypto stats code

[0] https://lore.kernel.org/linux-arm-kernel/20210302090118.30666-1-ardb@kernel.org/

Ard Biesheuvel (7):
  crypto: arm64/gcm-aes-ce - remove non-SIMD fallback path
  crypto: arm64/aes-neonbs - stop using SIMD helper for skciphers
  crypto: arm64/aes-ce - stop using SIMD helper for skciphers
  crypto: arm64/aes-ccm - yield NEON when processing auth-only data
  crypto: arm64/aes-ccm - remove non-SIMD fallback path
  crypto: arm64/aes-ccm - reduce NEON begin/end calls for common case
  crypto: arm64/aes-ccm - avoid by-ref argument for ce_aes_ccm_auth_data

 arch/arm64/crypto/Kconfig           |   6 -
 arch/arm64/crypto/aes-ce-ccm-core.S |  24 +--
 arch/arm64/crypto/aes-ce-ccm-glue.c | 203 ++++++-------------
 arch/arm64/crypto/aes-glue.c        | 102 ++--------
 arch/arm64/crypto/aes-neonbs-glue.c | 122 +-----------
 arch/arm64/crypto/ghash-ce-glue.c   | 209 +++++---------------
 6 files changed, 148 insertions(+), 518 deletions(-)

Comments

Ard Biesheuvel Aug. 29, 2021, 6:35 a.m. UTC | #1
On Fri, 27 Aug 2021 at 09:03, Ard Biesheuvel <ardb@kernel.org> wrote:
>
> This is a follow-up to [0], but given that the arm64 architectural
> pieces have been merged for arm64, the only remaining changes are crypto
> specific. Therefore, the audience has been reduced to those people who
> are somewhat more likely to care about these specifics.
>
> The AEAD and skcipher APIs may only be called from task or softirq
> context. This permits the arm64 AEAD and skcipher code to get rid of all
> scalar fallbacks, given that on this architecture, softirqs are now no
> longer served while the SIMD unit is being used in kernel mode, which
> means that the scalar fallbacks are never needed. These are removed in
> this series.
>
> Changes since v6:
> - add patch to yield the NEON every 4k of input when processing the AAD
> - add some more acks from Eric
>
> Changes since v5:
> - add Eric's R-b to patches #1 to #3
> - split CCM changes into 3 separate patches
>
> Changes since v4:
> - drop skcipher_walk layer change to deal with zero sized walks
> - drop aead/skcipher layer sanity checks on invocations from hardirq
>   context
> - add patch to clean up CCM a bit more after removing the SIMD code path
>
> Changes since v3:
> - clarify the nature of the issue addressed by patch #1, and apply the
>   same fix to the skcipher walker
> - update patches #2 and #3 so that the failures can be observed by the
>   crypto stats code
>
> [0] https://lore.kernel.org/linux-arm-kernel/20210302090118.30666-1-ardb@kernel.org/
>
> Ard Biesheuvel (7):
>   crypto: arm64/gcm-aes-ce - remove non-SIMD fallback path
>   crypto: arm64/aes-neonbs - stop using SIMD helper for skciphers
>   crypto: arm64/aes-ce - stop using SIMD helper for skciphers
>   crypto: arm64/aes-ccm - yield NEON when processing auth-only data
>   crypto: arm64/aes-ccm - remove non-SIMD fallback path
>   crypto: arm64/aes-ccm - reduce NEON begin/end calls for common case
>   crypto: arm64/aes-ccm - avoid by-ref argument for ce_aes_ccm_auth_data
>

Herbert,

Any chance we could get this queued for v5.15? If it's too late,
please consider taking only the first three patches as an alternative,
and I will resend the CCM ones for v5.16 once they have all been
reviewed.

Thanks,
Ard.

>  arch/arm64/crypto/Kconfig           |   6 -
>  arch/arm64/crypto/aes-ce-ccm-core.S |  24 +--
>  arch/arm64/crypto/aes-ce-ccm-glue.c | 203 ++++++-------------
>  arch/arm64/crypto/aes-glue.c        | 102 ++--------
>  arch/arm64/crypto/aes-neonbs-glue.c | 122 +-----------
>  arch/arm64/crypto/ghash-ce-glue.c   | 209 +++++---------------
>  6 files changed, 148 insertions(+), 518 deletions(-)
>
> --
> 2.30.2
>
Herbert Xu Aug. 30, 2021, 6:26 a.m. UTC | #2
On Sun, Aug 29, 2021 at 08:35:37AM +0200, Ard Biesheuvel wrote:
>
> Any chance we could get this queued for v5.15? If it's too late,
> please consider taking only the first three patches as an alternative,
> and I will resend the CCM ones for v5.16 once they have all been
> reviewed.

Sorry, it's too late for that.  If these are to serve as dependencies
for other work, perhaps you can just add my acks to them and
submit them to the trees where they are needed?

Cheers,
Ard Biesheuvel Aug. 30, 2021, 1:48 p.m. UTC | #3
On Mon, 30 Aug 2021 at 08:26, Herbert Xu <herbert@gondor.apana.org.au> wrote:
>
> On Sun, Aug 29, 2021 at 08:35:37AM +0200, Ard Biesheuvel wrote:
> >
> > Any chance we could get this queued for v5.15? If it's too late,
> > please consider taking only the first three patches as an alternative,
> > and I will resend the CCM ones for v5.16 once they have all been
> > reviewed.
>
> Sorry, it's too late for that.  If these are to serve as dependencies
> for other work, perhaps you can just add my acks to them and
> submit them to the trees where they are needed?
>

Fair enough. It would be nice to get rid of the crypto_simd dependency
at least for the aes-ce driver, but I might propose it as a backport
once it hits mainline.

Thanks,
Ard.
Herbert Xu Sept. 17, 2021, 3:19 a.m. UTC | #4
On Fri, Aug 27, 2021 at 09:03:35AM +0200, Ard Biesheuvel wrote:
> This is a follow-up to [0], but given that the arm64 architectural
> pieces have been merged for arm64, the only remaining changes are crypto
> specific. Therefore, the audience has been reduced to those people who
> are somewhat more likely to care about these specifics.
> 
> The AEAD and skcipher APIs may only be called from task or softirq
> context. This permits the arm64 AEAD and skcipher code to get rid of all
> scalar fallbacks, given that on this architecture, softirqs are now no
> longer served while the SIMD unit is being used in kernel mode, which
> means that the scalar fallbacks are never needed. These are removed in
> this series.
> 
> Changes since v6:
> - add patch to yield the NEON every 4k of input when processing the AAD
> - add some more acks from Eric
> 
> Changes since v5:
> - add Eric's R-b to patches #1 to #3
> - split CCM changes into 3 separate patches
> 
> Changes since v4:
> - drop skcipher_walk layer change to deal with zero sized walks
> - drop aead/skcipher layer sanity checks on invocations from hardirq
>   context
> - add patch to clean up CCM a bit more after removing the SIMD code path
> 
> Changes since v3:
> - clarify the nature of the issue addressed by patch #1, and apply the
>   same fix to the skcipher walker
> - update patches #2 and #3 so that the failures can be observed by the
>   crypto stats code
> 
> [0] https://lore.kernel.org/linux-arm-kernel/20210302090118.30666-1-ardb@kernel.org/
> 
> Ard Biesheuvel (7):
>   crypto: arm64/gcm-aes-ce - remove non-SIMD fallback path
>   crypto: arm64/aes-neonbs - stop using SIMD helper for skciphers
>   crypto: arm64/aes-ce - stop using SIMD helper for skciphers
>   crypto: arm64/aes-ccm - yield NEON when processing auth-only data
>   crypto: arm64/aes-ccm - remove non-SIMD fallback path
>   crypto: arm64/aes-ccm - reduce NEON begin/end calls for common case
>   crypto: arm64/aes-ccm - avoid by-ref argument for ce_aes_ccm_auth_data
> 
>  arch/arm64/crypto/Kconfig           |   6 -
>  arch/arm64/crypto/aes-ce-ccm-core.S |  24 +--
>  arch/arm64/crypto/aes-ce-ccm-glue.c | 203 ++++++-------------
>  arch/arm64/crypto/aes-glue.c        | 102 ++--------
>  arch/arm64/crypto/aes-neonbs-glue.c | 122 +-----------
>  arch/arm64/crypto/ghash-ce-glue.c   | 209 +++++---------------
>  6 files changed, 148 insertions(+), 518 deletions(-)

All applied.  Thanks.