[00/29] crypto: scatterlist handling improvements

Message

Eric Biggers Dec. 21, 2024, 9:10 a.m. UTC

This series cleans up and optimizes the code that translates between
scatterlists (the input to the API) and virtual addresses (what software
implementations operate on) for skcipher and aead algorithms.

This takes the form of cleanups and optimizations to the skcipher_walk
functions and a rework of the underlying scatter_walk functions.

This series is organized as follows:

- Patch 1-8 are cleanups and optimizations for skcipher_walk.
- Patch 9-10 are cleanups for drivers that were unnecessarily using
  scatter_walk when simpler approaches existed.
- Patch 11-14 improve scatter_walk, introducing easier-to-use functions
  and optimizing performance in some cases.
- Patch 15-27 convert users to use the new functions.
- Patch 28 removes functions that are no longer needed.
- Patch 29 optimizes the walker on !HIGHMEM platforms to start returning
  data segments that can cross a page boundary.  This can significantly
  improve performance in cases where messages can cross pages, such as
  IPsec.  Previously there was a large overhead caused by packets being
  unnecessarily divided into multiple parts by the walker, including
  hitting skcipher_next_slow() which uses a single-block bounce buffer.

Eric Biggers (29):
  crypto: skcipher - document skcipher_walk_done() and rename some vars
  crypto: skcipher - remove unnecessary page alignment of bounce buffer
  crypto: skcipher - remove redundant clamping to page size
  crypto: skcipher - remove redundant check for SKCIPHER_WALK_SLOW
  crypto: skcipher - fold skcipher_walk_skcipher() into
    skcipher_walk_virt()
  crypto: skcipher - clean up initialization of skcipher_walk::flags
  crypto: skcipher - optimize initializing skcipher_walk fields
  crypto: skcipher - call cond_resched() directly
  crypto: omap - switch from scatter_walk to plain offset
  crypto: powerpc/p10-aes-gcm - simplify handling of linear associated
    data
  crypto: scatterwalk - move to next sg entry just in time
  crypto: scatterwalk - add new functions for skipping data
  crypto: scatterwalk - add new functions for iterating through data
  crypto: scatterwalk - add new functions for copying data
  crypto: skcipher - use scatterwalk_start_at_pos()
  crypto: aegis - use the new scatterwalk functions
  crypto: arm/ghash - use the new scatterwalk functions
  crypto: arm64 - use the new scatterwalk functions
  crypto: keywrap - use the new scatterwalk functions
  crypto: nx - use the new scatterwalk functions
  crypto: s390/aes-gcm - use the new scatterwalk functions
  crypto: s5p-sss - use the new scatterwalk functions
  crypto: stm32 - use the new scatterwalk functions
  crypto: x86/aes-gcm - use the new scatterwalk functions
  crypto: x86/aegis - use the new scatterwalk functions
  net/tls: use the new scatterwalk functions
  crypto: skcipher - use the new scatterwalk functions
  crypto: scatterwalk - remove obsolete functions
  crypto: scatterwalk - don't split at page boundaries when !HIGHMEM

 arch/arm/crypto/ghash-ce-glue.c        |  15 +-
 arch/arm64/crypto/aes-ce-ccm-glue.c    |  17 +-
 arch/arm64/crypto/ghash-ce-glue.c      |  16 +-
 arch/arm64/crypto/sm4-ce-ccm-glue.c    |  27 ++-
 arch/arm64/crypto/sm4-ce-gcm-glue.c    |  31 ++-
 arch/powerpc/crypto/aes-gcm-p10-glue.c |   8 +-
 arch/s390/crypto/aes_s390.c            |  33 ++--
 arch/x86/crypto/aegis128-aesni-glue.c  |  10 +-
 arch/x86/crypto/aesni-intel_glue.c     |  28 +--
 crypto/aegis128-core.c                 |  10 +-
 crypto/keywrap.c                       |  48 +----
 crypto/scatterwalk.c                   |  91 +++++----
 crypto/skcipher.c                      | 253 ++++++++++---------------
 drivers/crypto/nx/nx-aes-ccm.c         |  16 +-
 drivers/crypto/nx/nx-aes-gcm.c         |  17 +-
 drivers/crypto/nx/nx.c                 |  31 +--
 drivers/crypto/nx/nx.h                 |   3 -
 drivers/crypto/omap-aes.c              |  34 ++--
 drivers/crypto/omap-aes.h              |   6 +-
 drivers/crypto/omap-des.c              |  40 ++--
 drivers/crypto/s5p-sss.c               |  38 ++--
 drivers/crypto/stm32/stm32-cryp.c      |  34 ++--
 include/crypto/internal/skcipher.h     |   2 +-
 include/crypto/scatterwalk.h           | 181 ++++++++++++++----
 net/tls/tls_device_fallback.c          |  16 +-
 25 files changed, 454 insertions(+), 551 deletions(-)


base-commit: f916e44487f56df4827069ff3a2070c0746dc511