[PATCHSET,v2,0/17] Improve async state handling

Message

Jens Axboe March 20, 2024, 10:55 p.m. UTC

Hi,

This patchset gets rid of on-stack state, that is then fixed up and
copied if we need to go async. Having to do this fixup is nasty
business, and this is the main motivation for the change.

Opcodes are converted to setting up their async context at prep time,
which means that everything is stable beyond that. No more special
io_req_prep_async() handling, and no more "oops we can't proceed,
let's now allocate memory, copy state, and be ready for a retry".
By default, opcodes are now always ready for a retry, and the issue
path can be simplified. This is most readily apparent in the read/write
handling, but can be seen on the net side too.

Lastly, the alloc cache is rewritten to be array based rather than list
based. List based isn't a great choice, as grabbing an element from the
list also means you have to touch the next one.

With all of that, performance is as good as before, or better, and we
drop quite a bit of code.

The diffstat reflects that, but doesn't even tell the full story. Most
of the added lines are trivial, whereas some of the removed lines are
pretty hairy.

Changes since v1:
- Cleanups
- Switch connect to using io_async_msghdr, now it gets recycling
  too
- Avoid recycling for read/write if io-wq is used
- Fix errant io_async_rw shadowing in io_write()
- Change alloc_cache to be array based
- Fix KASAN issues. Not with mem reuse, but just errors in my
  implementation of it for the mempool.
- Only mark iovec caching as REQ_F_NEED_CLEANUP
- Shuffle some hunks around between patches
- Fix an issue with send zerocopy and iovec freeing
- Move connect to io_async_msghdr so it can tap into the recycling
- Actually delete struct io_rw_state, not just its elements
- Add uring_cmd optimization that avoids sqe copy unless needed
- Rebase on for-6.10/io_uring

 include/linux/io_uring_types.h |   4 +-
 io_uring/alloc_cache.h         |  51 ++--
 io_uring/futex.c               |  26 +-
 io_uring/futex.h               |   5 +-
 io_uring/io_uring.c            |  71 ++---
 io_uring/io_uring.h            |   1 -
 io_uring/net.c                 | 550 +++++++++++++++++----------------------
 io_uring/net.h                 |  27 +-
 io_uring/opdef.c               |  65 ++---
 io_uring/opdef.h               |   9 +-
 io_uring/poll.c                |  11 +-
 io_uring/poll.h                |   7 +-
 io_uring/rsrc.c                |   9 +-
 io_uring/rsrc.h                |   5 +-
 io_uring/rw.c                  | 570 +++++++++++++++++++++--------------------
 io_uring/rw.h                  |  25 +-
 io_uring/uring_cmd.c           |  75 ++++--
 io_uring/uring_cmd.h           |   7 +-
 18 files changed, 707 insertions(+), 811 deletions(-)