[v3,for-next,0/3] io_uring: multishot recvmsg

Message

Dylan Yudaken July 14, 2022, 11:02 a.m. UTC

This series adds multishot support to recvmsg in io_uring.

The idea is that you submit a single multishot recvmsg and then receive
completions as and when data arrives. For recvmsg each completion also has
control data, and this is necessarily included in the same buffer as the
payload.

In order to do this a new structure is used: io_uring_recvmsg_out. This
specifies the length written of the name, control and payload. As well as
including the flags.
The layout of the buffer is <header><name><control><payload> where the
lengths are those specified in the original msghdr used to issue the recvmsg.

I suspect this API will be the most contentious part of this series and would
appreciate any comments on it.

For completeness I considered having the original struct msghdr as the header,
but size wise it is much bigger (72 bytes including an iovec vs 16 bytes here).
Testing also showed a 1% slowdown in terms of QPS.

Using a mini network tester [1] shows 14% QPS improvment using this API, however
this is likely to go down to ~8% with the latest allocation cache added by Jens.

[1]: https://github.com/DylanZA/netbench/tree/main

Patches 1,2 change the copy_msghdr code to take a user_msghdr as input
Patch 3 is the multishot feature

v3:
 * apply formatting comments
 * refactor io_recvmsg_prep_multishot to reduce casts
 * move some overflow logic into recvmsg_prep as only need to call it once

v2:
 * Rebase without netbuf recycling provided by io_uring
 * Fix payload field output with MSG_TRUNC set to match recvmsg(2)

Dylan Yudaken (3):
  net: copy from user before calling __copy_msghdr
  net: copy from user before calling __get_compat_msghdr
  io_uring: support multishot in recvmsg

 include/linux/socket.h        |   7 +-
 include/net/compat.h          |   5 +-
 include/uapi/linux/io_uring.h |   7 ++
 io_uring/net.c                | 216 ++++++++++++++++++++++++++++------
 io_uring/net.h                |   6 +
 net/compat.c                  |  39 +++---
 net/socket.c                  |  37 +++---
 7 files changed, 232 insertions(+), 85 deletions(-)


base-commit: 20898aeac6b82d8eb6247754494584b2f6cafd53

Comments

Jens Axboe July 14, 2022, 4:28 p.m. UTC | #1

On Thu, 14 Jul 2022 04:02:55 -0700, Dylan Yudaken wrote:
> This series adds multishot support to recvmsg in io_uring.
> 
> The idea is that you submit a single multishot recvmsg and then receive
> completions as and when data arrives. For recvmsg each completion also has
> control data, and this is necessarily included in the same buffer as the
> payload.
> 
> [...]

Applied, thanks!

[1/3] net: copy from user before calling __copy_msghdr
      commit: 03a3f428042c7752afa5dc5ffe3cf0b8f0e2acbf
[2/3] net: copy from user before calling __get_compat_msghdr
      commit: 1a3e4e94a1b95edb37cf0f66d7dfb32beb61df7f
[3/3] io_uring: support multishot in recvmsg
      commit: a8b38c4ce7240d869c820d457bcd51e452149510

Best regards,