[RFC,0/2] optimise UDP skb completion wakeups

Message

Pavel Begunkov Feb. 7, 2024, 2:23 p.m. UTC

sock_wfree() tries to wake even when there are only read waiters and no
write waiters, which is the common scenario for io_uring. It'd also
attempt to wake if the write queue is far from being full. To avoid most
of this overhead add SOCK_NOSPACE support for UDP.

It brings +5% to UDP throughput with a CPU bound benchmark, and I observed
it taking 0.5-2% according to profiles in more realistic workloads.

Patch 1 introduces a new destructor udp_wfree(). The optimisation can be
implemented in sock_wfree() but that would either require patching up all
poll callbacks across the tree that are used in couple with sock_wfree(),
or limiting it to UDP with a flag.

Another option considered was to split out a write waitqueue, perhaps
conditionally, but it's bulkier, and the current version should also
benefit epoll workloads.

Pavel Begunkov (2):
  udp: introduce udp specific skb destructor
  udp: optimise write wakeups with SOCK_NOSPACE

 drivers/net/veth.c | 10 +++++---
 include/net/sock.h |  1 +
 include/net/udp.h  |  1 +
 net/core/sock.c    | 10 +++++---
 net/ipv4/udp.c     | 58 ++++++++++++++++++++++++++++++++++++++++++++--
 net/ipv6/udp.c     |  5 +++-
 6 files changed, 76 insertions(+), 9 deletions(-)