[v6,0/4] fast poll multishot mode

Message

Hao Xu May 14, 2022, 2:20 p.m. UTC

Let multishot support multishot mode, currently only add accept as its
first consumer.
theoretical analysis:
  1) when connections come in fast
    - singleshot:
              add accept sqe(userspace) --> accept inline
                              ^                 |
                              |-----------------|
    - multishot:
             add accept sqe(userspace) --> accept inline
                                              ^     |
                                              |--*--|

    we do accept repeatedly in * place until get EAGAIN

  2) when connections come in at a low pressure
    similar thing like 1), we reduce a lot of userspace-kernel context
    switch and useless vfs_poll()


tests:
Did some tests, which goes in this way:

  server    client(multiple)
  accept    connect
  read      write
  write     read
  close     close

Basically, raise up a number of clients(on same machine with server) to
connect to the server, and then write some data to it, the server will
write those data back to the client after it receives them, and then
close the connection after write return. Then the client will read the
data and then close the connection. Here I test 10000 clients connect
one server, data size 128 bytes. And each client has a go routine for
it, so they come to the server in short time.
test 20 times before/after this patchset, time spent:(unit cycle, which
is the return value of clock())
before:
  1930136+1940725+1907981+1947601+1923812+1928226+1911087+1905897+1941075
  +1934374+1906614+1912504+1949110+1908790+1909951+1941672+1969525+1934984
  +1934226+1914385)/20.0 = 1927633.75
after:
  1858905+1917104+1895455+1963963+1892706+1889208+1874175+1904753+1874112
  +1874985+1882706+1884642+1864694+1906508+1916150+1924250+1869060+1889506
  +1871324+1940803)/20.0 = 1894750.45

(1927633.75 - 1894750.45) / 1927633.75 = 1.65%

v1->v2:
 - re-implement it against the reworked poll code

v2->v3:
 - fold in code tweak and clean from Jens
 - use io_issue_sqe rather than io_queue_sqe, since the former one
   return the internal error back which makes more sense
 - remove io_poll_clean() and its friends since they are not needed

v3->v4:
 - move the accept multishot flag to the proper patch
 - typo correction
 - remove improperly added signed-off-by

v4->v5:
 - address some email account issue

v5->v6:
 - support multishot accept with fixed file

Hao Xu (4):
  io_uring: add IORING_ACCEPT_MULTISHOT for accept
  io_uring: add REQ_F_APOLL_MULTISHOT for requests
  io_uring: let fast poll support multishot
  io_uring: implement multishot mode for accept

 fs/io_uring.c                 | 106 +++++++++++++++++++++++++++-------
 include/uapi/linux/io_uring.h |   5 ++
 2 files changed, 90 insertions(+), 21 deletions(-)


base-commit: 1b1d7b4bf1d9948c8dba5ee550459ce7c65ac019

Comments

Jens Axboe May 14, 2022, 3:15 p.m. UTC | #1

On Sat, 14 May 2022 22:20:42 +0800, Hao Xu wrote:
> Let multishot support multishot mode, currently only add accept as its
> first consumer.
> theoretical analysis:
>   1) when connections come in fast
>     - singleshot:
>               add accept sqe(userspace) --> accept inline
>                               ^                 |
>                               |-----------------|
>     - multishot:
>              add accept sqe(userspace) --> accept inline
>                                               ^     |
>                                               |--*--|
> 
> [...]

Applied, thanks!

[1/4] io_uring: add IORING_ACCEPT_MULTISHOT for accept
      commit: 390ed29b5e425ba00da2b6113b74a14949f71b02
[2/4] io_uring: add REQ_F_APOLL_MULTISHOT for requests
      commit: 227685ebfaba0bc7e2ddc47cef4556050b6d7a8f
[3/4] io_uring: let fast poll support multishot
      commit: dbc2564cfe0faff439dc46adb8c009589054ea46
[4/4] io_uring: implement multishot mode for accept
      commit: 4e86a2c980137f7be1ea600af5f1f5c8342ecc09

Best regards,