[v5,00/11] fixed worker

Message

Hao Xu June 27, 2022, 1:35 p.m. UTC

From: Hao Xu <[e-mail]>

This is the second version of fixed worker implementation.
Wrote a nop test program to test it, 3 fixed-workers VS 3 normal workers.
normal workers:
./run_nop_wqe.sh nop_wqe_normal 200000 100 3 1-3
        time spent: 17314274 usecs      IOPS: 1155116
        time spent: 17016942 usecs      IOPS: 1175299
        time spent: 17908684 usecs      IOPS: 1116776

fixed workers:
./run_nop_wqe.sh nop_wqe_fixed 200000 100 3 1-3
        time spent: 10464397 usecs      IOPS: 1911242
        time spent: 9610976 usecs       IOPS: 2080954
        time spent: 9807361 usecs       IOPS: 2039284

About 2x improvement. From perf result, almost no acct->lock contension.
Test program: https://github.com/HowHsu/liburing/tree/fixed_worker
liburing/test/nop_wqe.c

v3->v4:
 - make work in fixed worker's private worfixed worker
 - tweak the io_wqe_acct struct to make it clearer

v4->v5:
 - code clean
 - rebase against the for-5.20/io_uring


Hao Xu (11):
  io-wq: add a worker flag for individual exit
  io-wq: change argument of create_io_worker() for convienence
  io-wq: add infra data structure for fixed workers
  io-wq: tweak io_get_acct()
  io-wq: fixed worker initialization
  io-wq: fixed worker exit
  io-wq: implement fixed worker logic
  io-wq: batch the handling of fixed worker private works
  io_uring: add register fixed worker interface
  io-wq: add an work list for fixed worker
  io_uring: cancel works in exec work list for fixed worker

 include/uapi/linux/io_uring.h |  11 +
 io_uring/io-wq.c              | 495 ++++++++++++++++++++++++++++++----
 io_uring/io-wq.h              |   8 +
 io_uring/io_uring.c           |  71 +++++
 4 files changed, 537 insertions(+), 48 deletions(-)


base-commit: 094abe8fbccb0d79bef982c67eb7372e92452c0e

Comments

Hao Xu July 12, 2022, 8:32 a.m. UTC | #1

On 6/27/22 21:35, Hao Xu wrote:
> From: Hao Xu <[e-mail]>
> 
> This is the second version of fixed worker implementation.
> Wrote a nop test program to test it, 3 fixed-workers VS 3 normal workers.
> normal workers:
> ./run_nop_wqe.sh nop_wqe_normal 200000 100 3 1-3
>          time spent: 17314274 usecs      IOPS: 1155116
>          time spent: 17016942 usecs      IOPS: 1175299
>          time spent: 17908684 usecs      IOPS: 1116776
> 
> fixed workers:
> ./run_nop_wqe.sh nop_wqe_fixed 200000 100 3 1-3
>          time spent: 10464397 usecs      IOPS: 1911242
>          time spent: 9610976 usecs       IOPS: 2080954
>          time spent: 9807361 usecs       IOPS: 2039284
> 
> About 2x improvement. From perf result, almost no acct->lock contension.
> Test program: https://github.com/HowHsu/liburing/tree/fixed_worker
> liburing/test/nop_wqe.c
> 
> v3->v4:
>   - make work in fixed worker's private worfixed worker
>   - tweak the io_wqe_acct struct to make it clearer
> 
> v4->v5:
>   - code clean
>   - rebase against the for-5.20/io_uring
> 
> 
> Hao Xu (11):
>    io-wq: add a worker flag for individual exit
>    io-wq: change argument of create_io_worker() for convienence
>    io-wq: add infra data structure for fixed workers
>    io-wq: tweak io_get_acct()
>    io-wq: fixed worker initialization
>    io-wq: fixed worker exit
>    io-wq: implement fixed worker logic
>    io-wq: batch the handling of fixed worker private works
>    io_uring: add register fixed worker interface
>    io-wq: add an work list for fixed worker
>    io_uring: cancel works in exec work list for fixed worker
> 
>   include/uapi/linux/io_uring.h |  11 +
>   io_uring/io-wq.c              | 495 ++++++++++++++++++++++++++++++----
>   io_uring/io-wq.h              |   8 +
>   io_uring/io_uring.c           |  71 +++++
>   4 files changed, 537 insertions(+), 48 deletions(-)
> 
> 
> base-commit: 094abe8fbccb0d79bef982c67eb7372e92452c0e

ping...