| Message ID | 20230609122031.183730-1-hao.xu@linux.dev (mailing list archive) |
|---|---|
| Headers | show |
| Series | fixed worker | expand |
Hi Jens and all, On 6/9/23 20:20, Hao Xu wrote: > From: Hao Xu <howeyxu@tencent.com> > > The initial feature request by users is here: > https://github.com/axboe/liburing/issues/296 > > Fixed worker provide a way for users to control the io-wq threads. A > fixed worker is worker thread which exists no matter there are works > to do or not. We provide a new register api to register fixed workers, > and a register api to unregister them as well. The parameter of the > register api is the number of fixed workers users want. > Here is a liburing test case to show how it works: https://github.com/axboe/liburing/commit/bc9e862d1317f2466381adb6243be8cc86c3bd27 Regards, Hao
Gently ping this series No idea why the mail threading is a mess, it looks fine in fsdevel list. On 6/9/23 20:20, Hao Xu wrote: > From: Hao Xu <howeyxu@tencent.com> > > The initial feature request by users is here: > https://github.com/axboe/liburing/issues/296 > > Fixed worker provide a way for users to control the io-wq threads. A > fixed worker is worker thread which exists no matter there are works > to do or not. We provide a new register api to register fixed workers, > and a register api to unregister them as well. The parameter of the > register api is the number of fixed workers users want. > > For example: > > ```c > io_uring_register_iowq_fixed_workers(&ring, { .nr_workers = 5 }) > do I/O works > io_uring_unregister_iowq_fixed_workers(&ring) > > ``` > > After registration, there will be 5 fixed workers. User can setup their > affinity, priority etc. freely, without adding any new register api to > set up attributions. These workers won't be destroyed until users call > unregister api. > > Note, registering some fixed workers doesn't mean no creating normal > workers. When there is no free workers, new normal workers can be > created when works come. So a work may be picked up by fixed workers or > normal workers. > > If users want to offload works only to fixed workers, they can specify > a flag FIXED_ONLY when registering fixed workers. > > ```c > io_uring_register_iowq_fixed_workers(&ring, { .nr_workers = 5, .flags |= > FIXED_ONLY }) > > ``` > > In above case, no normal workers will be created before calling > io_uring_register_iowq_fixed_workers(). > > Note: > - When registering fixed workers, those fixed workers are per io-wq. > So if an io_uring instance is shared by multiple tasks, and you want > all tasks to use fixed workers, all tasks have to call the regitser > api. > - if specifying FIXED_ONLY when registering fixed workers, that is per > io_uring instance. all works in this instance are handled by fixed > workers. > > Therefore, if an io_uring instance is shared by two tasks, and you want > all requests in this instance to be handled only by fixed workers, you > have to call the register api in these two tasks and specify FIXED_ONLY > at least once when calling register api. > > > Hao Xu (11): > io-wq: fix worker counting after worker received exit signal > io-wq: add a new worker flag to indicate worker exit > io-wq: add a new type io-wq worker > io-wq: add fixed worker members in io_wq_acct > io-wq: add a new parameter for creating a new fixed worker > io-wq: return io_worker after successful inline worker creation > io_uring: add new api to register fixed workers > io_uring: add function to unregister fixed workers > io-wq: add strutures to allow to wait fixed workers exit > io-wq: distinguish fixed worker by its name > io_uring: add IORING_SETUP_FIXED_WORKER_ONLY and its friend > > include/uapi/linux/io_uring.h | 20 +++ > io_uring/io-wq.c | 275 ++++++++++++++++++++++++++++++---- > io_uring/io-wq.h | 3 + > io_uring/io_uring.c | 132 +++++++++++++++- > 4 files changed, 397 insertions(+), 33 deletions(-) >
From: Hao Xu <howeyxu@tencent.com> The initial feature request by users is here: https://github.com/axboe/liburing/issues/296 Fixed worker provide a way for users to control the io-wq threads. A fixed worker is worker thread which exists no matter there are works to do or not. We provide a new register api to register fixed workers, and a register api to unregister them as well. The parameter of the register api is the number of fixed workers users want. For example: ```c io_uring_register_iowq_fixed_workers(&ring, { .nr_workers = 5 }) do I/O works io_uring_unregister_iowq_fixed_workers(&ring) ``` After registration, there will be 5 fixed workers. User can setup their affinity, priority etc. freely, without adding any new register api to set up attributions. These workers won't be destroyed until users call unregister api. Note, registering some fixed workers doesn't mean no creating normal workers. When there is no free workers, new normal workers can be created when works come. So a work may be picked up by fixed workers or normal workers. If users want to offload works only to fixed workers, they can specify a flag FIXED_ONLY when registering fixed workers. ```c io_uring_register_iowq_fixed_workers(&ring, { .nr_workers = 5, .flags |= FIXED_ONLY }) ``` In above case, no normal workers will be created before calling io_uring_register_iowq_fixed_workers(). Note: - When registering fixed workers, those fixed workers are per io-wq. So if an io_uring instance is shared by multiple tasks, and you want all tasks to use fixed workers, all tasks have to call the regitser api. - if specifying FIXED_ONLY when registering fixed workers, that is per io_uring instance. all works in this instance are handled by fixed workers. Therefore, if an io_uring instance is shared by two tasks, and you want all requests in this instance to be handled only by fixed workers, you have to call the register api in these two tasks and specify FIXED_ONLY at least once when calling register api. Hao Xu (11): io-wq: fix worker counting after worker received exit signal io-wq: add a new worker flag to indicate worker exit io-wq: add a new type io-wq worker io-wq: add fixed worker members in io_wq_acct io-wq: add a new parameter for creating a new fixed worker io-wq: return io_worker after successful inline worker creation io_uring: add new api to register fixed workers io_uring: add function to unregister fixed workers io-wq: add strutures to allow to wait fixed workers exit io-wq: distinguish fixed worker by its name io_uring: add IORING_SETUP_FIXED_WORKER_ONLY and its friend include/uapi/linux/io_uring.h | 20 +++ io_uring/io-wq.c | 275 ++++++++++++++++++++++++++++++---- io_uring/io-wq.h | 3 + io_uring/io_uring.c | 132 +++++++++++++++- 4 files changed, 397 insertions(+), 33 deletions(-)