| Message ID | 20230609122031.183730-4-hao.xu@linux.dev (mailing list archive) |
|---|---|
| State | New |
| Headers | show |
| Series | fixed worker | expand |
On 6/9/23 13:20, Hao Xu wrote: > From: Hao Xu <howeyxu@tencent.com> > > Add a new type io-wq worker IO_WORKER_F_FIXED, this type of worker > exists during the whole io-wq lifecycle. > > Signed-off-by: Hao Xu <howeyxu@tencent.com> > --- > io_uring/io-wq.c | 13 ++++++++++++- > 1 file changed, 12 insertions(+), 1 deletion(-) > > diff --git a/io_uring/io-wq.c b/io_uring/io-wq.c > index 1717f1465613..7326fef58ca7 100644 > --- a/io_uring/io-wq.c > +++ b/io_uring/io-wq.c > @@ -30,6 +30,7 @@ enum { > IO_WORKER_F_FREE = 4, /* worker on free list */ > IO_WORKER_F_BOUND = 8, /* is doing bounded work */ > IO_WORKER_F_EXIT = 16, /* worker is exiting */ > + IO_WORKER_F_FIXED = 32, /* is a fixed worker */ > }; > > enum { > @@ -598,6 +599,11 @@ static bool is_worker_exiting(struct io_worker *worker) > return worker->flags & IO_WORKER_F_EXIT; > } > > +static bool is_fixed_worker(struct io_worker *worker) > +{ > + return worker->flags & IO_WORKER_F_FIXED; > +} You move it up in Patch 5/11, I suggest to move it to the top of the file here. > static int io_wq_worker(void *data) > { > struct io_worker *worker = data; > @@ -622,8 +628,13 @@ static int io_wq_worker(void *data) > /* > * Last sleep timed out. Exit if we're not the last worker, > * or if someone modified our affinity. > + * Note: fixed worker always have same lifecycle as io-wq > + * itself, and cpu affinity setting doesn't work well for > + * fixed worker, they can be manually reset to cpu other than > + * the cpuset indicated by io_wq_worker_affinity() > */ > - if (last_timeout && (exit_mask || acct->nr_workers > 1)) { > + if (!is_fixed_worker(worker) && last_timeout && > + (exit_mask || acct->nr_workers > 1)) { > acct->nr_workers--; > raw_spin_unlock(&wq->lock); > __set_current_state(TASK_RUNNING); If there is no work it'll continue to loop every WORKER_IDLE_TIMEOUT (5 * HZ), which sounds troublesome with many workers in the system. tm = is_fixed_worker(worker) ? MAX_SCHEDULE_TIMEOUT : WORKER_IDLE_TIMEOUT; schedule_timeout(tm); Maybe?
diff --git a/io_uring/io-wq.c b/io_uring/io-wq.c index 1717f1465613..7326fef58ca7 100644 --- a/io_uring/io-wq.c +++ b/io_uring/io-wq.c @@ -30,6 +30,7 @@ enum { IO_WORKER_F_FREE = 4, /* worker on free list */ IO_WORKER_F_BOUND = 8, /* is doing bounded work */ IO_WORKER_F_EXIT = 16, /* worker is exiting */ + IO_WORKER_F_FIXED = 32, /* is a fixed worker */ }; enum { @@ -598,6 +599,11 @@ static bool is_worker_exiting(struct io_worker *worker) return worker->flags & IO_WORKER_F_EXIT; } +static bool is_fixed_worker(struct io_worker *worker) +{ + return worker->flags & IO_WORKER_F_FIXED; +} + static int io_wq_worker(void *data) { struct io_worker *worker = data; @@ -622,8 +628,13 @@ static int io_wq_worker(void *data) /* * Last sleep timed out. Exit if we're not the last worker, * or if someone modified our affinity. + * Note: fixed worker always have same lifecycle as io-wq + * itself, and cpu affinity setting doesn't work well for + * fixed worker, they can be manually reset to cpu other than + * the cpuset indicated by io_wq_worker_affinity() */ - if (last_timeout && (exit_mask || acct->nr_workers > 1)) { + if (!is_fixed_worker(worker) && last_timeout && + (exit_mask || acct->nr_workers > 1)) { acct->nr_workers--; raw_spin_unlock(&wq->lock); __set_current_state(TASK_RUNNING);