| Message ID | 20200116134946.184711-1-sgarzare@redhat.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | io_uring: wakeup threads waiting for EPOLLOUT events | expand |
On 1/16/20 6:49 AM, Stefano Garzarella wrote: > io_uring_poll() sets EPOLLOUT flag if there is space in the > SQ ring, then we should wakeup threads waiting for EPOLLOUT > events when we expose the new SQ head to the userspace. > > Signed-off-by: Stefano Garzarella <sgarzare@redhat.com> > --- > > Do you think is better to change the name of 'cq_wait' and 'cq_fasync'? I honestly think it'd be better to have separate waits for in/out poll, the below patch will introduce some unfortunate cacheline traffic between the submitter and completer side.
On Thu, Jan 16, 2020 at 08:29:07AM -0700, Jens Axboe wrote: > On 1/16/20 6:49 AM, Stefano Garzarella wrote: > > io_uring_poll() sets EPOLLOUT flag if there is space in the > > SQ ring, then we should wakeup threads waiting for EPOLLOUT > > events when we expose the new SQ head to the userspace. > > > > Signed-off-by: Stefano Garzarella <sgarzare@redhat.com> > > --- > > > > Do you think is better to change the name of 'cq_wait' and 'cq_fasync'? > > I honestly think it'd be better to have separate waits for in/out poll, > the below patch will introduce some unfortunate cacheline traffic > between the submitter and completer side. Agree, make sense. I'll send a v2 with a new 'sq_wait'. About fasync, do you think could be useful the POLL_OUT support? In this case, maybe is not simple to have two separate fasync_struct, do you have any advice? Thanks, Stefano
On 1/16/20 8:55 AM, Stefano Garzarella wrote: > On Thu, Jan 16, 2020 at 08:29:07AM -0700, Jens Axboe wrote: >> On 1/16/20 6:49 AM, Stefano Garzarella wrote: >>> io_uring_poll() sets EPOLLOUT flag if there is space in the >>> SQ ring, then we should wakeup threads waiting for EPOLLOUT >>> events when we expose the new SQ head to the userspace. >>> >>> Signed-off-by: Stefano Garzarella <sgarzare@redhat.com> >>> --- >>> >>> Do you think is better to change the name of 'cq_wait' and 'cq_fasync'? >> >> I honestly think it'd be better to have separate waits for in/out poll, >> the below patch will introduce some unfortunate cacheline traffic >> between the submitter and completer side. > > Agree, make sense. I'll send a v2 with a new 'sq_wait'. > > About fasync, do you think could be useful the POLL_OUT support? > In this case, maybe is not simple to have two separate fasync_struct, > do you have any advice? The fasync should not matter, it's all in the checking of whether the sq side has any sleepers. This is rarely going to be the case, so as long as we can keep the check cheap, then I think we're fine. Since the use case is mostly single submitter, unless you're doing something funky or unusual, you're not going to be needing POLLOUT ever. Hence I don't want to add any cost for it, I'd even advocate just doing waitqueue_active() perhaps, if we can safely pull it off.
On Thu, Jan 16, 2020 at 09:00:24AM -0700, Jens Axboe wrote: > On 1/16/20 8:55 AM, Stefano Garzarella wrote: > > On Thu, Jan 16, 2020 at 08:29:07AM -0700, Jens Axboe wrote: > >> On 1/16/20 6:49 AM, Stefano Garzarella wrote: > >>> io_uring_poll() sets EPOLLOUT flag if there is space in the > >>> SQ ring, then we should wakeup threads waiting for EPOLLOUT > >>> events when we expose the new SQ head to the userspace. > >>> > >>> Signed-off-by: Stefano Garzarella <sgarzare@redhat.com> > >>> --- > >>> > >>> Do you think is better to change the name of 'cq_wait' and 'cq_fasync'? > >> > >> I honestly think it'd be better to have separate waits for in/out poll, > >> the below patch will introduce some unfortunate cacheline traffic > >> between the submitter and completer side. > > > > Agree, make sense. I'll send a v2 with a new 'sq_wait'. > > > > About fasync, do you think could be useful the POLL_OUT support? > > In this case, maybe is not simple to have two separate fasync_struct, > > do you have any advice? > > The fasync should not matter, it's all in the checking of whether the sq > side has any sleepers. This is rarely going to be the case, so as long > as we can keep the check cheap, then I think we're fine. Right. > > Since the use case is mostly single submitter, unless you're doing > something funky or unusual, you're not going to be needing POLLOUT ever. The case that I had in mind was with kernel side polling enabled and a single submitter that can use epoll() to wait free slots in the SQ ring. (I don't have a test, maybe I can write one...) > Hence I don't want to add any cost for it, I'd even advocate just doing > waitqueue_active() perhaps, if we can safely pull it off. I'll try! Thanks, Stefano
On 1/16/20 9:26 AM, Stefano Garzarella wrote: >> Since the use case is mostly single submitter, unless you're doing >> something funky or unusual, you're not going to be needing POLLOUT ever. > > The case that I had in mind was with kernel side polling enabled and > a single submitter that can use epoll() to wait free slots in the SQ > ring. (I don't have a test, maybe I can write one...) Right, I think that's the only use case where it makes sense, because you have someone else draining the sq side for you. A test case would indeed be nice, liburing has a good arsenal of test cases and this would be a good addition!
On Thu, Jan 16, 2020 at 09:30:12AM -0700, Jens Axboe wrote: > On 1/16/20 9:26 AM, Stefano Garzarella wrote: > >> Since the use case is mostly single submitter, unless you're doing > >> something funky or unusual, you're not going to be needing POLLOUT ever. > > > > The case that I had in mind was with kernel side polling enabled and > > a single submitter that can use epoll() to wait free slots in the SQ > > ring. (I don't have a test, maybe I can write one...) > > Right, I think that's the only use case where it makes sense, because > you have someone else draining the sq side for you. A test case would > indeed be nice, liburing has a good arsenal of test cases and this would > be a good addition! Sure, I'll send a test to liburing for this case! Thanks, Stefano
On 1/16/20 10:03 AM, Stefano Garzarella wrote: > On Thu, Jan 16, 2020 at 09:30:12AM -0700, Jens Axboe wrote: >> On 1/16/20 9:26 AM, Stefano Garzarella wrote: >>>> Since the use case is mostly single submitter, unless you're doing >>>> something funky or unusual, you're not going to be needing POLLOUT ever. >>> >>> The case that I had in mind was with kernel side polling enabled and >>> a single submitter that can use epoll() to wait free slots in the SQ >>> ring. (I don't have a test, maybe I can write one...) >> >> Right, I think that's the only use case where it makes sense, because >> you have someone else draining the sq side for you. A test case would >> indeed be nice, liburing has a good arsenal of test cases and this would >> be a good addition! > > Sure, I'll send a test to liburing for this case! Gentle ping on the test case :-)
On Thu, Jan 23, 2020 at 12:13:57PM -0700, Jens Axboe wrote: > On 1/16/20 10:03 AM, Stefano Garzarella wrote: > > On Thu, Jan 16, 2020 at 09:30:12AM -0700, Jens Axboe wrote: > >> On 1/16/20 9:26 AM, Stefano Garzarella wrote: > >>>> Since the use case is mostly single submitter, unless you're doing > >>>> something funky or unusual, you're not going to be needing POLLOUT ever. > >>> > >>> The case that I had in mind was with kernel side polling enabled and > >>> a single submitter that can use epoll() to wait free slots in the SQ > >>> ring. (I don't have a test, maybe I can write one...) > >> > >> Right, I think that's the only use case where it makes sense, because > >> you have someone else draining the sq side for you. A test case would > >> indeed be nice, liburing has a good arsenal of test cases and this would > >> be a good addition! > > > > Sure, I'll send a test to liburing for this case! > > Gentle ping on the test case :-) > Yes, you are right :-) I was a little busy this week to finish some works before DevConf. I hope to work on the test case these days, so by Monday I hope I have it ;-) Cheers, Stefano
On 1/23/20 2:45 PM, Stefano Garzarella wrote: > On Thu, Jan 23, 2020 at 12:13:57PM -0700, Jens Axboe wrote: >> On 1/16/20 10:03 AM, Stefano Garzarella wrote: >>> On Thu, Jan 16, 2020 at 09:30:12AM -0700, Jens Axboe wrote: >>>> On 1/16/20 9:26 AM, Stefano Garzarella wrote: >>>>>> Since the use case is mostly single submitter, unless you're doing >>>>>> something funky or unusual, you're not going to be needing POLLOUT ever. >>>>> >>>>> The case that I had in mind was with kernel side polling enabled and >>>>> a single submitter that can use epoll() to wait free slots in the SQ >>>>> ring. (I don't have a test, maybe I can write one...) >>>> >>>> Right, I think that's the only use case where it makes sense, because >>>> you have someone else draining the sq side for you. A test case would >>>> indeed be nice, liburing has a good arsenal of test cases and this would >>>> be a good addition! >>> >>> Sure, I'll send a test to liburing for this case! >> >> Gentle ping on the test case :-) >> > > Yes, you are right :-) > > I was a little busy this week to finish some works before DevConf. I > hope to work on the test case these days, so by Monday I hope I have > it ;-) Thanks, all good, just a gentle nudge ;-)
diff --git a/fs/io_uring.c b/fs/io_uring.c index 38b54051facd..5c6ff5f9e741 100644 --- a/fs/io_uring.c +++ b/fs/io_uring.c @@ -3687,6 +3687,11 @@ static void io_commit_sqring(struct io_ring_ctx *ctx) * write new data to them. */ smp_store_release(&rings->sq.head, ctx->cached_sq_head); + + if (wq_has_sleeper(&ctx->cq_wait)) { + wake_up_interruptible(&ctx->cq_wait); + kill_fasync(&ctx->cq_fasync, SIGIO, POLL_OUT); + } } }
io_uring_poll() sets EPOLLOUT flag if there is space in the SQ ring, then we should wakeup threads waiting for EPOLLOUT events when we expose the new SQ head to the userspace. Signed-off-by: Stefano Garzarella <sgarzare@redhat.com> --- Do you think is better to change the name of 'cq_wait' and 'cq_fasync'? Thanks, Stefano --- fs/io_uring.c | 5 +++++ 1 file changed, 5 insertions(+)