mbox series

[RFC,0/4] open/accept directly into io_uring fixed file table

Message ID cover.1625657451.git.asml.silence@gmail.com (mailing list archive)
Headers show
Series open/accept directly into io_uring fixed file table | expand

Message

Pavel Begunkov July 7, 2021, 11:39 a.m. UTC
Implement an old idea allowing open/accept io_uring requests to register
a newly created file as a io_uring's fixed file instead of placing it
into a task's file table. The switching is encoded in io_uring's SQEs
by setting sqe->buf_index/file_index, so restricted to 2^16-1. Don't
think we need more, but may be a good idea to scrap u32 somewhere
instead.

From the net side only needs a function doing __sys_accept4_file()
but not installing fd, see 2/4.

Only RFC for now, the new functionality is tested only for open yet.
I hope we can remember the author of the idea to add attribution.

Pavel Begunkov (4):
  io_uring: allow open directly into fixed fd table
  net: add an accept helper not installing fd
  io_uring: hand code io_accept()' fd installing
  io_uring: accept directly into fixed file table

 fs/io_uring.c                 | 113 +++++++++++++++++++++++++++++-----
 include/linux/socket.h        |   3 +
 include/uapi/linux/io_uring.h |   2 +
 net/socket.c                  |  71 +++++++++++----------
 4 files changed, 138 insertions(+), 51 deletions(-)

Comments

Jens Axboe July 7, 2021, 1:07 p.m. UTC | #1
On 7/7/21 5:39 AM, Pavel Begunkov wrote:
> Implement an old idea allowing open/accept io_uring requests to register
> a newly created file as a io_uring's fixed file instead of placing it
> into a task's file table. The switching is encoded in io_uring's SQEs
> by setting sqe->buf_index/file_index, so restricted to 2^16-1. Don't
> think we need more, but may be a good idea to scrap u32 somewhere
> instead.
> 
> From the net side only needs a function doing __sys_accept4_file()
> but not installing fd, see 2/4.
> 
> Only RFC for now, the new functionality is tested only for open yet.
> I hope we can remember the author of the idea to add attribution.

Pretty sure the original suggester of this as Josh, CC'ed.
Stefan Metzmacher July 7, 2021, 1:59 p.m. UTC | #2
Am 07.07.21 um 15:07 schrieb Jens Axboe:
> On 7/7/21 5:39 AM, Pavel Begunkov wrote:
>> Implement an old idea allowing open/accept io_uring requests to register
>> a newly created file as a io_uring's fixed file instead of placing it
>> into a task's file table. The switching is encoded in io_uring's SQEs
>> by setting sqe->buf_index/file_index, so restricted to 2^16-1. Don't
>> think we need more, but may be a good idea to scrap u32 somewhere
>> instead.
>>
>> From the net side only needs a function doing __sys_accept4_file()
>> but not installing fd, see 2/4.
>>
>> Only RFC for now, the new functionality is tested only for open yet.
>> I hope we can remember the author of the idea to add attribution.
> 
> Pretty sure the original suggester of this as Josh, CC'ed.

I also requested it for open :-)

metze
Jens Axboe July 7, 2021, 3:01 p.m. UTC | #3
On 7/7/21 7:59 AM, Stefan Metzmacher wrote:
> Am 07.07.21 um 15:07 schrieb Jens Axboe:
>> On 7/7/21 5:39 AM, Pavel Begunkov wrote:
>>> Implement an old idea allowing open/accept io_uring requests to register
>>> a newly created file as a io_uring's fixed file instead of placing it
>>> into a task's file table. The switching is encoded in io_uring's SQEs
>>> by setting sqe->buf_index/file_index, so restricted to 2^16-1. Don't
>>> think we need more, but may be a good idea to scrap u32 somewhere
>>> instead.
>>>
>>> From the net side only needs a function doing __sys_accept4_file()
>>> but not installing fd, see 2/4.
>>>
>>> Only RFC for now, the new functionality is tested only for open yet.
>>> I hope we can remember the author of the idea to add attribution.
>>
>> Pretty sure the original suggester of this as Josh, CC'ed.
> 
> I also requested it for open :-)

Indeed! I honestly forget the details, as some of it is implementation
detail. I think Josh was the first to suggest a private fd table could
be used, but that's mostly implementation detail as the point was to be
able to know which fd would be assigned.

But I think we're all in agreement here, it's a nifty feature :-)
Josh Triplett July 7, 2021, 3:04 p.m. UTC | #4
On Wed, Jul 07, 2021 at 07:07:52AM -0600, Jens Axboe wrote:
> On 7/7/21 5:39 AM, Pavel Begunkov wrote:
> > Implement an old idea allowing open/accept io_uring requests to register
> > a newly created file as a io_uring's fixed file instead of placing it
> > into a task's file table. The switching is encoded in io_uring's SQEs
> > by setting sqe->buf_index/file_index, so restricted to 2^16-1. Don't
> > think we need more, but may be a good idea to scrap u32 somewhere
> > instead.
> > 
> > From the net side only needs a function doing __sys_accept4_file()
> > but not installing fd, see 2/4.
> > 
> > Only RFC for now, the new functionality is tested only for open yet.
> > I hope we can remember the author of the idea to add attribution.
> 
> Pretty sure the original suggester of this as Josh, CC'ed.

Thanks for working on this, Pavel!

Original thread at
https://lore.kernel.org/io-uring/20200715004209.GA334456@localhost/T/ in
case that helps.
Hao Xu July 21, 2021, 7:11 a.m. UTC | #5
在 2021/7/7 下午7:39, Pavel Begunkov 写道:
> Implement an old idea allowing open/accept io_uring requests to register
> a newly created file as a io_uring's fixed file instead of placing it
> into a task's file table. The switching is encoded in io_uring's SQEs
> by setting sqe->buf_index/file_index, so restricted to 2^16-1. Don't
> think we need more, but may be a good idea to scrap u32 somewhere
> instead.
> 
>  From the net side only needs a function doing __sys_accept4_file()
> but not installing fd, see 2/4.
> 
> Only RFC for now, the new functionality is tested only for open yet.
> I hope we can remember the author of the idea to add attribution.
> 
Great feature! I believe this one leverages linked sqes, we may need to
remind users to be careful when they use this feature in shared sqthread
mode since linked sqes may be splited.
> Pavel Begunkov (4):
>    io_uring: allow open directly into fixed fd table
>    net: add an accept helper not installing fd
>    io_uring: hand code io_accept()' fd installing
>    io_uring: accept directly into fixed file table
> 
>   fs/io_uring.c                 | 113 +++++++++++++++++++++++++++++-----
>   include/linux/socket.h        |   3 +
>   include/uapi/linux/io_uring.h |   2 +
>   net/socket.c                  |  71 +++++++++++----------
>   4 files changed, 138 insertions(+), 51 deletions(-)
>
Pavel Begunkov Aug. 13, 2021, 4:30 p.m. UTC | #6
On 7/7/21 4:04 PM, Josh Triplett wrote:
> On Wed, Jul 07, 2021 at 07:07:52AM -0600, Jens Axboe wrote:
>> On 7/7/21 5:39 AM, Pavel Begunkov wrote:
>>> Implement an old idea allowing open/accept io_uring requests to register
>>> a newly created file as a io_uring's fixed file instead of placing it
>>> into a task's file table. The switching is encoded in io_uring's SQEs
>>> by setting sqe->buf_index/file_index, so restricted to 2^16-1. Don't
>>> think we need more, but may be a good idea to scrap u32 somewhere
>>> instead.
>>>
>>> From the net side only needs a function doing __sys_accept4_file()
>>> but not installing fd, see 2/4.
>>>
>>> Only RFC for now, the new functionality is tested only for open yet.
>>> I hope we can remember the author of the idea to add attribution.
>>
>> Pretty sure the original suggester of this as Josh, CC'ed.
> 
> Thanks for working on this, Pavel!
> 
> Original thread at

Totally was thinking it was only a discussion but not an actual patch,
and I even did comment on it! Sorry Josh, would have persuaded you to
finish it, if I remembered that.

> https://lore.kernel.org/io-uring/20200715004209.GA334456@localhost/T/ in
> case that helps.
>