| Message ID | 20210916234100.122368-1-logang@deltatee.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | Userspace P2PDMA with O_DIRECT NVMe devices | expand |
On Thu, Sep 16, 2021 at 05:40:40PM -0600, Logan Gunthorpe wrote: > Hi, > > This patchset continues my work to add userspace P2PDMA access using > O_DIRECT NVMe devices. My last posting[1] just included the first 13 > patches in this series, but the early P2PDMA cleanup and map_sg error > changes from that series have been merged into v5.15-rc1. To address > concerns that that series did not add any new functionality, I've added > back the userspcae functionality from the original RFC[2] (but improved > based on the original feedback). I really think this is the best series yet, it really looks nice overall. I know the sg flag was a bit of a debate at the start, but it serves an undeniable purpose and the resulting standard DMA APIs 'just working' is really clean. There is more possible here, we could also pass the new GUP flag in the ib_umem code.. After this gets merged I would make a series to split out the CMD genalloc related stuff and try and probably get something like VFIO to export this kind of memory as well, then it would have pretty nice coverage. Jason
On 2021-09-28 2:02 p.m., Jason Gunthorpe wrote: > On Thu, Sep 16, 2021 at 05:40:40PM -0600, Logan Gunthorpe wrote: >> Hi, >> >> This patchset continues my work to add userspace P2PDMA access using >> O_DIRECT NVMe devices. My last posting[1] just included the first 13 >> patches in this series, but the early P2PDMA cleanup and map_sg error >> changes from that series have been merged into v5.15-rc1. To address >> concerns that that series did not add any new functionality, I've added >> back the userspcae functionality from the original RFC[2] (but improved >> based on the original feedback). > > I really think this is the best series yet, it really looks nice > overall. I know the sg flag was a bit of a debate at the start, but it > serves an undeniable purpose and the resulting standard DMA APIs 'just > working' is really clean. Actually, so far, nobody has said anything negative about using the SG flag. > There is more possible here, we could also pass the new GUP flag in the > ib_umem code.. Yes, that would be very useful. > After this gets merged I would make a series to split out the CMD > genalloc related stuff and try and probably get something like VFIO to > export this kind of memory as well, then it would have pretty nice > coverage. Yup! Thanks for the review. Anything I didn't respond to I've either made changes for, or am still working on and will be addressed for subsequent postings. Logan
On Wed, Sep 29, 2021 at 03:50:02PM -0600, Logan Gunthorpe wrote: > > > On 2021-09-28 2:02 p.m., Jason Gunthorpe wrote: > > On Thu, Sep 16, 2021 at 05:40:40PM -0600, Logan Gunthorpe wrote: > >> Hi, > >> > >> This patchset continues my work to add userspace P2PDMA access using > >> O_DIRECT NVMe devices. My last posting[1] just included the first 13 > >> patches in this series, but the early P2PDMA cleanup and map_sg error > >> changes from that series have been merged into v5.15-rc1. To address > >> concerns that that series did not add any new functionality, I've added > >> back the userspcae functionality from the original RFC[2] (but improved > >> based on the original feedback). > > > > I really think this is the best series yet, it really looks nice > > overall. I know the sg flag was a bit of a debate at the start, but it > > serves an undeniable purpose and the resulting standard DMA APIs 'just > > working' is really clean. > > Actually, so far, nobody has said anything negative about using the SG flag. > > > There is more possible here, we could also pass the new GUP flag in the > > ib_umem code.. > > Yes, that would be very useful. You might actually prefer to do that then the bio changes to get the infrastructur merged as it seems less "core" Jason
On 2021-09-29 5:21 p.m., Jason Gunthorpe wrote: > On Wed, Sep 29, 2021 at 03:50:02PM -0600, Logan Gunthorpe wrote: >> >> >> On 2021-09-28 2:02 p.m., Jason Gunthorpe wrote: >>> On Thu, Sep 16, 2021 at 05:40:40PM -0600, Logan Gunthorpe wrote: >>>> Hi, >>>> >>>> This patchset continues my work to add userspace P2PDMA access using >>>> O_DIRECT NVMe devices. My last posting[1] just included the first 13 >>>> patches in this series, but the early P2PDMA cleanup and map_sg error >>>> changes from that series have been merged into v5.15-rc1. To address >>>> concerns that that series did not add any new functionality, I've added >>>> back the userspcae functionality from the original RFC[2] (but improved >>>> based on the original feedback). >>> >>> I really think this is the best series yet, it really looks nice >>> overall. I know the sg flag was a bit of a debate at the start, but it >>> serves an undeniable purpose and the resulting standard DMA APIs 'just >>> working' is really clean. >> >> Actually, so far, nobody has said anything negative about using the SG flag. >> >>> There is more possible here, we could also pass the new GUP flag in the >>> ib_umem code.. >> >> Yes, that would be very useful. > > You might actually prefer to do that then the bio changes to get the > infrastructur merged as it seems less "core" I'm a little bit more concerned about my patch set growing too large. It's already at 20 patches and I think I'll need to add a couple more based on the feedback you've already provided. So I'm leaning toward pushing more functionality as future work. Logan
On Wed, Sep 29, 2021 at 05:28:38PM -0600, Logan Gunthorpe wrote: > > > On 2021-09-29 5:21 p.m., Jason Gunthorpe wrote: > > On Wed, Sep 29, 2021 at 03:50:02PM -0600, Logan Gunthorpe wrote: > >> > >> > >> On 2021-09-28 2:02 p.m., Jason Gunthorpe wrote: > >>> On Thu, Sep 16, 2021 at 05:40:40PM -0600, Logan Gunthorpe wrote: > >>>> Hi, > >>>> > >>>> This patchset continues my work to add userspace P2PDMA access using > >>>> O_DIRECT NVMe devices. My last posting[1] just included the first 13 > >>>> patches in this series, but the early P2PDMA cleanup and map_sg error > >>>> changes from that series have been merged into v5.15-rc1. To address > >>>> concerns that that series did not add any new functionality, I've added > >>>> back the userspcae functionality from the original RFC[2] (but improved > >>>> based on the original feedback). > >>> > >>> I really think this is the best series yet, it really looks nice > >>> overall. I know the sg flag was a bit of a debate at the start, but it > >>> serves an undeniable purpose and the resulting standard DMA APIs 'just > >>> working' is really clean. > >> > >> Actually, so far, nobody has said anything negative about using the SG flag. > >> > >>> There is more possible here, we could also pass the new GUP flag in the > >>> ib_umem code.. > >> > >> Yes, that would be very useful. > > > > You might actually prefer to do that then the bio changes to get the > > infrastructur merged as it seems less "core" > > I'm a little bit more concerned about my patch set growing too large. > It's already at 20 patches and I think I'll need to add a couple more > based on the feedback you've already provided. So I'm leaning toward > pushing more functionality as future work. I mean you could postpone the three block related patches and use a single ib_umem patch instead as the consumer. Jason
On 2021-09-29 5:36 p.m., Jason Gunthorpe wrote: > On Wed, Sep 29, 2021 at 05:28:38PM -0600, Logan Gunthorpe wrote: >> >> >> On 2021-09-29 5:21 p.m., Jason Gunthorpe wrote: >>> On Wed, Sep 29, 2021 at 03:50:02PM -0600, Logan Gunthorpe wrote: >>>> >>>> >>>> On 2021-09-28 2:02 p.m., Jason Gunthorpe wrote: >>>>> On Thu, Sep 16, 2021 at 05:40:40PM -0600, Logan Gunthorpe wrote: >>>>>> Hi, >>>>>> >>>>>> This patchset continues my work to add userspace P2PDMA access using >>>>>> O_DIRECT NVMe devices. My last posting[1] just included the first 13 >>>>>> patches in this series, but the early P2PDMA cleanup and map_sg error >>>>>> changes from that series have been merged into v5.15-rc1. To address >>>>>> concerns that that series did not add any new functionality, I've added >>>>>> back the userspcae functionality from the original RFC[2] (but improved >>>>>> based on the original feedback). >>>>> >>>>> I really think this is the best series yet, it really looks nice >>>>> overall. I know the sg flag was a bit of a debate at the start, but it >>>>> serves an undeniable purpose and the resulting standard DMA APIs 'just >>>>> working' is really clean. >>>> >>>> Actually, so far, nobody has said anything negative about using the SG flag. >>>> >>>>> There is more possible here, we could also pass the new GUP flag in the >>>>> ib_umem code.. >>>> >>>> Yes, that would be very useful. >>> >>> You might actually prefer to do that then the bio changes to get the >>> infrastructur merged as it seems less "core" >> >> I'm a little bit more concerned about my patch set growing too large. >> It's already at 20 patches and I think I'll need to add a couple more >> based on the feedback you've already provided. So I'm leaning toward >> pushing more functionality as future work. > > I mean you could postpone the three block related patches and use a > single ib_umem patch instead as the consumer. I think that's not a very compelling use case given the only provider of these VMAs is an NVMe block device. My patch set enables a real world use (copying data between NVMe devices P2P through the CMB with O_DIRECT). Being able to read or write a CMB with RDMA and only RDMA is not very compelling. Logan