mbox series

[0/3] uio/dma-buf: Give UIO users access to DMA addresses.

Message ID 20250410-uio-dma-v1-0-6468ace2c786@bootlin.com (mailing list archive)
Headers show
Series uio/dma-buf: Give UIO users access to DMA addresses. | expand

Message

Bastien Curutchet April 10, 2025, 2:53 p.m. UTC
Hi all,

Many UIO users performing DMA from their UIO device need to access the
DMA addresses of the allocated buffers. There are out-of-tree drivers
that allow to do it but nothing in the mainline.

I know DMA shouldn't be handled by userspace but, IMHO, since UIO
drivers exist, it would be better if they offered a way of doing this.

This patch series use the dma-heap framework which already allows
userspace to allocate DMA buffers. I tried to avoid 'polluting' the
existing heaps to prevent inappropriate uses of this new feature by
introducing a new UIO heap, which is the only one implementing this
behavior.

PATCH 1 allows the creation of heaps that don't implement map/unmap_buf
operations as UIO heap doesn't use them.
PATCH 2 adds the DMA_BUF_IOCTL_GET_DMA_ADDR which transmits the DMA
addresses to userspace.
PATCH 3 implements the UIO heap.

It has been tested with the uio_pci_generic driver on a PowerPC.

Signed-off-by: Bastien Curutchet <bastien.curutchet@bootlin.com>
---
Bastien Curutchet (3):
      dma-buf: Allow heap that doesn't provide map_buf/unmap_buf
      dma-buf: Add DMA_BUF_IOCTL_GET_DMA_ADDR
      uio: Add UIO_DMABUF_HEAP

 drivers/dma-buf/dma-buf.c    |  29 +++++++++--
 drivers/uio/Kconfig          |   9 ++++
 drivers/uio/Makefile         |   1 +
 drivers/uio/uio.c            |   4 ++
 drivers/uio/uio_heap.c       | 120 +++++++++++++++++++++++++++++++++++++++++++
 include/linux/dma-buf.h      |   1 +
 include/linux/uio_driver.h   |   2 +
 include/uapi/linux/dma-buf.h |   1 +
 8 files changed, 164 insertions(+), 3 deletions(-)
---
base-commit: 5f13fa25acaa4f586aaed12efcf7436e004eeaf2
change-id: 20250408-uio-dma-9b011e9e7f0b

Best regards,

Comments

Christian König April 10, 2025, 4:29 p.m. UTC | #1
Am 10.04.25 um 16:53 schrieb Bastien Curutchet:
> Hi all,
>
> Many UIO users performing DMA from their UIO device need to access the
> DMA addresses of the allocated buffers. There are out-of-tree drivers
> that allow to do it but nothing in the mainline.

Well that basically disqualifies this patch set in the first paragraph.

To justify some kernel change we always need an in kernel user of the interface, since this is purely for out-of-tree drivers this is a no-go to begin with.

> I know DMA shouldn't be handled by userspace but, IMHO, since UIO
> drivers exist, it would be better if they offered a way of doing this.

Leaking DMA addresses to userspace is usually seen as quite some security hole, but on the other hand with UIO you don't have much other choice.

> This patch series use the dma-heap framework which already allows
> userspace to allocate DMA buffers. I tried to avoid 'polluting' the
> existing heaps to prevent inappropriate uses of this new feature by
> introducing a new UIO heap, which is the only one implementing this
> behavior.

Yeah, that won't fly at all.

What you could potentially do is to create an UIO driver which imports DMA-bufs, pins them and then provide the DMA addresses to userspace.

But please be aware that DMA-fences are fundamentally incompatible with UIO. So you won't be able to do any form of synchronization which probably makes the implementation pretty limited.

Regards,
Christian.

>
> PATCH 1 allows the creation of heaps that don't implement map/unmap_buf
> operations as UIO heap doesn't use them.
> PATCH 2 adds the DMA_BUF_IOCTL_GET_DMA_ADDR which transmits the DMA
> addresses to userspace.
> PATCH 3 implements the UIO heap.
>
> It has been tested with the uio_pci_generic driver on a PowerPC.
>
> Signed-off-by: Bastien Curutchet <bastien.curutchet@bootlin.com>
> ---
> Bastien Curutchet (3):
>       dma-buf: Allow heap that doesn't provide map_buf/unmap_buf
>       dma-buf: Add DMA_BUF_IOCTL_GET_DMA_ADDR
>       uio: Add UIO_DMABUF_HEAP
>
>  drivers/dma-buf/dma-buf.c    |  29 +++++++++--
>  drivers/uio/Kconfig          |   9 ++++
>  drivers/uio/Makefile         |   1 +
>  drivers/uio/uio.c            |   4 ++
>  drivers/uio/uio_heap.c       | 120 +++++++++++++++++++++++++++++++++++++++++++
>  include/linux/dma-buf.h      |   1 +
>  include/linux/uio_driver.h   |   2 +
>  include/uapi/linux/dma-buf.h |   1 +
>  8 files changed, 164 insertions(+), 3 deletions(-)
> ---
> base-commit: 5f13fa25acaa4f586aaed12efcf7436e004eeaf2
> change-id: 20250408-uio-dma-9b011e9e7f0b
>
> Best regards,
Thomas Petazzoni April 10, 2025, 7:43 p.m. UTC | #2
Hello Christian,

Thanks for your feedback!

On Thu, 10 Apr 2025 18:29:12 +0200
Christian König <christian.koenig@amd.com> wrote:

> > Many UIO users performing DMA from their UIO device need to access the
> > DMA addresses of the allocated buffers. There are out-of-tree drivers
> > that allow to do it but nothing in the mainline.  
> 
> Well that basically disqualifies this patch set in the first paragraph.
> 
> To justify some kernel change we always need an in kernel user of the
> interface, since this is purely for out-of-tree drivers this is a
> no-go to begin with.

I'm not sure to understand your comment here. This patch series is
about extending the UIO interface... which is a user-space interface.
So obviously it has no "in-kernel user" because it's meant to be used
from user-space. Could you clarify what you meant here?


> What you could potentially do is to create an UIO driver which
> imports DMA-bufs, pins them and then provide the DMA addresses to
> userspace.
> 
> But please be aware that DMA-fences are fundamentally incompatible
> with UIO. So you won't be able to do any form of synchronization
> which probably makes the implementation pretty limited.

Could you clarify why DMA-fences would be needed here, and for what
synchronization?

The DMA buffers allocated here are DMA coherent buffers. So the
user-space application that uses UIO will allocate such buffers once at
application start, retrieve their DMA address, and then program DMA
transfers as it sees fit using the memory-mapped registers accessible
through UIO. I'm not sure which synchronization you are referring to.
We are not "chaining" DMA transfers, with for example a camera
interface feeding its DMA buffers to an ISP or something like that. The
typical use case here is some IP block in an FPGA that does DMA
transfers to transfer data to/from some sort of specialized I/O
interface. We get an interrupt when the transfer is done, and we can
re-use the buffer for the next transfer.

If you could clarify here as well, it would definitely help in
understanding the shortcomings/limitations.

Thanks a lot!

Thomas Petazzoni
Bastien Curutchet April 11, 2025, 8:14 a.m. UTC | #3
Hi Christian,

On 4/11/25 9:30 AM, Christian König wrote:
> Hi Thomas,
> 
> Am 10.04.25 um 21:43 schrieb Thomas Petazzoni:
>> Hello Christian,
>>
>> Thanks for your feedback!
>>
>> On Thu, 10 Apr 2025 18:29:12 +0200
>> Christian König<christian.koenig@amd.com> wrote:
>>
>>>> Many UIO users performing DMA from their UIO device need to access the
>>>> DMA addresses of the allocated buffers. There are out-of-tree drivers
>>>> that allow to do it but nothing in the mainline.
>>> Well that basically disqualifies this patch set in the first paragraph.
>>>
>>> To justify some kernel change we always need an in kernel user of the
>>> interface, since this is purely for out-of-tree drivers this is a
>>> no-go to begin with.
>> I'm not sure to understand your comment here. This patch series is
>> about extending the UIO interface... which is a user-space interface.
>> So obviously it has no "in-kernel user" because it's meant to be used
>> from user-space. Could you clarify what you meant here?
> 
> Bastien wrote about "out-of-tree drivers" which is something the 
> upstream kernel explicitly does not support.
> 

Sorry maybe it wasn't clear, but what I meant is that the goal of this 
series is to replace 'out-of-tree drivers' with something upstream.

> When you make that UIO API and have an open source userspace driver then 
> that is probably a good justification to do this.
> 
> What the kernel community tries to prevent here is that people start 
> using the UAPI to write closed source drivers in userspace.
> 

>>> What you could potentially do is to create an UIO driver which
>>> imports DMA-bufs, pins them and then provide the DMA addresses to
>>> userspace.
>>>
>>> But please be aware that DMA-fences are fundamentally incompatible
>>> with UIO. So you won't be able to do any form of synchronization
>>> which probably makes the implementation pretty limited.
>> Could you clarify why DMA-fences would be needed here, and for what
>> synchronization?
> 
> In general DMA-buf is an interface which enables you do share device 
> specific buffers between different drivers as well as between userspace 
> processes.
> 
> For this to work with for example cameras, GPUs or RDMA NICs you need to 
> some kind of synchronization primitive, e.g. device A can only starts 
> it's DMA transaction when device B has completed his.
> 
> The problem is that this synchronization approach is fundamentally 
> incompatible with UIO. See here for more details: https:// 
> www.kernel.org/doc/html/latest/driver-api/dma-buf.html#indefinite-dma-fences
>
>> The DMA buffers allocated here are DMA coherent buffers. So the
>> user-space application that uses UIO will allocate such buffers once at
>> application start, retrieve their DMA address, and then program DMA
>> transfers as it sees fit using the memory-mapped registers accessible
>> through UIO. I'm not sure which synchronization you are referring to.
>> We are not "chaining" DMA transfers, with for example a camera
>> interface feeding its DMA buffers to an ISP or something like that. The
>> typical use case here is some IP block in an FPGA that does DMA
>> transfers to transfer data to/from some sort of specialized I/O
>> interface. We get an interrupt when the transfer is done, and we can
>> re-use the buffer for the next transfer.
> 
> Well why do you then want to use DMA-buf in the first place? As far as I 
> know what you describe can perfectly be done with the normal UIO memory 
> management interfaces.
> 
> DMA-buf is only interesting when you actually want to share something in 
> between devices or between applications.
>

I wanted to use DMA-buf because it allows to dynamically 
allocate/release coherent buffers from userspace. UIO doesn't provide 
such interface.
I'm aware that exposing DMA addresses to userspace isn't a good 
practice. That's why this series create a new heap specific to UIO that 
will be the only one implementing the new ioctl.


Best regards,
Bastien
Christian König April 11, 2025, 12:41 p.m. UTC | #4
Hi Bastien,

Am 11.04.25 um 10:14 schrieb Bastien Curutchet:
> Hi Christian,
>
> On 4/11/25 9:30 AM, Christian König wrote:
>> Hi Thomas,
>>
>> Am 10.04.25 um 21:43 schrieb Thomas Petazzoni:
>>> Hello Christian,
>>>
>>> Thanks for your feedback!
>>>
>>> On Thu, 10 Apr 2025 18:29:12 +0200
>>> Christian König<christian.koenig@amd.com> wrote:
>>>
>>>>> Many UIO users performing DMA from their UIO device need to access the
>>>>> DMA addresses of the allocated buffers. There are out-of-tree drivers
>>>>> that allow to do it but nothing in the mainline.
>>>> Well that basically disqualifies this patch set in the first paragraph.
>>>>
>>>> To justify some kernel change we always need an in kernel user of the
>>>> interface, since this is purely for out-of-tree drivers this is a
>>>> no-go to begin with.
>>> I'm not sure to understand your comment here. This patch series is
>>> about extending the UIO interface... which is a user-space interface.
>>> So obviously it has no "in-kernel user" because it's meant to be used
>>> from user-space. Could you clarify what you meant here?
>>
>> Bastien wrote about "out-of-tree drivers" which is something the upstream kernel explicitly does not support.
>>
>
> Sorry maybe it wasn't clear, but what I meant is that the goal of this series is to replace 'out-of-tree drivers' with something upstream.

Ah! Yeah that wasn't really clear from the description.

But anyway please note that when you want to create new UAPI you need to provide an open source user of it. E.g. link to a repository or something similar in the covert letter should do it.

>> Well why do you then want to use DMA-buf in the first place? As far as I know what you describe can perfectly be done with the normal UIO memory management interfaces.
>>
>> DMA-buf is only interesting when you actually want to share something in between devices or between applications.
>>
>
> I wanted to use DMA-buf because it allows to dynamically allocate/release coherent buffers from userspace. UIO doesn't provide such interface.
> I'm aware that exposing DMA addresses to userspace isn't a good practice. That's why this series create a new heap specific to UIO that will be the only one implementing the new ioctl.

I don't know the UIO interfaces that well, but that is pretty clearly an abuse of DMA-buf and won't fly at all.

If you want coherent memory for your device you should use dma_alloc_coherent() for that.

Regards,
Christian.

>
>
> Best regards,
> Bastien
>
>
>
Christoph Hellwig April 14, 2025, 5:55 a.m. UTC | #5
On Thu, Apr 10, 2025 at 04:53:17PM +0200, Bastien Curutchet wrote:
> Hi all,
> 
> Many UIO users performing DMA from their UIO device need to access the
> DMA addresses of the allocated buffers. There are out-of-tree drivers
> that allow to do it but nothing in the mainline.

In which case it does not matter at all for mainline.

FYI the proper and safe way to do DMA from userspace is to use
vfio/iommufd.
Thomas Petazzoni April 14, 2025, 8:17 a.m. UTC | #6
Hello Christian,

On Fri, 11 Apr 2025 14:41:56 +0200
Christian König <christian.koenig@amd.com> wrote:

> But anyway please note that when you want to create new UAPI you need
> to provide an open source user of it. E.g. link to a repository or
> something similar in the covert letter should do it.

Could you clarify what is the "reference" user-space user of UIO that
exists today?

Thomas
Thomas Petazzoni April 14, 2025, 8:24 a.m. UTC | #7
Hello Christoph,

On Sun, 13 Apr 2025 22:55:02 -0700
Christoph Hellwig <hch@infradead.org> wrote:

> On Thu, Apr 10, 2025 at 04:53:17PM +0200, Bastien Curutchet wrote:
> > Hi all,
> > 
> > Many UIO users performing DMA from their UIO device need to access the
> > DMA addresses of the allocated buffers. There are out-of-tree drivers
> > that allow to do it but nothing in the mainline.  
> 
> In which case it does not matter at all for mainline.

It is impressive how "out-of-tree" triggers kernel maintainers, and
then end up not even reading what the patch series is all about (but I
forgive you, because it triggers me in the same way).

This patch series is NOT about adding new kernel APIs meant to be used
by out-of-tree drivers, which of course would be bad, and we would have
never proposed.

What this patch series is about is to add new user-space interface to
extend the existing UIO subsystem. What my colleague Bastien was
mentioning about out-of-tree drivers is that there are a LOT of
out-of-tree drivers extending UIO to allow user-space applications to
do DMA, and our proposal is that considering how many people need that
and implement ugly out-of-tree drivers to solve this issue, we suggest
the mainline kernel should have a built in solution.

Please re-read again, and realize that we are NOT adding new kernel
APIs for out-of-tree drivers.

> FYI the proper and safe way to do DMA from userspace is to use
> vfio/iommufd.

I am not sure how this can work in our use-case. We have a very simple
set of IP blocks implemented in a FPGA, some of those IP blocks are
able to perform DMA operations. The register of those IP blocks are
mapped into a user-space application using the existing, accepted
upstream, UIO subsystem. Some of those registers allow to program DMA
transfers. So far, we can do all what we need, except program those DMA
transfers. Lots of people are having the same issue, and zillions of
ugly out-of-tree solutions flourish all over, and we're trying to see
if we can constructively find a solution that would be acceptable
upstream to resolve this use-case. Our platform is an old PowerPC with
no IOMMU.

Note that the safety argument doesn't hold: UIO already allows to map
registers into user-space applications, which can already program DMA
transfers to arbitrary locations in memory. The safety comes from the
fact that such UIO devices are associated with permissions that the
system administrator controls, to decide which applications are safe
enough to be granted access to those UIO devices. Therefore, providing
access through UIO of the physical address of well-defined DMA buffers
properly allocated through a sane API is not reducing the safety of
anything compared to what UIO already allows today.

Best regards,

Thomas
Christian König April 14, 2025, 8:59 a.m. UTC | #8
Am 14.04.25 um 10:17 schrieb Thomas Petazzoni:
> Hello Christian,
>
> On Fri, 11 Apr 2025 14:41:56 +0200
> Christian König <christian.koenig@amd.com> wrote:
>
>> But anyway please note that when you want to create new UAPI you need
>> to provide an open source user of it. E.g. link to a repository or
>> something similar in the covert letter should do it.
> Could you clarify what is the "reference" user-space user of UIO that
> exists today?

I honestly don't know. I have never looked into UIO before this patch set arrived.

What I mean is that the general rule to justify adding UAPI to the Linux kernel is that you need to have an open source user of that UAPI.

In other words for UIO this means that you *can't* do things like exposing some kernel functionality like DMA-buf through UIO and then write a binary only driver on top of it.

Regards,
Christian.

>
> Thomas
Christian König April 14, 2025, 9:11 a.m. UTC | #9
Am 14.04.25 um 10:24 schrieb Thomas Petazzoni:
> Hello Christoph,
>
> On Sun, 13 Apr 2025 22:55:02 -0700
> Christoph Hellwig <hch@infradead.org> wrote:
>
>> On Thu, Apr 10, 2025 at 04:53:17PM +0200, Bastien Curutchet wrote:
>>> Hi all,
>>>
>>> Many UIO users performing DMA from their UIO device need to access the
>>> DMA addresses of the allocated buffers. There are out-of-tree drivers
>>> that allow to do it but nothing in the mainline.  
>> In which case it does not matter at all for mainline.
> It is impressive how "out-of-tree" triggers kernel maintainers, and
> then end up not even reading what the patch series is all about (but I
> forgive you, because it triggers me in the same way).
>
> This patch series is NOT about adding new kernel APIs meant to be used
> by out-of-tree drivers, which of course would be bad, and we would have
> never proposed.
>
> What this patch series is about is to add new user-space interface to
> extend the existing UIO subsystem. What my colleague Bastien was
> mentioning about out-of-tree drivers is that there are a LOT of
> out-of-tree drivers extending UIO to allow user-space applications to
> do DMA, and our proposal is that considering how many people need that
> and implement ugly out-of-tree drivers to solve this issue, we suggest
> the mainline kernel should have a built in solution.
>
> Please re-read again, and realize that we are NOT adding new kernel
> APIs for out-of-tree drivers.
>
>> FYI the proper and safe way to do DMA from userspace is to use
>> vfio/iommufd.
> I am not sure how this can work in our use-case. We have a very simple
> set of IP blocks implemented in a FPGA, some of those IP blocks are
> able to perform DMA operations. The register of those IP blocks are
> mapped into a user-space application using the existing, accepted
> upstream, UIO subsystem. Some of those registers allow to program DMA
> transfers. So far, we can do all what we need, except program those DMA
> transfers. Lots of people are having the same issue, and zillions of
> ugly out-of-tree solutions flourish all over, and we're trying to see
> if we can constructively find a solution that would be acceptable
> upstream to resolve this use-case. Our platform is an old PowerPC with
> no IOMMU.

Maybe I should try to better explain the concern here. The question is "Where is the source code of your FPGA driver?".

I mean that you are trying to replace the out-of-tree solution is rather welcomed, but the out-of-tree solutions are out-of-tree because they don't fit with requirements to be added to the core Linux tree.

And one of those requirements is that you need to provide the source code of the userspace user of this interface, in this case here that is your FPGA driver. An MIT/X11 license is usually sufficient, GPL is of course seen as better and it must not be a toy application, but rather the real thing.

And that is what people usually don't want and that's why no in-tree solution exists for this.

Regards,
Christian.

>
> Note that the safety argument doesn't hold: UIO already allows to map
> registers into user-space applications, which can already program DMA
> transfers to arbitrary locations in memory. The safety comes from the
> fact that such UIO devices are associated with permissions that the
> system administrator controls, to decide which applications are safe
> enough to be granted access to those UIO devices. Therefore, providing
> access through UIO of the physical address of well-defined DMA buffers
> properly allocated through a sane API is not reducing the safety of
> anything compared to what UIO already allows today.
>
> Best regards,
>
> Thomas
Christoph Hellwig April 14, 2025, 11:24 a.m. UTC | #10
On Mon, Apr 14, 2025 at 10:24:55AM +0200, Thomas Petazzoni wrote:
> What this patch series is about is to add new user-space interface to
> extend the existing UIO subsystem.

Which as I explained to you is fundamentally broken and unsafe.  If you
need to do DMA from userspae you need to use vfio/iommufd.

> I am not sure how this can work in our use-case. We have a very simple
> set of IP blocks implemented in a FPGA, some of those IP blocks are
> able to perform DMA operations. The register of those IP blocks are
> mapped into a user-space application using the existing, accepted
> upstream, UIO subsystem. Some of those registers allow to program DMA
> transfers. So far, we can do all what we need, except program those DMA
> transfers. Lots of people are having the same issue, and zillions of
> ugly out-of-tree solutions flourish all over, and we're trying to see
> if we can constructively find a solution that would be acceptable
> upstream to resolve this use-case. Our platform is an old PowerPC with
> no IOMMU.

Then your driver design can't work and you need to replace it with a
proper in-kernel driver.
Thomas Petazzoni April 14, 2025, 11:48 a.m. UTC | #11
Hello Christoph,

On Mon, 14 Apr 2025 04:24:21 -0700
Christoph Hellwig <hch@infradead.org> wrote:

> On Mon, Apr 14, 2025 at 10:24:55AM +0200, Thomas Petazzoni wrote:
> > What this patch series is about is to add new user-space interface to
> > extend the existing UIO subsystem.  
> 
> Which as I explained to you is fundamentally broken and unsafe.  If you
> need to do DMA from userspae you need to use vfio/iommufd.

I'm still unclear as to why it is more "broken and unsafe" than UIO
already is. As I already replied in this thread: UIO allows to remap
MMIO registers into a user-space application, which can then do
whatever it wants with the IP block behind those MMIO registers. If
this IP block supports DMA, it already means that _today_ with the
current UIO subsystem as it is, the user-space application can program
a DMA transfer to read/write to any location in memory.

Therefore, providing a way to cleanly allocate DMA buffers and get
their physical address will not make things any better or worse in
terms of safety.

The fact that it is reasonably safe is solely based on access control
to the UIO device, done using usual Unix permissions, and that is
already the case today.

> > I am not sure how this can work in our use-case. We have a very simple
> > set of IP blocks implemented in a FPGA, some of those IP blocks are
> > able to perform DMA operations. The register of those IP blocks are
> > mapped into a user-space application using the existing, accepted
> > upstream, UIO subsystem. Some of those registers allow to program DMA
> > transfers. So far, we can do all what we need, except program those DMA
> > transfers. Lots of people are having the same issue, and zillions of
> > ugly out-of-tree solutions flourish all over, and we're trying to see
> > if we can constructively find a solution that would be acceptable
> > upstream to resolve this use-case. Our platform is an old PowerPC with
> > no IOMMU.  
> 
> Then your driver design can't work and you need to replace it with a
> proper in-kernel driver.

See above: your point is moot because providing capabilities to
allocate a buffer and get its physical address so that a UIO-based
user-space application can do DMA transfer does not make things any
more unsafe than they already are.

Best regards,

Thomas
Thomas Petazzoni April 14, 2025, 11:52 a.m. UTC | #12
Hello Christian,

On Mon, 14 Apr 2025 11:11:48 +0200
Christian König <christian.koenig@amd.com> wrote:

> Maybe I should try to better explain the concern here. The question
> is "Where is the source code of your FPGA driver?".
> 
> I mean that you are trying to replace the out-of-tree solution is
> rather welcomed, but the out-of-tree solutions are out-of-tree
> because they don't fit with requirements to be added to the core
> Linux tree.
> 
> And one of those requirements is that you need to provide the source
> code of the userspace user of this interface, in this case here that
> is your FPGA driver. An MIT/X11 license is usually sufficient, GPL is
> of course seen as better and it must not be a toy application, but
> rather the real thing.

Where is this requirement for UIO? The UIO subsystem does not have such
a requirement, unlike indeed some other kernel subsystems such as DRM.

But the practical situation is different: for DRM it makes a lot of
sense to enforce having open-source code in user-space, as we want to
force GPU vendors to open their OpenGL/Vulkan implementations. However,
for UIO it makes little sense, because UIO is typically used to control
some super-specific FPGA IP blocks that are totally irrelevant outside
of the very specific product they are included in. Most likely if those
drivers were open-sourced and tried to be upstream they would be
rejected because their usefulness in the upstream kernel is basically
zero (but they would have an on-going maintenance effort for the whole
community).

> And that is what people usually don't want and that's why no in-tree
> solution exists for this.

And that doesn't make sense because UIO already exists, and allows to
achieve 95% of what people already need, to the exception of this DMA
issue.

Thomas
Andrew Davis April 14, 2025, 5:08 p.m. UTC | #13
On 4/14/25 6:48 AM, Thomas Petazzoni wrote:
> Hello Christoph,
> 
> On Mon, 14 Apr 2025 04:24:21 -0700
> Christoph Hellwig <hch@infradead.org> wrote:
> 
>> On Mon, Apr 14, 2025 at 10:24:55AM +0200, Thomas Petazzoni wrote:
>>> What this patch series is about is to add new user-space interface to
>>> extend the existing UIO subsystem.
>>
>> Which as I explained to you is fundamentally broken and unsafe.  If you
>> need to do DMA from userspae you need to use vfio/iommufd.
> 
> I'm still unclear as to why it is more "broken and unsafe" than UIO
> already is. As I already replied in this thread: UIO allows to remap
> MMIO registers into a user-space application, which can then do
> whatever it wants with the IP block behind those MMIO registers. If
> this IP block supports DMA, it already means that _today_ with the
> current UIO subsystem as it is, the user-space application can program
> a DMA transfer to read/write to any location in memory.
> 
> Therefore, providing a way to cleanly allocate DMA buffers and get
> their physical address will not make things any better or worse in
> terms of safety.
> 
> The fact that it is reasonably safe is solely based on access control
> to the UIO device, done using usual Unix permissions, and that is
> already the case today.
> 
>>> I am not sure how this can work in our use-case. We have a very simple
>>> set of IP blocks implemented in a FPGA, some of those IP blocks are
>>> able to perform DMA operations. The register of those IP blocks are
>>> mapped into a user-space application using the existing, accepted
>>> upstream, UIO subsystem. Some of those registers allow to program DMA
>>> transfers. So far, we can do all what we need, except program those DMA
>>> transfers. Lots of people are having the same issue, and zillions of
>>> ugly out-of-tree solutions flourish all over, and we're trying to see
>>> if we can constructively find a solution that would be acceptable
>>> upstream to resolve this use-case. Our platform is an old PowerPC with
>>> no IOMMU.
>>
>> Then your driver design can't work and you need to replace it with a
>> proper in-kernel driver.
> 
> See above: your point is moot because providing capabilities to
> allocate a buffer and get its physical address so that a UIO-based
> user-space application can do DMA transfer does not make things any
> more unsafe than they already are.
> 

"UIO is a broken legacy mess, so let's add more broken things
to it as broken + broken => still broken, so no harm done", am I
getting that right?

If your FPGA IP can do DMA then you should not be using UIO in
the first place, see UIO docs:

> Please note that UIO is not an universal driver interface. Devices that
> are already handled well by other kernel subsystems (like networking or
> serial or USB) are no candidates for an UIO driver.

The DMA subsystem already handles DMA devices, so write a DMA driver.

Andrew

> Best regards,
> 
> Thomas
Thomas Petazzoni April 14, 2025, 7:21 p.m. UTC | #14
Hello Andrew,

On Mon, 14 Apr 2025 12:08:44 -0500
Andrew Davis <afd@ti.com> wrote:

> "UIO is a broken legacy mess, so let's add more broken things
> to it as broken + broken => still broken, so no harm done", am I
> getting that right?

Who says UIO is a "broken legacy mess"? Only you says so. I don't see
any indication anywhere in the kernel tree suggesting that UIO is
considered a broken legacy mess.

Keep in mind that when you're running code as root, you can load a
kernel module, which can do anything on the system security-wise. So
letting UIO expose MMIO registers of devices to userspace applications
running as root is not any worse than that.

> If your FPGA IP can do DMA then you should not be using UIO in
> the first place, see UIO docs:
> 
> > Please note that UIO is not an universal driver interface. Devices that
> > are already handled well by other kernel subsystems (like networking or
> > serial or USB) are no candidates for an UIO driver.  
> 
> The DMA subsystem already handles DMA devices, so write a DMA driver.

My FPGA IP block is not a DMA controller that would fit the dmaengine
kernel subsystem. It's a weird custom device that doesn't fit in any
existing subsystem, and that happens to do "peripheral DMA" (i.e the IP
block is DMA-capable itself, without relying on a separate DMA
controller). So this (very valid) recommendation from the UIO
documentation doesn't apply to my device.

Best regards,

Thomas
Andrew Davis April 14, 2025, 8:13 p.m. UTC | #15
On 4/14/25 2:21 PM, Thomas Petazzoni wrote:
> Hello Andrew,
> 
> On Mon, 14 Apr 2025 12:08:44 -0500
> Andrew Davis <afd@ti.com> wrote:
> 
>> "UIO is a broken legacy mess, so let's add more broken things
>> to it as broken + broken => still broken, so no harm done", am I
>> getting that right?
> 
> Who says UIO is a "broken legacy mess"? Only you says so. I don't see
> any indication anywhere in the kernel tree suggesting that UIO is
> considered a broken legacy mess.
> 

I'm not saying that*, I'm pointing out your argument is that even
though what you are trying to do is broken and unsafe, it is okay to
do because it isn't any "more "broken and unsafe" than UIO already is."

*It is, but that is an argument to have outside of this thread :)

> Keep in mind that when you're running code as root, you can load a
> kernel module, which can do anything on the system security-wise. So
> letting UIO expose MMIO registers of devices to userspace applications
> running as root is not any worse than that.
> 

You can take your computer out back and shoot it too, but we shouldn't
encourage that either :) According to the original docs, UIO was created
to support "industrial I/O cards", think old one-off custom ISA cards by
vendors that had no intention of ever writing a proper driver and just
wanted to poke registers and wait on an IRQ.

IMHO we shouldn't be encouraging that, and trying to modernize UIO does just
that. It gives the impression that is how drivers should still be written.
If you setup your FPGA card to go blink an LED, sure UIO driver it is,
anything more complex, then writing a proper driver is the way to go.

>> If your FPGA IP can do DMA then you should not be using UIO in
>> the first place, see UIO docs:
>>
>>> Please note that UIO is not an universal driver interface. Devices that
>>> are already handled well by other kernel subsystems (like networking or
>>> serial or USB) are no candidates for an UIO driver.
>>
>> The DMA subsystem already handles DMA devices, so write a DMA driver.
> 
> My FPGA IP block is not a DMA controller that would fit the dmaengine
> kernel subsystem. It's a weird custom device that doesn't fit in any
> existing subsystem, and that happens to do "peripheral DMA" (i.e the IP
> block is DMA-capable itself, without relying on a separate DMA
> controller). So this (very valid) recommendation from the UIO
> documentation doesn't apply to my device.

Peripheral DMA is the much more common case, nothing new here. Could
you give a hint as to what this device does that doesn't fit *any*
current subsystem? Or are we talking a hypothetical device (which
for the sake of argument is a valid thing to say, I'm sure with an
FPGA card I could make something that doesn't fit any current
framework too). Just want to know if you are trying to solve a
specific issue or a generic issue here.

Andrew

> 
> Best regards,
> 
> Thomas