[1/3] crypto: marvell/cesa - replace dma_to_phys with dma_map_single
diff mbox

Message ID 1458252137-24497-1-git-send-email-okaya@codeaurora.org
State New, archived
Headers show

Commit Message

Sinan Kaya March 17, 2016, 10:02 p.m. UTC
Getting ready to remove dma_to_phys API. Drivers should not be
using this API for DMA operations. Instead, they should go
through the dma_map or dma_alloc APIs.

Signed-off-by: Sinan Kaya <okaya@codeaurora.org>
---
 drivers/crypto/marvell/cesa.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

Comments

Russell King - ARM Linux March 17, 2016, 10:54 p.m. UTC | #1
On Thu, Mar 17, 2016 at 06:02:15PM -0400, Sinan Kaya wrote:
> Getting ready to remove dma_to_phys API. Drivers should not be
> using this API for DMA operations. Instead, they should go
> through the dma_map or dma_alloc APIs.
> 
> Signed-off-by: Sinan Kaya <okaya@codeaurora.org>
> ---
>  drivers/crypto/marvell/cesa.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/crypto/marvell/cesa.c b/drivers/crypto/marvell/cesa.c
> index c0656e7..52ddfa4 100644
> --- a/drivers/crypto/marvell/cesa.c
> +++ b/drivers/crypto/marvell/cesa.c
> @@ -350,8 +350,8 @@ static int mv_cesa_get_sram(struct platform_device *pdev, int idx)
>  	if (IS_ERR(engine->sram))
>  		return PTR_ERR(engine->sram);
>  
> -	engine->sram_dma = phys_to_dma(cesa->dev,
> -				       (phys_addr_t)res->start);
> +	engine->sram_dma = dma_map_single(cesa->dev, engine->sram,
> +					  DMA_TO_DEVICE);

Documentation/DMA-API.txt

dma_addr_t
dma_map_single(struct device *dev, void *cpu_addr, size_t size,
                      enum dma_data_direction direction)

Notes:  Not all memory regions in a machine can be mapped by this API.
Further, contiguous kernel virtual space may not be contiguous as
physical memory.  Since this API does not provide any scatter/gather
capability, it will fail if the user tries to map a non-physically
contiguous piece of memory.  For this reason, memory to be mapped by
this API should be obtained from sources which guarantee it to be
physically contiguous (like kmalloc).

Specifically, ioremapped memory will *not* work as you expect with
dma_map_single().
Sinan Kaya March 17, 2016, 11:17 p.m. UTC | #2
On 2016-03-17 18:54, Russell King - ARM Linux wrote:
> On Thu, Mar 17, 2016 at 06:02:15PM -0400, Sinan Kaya wrote:
>> Getting ready to remove dma_to_phys API. Drivers should not be
>> using this API for DMA operations. Instead, they should go
>> through the dma_map or dma_alloc APIs.
>> 
>> Signed-off-by: Sinan Kaya <okaya@codeaurora.org>
>> ---
>>  drivers/crypto/marvell/cesa.c | 4 ++--
>>  1 file changed, 2 insertions(+), 2 deletions(-)
>> 
>> diff --git a/drivers/crypto/marvell/cesa.c 
>> b/drivers/crypto/marvell/cesa.c
>> index c0656e7..52ddfa4 100644
>> --- a/drivers/crypto/marvell/cesa.c
>> +++ b/drivers/crypto/marvell/cesa.c
>> @@ -350,8 +350,8 @@ static int mv_cesa_get_sram(struct platform_device 
>> *pdev, int idx)
>>  	if (IS_ERR(engine->sram))
>>  		return PTR_ERR(engine->sram);
>> 
>> -	engine->sram_dma = phys_to_dma(cesa->dev,
>> -				       (phys_addr_t)res->start);
>> +	engine->sram_dma = dma_map_single(cesa->dev, engine->sram,
>> +					  DMA_TO_DEVICE);
> 
> Documentation/DMA-API.txt
> 
> dma_addr_t
> dma_map_single(struct device *dev, void *cpu_addr, size_t size,
>                       enum dma_data_direction direction)
> 
> Notes:  Not all memory regions in a machine can be mapped by this API.
> Further, contiguous kernel virtual space may not be contiguous as
> physical memory.  Since this API does not provide any scatter/gather
> capability, it will fail if the user tries to map a non-physically
> contiguous piece of memory.  For this reason, memory to be mapped by
> this API should be obtained from sources which guarantee it to be
> physically contiguous (like kmalloc).
> 
> Specifically, ioremapped memory will *not* work as you expect with
> dma_map_single().


What is the correct way? I don't want to write engine->sram_dma = sram
Russell King - ARM Linux March 17, 2016, 11:50 p.m. UTC | #3
On Thu, Mar 17, 2016 at 07:17:24PM -0400, okaya@codeaurora.org wrote:
> What is the correct way? I don't want to write engine->sram_dma = sram

Well, what the driver _is_ wanting to do is to go from a CPU physical
address to a device DMA address.  phys_to_dma() looks like the correct
thing there to me, but I guess that's just an offset and doesn't take
account of any IOMMU that may be in the way.

If you have an IOMMU, then the whole phys_to_dma() thing is a total
failure as it only does a linear translation, and there are no
interfaces in the kernel to take account of an IOMMU in the way.  So,
it needs something designed for the job, implemented and discussed by
the normal methods of proposing a new cross-arch interface for drivers
to use.

What I'm certain of, though, is that the change proposed in this patch
will break current users of this driver: virt_to_page() on an address
returned by ioremap() is completely undefined, and will result in
either a kernel oops, or if not poking at memory which isn't a struct
page, ultimately resulting in something that isn't SRAM being pointed
to by "engine->sram_dma".
Boris BREZILLON March 18, 2016, 9:30 a.m. UTC | #4
On Thu, 17 Mar 2016 23:50:20 +0000
Russell King - ARM Linux <linux@arm.linux.org.uk> wrote:

> On Thu, Mar 17, 2016 at 07:17:24PM -0400, okaya@codeaurora.org wrote:
> > What is the correct way? I don't want to write engine->sram_dma = sram
> 
> Well, what the driver _is_ wanting to do is to go from a CPU physical
> address to a device DMA address.  phys_to_dma() looks like the correct
> thing there to me, but I guess that's just an offset and doesn't take
> account of any IOMMU that may be in the way.
> 
> If you have an IOMMU, then the whole phys_to_dma() thing is a total
> failure as it only does a linear translation, and there are no
> interfaces in the kernel to take account of an IOMMU in the way.  So,
> it needs something designed for the job, implemented and discussed by
> the normal methods of proposing a new cross-arch interface for drivers
> to use.
> 
> What I'm certain of, though, is that the change proposed in this patch
> will break current users of this driver: virt_to_page() on an address
> returned by ioremap() is completely undefined, and will result in
> either a kernel oops, or if not poking at memory which isn't a struct
> page, ultimately resulting in something that isn't SRAM being pointed
> to by "engine->sram_dma".
> 

Or we could just do

	engine->sram_dma = res->start;

which is pretty much what the SRAM/genalloc code is doing already.
Robin Murphy March 18, 2016, 11:25 a.m. UTC | #5
On 18/03/16 09:30, Boris Brezillon wrote:
> On Thu, 17 Mar 2016 23:50:20 +0000
> Russell King - ARM Linux <linux@arm.linux.org.uk> wrote:
>
>> On Thu, Mar 17, 2016 at 07:17:24PM -0400, okaya@codeaurora.org wrote:
>>> What is the correct way? I don't want to write engine->sram_dma = sram
>>
>> Well, what the driver _is_ wanting to do is to go from a CPU physical
>> address to a device DMA address.  phys_to_dma() looks like the correct
>> thing there to me, but I guess that's just an offset and doesn't take
>> account of any IOMMU that may be in the way.
>>
>> If you have an IOMMU, then the whole phys_to_dma() thing is a total
>> failure as it only does a linear translation, and there are no
>> interfaces in the kernel to take account of an IOMMU in the way.  So,
>> it needs something designed for the job, implemented and discussed by
>> the normal methods of proposing a new cross-arch interface for drivers
>> to use.
>>
>> What I'm certain of, though, is that the change proposed in this patch
>> will break current users of this driver: virt_to_page() on an address
>> returned by ioremap() is completely undefined, and will result in
>> either a kernel oops, or if not poking at memory which isn't a struct
>> page, ultimately resulting in something that isn't SRAM being pointed
>> to by "engine->sram_dma".
>>
>
> Or we could just do
>
> 	engine->sram_dma = res->start;
>
> which is pretty much what the SRAM/genalloc code is doing already.

As Russell points out this is yet another type of "set up a DMA master 
to access something other than kernel RAM" - there's already discussion 
in progress over how to handle this for dmaengine slaves[1], so 
gathering more use-cases might help distil exactly what the design of 
not-strictly-DMA-but-so-closely-coupled-it-can't-really-live-anywhere-else 
needs to be.

Robin.

[1]:http://lists.infradead.org/pipermail/linux-arm-kernel/2016-March/414422.html
Boris BREZILLON March 18, 2016, 11:32 a.m. UTC | #6
On Fri, 18 Mar 2016 11:25:48 +0000
Robin Murphy <robin.murphy@arm.com> wrote:

> On 18/03/16 09:30, Boris Brezillon wrote:
> > On Thu, 17 Mar 2016 23:50:20 +0000
> > Russell King - ARM Linux <linux@arm.linux.org.uk> wrote:
> >
> >> On Thu, Mar 17, 2016 at 07:17:24PM -0400, okaya@codeaurora.org wrote:
> >>> What is the correct way? I don't want to write engine->sram_dma = sram
> >>
> >> Well, what the driver _is_ wanting to do is to go from a CPU physical
> >> address to a device DMA address.  phys_to_dma() looks like the correct
> >> thing there to me, but I guess that's just an offset and doesn't take
> >> account of any IOMMU that may be in the way.
> >>
> >> If you have an IOMMU, then the whole phys_to_dma() thing is a total
> >> failure as it only does a linear translation, and there are no
> >> interfaces in the kernel to take account of an IOMMU in the way.  So,
> >> it needs something designed for the job, implemented and discussed by
> >> the normal methods of proposing a new cross-arch interface for drivers
> >> to use.
> >>
> >> What I'm certain of, though, is that the change proposed in this patch
> >> will break current users of this driver: virt_to_page() on an address
> >> returned by ioremap() is completely undefined, and will result in
> >> either a kernel oops, or if not poking at memory which isn't a struct
> >> page, ultimately resulting in something that isn't SRAM being pointed
> >> to by "engine->sram_dma".
> >>
> >
> > Or we could just do
> >
> > 	engine->sram_dma = res->start;
> >
> > which is pretty much what the SRAM/genalloc code is doing already.
> 
> As Russell points out this is yet another type of "set up a DMA master 
> to access something other than kernel RAM" - there's already discussion 
> in progress over how to handle this for dmaengine slaves[1], so 
> gathering more use-cases might help distil exactly what the design of 
> not-strictly-DMA-but-so-closely-coupled-it-can't-really-live-anywhere-else 
> needs to be.
> 
> Robin.
> 
> [1]:http://lists.infradead.org/pipermail/linux-arm-kernel/2016-March/414422.html
> 

Hm, interesting, thanks for the pointer.
Sinan Kaya March 18, 2016, 1:51 p.m. UTC | #7
On 3/18/2016 7:25 AM, Robin Murphy wrote:
> On 18/03/16 09:30, Boris Brezillon wrote:
>> On Thu, 17 Mar 2016 23:50:20 +0000
>> Russell King - ARM Linux <linux@arm.linux.org.uk> wrote:
>>
>>> On Thu, Mar 17, 2016 at 07:17:24PM -0400, okaya@codeaurora.org wrote:
>>>> What is the correct way? I don't want to write engine->sram_dma = sram
>>>
>>> Well, what the driver _is_ wanting to do is to go from a CPU physical
>>> address to a device DMA address.  phys_to_dma() looks like the correct
>>> thing there to me, but I guess that's just an offset and doesn't take
>>> account of any IOMMU that may be in the way.
>>>
>>> If you have an IOMMU, then the whole phys_to_dma() thing is a total
>>> failure as it only does a linear translation, and there are no
>>> interfaces in the kernel to take account of an IOMMU in the way.  So,
>>> it needs something designed for the job, implemented and discussed by
>>> the normal methods of proposing a new cross-arch interface for drivers
>>> to use.
>>>
>>> What I'm certain of, though, is that the change proposed in this patch
>>> will break current users of this driver: virt_to_page() on an address
>>> returned by ioremap() is completely undefined, and will result in
>>> either a kernel oops, or if not poking at memory which isn't a struct
>>> page, ultimately resulting in something that isn't SRAM being pointed
>>> to by "engine->sram_dma".
>>>
>>
>> Or we could just do
>>
>>     engine->sram_dma = res->start;
>>
>> which is pretty much what the SRAM/genalloc code is doing already.
> 
> As Russell points out this is yet another type of "set up a DMA master to access something other than kernel RAM" - there's already discussion in progress over how to handle this for dmaengine slaves[1], so gathering more use-cases might help distil exactly what the design of not-strictly-DMA-but-so-closely-coupled-it-can't-really-live-anywhere-else needs to be.
> 
> Robin.
> 
> [1]:http://lists.infradead.org/pipermail/linux-arm-kernel/2016-March/414422.html
> 

Thanks for the link. 

dma_map_resource looks like to be the correct way of doing things. Just from
the purist point of view, a driver is not supposed to know the physical address
of a DMA address. That kills the intent of using DMA API. When programming descriptors,
the DMA addresses should be programmed not physical addresses so that the same 
driver can be used in a system with IOMMU. The IOMMU DMA ops will remap the DMA 
address to a bus address that is not physical address. All of this operation needs
to be isolated from the device driver.


I don't know the architecture or the driver enough to write this. This is not ideally
right but I can do this if Boris you are OK with this. 

     engine->sram_dma = res->start;

Another option is I can write

     engine->sram_dma = swiotlb_dma_to_phys(res->start)

I agree that dma_map_single is not the right thing.
Sinan Kaya March 18, 2016, 2 p.m. UTC | #8
On 3/18/2016 9:51 AM, Sinan Kaya wrote:
> Another option is I can write
> 
>      engine->sram_dma = swiotlb_dma_to_phys(res->start)

I realized that I made a mistake in the commit message and the code above.

The code is trying to find DMA address from physical address. Not the other
way around. I'll fix it on the next version. 

The correct suggestion above would be 

      engine->sram_dma = swiotlb_phys_to_dmares->start)
Boris BREZILLON March 18, 2016, 2:20 p.m. UTC | #9
On Fri, 18 Mar 2016 09:51:37 -0400
Sinan Kaya <okaya@codeaurora.org> wrote:

> On 3/18/2016 7:25 AM, Robin Murphy wrote:
> > On 18/03/16 09:30, Boris Brezillon wrote:
> >> On Thu, 17 Mar 2016 23:50:20 +0000
> >> Russell King - ARM Linux <linux@arm.linux.org.uk> wrote:
> >>
> >>> On Thu, Mar 17, 2016 at 07:17:24PM -0400, okaya@codeaurora.org wrote:
> >>>> What is the correct way? I don't want to write engine->sram_dma = sram
> >>>
> >>> Well, what the driver _is_ wanting to do is to go from a CPU physical
> >>> address to a device DMA address.  phys_to_dma() looks like the correct
> >>> thing there to me, but I guess that's just an offset and doesn't take
> >>> account of any IOMMU that may be in the way.
> >>>
> >>> If you have an IOMMU, then the whole phys_to_dma() thing is a total
> >>> failure as it only does a linear translation, and there are no
> >>> interfaces in the kernel to take account of an IOMMU in the way.  So,
> >>> it needs something designed for the job, implemented and discussed by
> >>> the normal methods of proposing a new cross-arch interface for drivers
> >>> to use.
> >>>
> >>> What I'm certain of, though, is that the change proposed in this patch
> >>> will break current users of this driver: virt_to_page() on an address
> >>> returned by ioremap() is completely undefined, and will result in
> >>> either a kernel oops, or if not poking at memory which isn't a struct
> >>> page, ultimately resulting in something that isn't SRAM being pointed
> >>> to by "engine->sram_dma".
> >>>
> >>
> >> Or we could just do
> >>
> >>     engine->sram_dma = res->start;
> >>
> >> which is pretty much what the SRAM/genalloc code is doing already.
> > 
> > As Russell points out this is yet another type of "set up a DMA master to access something other than kernel RAM" - there's already discussion in progress over how to handle this for dmaengine slaves[1], so gathering more use-cases might help distil exactly what the design of not-strictly-DMA-but-so-closely-coupled-it-can't-really-live-anywhere-else needs to be.
> > 
> > Robin.
> > 
> > [1]:http://lists.infradead.org/pipermail/linux-arm-kernel/2016-March/414422.html
> > 
> 
> Thanks for the link. 
> 
> dma_map_resource looks like to be the correct way of doing things. Just from
> the purist point of view, a driver is not supposed to know the physical address
> of a DMA address. That kills the intent of using DMA API. When programming descriptors,
> the DMA addresses should be programmed not physical addresses so that the same 
> driver can be used in a system with IOMMU. The IOMMU DMA ops will remap the DMA 
> address to a bus address that is not physical address. All of this operation needs
> to be isolated from the device driver.
> 
> 
> I don't know the architecture or the driver enough to write this. This is not ideally
> right but I can do this if Boris you are OK with this. 
> 
>      engine->sram_dma = res->start;

I don't know.

How about waiting for the 'dma_{map,unmap}_resource' discussion to
settle down before removing phy_to_dma()/dma_to_phys() APIs (as
suggested by Robin and Russell)?
Sinan Kaya March 18, 2016, 2:21 p.m. UTC | #10
On 3/18/2016 10:20 AM, Boris Brezillon wrote:
> On Fri, 18 Mar 2016 09:51:37 -0400
> Sinan Kaya <okaya@codeaurora.org> wrote:
> 
>> On 3/18/2016 7:25 AM, Robin Murphy wrote:
>>> On 18/03/16 09:30, Boris Brezillon wrote:
>>>> On Thu, 17 Mar 2016 23:50:20 +0000
>>>> Russell King - ARM Linux <linux@arm.linux.org.uk> wrote:
>>>>
>>>>> On Thu, Mar 17, 2016 at 07:17:24PM -0400, okaya@codeaurora.org wrote:
>>>>>> What is the correct way? I don't want to write engine->sram_dma = sram
>>>>>
>>>>> Well, what the driver _is_ wanting to do is to go from a CPU physical
>>>>> address to a device DMA address.  phys_to_dma() looks like the correct
>>>>> thing there to me, but I guess that's just an offset and doesn't take
>>>>> account of any IOMMU that may be in the way.
>>>>>
>>>>> If you have an IOMMU, then the whole phys_to_dma() thing is a total
>>>>> failure as it only does a linear translation, and there are no
>>>>> interfaces in the kernel to take account of an IOMMU in the way.  So,
>>>>> it needs something designed for the job, implemented and discussed by
>>>>> the normal methods of proposing a new cross-arch interface for drivers
>>>>> to use.
>>>>>
>>>>> What I'm certain of, though, is that the change proposed in this patch
>>>>> will break current users of this driver: virt_to_page() on an address
>>>>> returned by ioremap() is completely undefined, and will result in
>>>>> either a kernel oops, or if not poking at memory which isn't a struct
>>>>> page, ultimately resulting in something that isn't SRAM being pointed
>>>>> to by "engine->sram_dma".
>>>>>
>>>>
>>>> Or we could just do
>>>>
>>>>     engine->sram_dma = res->start;
>>>>
>>>> which is pretty much what the SRAM/genalloc code is doing already.
>>>
>>> As Russell points out this is yet another type of "set up a DMA master to access something other than kernel RAM" - there's already discussion in progress over how to handle this for dmaengine slaves[1], so gathering more use-cases might help distil exactly what the design of not-strictly-DMA-but-so-closely-coupled-it-can't-really-live-anywhere-else needs to be.
>>>
>>> Robin.
>>>
>>> [1]:http://lists.infradead.org/pipermail/linux-arm-kernel/2016-March/414422.html
>>>
>>
>> Thanks for the link. 
>>
>> dma_map_resource looks like to be the correct way of doing things. Just from
>> the purist point of view, a driver is not supposed to know the physical address
>> of a DMA address. That kills the intent of using DMA API. When programming descriptors,
>> the DMA addresses should be programmed not physical addresses so that the same 
>> driver can be used in a system with IOMMU. The IOMMU DMA ops will remap the DMA 
>> address to a bus address that is not physical address. All of this operation needs
>> to be isolated from the device driver.
>>
>>
>> I don't know the architecture or the driver enough to write this. This is not ideally
>> right but I can do this if Boris you are OK with this. 
>>
>>      engine->sram_dma = res->start;
> 
> I don't know.
> 
> How about waiting for the 'dma_{map,unmap}_resource' discussion to
> settle down before removing phy_to_dma()/dma_to_phys() APIs (as
> suggested by Robin and Russell)?
> 
> 

Sure, that's fine for me.
kernel test robot March 18, 2016, 8:18 p.m. UTC | #11
Hi Sinan,

[auto build test ERROR on arm64/for-next/core]
[also build test ERROR on v4.5 next-20160318]
[if your patch is applied to the wrong git tree, please drop us a note to help improving the system]

url:    https://github.com/0day-ci/linux/commits/Sinan-Kaya/crypto-marvell-cesa-replace-dma_to_phys-with-dma_map_single/20160318-060640
base:   https://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux for-next/core
config: arm-allmodconfig (attached as .config)
reproduce:
        wget https://git.kernel.org/cgit/linux/kernel/git/wfg/lkp-tests.git/plain/sbin/make.cross -O ~/bin/make.cross
        chmod +x ~/bin/make.cross
        # save the attached .config to linux build tree
        make.cross ARCH=arm 

All errors (new ones prefixed by >>):

   drivers/crypto/marvell/cesa.c: In function 'mv_cesa_get_sram':
>> drivers/crypto/marvell/cesa.c:354:21: error: macro "dma_map_single" requires 4 arguments, but only 3 given
           DMA_TO_DEVICE);
                        ^
>> drivers/crypto/marvell/cesa.c:353:21: error: 'dma_map_single' undeclared (first use in this function)
     engine->sram_dma = dma_map_single(cesa->dev, engine->sram,
                        ^
   drivers/crypto/marvell/cesa.c:353:21: note: each undeclared identifier is reported only once for each function it appears in

vim +/dma_map_single +354 drivers/crypto/marvell/cesa.c

   347			return -EINVAL;
   348	
   349		engine->sram = devm_ioremap_resource(cesa->dev, res);
   350		if (IS_ERR(engine->sram))
   351			return PTR_ERR(engine->sram);
   352	
 > 353		engine->sram_dma = dma_map_single(cesa->dev, engine->sram,
 > 354						  DMA_TO_DEVICE);
   355	
   356		return 0;
   357	}

---
0-DAY kernel test infrastructure                Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all                   Intel Corporation

Patch
diff mbox

diff --git a/drivers/crypto/marvell/cesa.c b/drivers/crypto/marvell/cesa.c
index c0656e7..52ddfa4 100644
--- a/drivers/crypto/marvell/cesa.c
+++ b/drivers/crypto/marvell/cesa.c
@@ -350,8 +350,8 @@  static int mv_cesa_get_sram(struct platform_device *pdev, int idx)
 	if (IS_ERR(engine->sram))
 		return PTR_ERR(engine->sram);
 
-	engine->sram_dma = phys_to_dma(cesa->dev,
-				       (phys_addr_t)res->start);
+	engine->sram_dma = dma_map_single(cesa->dev, engine->sram,
+					  DMA_TO_DEVICE);
 
 	return 0;
 }