[v2,5/7] ARM: of: introduce common routine for DMA configuration

Commit Message

Santosh Shilimkar Feb. 27, 2014, 9:17 p.m. UTC

From: Grygorii Strashko <[e-mail]>

This patch introduces ARM specific function arm_dt_dma_configure()
which intended to retrieve DMA configuration from DT and setup Platform
device's DMA parameters.

The DMA configuration in DT has to be specified using "dma-ranges"
and "dam-coherent" properties if supported. The DMA configuration applied
by arm_dt_dma_configure() as following:
 - call of_get_dma_range() and get supported DMA range info
   (dma_addr, cpu_addr, dma_size);
 - if "not found" then fill dma_mask as DMA_BIT_MASK(32)
   (this is default behaviour);
 - if "failed" then clean up dma_mask (DMA not supported)
 - if ok then update devices DMA configuration:
      set dma_mask to (dma_addr + dma_size - 1)
      set dma_pfn_offset to PFN_DOWN(cpu_addr - dma_addr)

Cc: Russell King <[e-mail]>
Cc: Arnd Bergmann <[e-mail]>
Cc: Olof Johansson <[e-mail]>
Cc: Grant Likely <[e-mail]>
Cc: Rob Herring <[e-mail]>
Signed-off-by: Grygorii Strashko <[e-mail]>
Signed-off-by: Santosh Shilimkar <[e-mail]>
---
 arch/arm/include/asm/prom.h |    3 +++
 arch/arm/kernel/devtree.c   |   61 +++++++++++++++++++++++++++++++++++++++++++
 drivers/of/platform.c       |    5 +++-
 include/linux/of.h          |    2 +-
 4 files changed, 69 insertions(+), 2 deletions(-)

Comments

Arnd Bergmann Feb. 28, 2014, 10 a.m. UTC | #1

On Thursday 27 February 2014 16:17:50 Santosh Shilimkar wrote:
> diff --git a/arch/arm/kernel/devtree.c b/arch/arm/kernel/devtree.c
> index f751714..926b5dd 100644
> --- a/arch/arm/kernel/devtree.c
> +++ b/arch/arm/kernel/devtree.c
> @@ -235,3 +238,61 @@ const struct machine_desc * __init setup_machine_fdt(unsigned int dt_phys)
>  
>  	return mdesc;
>  }
> +
> +void arm_dt_dma_configure(struct device *dev)
> +{
> +	dma_addr_t dma_addr;
> +	phys_addr_t paddr, size;
> +	dma_addr_t dma_mask;
> +	int ret;
> +
> +	/*
> +	 * if dma-ranges property doesn't exist - use 32 bits DMA mask
> +	 * by default and don't set skip archdata.dma_pfn_offset
> +	 */
> +	ret = of_dma_get_range(dev->of_node, &dma_addr, &paddr, &size);
> +	if (ret == -ENODEV) {
> +		dev->coherent_dma_mask = DMA_BIT_MASK(32);
> +		if (!dev->dma_mask)
> +			dev->dma_mask = &dev->coherent_dma_mask;
> +		return;
> +	}

I think this is a reasonable default, but I also want Russell's
opinion on this, since I suspect he will argue that we shouldn't
default to setting a DMA mask for devices that are not DMA capable.

Maybe someone has an idea how we can detect all three important cases:

a) A device is marked as DMA capable using a dma-ranges property
b) A device is known not to be DMA capable
c) we don't have any dma-ranges properties in an old dtb file
   but still want 32 bit masks by default.

> +	/* if failed - disable DMA for device */
> +	if (ret < 0) {
> +		dev_err(dev, "failed to configure DMA\n");
> +		return;
> +	}

I guess this is also where other platforms (shmobile, highbank, ...)
will want the IOMMU detection to happen.

> +	/* DMA ranges found. Calculate and set dma_pfn_offset */
> +	dev->archdata.dma_pfn_offset = PFN_DOWN(paddr - dma_addr);
> +
> +	/* Configure DMA mask */
> +	dev->dma_mask = kmalloc(sizeof(*dev->dma_mask), GFP_KERNEL);
> +	if (!dev->dma_mask)
> +		return;

Do we have to worry about freeing this? We could in theory put the
mask into pdev_archdata (as on microblaze), or point to
coherent_dma_mask (as of_platform_device_create_pdata does).
I can't think of a case where the latter won't actually work,
since coherent_dma_mask!=*dma_mask doesn't happen on any platform
device I have ever seen. coherent_dma_mask was introduced to handle
some special requirements of PCI devices on ia64 or parisc.

> +       dma_mask = dma_addr + size - 1;

I can never remember if this is actually correct, or if it would have to
be 

	dma_mask = size - 1;

instead. Russell knows.

> +	ret = arm_dma_set_mask(dev, dma_mask);
> +	if (ret < 0) {
> +		dev_err(dev, "failed to set DMA mask %#08x\n", dma_mask);
> +		kfree(dev->dma_mask);
> +		dev->dma_mask = NULL;
> +		return;
> +	}

Again I'm hoping for Russell to provide the correct answer: Should we
set the correct mask initially for the device here, or should we
rely on dma_set_mask() to refuse a mask that is larger than we
can handle?

For PCI devices, we normally assume that we can always set a 32-bit
DMA mask, but drivers can set a smaller mask if the device can
support a smaller space than the bus can. In this case, the mask
is already the intersection of what the device and all the parent
buses support, and I'm not sure how the API describe in
Documentation/DMA-API-HOWTO.txt would deal with this.

> diff --git a/drivers/of/platform.c b/drivers/of/platform.c
> index 404d1da..97d5533 100644
> --- a/drivers/of/platform.c
> +++ b/drivers/of/platform.c
> @@ -213,10 +213,13 @@ static struct platform_device *of_platform_device_create_pdata(
>  
>  #if defined(CONFIG_MICROBLAZE)
>  	dev->archdata.dma_mask = 0xffffffffUL;
> -#endif
> +#elif defined(CONFIG_ARM_LPAE)
> +	arm_dt_dma_configure(&dev->dev);
> +#else
>  	dev->dev.coherent_dma_mask = DMA_BIT_MASK(32);
>  	if (!dev->dev.dma_mask)
>  		dev->dev.dma_mask = &dev->dev.coherent_dma_mask;
> +#endif

The dependency on CONFIG_ARM_LPAE is not correct the general case,
that would be a special case on keystone. I'd suggest using
CONFIG_ARM here, and finding a different way to return false
for dma_is_coherent() on keystone with LPAE disabled.

	Arnd

Arnd Bergmann Feb. 28, 2014, 11:14 a.m. UTC | #2

On Friday 28 February 2014 13:49:34 Grygorii Strashko wrote:
> On 02/28/2014 12:00 PM, Arnd Bergmann wrote:
> > On Thursday 27 February 2014 16:17:50 Santosh Shilimkar wrote:
> >> diff --git a/arch/arm/kernel/devtree.c b/arch/arm/kernel/devtree.c
> >> +
> >> +void arm_dt_dma_configure(struct device *dev)
> >> +{
> >> +	dma_addr_t dma_addr;
> >> +	phys_addr_t paddr, size;
> >> +	dma_addr_t dma_mask;
> >> +	int ret;
> >> +
> >> +	/*
> >> +	 * if dma-ranges property doesn't exist - use 32 bits DMA mask
> >> +	 * by default and don't set skip archdata.dma_pfn_offset
> >> +	 */
> >> +	ret = of_dma_get_range(dev->of_node, &dma_addr, &paddr, &size);
> >> +	if (ret == -ENODEV) {
> >> +		dev->coherent_dma_mask = DMA_BIT_MASK(32);
> >> +		if (!dev->dma_mask)
> >> +			dev->dma_mask = &dev->coherent_dma_mask;
> >> +		return;
> >> +	}
> > 
> > I think this is a reasonable default, but I also want Russell's
> > opinion on this, since I suspect he will argue that we shouldn't
> > default to setting a DMA mask for devices that are not DMA capable.
> 
> Just to note, that's current default behavior used in of_platform_device_create_pdata()

Right, I realized that later.

> > Maybe someone has an idea how we can detect all three important cases:
> > 
> > a) A device is marked as DMA capable using a dma-ranges property
> > b) A device is known not to be DMA capable
> > c) we don't have any dma-ranges properties in an old dtb file
> >     but still want 32 bit masks by default.
> 
> Yep, This patch set supports [a, c]. But, case be [b] can be patched 
> by arch/mach code using Platform Bus notifier if needed.
> (Platform Bus notifiers will be called after arm_dt_dma_configure is 
> finished).

It would be nice to have a way to do it without a platform specific
notifier, I just haven't found a nice way to express that in DT.

> >> +	/* if failed - disable DMA for device */
> >> +	if (ret < 0) {
> >> +		dev_err(dev, "failed to configure DMA\n");
> >> +		return;
> >> +	}
> > 
> > I guess this is also where other platforms (shmobile, highbank, ...)
> > will want the IOMMU detection to happen.
> 
> This error path handling - means, DT contains wrong data :)

I wasn't referring to the error path here, sorry for being
ambiguous. What I meant that we could add code after this line
to look for an IOMMU.

> >> +	/* DMA ranges found. Calculate and set dma_pfn_offset */
> >> +	dev->archdata.dma_pfn_offset = PFN_DOWN(paddr - dma_addr);
> >> +
> >> +	/* Configure DMA mask */
> >> +	dev->dma_mask = kmalloc(sizeof(*dev->dma_mask), GFP_KERNEL);
> >> +	if (!dev->dma_mask)
> >> +		return;
> > 
> > Do we have to worry about freeing this? We could in theory put the
> > mask into pdev_archdata (as on microblaze), or point to
> > coherent_dma_mask (as of_platform_device_create_pdata does).
> > I can't think of a case where the latter won't actually work,
> > since coherent_dma_mask!=*dma_mask doesn't happen on any platform
> > device I have ever seen. coherent_dma_mask was introduced to handle
> > some special requirements of PCI devices on ia64 or parisc.
> 
> I've used platform_device_register_full() as ref here. It actually contains
> good comment regarding this mem leak issue:
> /*
>  * This memory isn't freed when the device is put,
>  * I don't have a nice idea for that though.  Conceptually
>  * dma_mask in struct device should not be a pointer.
>  * See http://thread.gmane.org/gmane.linux.kernel.pci/9081
>  */

Right. Maybe the best solution for that code path however is to
make it the same as the of_platform code where we today set the
mask pointer to &dev->coherent_mask.

> > Again I'm hoping for Russell to provide the correct answer: Should we
> > set the correct mask initially for the device here, or should we
> > rely on dma_set_mask() to refuse a mask that is larger than we
> > can handle?
> > 
> > For PCI devices, we normally assume that we can always set a 32-bit
> > DMA mask, but drivers can set a smaller mask if the device can
> > support a smaller space than the bus can. In this case, the mask
> > is already the intersection of what the device and all the parent
> > buses support, and I'm not sure how the API describe in
> > Documentation/DMA-API-HOWTO.txt would deal with this.
> 
> As mentioned by Santosh in cover letter,
> PCI (and other buses) is problem here as they may have different "dma-ranges"
> prop format (PCI #address-cells = <3>) and need to handled in different way. 
> 
> May be, this code can be limited to platform_bus_type devices only somehow.

Doesn't that already get handled correctly by of_bus_pci_translate()?
We have bus specific translation functions that should work for
both 'ranges' and 'dma-ranges'.

	Arnd

Grygorii Strashko Feb. 28, 2014, 11:49 a.m. UTC | #3

Hi Arnd,

On 02/28/2014 12:00 PM, Arnd Bergmann wrote:
> On Thursday 27 February 2014 16:17:50 Santosh Shilimkar wrote:
>> diff --git a/arch/arm/kernel/devtree.c b/arch/arm/kernel/devtree.c
>> index f751714..926b5dd 100644
>> --- a/arch/arm/kernel/devtree.c
>> +++ b/arch/arm/kernel/devtree.c
>> @@ -235,3 +238,61 @@ const struct machine_desc * __init setup_machine_fdt(unsigned int dt_phys)
>>   
>>   	return mdesc;
>>   }
>> +
>> +void arm_dt_dma_configure(struct device *dev)
>> +{
>> +	dma_addr_t dma_addr;
>> +	phys_addr_t paddr, size;
>> +	dma_addr_t dma_mask;
>> +	int ret;
>> +
>> +	/*
>> +	 * if dma-ranges property doesn't exist - use 32 bits DMA mask
>> +	 * by default and don't set skip archdata.dma_pfn_offset
>> +	 */
>> +	ret = of_dma_get_range(dev->of_node, &dma_addr, &paddr, &size);
>> +	if (ret == -ENODEV) {
>> +		dev->coherent_dma_mask = DMA_BIT_MASK(32);
>> +		if (!dev->dma_mask)
>> +			dev->dma_mask = &dev->coherent_dma_mask;
>> +		return;
>> +	}
> 
> I think this is a reasonable default, but I also want Russell's
> opinion on this, since I suspect he will argue that we shouldn't
> default to setting a DMA mask for devices that are not DMA capable.

Just to note, that's current default behavior used in of_platform_device_create_pdata()

> 
> Maybe someone has an idea how we can detect all three important cases:
> 
> a) A device is marked as DMA capable using a dma-ranges property
> b) A device is known not to be DMA capable
> c) we don't have any dma-ranges properties in an old dtb file
>     but still want 32 bit masks by default.

Yep, This patch set supports [a, c]. But, case be [b] can be patched 
by arch/mach code using Platform Bus notifier if needed.
(Platform Bus notifiers will be called after arm_dt_dma_configure is 
finished).

> 
>> +	/* if failed - disable DMA for device */
>> +	if (ret < 0) {
>> +		dev_err(dev, "failed to configure DMA\n");
>> +		return;
>> +	}
> 
> I guess this is also where other platforms (shmobile, highbank, ...)
> will want the IOMMU detection to happen.

This error path handling - means, DT contains wrong data :)

> 
>> +	/* DMA ranges found. Calculate and set dma_pfn_offset */
>> +	dev->archdata.dma_pfn_offset = PFN_DOWN(paddr - dma_addr);
>> +
>> +	/* Configure DMA mask */
>> +	dev->dma_mask = kmalloc(sizeof(*dev->dma_mask), GFP_KERNEL);
>> +	if (!dev->dma_mask)
>> +		return;
> 
> Do we have to worry about freeing this? We could in theory put the
> mask into pdev_archdata (as on microblaze), or point to
> coherent_dma_mask (as of_platform_device_create_pdata does).
> I can't think of a case where the latter won't actually work,
> since coherent_dma_mask!=*dma_mask doesn't happen on any platform
> device I have ever seen. coherent_dma_mask was introduced to handle
> some special requirements of PCI devices on ia64 or parisc.

I've used platform_device_register_full() as ref here. It actually contains
good comment regarding this mem leak issue:
/*
 * This memory isn't freed when the device is put,
 * I don't have a nice idea for that though.  Conceptually
 * dma_mask in struct device should not be a pointer.
 * See http://thread.gmane.org/gmane.linux.kernel.pci/9081
 */


> 
>> +       dma_mask = dma_addr + size - 1;
> 
> I can never remember if this is actually correct, or if it would have to
> be
> 
> 	dma_mask = size - 1;
> 
> instead. Russell knows.
> 
>> +	ret = arm_dma_set_mask(dev, dma_mask);
>> +	if (ret < 0) {
>> +		dev_err(dev, "failed to set DMA mask %#08x\n", dma_mask);
>> +		kfree(dev->dma_mask);
>> +		dev->dma_mask = NULL;
>> +		return;
>> +	}
> 
> Again I'm hoping for Russell to provide the correct answer: Should we
> set the correct mask initially for the device here, or should we
> rely on dma_set_mask() to refuse a mask that is larger than we
> can handle?
> 
> For PCI devices, we normally assume that we can always set a 32-bit
> DMA mask, but drivers can set a smaller mask if the device can
> support a smaller space than the bus can. In this case, the mask
> is already the intersection of what the device and all the parent
> buses support, and I'm not sure how the API describe in
> Documentation/DMA-API-HOWTO.txt would deal with this.

As mentioned by Santosh in cover letter,
PCI (and other buses) is problem here as they may have different "dma-ranges"
prop format (PCI #address-cells = <3>) and need to handled in different way. 

May be, this code can be limited to platform_bus_type devices only somehow.

> 
>> diff --git a/drivers/of/platform.c b/drivers/of/platform.c
>> index 404d1da..97d5533 100644
>> --- a/drivers/of/platform.c
>> +++ b/drivers/of/platform.c
>> @@ -213,10 +213,13 @@ static struct platform_device *of_platform_device_create_pdata(
>>   
>>   #if defined(CONFIG_MICROBLAZE)
>>   	dev->archdata.dma_mask = 0xffffffffUL;
>> -#endif
>> +#elif defined(CONFIG_ARM_LPAE)

ops, should be:
#endif
+#if defined(CONFIG_ARM_LPAE)

>> +	arm_dt_dma_configure(&dev->dev);
>> +#else
>>   	dev->dev.coherent_dma_mask = DMA_BIT_MASK(32);
>>   	if (!dev->dev.dma_mask)
>>   		dev->dev.dma_mask = &dev->dev.coherent_dma_mask;
>> +#endif
> 
> The dependency on CONFIG_ARM_LPAE is not correct the general case,
> that would be a special case on keystone. I'd suggest using
> CONFIG_ARM here, and finding a different way to return false
> for dma_is_coherent() on keystone with LPAE disabled.
> 

Thanks, for your comments.

Regards,
-grygorii

Rob Herring Feb. 28, 2014, 2:56 p.m. UTC | #4

On Thu, Feb 27, 2014 at 3:17 PM, Santosh Shilimkar
<[e-mail]> wrote:
> From: Grygorii Strashko <[e-mail]>
>
> This patch introduces ARM specific function arm_dt_dma_configure()
> which intended to retrieve DMA configuration from DT and setup Platform
> device's DMA parameters.
>
> The DMA configuration in DT has to be specified using "dma-ranges"
> and "dam-coherent" properties if supported. The DMA configuration applied

s/dam/dma/

> by arm_dt_dma_configure() as following:
>  - call of_get_dma_range() and get supported DMA range info
>    (dma_addr, cpu_addr, dma_size);
>  - if "not found" then fill dma_mask as DMA_BIT_MASK(32)
>    (this is default behaviour);
>  - if "failed" then clean up dma_mask (DMA not supported)
>  - if ok then update devices DMA configuration:
>       set dma_mask to (dma_addr + dma_size - 1)
>       set dma_pfn_offset to PFN_DOWN(cpu_addr - dma_addr)
>
> Cc: Russell King <[e-mail]>
> Cc: Arnd Bergmann <[e-mail]>
> Cc: Olof Johansson <[e-mail]>
> Cc: Grant Likely <[e-mail]>
> Cc: Rob Herring <[e-mail]>
> Signed-off-by: Grygorii Strashko <[e-mail]>
> Signed-off-by: Santosh Shilimkar <[e-mail]>
> ---
>  arch/arm/include/asm/prom.h |    3 +++
>  arch/arm/kernel/devtree.c   |   61 +++++++++++++++++++++++++++++++++++++++++++
>  drivers/of/platform.c       |    5 +++-
>  include/linux/of.h          |    2 +-
>  4 files changed, 69 insertions(+), 2 deletions(-)
>
> diff --git a/arch/arm/include/asm/prom.h b/arch/arm/include/asm/prom.h
> index b681575..1acb732 100644
> --- a/arch/arm/include/asm/prom.h
> +++ b/arch/arm/include/asm/prom.h
> @@ -11,11 +11,13 @@
>  #ifndef __ASMARM_PROM_H
>  #define __ASMARM_PROM_H
>
> +struct device;
>  #ifdef CONFIG_OF
>
>  extern const struct machine_desc *setup_machine_fdt(unsigned int dt_phys);
>  extern void arm_dt_memblock_reserve(void);
>  extern void __init arm_dt_init_cpu_maps(void);
> +extern void arm_dt_dma_configure(struct device *dev);
>
>  #else /* CONFIG_OF */
>
> @@ -26,6 +28,7 @@ static inline const struct machine_desc *setup_machine_fdt(unsigned int dt_phys)
>
>  static inline void arm_dt_memblock_reserve(void) { }
>  static inline void arm_dt_init_cpu_maps(void) { }
> +static inline void arm_dt_dma_configure(struct device *dev) { }
>
>  #endif /* CONFIG_OF */
>  #endif /* ASMARM_PROM_H */
> diff --git a/arch/arm/kernel/devtree.c b/arch/arm/kernel/devtree.c
> index f751714..926b5dd 100644
> --- a/arch/arm/kernel/devtree.c
> +++ b/arch/arm/kernel/devtree.c
> @@ -18,6 +18,9 @@
>  #include <linux/of_fdt.h>
>  #include <linux/of_irq.h>
>  #include <linux/of_platform.h>
> +#include <linux/of_dma.h>
> +#include <linux/dma-mapping.h>
> +#include <linux/slab.h>
>
>  #include <asm/cputype.h>
>  #include <asm/setup.h>
> @@ -235,3 +238,61 @@ const struct machine_desc * __init setup_machine_fdt(unsigned int dt_phys)
>
>         return mdesc;
>  }
> +
> +void arm_dt_dma_configure(struct device *dev)

The implementation may be ARM specific, but the need for the function
should not be.

> +{
> +       dma_addr_t dma_addr;
> +       phys_addr_t paddr, size;
> +       dma_addr_t dma_mask;
> +       int ret;
> +
> +       /*
> +        * if dma-ranges property doesn't exist - use 32 bits DMA mask
> +        * by default and don't set skip archdata.dma_pfn_offset
> +        */
> +       ret = of_dma_get_range(dev->of_node, &dma_addr, &paddr, &size);
> +       if (ret == -ENODEV) {
> +               dev->coherent_dma_mask = DMA_BIT_MASK(32);
> +               if (!dev->dma_mask)
> +                       dev->dma_mask = &dev->coherent_dma_mask;
> +               return;
> +       }
> +
> +       /* if failed - disable DMA for device */
> +       if (ret < 0) {
> +               dev_err(dev, "failed to configure DMA\n");
> +               return;
> +       }
> +
> +       /* DMA ranges found. Calculate and set dma_pfn_offset */
> +       dev->archdata.dma_pfn_offset = PFN_DOWN(paddr - dma_addr);
> +
> +       /* Configure DMA mask */
> +       dev->dma_mask = kmalloc(sizeof(*dev->dma_mask), GFP_KERNEL);

I don't believe that any ARM platform needs dma_mask to be different
from coherent_dma_mask. I traced the history to when 2 masks were
defined and this was my conclusion. Russell has also recently stated
the same:

https://lkml.org/lkml/2013/8/9/205

> +       if (!dev->dma_mask)
> +               return;
> +
> +       dma_mask = dma_addr + size - 1;
> +
> +       ret = arm_dma_set_mask(dev, dma_mask);
> +       if (ret < 0) {
> +               dev_err(dev, "failed to set DMA mask %#08x\n", dma_mask);
> +               kfree(dev->dma_mask);
> +               dev->dma_mask = NULL;
> +               return;
> +       }
> +
> +       dev_dbg(dev, "dma_pfn_offset(%#08lx) dma_mask(%#016llx)\n",
> +               dev->archdata.dma_pfn_offset, *dev->dma_mask);
> +
> +       if (of_dma_is_coherent(dev->of_node)) {
> +               set_dma_ops(dev, &arm_coherent_dma_ops);

All but this line could be in a common function. So make an arm
version of the function that calls the common function and then does
this. Or perhaps a per arch "get dma ops" function is all that is
needed.

> +               dev_dbg(dev, "device is dma coherent\n");
> +       }
> +
> +       ret = dma_set_coherent_mask(dev, dma_mask);
> +       if (ret < 0) {
> +               dev_err(dev, "failed to set coherent DMA mask %#08x\n",
> +                       dma_mask);
> +       }
> +}
> diff --git a/drivers/of/platform.c b/drivers/of/platform.c
> index 404d1da..97d5533 100644
> --- a/drivers/of/platform.c
> +++ b/drivers/of/platform.c
> @@ -213,10 +213,13 @@ static struct platform_device *of_platform_device_create_pdata(
>
>  #if defined(CONFIG_MICROBLAZE)
>         dev->archdata.dma_mask = 0xffffffffUL;

This should be moved into a microblaze specific version of the function.

> -#endif
> +#elif defined(CONFIG_ARM_LPAE)
> +       arm_dt_dma_configure(&dev->dev);
> +#else
>         dev->dev.coherent_dma_mask = DMA_BIT_MASK(32);
>         if (!dev->dev.dma_mask)
>                 dev->dev.dma_mask = &dev->dev.coherent_dma_mask;
> +#endif
>         dev->dev.bus = &platform_bus_type;
>         dev->dev.platform_data = platform_data;
>
> diff --git a/include/linux/of.h b/include/linux/of.h
> index 70c64ba..a321058 100644
> --- a/include/linux/of.h
> +++ b/include/linux/of.h
> @@ -136,7 +136,7 @@ static inline unsigned long of_read_ulong(const __be32 *cell, int size)
>         return of_read_number(cell, size);
>  }
>
> -#if defined(CONFIG_SPARC)
> +#if defined(CONFIG_SPARC) || defined(CONFIG_ARM_LPAE)
>  #include <asm/prom.h>

No. The idea here is to get rid of including prom.h.

Rob

Santosh Shilimkar Feb. 28, 2014, 3:06 p.m. UTC | #5

On Friday 28 February 2014 05:00 AM, Arnd Bergmann wrote:
>> diff --git a/drivers/of/platform.c b/drivers/of/platform.c
>> > index 404d1da..97d5533 100644
>> > --- a/drivers/of/platform.c
>> > +++ b/drivers/of/platform.c
>> > @@ -213,10 +213,13 @@ static struct platform_device *of_platform_device_create_pdata(
>> >  
>> >  #if defined(CONFIG_MICROBLAZE)
>> >  	dev->archdata.dma_mask = 0xffffffffUL;
>> > -#endif
>> > +#elif defined(CONFIG_ARM_LPAE)
>> > +	arm_dt_dma_configure(&dev->dev);
>> > +#else
>> >  	dev->dev.coherent_dma_mask = DMA_BIT_MASK(32);
>> >  	if (!dev->dev.dma_mask)
>> >  		dev->dev.dma_mask = &dev->dev.coherent_dma_mask;
>> > +#endif
> The dependency on CONFIG_ARM_LPAE is not correct the general case,
> that would be a special case on keystone. I'd suggest using
> CONFIG_ARM here, and finding a different way to return false
> for dma_is_coherent() on keystone with LPAE disabled.
> 
I made that LPAE specific assuming the 32 machines anyway are
happy with default as they are today. We can keep CONFIG_ARM
and handle the special case in machine platform notifier.

Regards,
Santosh

Arnd Bergmann Feb. 28, 2014, 3:31 p.m. UTC | #6

On Friday 28 February 2014 10:06:25 Santosh Shilimkar wrote:
> I made that LPAE specific assuming the 32 machines anyway are
> happy with default as they are today. We can keep CONFIG_ARM
> and handle the special case in machine platform notifier.

I think all other machines that support LPAE can also run with
LPAE disabled and still use coherent DMA, they may just not
support all the available RAM.

	Arnd

Santosh Shilimkar Feb. 28, 2014, 3:35 p.m. UTC | #7

On Friday 28 February 2014 10:31 AM, Arnd Bergmann wrote:
> On Friday 28 February 2014 10:06:25 Santosh Shilimkar wrote:
>> I made that LPAE specific assuming the 32 machines anyway are
>> happy with default as they are today. We can keep CONFIG_ARM
>> and handle the special case in machine platform notifier.
> 
> I think all other machines that support LPAE can also run with
> LPAE disabled and still use coherent DMA, they may just not
> support all the available RAM.
> 
Fair enough.

Regards,
Santosh

Linus Walleij March 7, 2014, 3:15 a.m. UTC | #8

On Fri, Feb 28, 2014 at 6:00 PM, Arnd Bergmann <[e-mail]> wrote:
> On Thursday 27 February 2014 16:17:50 Santosh Shilimkar wrote:
>> diff --git a/arch/arm/kernel/devtree.c b/arch/arm/kernel/devtree.c
>> index f751714..926b5dd 100644
>> --- a/arch/arm/kernel/devtree.c
>> +++ b/arch/arm/kernel/devtree.c
>> @@ -235,3 +238,61 @@ const struct machine_desc * __init setup_machine_fdt(unsigned int dt_phys)
>>
>>       return mdesc;
>>  }
>> +
>> +void arm_dt_dma_configure(struct device *dev)
>> +{
>> +     dma_addr_t dma_addr;
>> +     phys_addr_t paddr, size;
>> +     dma_addr_t dma_mask;
>> +     int ret;
>> +
>> +     /*
>> +      * if dma-ranges property doesn't exist - use 32 bits DMA mask
>> +      * by default and don't set skip archdata.dma_pfn_offset
>> +      */
>> +     ret = of_dma_get_range(dev->of_node, &dma_addr, &paddr, &size);
>> +     if (ret == -ENODEV) {
>> +             dev->coherent_dma_mask = DMA_BIT_MASK(32);
>> +             if (!dev->dma_mask)
>> +                     dev->dma_mask = &dev->coherent_dma_mask;
>> +             return;
>> +     }
>
> I think this is a reasonable default, but I also want Russell's
> opinion on this, since I suspect he will argue that we shouldn't
> default to setting a DMA mask for devices that are not DMA capable.

of_platform_device_create_pdata() in drivers/of/platform.c
does the same assumption since ages.

> Maybe someone has an idea how we can detect all three important cases:
>
> a) A device is marked as DMA capable using a dma-ranges property
> b) A device is known not to be DMA capable
> c) we don't have any dma-ranges properties in an old dtb file
>    but still want 32 bit masks by default.

The exact same solution would apply to the of core right?

Yours,
Linus Walleij

Patch

diff --git a/arch/arm/include/asm/prom.h b/arch/arm/include/asm/prom.h
index b681575..1acb732 100644
--- a/arch/arm/include/asm/prom.h
+++ b/arch/arm/include/asm/prom.h
@@ -11,11 +11,13 @@ 
 #ifndef __ASMARM_PROM_H
 #define __ASMARM_PROM_H
 
+struct device;
 #ifdef CONFIG_OF
 
 extern const struct machine_desc *setup_machine_fdt(unsigned int dt_phys);
 extern void arm_dt_memblock_reserve(void);
 extern void __init arm_dt_init_cpu_maps(void);
+extern void arm_dt_dma_configure(struct device *dev);
 
 #else /* CONFIG_OF */
 
@@ -26,6 +28,7 @@  static inline const struct machine_desc *setup_machine_fdt(unsigned int dt_phys)
 
 static inline void arm_dt_memblock_reserve(void) { }
 static inline void arm_dt_init_cpu_maps(void) { }
+static inline void arm_dt_dma_configure(struct device *dev) { }
 
 #endif /* CONFIG_OF */
 #endif /* ASMARM_PROM_H */
diff --git a/arch/arm/kernel/devtree.c b/arch/arm/kernel/devtree.c
index f751714..926b5dd 100644
--- a/arch/arm/kernel/devtree.c
+++ b/arch/arm/kernel/devtree.c
@@ -18,6 +18,9 @@ 
 #include <linux/of_fdt.h>
 #include <linux/of_irq.h>
 #include <linux/of_platform.h>
+#include <linux/of_dma.h>
+#include <linux/dma-mapping.h>
+#include <linux/slab.h>
 
 #include <asm/cputype.h>
 #include <asm/setup.h>
@@ -235,3 +238,61 @@  const struct machine_desc * __init setup_machine_fdt(unsigned int dt_phys)
 
 	return mdesc;
 }
+
+void arm_dt_dma_configure(struct device *dev)
+{
+	dma_addr_t dma_addr;
+	phys_addr_t paddr, size;
+	dma_addr_t dma_mask;
+	int ret;
+
+	/*
+	 * if dma-ranges property doesn't exist - use 32 bits DMA mask
+	 * by default and don't set skip archdata.dma_pfn_offset
+	 */
+	ret = of_dma_get_range(dev->of_node, &dma_addr, &paddr, &size);
+	if (ret == -ENODEV) {
+		dev->coherent_dma_mask = DMA_BIT_MASK(32);
+		if (!dev->dma_mask)
+			dev->dma_mask = &dev->coherent_dma_mask;
+		return;
+	}
+
+	/* if failed - disable DMA for device */
+	if (ret < 0) {
+		dev_err(dev, "failed to configure DMA\n");
+		return;
+	}
+
+	/* DMA ranges found. Calculate and set dma_pfn_offset */
+	dev->archdata.dma_pfn_offset = PFN_DOWN(paddr - dma_addr);
+
+	/* Configure DMA mask */
+	dev->dma_mask = kmalloc(sizeof(*dev->dma_mask), GFP_KERNEL);
+	if (!dev->dma_mask)
+		return;
+
+	dma_mask = dma_addr + size - 1;
+
+	ret = arm_dma_set_mask(dev, dma_mask);
+	if (ret < 0) {
+		dev_err(dev, "failed to set DMA mask %#08x\n", dma_mask);
+		kfree(dev->dma_mask);
+		dev->dma_mask = NULL;
+		return;
+	}
+
+	dev_dbg(dev, "dma_pfn_offset(%#08lx) dma_mask(%#016llx)\n",
+		dev->archdata.dma_pfn_offset, *dev->dma_mask);
+
+	if (of_dma_is_coherent(dev->of_node)) {
+		set_dma_ops(dev, &arm_coherent_dma_ops);
+		dev_dbg(dev, "device is dma coherent\n");
+	}
+
+	ret = dma_set_coherent_mask(dev, dma_mask);
+	if (ret < 0) {
+		dev_err(dev, "failed to set coherent DMA mask %#08x\n",
+			dma_mask);
+	}
+}
diff --git a/drivers/of/platform.c b/drivers/of/platform.c
index 404d1da..97d5533 100644
--- a/drivers/of/platform.c
+++ b/drivers/of/platform.c
@@ -213,10 +213,13 @@  static struct platform_device *of_platform_device_create_pdata(
 
 #if defined(CONFIG_MICROBLAZE)
 	dev->archdata.dma_mask = 0xffffffffUL;
-#endif
+#elif defined(CONFIG_ARM_LPAE)
+	arm_dt_dma_configure(&dev->dev);
+#else
 	dev->dev.coherent_dma_mask = DMA_BIT_MASK(32);
 	if (!dev->dev.dma_mask)
 		dev->dev.dma_mask = &dev->dev.coherent_dma_mask;
+#endif
 	dev->dev.bus = &platform_bus_type;
 	dev->dev.platform_data = platform_data;
 
diff --git a/include/linux/of.h b/include/linux/of.h
index 70c64ba..a321058 100644
--- a/include/linux/of.h
+++ b/include/linux/of.h
@@ -136,7 +136,7 @@  static inline unsigned long of_read_ulong(const __be32 *cell, int size)
 	return of_read_number(cell, size);
 }
 
-#if defined(CONFIG_SPARC)
+#if defined(CONFIG_SPARC) || defined(CONFIG_ARM_LPAE)
 #include <asm/prom.h>
 #endif