| Message ID | 20150128110523.GC6646@e104818-lin.cambridge.arm.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
On Wed, Jan 28, 2015 at 5:05 AM, Catalin Marinas <catalin.marinas@arm.com> wrote: > On Tue, Jan 27, 2015 at 06:55:15PM +0000, Murali Karicheri wrote: >> On 01/27/2015 06:27 AM, Robin Murphy wrote: >> > On 23/01/15 22:32, Murali Karicheri wrote: >> >> Fix the dma-range size when the DT attribute is missing. i.e set size to >> >> dev->coherent_dma_mask + 1 instead of dev->coherent_dma_mask. To detect >> >> overflow when mask is set to max of u64, add a check, log error and >> >> return. >> >> Some platform use mask format for size in DTS. So add a work around to >> >> catch this and fix. >> >> >> >> Cc: Joerg Roedel <joro@8bytes.org> >> >> Cc: Grant Likely <grant.likely@linaro.org> >> >> Cc: Rob Herring <robh+dt@kernel.org> >> >> Cc: Bjorn Helgaas <bhelgaas@google.com> >> >> Cc: Will Deacon <will.deacon@arm.com> >> >> Cc: Russell King <linux@arm.linux.org.uk> >> >> Cc: Arnd Bergmann <arnd@arndb.de> >> >> Cc: Suravee Suthikulpanit <Suravee.Suthikulpanit@amd.com> >> >> >> >> Signed-off-by: Murali Karicheri <m-karicheri2@ti.com> >> >> --- >> >> drivers/of/device.c | 14 +++++++++++++- >> >> 1 file changed, 13 insertions(+), 1 deletion(-) >> >> >> >> diff --git a/drivers/of/device.c b/drivers/of/device.c >> >> index 2de320d..0a5ff54 100644 >> >> --- a/drivers/of/device.c >> >> +++ b/drivers/of/device.c >> >> @@ -105,12 +105,24 @@ void of_dma_configure(struct device *dev, struct >> >> device_node *np) >> >> ret = of_dma_get_range(np, &dma_addr, &paddr, &size); >> >> if (ret < 0) { >> >> dma_addr = offset = 0; >> >> - size = dev->coherent_dma_mask; >> >> + size = dev->coherent_dma_mask + 1; >> >> } else { >> >> offset = PFN_DOWN(paddr - dma_addr); >> >> + /* >> >> + * Add a work around to treat the size as mask + 1 in case >> >> + * it is defined in DT as a mask. >> >> + */ >> >> + if (size & 1) >> >> + size = size + 1; >> >> dev_dbg(dev, "dma_pfn_offset(%#08lx)\n", offset); >> >> } >> >> >> >> + /* if size is 0, we have an overflow of u64 */ >> >> + if (!size) { >> >> + dev_err(dev, "invalid size\n"); >> >> + return; >> >> + } >> >> + >> > >> > This seems potentially fragile to dodgy DTs given that we might also be >> > using size to make a mask later. Would it make sense to double-up a >> > sanity check as mask-format detection? Something like: >> > >> > if is_power_of_2(size) >> > // use size >> > else if is_power_of_2(size + 1) >> > // use size + 1 >> > else >> > // cry >> >> How about having the logic like this? >> >> ret = of_dma_get_range(np, &dma_addr, &paddr, &size); >> if (ret < 0) { >> dma_addr = offset = 0; >> size = dev->coherent_dma_mask + 1; >> } else { >> offset = PFN_DOWN(paddr - dma_addr); >> dev_dbg(dev, "dma_pfn_offset(%#08lx)\n", offset); >> } >> >> if (is_power_of_2(size + 1)) >> size = size + 1; >> else if (!is_power_of_2(size)) >> { >> dev_err(dev, "invalid size\n"); >> return; >> } > > In of_dma_configure(), we currently assume that the default coherent > mask is 32-bit. In this thread: > > http://article.gmane.org/gmane.linux.kernel/1835096 > > we talked about setting the coherent mask based on size automatically. > I'm not sure about the size but I think we can assume is 32-bit mask + 1 > if it is not specified in the DT. Instead of just assuming a default > mask, let's assume a default size and create the mask based on this > (untested): > > diff --git a/drivers/of/platform.c b/drivers/of/platform.c > index 5b33c6a21807..9ff8d1286b44 100644 > --- a/drivers/of/platform.c > +++ b/drivers/of/platform.c > @@ -170,10 +170,10 @@ static void of_dma_configure(struct device *dev) > struct iommu_ops *iommu; > > /* > - * Set default dma-mask to 32 bit. Drivers are expected to setup > - * the correct supported dma_mask. > + * Set default size to cover the 32-bit. Drivers are expected to setup > + * the correct size and dma_mask. > */ > - dev->coherent_dma_mask = DMA_BIT_MASK(32); > + size = 1ULL << 32; > > /* > * Set it to coherent_dma_mask by default if the architecture > @@ -185,13 +185,24 @@ static void of_dma_configure(struct device *dev) > ret = of_dma_get_range(dev->of_node, &dma_addr, &paddr, &size); > if (ret < 0) { > dma_addr = offset = 0; > - size = dev->coherent_dma_mask; Are we assuming dma_addr, paddr and size are not touched on error? If so, we can get rid of this clause entirely. > } else { > offset = PFN_DOWN(paddr - dma_addr); > dev_dbg(dev, "dma_pfn_offset(%#08lx)\n", dev->dma_pfn_offset); > } > dev->dma_pfn_offset = offset; > > + /* > + * Workaround for DTs setting the size to a mask or 0. > + */ > + if (is_power_of_2(size + 1)) > + size += 1; As I mentioned, I think power of 2 is too restrictive (from a DT perspective even though the kernel may require a power of 2 here for the mask). Just checking bit0 set should be enough. Also, we need a WARN here so DTs get fixed. > + > + /* > + * Coherent DMA masks larger than 32-bit must be explicitly set by the > + * driver. > + */ > + dev->coherent_dma_mask = min(DMA_BIT_MASK(32), DMA_BIT_MASK(ilog2(size))); > + > coherent = of_dma_is_coherent(dev->of_node); > dev_dbg(dev, "device is%sdma coherent\n", > coherent ? " " : " not "); > > -- > Catalin
On 28/01/15 11:05, Catalin Marinas wrote: > On Tue, Jan 27, 2015 at 06:55:15PM +0000, Murali Karicheri wrote: >> On 01/27/2015 06:27 AM, Robin Murphy wrote: >>> On 23/01/15 22:32, Murali Karicheri wrote: >>>> Fix the dma-range size when the DT attribute is missing. i.e set size to >>>> dev->coherent_dma_mask + 1 instead of dev->coherent_dma_mask. To detect >>>> overflow when mask is set to max of u64, add a check, log error and >>>> return. >>>> Some platform use mask format for size in DTS. So add a work around to >>>> catch this and fix. >>>> >>>> Cc: Joerg Roedel <joro@8bytes.org> >>>> Cc: Grant Likely <grant.likely@linaro.org> >>>> Cc: Rob Herring <robh+dt@kernel.org> >>>> Cc: Bjorn Helgaas <bhelgaas@google.com> >>>> Cc: Will Deacon <will.deacon@arm.com> >>>> Cc: Russell King <linux@arm.linux.org.uk> >>>> Cc: Arnd Bergmann <arnd@arndb.de> >>>> Cc: Suravee Suthikulpanit <Suravee.Suthikulpanit@amd.com> >>>> >>>> Signed-off-by: Murali Karicheri <m-karicheri2@ti.com> >>>> --- >>>> drivers/of/device.c | 14 +++++++++++++- >>>> 1 file changed, 13 insertions(+), 1 deletion(-) >>>> >>>> diff --git a/drivers/of/device.c b/drivers/of/device.c >>>> index 2de320d..0a5ff54 100644 >>>> --- a/drivers/of/device.c >>>> +++ b/drivers/of/device.c >>>> @@ -105,12 +105,24 @@ void of_dma_configure(struct device *dev, struct >>>> device_node *np) >>>> ret = of_dma_get_range(np, &dma_addr, &paddr, &size); >>>> if (ret < 0) { >>>> dma_addr = offset = 0; >>>> - size = dev->coherent_dma_mask; >>>> + size = dev->coherent_dma_mask + 1; >>>> } else { >>>> offset = PFN_DOWN(paddr - dma_addr); >>>> + /* >>>> + * Add a work around to treat the size as mask + 1 in case >>>> + * it is defined in DT as a mask. >>>> + */ >>>> + if (size & 1) >>>> + size = size + 1; >>>> dev_dbg(dev, "dma_pfn_offset(%#08lx)\n", offset); >>>> } >>>> >>>> + /* if size is 0, we have an overflow of u64 */ >>>> + if (!size) { >>>> + dev_err(dev, "invalid size\n"); >>>> + return; >>>> + } >>>> + >>> >>> This seems potentially fragile to dodgy DTs given that we might also be >>> using size to make a mask later. Would it make sense to double-up a >>> sanity check as mask-format detection? Something like: >>> >>> if is_power_of_2(size) >>> // use size >>> else if is_power_of_2(size + 1) >>> // use size + 1 >>> else >>> // cry >> >> How about having the logic like this? >> >> ret = of_dma_get_range(np, &dma_addr, &paddr, &size); >> if (ret < 0) { >> dma_addr = offset = 0; >> size = dev->coherent_dma_mask + 1; >> } else { >> offset = PFN_DOWN(paddr - dma_addr); >> dev_dbg(dev, "dma_pfn_offset(%#08lx)\n", offset); >> } >> >> if (is_power_of_2(size + 1)) >> size = size + 1; >> else if (!is_power_of_2(size)) >> { >> dev_err(dev, "invalid size\n"); >> return; >> } > > In of_dma_configure(), we currently assume that the default coherent > mask is 32-bit. In this thread: > > http://article.gmane.org/gmane.linux.kernel/1835096 > > we talked about setting the coherent mask based on size automatically. > I'm not sure about the size but I think we can assume is 32-bit mask + 1 > if it is not specified in the DT. Instead of just assuming a default > mask, let's assume a default size and create the mask based on this > (untested): > > diff --git a/drivers/of/platform.c b/drivers/of/platform.c > index 5b33c6a21807..9ff8d1286b44 100644 > --- a/drivers/of/platform.c > +++ b/drivers/of/platform.c > @@ -170,10 +170,10 @@ static void of_dma_configure(struct device *dev) > struct iommu_ops *iommu; > > /* > - * Set default dma-mask to 32 bit. Drivers are expected to setup > - * the correct supported dma_mask. > + * Set default size to cover the 32-bit. Drivers are expected to setup > + * the correct size and dma_mask. > */ > - dev->coherent_dma_mask = DMA_BIT_MASK(32); > + size = 1ULL << 32; > > /* > * Set it to coherent_dma_mask by default if the architecture > @@ -185,13 +185,24 @@ static void of_dma_configure(struct device *dev) > ret = of_dma_get_range(dev->of_node, &dma_addr, &paddr, &size); > if (ret < 0) { > dma_addr = offset = 0; > - size = dev->coherent_dma_mask; > } else { > offset = PFN_DOWN(paddr - dma_addr); > dev_dbg(dev, "dma_pfn_offset(%#08lx)\n", dev->dma_pfn_offset); > } > dev->dma_pfn_offset = offset; > > + /* > + * Workaround for DTs setting the size to a mask or 0. > + */ > + if (is_power_of_2(size + 1)) > + size += 1; In fact, since the ilog2 below ends up effectively rounding down, we might as well do away with this check as well and just add 1 unconditionally. The only time it makes any difference is when we want it to anyway! I like this approach, it ends up looking a lot neater. Robin. > + > + /* > + * Coherent DMA masks larger than 32-bit must be explicitly set by the > + * driver. > + */ > + dev->coherent_dma_mask = min(DMA_BIT_MASK(32), DMA_BIT_MASK(ilog2(size))); > + > coherent = of_dma_is_coherent(dev->of_node); > dev_dbg(dev, "device is%sdma coherent\n", > coherent ? " " : " not "); >
On Wed, Jan 28, 2015 at 03:45:19PM +0000, Rob Herring wrote: > On Wed, Jan 28, 2015 at 5:05 AM, Catalin Marinas > <catalin.marinas@arm.com> wrote: > > On Tue, Jan 27, 2015 at 06:55:15PM +0000, Murali Karicheri wrote: > >> How about having the logic like this? > >> > >> ret = of_dma_get_range(np, &dma_addr, &paddr, &size); > >> if (ret < 0) { > >> dma_addr = offset = 0; > >> size = dev->coherent_dma_mask + 1; > >> } else { > >> offset = PFN_DOWN(paddr - dma_addr); > >> dev_dbg(dev, "dma_pfn_offset(%#08lx)\n", offset); > >> } > >> > >> if (is_power_of_2(size + 1)) > >> size = size + 1; > >> else if (!is_power_of_2(size)) > >> { > >> dev_err(dev, "invalid size\n"); > >> return; > >> } > > > > In of_dma_configure(), we currently assume that the default coherent > > mask is 32-bit. In this thread: > > > > http://article.gmane.org/gmane.linux.kernel/1835096 > > > > we talked about setting the coherent mask based on size automatically. > > I'm not sure about the size but I think we can assume is 32-bit mask + 1 > > if it is not specified in the DT. Instead of just assuming a default > > mask, let's assume a default size and create the mask based on this > > (untested): > > > > diff --git a/drivers/of/platform.c b/drivers/of/platform.c > > index 5b33c6a21807..9ff8d1286b44 100644 > > --- a/drivers/of/platform.c > > +++ b/drivers/of/platform.c > > @@ -170,10 +170,10 @@ static void of_dma_configure(struct device *dev) > > struct iommu_ops *iommu; > > > > /* > > - * Set default dma-mask to 32 bit. Drivers are expected to setup > > - * the correct supported dma_mask. > > + * Set default size to cover the 32-bit. Drivers are expected to setup > > + * the correct size and dma_mask. > > */ > > - dev->coherent_dma_mask = DMA_BIT_MASK(32); > > + size = 1ULL << 32; > > > > /* > > * Set it to coherent_dma_mask by default if the architecture > > @@ -185,13 +185,24 @@ static void of_dma_configure(struct device *dev) > > ret = of_dma_get_range(dev->of_node, &dma_addr, &paddr, &size); > > if (ret < 0) { > > dma_addr = offset = 0; > > - size = dev->coherent_dma_mask; > > Are we assuming dma_addr, paddr and size are not touched on error? If > so, we can get rid of this clause entirely. We can if we initialise dma_addr and offset to 0 when declared in this function. The dma_addr and size variables are later passed to the arch_setup_dma_ops(), so they need to have some sane values independent of the presence of dma-ranges in the DT. > > } else { > > offset = PFN_DOWN(paddr - dma_addr); > > dev_dbg(dev, "dma_pfn_offset(%#08lx)\n", dev->dma_pfn_offset); > > } > > dev->dma_pfn_offset = offset; > > > > + /* > > + * Workaround for DTs setting the size to a mask or 0. > > + */ > > + if (is_power_of_2(size + 1)) > > + size += 1; > > As I mentioned, I think power of 2 is too restrictive (from a DT > perspective even though the kernel may require a power of 2 here for > the mask). Just checking bit0 set should be enough. The power of 2 was mainly to cover the case where the size is wrongly written as a mask in the DT. If the size is deliberately not a power of two and not a full mask, the above check won't change it. With my proposal, ilog2 gets rid of extra bits in size, only that if the size was a mask because of DT error, we lose a bit in the coherent_dma_mask. > Also, we need a WARN here so DTs get fixed. I agree.
On Wed, Jan 28, 2015 at 03:55:57PM +0000, Robin Murphy wrote: > On 28/01/15 11:05, Catalin Marinas wrote: > > On Tue, Jan 27, 2015 at 06:55:15PM +0000, Murali Karicheri wrote: > >> How about having the logic like this? > >> > >> ret = of_dma_get_range(np, &dma_addr, &paddr, &size); > >> if (ret < 0) { > >> dma_addr = offset = 0; > >> size = dev->coherent_dma_mask + 1; > >> } else { > >> offset = PFN_DOWN(paddr - dma_addr); > >> dev_dbg(dev, "dma_pfn_offset(%#08lx)\n", offset); > >> } > >> > >> if (is_power_of_2(size + 1)) > >> size = size + 1; > >> else if (!is_power_of_2(size)) > >> { > >> dev_err(dev, "invalid size\n"); > >> return; > >> } > > > > In of_dma_configure(), we currently assume that the default coherent > > mask is 32-bit. In this thread: > > > > http://article.gmane.org/gmane.linux.kernel/1835096 > > > > we talked about setting the coherent mask based on size automatically. > > I'm not sure about the size but I think we can assume is 32-bit mask + 1 > > if it is not specified in the DT. Instead of just assuming a default > > mask, let's assume a default size and create the mask based on this > > (untested): > > > > diff --git a/drivers/of/platform.c b/drivers/of/platform.c > > index 5b33c6a21807..9ff8d1286b44 100644 > > --- a/drivers/of/platform.c > > +++ b/drivers/of/platform.c > > @@ -170,10 +170,10 @@ static void of_dma_configure(struct device *dev) > > struct iommu_ops *iommu; > > > > /* > > - * Set default dma-mask to 32 bit. Drivers are expected to setup > > - * the correct supported dma_mask. > > + * Set default size to cover the 32-bit. Drivers are expected to setup > > + * the correct size and dma_mask. > > */ > > - dev->coherent_dma_mask = DMA_BIT_MASK(32); > > + size = 1ULL << 32; > > > > /* > > * Set it to coherent_dma_mask by default if the architecture > > @@ -185,13 +185,24 @@ static void of_dma_configure(struct device *dev) > > ret = of_dma_get_range(dev->of_node, &dma_addr, &paddr, &size); > > if (ret < 0) { > > dma_addr = offset = 0; > > - size = dev->coherent_dma_mask; > > } else { > > offset = PFN_DOWN(paddr - dma_addr); > > dev_dbg(dev, "dma_pfn_offset(%#08lx)\n", dev->dma_pfn_offset); > > } > > dev->dma_pfn_offset = offset; > > > > + /* > > + * Workaround for DTs setting the size to a mask or 0. > > + */ > > + if (is_power_of_2(size + 1)) > > + size += 1; > > In fact, since the ilog2 below ends up effectively rounding down, we > might as well do away with this check as well and just add 1 > unconditionally. The only time it makes any difference is when we want > it to anyway! Well, we could simply ignore the is_power_of_2() check but without incrementing size as we don't know what arch_setup_dma_ops() does with it. I think we can remove this check altogether (we leaved without it for a while) but we need to add 1 when calculating the mask: dev->coherent_dma_mask = min(DMA_BIT_MASK(32), DMA_BIT_MASK(ilog2(size + 1)));
On 01/28/2015 10:45 AM, Rob Herring wrote: > On Wed, Jan 28, 2015 at 5:05 AM, Catalin Marinas > <catalin.marinas@arm.com> wrote: >> On Tue, Jan 27, 2015 at 06:55:15PM +0000, Murali Karicheri wrote: >>> On 01/27/2015 06:27 AM, Robin Murphy wrote: >>>> On 23/01/15 22:32, Murali Karicheri wrote: >>>>> Fix the dma-range size when the DT attribute is missing. i.e set size to >>>>> dev->coherent_dma_mask + 1 instead of dev->coherent_dma_mask. To detect >>>>> overflow when mask is set to max of u64, add a check, log error and >>>>> return. >>>>> Some platform use mask format for size in DTS. So add a work around to >>>>> catch this and fix. >>>>> >>>>> Cc: Joerg Roedel<joro@8bytes.org> >>>>> Cc: Grant Likely<grant.likely@linaro.org> >>>>> Cc: Rob Herring<robh+dt@kernel.org> >>>>> Cc: Bjorn Helgaas<bhelgaas@google.com> >>>>> Cc: Will Deacon<will.deacon@arm.com> >>>>> Cc: Russell King<linux@arm.linux.org.uk> >>>>> Cc: Arnd Bergmann<arnd@arndb.de> >>>>> Cc: Suravee Suthikulpanit<Suravee.Suthikulpanit@amd.com> >>>>> >>>>> Signed-off-by: Murali Karicheri<m-karicheri2@ti.com> >>>>> --- >>>>> drivers/of/device.c | 14 +++++++++++++- >>>>> 1 file changed, 13 insertions(+), 1 deletion(-) >>>>> >>>>> diff --git a/drivers/of/device.c b/drivers/of/device.c >>>>> index 2de320d..0a5ff54 100644 >>>>> --- a/drivers/of/device.c >>>>> +++ b/drivers/of/device.c >>>>> @@ -105,12 +105,24 @@ void of_dma_configure(struct device *dev, struct >>>>> device_node *np) >>>>> ret = of_dma_get_range(np,&dma_addr,&paddr,&size); >>>>> if (ret< 0) { >>>>> dma_addr = offset = 0; >>>>> - size = dev->coherent_dma_mask; >>>>> + size = dev->coherent_dma_mask + 1; >>>>> } else { >>>>> offset = PFN_DOWN(paddr - dma_addr); >>>>> + /* >>>>> + * Add a work around to treat the size as mask + 1 in case >>>>> + * it is defined in DT as a mask. >>>>> + */ >>>>> + if (size& 1) >>>>> + size = size + 1; >>>>> dev_dbg(dev, "dma_pfn_offset(%#08lx)\n", offset); >>>>> } >>>>> >>>>> + /* if size is 0, we have an overflow of u64 */ >>>>> + if (!size) { >>>>> + dev_err(dev, "invalid size\n"); >>>>> + return; >>>>> + } >>>>> + >>>> >>>> This seems potentially fragile to dodgy DTs given that we might also be >>>> using size to make a mask later. Would it make sense to double-up a >>>> sanity check as mask-format detection? Something like: >>>> >>>> if is_power_of_2(size) >>>> // use size >>>> else if is_power_of_2(size + 1) >>>> // use size + 1 >>>> else >>>> // cry >>> >>> How about having the logic like this? >>> >>> ret = of_dma_get_range(np,&dma_addr,&paddr,&size); >>> if (ret< 0) { >>> dma_addr = offset = 0; >>> size = dev->coherent_dma_mask + 1; >>> } else { >>> offset = PFN_DOWN(paddr - dma_addr); >>> dev_dbg(dev, "dma_pfn_offset(%#08lx)\n", offset); >>> } >>> >>> if (is_power_of_2(size + 1)) >>> size = size + 1; >>> else if (!is_power_of_2(size)) >>> { >>> dev_err(dev, "invalid size\n"); >>> return; >>> } >> >> In of_dma_configure(), we currently assume that the default coherent >> mask is 32-bit. In this thread: >> >> http://article.gmane.org/gmane.linux.kernel/1835096 >> >> we talked about setting the coherent mask based on size automatically. >> I'm not sure about the size but I think we can assume is 32-bit mask + 1 >> if it is not specified in the DT. Instead of just assuming a default >> mask, let's assume a default size and create the mask based on this >> (untested): >> >> diff --git a/drivers/of/platform.c b/drivers/of/platform.c >> index 5b33c6a21807..9ff8d1286b44 100644 >> --- a/drivers/of/platform.c >> +++ b/drivers/of/platform.c >> @@ -170,10 +170,10 @@ static void of_dma_configure(struct device *dev) >> struct iommu_ops *iommu; >> >> /* >> - * Set default dma-mask to 32 bit. Drivers are expected to setup >> - * the correct supported dma_mask. >> + * Set default size to cover the 32-bit. Drivers are expected to setup >> + * the correct size and dma_mask. >> */ >> - dev->coherent_dma_mask = DMA_BIT_MASK(32); >> + size = 1ULL<< 32; >> >> /* >> * Set it to coherent_dma_mask by default if the architecture >> @@ -185,13 +185,24 @@ static void of_dma_configure(struct device *dev) >> ret = of_dma_get_range(dev->of_node,&dma_addr,&paddr,&size); >> if (ret< 0) { >> dma_addr = offset = 0; >> - size = dev->coherent_dma_mask; > > Are we assuming dma_addr, paddr and size are not touched on error? If > so, we can get rid of this clause entirely. Checking the code for of_dma_get_range() I see paddr is modified on error case, but is used only for success case in this function. dma_addr and size are not modified. So setting dma_addr and offset to zero before hand like size might work as below dma_addr = offset = 0; size = 1ULL << 32; ret = of_dma_get_range(dev->of_node,&dma_addr,&paddr,&size); if (!ret) { offset = PFN_DOWN(paddr - dma_addr); } .. rest of the code. Murali > >> } else { >> offset = PFN_DOWN(paddr - dma_addr); >> dev_dbg(dev, "dma_pfn_offset(%#08lx)\n", dev->dma_pfn_offset); >> } >> dev->dma_pfn_offset = offset; >> >> + /* >> + * Workaround for DTs setting the size to a mask or 0. >> + */ >> + if (is_power_of_2(size + 1)) >> + size += 1; > > As I mentioned, I think power of 2 is too restrictive (from a DT > perspective even though the kernel may require a power of 2 here for > the mask). Just checking bit0 set should be enough. > > Also, we need a WARN here so DTs get fixed. > >> + >> + /* >> + * Coherent DMA masks larger than 32-bit must be explicitly set by the >> + * driver. >> + */ >> + dev->coherent_dma_mask = min(DMA_BIT_MASK(32), DMA_BIT_MASK(ilog2(size))); >> + >> coherent = of_dma_is_coherent(dev->of_node); >> dev_dbg(dev, "device is%sdma coherent\n", >> coherent ? " " : " not "); >> >> -- >> Catalin
On 01/28/2015 12:30 PM, Catalin Marinas wrote: > On Wed, Jan 28, 2015 at 03:55:57PM +0000, Robin Murphy wrote: >> On 28/01/15 11:05, Catalin Marinas wrote: >>> On Tue, Jan 27, 2015 at 06:55:15PM +0000, Murali Karicheri wrote: >>>> How about having the logic like this? >>>> >>>> ret = of_dma_get_range(np,&dma_addr,&paddr,&size); >>>> if (ret< 0) { >>>> dma_addr = offset = 0; >>>> size = dev->coherent_dma_mask + 1; >>>> } else { >>>> offset = PFN_DOWN(paddr - dma_addr); >>>> dev_dbg(dev, "dma_pfn_offset(%#08lx)\n", offset); >>>> } >>>> >>>> if (is_power_of_2(size + 1)) >>>> size = size + 1; >>>> else if (!is_power_of_2(size)) >>>> { >>>> dev_err(dev, "invalid size\n"); >>>> return; >>>> } >>> >>> In of_dma_configure(), we currently assume that the default coherent >>> mask is 32-bit. In this thread: >>> >>> http://article.gmane.org/gmane.linux.kernel/1835096 >>> >>> we talked about setting the coherent mask based on size automatically. >>> I'm not sure about the size but I think we can assume is 32-bit mask + 1 >>> if it is not specified in the DT. Instead of just assuming a default >>> mask, let's assume a default size and create the mask based on this >>> (untested): >>> >>> diff --git a/drivers/of/platform.c b/drivers/of/platform.c >>> index 5b33c6a21807..9ff8d1286b44 100644 >>> --- a/drivers/of/platform.c >>> +++ b/drivers/of/platform.c >>> @@ -170,10 +170,10 @@ static void of_dma_configure(struct device *dev) >>> struct iommu_ops *iommu; >>> >>> /* >>> - * Set default dma-mask to 32 bit. Drivers are expected to setup >>> - * the correct supported dma_mask. >>> + * Set default size to cover the 32-bit. Drivers are expected to setup >>> + * the correct size and dma_mask. >>> */ >>> - dev->coherent_dma_mask = DMA_BIT_MASK(32); >>> + size = 1ULL<< 32; >>> >>> /* >>> * Set it to coherent_dma_mask by default if the architecture >>> @@ -185,13 +185,24 @@ static void of_dma_configure(struct device *dev) >>> ret = of_dma_get_range(dev->of_node,&dma_addr,&paddr,&size); >>> if (ret< 0) { >>> dma_addr = offset = 0; >>> - size = dev->coherent_dma_mask; >>> } else { >>> offset = PFN_DOWN(paddr - dma_addr); >>> dev_dbg(dev, "dma_pfn_offset(%#08lx)\n", dev->dma_pfn_offset); >>> } >>> dev->dma_pfn_offset = offset; >>> >>> + /* >>> + * Workaround for DTs setting the size to a mask or 0. >>> + */ >>> + if (is_power_of_2(size + 1)) >>> + size += 1; >> >> In fact, since the ilog2 below ends up effectively rounding down, we >> might as well do away with this check as well and just add 1 >> unconditionally. The only time it makes any difference is when we want >> it to anyway! > > Well, we could simply ignore the is_power_of_2() check but without > incrementing size as we don't know what arch_setup_dma_ops() does with > it. > > I think we can remove this check altogether (we leaved without it for a > while) but we need to add 1 when calculating the mask: > > dev->coherent_dma_mask = min(DMA_BIT_MASK(32), > DMA_BIT_MASK(ilog2(size + 1))); > For Keystone, the dma_addr is to be taken care as well to determine the mask. The above will not work. Based on the discussion so far, this is the function I have come up with incorporating the suggestions. Please review this and see if I have missed out any. This works fine on Keystone. void of_dma_configure(struct device *dev, struct device_node *np) { u64 dma_addr = 0, paddr, size; int ret; bool coherent; unsigned long offset = 0; struct iommu_ops *iommu; /* * Set default size to cover the 32-bit. Drivers are expected to setup * the correct size and dma_mask. */ size = 1ULL << 32; ret = of_dma_get_range(np, &dma_addr, &paddr, &size); if (!ret) { offset = PFN_DOWN(paddr - dma_addr); if (!size) { dev_err(dev, "Invalid size (%llx)\n", size); return; } if (size & 1) { size = size + 1; dev_warn(dev, "Incorrect usage of size (%llx)\n", size); } dev_dbg(dev, "dma_pfn_offset(%#08lx)\n", offset); } dev->dma_pfn_offset = offset; /* * Coherent DMA masks larger than 32-bit must be explicitly set by the * driver. */ dev->coherent_dma_mask = min(DMA_BIT_MASK(32), DMA_BIT_MASK(ilog2(dma_addr + size))); /* * Set dma_mask to coherent_dma_mask by default if the architecture * code has not set it. */ if (!dev->dma_mask) dev->dma_mask = &dev->coherent_dma_mask; coherent = of_dma_is_coherent(np); dev_dbg(dev, "device is%sdma coherent\n", coherent ? " " : " not "); iommu = of_iommu_configure(dev, np); dev_dbg(dev, "device is%sbehind an iommu\n", iommu ? " " : " not "); arch_setup_dma_ops(dev, dma_addr, size, iommu, coherent); }
On Fri, Jan 30, 2015 at 06:06:27PM +0000, Murali Karicheri wrote: > On 01/28/2015 12:30 PM, Catalin Marinas wrote: > > I think we can remove this check altogether (we leaved without it for a > > while) but we need to add 1 when calculating the mask: > > > > dev->coherent_dma_mask = min(DMA_BIT_MASK(32), > > DMA_BIT_MASK(ilog2(size + 1))); > > > For Keystone, the dma_addr is to be taken care as well to determine the > mask. The above will not work. This was discussed before (not on this thread) and dma_addr should not affect the mask, it only affects the pfn offset. > Based on the discussion so far, this is the function I have come up with > incorporating the suggestions. Please review this and see if I have > missed out any. This works fine on Keystone. > > void of_dma_configure(struct device *dev, struct device_node *np) > { > u64 dma_addr = 0, paddr, size; > int ret; > bool coherent; > unsigned long offset = 0; > struct iommu_ops *iommu; > > /* > * Set default size to cover the 32-bit. Drivers are expected to setup > * the correct size and dma_mask. > */ > size = 1ULL << 32; > > ret = of_dma_get_range(np, &dma_addr, &paddr, &size); > if (!ret) { > offset = PFN_DOWN(paddr - dma_addr); > if (!size) { > dev_err(dev, "Invalid size (%llx)\n", > size); > return; > } > if (size & 1) { > size = size + 1; > dev_warn(dev, "Incorrect usage of size (%llx)\n", > size); > } > dev_dbg(dev, "dma_pfn_offset(%#08lx)\n", offset); > } > dev->dma_pfn_offset = offset; > > /* > * Coherent DMA masks larger than 32-bit must be explicitly set by the > * driver. > */ > dev->coherent_dma_mask = min(DMA_BIT_MASK(32), > DMA_BIT_MASK(ilog2(dma_addr + size))); That's not correct, coherent_dma_mask should still be calculated solely based on size, not dma_addr. Functions like swiotlb_dma_supported() use phys_to_dma() which on ARM (32-bit) subtracts the dma_pfn_offset, so the mask based on size works fine. In the arm64 tree, we haven't taken dma_pfn_offset into account for phys_to_dma() yet but if needed for a SoC, we'll add it.
On 02/02/2015 07:18 AM, Catalin Marinas wrote: > On Fri, Jan 30, 2015 at 06:06:27PM +0000, Murali Karicheri wrote: >> On 01/28/2015 12:30 PM, Catalin Marinas wrote: >>> I think we can remove this check altogether (we leaved without it for a >>> while) but we need to add 1 when calculating the mask: >>> >>> dev->coherent_dma_mask = min(DMA_BIT_MASK(32), >>> DMA_BIT_MASK(ilog2(size + 1))); >>> >> For Keystone, the dma_addr is to be taken care as well to determine the >> mask. The above will not work. > > This was discussed before (not on this thread) and dma_addr should not > affect the mask, it only affects the pfn offset. > >> Based on the discussion so far, this is the function I have come up with >> incorporating the suggestions. Please review this and see if I have >> missed out any. This works fine on Keystone. >> >> void of_dma_configure(struct device *dev, struct device_node *np) >> { >> u64 dma_addr = 0, paddr, size; >> int ret; >> bool coherent; >> unsigned long offset = 0; >> struct iommu_ops *iommu; >> >> /* >> * Set default size to cover the 32-bit. Drivers are expected to setup >> * the correct size and dma_mask. >> */ >> size = 1ULL<< 32; >> >> ret = of_dma_get_range(np,&dma_addr,&paddr,&size); >> if (!ret) { >> offset = PFN_DOWN(paddr - dma_addr); >> if (!size) { >> dev_err(dev, "Invalid size (%llx)\n", >> size); >> return; >> } >> if (size& 1) { >> size = size + 1; >> dev_warn(dev, "Incorrect usage of size (%llx)\n", >> size); >> } >> dev_dbg(dev, "dma_pfn_offset(%#08lx)\n", offset); >> } >> dev->dma_pfn_offset = offset; >> >> /* >> * Coherent DMA masks larger than 32-bit must be explicitly set by the >> * driver. >> */ >> dev->coherent_dma_mask = min(DMA_BIT_MASK(32), >> DMA_BIT_MASK(ilog2(dma_addr + size))); > > That's not correct, coherent_dma_mask should still be calculated solely > based on size, not dma_addr. > > Functions like swiotlb_dma_supported() use phys_to_dma() which on ARM > (32-bit) subtracts the dma_pfn_offset, so the mask based on size works > fine. > > In the arm64 tree, we haven't taken dma_pfn_offset into account for > phys_to_dma() yet but if needed for a SoC, we'll add it. > I need to hear Arnd's comment on this. I am seeing an issue without this change. Probably it needs a change else where. I will post the error I am getting to this list. Murali
On 02/02/2015 07:18 AM, Catalin Marinas wrote: > On Fri, Jan 30, 2015 at 06:06:27PM +0000, Murali Karicheri wrote: >> On 01/28/2015 12:30 PM, Catalin Marinas wrote: >>> I think we can remove this check altogether (we leaved without it for a >>> while) but we need to add 1 when calculating the mask: >>> >>> dev->coherent_dma_mask = min(DMA_BIT_MASK(32), >>> DMA_BIT_MASK(ilog2(size + 1))); >>> >> For Keystone, the dma_addr is to be taken care as well to determine the >> mask. The above will not work. > > This was discussed before (not on this thread) and dma_addr should not > affect the mask, it only affects the pfn offset. > >> Based on the discussion so far, this is the function I have come up with >> incorporating the suggestions. Please review this and see if I have >> missed out any. This works fine on Keystone. >> >> void of_dma_configure(struct device *dev, struct device_node *np) >> { >> u64 dma_addr = 0, paddr, size; >> int ret; >> bool coherent; >> unsigned long offset = 0; >> struct iommu_ops *iommu; >> >> /* >> * Set default size to cover the 32-bit. Drivers are expected to setup >> * the correct size and dma_mask. >> */ >> size = 1ULL<< 32; >> >> ret = of_dma_get_range(np,&dma_addr,&paddr,&size); >> if (!ret) { >> offset = PFN_DOWN(paddr - dma_addr); >> if (!size) { >> dev_err(dev, "Invalid size (%llx)\n", >> size); >> return; >> } >> if (size& 1) { >> size = size + 1; >> dev_warn(dev, "Incorrect usage of size (%llx)\n", >> size); >> } >> dev_dbg(dev, "dma_pfn_offset(%#08lx)\n", offset); >> } >> dev->dma_pfn_offset = offset; >> >> /* >> * Coherent DMA masks larger than 32-bit must be explicitly set by the >> * driver. >> */ >> dev->coherent_dma_mask = min(DMA_BIT_MASK(32), >> DMA_BIT_MASK(ilog2(dma_addr + size))); > > That's not correct, coherent_dma_mask should still be calculated solely > based on size, not dma_addr. > > Functions like swiotlb_dma_supported() use phys_to_dma() which on ARM > (32-bit) subtracts the dma_pfn_offset, so the mask based on size works > fine. > > In the arm64 tree, we haven't taken dma_pfn_offset into account for > phys_to_dma() yet but if needed for a SoC, we'll add it. > Catalin, The size based dma mask calculation itself can be a separate topic patch. This series is adding important fix to get the PCI driver functional and I would like to get this merged as soon as possible. I also want to hear from Arnd about yout comment as we had discussed this in the initial discussion of this patch series and 8/8 is essentially added based on that discussion. I will add a simple check to catch and warn the incorrect size setting in DT for dma-range as suggested by Rob Herring and create a new patch to for size based mask calculation. I will be sending v6 (expected to be merged soon) today and will work to add a new patch. Before that we need to agree on what is the content of the patch. 1. On keystone, DMA address start at 0x8000_0000 and DMA-able memory is 2G from the above base address. So without taking into account the dma_addr, mask calculated will be 0x7fff_ffff where as we need that to be 0xffff_ffff for keystone. So need to use this in the calculation. 2. W.r.t arm_pfn_offset, this was added to support Keystone as the DDR physical address for LPAE startes at 0x8_0000_0000 and the pfn offset is calculated as the PFN of (0x8_0000_0000 - 0x8000_0000) to do the dma address to DDR address translation. I haven't looked at swiotlb_dma_supported() but will do so. Murali
Murali, On Thu, Feb 05, 2015 at 09:42:27PM +0000, Murali Karicheri wrote: > On 02/02/2015 07:18 AM, Catalin Marinas wrote: > > On Fri, Jan 30, 2015 at 06:06:27PM +0000, Murali Karicheri wrote: > >> On 01/28/2015 12:30 PM, Catalin Marinas wrote: > >>> I think we can remove this check altogether (we leaved without it for a > >>> while) but we need to add 1 when calculating the mask: > >>> > >>> dev->coherent_dma_mask = min(DMA_BIT_MASK(32), > >>> DMA_BIT_MASK(ilog2(size + 1))); > >>> > >> For Keystone, the dma_addr is to be taken care as well to determine the > >> mask. The above will not work. > > > > This was discussed before (not on this thread) and dma_addr should not > > affect the mask, it only affects the pfn offset. > > > >> Based on the discussion so far, this is the function I have come up with > >> incorporating the suggestions. Please review this and see if I have > >> missed out any. This works fine on Keystone. > >> > >> void of_dma_configure(struct device *dev, struct device_node *np) > >> { > >> u64 dma_addr = 0, paddr, size; > >> int ret; > >> bool coherent; > >> unsigned long offset = 0; > >> struct iommu_ops *iommu; > >> > >> /* > >> * Set default size to cover the 32-bit. Drivers are expected to setup > >> * the correct size and dma_mask. > >> */ > >> size = 1ULL<< 32; > >> > >> ret = of_dma_get_range(np,&dma_addr,&paddr,&size); > >> if (!ret) { > >> offset = PFN_DOWN(paddr - dma_addr); > >> if (!size) { > >> dev_err(dev, "Invalid size (%llx)\n", > >> size); > >> return; > >> } > >> if (size& 1) { > >> size = size + 1; > >> dev_warn(dev, "Incorrect usage of size (%llx)\n", > >> size); > >> } > >> dev_dbg(dev, "dma_pfn_offset(%#08lx)\n", offset); > >> } > >> dev->dma_pfn_offset = offset; > >> > >> /* > >> * Coherent DMA masks larger than 32-bit must be explicitly set by the > >> * driver. > >> */ > >> dev->coherent_dma_mask = min(DMA_BIT_MASK(32), > >> DMA_BIT_MASK(ilog2(dma_addr + size))); > > > > That's not correct, coherent_dma_mask should still be calculated solely > > based on size, not dma_addr. > > > > Functions like swiotlb_dma_supported() use phys_to_dma() which on ARM > > (32-bit) subtracts the dma_pfn_offset, so the mask based on size works > > fine. > > > > In the arm64 tree, we haven't taken dma_pfn_offset into account for > > phys_to_dma() yet but if needed for a SoC, we'll add it. > > The size based dma mask calculation itself can be a separate topic > patch. This series is adding important fix to get the PCI driver > functional and I would like to get this merged as soon as possible. Well as long as you don't break the existing users of of_dma_configure() ;). > I also want to hear from Arnd about yout comment as we had discussed > this in the initial discussion of this patch series and 8/8 is > essentially added based on that discussion. I will add a simple check > to catch and warn the incorrect size setting in DT for dma-range as > suggested by Rob Herring and create a new patch to for size based mask > calculation. I will be sending v6 (expected to be merged soon) today > and will work to add a new patch. Before that we need to agree on what > is the content of the patch. > > 1. On keystone, DMA address start at 0x8000_0000 and DMA-able memory is > 2G from the above base address. So without taking into account the > dma_addr, mask calculated will be 0x7fff_ffff where as we need that to > be 0xffff_ffff for keystone. So need to use this in the calculation. Ah, you are right. I confused dma_addr in your patch with the offset. > 2. W.r.t arm_pfn_offset, this was added to support Keystone as the DDR > physical address for LPAE startes at 0x8_0000_0000 and the pfn offset is > calculated as the PFN of (0x8_0000_0000 - 0x8000_0000) to do the dma > address to DDR address translation. Indeed. I'll look at your patches again tomorrow.
diff --git a/drivers/of/platform.c b/drivers/of/platform.c index 5b33c6a21807..9ff8d1286b44 100644 --- a/drivers/of/platform.c +++ b/drivers/of/platform.c @@ -170,10 +170,10 @@ static void of_dma_configure(struct device *dev) struct iommu_ops *iommu; /* - * Set default dma-mask to 32 bit. Drivers are expected to setup - * the correct supported dma_mask. + * Set default size to cover the 32-bit. Drivers are expected to setup + * the correct size and dma_mask. */ - dev->coherent_dma_mask = DMA_BIT_MASK(32); + size = 1ULL << 32; /* * Set it to coherent_dma_mask by default if the architecture @@ -185,13 +185,24 @@ static void of_dma_configure(struct device *dev) ret = of_dma_get_range(dev->of_node, &dma_addr, &paddr, &size); if (ret < 0) { dma_addr = offset = 0; - size = dev->coherent_dma_mask; } else { offset = PFN_DOWN(paddr - dma_addr); dev_dbg(dev, "dma_pfn_offset(%#08lx)\n", dev->dma_pfn_offset); } dev->dma_pfn_offset = offset; + /* + * Workaround for DTs setting the size to a mask or 0. + */ + if (is_power_of_2(size + 1)) + size += 1; + + /* + * Coherent DMA masks larger than 32-bit must be explicitly set by the + * driver. + */ + dev->coherent_dma_mask = min(DMA_BIT_MASK(32), DMA_BIT_MASK(ilog2(size))); + coherent = of_dma_is_coherent(dev->of_node); dev_dbg(dev, "device is%sdma coherent\n", coherent ? " " : " not ");