diff mbox

[v2] devicetree: Add generic IOMMU device tree bindings

Message ID 1400877395-4235-1-git-send-email-thierry.reding@gmail.com (mailing list archive)
State New, archived
Headers show

Commit Message

Thierry Reding May 23, 2014, 8:36 p.m. UTC
From: Thierry Reding <treding@nvidia.com>

This commit introduces a generic device tree binding for IOMMU devices.
Only a very minimal subset is described here, but it is enough to cover
the requirements of both the Exynos System MMU and Tegra SMMU as
discussed here:

    https://lkml.org/lkml/2014/4/27/346

Signed-off-by: Thierry Reding <treding@nvidia.com>
---
Apologies for the noise, but apparently I mistyped one of the email
addresses, should be fixed now.

Changes in v2:
- add notes about "dma-ranges" property (drop note from commit message)
- document priorities of "iommus" property vs. "dma-ranges" property
- drop #iommu-cells in favour of #address-cells and #size-cells
- remove multiple-master device example

 Documentation/devicetree/bindings/iommu/iommu.txt | 167 ++++++++++++++++++++++
 1 file changed, 167 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/iommu/iommu.txt

Comments

Stephen Warren May 29, 2014, 3:52 p.m. UTC | #1
On 05/23/2014 02:36 PM, Thierry Reding wrote:
> From: Thierry Reding <treding@nvidia.com>
> 
> This commit introduces a generic device tree binding for IOMMU devices.
> Only a very minimal subset is described here, but it is enough to cover
> the requirements of both the Exynos System MMU and Tegra SMMU as
> discussed here:
> 
>     https://lkml.org/lkml/2014/4/27/346
> 
> Signed-off-by: Thierry Reding <treding@nvidia.com>
> ---
> Apologies for the noise, but apparently I mistyped one of the email
> addresses, should be fixed now.
> 
> Changes in v2:
> - add notes about "dma-ranges" property (drop note from commit message)
> - document priorities of "iommus" property vs. "dma-ranges" property
> - drop #iommu-cells in favour of #address-cells and #size-cells

I think this is a mistake. address-cells/size-cells are for transactions
flowing down the bus (from the CPU to date). Describing a connection
from a device to an IOMMU is something completely different, and should
therefore simply use an iommu-cells property to describe any necessary
information. If we start re-using properties for different things in
different contexts, how is anyone going to know what they mean, and how
will conflicts be resolved. For example, what if there's a single HW
module that both acts as a regular register bus with children (where
address-cells/size-cells defines how transactions reach the children
from the parent), and is also an IOMMU (where according to this binding
proposal, address-cells/size-cells represent some aspect of the IOMMU
feature). Using different properties for different things is the only
sane way to keep different concepts separate. Another alternative would
be to represent the single HW module as separate nodes in DT, but I
think that will only make our lives harder, and where I've done that in
the past, I've regretted it.
Thierry Reding May 30, 2014, 7:30 a.m. UTC | #2
On Thu, May 29, 2014 at 09:52:22AM -0600, Stephen Warren wrote:
> On 05/23/2014 02:36 PM, Thierry Reding wrote:
> > From: Thierry Reding <treding@nvidia.com>
> > 
> > This commit introduces a generic device tree binding for IOMMU devices.
> > Only a very minimal subset is described here, but it is enough to cover
> > the requirements of both the Exynos System MMU and Tegra SMMU as
> > discussed here:
> > 
> >     https://lkml.org/lkml/2014/4/27/346
> > 
> > Signed-off-by: Thierry Reding <treding@nvidia.com>
> > ---
> > Apologies for the noise, but apparently I mistyped one of the email
> > addresses, should be fixed now.
> > 
> > Changes in v2:
> > - add notes about "dma-ranges" property (drop note from commit message)
> > - document priorities of "iommus" property vs. "dma-ranges" property
> > - drop #iommu-cells in favour of #address-cells and #size-cells
> 
> I think this is a mistake. address-cells/size-cells are for transactions
> flowing down the bus (from the CPU to date). Describing a connection
> from a device to an IOMMU is something completely different, and should
> therefore simply use an iommu-cells property to describe any necessary
> information. If we start re-using properties for different things in
> different contexts, how is anyone going to know what they mean, and how
> will conflicts be resolved. For example, what if there's a single HW
> module that both acts as a regular register bus with children (where
> address-cells/size-cells defines how transactions reach the children
> from the parent), and is also an IOMMU (where according to this binding
> proposal, address-cells/size-cells represent some aspect of the IOMMU
> feature). Using different properties for different things is the only
> sane way to keep different concepts separate. Another alternative would
> be to represent the single HW module as separate nodes in DT, but I
> think that will only make our lives harder, and where I've done that in
> the past, I've regretted it.

There was some back-and-forth on this topic and the latest concensus
when I wrote the second version was that #address-cells and #size-cells
were to be used.

But there was some bore back-and-forth after that, and it seems like
Arnd no longer thinks that using #address-cells and #size-cells is a
good idea either[0].

Arnd, can you take another look at this binding and see if there's
anything else missing? If not I'll go through the document again and
update all #address-cells/#size-cells references with #iommu-cells as
appropriate and submit v3.

Thanks,
Thierry

[0]: https://lkml.org/lkml/2014/5/20/609
Dave Martin May 30, 2014, 11:22 a.m. UTC | #3
On Fri, May 23, 2014 at 10:36:35PM +0200, Thierry Reding wrote:
> From: Thierry Reding <treding@nvidia.com>
> 
> This commit introduces a generic device tree binding for IOMMU devices.
> Only a very minimal subset is described here, but it is enough to cover
> the requirements of both the Exynos System MMU and Tegra SMMU as
> discussed here:
> 
>     https://lkml.org/lkml/2014/4/27/346
> 
> Signed-off-by: Thierry Reding <treding@nvidia.com>
> ---
> Apologies for the noise, but apparently I mistyped one of the email
> addresses, should be fixed now.
> 
> Changes in v2:
> - add notes about "dma-ranges" property (drop note from commit message)
> - document priorities of "iommus" property vs. "dma-ranges" property
> - drop #iommu-cells in favour of #address-cells and #size-cells
> - remove multiple-master device example
> 
>  Documentation/devicetree/bindings/iommu/iommu.txt | 167 ++++++++++++++++++++++
>  1 file changed, 167 insertions(+)
>  create mode 100644 Documentation/devicetree/bindings/iommu/iommu.txt
> 
> diff --git a/Documentation/devicetree/bindings/iommu/iommu.txt b/Documentation/devicetree/bindings/iommu/iommu.txt
> new file mode 100644
> index 000000000000..6ce759afcc94
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/iommu/iommu.txt
> @@ -0,0 +1,167 @@
> +This document describes the generic device tree binding for IOMMUs and their
> +master(s).
> +
> +
> +IOMMU device node:
> +==================
> +
> +An IOMMU can provide the following services:
> +
> +* Remap address space to allow devices to access physical memory ranges that
> +  they otherwise wouldn't be capable of accessing.
> +
> +  Example: 32-bit DMA to 64-bit physical addresses
> +
> +* Implement scatter-gather at page level granularity so that the device does
> +  not have to.
> +
> +* Provide system protection against "rogue" DMA by forcing all accesses to go
> +  through the IOMMU and faulting when encountering accesses to unmapped
> +  address regions.
> +
> +* Provide address space isolation between multiple contexts.
> +
> +  Example: Virtualization
> +
> +Device nodes compatible with this binding represent hardware with some of the
> +above capabilities.
> +
> +IOMMUs can be single-master or multiple-master. Single-master IOMMU devices
> +typically have a fixed association to the master device, whereas multiple-
> +master IOMMU devices can translate accesses from more than one master.
> +
> +The device tree node of the IOMMU device's parent bus must contain a valid
> +"dma-ranges" property that describes how the physical address space of the
> +IOMMU maps to memory. An empty "dma-ranges" property means that there is a
> +1:1 mapping from IOMMU to memory.
> +
> +Required properties:
> +--------------------
> +- #address-cells: The number of cells in an IOMMU specifier needed to encode
> +  an address.
> +- #size-cells: The number of cells in an IOMMU specifier needed to represent
> +  the length of an address range.
> +
> +Typical values for the above include:
> +- #address-cells = <0>, size-cells = <0>: Single master IOMMU devices are not
> +  configurable and therefore no additional information needs to be encoded in
> +  the specifier. This may also apply to multiple master IOMMU devices that do
> +  not allow the association of masters to be configured.
> +- #address-cells = <1>, size-cells = <0>: Multiple master IOMMU devices may
> +  need to be configured in order to enable translation for a given master. In
> +  such cases the single address cell corresponds to the master device's ID.
> +- #address-cells = <2>, size-cells = <2>: Some IOMMU devices allow the DMA
> +  window for masters to be configured. The first cell of the address in this
> +  may contain the master device's ID for example, while the second cell could
> +  contain the start of the DMA window for the given device. The length of the
> +  DMA window is specified by two additional cells.
> +
> +
> +IOMMU master node:
> +==================
> +
> +Devices that access memory through an IOMMU are called masters. A device can
> +have multiple master interfaces (to one or more IOMMU devices).
> +
> +Required properties:
> +--------------------
> +- iommus: A list of phandle and IOMMU specifier pairs that describe the IOMMU
> +  master interfaces of the device. One entry in the list describes one master
> +  interface of the device.
> +
> +When an "iommus" property is specified in a device tree node, the IOMMU will
> +be used for address translation. If a "dma-ranges" property exists in the
> +device's parent node it will be ignored. An exception to this rule is if the
> +referenced IOMMU is disabled, in which case the "dma-ranges" property of the
> +parent shall take effect.
> +
> +Optional properties:
> +--------------------
> +- iommu-names: A list of names identifying each entry in the "iommus"
> +  property.
> +
> +
> +Notes:
> +======
> +
> +One possible extension to the above is to use an "iommus" property along with
> +a "dma-ranges" property in a bus device node (such as PCI host bridges). This
> +can be useful to describe how children on the bus relate to the IOMMU if they
> +are not explicitly listed in the device tree (e.g. PCI devices). However, the
> +requirements of that use-case haven't been fully determined yet. Implementing
> +this is therefore not recommended without further discussion and extension of
> +this binding.

Sounds sensible.

> +
> +
> +Examples:
> +=========
> +
> +Single-master IOMMU:
> +--------------------
> +
> +	iommu {
> +		#address-cells = <0>;
> +		#size-cells = <0>;
> +	};
> +
> +	master {
> +		iommus = <&/iommu>;
> +	};
> +
> +Multiple-master IOMMU with fixed associations:
> +----------------------------------------------
> +
> +	/* multiple-master IOMMU */
> +	iommu {
> +		/*
> +		 * Masters are statically associated with this IOMMU and
> +		 * address translation is always enabled.
> +		 */
> +		#address-cells = <0>;
> +		#size-cells = <0>;

In this example, can different translations be set up for the different
masters?

With no cells available to contain any sort of ID, it looks like this
is not possible.

> +	};
> +
> +	/* static association with IOMMU */
> +	master@1 {
> +		reg = <1>;
> +		iommus = <&/iommu>;
> +	};
> +
> +	/* static association with IOMMU */
> +	master@2 {
> +		reg = <2>;
> +		iommus = <&/iommu>;
> +	};
> +
> +Multiple-master IOMMU:
> +----------------------
> +
> +	iommu {
> +		/* the specifier represents the ID of the master */
> +		#address-cells = <1>;
> +		#size-cells = <0>;
> +	};
> +
> +	master {
> +		/* device has master ID 42 in the IOMMU */
> +		iommus = <&/iommu 42>;
> +	};
> +
> +Multiple-master IOMMU with configurable DMA window:
> +---------------------------------------------------
> +
> +	/ {
> +		#address-cells = <1>;
> +		#size-cells = <1>;
> +
> +		iommu {
> +			/* master ID, address of DMA window */
> +			#address-cells = <2>;
> +			#size-cells = <2>;
> +		};
> +
> +		master {
> +			/* master ID 42, 4 GiB DMA window starting at 0 */
> +			iommus = <&/iommu  42 0  0x1 0x0>;

I'm still concerned that in order to deal with future cases we will have
to invent multiple ways to parse the "iommus" property.  For example, if
we have a PCEe RC mastering through an IOMMU, it will pass a huge set
of possible master IDs to the IOMMU, not just noe or two.

Do you have a solution in mind for that which doesn't break backwards
compatibility?

One option is to include an extra cell to the IOMMUs property
that indicates how to parse it.  For now, only a single value would
be defined.  For example:

	iommus = <&/iommu IOMMU_SIMPLE 42>;

Then maybe later

	iommus = <&/iommu IOMMU_RANGE 0x10000 0x10000>;

(I'm not suggesting what IOMMU_RANGE might mean.)


Other options are to introduce a new property name

	range-iommus = <&/iommu 0x10000 0x10000>;

or control the parsing of incompatible iommus properties via a compatible
string somewhere.

Cheers
---Dave
Dave Martin May 30, 2014, 11:27 a.m. UTC | #4
On Fri, May 30, 2014 at 08:30:08AM +0100, Thierry Reding wrote:
> On Thu, May 29, 2014 at 09:52:22AM -0600, Stephen Warren wrote:
> > On 05/23/2014 02:36 PM, Thierry Reding wrote:
> > > From: Thierry Reding <treding@nvidia.com>
> > > 
> > > This commit introduces a generic device tree binding for IOMMU devices.
> > > Only a very minimal subset is described here, but it is enough to cover
> > > the requirements of both the Exynos System MMU and Tegra SMMU as
> > > discussed here:
> > > 
> > >     https://lkml.org/lkml/2014/4/27/346
> > > 
> > > Signed-off-by: Thierry Reding <treding@nvidia.com>
> > > ---
> > > Apologies for the noise, but apparently I mistyped one of the email
> > > addresses, should be fixed now.
> > > 
> > > Changes in v2:
> > > - add notes about "dma-ranges" property (drop note from commit message)
> > > - document priorities of "iommus" property vs. "dma-ranges" property
> > > - drop #iommu-cells in favour of #address-cells and #size-cells
> > 
> > I think this is a mistake. address-cells/size-cells are for transactions
> > flowing down the bus (from the CPU to date). Describing a connection
> > from a device to an IOMMU is something completely different, and should
> > therefore simply use an iommu-cells property to describe any necessary
> > information. If we start re-using properties for different things in
> > different contexts, how is anyone going to know what they mean, and how
> > will conflicts be resolved. For example, what if there's a single HW
> > module that both acts as a regular register bus with children (where
> > address-cells/size-cells defines how transactions reach the children
> > from the parent), and is also an IOMMU (where according to this binding
> > proposal, address-cells/size-cells represent some aspect of the IOMMU
> > feature). Using different properties for different things is the only
> > sane way to keep different concepts separate. Another alternative would
> > be to represent the single HW module as separate nodes in DT, but I
> > think that will only make our lives harder, and where I've done that in
> > the past, I've regretted it.
> 
> There was some back-and-forth on this topic and the latest concensus
> when I wrote the second version was that #address-cells and #size-cells
> were to be used.
> 
> But there was some bore back-and-forth after that, and it seems like
> Arnd no longer thinks that using #address-cells and #size-cells is a
> good idea either[0].

Mistake or not, ePAPR already (ab)uses the address concept for PCI.
Unless ePAPR is wrong, I don't think it makes sense to argue that
the address concept cannot be repurposed.

The reason why this is abused for PCI is the same as our reason here:
different masters really are treated as distinct even when accessing
the same destination address, and DT has no general native way to
describe that.

One clear advantage of using #address-cells etc. is that ePAPR already
has very clear and well-defined ways of how to specify range mappings.
This gives us a ready-made way to describe windowed IOMMUs and 1:1
remappings of whole blocks of master IDs, which are the most obvious
cases other than having a small-integer set of explicit IDs.

That said, the PCI pseudo-address thing is not something we _necessarily_
want to repeat.

> Arnd, can you take another look at this binding and see if there's
> anything else missing? If not I'll go through the document again and
> update all #address-cells/#size-cells references with #iommu-cells as
> appropriate and submit v3.

How do you envisage propagation of the master ID bits downstream of the
IOMMU would be described?

We will definitely need a way to describe this for GICv3.  How those
values are propagated is likely to vary between related SoCs and doesn't
feel like it should be baked into a driver, especially for the ARM SMMU
which may get reused in radically different SoC families from different
vendors.

The most likely types of remapping are the adding of a base offset or
some extra bits to the ID -- because not all MSIs to the GIC will
necessarily pass through the IOMMU.  It's also possible that we might
see ID squashing or folding in some systems.


For types of remapping which mix the ID and address together, I now
do tend to agree that any flexibility arising from describing that
in a general way that is unlikely to repay the cost of trying to
interpret and analyse the DT.  Defining a few sterotypical kinds
of mapping explicitly, as needed, looks more sensible for now.  The
windowed-IOMMU case is one example.

Cheers
---Dave
Arnd Bergmann May 30, 2014, 7:01 p.m. UTC | #5
On Friday 30 May 2014 12:22:32 Dave Martin wrote:
> > +
> > +Examples:
> > +=========
> > +
> > +Single-master IOMMU:
> > +--------------------
> > +
> > +	iommu {
> > +		#address-cells = <0>;
> > +		#size-cells = <0>;
> > +	};
> > +
> > +	master {
> > +		iommus = <&/iommu>;
> > +	};
> > +
> > +Multiple-master IOMMU with fixed associations:
> > +----------------------------------------------
> > +
> > +	/* multiple-master IOMMU */
> > +	iommu {
> > +		/*
> > +		 * Masters are statically associated with this IOMMU and
> > +		 * address translation is always enabled.
> > +		 */
> > +		#address-cells = <0>;
> > +		#size-cells = <0>;
> 
> In this example, can different translations be set up for the different
> masters?
> 
> With no cells available to contain any sort of ID, it looks like this
> is not possible.

Correct, this example is for an IOMMU that does not use IDs but has a
shared address space for all devices.

> > +Multiple-master IOMMU with configurable DMA window:
> > +---------------------------------------------------
> > +
> > +	/ {
> > +		#address-cells = <1>;
> > +		#size-cells = <1>;
> > +
> > +		iommu {
> > +			/* master ID, address of DMA window */
> > +			#address-cells = <2>;
> > +			#size-cells = <2>;
> > +		};
> > +
> > +		master {
> > +			/* master ID 42, 4 GiB DMA window starting at 0 */
> > +			iommus = <&/iommu  42 0  0x1 0x0>;
> 
> I'm still concerned that in order to deal with future cases we will have
> to invent multiple ways to parse the "iommus" property.  For example, if
> we have a PCEe RC mastering through an IOMMU, it will pass a huge set
> of possible master IDs to the IOMMU, not just noe or two.
> 
> Do you have a solution in mind for that which doesn't break backwards
> compatibility?

I think we can treat PCI as a special case here and have an interface
that gets used by the PCI core code to talk to the IOMMU core code
when setting up a the dma_map_ops for a PCI function. As long as the
IOMMU driver understands what PCI is, we don't have to describe the
mapping in detail.

> One option is to include an extra cell to the IOMMUs property
> that indicates how to parse it.  For now, only a single value would
> be defined.  For example:
> 
> 	iommus = <&/iommu IOMMU_SIMPLE 42>;
> 
> Then maybe later
> 
> 	iommus = <&/iommu IOMMU_RANGE 0x10000 0x10000>;
> 
> (I'm not suggesting what IOMMU_RANGE might mean.)
> 

This can really be left up to the specific IOMMU driver itself.
We can have drivers that support both #address-cells=<1>
and #address-cells=<2> and behave differently based on that.
I don't see a reason to define that across IOMMU implementations.

> Other options are to introduce a new property name
> 
> 	range-iommus = <&/iommu 0x10000 0x10000>;
> 
> or control the parsing of incompatible iommus properties via a compatible
> string somewhere.

Introducing a new compatible string is always an option as the last resort.

	Arnd
Arnd Bergmann May 30, 2014, 7:11 p.m. UTC | #6
On Friday 30 May 2014 12:27:28 Dave Martin wrote:
> On Fri, May 30, 2014 at 08:30:08AM +0100, Thierry Reding wrote:
> > On Thu, May 29, 2014 at 09:52:22AM -0600, Stephen Warren wrote:
> > > On 05/23/2014 02:36 PM, Thierry Reding wrote:
> > > I think this is a mistake. address-cells/size-cells are for transactions
> > > flowing down the bus (from the CPU to date). Describing a connection
> > > from a device to an IOMMU is something completely different, and should
> > > therefore simply use an iommu-cells property to describe any necessary
> > > information. If we start re-using properties for different things in
> > > different contexts, how is anyone going to know what they mean, and how
> > > will conflicts be resolved. For example, what if there's a single HW
> > > module that both acts as a regular register bus with children (where
> > > address-cells/size-cells defines how transactions reach the children
> > > from the parent), and is also an IOMMU (where according to this binding
> > > proposal, address-cells/size-cells represent some aspect of the IOMMU
> > > feature). Using different properties for different things is the only
> > > sane way to keep different concepts separate. Another alternative would
> > > be to represent the single HW module as separate nodes in DT, but I
> > > think that will only make our lives harder, and where I've done that in
> > > the past, I've regretted it.
> > 
> > There was some back-and-forth on this topic and the latest concensus
> > when I wrote the second version was that #address-cells and #size-cells
> > were to be used.
> > 
> > But there was some bore back-and-forth after that, and it seems like
> > Arnd no longer thinks that using #address-cells and #size-cells is a
> > good idea either[0].
> 
> Mistake or not, ePAPR already (ab)uses the address concept for PCI.
> Unless ePAPR is wrong, I don't think it makes sense to argue that
> the address concept cannot be repurposed.
> 
> The reason why this is abused for PCI is the same as our reason here:
> different masters really are treated as distinct even when accessing
> the same destination address, and DT has no general native way to
> describe that.
> 
> One clear advantage of using #address-cells etc. is that ePAPR already
> has very clear and well-defined ways of how to specify range mappings.
> This gives us a ready-made way to describe windowed IOMMUs and 1:1
> remappings of whole blocks of master IDs, which are the most obvious
> cases other than having a small-integer set of explicit IDs.
> 
> That said, the PCI pseudo-address thing is not something we _necessarily_
> want to repeat.

I'm really impartial to the question of whether to use #address-cells
or #iommus now. It's possible that we can use the addresses in some
meaningful way, but it's not clear to me that this will help make the
representation cleaner or that we will actually want it for SMMU.

> > Arnd, can you take another look at this binding and see if there's
> > anything else missing? If not I'll go through the document again and
> > update all #address-cells/#size-cells references with #iommu-cells as
> > appropriate and submit v3.
> 
> How do you envisage propagation of the master ID bits downstream of the
> IOMMU would be described?
> 
> We will definitely need a way to describe this for GICv3.  How those
> values are propagated is likely to vary between related SoCs and doesn't
> feel like it should be baked into a driver, especially for the ARM SMMU
> which may get reused in radically different SoC families from different
> vendors.
> 
> The most likely types of remapping are the adding of a base offset or
> some extra bits to the ID -- because not all MSIs to the GIC will
> necessarily pass through the IOMMU.  It's also possible that we might
> see ID squashing or folding in some systems.
> 
> 
> For types of remapping which mix the ID and address together, I now
> do tend to agree that any flexibility arising from describing that
> in a general way that is unlikely to repay the cost of trying to
> interpret and analyse the DT.  Defining a few sterotypical kinds
> of mapping explicitly, as needed, looks more sensible for now.  The
> windowed-IOMMU case is one example.

For MSI, my feeling is that we'd just be best off doing an end-to-end
description. Any device using an MSI would have an interrupt specifier
that is sufficient for the interrupt controller to understand where
the IRQ comes from, and return an address/value pair back to the driver
to program into some register.

	Arnd
Dave Martin June 2, 2014, 10:56 a.m. UTC | #7
On Fri, May 30, 2014 at 09:11:07PM +0200, Arnd Bergmann wrote:
> On Friday 30 May 2014 12:27:28 Dave Martin wrote:
> > On Fri, May 30, 2014 at 08:30:08AM +0100, Thierry Reding wrote:
> > > On Thu, May 29, 2014 at 09:52:22AM -0600, Stephen Warren wrote:
> > > > On 05/23/2014 02:36 PM, Thierry Reding wrote:
> > > > I think this is a mistake. address-cells/size-cells are for transactions
> > > > flowing down the bus (from the CPU to date). Describing a connection
> > > > from a device to an IOMMU is something completely different, and should
> > > > therefore simply use an iommu-cells property to describe any necessary
> > > > information. If we start re-using properties for different things in
> > > > different contexts, how is anyone going to know what they mean, and how
> > > > will conflicts be resolved. For example, what if there's a single HW
> > > > module that both acts as a regular register bus with children (where
> > > > address-cells/size-cells defines how transactions reach the children
> > > > from the parent), and is also an IOMMU (where according to this binding
> > > > proposal, address-cells/size-cells represent some aspect of the IOMMU
> > > > feature). Using different properties for different things is the only
> > > > sane way to keep different concepts separate. Another alternative would
> > > > be to represent the single HW module as separate nodes in DT, but I
> > > > think that will only make our lives harder, and where I've done that in
> > > > the past, I've regretted it.
> > > 
> > > There was some back-and-forth on this topic and the latest concensus
> > > when I wrote the second version was that #address-cells and #size-cells
> > > were to be used.
> > > 
> > > But there was some bore back-and-forth after that, and it seems like
> > > Arnd no longer thinks that using #address-cells and #size-cells is a
> > > good idea either[0].
> > 
> > Mistake or not, ePAPR already (ab)uses the address concept for PCI.
> > Unless ePAPR is wrong, I don't think it makes sense to argue that
> > the address concept cannot be repurposed.
> > 
> > The reason why this is abused for PCI is the same as our reason here:
> > different masters really are treated as distinct even when accessing
> > the same destination address, and DT has no general native way to
> > describe that.
> > 
> > One clear advantage of using #address-cells etc. is that ePAPR already
> > has very clear and well-defined ways of how to specify range mappings.
> > This gives us a ready-made way to describe windowed IOMMUs and 1:1
> > remappings of whole blocks of master IDs, which are the most obvious
> > cases other than having a small-integer set of explicit IDs.
> > 
> > That said, the PCI pseudo-address thing is not something we _necessarily_
> > want to repeat.
> 
> I'm really impartial to the question of whether to use #address-cells
> or #iommus now. It's possible that we can use the addresses in some
> meaningful way, but it's not clear to me that this will help make the
> representation cleaner or that we will actually want it for SMMU.
> 
> > > Arnd, can you take another look at this binding and see if there's
> > > anything else missing? If not I'll go through the document again and
> > > update all #address-cells/#size-cells references with #iommu-cells as
> > > appropriate and submit v3.
> > 
> > How do you envisage propagation of the master ID bits downstream of the
> > IOMMU would be described?
> > 
> > We will definitely need a way to describe this for GICv3.  How those
> > values are propagated is likely to vary between related SoCs and doesn't
> > feel like it should be baked into a driver, especially for the ARM SMMU
> > which may get reused in radically different SoC families from different
> > vendors.
> > 
> > The most likely types of remapping are the adding of a base offset or
> > some extra bits to the ID -- because not all MSIs to the GIC will
> > necessarily pass through the IOMMU.  It's also possible that we might
> > see ID squashing or folding in some systems.
> > 
> > 
> > For types of remapping which mix the ID and address together, I now
> > do tend to agree that any flexibility arising from describing that
> > in a general way that is unlikely to repay the cost of trying to
> > interpret and analyse the DT.  Defining a few sterotypical kinds
> > of mapping explicitly, as needed, looks more sensible for now.  The
> > windowed-IOMMU case is one example.
> 
> For MSI, my feeling is that we'd just be best off doing an end-to-end
> description. Any device using an MSI would have an interrupt specifier
> that is sufficient for the interrupt controller to understand where
> the IRQ comes from, and return an address/value pair back to the driver
> to program into some register.

This is OK so long as the number of endpoint pairings is trivial.

If there are multiple GICv3 ITSes addressable by a master, then we
might have to describe each one per master, or impose policy on the
DT whereby we choose one or two arbitrary or preferred ITSes for a
particular master and describe just those.

It's worth nothing that ePAPR strenuously avoids such an approach
for describing DMA: device-device DMA would require information about
every possible destination in every master node.

Of course, device-device DMA turns out not to be a hot topic in
practice most of the time, and likewise with this master ID stuff the
number of pairings that are really relevant in useful situations is
much smaller than the number of possible pairings.

The element of gamble here, and the design of DT around Linux's
present-day limitations makes me feel uneasy, but there's no
knockdown argument that I'm aware of.

_If_ we end up with a combinatorial explosion of pairings in the
future, do you have a solution in mind?

Cheers
---Dave
Dave Martin June 2, 2014, 11:44 a.m. UTC | #8
On Fri, May 30, 2014 at 09:01:19PM +0200, Arnd Bergmann wrote:
> On Friday 30 May 2014 12:22:32 Dave Martin wrote:
> > > +
> > > +Examples:
> > > +=========
> > > +
> > > +Single-master IOMMU:
> > > +--------------------
> > > +
> > > +	iommu {
> > > +		#address-cells = <0>;
> > > +		#size-cells = <0>;
> > > +	};
> > > +
> > > +	master {
> > > +		iommus = <&/iommu>;
> > > +	};
> > > +
> > > +Multiple-master IOMMU with fixed associations:
> > > +----------------------------------------------
> > > +
> > > +	/* multiple-master IOMMU */
> > > +	iommu {
> > > +		/*
> > > +		 * Masters are statically associated with this IOMMU and
> > > +		 * address translation is always enabled.
> > > +		 */
> > > +		#address-cells = <0>;
> > > +		#size-cells = <0>;
> > 
> > In this example, can different translations be set up for the different
> > masters?
> > 
> > With no cells available to contain any sort of ID, it looks like this
> > is not possible.
> 
> Correct, this example is for an IOMMU that does not use IDs but has a
> shared address space for all devices.
> 
> > > +Multiple-master IOMMU with configurable DMA window:
> > > +---------------------------------------------------
> > > +
> > > +	/ {
> > > +		#address-cells = <1>;
> > > +		#size-cells = <1>;
> > > +
> > > +		iommu {
> > > +			/* master ID, address of DMA window */
> > > +			#address-cells = <2>;
> > > +			#size-cells = <2>;
> > > +		};
> > > +
> > > +		master {
> > > +			/* master ID 42, 4 GiB DMA window starting at 0 */
> > > +			iommus = <&/iommu  42 0  0x1 0x0>;
> > 
> > I'm still concerned that in order to deal with future cases we will have
> > to invent multiple ways to parse the "iommus" property.  For example, if
> > we have a PCEe RC mastering through an IOMMU, it will pass a huge set
> > of possible master IDs to the IOMMU, not just noe or two.
> > 
> > Do you have a solution in mind for that which doesn't break backwards
> > compatibility?
> 
> I think we can treat PCI as a special case here and have an interface
> that gets used by the PCI core code to talk to the IOMMU core code
> when setting up a the dma_map_ops for a PCI function. As long as the
> IOMMU driver understands what PCI is, we don't have to describe the
> mapping in detail.

PCI is only an example, but I admit it's likely to be the most important
example of a peripheral using a huge ID space.

There may still be integration-specific parameters which would need to
be fed in via DT regarding how IDs coming out of the RC map onto the
IOMMU and GIC.

> 
> > One option is to include an extra cell to the IOMMUs property
> > that indicates how to parse it.  For now, only a single value would
> > be defined.  For example:
> > 
> > 	iommus = <&/iommu IOMMU_SIMPLE 42>;
> > 
> > Then maybe later
> > 
> > 	iommus = <&/iommu IOMMU_RANGE 0x10000 0x10000>;
> > 
> > (I'm not suggesting what IOMMU_RANGE might mean.)
> > 
> 
> This can really be left up to the specific IOMMU driver itself.
> We can have drivers that support both #address-cells=<1>
> and #address-cells=<2> and behave differently based on that.
> I don't see a reason to define that across IOMMU implementations.

There's a risk we have to come up with multiple solutions to the same
problem there -- i.e., one solution per IOMMU implementation and then
another solution for the GIC.

Ideally that would be avoided, but the number of times this problem
would need to be solved is probably not that large.

Cheers
---Dave
Thierry Reding June 4, 2014, 9:12 p.m. UTC | #9
On Fri, May 30, 2014 at 12:27:28PM +0100, Dave Martin wrote:
> On Fri, May 30, 2014 at 08:30:08AM +0100, Thierry Reding wrote:
[...]
> > Arnd, can you take another look at this binding and see if there's
> > anything else missing? If not I'll go through the document again and
> > update all #address-cells/#size-cells references with #iommu-cells as
> > appropriate and submit v3.
> 
> How do you envisage propagation of the master ID bits downstream of the
> IOMMU would be described?
> 
> We will definitely need a way to describe this for GICv3.  How those
> values are propagated is likely to vary between related SoCs and doesn't
> feel like it should be baked into a driver, especially for the ARM SMMU
> which may get reused in radically different SoC families from different
> vendors.

Well, we've had cases like these in the past (power sequences come to
mind). Some concepts are just too difficult or unwieldy to be put into
device tree. I think that this is one of them.

> The most likely types of remapping are the adding of a base offset or
> some extra bits to the ID -- because not all MSIs to the GIC will
> necessarily pass through the IOMMU.  It's also possible that we might
> see ID squashing or folding in some systems.

It can easily be argued that if the algorithm used to remap the ID
varies, the compatibility of the device changes. Therefore I would
expect any variant of the GICv3 that deviates from the "standard"
mapping (if there is such a thing) to have its own compatible string.

At that point there are two possibilities: a) handle the remapping in
code hooked into the driver via matching of the compatible string or
b) specifying a different specifier for each of these and keep the
description in DT. From an implementation point of view, variant b) is
likely going to be very similar to a), though (I'm thinking .of_xlate
type of functions here).

Thierry
Thierry Reding June 4, 2014, 9:32 p.m. UTC | #10
On Fri, May 30, 2014 at 09:01:19PM +0200, Arnd Bergmann wrote:
> On Friday 30 May 2014 12:22:32 Dave Martin wrote:
> > > +
> > > +Examples:
> > > +=========
> > > +
> > > +Single-master IOMMU:
> > > +--------------------
> > > +
> > > +	iommu {
> > > +		#address-cells = <0>;
> > > +		#size-cells = <0>;
> > > +	};
> > > +
> > > +	master {
> > > +		iommus = <&/iommu>;
> > > +	};
> > > +
> > > +Multiple-master IOMMU with fixed associations:
> > > +----------------------------------------------
> > > +
> > > +	/* multiple-master IOMMU */
> > > +	iommu {
> > > +		/*
> > > +		 * Masters are statically associated with this IOMMU and
> > > +		 * address translation is always enabled.
> > > +		 */
> > > +		#address-cells = <0>;
> > > +		#size-cells = <0>;
> > 
> > In this example, can different translations be set up for the different
> > masters?
> > 
> > With no cells available to contain any sort of ID, it looks like this
> > is not possible.
> 
> Correct, this example is for an IOMMU that does not use IDs but has a
> shared address space for all devices.

Couldn't these device all still have separate address spaces?

Thierry
Arnd Bergmann June 5, 2014, 9:42 a.m. UTC | #11
On Wednesday 04 June 2014 23:32:00 Thierry Reding wrote:
> On Fri, May 30, 2014 at 09:01:19PM +0200, Arnd Bergmann wrote:
> > On Friday 30 May 2014 12:22:32 Dave Martin wrote:
> > > > +
> > > > +Examples:
> > > > +=========
> > > > +
> > > > +Single-master IOMMU:
> > > > +--------------------
> > > > +
> > > > + iommu {
> > > > +         #address-cells = <0>;
> > > > +         #size-cells = <0>;
> > > > + };
> > > > +
> > > > + master {
> > > > +         iommus = <&/iommu>;
> > > > + };
> > > > +
> > > > +Multiple-master IOMMU with fixed associations:
> > > > +----------------------------------------------
> > > > +
> > > > + /* multiple-master IOMMU */
> > > > + iommu {
> > > > +         /*
> > > > +          * Masters are statically associated with this IOMMU and
> > > > +          * address translation is always enabled.
> > > > +          */
> > > > +         #address-cells = <0>;
> > > > +         #size-cells = <0>;
> > > 
> > > In this example, can different translations be set up for the different
> > > masters?
> > > 
> > > With no cells available to contain any sort of ID, it looks like this
> > > is not possible.
> > 
> > Correct, this example is for an IOMMU that does not use IDs but has a
> > shared address space for all devices.
> 
> Couldn't these device all still have separate address spaces?

No. If they had separate address spaces, they would require a more
sophisticated IOMMU. A simple IOMMU without IDs can only be used
for overcoming address space limits (e.g. for 32-bit DMA masters on
systems with more than 4GB RAM) but not for strict isolation.

You basically have one page table shared across all devices connected
to the IOMMU, and every call to dma_alloc_coherent or dma_map_*
allocates a new IOVA that isn't used by any of the other devices
already, but you can't prevent a malicious user from getting a device
to do DMA to an IOVA that has been set up for another device.

You could have one such IOMMU per device of course, but I guess that's
not what you mean.

	Arnd
Will Deacon June 16, 2014, 12:57 p.m. UTC | #12
On Wed, Jun 04, 2014 at 10:12:38PM +0100, Thierry Reding wrote:
> On Fri, May 30, 2014 at 12:27:28PM +0100, Dave Martin wrote:
> > On Fri, May 30, 2014 at 08:30:08AM +0100, Thierry Reding wrote:
> [...]
> > > Arnd, can you take another look at this binding and see if there's
> > > anything else missing? If not I'll go through the document again and
> > > update all #address-cells/#size-cells references with #iommu-cells as
> > > appropriate and submit v3.
> > 
> > How do you envisage propagation of the master ID bits downstream of the
> > IOMMU would be described?
> > 
> > We will definitely need a way to describe this for GICv3.  How those
> > values are propagated is likely to vary between related SoCs and doesn't
> > feel like it should be baked into a driver, especially for the ARM SMMU
> > which may get reused in radically different SoC families from different
> > vendors.
> 
> Well, we've had cases like these in the past (power sequences come to
> mind). Some concepts are just too difficult or unwieldy to be put into
> device tree. I think that this is one of them.
> 
> > The most likely types of remapping are the adding of a base offset or
> > some extra bits to the ID -- because not all MSIs to the GIC will
> > necessarily pass through the IOMMU.  It's also possible that we might
> > see ID squashing or folding in some systems.
> 
> It can easily be argued that if the algorithm used to remap the ID
> varies, the compatibility of the device changes. Therefore I would
> expect any variant of the GICv3 that deviates from the "standard"
> mapping (if there is such a thing) to have its own compatible string.

There is no standard mapping; it's a property defined at system integration
time. I fully expect different SoCs to do different things here.

Will
Thierry Reding June 17, 2014, 11:58 a.m. UTC | #13
On Mon, Jun 16, 2014 at 01:57:04PM +0100, Will Deacon wrote:
> On Wed, Jun 04, 2014 at 10:12:38PM +0100, Thierry Reding wrote:
> > On Fri, May 30, 2014 at 12:27:28PM +0100, Dave Martin wrote:
> > > On Fri, May 30, 2014 at 08:30:08AM +0100, Thierry Reding wrote:
> > [...]
> > > > Arnd, can you take another look at this binding and see if there's
> > > > anything else missing? If not I'll go through the document again and
> > > > update all #address-cells/#size-cells references with #iommu-cells as
> > > > appropriate and submit v3.
> > > 
> > > How do you envisage propagation of the master ID bits downstream of the
> > > IOMMU would be described?
> > > 
> > > We will definitely need a way to describe this for GICv3.  How those
> > > values are propagated is likely to vary between related SoCs and doesn't
> > > feel like it should be baked into a driver, especially for the ARM SMMU
> > > which may get reused in radically different SoC families from different
> > > vendors.
> > 
> > Well, we've had cases like these in the past (power sequences come to
> > mind). Some concepts are just too difficult or unwieldy to be put into
> > device tree. I think that this is one of them.
> > 
> > > The most likely types of remapping are the adding of a base offset or
> > > some extra bits to the ID -- because not all MSIs to the GIC will
> > > necessarily pass through the IOMMU.  It's also possible that we might
> > > see ID squashing or folding in some systems.
> > 
> > It can easily be argued that if the algorithm used to remap the ID
> > varies, the compatibility of the device changes. Therefore I would
> > expect any variant of the GICv3 that deviates from the "standard"
> > mapping (if there is such a thing) to have its own compatible string.
> 
> There is no standard mapping; it's a property defined at system integration
> time. I fully expect different SoCs to do different things here.

My point was that the mapping itself seems to be fundamental enough to
make devices with different mappings "incompatible". Therefore I think
this could probably be handled by using different compatible values,
something along the lines of this:

	compatible = "vendor,soc-gicv3", "arm,gicv3";

Then the mapping can be described in code, which should be a whole lot
easier and more flexible than a more or less generic notation in device
tree.

Thierry
Will Deacon June 17, 2014, 12:18 p.m. UTC | #14
On Tue, Jun 17, 2014 at 12:58:30PM +0100, Thierry Reding wrote:
> On Mon, Jun 16, 2014 at 01:57:04PM +0100, Will Deacon wrote:
> > On Wed, Jun 04, 2014 at 10:12:38PM +0100, Thierry Reding wrote:
> > > It can easily be argued that if the algorithm used to remap the ID
> > > varies, the compatibility of the device changes. Therefore I would
> > > expect any variant of the GICv3 that deviates from the "standard"
> > > mapping (if there is such a thing) to have its own compatible string.
> > 
> > There is no standard mapping; it's a property defined at system integration
> > time. I fully expect different SoCs to do different things here.
> 
> My point was that the mapping itself seems to be fundamental enough to
> make devices with different mappings "incompatible". Therefore I think
> this could probably be handled by using different compatible values,
> something along the lines of this:
> 
> 	compatible = "vendor,soc-gicv3", "arm,gicv3";
> 
> Then the mapping can be described in code, which should be a whole lot
> easier and more flexible than a more or less generic notation in device
> tree.

I don't think that scales well beyond a handful of unique mappings, and I
really anticipate everybody doing something different based on their
integration constraints.

You'd very quickly end up with sets of tables for each SoC, describing the
topology and associated IDs in the kernel source, which feels like a giant
step backwards from where we are today with device tree.

If, for example, the GIC architecture prescribed a fixed set of Device IDs
and an algorithm for converting from Stream IDs then your approach may have
some merits, but that's not where we are today (and I also don't think it's
practical to try and enforce such system-wide properties into an interrupt
controller architecture).

Will
Thierry Reding June 17, 2014, 11:37 p.m. UTC | #15
On Tue, Jun 17, 2014 at 01:18:11PM +0100, Will Deacon wrote:
> On Tue, Jun 17, 2014 at 12:58:30PM +0100, Thierry Reding wrote:
> > On Mon, Jun 16, 2014 at 01:57:04PM +0100, Will Deacon wrote:
> > > On Wed, Jun 04, 2014 at 10:12:38PM +0100, Thierry Reding wrote:
> > > > It can easily be argued that if the algorithm used to remap the ID
> > > > varies, the compatibility of the device changes. Therefore I would
> > > > expect any variant of the GICv3 that deviates from the "standard"
> > > > mapping (if there is such a thing) to have its own compatible string.
> > > 
> > > There is no standard mapping; it's a property defined at system integration
> > > time. I fully expect different SoCs to do different things here.
> > 
> > My point was that the mapping itself seems to be fundamental enough to
> > make devices with different mappings "incompatible". Therefore I think
> > this could probably be handled by using different compatible values,
> > something along the lines of this:
> > 
> > 	compatible = "vendor,soc-gicv3", "arm,gicv3";
> > 
> > Then the mapping can be described in code, which should be a whole lot
> > easier and more flexible than a more or less generic notation in device
> > tree.
> 
> I don't think that scales well beyond a handful of unique mappings, and I
> really anticipate everybody doing something different based on their
> integration constraints.
> 
> You'd very quickly end up with sets of tables for each SoC, describing the
> topology and associated IDs in the kernel source, which feels like a giant
> step backwards from where we are today with device tree.

Well, today we don't have a generic binding at all, so anything will
really be a giant step forward in my opinion.

But seriously, from what you said earlier I got the impression that some
of the mappings may not be easy or possible to represent in DT, which is
why I proposed to encode it into the compatible property so that it can
be handled in code instead.

We've had similar discussions before (power sequences anyone?) where we
tried to come up with a generic way to describe something in device tree
that just didn't work out too well. Some things are better done in code,
so I think we should at least consider that possibility rather than
blindly try and force everything into device tree.

Thierry
Will Deacon June 18, 2014, 10:14 a.m. UTC | #16
On Wed, Jun 18, 2014 at 12:37:16AM +0100, Thierry Reding wrote:
> On Tue, Jun 17, 2014 at 01:18:11PM +0100, Will Deacon wrote:
> > On Tue, Jun 17, 2014 at 12:58:30PM +0100, Thierry Reding wrote:
> > > On Mon, Jun 16, 2014 at 01:57:04PM +0100, Will Deacon wrote:
> > > > On Wed, Jun 04, 2014 at 10:12:38PM +0100, Thierry Reding wrote:
> > > > > It can easily be argued that if the algorithm used to remap the ID
> > > > > varies, the compatibility of the device changes. Therefore I would
> > > > > expect any variant of the GICv3 that deviates from the "standard"
> > > > > mapping (if there is such a thing) to have its own compatible string.
> > > > 
> > > > There is no standard mapping; it's a property defined at system integration
> > > > time. I fully expect different SoCs to do different things here.
> > > 
> > > My point was that the mapping itself seems to be fundamental enough to
> > > make devices with different mappings "incompatible". Therefore I think
> > > this could probably be handled by using different compatible values,
> > > something along the lines of this:
> > > 
> > > 	compatible = "vendor,soc-gicv3", "arm,gicv3";
> > > 
> > > Then the mapping can be described in code, which should be a whole lot
> > > easier and more flexible than a more or less generic notation in device
> > > tree.
> > 
> > I don't think that scales well beyond a handful of unique mappings, and I
> > really anticipate everybody doing something different based on their
> > integration constraints.
> > 
> > You'd very quickly end up with sets of tables for each SoC, describing the
> > topology and associated IDs in the kernel source, which feels like a giant
> > step backwards from where we are today with device tree.
> 
> Well, today we don't have a generic binding at all, so anything will
> really be a giant step forward in my opinion.

Oh, I'm not disputing that at all. I just think it's worth considering how
we can extend the binding in future to describe some of the ID routing and
remapping that we've discussed.

> But seriously, from what you said earlier I got the impression that some
> of the mappings may not be easy or possible to represent in DT, which is
> why I proposed to encode it into the compatible property so that it can
> be handled in code instead.

For any old arbitrary mappings, we're going to have a rough time, but that
doesn't mean we need to support that until we're forced to (and then we can
consider our options). What I *do* think we need to describe is:

  - Each master has a set of fixed StreamIDs
  - StreamIDs can be remastered by adding a constant offset (this could also
    be used to describe RequesterID -> StreamID mapping)

I'd hope this would be sufficient for most people. Dynamic ID assignment can
be worked out later (I'm not even sure it belongs in this binding) and any
mappings other than `add a constant offset' can be treated on a case-by-case
basis. We don't want to throw the kitchen sink at a language for describing
arbitrary transformations!

> We've had similar discussions before (power sequences anyone?) where we
> tried to come up with a generic way to describe something in device tree
> that just didn't work out too well. Some things are better done in code,
> so I think we should at least consider that possibility rather than
> blindly try and force everything into device tree.

If we can support 90% of SoCs with a simple DT-based description, we can
address the corner cases as they arise. I'm not ruling our hardcoding
topology if we have no choice, but I don't think that's a healthy place to
start from.

Will
Arnd Bergmann June 20, 2014, 3:53 p.m. UTC | #17
On Wednesday 18 June 2014 11:14:39 Will Deacon wrote:
> On Wed, Jun 18, 2014 at 12:37:16AM +0100, Thierry Reding wrote:
>   - Each master has a set of fixed StreamIDs
>   - StreamIDs can be remastered by adding a constant offset (this could also
>     be used to describe RequesterID -> StreamID mapping)
> 
> I'd hope this would be sufficient for most people. Dynamic ID assignment can
> be worked out later (I'm not even sure it belongs in this binding) and any
> mappings other than `add a constant offset' can be treated on a case-by-case
> basis. We don't want to throw the kitchen sink at a language for describing
> arbitrary transformations!
> 
> > We've had similar discussions before (power sequences anyone?) where we
> > tried to come up with a generic way to describe something in device tree
> > that just didn't work out too well. Some things are better done in code,
> > so I think we should at least consider that possibility rather than
> > blindly try and force everything into device tree.
> 
> If we can support 90% of SoCs with a simple DT-based description, we can
> address the corner cases as they arise. I'm not ruling our hardcoding
> topology if we have no choice, but I don't think that's a healthy place to
> start from.

So we could use the "arm,gicv3" comaptible string for all those that
have a relatively simple mapping, and describe that mapping entirely
in DT properties, but use a different compatible string for those
SoCs that have a mapping which we can't easily describe, and then
put that into code?

	Arnd
Will Deacon June 20, 2014, 5:50 p.m. UTC | #18
On Fri, Jun 20, 2014 at 04:53:08PM +0100, Arnd Bergmann wrote:
> On Wednesday 18 June 2014 11:14:39 Will Deacon wrote:
> > On Wed, Jun 18, 2014 at 12:37:16AM +0100, Thierry Reding wrote:
> >   - Each master has a set of fixed StreamIDs
> >   - StreamIDs can be remastered by adding a constant offset (this could also
> >     be used to describe RequesterID -> StreamID mapping)
> > 
> > I'd hope this would be sufficient for most people. Dynamic ID assignment can
> > be worked out later (I'm not even sure it belongs in this binding) and any
> > mappings other than `add a constant offset' can be treated on a case-by-case
> > basis. We don't want to throw the kitchen sink at a language for describing
> > arbitrary transformations!
> > 
> > > We've had similar discussions before (power sequences anyone?) where we
> > > tried to come up with a generic way to describe something in device tree
> > > that just didn't work out too well. Some things are better done in code,
> > > so I think we should at least consider that possibility rather than
> > > blindly try and force everything into device tree.
> > 
> > If we can support 90% of SoCs with a simple DT-based description, we can
> > address the corner cases as they arise. I'm not ruling our hardcoding
> > topology if we have no choice, but I don't think that's a healthy place to
> > start from.
> 
> So we could use the "arm,gicv3" comaptible string for all those that
> have a relatively simple mapping, and describe that mapping entirely
> in DT properties, but use a different compatible string for those
> SoCs that have a mapping which we can't easily describe, and then
> put that into code?

That doesn't sound unreasonable, but I don't think we should commit to
putting things into code until they come along and we can't describe them.

Will
Arnd Bergmann June 20, 2014, 6:55 p.m. UTC | #19
On Friday 20 June 2014 18:50:51 Will Deacon wrote:
> On Fri, Jun 20, 2014 at 04:53:08PM +0100, Arnd Bergmann wrote:
> > On Wednesday 18 June 2014 11:14:39 Will Deacon wrote:
> > > On Wed, Jun 18, 2014 at 12:37:16AM +0100, Thierry Reding wrote:
> > >   - Each master has a set of fixed StreamIDs
> > >   - StreamIDs can be remastered by adding a constant offset (this could also
> > >     be used to describe RequesterID -> StreamID mapping)
> > > 
> > > I'd hope this would be sufficient for most people. Dynamic ID assignment can
> > > be worked out later (I'm not even sure it belongs in this binding) and any
> > > mappings other than `add a constant offset' can be treated on a case-by-case
> > > basis. We don't want to throw the kitchen sink at a language for describing
> > > arbitrary transformations!
> > > 
> > > > We've had similar discussions before (power sequences anyone?) where we
> > > > tried to come up with a generic way to describe something in device tree
> > > > that just didn't work out too well. Some things are better done in code,
> > > > so I think we should at least consider that possibility rather than
> > > > blindly try and force everything into device tree.
> > > 
> > > If we can support 90% of SoCs with a simple DT-based description, we can
> > > address the corner cases as they arise. I'm not ruling our hardcoding
> > > topology if we have no choice, but I don't think that's a healthy place to
> > > start from.
> > 
> > So we could use the "arm,gicv3" comaptible string for all those that
> > have a relatively simple mapping, and describe that mapping entirely
> > in DT properties, but use a different compatible string for those
> > SoCs that have a mapping which we can't easily describe, and then
> > put that into code?
> 
> That doesn't sound unreasonable, but I don't think we should commit to
> putting things into code until they come along and we can't describe them.

Yes, that makes sense. But it's good to know that we have the option so
we don't have to cover all corner cases with the binding.

	Arnd
diff mbox

Patch

diff --git a/Documentation/devicetree/bindings/iommu/iommu.txt b/Documentation/devicetree/bindings/iommu/iommu.txt
new file mode 100644
index 000000000000..6ce759afcc94
--- /dev/null
+++ b/Documentation/devicetree/bindings/iommu/iommu.txt
@@ -0,0 +1,167 @@ 
+This document describes the generic device tree binding for IOMMUs and their
+master(s).
+
+
+IOMMU device node:
+==================
+
+An IOMMU can provide the following services:
+
+* Remap address space to allow devices to access physical memory ranges that
+  they otherwise wouldn't be capable of accessing.
+
+  Example: 32-bit DMA to 64-bit physical addresses
+
+* Implement scatter-gather at page level granularity so that the device does
+  not have to.
+
+* Provide system protection against "rogue" DMA by forcing all accesses to go
+  through the IOMMU and faulting when encountering accesses to unmapped
+  address regions.
+
+* Provide address space isolation between multiple contexts.
+
+  Example: Virtualization
+
+Device nodes compatible with this binding represent hardware with some of the
+above capabilities.
+
+IOMMUs can be single-master or multiple-master. Single-master IOMMU devices
+typically have a fixed association to the master device, whereas multiple-
+master IOMMU devices can translate accesses from more than one master.
+
+The device tree node of the IOMMU device's parent bus must contain a valid
+"dma-ranges" property that describes how the physical address space of the
+IOMMU maps to memory. An empty "dma-ranges" property means that there is a
+1:1 mapping from IOMMU to memory.
+
+Required properties:
+--------------------
+- #address-cells: The number of cells in an IOMMU specifier needed to encode
+  an address.
+- #size-cells: The number of cells in an IOMMU specifier needed to represent
+  the length of an address range.
+
+Typical values for the above include:
+- #address-cells = <0>, size-cells = <0>: Single master IOMMU devices are not
+  configurable and therefore no additional information needs to be encoded in
+  the specifier. This may also apply to multiple master IOMMU devices that do
+  not allow the association of masters to be configured.
+- #address-cells = <1>, size-cells = <0>: Multiple master IOMMU devices may
+  need to be configured in order to enable translation for a given master. In
+  such cases the single address cell corresponds to the master device's ID.
+- #address-cells = <2>, size-cells = <2>: Some IOMMU devices allow the DMA
+  window for masters to be configured. The first cell of the address in this
+  may contain the master device's ID for example, while the second cell could
+  contain the start of the DMA window for the given device. The length of the
+  DMA window is specified by two additional cells.
+
+
+IOMMU master node:
+==================
+
+Devices that access memory through an IOMMU are called masters. A device can
+have multiple master interfaces (to one or more IOMMU devices).
+
+Required properties:
+--------------------
+- iommus: A list of phandle and IOMMU specifier pairs that describe the IOMMU
+  master interfaces of the device. One entry in the list describes one master
+  interface of the device.
+
+When an "iommus" property is specified in a device tree node, the IOMMU will
+be used for address translation. If a "dma-ranges" property exists in the
+device's parent node it will be ignored. An exception to this rule is if the
+referenced IOMMU is disabled, in which case the "dma-ranges" property of the
+parent shall take effect.
+
+Optional properties:
+--------------------
+- iommu-names: A list of names identifying each entry in the "iommus"
+  property.
+
+
+Notes:
+======
+
+One possible extension to the above is to use an "iommus" property along with
+a "dma-ranges" property in a bus device node (such as PCI host bridges). This
+can be useful to describe how children on the bus relate to the IOMMU if they
+are not explicitly listed in the device tree (e.g. PCI devices). However, the
+requirements of that use-case haven't been fully determined yet. Implementing
+this is therefore not recommended without further discussion and extension of
+this binding.
+
+
+Examples:
+=========
+
+Single-master IOMMU:
+--------------------
+
+	iommu {
+		#address-cells = <0>;
+		#size-cells = <0>;
+	};
+
+	master {
+		iommus = <&/iommu>;
+	};
+
+Multiple-master IOMMU with fixed associations:
+----------------------------------------------
+
+	/* multiple-master IOMMU */
+	iommu {
+		/*
+		 * Masters are statically associated with this IOMMU and
+		 * address translation is always enabled.
+		 */
+		#address-cells = <0>;
+		#size-cells = <0>;
+	};
+
+	/* static association with IOMMU */
+	master@1 {
+		reg = <1>;
+		iommus = <&/iommu>;
+	};
+
+	/* static association with IOMMU */
+	master@2 {
+		reg = <2>;
+		iommus = <&/iommu>;
+	};
+
+Multiple-master IOMMU:
+----------------------
+
+	iommu {
+		/* the specifier represents the ID of the master */
+		#address-cells = <1>;
+		#size-cells = <0>;
+	};
+
+	master {
+		/* device has master ID 42 in the IOMMU */
+		iommus = <&/iommu 42>;
+	};
+
+Multiple-master IOMMU with configurable DMA window:
+---------------------------------------------------
+
+	/ {
+		#address-cells = <1>;
+		#size-cells = <1>;
+
+		iommu {
+			/* master ID, address of DMA window */
+			#address-cells = <2>;
+			#size-cells = <2>;
+		};
+
+		master {
+			/* master ID 42, 4 GiB DMA window starting at 0 */
+			iommus = <&/iommu  42 0  0x1 0x0>;
+		};
+	};