[v8,8/8] xen/arm: add dom0-less device assignment info to docs
diff mbox series

Message ID 20191003013526.30768-8-sstabellini@kernel.org
State New
Headers show
Series
  • [v8,1/8] xen/arm: introduce handle_device_interrupts
Related show

Commit Message

Stefano Stabellini Oct. 3, 2019, 1:35 a.m. UTC
Add info about the SPI used for the virtual pl011.

Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>

---
Changes in v8:
- remove sentence about xen,path being optional

Changes in v7:
- add xen,force-assign-without-iommu
- clarify xen,reg and xen,path go together
- remove acked-by due to changes

Changes in v6:
- fix nr_spis description
- add ack

Changes in v5:
- improve wording

Changes in v4:
- fix spelling
- add "multiboot,module"
- improve commit message
- improve doc
- expand the nr_spis and vpl011 sections and include information about
  the vpl011 SPI
- move passthrough information to docs/misc/arm/passthrough.txt

Changes in v3:
- add nr_spis
- change description of interrupts and interrupt-parent

Changes in v2:
- device tree fragment loaded in cacheable memory
- rename multiboot,dtb to multiboot,device-tree
- rename "path" to "xen,path"
- add a note about device memory mapping
- introduce xen,reg
- specify only the GIC is supported
---
 docs/misc/arm/device-tree/booting.txt |  44 ++++++++++-
 docs/misc/arm/passthrough.txt         | 106 ++++++++++++++++++++++++++
 2 files changed, 149 insertions(+), 1 deletion(-)

Comments

Julien Grall Oct. 3, 2019, 9:36 a.m. UTC | #1
Hi Stefano,

On 03/10/2019 02:35, Stefano Stabellini wrote:
> Add info about the SPI used for the virtual pl011.
> 
> Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>

Acked-by: Julien Grall <julien.grall@arm.com>

Cheers,
Julien Grall Oct. 3, 2019, 9:58 a.m. UTC | #2
Hi Stefano,

On 03/10/2019 02:35, Stefano Stabellini wrote:
> Add info about the SPI used for the virtual pl011.
> 
> Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
> 
> ---
> Changes in v8:
> - remove sentence about xen,path being optional
> 
> Changes in v7:
> - add xen,force-assign-without-iommu
> - clarify xen,reg and xen,path go together
> - remove acked-by due to changes
> 
> Changes in v6:
> - fix nr_spis description
> - add ack
> 
> Changes in v5:
> - improve wording
> 
> Changes in v4:
> - fix spelling
> - add "multiboot,module"
> - improve commit message
> - improve doc
> - expand the nr_spis and vpl011 sections and include information about
>    the vpl011 SPI
> - move passthrough information to docs/misc/arm/passthrough.txt
> 
> Changes in v3:
> - add nr_spis
> - change description of interrupts and interrupt-parent
> 
> Changes in v2:
> - device tree fragment loaded in cacheable memory
> - rename multiboot,dtb to multiboot,device-tree
> - rename "path" to "xen,path"
> - add a note about device memory mapping
> - introduce xen,reg
> - specify only the GIC is supported
> ---
>   docs/misc/arm/device-tree/booting.txt |  44 ++++++++++-
>   docs/misc/arm/passthrough.txt         | 106 ++++++++++++++++++++++++++
>   2 files changed, 149 insertions(+), 1 deletion(-)
> 
> diff --git a/docs/misc/arm/device-tree/booting.txt b/docs/misc/arm/device-tree/booting.txt
> index 317a9e962a..649e00d09f 100644
> --- a/docs/misc/arm/device-tree/booting.txt
> +++ b/docs/misc/arm/device-tree/booting.txt
> @@ -146,7 +146,18 @@ with the following properties:
>   
>   - vpl011
>   
> -    An empty property to enable/disable a virtual pl011 for the guest to use.
> +    An empty property to enable/disable a virtual pl011 for the guest to
> +    use. The virtual pl011 uses SPI number 0 (see GUEST_VPL011_SPI).
> +    Please note that the SPI used for the virtual pl011 could clash with the
> +    physical SPI of a physical device assigned to the guest.
> +
> +- nr_spis
> +
> +    Optional. A 32-bit integer specifying the number of SPIs (Shared
> +    Peripheral Interrupts) to allocate for the domain. If nr_spis is
> +    missing, the max number of SPIs supported by the physical GIC is
> +    used, or GUEST_VPL011_SPI+1 if vpl011 is enabled, whichever is
> +    greater.
>   
>   - #address-cells and #size-cells
>   
> @@ -226,3 +237,34 @@ chosen {
>           };
>       };
>   };
> +
> +
> +Device Assignment
> +=================
> +
> +Device Assignment (Passthrough) is supported by adding another module,
> +alongside the kernel and ramdisk, with the device tree fragment
> +corresponding to the device node to assign to the guest.
> +
> +The dtb sub-node should have the following properties:
> +
> +- compatible
> +
> +    "multiboot,device-tree" and "multiboot,module"
> +
> +- reg
> +
> +    Specifies the physical address of the device tree binary fragment
> +    RAM and its length.
> +
> +As an example:
> +
> +        module@0xc000000 {
> +            compatible = "multiboot,device-tree", "multiboot,module";
> +            reg = <0x0 0xc000000 0xffffff>;
> +        };
> +
> +The DTB fragment is loaded at 0xc000000 in the example above. It should
> +follow the convention explained in docs/misc/arm/passthrough.txt. The
> +DTB fragment will be added to the guest device tree, so that the guest
> +kernel will be able to discover the device.
> diff --git a/docs/misc/arm/passthrough.txt b/docs/misc/arm/passthrough.txt
> index 0efbd122de..6826e1f341 100644
> --- a/docs/misc/arm/passthrough.txt
> +++ b/docs/misc/arm/passthrough.txt
> @@ -80,6 +80,112 @@ SPI numbers start from 32, in this example 80 + 32 = 112.
>   See man [xl.cfg] for the iomem format. The reg property is just a pair
>   of address, then size numbers, each of them can occupy 1 or 2 cells.
>   
> +
> +Dom0-less Device Passthrough
> +============================
> +
> +The partial device tree for dom0-less guests should have the following
> +properties for each node corresponding to a physical device to assign to
> +the guest:
> +
> +- xen,reg
> +
> +  The xen,reg property is an array of:
> +
> +    <phys_addr size guest_addr>
> +
> +  They specify the physical address and size of the device memory
> +  ranges together with the corresponding guest address to map them to.
> +  The size of `phys_addr' and `guest_addr' is determined by
> +  #address-cells, the size of `size' is determined by #size-cells, of
> +  the partial device tree.
> +  The memory will be mapped as device memory in the guest (Device-nGnRE).
> +
> +- xen,path
> +
> +  A string property representing the path in the host device tree to the
> +  corresponding device node.
> +
> +- xen,force-assign-without-iommu
> +  If xen,force-assign-without-iommu is present Xen continues booting
> +  even on IOMMU setup errors for the device (i.e. the device is not
> +  protected by an IOMMU).

Reading again, this suggest the option can be used to force assignment if a 
Device is behind an IOMMU but the setup failed.

All IOMMUs should be configured to deny any transaction. So your device is not 
going to work. We should also probably state the consequence of using this option.

It would be better to say:

"If xen,force-assign-without-iommu is present, Xen will allow to assign a device 
even if it is not behind an IOMMU. This will render your platform *unsafe* if 
the device is DMA-capable".

Cheers,
Stefano Stabellini Oct. 3, 2019, 5:36 p.m. UTC | #3
On Thu, 3 Oct 2019, Julien Grall wrote:
> Hi Stefano,
> 
> On 03/10/2019 02:35, Stefano Stabellini wrote:
> > Add info about the SPI used for the virtual pl011.
> > 
> > Signed-off-by: Stefano Stabellini <stefanos@xilinx.com>
> > 
> > ---
> > Changes in v8:
> > - remove sentence about xen,path being optional
> > 
> > Changes in v7:
> > - add xen,force-assign-without-iommu
> > - clarify xen,reg and xen,path go together
> > - remove acked-by due to changes
> > 
> > Changes in v6:
> > - fix nr_spis description
> > - add ack
> > 
> > Changes in v5:
> > - improve wording
> > 
> > Changes in v4:
> > - fix spelling
> > - add "multiboot,module"
> > - improve commit message
> > - improve doc
> > - expand the nr_spis and vpl011 sections and include information about
> >    the vpl011 SPI
> > - move passthrough information to docs/misc/arm/passthrough.txt
> > 
> > Changes in v3:
> > - add nr_spis
> > - change description of interrupts and interrupt-parent
> > 
> > Changes in v2:
> > - device tree fragment loaded in cacheable memory
> > - rename multiboot,dtb to multiboot,device-tree
> > - rename "path" to "xen,path"
> > - add a note about device memory mapping
> > - introduce xen,reg
> > - specify only the GIC is supported
> > ---
> >   docs/misc/arm/device-tree/booting.txt |  44 ++++++++++-
> >   docs/misc/arm/passthrough.txt         | 106 ++++++++++++++++++++++++++
> >   2 files changed, 149 insertions(+), 1 deletion(-)
> > 
> > diff --git a/docs/misc/arm/device-tree/booting.txt
> > b/docs/misc/arm/device-tree/booting.txt
> > index 317a9e962a..649e00d09f 100644
> > --- a/docs/misc/arm/device-tree/booting.txt
> > +++ b/docs/misc/arm/device-tree/booting.txt
> > @@ -146,7 +146,18 @@ with the following properties:
> >     - vpl011
> >   -    An empty property to enable/disable a virtual pl011 for the guest to
> > use.
> > +    An empty property to enable/disable a virtual pl011 for the guest to
> > +    use. The virtual pl011 uses SPI number 0 (see GUEST_VPL011_SPI).
> > +    Please note that the SPI used for the virtual pl011 could clash with
> > the
> > +    physical SPI of a physical device assigned to the guest.
> > +
> > +- nr_spis
> > +
> > +    Optional. A 32-bit integer specifying the number of SPIs (Shared
> > +    Peripheral Interrupts) to allocate for the domain. If nr_spis is
> > +    missing, the max number of SPIs supported by the physical GIC is
> > +    used, or GUEST_VPL011_SPI+1 if vpl011 is enabled, whichever is
> > +    greater.
> >     - #address-cells and #size-cells
> >   @@ -226,3 +237,34 @@ chosen {
> >           };
> >       };
> >   };
> > +
> > +
> > +Device Assignment
> > +=================
> > +
> > +Device Assignment (Passthrough) is supported by adding another module,
> > +alongside the kernel and ramdisk, with the device tree fragment
> > +corresponding to the device node to assign to the guest.
> > +
> > +The dtb sub-node should have the following properties:
> > +
> > +- compatible
> > +
> > +    "multiboot,device-tree" and "multiboot,module"
> > +
> > +- reg
> > +
> > +    Specifies the physical address of the device tree binary fragment
> > +    RAM and its length.
> > +
> > +As an example:
> > +
> > +        module@0xc000000 {
> > +            compatible = "multiboot,device-tree", "multiboot,module";
> > +            reg = <0x0 0xc000000 0xffffff>;
> > +        };
> > +
> > +The DTB fragment is loaded at 0xc000000 in the example above. It should
> > +follow the convention explained in docs/misc/arm/passthrough.txt. The
> > +DTB fragment will be added to the guest device tree, so that the guest
> > +kernel will be able to discover the device.
> > diff --git a/docs/misc/arm/passthrough.txt b/docs/misc/arm/passthrough.txt
> > index 0efbd122de..6826e1f341 100644
> > --- a/docs/misc/arm/passthrough.txt
> > +++ b/docs/misc/arm/passthrough.txt
> > @@ -80,6 +80,112 @@ SPI numbers start from 32, in this example 80 + 32 =
> > 112.
> >   See man [xl.cfg] for the iomem format. The reg property is just a pair
> >   of address, then size numbers, each of them can occupy 1 or 2 cells.
> >   +
> > +Dom0-less Device Passthrough
> > +============================
> > +
> > +The partial device tree for dom0-less guests should have the following
> > +properties for each node corresponding to a physical device to assign to
> > +the guest:
> > +
> > +- xen,reg
> > +
> > +  The xen,reg property is an array of:
> > +
> > +    <phys_addr size guest_addr>
> > +
> > +  They specify the physical address and size of the device memory
> > +  ranges together with the corresponding guest address to map them to.
> > +  The size of `phys_addr' and `guest_addr' is determined by
> > +  #address-cells, the size of `size' is determined by #size-cells, of
> > +  the partial device tree.
> > +  The memory will be mapped as device memory in the guest (Device-nGnRE).
> > +
> > +- xen,path
> > +
> > +  A string property representing the path in the host device tree to the
> > +  corresponding device node.
> > +
> > +- xen,force-assign-without-iommu
> > +  If xen,force-assign-without-iommu is present Xen continues booting
> > +  even on IOMMU setup errors for the device (i.e. the device is not
> > +  protected by an IOMMU).
> 
> Reading again, this suggest the option can be used to force assignment if a
> Device is behind an IOMMU but the setup failed.
> 
> All IOMMUs should be configured to deny any transaction. So your device is not
> going to work. We should also probably state the consequence of using this
> option.
> 
> It would be better to say:
> 
> "If xen,force-assign-without-iommu is present, Xen will allow to assign a
> device even if it is not behind an IOMMU. This will render your platform
> *unsafe* if the device is DMA-capable".

Yes, this is clear, I'll use your version. Can I retain the acked-by
while I make this change?

Patch
diff mbox series

diff --git a/docs/misc/arm/device-tree/booting.txt b/docs/misc/arm/device-tree/booting.txt
index 317a9e962a..649e00d09f 100644
--- a/docs/misc/arm/device-tree/booting.txt
+++ b/docs/misc/arm/device-tree/booting.txt
@@ -146,7 +146,18 @@  with the following properties:
 
 - vpl011
 
-    An empty property to enable/disable a virtual pl011 for the guest to use.
+    An empty property to enable/disable a virtual pl011 for the guest to
+    use. The virtual pl011 uses SPI number 0 (see GUEST_VPL011_SPI).
+    Please note that the SPI used for the virtual pl011 could clash with the
+    physical SPI of a physical device assigned to the guest.
+
+- nr_spis
+
+    Optional. A 32-bit integer specifying the number of SPIs (Shared
+    Peripheral Interrupts) to allocate for the domain. If nr_spis is
+    missing, the max number of SPIs supported by the physical GIC is
+    used, or GUEST_VPL011_SPI+1 if vpl011 is enabled, whichever is
+    greater.
 
 - #address-cells and #size-cells
 
@@ -226,3 +237,34 @@  chosen {
         };
     };
 };
+
+
+Device Assignment
+=================
+
+Device Assignment (Passthrough) is supported by adding another module,
+alongside the kernel and ramdisk, with the device tree fragment
+corresponding to the device node to assign to the guest.
+
+The dtb sub-node should have the following properties:
+
+- compatible
+
+    "multiboot,device-tree" and "multiboot,module"
+
+- reg
+
+    Specifies the physical address of the device tree binary fragment
+    RAM and its length.
+
+As an example:
+
+        module@0xc000000 {
+            compatible = "multiboot,device-tree", "multiboot,module";
+            reg = <0x0 0xc000000 0xffffff>;
+        };
+
+The DTB fragment is loaded at 0xc000000 in the example above. It should
+follow the convention explained in docs/misc/arm/passthrough.txt. The
+DTB fragment will be added to the guest device tree, so that the guest
+kernel will be able to discover the device.
diff --git a/docs/misc/arm/passthrough.txt b/docs/misc/arm/passthrough.txt
index 0efbd122de..6826e1f341 100644
--- a/docs/misc/arm/passthrough.txt
+++ b/docs/misc/arm/passthrough.txt
@@ -80,6 +80,112 @@  SPI numbers start from 32, in this example 80 + 32 = 112.
 See man [xl.cfg] for the iomem format. The reg property is just a pair
 of address, then size numbers, each of them can occupy 1 or 2 cells.
 
+
+Dom0-less Device Passthrough
+============================
+
+The partial device tree for dom0-less guests should have the following
+properties for each node corresponding to a physical device to assign to
+the guest:
+
+- xen,reg
+
+  The xen,reg property is an array of:
+
+    <phys_addr size guest_addr>
+
+  They specify the physical address and size of the device memory
+  ranges together with the corresponding guest address to map them to.
+  The size of `phys_addr' and `guest_addr' is determined by
+  #address-cells, the size of `size' is determined by #size-cells, of
+  the partial device tree.
+  The memory will be mapped as device memory in the guest (Device-nGnRE).
+
+- xen,path
+
+  A string property representing the path in the host device tree to the
+  corresponding device node.
+
+- xen,force-assign-without-iommu
+  If xen,force-assign-without-iommu is present Xen continues booting
+  even on IOMMU setup errors for the device (i.e. the device is not
+  protected by an IOMMU).
+
+In addition, a special /gic node is expected as a placeholder for the
+full GIC node that will be added by Xen for the guest. /gic can be
+referenced by other properties in the device tree fragment. For
+instance, it can be referenced by interrupt-parent under a device node.
+Xen will take care of replacing the "gic" placeholder node for a
+complete GIC node while retaining all the references correctly. The new
+GIC node created by Xen is a regular interrupt-controller@<unit> node.
+
+    gic: gic {
+        #interrupt-cells = <0x3>;
+        interrupt-controller;
+    };
+
+Note that the #interrupt-cells and interrupt-controller properties are
+not actually required, however, DTC expects them to be present if gic is
+referenced by interrupt-parent or similar.
+
+
+Example
+=======
+
+The following is a real-world example of a device tree fragment to
+assign a network card to a dom0-less guest on Xilinx Ultrascale+ MPSoC:
+
+/dts-v1/;
+
+/ {
+    #address-cells = <2>;
+    #size-cells = <1>;
+
+    gic: gic {
+        #interrupt-cells = <3>;
+        interrupt-controller;
+    };
+
+    passthrough {
+        compatible = "simple-bus";
+        ranges;
+        #address-cells = <2>;
+        #size-cells = <1>;
+
+        misc_clk: misc_clk {
+            #clock-cells = <0>;
+            clock-frequency = <0x7735940>;
+            compatible = "fixed-clock";
+        };
+
+        ethernet@ff0e0000 {
+            compatible = "cdns,zynqmp-gem";
+            status = "okay";
+            reg = <0x0 0xff0e0000 0x1000>;
+            clock-names = "pclk", "hclk", "tx_clk", "rx_clk";
+            #address-cells = <1>;
+            #size-cells = <0>;
+            clocks = <&misc_clk &misc_clk &misc_clk &misc_clk>;
+            phy-mode = "rgmii-id";
+            xlnx,ptp-enet-clock = <0>;
+            local-mac-address = [00 0a 35 00 22 01];
+            interrupt-parent = <&gic>;
+            interrupts = <0 63 0x4 0 63 0x4>;
+            xen,path = "/amba/ethernet@ff0e0000";
+            xen,reg = <0x0 0xff0e0000 0x1000 0x0 0xff0e0000>;
+
+            phy@c {
+                reg = <0xc>;
+                ti,rx-internal-delay = <0x8>;
+                ti,tx-internal-delay = <0xa>;
+                ti,fifo-depth = <0x1>;
+                ti,rxctrl-strap-worka;
+            };
+        };
+    };
+};
+
+
 [arm,gic.txt]: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/devicetree/bindings/interrupt-controller/arm,gic.txt
 [arm,gic-v3.txt]: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/devicetree/bindings/interrupt-controller/arm,gic-v3.txt
 [xl.cfg]: https://xenbits.xen.org/docs/unstable/man/xl.cfg.5.html