| Message ID | 1345131377-14713-6-git-send-email-stefano.stabellini@eu.citrix.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
On 08/16/2012 10:35 AM, Stefano Stabellini wrote: > Add a doc to describe the Xen ARM device tree bindings > > Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com> > CC: devicetree-discuss@lists.ozlabs.org > CC: David Vrabel <david.vrabel@citrix.com> > --- > Documentation/devicetree/bindings/arm/xen.txt | 22 ++++++++++++++++++++++ > 1 files changed, 22 insertions(+), 0 deletions(-) > create mode 100644 Documentation/devicetree/bindings/arm/xen.txt > > diff --git a/Documentation/devicetree/bindings/arm/xen.txt b/Documentation/devicetree/bindings/arm/xen.txt > new file mode 100644 > index 0000000..ec6d884 > --- /dev/null > +++ b/Documentation/devicetree/bindings/arm/xen.txt > @@ -0,0 +1,22 @@ > +* Xen hypervisor device tree bindings > + > +Xen ARM virtual platforms shall have the following properties: > + > +- compatible: > + compatible = "xen,xen", "xen,xen-<version>"; > + where <version> is the version of the Xen ABI of the platform. > + > +- reg: specifies the base physical address and size of a region in > + memory where the grant table should be mapped to, using an > + HYPERVISOR_memory_op hypercall. > + > +- interrupts: the interrupt used by Xen to inject event notifications. You should look at ePAPR 1.1 which defines hypervisor related bindings. While it is a PPC doc, we should reuse or extend what makes sense. See section 7.5: https://www.power.org/resources/downloads/Power_ePAPR_APPROVED_v1.1.pdf Rob > + > + > +Example: > + > +hypervisor { > + compatible = "xen,xen", "xen,xen-4.3"; > + reg = <0xb0000000 0x20000>; > + interrupts = <1 15 0xf08>; > +}; >
On Tue, 28 Aug 2012, Rob Herring wrote: > On 08/16/2012 10:35 AM, Stefano Stabellini wrote: > > Add a doc to describe the Xen ARM device tree bindings > > > > Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com> > > CC: devicetree-discuss@lists.ozlabs.org > > CC: David Vrabel <david.vrabel@citrix.com> > > --- > > Documentation/devicetree/bindings/arm/xen.txt | 22 ++++++++++++++++++++++ > > 1 files changed, 22 insertions(+), 0 deletions(-) > > create mode 100644 Documentation/devicetree/bindings/arm/xen.txt > > > > diff --git a/Documentation/devicetree/bindings/arm/xen.txt b/Documentation/devicetree/bindings/arm/xen.txt > > new file mode 100644 > > index 0000000..ec6d884 > > --- /dev/null > > +++ b/Documentation/devicetree/bindings/arm/xen.txt > > @@ -0,0 +1,22 @@ > > +* Xen hypervisor device tree bindings > > + > > +Xen ARM virtual platforms shall have the following properties: > > + > > +- compatible: > > + compatible = "xen,xen", "xen,xen-<version>"; > > + where <version> is the version of the Xen ABI of the platform. > > + > > +- reg: specifies the base physical address and size of a region in > > + memory where the grant table should be mapped to, using an > > + HYPERVISOR_memory_op hypercall. > > + > > +- interrupts: the interrupt used by Xen to inject event notifications. > > You should look at ePAPR 1.1 which defines hypervisor related bindings. > While it is a PPC doc, we should reuse or extend what makes sense. > > See section 7.5: > > https://www.power.org/resources/downloads/Power_ePAPR_APPROVED_v1.1.pdf Thanks for the link, I wasn't aware of ePAPR. The hypervisor node defined by ePAPR is not very different from what I am proposing. Should I try to be compatible with the hypervisor specification above (as in compatible = "epapr,hypervisor-1.1")? Or should I just use it as a reference for my own specification? Personally I would rather avoid full compatibility with ePAPR.
On Wed, 12 Sep 2012, Stefano Stabellini wrote: > On Tue, 28 Aug 2012, Rob Herring wrote: > > You should look at ePAPR 1.1 which defines hypervisor related bindings. > > While it is a PPC doc, we should reuse or extend what makes sense. > > > > See section 7.5: > > > > https://www.power.org/resources/downloads/Power_ePAPR_APPROVED_v1.1.pdf > > Thanks for the link, I wasn't aware of ePAPR. > > The hypervisor node defined by ePAPR is not very different from what I > am proposing. Should I try to be compatible with the hypervisor > specification above (as in compatible = "epapr,hypervisor-1.1")? > Or should I just use it as a reference for my own specification? > > Personally I would rather avoid full compatibility with ePAPR. In particular reading section 7.5, these are the required properties of the ePAPR hypervisor node: - compatible = "epapr,hypervisor-1.1" compared to what I am proposing, it is laking information about what hypervisor we are talking about (xen, kvm, vmware, etc) and the version of the ABI (xen-4.2). - hcall-instructions potentially interesting, but given that for Xen we are quite happy with HVC, we are not going to add any secondary hypercall mechanisms, therefore at the moment it would just result in a BUG if the specified hcall instruction is != HVC. Besides if somebody else wanted to implemented the Xen hypercall interface in a different way they could just reimplement the hypercall wrappers, that would be easier than trying to do it with this property. - guest-id we usually make a point in trying not to tell the guest its own domid because if the VM gets migrated to a different host it acquires a new domid, therefore it should not rely on it. - guest-name we could pass the guest name here, but I don't see how it could be of any use. On the other hand, thinking more about what Xen needs in the device tree, I think we could improve the current spec by clarifying the meaning of the memory region and interrupt properties we currently require. I thought about moving them to two separate children node with an explicit name: --- * Xen hypervisor device tree bindings Xen ARM virtual platforms shall have the following properties and children nodes: - compatible property: compatible = "xen,xen", "xen,xen-<version>"; where <version> is the version of the Xen ABI of the platform. - grant_table child with the following properties: - name: name = "grant_table"; - reg: specifies the base physical address and size of a region in memory where the grant table should be mapped to, using an HYPERVISOR_memory_op hypercall. - events child with the following properties: - name: name = "events"; - interrupts: the interrupt used by Xen to inject event notifications. Example: hypervisor { compatible = "xen,xen", "xen,xen-4.2"; #address-cells = <1>; #size-cells = <1>; #interrupt-cells = <3>; ranges = <0xb0000000 0xb0000000 0x20000>; grant_table { name = "grant_table"; reg = <0xb0000000 0x20000>; }; events { name = "events"; interrupts = <1 15 0xf08>; }; };
On 09/12/2012 01:14 PM, Stefano Stabellini wrote: > On Wed, 12 Sep 2012, Stefano Stabellini wrote: >> On Tue, 28 Aug 2012, Rob Herring wrote: >>> You should look at ePAPR 1.1 which defines hypervisor related bindings. >>> While it is a PPC doc, we should reuse or extend what makes sense. >>> >>> See section 7.5: >>> >>> https://www.power.org/resources/downloads/Power_ePAPR_APPROVED_v1.1.pdf >> >> Thanks for the link, I wasn't aware of ePAPR. >> >> The hypervisor node defined by ePAPR is not very different from what I >> am proposing. Should I try to be compatible with the hypervisor >> specification above (as in compatible = "epapr,hypervisor-1.1")? >> Or should I just use it as a reference for my own specification? >> >> Personally I would rather avoid full compatibility with ePAPR. > > In particular reading section 7.5, these are the required properties of > the ePAPR hypervisor node: > > - compatible = "epapr,hypervisor-1.1" > compared to what I am proposing, it is laking information about what > hypervisor we are talking about (xen, kvm, vmware, etc) and the version > of the ABI (xen-4.2). compatible properties are often changed. If we do deviate on the rest of the binding, then it needs to be different. Turns out that powerpc KVM guests use "linux,kvm" and "epapr,hypervisor" doesn't even appear in the kernel. We also perhaps have to consider the possibility of Xen on PowerPC. Then alignment is more important. > - hcall-instructions > potentially interesting, but given that for Xen we are quite happy with > HVC, we are not going to add any secondary hypercall mechanisms, > therefore at the moment it would just result in a BUG if the specified > hcall instruction is != HVC. Besides if somebody else wanted to > implemented the Xen hypercall interface in a different way they could > just reimplement the hypercall wrappers, that would be easier than > trying to do it with this property. It's really about the parameters passed with the HVC. The ePAPR may be a good model for defining this. Just grepping "hypervisor" under arch/powerpc, it's pretty clear hypervisor support happened first and the ePAPR came second. Hopefully we can avoid that for ARM. > - guest-id > we usually make a point in trying not to tell the guest its own domid > because if the VM gets migrated to a different host it acquires a new > domid, therefore it should not rely on it. > > - guest-name > we could pass the guest name here, but I don't see how it could be > of any use. > I have no issue with these being optional. > > On the other hand, thinking more about what Xen needs in the device > tree, I think we could improve the current spec by clarifying the > meaning of the memory region and interrupt properties we currently > require. I thought about moving them to two separate children node with > an explicit name: > > --- > > * Xen hypervisor device tree bindings > I really think we need to define ARM hypervisor device tree bindings first, then Xen specific bindings as an addition to that. I worry that the KVM folks aren't paying attention and then want something different later on. > Xen ARM virtual platforms shall have the following properties and > children nodes: > > - compatible property: > compatible = "xen,xen", "xen,xen-<version>"; "xen,xen" should be last as it is less specific. > where <version> is the version of the Xen ABI of the platform. > > - grant_table child with the following properties: > - name: > name = "grant_table"; What's a grant table? > - reg: specifies the base physical address and size of a region in > memory where the grant table should be mapped to, using an > HYPERVISOR_memory_op hypercall. > > - events child with the following properties: > - name: > name = "events"; > - interrupts: the interrupt used by Xen to inject event notifications. Why a child node? Just an interrupts property alone should be fine. If you have cases with different number of interrupts, the compatible property can distinguish that. Rob > > > Example: > hypervisor { > compatible = "xen,xen", "xen,xen-4.2"; > #address-cells = <1>; > #size-cells = <1>; > #interrupt-cells = <3>; > ranges = <0xb0000000 0xb0000000 0x20000>; > > grant_table { > name = "grant_table"; > reg = <0xb0000000 0x20000>; > }; > > events { > name = "events"; > interrupts = <1 15 0xf08>; > }; > }; >
On Wed, Sep 12, 2012 at 07:14:58PM +0100, Stefano Stabellini wrote: > On Wed, 12 Sep 2012, Stefano Stabellini wrote: > > On Tue, 28 Aug 2012, Rob Herring wrote: > > > You should look at ePAPR 1.1 which defines hypervisor related bindings. > > > While it is a PPC doc, we should reuse or extend what makes sense. > > > > > > See section 7.5: > > > > > > https://www.power.org/resources/downloads/Power_ePAPR_APPROVED_v1.1.pdf > > > > Thanks for the link, I wasn't aware of ePAPR. > > > > The hypervisor node defined by ePAPR is not very different from what I > > am proposing. Should I try to be compatible with the hypervisor > > specification above (as in compatible = "epapr,hypervisor-1.1")? > > Or should I just use it as a reference for my own specification? > > > > Personally I would rather avoid full compatibility with ePAPR. > > In particular reading section 7.5, these are the required properties of > the ePAPR hypervisor node: > > - compatible = "epapr,hypervisor-1.1" > compared to what I am proposing, it is laking information about what > hypervisor we are talking about (xen, kvm, vmware, etc) and the version > of the ABI (xen-4.2). > > - hcall-instructions > potentially interesting, but given that for Xen we are quite happy with > HVC, we are not going to add any secondary hypercall mechanisms, > therefore at the moment it would just result in a BUG if the specified > hcall instruction is != HVC. Besides if somebody else wanted to > implemented the Xen hypercall interface in a different way they could > just reimplement the hypercall wrappers, that would be easier than > trying to do it with this property. Some thoughts on this: We decided that embedding machine instructions into the DT is a fairly awful idea when discussing how to describe low-level debug UARTs in the DT. I don't think it's a lot better in this case (never mind issues like ARM versus Thumb, endianness etc.) If we are going to attempt to describe the call interface, we should do it symbolically, allowing the hypervisor interface code in the kernel to choose (or, if necessary, generate) the right call wrappers. We will have this issue for descrbing power firmware interfaces for example: we already know that this functionality might require an SMC or HVC instruction to call it, depending on the platform. A hypervisor with only one call ABI could leave this to be implicit, providing there is a version number property of similar to allow future changes to be accommodated. Cheers ---Dave
On Wed, Sep 12, 2012 at 09:34:28PM -0500, Rob Herring wrote: > On 09/12/2012 01:14 PM, Stefano Stabellini wrote: > > On Wed, 12 Sep 2012, Stefano Stabellini wrote: > >> On Tue, 28 Aug 2012, Rob Herring wrote: > >>> You should look at ePAPR 1.1 which defines hypervisor related bindings. > >>> While it is a PPC doc, we should reuse or extend what makes sense. > >>> > >>> See section 7.5: > >>> > >>> https://www.power.org/resources/downloads/Power_ePAPR_APPROVED_v1.1.pdf > >> > >> Thanks for the link, I wasn't aware of ePAPR. > >> > >> The hypervisor node defined by ePAPR is not very different from what I > >> am proposing. Should I try to be compatible with the hypervisor > >> specification above (as in compatible = "epapr,hypervisor-1.1")? > >> Or should I just use it as a reference for my own specification? > >> > >> Personally I would rather avoid full compatibility with ePAPR. > > > > In particular reading section 7.5, these are the required properties of > > the ePAPR hypervisor node: > > > > - compatible = "epapr,hypervisor-1.1" > > compared to what I am proposing, it is laking information about what > > hypervisor we are talking about (xen, kvm, vmware, etc) and the version > > of the ABI (xen-4.2). > > compatible properties are often changed. If we do deviate on the rest of > the binding, then it needs to be different. > > Turns out that powerpc KVM guests use "linux,kvm" and "epapr,hypervisor" > doesn't even appear in the kernel. > > We also perhaps have to consider the possibility of Xen on PowerPC. Then > alignment is more important. > > > - hcall-instructions > > potentially interesting, but given that for Xen we are quite happy with > > HVC, we are not going to add any secondary hypercall mechanisms, > > therefore at the moment it would just result in a BUG if the specified > > hcall instruction is != HVC. Besides if somebody else wanted to > > implemented the Xen hypercall interface in a different way they could > > just reimplement the hypercall wrappers, that would be easier than > > trying to do it with this property. > > It's really about the parameters passed with the HVC. The ePAPR may be a > good model for defining this. Just grepping "hypervisor" under > arch/powerpc, it's pretty clear hypervisor support happened first and > the ePAPR came second. Hopefully we can avoid that for ARM. Do you think it's feasible to standardise on some interoperable ABI for kvm and Xen? This sounds pretty optimistic, but I'm not aware of all the technicalities, or what possible third-party hypervisors are out there. If we could do it, it would be good. But I have my doubts about the feasibility and the benefits. If different hypervisors are significantly imcompatible, then having a common low-level ABI doesn't help all that much. Cheers ---Dave
On Thu, 13 Sep 2012, Dave Martin wrote: > On Wed, Sep 12, 2012 at 07:14:58PM +0100, Stefano Stabellini wrote: > > On Wed, 12 Sep 2012, Stefano Stabellini wrote: > > - hcall-instructions > > potentially interesting, but given that for Xen we are quite happy with > > HVC, we are not going to add any secondary hypercall mechanisms, > > therefore at the moment it would just result in a BUG if the specified > > hcall instruction is != HVC. Besides if somebody else wanted to > > implemented the Xen hypercall interface in a different way they could > > just reimplement the hypercall wrappers, that would be easier than > > trying to do it with this property. > > Some thoughts on this: > > We decided that embedding machine instructions into the DT is a fairly > awful idea when discussing how to describe low-level debug UARTs in the > DT. I don't think it's a lot better in this case (never mind issues > like ARM versus Thumb, endianness etc.) > > If we are going to attempt to describe the call interface, we should > do it symbolically, allowing the hypervisor interface code in the kernel > to choose (or, if necessary, generate) the right call wrappers. > > We will have this issue for descrbing power firmware interfaces for > example: we already know that this functionality might require an SMC > or HVC instruction to call it, depending on the platform. > > A hypervisor with only one call ABI could leave this to be implicit, > providing there is a version number property of similar to allow future > changes to be accommodated. I completely agree with Dave. I have no problems adding a symbolic property to say "we are using hvc with parameters on registers". I just want to avoid having actual machine instructions (and potentially dealing with executing them) into the DT. Maybe we could have a "calling-convention" property that can be "xen" or something else. When a new hypervisor vendor comes along it can change the value of "calling-convention" to "foo".
On Thu, 13 Sep 2012, Dave Martin wrote: > Do you think it's feasible to standardise on some interoperable ABI for > kvm and Xen? This sounds pretty optimistic, but I'm not aware of all > the technicalities, or what possible third-party hypervisors are out > there. > > If we could do it, it would be good. But I have my doubts about the > feasibility and the benefits. If different hypervisors are significantly > imcompatible, then having a common low-level ABI doesn't help all that > much. It is not really possible because each hypervisor offers a different set of hypercalls and has a different calling convention, so there is very little we can do to unify them. For example many of the current Xen hypercalls are to deal with the grant table and event channels, that are Xen specific concepts completely missing in KVM.
On Thu, 13 Sep 2012, Rob Herring wrote: > On 09/12/2012 01:14 PM, Stefano Stabellini wrote: > > On Wed, 12 Sep 2012, Stefano Stabellini wrote: > >> On Tue, 28 Aug 2012, Rob Herring wrote: > >>> You should look at ePAPR 1.1 which defines hypervisor related bindings. > >>> While it is a PPC doc, we should reuse or extend what makes sense. > >>> > >>> See section 7.5: > >>> > >>> https://www.power.org/resources/downloads/Power_ePAPR_APPROVED_v1.1.pdf > >> > >> Thanks for the link, I wasn't aware of ePAPR. > >> > >> The hypervisor node defined by ePAPR is not very different from what I > >> am proposing. Should I try to be compatible with the hypervisor > >> specification above (as in compatible = "epapr,hypervisor-1.1")? > >> Or should I just use it as a reference for my own specification? > >> > >> Personally I would rather avoid full compatibility with ePAPR. > > > > In particular reading section 7.5, these are the required properties of > > the ePAPR hypervisor node: > > > > - compatible = "epapr,hypervisor-1.1" > > compared to what I am proposing, it is laking information about what > > hypervisor we are talking about (xen, kvm, vmware, etc) and the version > > of the ABI (xen-4.2). > > compatible properties are often changed. If we do deviate on the rest of > the binding, then it needs to be different. > > Turns out that powerpc KVM guests use "linux,kvm" and "epapr,hypervisor" > doesn't even appear in the kernel. > > We also perhaps have to consider the possibility of Xen on PowerPC. Then > alignment is more important. OK. In that case I think that we can just use "xen,xen-4.2". > > - hcall-instructions > > potentially interesting, but given that for Xen we are quite happy with > > HVC, we are not going to add any secondary hypercall mechanisms, > > therefore at the moment it would just result in a BUG if the specified > > hcall instruction is != HVC. Besides if somebody else wanted to > > implemented the Xen hypercall interface in a different way they could > > just reimplement the hypercall wrappers, that would be easier than > > trying to do it with this property. > > It's really about the parameters passed with the HVC. The ePAPR may be a > good model for defining this. Just grepping "hypervisor" under > arch/powerpc, it's pretty clear hypervisor support happened first and > the ePAPR came second. Hopefully we can avoid that for ARM. Right. As I wrote in the other email, we could have a new property to select the calling convention (and therefore the hypercall wrappers) to be used with the hypervisor. > > - guest-id > > we usually make a point in trying not to tell the guest its own domid > > because if the VM gets migrated to a different host it acquires a new > > domid, therefore it should not rely on it. > > > > - guest-name > > we could pass the guest name here, but I don't see how it could be > > of any use. > > > > I have no issue with these being optional. OK, good. > > On the other hand, thinking more about what Xen needs in the device > > tree, I think we could improve the current spec by clarifying the > > meaning of the memory region and interrupt properties we currently > > require. I thought about moving them to two separate children node with > > an explicit name: > > > > --- > > > > * Xen hypervisor device tree bindings > > > > I really think we need to define ARM hypervisor device tree bindings > first, then Xen specific bindings as an addition to that. I worry that > the KVM folks aren't paying attention and then want something different > later on. The problem is that there isn't much in common between Xen and KVM at least from the DT point of view. I am not sure what would go into this common hypervisor node. The three key pieces of information that we are currently passing in the DT (xen-4.2, a memory region, a PPI) are Xen specific. If one day KVM (or another hypervisor vendor) decides to support the Xen interface, can't they just have the Xen specific binding with a slightly different compatible node? For example: compatible = "linux,kvm", "xen,xen-4.2" wouldn't that mean "I am KVM but I can support the Xen interface"? > > Xen ARM virtual platforms shall have the following properties and > > children nodes: > > > > - compatible property: > > compatible = "xen,xen", "xen,xen-<version>"; > > "xen,xen" should be last as it is less specific. Ah OK, thanks. > > where <version> is the version of the Xen ABI of the platform. > > > > - grant_table child with the following properties: > > - name: > > name = "grant_table"; > > What's a grant table? A Xen specific mechanism to share pages between guests. > > - reg: specifies the base physical address and size of a region in > > memory where the grant table should be mapped to, using an > > HYPERVISOR_memory_op hypercall. > > > > - events child with the following properties: > > - name: > > name = "events"; > > - interrupts: the interrupt used by Xen to inject event notifications. > > Why a child node? Just an interrupts property alone should be fine. If > you have cases with different number of interrupts, the compatible > property can distinguish that. I see, that's a good point. In that case maybe it doesn't make sense to move the memory region and the interrupt into two different children nodes after all. I should probably leave the spec as it was. > > > > > > Example: > > hypervisor { > > compatible = "xen,xen", "xen,xen-4.2"; > > #address-cells = <1>; > > #size-cells = <1>; > > #interrupt-cells = <3>; > > ranges = <0xb0000000 0xb0000000 0x20000>; > > > > grant_table { > > name = "grant_table"; > > reg = <0xb0000000 0x20000>; > > }; > > > > events { > > name = "events"; > > interrupts = <1 15 0xf08>; > > }; > > }; > > >
diff --git a/Documentation/devicetree/bindings/arm/xen.txt b/Documentation/devicetree/bindings/arm/xen.txt new file mode 100644 index 0000000..ec6d884 --- /dev/null +++ b/Documentation/devicetree/bindings/arm/xen.txt @@ -0,0 +1,22 @@ +* Xen hypervisor device tree bindings + +Xen ARM virtual platforms shall have the following properties: + +- compatible: + compatible = "xen,xen", "xen,xen-<version>"; + where <version> is the version of the Xen ABI of the platform. + +- reg: specifies the base physical address and size of a region in + memory where the grant table should be mapped to, using an + HYPERVISOR_memory_op hypercall. + +- interrupts: the interrupt used by Xen to inject event notifications. + + +Example: + +hypervisor { + compatible = "xen,xen", "xen,xen-4.3"; + reg = <0xb0000000 0x20000>; + interrupts = <1 15 0xf08>; +};
Add a doc to describe the Xen ARM device tree bindings Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com> CC: devicetree-discuss@lists.ozlabs.org CC: David Vrabel <david.vrabel@citrix.com> --- Documentation/devicetree/bindings/arm/xen.txt | 22 ++++++++++++++++++++++ 1 files changed, 22 insertions(+), 0 deletions(-) create mode 100644 Documentation/devicetree/bindings/arm/xen.txt