[v3] xen/evtchn: Add design for static event channel signaling

Commit Message

Rahul Singh May 13, 2022, 2:38 p.m. UTC

This patch introduces a new feature to support the signaling between
two domains in dom0less system.

Signed-off-by: Rahul Singh <[e-mail]>
---
v3 changes:
- add dt node example for dom0 and domU.
- add info about "xen,enhanced" property to enable the event-channel interface
  for domU guests.

v2 changes:
- switch to the one-subnode-per-evtchn under xen,domain" compatible node.
- Add more detail about event-channel.
---
 docs/designs/dom0less-evtchn.md | 144 ++++++++++++++++++++++++++++++++
 1 file changed, 144 insertions(+)
 create mode 100644 docs/designs/dom0less-evtchn.md

Comments

Stefano Stabellini June 11, 2022, 12:23 a.m. UTC | #1

On Fri, 13 May 2022, Rahul Singh wrote:
> This patch introduces a new feature to support the signaling between
> two domains in dom0less system.
> 
> Signed-off-by: Rahul Singh <[e-mail]>
> ---
> v3 changes:
> - add dt node example for dom0 and domU.
> - add info about "xen,enhanced" property to enable the event-channel interface
>   for domU guests.
> 
> v2 changes:
> - switch to the one-subnode-per-evtchn under xen,domain" compatible node.
> - Add more detail about event-channel.
> ---
>  docs/designs/dom0less-evtchn.md | 144 ++++++++++++++++++++++++++++++++
>  1 file changed, 144 insertions(+)
>  create mode 100644 docs/designs/dom0less-evtchn.md
> 
> diff --git a/docs/designs/dom0less-evtchn.md b/docs/designs/dom0less-evtchn.md
> new file mode 100644
> index 0000000000..3c89a9fb7d
> --- /dev/null
> +++ b/docs/designs/dom0less-evtchn.md
> @@ -0,0 +1,144 @@
> +# Signaling support between two domUs on dom0less system
> +
> +## Current state: Draft version
> +
> +## Proposer(s): Rahul Singh, Bertrand Marquis
> +
> +## Problem Statement:
> +
> +Dom0less guests would benefit from a statically-defined memory sharing and
> +signally system for communication. One that would be immediately available at
> +boot without any need for dynamic configurations.
> +
> +In embedded a great variety of guest operating system kernels exist, many of
> +which don't have support for xenstore, grant table, or other complex drivers.
> +Some of them are small kernel-space applications (often called "baremetal",
> +not to be confused with the term "baremetal" used in the data center which
> +means "without hypervisors") or RTOSes. Additionally, for safety reasons, users
> +often need to be able to configure the full system statically so that it can
> +be verified statically.
> +
> +Event channels are very simple and can be added even to baremetal applications.
> +This proposal introduces a way to define them statically to make them suitable
> +for dom0less embedded deployments.
> +
> +## Proposal:
> +
> +Event channels are the basic primitive provided by Xen for event notifications.
> +An event channel is a logical connection between 2 domains (more specifically
> +between dom1,port1, and dom2,port2). Each event has a pending and a masked bit.
> +The pending bit indicates the event has been raised. The masked bit is used by
> +the domain to prevent the delivery of that specific event. Xen only performs a
> +0 -> 1 transition on the pending bits and does not touch the mask bit. The
> +domain may toggle masked bits in the masked bit field and should clear the
> +pending bit when an event has been processed
> +
> +Events are received by a domain via an interrupt from Xen to the domain,
> +indicating when an event arrives (setting the bit). Further notifications are
> +blocked until the bit is cleared again. Events are delivered asynchronously to
> +a domain and are enqueued when the domain is not running. Xen supports two
> +different ABIs for event channel: FIFO and 2L.
> +
> +The event channel communication will be established statically between two
> +domains (dom0 and domU also) before unpausing the domains after domain creation.
> +Event channel connection information between domains will be passed to Xen via
> +the device tree node. The event channel will be created and established
> +in Xen before the domain started. The domain doesn’t need to do any operation
> +to establish a connection. Domain only needs hypercall
> +EVTCHNOP_send(local port) to send notifications to the remote guest.
> +
> +There is no need to describe the static event channel info in the domU device
> +tree. Static event channels are only useful in fully static configurations,
> +and in those configurations the domU device tree dynamically generated by Xen
> +is not needed.
> +
> +To enable the event-channel interface for domU guests include the
> +"xen,enhanced" property with an empty string ( or with the value
> +"enabled” or "evtchn") in domU Xen device tree node.
> +
> +Under the "xen,domain" compatible node, there need to be sub-nodes with
> +compatible "xen,evtchn" that describe the event channel connection between two
> +domains(dom0 and domU also).
> +
> +The event channel sub-node has the following properties:
> +
> +- compatible
> +
> +    "xen,evtchn"
> +
> +- xen,evtchn
> +
> +    The property is tuples of two numbers
> +    (local-evtchn link-to-foreign-evtchn) where:
> +
> +    local-evtchn is an integer value that will be used to allocate local port
> +    for a domain to send and receive event notifications to/from the remote
> +    domain. Maximum supported value is 2^17 for FIFO ABI and 4096 for 2L ABI.
> +
> +    link-to-foreign-evtchn is a single phandle to a foreign evtchn to which
> +    local-evtchn will be connected.
> +
> +
> +Example:
> +
> +    chosen {
> +        ....
> +
> +        module@0 {
> +            compatible = "multiboot,kernel", "multiboot,module";
> +            xen,uefi-binary = "...";
> +            bootargs = "...";
> +
> +            /* one sub-node per local event channel */
> +            ec1: evtchn@1 {
> +                compatible = "xen,evtchn-v1";
> +                /* local-evtchn link-to-foreign-evtchn */
> +                xen,evtchn = <0xa &ec2>;
> +            };
> +        };

Great that you added a dom0 example. I wish we had a dom0 node for dom0
but what you have done here is the easiest thing we can do and less
disruptive for the existing bindings.

Reviewed-by: Stefano Stabellini <[e-mail]>


> +        domU1: domU1 {
> +            compatible = "xen,domain";
> +
> +            /* one sub-node per local event channel */
> +            ec2: evtchn@2 {
> +                compatible = "xen,evtchn-v1";
> +                /* local-evtchn link-to-foreign-evtchn */
> +                xen,evtchn = <0xa &ec1>;
> +            };
> +
> +            ec3: evtchn@3 {
> +                compatible = "xen,evtchn-v1";
> +                xen,evtchn = <0xb &ec5>;
> +            };
> +
> +            ec4: evtchn@4 {
> +                compatible = "xen,evtchn-v1";
> +                xen,evtchn = <0xc &ec6>;
> +            };
> +            ....
> +        };
> +
> +        domU2: domU2 {
> +            compatible = "xen,domain";
> +
> +            /* one sub-node per local event channel */
> +            ec5: evtchn@5 {
> +                compatible = "xen,evtchn-v1";
> +                /* local-evtchn link-to-foreign-evtchn */
> +                xen,evtchn = <0xa &ec3>;
> +            };
> +
> +            ec6: evtchn@6 {
> +                compatible = "xen,evtchn-v1";
> +                xen,evtchn = <0xb &ec4>;
> +            };
> +            ....
> +        };
> +    };
> +
> +In above example three event channel comunications will be established:
> +
> +    dom0  (port 0xa) <-----------------> domU1 (port 0xa)
> +    domU1 (port 0xb) <-----------------> domU2 (port 0xa)
> +    domU1 (port 0xc) <-----------------> domU2 (port 0xb)
> -- 
> 2.25.1
>

Patch

diff --git a/docs/designs/dom0less-evtchn.md b/docs/designs/dom0less-evtchn.md
new file mode 100644
index 0000000000..3c89a9fb7d
--- /dev/null
+++ b/docs/designs/dom0less-evtchn.md
@@ -0,0 +1,144 @@ 
+# Signaling support between two domUs on dom0less system
+
+## Current state: Draft version
+
+## Proposer(s): Rahul Singh, Bertrand Marquis
+
+## Problem Statement:
+
+Dom0less guests would benefit from a statically-defined memory sharing and
+signally system for communication. One that would be immediately available at
+boot without any need for dynamic configurations.
+
+In embedded a great variety of guest operating system kernels exist, many of
+which don't have support for xenstore, grant table, or other complex drivers.
+Some of them are small kernel-space applications (often called "baremetal",
+not to be confused with the term "baremetal" used in the data center which
+means "without hypervisors") or RTOSes. Additionally, for safety reasons, users
+often need to be able to configure the full system statically so that it can
+be verified statically.
+
+Event channels are very simple and can be added even to baremetal applications.
+This proposal introduces a way to define them statically to make them suitable
+for dom0less embedded deployments.
+
+## Proposal:
+
+Event channels are the basic primitive provided by Xen for event notifications.
+An event channel is a logical connection between 2 domains (more specifically
+between dom1,port1, and dom2,port2). Each event has a pending and a masked bit.
+The pending bit indicates the event has been raised. The masked bit is used by
+the domain to prevent the delivery of that specific event. Xen only performs a
+0 -> 1 transition on the pending bits and does not touch the mask bit. The
+domain may toggle masked bits in the masked bit field and should clear the
+pending bit when an event has been processed
+
+Events are received by a domain via an interrupt from Xen to the domain,
+indicating when an event arrives (setting the bit). Further notifications are
+blocked until the bit is cleared again. Events are delivered asynchronously to
+a domain and are enqueued when the domain is not running. Xen supports two
+different ABIs for event channel: FIFO and 2L.
+
+The event channel communication will be established statically between two
+domains (dom0 and domU also) before unpausing the domains after domain creation.
+Event channel connection information between domains will be passed to Xen via
+the device tree node. The event channel will be created and established
+in Xen before the domain started. The domain doesn’t need to do any operation
+to establish a connection. Domain only needs hypercall
+EVTCHNOP_send(local port) to send notifications to the remote guest.
+
+There is no need to describe the static event channel info in the domU device
+tree. Static event channels are only useful in fully static configurations,
+and in those configurations the domU device tree dynamically generated by Xen
+is not needed.
+
+To enable the event-channel interface for domU guests include the
+"xen,enhanced" property with an empty string ( or with the value
+"enabled” or "evtchn") in domU Xen device tree node.
+
+Under the "xen,domain" compatible node, there need to be sub-nodes with
+compatible "xen,evtchn" that describe the event channel connection between two
+domains(dom0 and domU also).
+
+The event channel sub-node has the following properties:
+
+- compatible
+
+    "xen,evtchn"
+
+- xen,evtchn
+
+    The property is tuples of two numbers
+    (local-evtchn link-to-foreign-evtchn) where:
+
+    local-evtchn is an integer value that will be used to allocate local port
+    for a domain to send and receive event notifications to/from the remote
+    domain. Maximum supported value is 2^17 for FIFO ABI and 4096 for 2L ABI.
+
+    link-to-foreign-evtchn is a single phandle to a foreign evtchn to which
+    local-evtchn will be connected.
+
+
+Example:
+
+    chosen {
+        ....
+
+        module@0 {
+            compatible = "multiboot,kernel", "multiboot,module";
+            xen,uefi-binary = "...";
+            bootargs = "...";
+
+            /* one sub-node per local event channel */
+            ec1: evtchn@1 {
+                compatible = "xen,evtchn-v1";
+                /* local-evtchn link-to-foreign-evtchn */
+                xen,evtchn = <0xa &ec2>;
+            };
+        };
+
+        domU1: domU1 {
+            compatible = "xen,domain";
+
+            /* one sub-node per local event channel */
+            ec2: evtchn@2 {
+                compatible = "xen,evtchn-v1";
+                /* local-evtchn link-to-foreign-evtchn */
+                xen,evtchn = <0xa &ec1>;
+            };
+
+            ec3: evtchn@3 {
+                compatible = "xen,evtchn-v1";
+                xen,evtchn = <0xb &ec5>;
+            };
+
+            ec4: evtchn@4 {
+                compatible = "xen,evtchn-v1";
+                xen,evtchn = <0xc &ec6>;
+            };
+            ....
+        };
+
+        domU2: domU2 {
+            compatible = "xen,domain";
+
+            /* one sub-node per local event channel */
+            ec5: evtchn@5 {
+                compatible = "xen,evtchn-v1";
+                /* local-evtchn link-to-foreign-evtchn */
+                xen,evtchn = <0xa &ec3>;
+            };
+
+            ec6: evtchn@6 {
+                compatible = "xen,evtchn-v1";
+                xen,evtchn = <0xb &ec4>;
+            };
+            ....
+        };
+    };
+
+In above example three event channel comunications will be established:
+
+    dom0  (port 0xa) <-----------------> domU1 (port 0xa)
+    domU1 (port 0xb) <-----------------> domU2 (port 0xa)
+    domU1 (port 0xc) <-----------------> domU2 (port 0xb)