diff mbox series

[v2,01/11] docs: gunyah: Introduce Gunyah Hypervisor

Message ID 20220801211240.597859-2-quic_eberman@quicinc.com (mailing list archive)
State New, archived
Headers show
Series Drivers for gunyah hypervisor | expand

Commit Message

Elliot Berman Aug. 1, 2022, 9:12 p.m. UTC
Gunyah is an open-source Type-1 hypervisor developed by Qualcomm. It
does not depend on any lower-privileged OS/kernel code for its core
functionality. This increases its security and can support a smaller
trusted computing based when compared to Type-2 hypervisors.

Add documentation describing the Gunyah hypervisor and the main
components of the Gunyah hypervisor which are of interest to Linux
virtualization development.

Signed-off-by: Elliot Berman <quic_eberman@quicinc.com>
---
 Documentation/virt/gunyah/index.rst         | 92 +++++++++++++++++++++
 Documentation/virt/gunyah/message-queue.rst | 52 ++++++++++++
 Documentation/virt/index.rst                |  1 +
 MAINTAINERS                                 |  7 ++
 4 files changed, 152 insertions(+)
 create mode 100644 Documentation/virt/gunyah/index.rst
 create mode 100644 Documentation/virt/gunyah/message-queue.rst

Comments

Jeffrey Hugo Aug. 1, 2022, 9:29 p.m. UTC | #1
On Mon, Aug 1, 2022 at 3:16 PM Elliot Berman <quic_eberman@quicinc.com> wrote:
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 04ec80ee7352..18fb034526e1 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -8736,6 +8736,13 @@ L:       linux-efi@vger.kernel.org
>  S:     Maintained
>  F:     block/partitions/efi.*
>
> +GUNYAH HYPERVISOR DRIVER
> +M:     Elliot Berman <quic_eberman@quicinc.com>
> +M:     Murali Nalajala <quic_mnalajal@quicinc.com>
> +L:     linux-arm-msm@vger.kernel.org
> +S:     Maintained

Supported?  Sure seems like something you'd be paid to maintain....

> +F:     Documentation/virt/gunyah/
> +
>  HABANALABS PCI DRIVER
>  M:     Oded Gabbay <ogabbay@kernel.org>
>  S:     Supported
> --
> 2.25.1
>
Bagas Sanjaya Aug. 5, 2022, 3:18 a.m. UTC | #2
On Mon, Aug 01, 2022 at 02:12:30PM -0700, Elliot Berman wrote:
> +Communication with the resource manager from each guest VM happens with message-queue.rst. Details
> +about the specific messages can be found in drivers/virt/gunyah/rsc_mgr.c
> +
> +::
> +  +-------+   +--------+   +--------+
> +  |  RM   |   |  VM_A  |   |  VM_B  |
> +  +-.-.-.-+   +---.----+   +---.----+
> +    | |           |            |
> +  +-.-.-----------.------------.----+
> +  | | \==========/             |    |
> +  |  \========================/     |
> +  |            Gunyah               |
> +  +---------------------------------+
> +
 
Hi,

The diagram above triggers htmldocs warnings:

Documentation/virt/gunyah/index.rst:71: WARNING: Unexpected indentation.
Documentation/virt/gunyah/index.rst:72: WARNING: Block quote ends without a blank line; unexpected unindent.

I have applied the fixup:

diff --git a/Documentation/virt/gunyah/index.rst b/Documentation/virt/gunyah/index.rst
index 95ba9b71ab30d2..b74f7a6f9d4904 100644
--- a/Documentation/virt/gunyah/index.rst
+++ b/Documentation/virt/gunyah/index.rst
@@ -65,6 +65,7 @@ Communication with the resource manager from each guest VM happens with message-
 about the specific messages can be found in drivers/virt/gunyah/rsc_mgr.c
 
 ::
+
   +-------+   +--------+   +--------+
   |  RM   |   |  VM_A  |   |  VM_B  |
   +-.-.-.-+   +---.----+   +---.----+

Thanks.
Elliot Berman Aug. 5, 2022, 3:48 p.m. UTC | #3
On 8/4/2022 8:18 PM, Bagas Sanjaya wrote:
> On Mon, Aug 01, 2022 at 02:12:30PM -0700, Elliot Berman wrote:
>> +Communication with the resource manager from each guest VM happens with message-queue.rst. Details
>> +about the specific messages can be found in drivers/virt/gunyah/rsc_mgr.c
>> +
>> +::
>> +  +-------+   +--------+   +--------+
>> +  |  RM   |   |  VM_A  |   |  VM_B  |
>> +  +-.-.-.-+   +---.----+   +---.----+
>> +    | |           |            |
>> +  +-.-.-----------.------------.----+
>> +  | | \==========/             |    |
>> +  |  \========================/     |
>> +  |            Gunyah               |
>> +  +---------------------------------+
>> +
>   
> Hi,
> 
> The diagram above triggers htmldocs warnings:
> 
> Documentation/virt/gunyah/index.rst:71: WARNING: Unexpected indentation.
> Documentation/virt/gunyah/index.rst:72: WARNING: Block quote ends without a blank line; unexpected unindent.
> 
> I have applied the fixup:
> 
> diff --git a/Documentation/virt/gunyah/index.rst b/Documentation/virt/gunyah/index.rst
> index 95ba9b71ab30d2..b74f7a6f9d4904 100644
> --- a/Documentation/virt/gunyah/index.rst
> +++ b/Documentation/virt/gunyah/index.rst
> @@ -65,6 +65,7 @@ Communication with the resource manager from each guest VM happens with message-
>   about the specific messages can be found in drivers/virt/gunyah/rsc_mgr.c
>   
>   ::
> +
>     +-------+   +--------+   +--------+
>     |  RM   |   |  VM_A  |   |  VM_B  |
>     +-.-.-.-+   +---.----+   +---.----+
> 
> Thanks.
> 

Thanks! Updated for v3.
kernel test robot Aug. 6, 2022, 3:31 p.m. UTC | #4
Hi Elliot,

I love your patch! Perhaps something to improve:

[auto build test WARNING on arm64/for-next/core]
[also build test WARNING on v5.19]
[cannot apply to linus/master next-20220805]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/Elliot-Berman/Drivers-for-gunyah-hypervisor/20220802-051534
base:   https://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-next/core
reproduce: make htmldocs

If you fix the issue, kindly add following tag where applicable
Reported-by: kernel test robot <lkp@intel.com>

All warnings (new ones prefixed by >>):

>> Documentation/virt/gunyah/index.rst:71: WARNING: Unexpected indentation.
>> Documentation/virt/gunyah/index.rst:72: WARNING: Block quote ends without a blank line; unexpected unindent.

vim +71 Documentation/virt/gunyah/index.rst

    66	
    67	::
    68	  +-------+   +--------+   +--------+
    69	  |  RM   |   |  VM_A  |   |  VM_B  |
    70	  +-.-.-.-+   +---.----+   +---.----+
  > 71	    | |           |            |
  > 72	  +-.-.-----------.------------.----+
    73	  | | \==========/             |    |
    74	  |  \========================/     |
    75	  |            Gunyah               |
    76	  +---------------------------------+
    77
diff mbox series

Patch

diff --git a/Documentation/virt/gunyah/index.rst b/Documentation/virt/gunyah/index.rst
new file mode 100644
index 000000000000..e7bb2b14543e
--- /dev/null
+++ b/Documentation/virt/gunyah/index.rst
@@ -0,0 +1,92 @@ 
+.. SPDX-License-Identifier: GPL-2.0
+
+=================
+Gunyah Hypervisor
+=================
+
+.. toctree::
+   :maxdepth: 1
+
+   message-queue
+
+Gunyah is a Type-1 hypervisor which is independent of any OS kernel, and runs in
+a higher CPU privilege level. It does not depend on any lower-privileged operating system
+for its core functionality. This increases its security and can support a much smaller
+trusted computing base than a Type-2 hypervisor.
+
+Gunyah is an open source hypervisor. The source repo is available at
+https://github.com/quic/gunyah-hypervisor.
+
+Gunyah provides these following features.
+
+- Scheduling:
+  A scheduler for virtual CPUs (VCPUs) on physical CPUs and enables time-sharing
+  of the CPUs.
+- Memory Management:
+  APIs handling memory, abstracted as objects, limiting direct use of physical
+  addresses. Memory ownership and usage tracking of all memory under its control.
+  Memory partitioning between VMs is a fundamental security feature.
+- Interrupt Virtualization:
+  Uses CPU hardware interrupt virtualization capabilities. Interrupts are handled
+  in the hypervisor and routed to the assigned VM.
+- Inter-VM Communication:
+  There are several different mechanisms provided for communicating between VMs.
+- Virtual platform:
+  Architectural devices such as interrupt controllers and CPU timers are directly provided
+  by the hypervisor as well as core virtual platform devices and system APIs such as ARM PSCI.
+- Device Virtualization:
+  Para-virtualization of devices is supported using inter-VM communication.
+
+Architectures supported
+=======================
+AArch64 with a GIC
+
+Resources and Capabilities
+==========================
+
+Some services or resources provided by the Gunyah hypervisor are described by capability IDs.
+For instance, inter-VM communication is performed with doorbells and message queues. The specific
+instance of a doorbell is described by a capability ID. These devices are described in Linux as a
+struct gunyah_device.
+
+High level management of these resources is performed by the resource manager VM. RM informs a
+guest VM about resources it can access through either the device tree or via guest-initiated RPC.
+
+Resource Manager
+================
+
+The resource manager (RM) is a privileged application VM supporting the Gunyah Hypervisor.
+It provides policy enforcement aspects of the virtualization system. The resource manager can
+be treated as an extension of the Hypervisor but is separated to its own partition to ensure
+that the hypervisor layer itself remains small and secure and to maintain a separation of policy
+and mechanism in the platform. On arm64, RM runs at NS-EL1 similar to other virtual machines.
+
+Communication with the resource manager from each guest VM happens with message-queue.rst. Details
+about the specific messages can be found in drivers/virt/gunyah/rsc_mgr.c
+
+::
+  +-------+   +--------+   +--------+
+  |  RM   |   |  VM_A  |   |  VM_B  |
+  +-.-.-.-+   +---.----+   +---.----+
+    | |           |            |
+  +-.-.-----------.------------.----+
+  | | \==========/             |    |
+  |  \========================/     |
+  |            Gunyah               |
+  +---------------------------------+
+
+The source for the resource manager is available at https://github.com/quic/gunyah-resource-manager.
+
+The resource manager provides the following features:
+
+- Generate device-tree overlay
+- VM creation and deletion
+- VM device-tree management
+- VM access control policy
+- Interrupt routing configuration
+
+When booting a virtual machine which uses a devicetree, resource manager overlays a
+/hypervisor node. This node can let Linux know it is running as a Gunyah guest VM,
+how to communicate with resource manager, and basic description and capabilities of
+this VM. See Documentation/devicetree/bindings/gunyah/qcom,hypervisor.yml for a description
+of this node.
diff --git a/Documentation/virt/gunyah/message-queue.rst b/Documentation/virt/gunyah/message-queue.rst
new file mode 100644
index 000000000000..afd405f3a5e1
--- /dev/null
+++ b/Documentation/virt/gunyah/message-queue.rst
@@ -0,0 +1,52 @@ 
+.. SPDX-License-Identifier: GPL-2.0
+
+Message Queues
+==============
+Message queue is a simple low-capacity IPC channel between two VMs. It is
+intended for sending small control and configuration messages. Each message
+queue object is unidirectional, so a full-duplex IPC channel requires a pair of
+objects.
+
+Messages can be up to 240 bytes in length. Longer messages require a further
+protocol on top of the message queue messages themselves. For instance, communication
+with the resource manager adds a header field for sending longer messages via multiple
+message fragments.
+
+The diagram below shows how message queue works. A typical configuration involves
+2 message queues. Message queue 1 allows VM_A to send messages to VM_B. Message
+queue 2 allows VM_B to send messages to VM_A.
+
+1. VM_A sends a message of up to 240 bytes in length. It raises a hypercall
+   with the message to inform the hypervisor to add the message to
+   message queue 1's queue.
+2. Gunyah raises the corresponding interrupt for VM_B when any of these happens:
+   a. gh_msgq_send has PUSH flag. Queue is immediately flushed. This is the typical case.
+   b. Explicility with gh_msgq_push command from VM_A.
+   c. Message queue has reached a threshold depth.
+3. VM_B calls gh_msgq_recv and Gunyah copies message to requested buffer.
+
+For VM_B to send a message to VM_A, the process is identical, except that hypercalls
+reference message queue 2's capability ID.
+
+::
+
+      +---------------+         +-----------------+         +---------------+
+      |      VM_A     |         |Gunyah hypervisor|         |      VM_B     |
+      |               |         |                 |         |               |
+      |               |         |                 |         |               |
+      |               |   Tx    |                 |         |               |
+      |               |-------->|                 | Rx vIRQ |               |
+      |gh_msgq_send() | Tx vIRQ |Message queue 1  |-------->|gh_msgq_recv() |
+      |               |<------- |                 |         |               |
+      |               |         |                 |         |               |
+      | Message Queue |         |                 |         | Message Queue |
+      | driver        |         |                 |         | driver        |
+      |               |         |                 |         |               |
+      |               |         |                 |         |               |
+      |               |         |                 |   Tx    |               |
+      |               | Rx vIRQ |                 |<--------|               |
+      |gh_msgq_recv() |<--------|Message queue 2  | Tx vIRQ |gh_msgq_send() |
+      |               |         |                 |-------->|               |
+      |               |         |                 |         |               |
+      |               |         |                 |         |               |
+      +---------------+         +-----------------+         +---------------+
diff --git a/Documentation/virt/index.rst b/Documentation/virt/index.rst
index 492f0920b988..dd4e8ef284eb 100644
--- a/Documentation/virt/index.rst
+++ b/Documentation/virt/index.rst
@@ -14,6 +14,7 @@  Linux Virtualization Support
    ne_overview
    acrn/index
    coco/sev-guest
+   gunyah/index
 
 .. only:: html and subproject
 
diff --git a/MAINTAINERS b/MAINTAINERS
index 04ec80ee7352..18fb034526e1 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -8736,6 +8736,13 @@  L:	linux-efi@vger.kernel.org
 S:	Maintained
 F:	block/partitions/efi.*
 
+GUNYAH HYPERVISOR DRIVER
+M:	Elliot Berman <quic_eberman@quicinc.com>
+M:	Murali Nalajala <quic_mnalajal@quicinc.com>
+L:	linux-arm-msm@vger.kernel.org
+S:	Maintained
+F:	Documentation/virt/gunyah/
+
 HABANALABS PCI DRIVER
 M:	Oded Gabbay <ogabbay@kernel.org>
 S:	Supported