[v2,2/2] docs: Introduce some hypercall page documentation

Commit Message

Andrew Cooper May 29, 2019, 4:10 a.m. UTC

This also introduced the top-level Guest Documentation section.

Signed-off-by: Andrew Cooper <[e-mail]>
---
CC: George Dunlap <[e-mail]>
CC: Ian Jackson <[e-mail]>
CC: Jan Beulich <[e-mail]>
CC: Konrad Rzeszutek Wilk <[e-mail]>
CC: Stefano Stabellini <[e-mail]>
CC: Tim Deegan <[e-mail]>
CC: Wei Liu <[e-mail]>
CC: Julien Grall <[e-mail]>

v2:
 * Drop AT&T ligatures
 * Move into an x86 directory

The rendered version can be viewed at:

https://andrewcoop-xen.readthedocs.io/en/docs-devel/guest-guide/x86/hypercall-abi.html
---
 docs/guest-guide/index.rst             |   7 ++
 docs/guest-guide/x86/hypercall-abi.rst | 127 +++++++++++++++++++++++++++++++++
 docs/guest-guide/x86/index.rst         |   7 ++
 docs/index.rst                         |  13 ++++
 4 files changed, 154 insertions(+)
 create mode 100644 docs/guest-guide/index.rst
 create mode 100644 docs/guest-guide/x86/hypercall-abi.rst
 create mode 100644 docs/guest-guide/x86/index.rst

Comments

Jan Beulich May 29, 2019, 8:45 a.m. UTC | #1

>>> On 29.05.19 at 06:10, <[e-mail]> wrote:
> This also introduced the top-level Guest Documentation section.
> 
> Signed-off-by: Andrew Cooper <[e-mail]>

Acked-by: Jan Beulich <[e-mail]>
with one further remark / question:

> +Hypercall Page
> +==============
> +
> +The hypercall page is a page of guest RAM into which Xen will write suitable
> +transfer stubs.
> +
> +Creating a hypercall page is an isolated operation from Xen's point of view.
> +It is the guests responsibility to ensure that the hypercall page, once
> +written by Xen, is mapped with executable permissions so it may be used.
> +Multiple hypercall pages may be created by the guest, if it wishes.
> +
> +The stubs are arranged by hypercall index, and start on 32-byte boundaries.
> +To invoke a specific hypercall, ``call`` the relevant stub [3]_:
> +
> +.. code-block:: none
> +
> +  call hypercall_page + index * 32
> +
> +There result is an ABI which is invariant of the exact operating mode or
> +hardware vendor.  This is intended to simplify guest kernel interfaces by
> +abstracting away the details of how it is currently running.

Is it worth mentioning here that certain aspects of the hypervisor interface
(shared data structures) are influenced by the mode the guest is in at the
time the most recent hypercall page registration (oddly enough alternatively
the most recent setting of HVM_PARAM_CALLBACK_IRQ) has occurred?

Jan

Andrew Cooper May 29, 2019, 11:50 a.m. UTC | #2

On 29/05/2019 09:45, Jan Beulich wrote:
>>>> On 29.05.19 at 06:10, <[e-mail]> wrote:
>> This also introduced the top-level Guest Documentation section.
>>
>> Signed-off-by: Andrew Cooper <[e-mail]>
> Acked-by: Jan Beulich <[e-mail]>

Thanks.

> with one further remark / question:
>
>> +Hypercall Page
>> +==============
>> +
>> +The hypercall page is a page of guest RAM into which Xen will write suitable
>> +transfer stubs.
>> +
>> +Creating a hypercall page is an isolated operation from Xen's point of view.
>> +It is the guests responsibility to ensure that the hypercall page, once
>> +written by Xen, is mapped with executable permissions so it may be used.
>> +Multiple hypercall pages may be created by the guest, if it wishes.
>> +
>> +The stubs are arranged by hypercall index, and start on 32-byte boundaries.
>> +To invoke a specific hypercall, ``call`` the relevant stub [3]_:
>> +
>> +.. code-block:: none
>> +
>> +  call hypercall_page + index * 32
>> +
>> +There result is an ABI which is invariant of the exact operating mode or
>> +hardware vendor.  This is intended to simplify guest kernel interfaces by
>> +abstracting away the details of how it is currently running.
> Is it worth mentioning here that certain aspects of the hypervisor interface
> (shared data structures) are influenced by the mode the guest is in at the
> time the most recent hypercall page registration (oddly enough alternatively
> the most recent setting of HVM_PARAM_CALLBACK_IRQ) has occurred?

That aspect had crossed my mind, but I decided to leave it for now.

Details on the format of the shared_info page should live primarily in
the shared_info documentation, but I do intend to cross reference
appropriate documentation when both sides are in place.  I'd prefer that
any addition to this document happens when the main shared_info
documentation is done, rather than having an unqualified fraction of the
end result.

Note however that I am taking this opportunity to evaluate the current
behaviour of the areas being documented (hence the cleanup patches), and
I make no guarantees that the details of shared_info latching will be
identical to how they currently are, when the documentation is
eventually complete.

~Andrew

Patch

diff --git a/docs/guest-guide/index.rst b/docs/guest-guide/index.rst
new file mode 100644
index 0000000..108e0b8
--- /dev/null
+++ b/docs/guest-guide/index.rst
@@ -0,0 +1,7 @@ 
+Guest documentation
+===================
+
+.. toctree::
+  :maxdepth: 2
+
+  x86/index
diff --git a/docs/guest-guide/x86/hypercall-abi.rst b/docs/guest-guide/x86/hypercall-abi.rst
new file mode 100644
index 0000000..dee2585
--- /dev/null
+++ b/docs/guest-guide/x86/hypercall-abi.rst
@@ -0,0 +1,127 @@ 
+Hypercall ABI
+=============
+
+Hypercalls are system calls to Xen.  Two modes of guest operation are
+supported, and up to 6 individual parameters are supported.
+
+Hypercalls may only be issued by kernel-level software [1]_.
+
+Registers
+---------
+
+The registers used for hypercalls depends on the operating mode of the guest.
+
+.. list-table::
+  :header-rows: 1
+
+  * - ABI
+    - Hypercall Index
+    - Parameters (1 - 6)
+    - Result
+
+  * - 64bit
+    - RAX
+    - RDI RSI RDX R10 R8 R9
+    - RAX
+
+  * - 32bit
+    - EAX
+    - EBX ECX EDX ESI EDI EBP
+    - EAX
+
+32 and 64bit PV guests have an ABI fixed by their guest type.  The ABI for an
+HVM guest depends on whether the vCPU is operating in a 64bit segment or not
+[2]_.
+
+
+Parameters
+----------
+
+Different hypercalls take a different number of parameters.  Each hypercall
+potentially clobbers each of its parameter registers; a guest may not rely on
+the parameter registers staying the same.  A debug build of Xen checks this by
+deliberately poisoning the parameter registers before returning back to the
+guest.
+
+
+Mode transfer
+-------------
+
+The exact sequence of instructions required to issue a hypercall differs
+between virtualisation mode and hardware vendor.
+
+.. list-table::
+  :header-rows: 1
+
+  * - Guest
+    - Transfer instruction
+
+  * - 32bit PV
+    - INT 0x82
+
+  * - 64bit PV
+    - SYSCALL
+
+  * - Intel HVM
+    - VMCALL
+
+  * - AMD HVM
+    - VMMCALL
+
+To abstract away the details, Xen implements an interface known as the
+Hypercall Page.  This allows a guest to make a hypercall without needing to
+perform mode-specific or vendor-specific setup.
+
+
+Hypercall Page
+==============
+
+The hypercall page is a page of guest RAM into which Xen will write suitable
+transfer stubs.
+
+Creating a hypercall page is an isolated operation from Xen's point of view.
+It is the guests responsibility to ensure that the hypercall page, once
+written by Xen, is mapped with executable permissions so it may be used.
+Multiple hypercall pages may be created by the guest, if it wishes.
+
+The stubs are arranged by hypercall index, and start on 32-byte boundaries.
+To invoke a specific hypercall, ``call`` the relevant stub [3]_:
+
+.. code-block:: none
+
+  call hypercall_page + index * 32
+
+There result is an ABI which is invariant of the exact operating mode or
+hardware vendor.  This is intended to simplify guest kernel interfaces by
+abstracting away the details of how it is currently running.
+
+
+Creating Hypercall Pages
+------------------------
+
+Guests which are started using the PV boot protocol may set set
+``XEN_ELFNOTE_HYPERCALL_PAGE`` to have the nominated page written as a
+hypercall page during construction.  This mechanism is common for PV guests,
+and allows hypercalls to be issued with no additional setup.
+
+Any guest can locate the Xen CPUID leaves and read the *hypercall transfer
+page* information, which specifies an MSR that can be used to create
+additional hypercall pages.  When a guest physical address is written to the
+MSR, Xen writes a hypercall page into the nominated guest page.  This
+mechanism is common for HVM guests which are typically started via legacy
+means.
+
+
+.. rubric:: Footnotes
+
+.. [1] For HVM guests, ``HVMOP_guest_request_vm_event`` may be configured to
+       be usable from userspace, but this behaviour is not default.
+
+.. [2] While it is possible to use compatibility mode segments in a 64bit
+       kernel, hypercalls issues from such a mode will be interpreted with the
+       32bit ABI.  Such a setup is not expected in production scenarios.
+
+.. [3] ``HYPERCALL_iret`` is special.  It is only implemented for PV guests
+       and takes all its parameters on the stack.  This stub should be
+       ``jmp``'d to, rather than ``call``'d.  HVM guests have this stub
+       implemented as ``ud2a`` to prevent accidental use.
diff --git a/docs/guest-guide/x86/index.rst b/docs/guest-guide/x86/index.rst
new file mode 100644
index 0000000..a368392
--- /dev/null
+++ b/docs/guest-guide/x86/index.rst
@@ -0,0 +1,7 @@ 
+x86
+===
+
+.. toctree::
+  :maxdepth: 2
+
+  hypercall-abi
diff --git a/docs/index.rst b/docs/index.rst
index 9e2e256..31bb892 100644
--- a/docs/index.rst
+++ b/docs/index.rst
@@ -21,6 +21,19 @@  configure the system.
   admin-guide/index
 
 
+Guest documentation
+-------------------
+
+This documentation concerns the APIs and ABIs available to guests.  It is
+intended for OS developers trying to use a Xen feature, and for Xen developers
+to avoid breaking things.
+
+.. toctree::
+  :maxdepth: 3
+
+  guest-guide/index
+
+
 Hypervisor developer documentation
 ----------------------------------