[v2,00/14] Nested PAPR API (KVM on PowerVM)

Message

Harsh Prateek Bora Oct. 12, 2023, 10:49 a.m. UTC

There is an existing Nested-HV API to enable nested guests on powernv
machines. However, that is not supported on pseries/PowerVM LPARs.
This patch series implements required hcall interfaces to enable nested
guests with KVM on PowerVM.
Unlike Nested-HV, with this API, entire L2 state is retained by L0
during guest entry/exit and uses pre-defined Guest State Buffer (GSB)
format to communicate guest state between L1 and L2 via L0.

L0 here refers to the phyp/PowerVM, or launching a Qemu TCG L0 with the
newly introduced option cap-nested-papr=true.
L1 refers to the LPAR host on PowerVM or Linux booted on Qemu TCG with
above mentioned option cap-nested-papr=true.
L2 refers to nested guest running on top of L1 using KVM.
No SW changes needed for Qemu running in L1 Linux as well as L2 Kernel.

There is a Linux Kernel side patch series to enable support for Nested
PAPR in L1 and same can be found at below url:

Linux Kernel patch series:
- https://lore.kernel.org/linuxppc-dev/[e-mail]/

For more details, documentation can be referred in either of patch
series.

There are scripts available to assist in setting up an environment for
testing nested guests at https://github.com/iamjpn/kvm-powervm-test

A tree with this series is available at:
https://github.com/planetharsh/qemu/tree/upstream-kop-1012

Thanks to Michael Neuling, Shivaprasad Bhat, Kautuk Consul, Vaibhav Jain
and Jordan Niethe.

Changelog:

v2:
    - Addressed review comments from Nick on v1 series.
v1:
    - https://lore.kernel.org/qemu-devel/[e-mail]/

Harsh Prateek Bora (14):
  spapr: nested:  move nested part of spapr_get_pate into spapr_nested.c
  spapr: nested: Introduce SpaprMachineStateNested to store related
    info.
  spapr: nested: Document Nested PAPR API
  spapr: nested: Introduce cap-nested-papr for Nested PAPR API
  spapr: nested: register nested-hv api hcalls only for cap-nested-hv
  spapr: nested: Introduce H_GUEST_[GET|SET]_CAPABILITIES hcalls.
  spapr: nested: Introduce H_GUEST_[CREATE|DELETE] hcalls.
  spapr: nested: Introduce H_GUEST_CREATE_VPCU hcall.
  spapr: nested: Initialize the GSB elements lookup table.
  spapr: nested: Introduce H_GUEST_[GET|SET]_STATE hcalls.
  spapr: nested: Use correct source for parttbl info for nested PAPR
    API.
  spapr: nested: rename nested_host_state to nested_hv_host
  spapr: nested: keep nested-hv exit code restricted to its API.
  spapr: nested: Introduce H_GUEST_RUN_VCPU hcall.

 docs/devel/nested-papr.txt      |  500 +++++++++++
 hw/ppc/spapr.c                  |   32 +-
 hw/ppc/spapr_caps.c             |   63 ++
 hw/ppc/spapr_hcall.c            |    2 -
 hw/ppc/spapr_nested.c           | 1439 ++++++++++++++++++++++++++++++-
 include/hw/ppc/spapr.h          |   21 +-
 include/hw/ppc/spapr_cpu_core.h |    7 +-
 include/hw/ppc/spapr_nested.h   |  361 ++++++++
 target/ppc/cpu.h                |    2 +
 9 files changed, 2368 insertions(+), 59 deletions(-)
 create mode 100644 docs/devel/nested-papr.txt

Comments

Nicholas Piggin Nov. 13, 2023, 1:15 p.m. UTC | #1

This will have to wait until the next release. There should not be
big changes to rebase on. I'll take a better look before then.

Linux now has this merged upstream so it will be much easier to test.

I posted some RFCs for new avocado tests including a KVM guest boot
(https://lists.gnu.org/archive/html/qemu-ppc/2023-10/msg00260.html).
Byt I guess it won't be too easy to adapt that to test the new API
until there is a usable distro image with support. We should open an
issue for that.

Thanks,
Nick

On Thu Oct 12, 2023 at 8:49 PM AEST, Harsh Prateek Bora wrote:
> There is an existing Nested-HV API to enable nested guests on powernv
> machines. However, that is not supported on pseries/PowerVM LPARs.
> This patch series implements required hcall interfaces to enable nested
> guests with KVM on PowerVM.
> Unlike Nested-HV, with this API, entire L2 state is retained by L0
> during guest entry/exit and uses pre-defined Guest State Buffer (GSB)
> format to communicate guest state between L1 and L2 via L0.
>
> L0 here refers to the phyp/PowerVM, or launching a Qemu TCG L0 with the
> newly introduced option cap-nested-papr=true.
> L1 refers to the LPAR host on PowerVM or Linux booted on Qemu TCG with
> above mentioned option cap-nested-papr=true.
> L2 refers to nested guest running on top of L1 using KVM.
> No SW changes needed for Qemu running in L1 Linux as well as L2 Kernel.
>
> There is a Linux Kernel side patch series to enable support for Nested
> PAPR in L1 and same can be found at below url:
>
> Linux Kernel patch series:
> - https://lore.kernel.org/linuxppc-dev/[e-mail]/
>
> For more details, documentation can be referred in either of patch
> series.
>
> There are scripts available to assist in setting up an environment for
> testing nested guests at https://github.com/iamjpn/kvm-powervm-test
>
> A tree with this series is available at:
> https://github.com/planetharsh/qemu/tree/upstream-kop-1012
>
> Thanks to Michael Neuling, Shivaprasad Bhat, Kautuk Consul, Vaibhav Jain
> and Jordan Niethe.
>
> Changelog:
>
> v2:
>     - Addressed review comments from Nick on v1 series.
> v1:
>     - https://lore.kernel.org/qemu-devel/[e-mail]/
>
> Harsh Prateek Bora (14):
>   spapr: nested:  move nested part of spapr_get_pate into spapr_nested.c
>   spapr: nested: Introduce SpaprMachineStateNested to store related
>     info.
>   spapr: nested: Document Nested PAPR API
>   spapr: nested: Introduce cap-nested-papr for Nested PAPR API
>   spapr: nested: register nested-hv api hcalls only for cap-nested-hv
>   spapr: nested: Introduce H_GUEST_[GET|SET]_CAPABILITIES hcalls.
>   spapr: nested: Introduce H_GUEST_[CREATE|DELETE] hcalls.
>   spapr: nested: Introduce H_GUEST_CREATE_VPCU hcall.
>   spapr: nested: Initialize the GSB elements lookup table.
>   spapr: nested: Introduce H_GUEST_[GET|SET]_STATE hcalls.
>   spapr: nested: Use correct source for parttbl info for nested PAPR
>     API.
>   spapr: nested: rename nested_host_state to nested_hv_host
>   spapr: nested: keep nested-hv exit code restricted to its API.
>   spapr: nested: Introduce H_GUEST_RUN_VCPU hcall.
>
>  docs/devel/nested-papr.txt      |  500 +++++++++++
>  hw/ppc/spapr.c                  |   32 +-
>  hw/ppc/spapr_caps.c             |   63 ++
>  hw/ppc/spapr_hcall.c            |    2 -
>  hw/ppc/spapr_nested.c           | 1439 ++++++++++++++++++++++++++++++-
>  include/hw/ppc/spapr.h          |   21 +-
>  include/hw/ppc/spapr_cpu_core.h |    7 +-
>  include/hw/ppc/spapr_nested.h   |  361 ++++++++
>  target/ppc/cpu.h                |    2 +
>  9 files changed, 2368 insertions(+), 59 deletions(-)
>  create mode 100644 docs/devel/nested-papr.txt