mbox series

[v2,00/11] KVM: arm64: Implement PSCI SYSTEM_SUSPEND support

Message ID 20210923191610.3814698-1-oupton@google.com (mailing list archive)
Headers show
Series KVM: arm64: Implement PSCI SYSTEM_SUSPEND support | expand

Message

Oliver Upton Sept. 23, 2021, 7:15 p.m. UTC
Certain VMMs/operators may wish to give their guests the ability to
initiate a system suspend that could result in the VM being saved to
persistent storage to be resumed at a later time. The PSCI v1.0
specification describes an SMC, SYSTEM_SUSPEND, that allows a kernel to
request a system suspend. This call is optional for v1.0, and KVM
elected to not support the call in its v1.0 implementation.

This series adds support for the SYSTEM_SUSPEND PSCI call to KVM/arm64.
Since this is a system-scoped event, KVM cannot quiesce the VM on its
own. We add a new system exit type in this series to clue in userspace
that a suspend was requested. Per the KVM_EXIT_SYSTEM_EVENT ABI, a VMM
that doesn't care about this event can simply resume the guest without
issue (we set up the calling vCPU to come out of reset correctly on next
KVM_RUN). If a VMM would like to have KVM emulate the suspend, it can do
so by setting the vCPU's MP state to KVM_MP_STATE_HALTED. Support for
this state has been added in this series.

Patch 1 is an unrelated cleanup, dropping an unused parameter

Patch 2 simplifies how KVM filters SMC64 functions for AArch32 guests.

Patch 3 wraps up the vCPU reset logic used by the PSCI CPU_ON
implementation in KVM for subsequent use, as we must queue up a reset
for the vCPU that requested a system suspend.

Patch 4 is another unrelated cleanup, fixing the naming for the
KVM_REQ_SLEEP handler to avoid confusion and remain consistent with the
handler introduced in this series.

Patch 5 changes how WFI-like events are handled in KVM (WFI instruction,
PSCI CPU_SUSPEND). Instead of directly blocking the vCPU in the
respective handlers, set a request bit and block before resuming the
guest. WFI and PSCI CPU_SUSPEND do not require deferral of
kvm_vcpu_block(), but SYSTEM_SUSPEND does. Rather than adding a deferral
mechanism just for SYSTEM_SUSPEND, it is a bit cleaner to have all
blocking events just request the event.

Patch 6 actually adds PSCI SYSTEM_SUSPEND support to KVM, and adds the
necessary UAPI to pair with the call.

Patch 7 renames the PSCI selftest to something more generic, as we will
test more than just CPU_ON.

Patch 8 creates a common helper for making SMC64 calls in KVM selftests,
rather than having tests open-code their own approach.

Patch 9 makes the PSCI test use KVM_SET_MP_STATE for powering off a vCPU
rather than the vCPU init flag. This change is necessary to separate
generic VM setup from the setup for a particular PSCI test.

Patch 10 reworks psci_test into a bunch of helpers, making it easier to
build additional test cases with the common parts.

Finally, patch 11 adds 2 test cases for the SYSTEM_SUSPEND PSCI call.
Verify that the call succeeds if all other vCPUs have been powered off
and that it fails if more than the calling vCPU is powered on.

This series applies cleanly to v5.15-rc2. Testing was performed on an
Ampere Mt. Jade system.

Oliver Upton (11):
  KVM: arm64: Drop unused vcpu param to kvm_psci_valid_affinity()
  KVM: arm64: Clean up SMC64 PSCI filtering for AArch32 guests
  KVM: arm64: Encapsulate reset request logic in a helper function
  KVM: arm64: Rename the KVM_REQ_SLEEP handler
  KVM: arm64: Defer WFI emulation as a requested event
  KVM: arm64: Add support for SYSTEM_SUSPEND PSCI call
  selftests: KVM: Rename psci_cpu_on_test to psci_test
  selftests: KVM: Create helper for making SMCCC calls
  selftests: KVM: Use KVM_SET_MP_STATE to power off vCPU in psci_test
  selftests: KVM: Refactor psci_test to make it amenable to new tests
  selftests: KVM: Test SYSTEM_SUSPEND PSCI call

 Documentation/virt/kvm/api.rst                |   6 +
 arch/arm64/include/asm/kvm_host.h             |   4 +
 arch/arm64/kvm/arm.c                          |  21 +-
 arch/arm64/kvm/handle_exit.c                  |   3 +-
 arch/arm64/kvm/psci.c                         | 138 ++++++++---
 include/uapi/linux/kvm.h                      |   2 +
 tools/testing/selftests/kvm/.gitignore        |   2 +-
 tools/testing/selftests/kvm/Makefile          |   2 +-
 .../selftests/kvm/aarch64/psci_cpu_on_test.c  | 121 ----------
 .../testing/selftests/kvm/aarch64/psci_test.c | 218 ++++++++++++++++++
 .../selftests/kvm/include/aarch64/processor.h |  22 ++
 .../selftests/kvm/lib/aarch64/processor.c     |  25 ++
 tools/testing/selftests/kvm/steal_time.c      |  13 +-
 13 files changed, 403 insertions(+), 174 deletions(-)
 delete mode 100644 tools/testing/selftests/kvm/aarch64/psci_cpu_on_test.c
 create mode 100644 tools/testing/selftests/kvm/aarch64/psci_test.c

Comments

Oliver Upton Sept. 23, 2021, 8:15 p.m. UTC | #1
On Thu, Sep 23, 2021 at 07:15:59PM +0000, Oliver Upton wrote:
> Certain VMMs/operators may wish to give their guests the ability to
> initiate a system suspend that could result in the VM being saved to
> persistent storage to be resumed at a later time. The PSCI v1.0
> specification describes an SMC, SYSTEM_SUSPEND, that allows a kernel to
> request a system suspend. This call is optional for v1.0, and KVM
> elected to not support the call in its v1.0 implementation.
> 
> This series adds support for the SYSTEM_SUSPEND PSCI call to KVM/arm64.
> Since this is a system-scoped event, KVM cannot quiesce the VM on its
> own. We add a new system exit type in this series to clue in userspace
> that a suspend was requested. Per the KVM_EXIT_SYSTEM_EVENT ABI, a VMM
> that doesn't care about this event can simply resume the guest without
> issue (we set up the calling vCPU to come out of reset correctly on next
> KVM_RUN). If a VMM would like to have KVM emulate the suspend, it can do
> so by setting the vCPU's MP state to KVM_MP_STATE_HALTED. Support for
> this state has been added in this series.
> 
> Patch 1 is an unrelated cleanup, dropping an unused parameter
> 
> Patch 2 simplifies how KVM filters SMC64 functions for AArch32 guests.
> 
> Patch 3 wraps up the vCPU reset logic used by the PSCI CPU_ON
> implementation in KVM for subsequent use, as we must queue up a reset
> for the vCPU that requested a system suspend.
> 
> Patch 4 is another unrelated cleanup, fixing the naming for the
> KVM_REQ_SLEEP handler to avoid confusion and remain consistent with the
> handler introduced in this series.
> 
> Patch 5 changes how WFI-like events are handled in KVM (WFI instruction,
> PSCI CPU_SUSPEND). Instead of directly blocking the vCPU in the
> respective handlers, set a request bit and block before resuming the
> guest. WFI and PSCI CPU_SUSPEND do not require deferral of
> kvm_vcpu_block(), but SYSTEM_SUSPEND does. Rather than adding a deferral
> mechanism just for SYSTEM_SUSPEND, it is a bit cleaner to have all
> blocking events just request the event.
> 
> Patch 6 actually adds PSCI SYSTEM_SUSPEND support to KVM, and adds the
> necessary UAPI to pair with the call.
> 
> Patch 7 renames the PSCI selftest to something more generic, as we will
> test more than just CPU_ON.
> 
> Patch 8 creates a common helper for making SMC64 calls in KVM selftests,
> rather than having tests open-code their own approach.
> 
> Patch 9 makes the PSCI test use KVM_SET_MP_STATE for powering off a vCPU
> rather than the vCPU init flag. This change is necessary to separate
> generic VM setup from the setup for a particular PSCI test.
> 
> Patch 10 reworks psci_test into a bunch of helpers, making it easier to
> build additional test cases with the common parts.
> 
> Finally, patch 11 adds 2 test cases for the SYSTEM_SUSPEND PSCI call.
> Verify that the call succeeds if all other vCPUs have been powered off
> and that it fails if more than the calling vCPU is powered on.
> 
> This series applies cleanly to v5.15-rc2. Testing was performed on an
> Ampere Mt. Jade system.

Gah, forgot to summarize updates:

v1 -> v2:
 - Rebase to 5.15-rc2
 - Allow userspace to request in-kernel suspend emulation (Marc)
 - Add another test case for SYSTEM_SUSPEND, cleaning up the PSCI
   selftest
 - Create a common SMCCC function for KVM selftests

v1: http://lore.kernel.org/r/20210819223640.3564975-1-oupton@google.com

> Oliver Upton (11):
>   KVM: arm64: Drop unused vcpu param to kvm_psci_valid_affinity()
>   KVM: arm64: Clean up SMC64 PSCI filtering for AArch32 guests
>   KVM: arm64: Encapsulate reset request logic in a helper function
>   KVM: arm64: Rename the KVM_REQ_SLEEP handler
>   KVM: arm64: Defer WFI emulation as a requested event
>   KVM: arm64: Add support for SYSTEM_SUSPEND PSCI call
>   selftests: KVM: Rename psci_cpu_on_test to psci_test
>   selftests: KVM: Create helper for making SMCCC calls
>   selftests: KVM: Use KVM_SET_MP_STATE to power off vCPU in psci_test
>   selftests: KVM: Refactor psci_test to make it amenable to new tests
>   selftests: KVM: Test SYSTEM_SUSPEND PSCI call
> 
>  Documentation/virt/kvm/api.rst                |   6 +
>  arch/arm64/include/asm/kvm_host.h             |   4 +
>  arch/arm64/kvm/arm.c                          |  21 +-
>  arch/arm64/kvm/handle_exit.c                  |   3 +-
>  arch/arm64/kvm/psci.c                         | 138 ++++++++---
>  include/uapi/linux/kvm.h                      |   2 +
>  tools/testing/selftests/kvm/.gitignore        |   2 +-
>  tools/testing/selftests/kvm/Makefile          |   2 +-
>  .../selftests/kvm/aarch64/psci_cpu_on_test.c  | 121 ----------
>  .../testing/selftests/kvm/aarch64/psci_test.c | 218 ++++++++++++++++++
>  .../selftests/kvm/include/aarch64/processor.h |  22 ++
>  .../selftests/kvm/lib/aarch64/processor.c     |  25 ++
>  tools/testing/selftests/kvm/steal_time.c      |  13 +-
>  13 files changed, 403 insertions(+), 174 deletions(-)
>  delete mode 100644 tools/testing/selftests/kvm/aarch64/psci_cpu_on_test.c
>  create mode 100644 tools/testing/selftests/kvm/aarch64/psci_test.c
> 
> -- 
> 2.33.0.685.g46640cef36-goog
>