[v7,00/10] arm64: Stolen time support
mbox series

Message ID 20191021152823.14882-1-steven.price@arm.com
Headers show
Series
  • arm64: Stolen time support
Related show

Message

Steven Price Oct. 21, 2019, 3:28 p.m. UTC
This series add support for paravirtualized time for arm64 guests and
KVM hosts following the specification in Arm's document DEN 0057A:

https://developer.arm.com/docs/den0057/a

It implements support for stolen time, allowing the guest to
identify time when it is forcibly not executing.

Note that Live Physical Time (LPT) which was previously part of the
above specification has now been removed.

Also available as a git tree:
git://linux-arm.org/linux-sp.git stolen_time/v7

Changes from v6:
https://lore.kernel.org/kvm/20191011125930.40834-1-steven.price@arm.com/
 * Rebased onto Mark Rutland's SMCCC cleanup
 * Added comment about putting stolen time structures in their own 64k
   page
 * Split 'init' functionality out from kvm_update_stolen_time()
 * Enforce address provided by user space for structure is in a valid
   memslot
 * Rename functions/structures in paravirt.c to avoid 'KVM' since the
   functionality isn't specific to KVM
 * Minor cleanups from review comments

Changes from v5:
https://lore.kernel.org/kvm/20191002145037.51630-1-steven.price@arm.com/
 * Convert document to RST format
 * Rename PV_FEATURES to PV_TIME_FEATURES to match spec
 * Correct SMC number of PV_TIME_ST

Changes from v4:
https://lore.kernel.org/kvm/20190830084255.55113-1-steven.price@arm.com/
 * Rebased to v5.4-rc1
 * Renamed KVM_ARM_VCPU_PVTIME_SET_IPA to remove _SET as it is used for
   both set/get operations
 * Added kvm/arm_hypercalls.h to header-test-$(CONFIG_ARM{,64}) as it is
   only buildable on arm/arm64
 * Documented no-steal-acc kernel parameter

Changes from v3:
https://lore.kernel.org/lkml/20190821153656.33429-1-steven.price@arm.com/
 * There's no longer a PV_TIME device, instead there are attributes on
   the VCPU. This allows the stolen time structures to be places
   arbitrarily by user space (subject to 64 byte alignment).
 * Split documentation between information on the hypercalls and the
   attributes on the VCPU
 * Fixed the type of SMCCC functions to return long not int

Changes from v2:
https://lore.kernel.org/lkml/20190819140436.12207-1-steven.price@arm.com/
 * Switched from using gfn_to_hva_cache to a new macro kvm_put_guest()
   that can provide the single-copy atomicity required (on arm64). This
   macro is added in patch 4.
 * Tidied up the locking for kvm_update_stolen_time().
   pagefault_disable() was unnecessary and the caller didn't need to
   take kvm->srcu as the function does it itself.
 * Removed struct kvm_arch_pvtime from the arm implementation, replaced
   instead with inline static functions which are empty for arm.
 * Fixed a few checkpatch --strict warnings.

Changes from v1:
https://lore.kernel.org/lkml/20190802145017.42543-1-steven.price@arm.com/
 * Host kernel no longer allocates the stolen time structure, instead it
   is allocated by user space. This means the save/restore functionality
   can be removed.
 * Refactored the code so arm has stub implementations and to avoid
   initcall
 * Rebased to pick up Documentation/{virt->virtual} change
 * Bunch of typo fixes

Christoffer Dall (1):
  KVM: arm/arm64: Factor out hypercall handling from PSCI code

Steven Price (9):
  KVM: arm64: Document PV-time interface
  KVM: arm64: Implement PV_TIME_FEATURES call
  KVM: Implement kvm_put_guest()
  KVM: arm64: Support stolen time reporting via shared structure
  KVM: Allow kvm_device_ops to be const
  KVM: arm64: Provide VCPU attributes for stolen time
  arm/arm64: Provide a wrapper for SMCCC 1.1 calls
  arm/arm64: Make use of the SMCCC 1.1 wrapper
  arm64: Retrieve stolen time as paravirtualized guest

 .../admin-guide/kernel-parameters.txt         |   6 +-
 Documentation/virt/kvm/arm/pvtime.rst         |  81 ++++++++++
 Documentation/virt/kvm/devices/vcpu.txt       |  14 ++
 arch/arm/include/asm/kvm_host.h               |  25 ++++
 arch/arm/kvm/Makefile                         |   2 +-
 arch/arm/kvm/handle_exit.c                    |   2 +-
 arch/arm/mm/proc-v7-bugs.c                    |  13 +-
 arch/arm64/include/asm/kvm_host.h             |  29 ++++
 arch/arm64/include/asm/paravirt.h             |   9 +-
 arch/arm64/include/asm/pvclock-abi.h          |  17 +++
 arch/arm64/include/uapi/asm/kvm.h             |   2 +
 arch/arm64/kernel/cpu_errata.c                |  81 ++++------
 arch/arm64/kernel/paravirt.c                  | 140 ++++++++++++++++++
 arch/arm64/kernel/time.c                      |   3 +
 arch/arm64/kvm/Kconfig                        |   1 +
 arch/arm64/kvm/Makefile                       |   2 +
 arch/arm64/kvm/guest.c                        |   9 ++
 arch/arm64/kvm/handle_exit.c                  |   4 +-
 include/Kbuild                                |   2 +
 include/kvm/arm_hypercalls.h                  |  43 ++++++
 include/kvm/arm_psci.h                        |   2 +-
 include/linux/arm-smccc.h                     |  59 ++++++++
 include/linux/cpuhotplug.h                    |   1 +
 include/linux/kvm_host.h                      |  26 +++-
 include/linux/kvm_types.h                     |   2 +
 include/uapi/linux/kvm.h                      |   2 +
 virt/kvm/arm/arm.c                            |  11 ++
 virt/kvm/arm/hypercalls.c                     |  71 +++++++++
 virt/kvm/arm/psci.c                           |  84 +----------
 virt/kvm/arm/pvtime.c                         | 131 ++++++++++++++++
 virt/kvm/kvm_main.c                           |   6 +-
 31 files changed, 724 insertions(+), 156 deletions(-)
 create mode 100644 Documentation/virt/kvm/arm/pvtime.rst
 create mode 100644 arch/arm64/include/asm/pvclock-abi.h
 create mode 100644 include/kvm/arm_hypercalls.h
 create mode 100644 virt/kvm/arm/hypercalls.c
 create mode 100644 virt/kvm/arm/pvtime.c

Comments

Marc Zyngier Oct. 21, 2019, 6:37 p.m. UTC | #1
On Mon, 21 Oct 2019 16:28:13 +0100,
Steven Price <steven.price@arm.com> wrote:
> 
> This series add support for paravirtualized time for arm64 guests and
> KVM hosts following the specification in Arm's document DEN 0057A:
> 
> https://developer.arm.com/docs/den0057/a
> 
> It implements support for stolen time, allowing the guest to
> identify time when it is forcibly not executing.
> 
> Note that Live Physical Time (LPT) which was previously part of the
> above specification has now been removed.
> 
> Also available as a git tree:
> git://linux-arm.org/linux-sp.git stolen_time/v7

I've provisionally applied this series on its own branch, and merged
it into kvmarm/next. It's not a done deal though, as I'd like Will or
Catalin's Ack on patches #9 and #10, as well as Russell's Ack for #9.

Thanks,

        M.
Marc Zyngier Oct. 23, 2019, 12:39 p.m. UTC | #2
Hi Steven,

On 2019-10-21 16:28, Steven Price wrote:
> This series add support for paravirtualized time for arm64 guests and
> KVM hosts following the specification in Arm's document DEN 0057A:
>
> https://developer.arm.com/docs/den0057/a
>
> It implements support for stolen time, allowing the guest to
> identify time when it is forcibly not executing.
>
> Note that Live Physical Time (LPT) which was previously part of the
> above specification has now been removed.
>
> Also available as a git tree:
> git://linux-arm.org/linux-sp.git stolen_time/v7

Can you please point me to userspace patches that I could apply to
kvmtool? I'd like to give this series a go as part of my normal 
testing.

Thanks,

         M.
Steven Price Oct. 23, 2019, 1:50 p.m. UTC | #3
On 23/10/2019 13:39, Marc Zyngier wrote:
> Hi Steven,
> 
> On 2019-10-21 16:28, Steven Price wrote:
>> This series add support for paravirtualized time for arm64 guests and
>> KVM hosts following the specification in Arm's document DEN 0057A:
>>
>> https://developer.arm.com/docs/den0057/a
>>
>> It implements support for stolen time, allowing the guest to
>> identify time when it is forcibly not executing.
>>
>> Note that Live Physical Time (LPT) which was previously part of the
>> above specification has now been removed.
>>
>> Also available as a git tree:
>> git://linux-arm.org/linux-sp.git stolen_time/v7
> 
> Can you please point me to userspace patches that I could apply to
> kvmtool? I'd like to give this series a go as part of my normal testing.

I don't have a proper patch yet, but the below is what I've been testing
with (this breaks with kernels without PV-time).

Steve

----8<----
From 8f6540cfbe2842f3ee422c07fbd1f590534cc90a Mon Sep 17 00:00:00 2001
From: Steven Price <steven.price@arm.com>
Date: Thu, 8 Nov 2018 13:43:56 +0000
Subject: [PATCH kvmtool] ARM PV time support

Signed-off-by: Steven Price <steven.price@arm.com>
---
 Makefile                 |  2 +-
 arm/kvm-cpu.c            |  4 ++++
 arm/pvtime.c             | 47 ++++++++++++++++++++++++++++++++++++++++
 include/kvm/arm-pvtime.h |  6 +++++
 4 files changed, 58 insertions(+), 1 deletion(-)
 create mode 100644 arm/pvtime.c
 create mode 100644 include/kvm/arm-pvtime.h

diff --git a/Makefile b/Makefile
index 3862112..a79956b 100644
--- a/Makefile
+++ b/Makefile
@@ -158,7 +158,7 @@ endif
 # ARM
 OBJS_ARM_COMMON		:= arm/fdt.o arm/gic.o arm/gicv2m.o arm/ioport.o \
 			   arm/kvm.o arm/kvm-cpu.o arm/pci.o arm/timer.o \
-			   arm/pmu.o
+			   arm/pmu.o arm/pvtime.o
 HDRS_ARM_COMMON		:= arm/include
 ifeq ($(ARCH), arm)
 	DEFINES		+= -DCONFIG_ARM
diff --git a/arm/kvm-cpu.c b/arm/kvm-cpu.c
index 7780251..c903b05 100644
--- a/arm/kvm-cpu.c
+++ b/arm/kvm-cpu.c
@@ -1,5 +1,6 @@
 #include "kvm/kvm.h"
 #include "kvm/kvm-cpu.h"
+#include "kvm/arm-pvtime.h"
 
 static int debug_fd;
 
@@ -122,6 +123,9 @@ struct kvm_cpu *kvm_cpu__arch_init(struct kvm *kvm, unsigned long cpu_id)
 	vcpu->cpu_compatible	= target->compatible;
 	vcpu->is_running	= true;
 
+	if (pvtime_vcpu_init(vcpu))
+		die("Unable to initialise pvtime");
+
 	return vcpu;
 }
 
diff --git a/arm/pvtime.c b/arm/pvtime.c
new file mode 100644
index 0000000..fc152d3
--- /dev/null
+++ b/arm/pvtime.c
@@ -0,0 +1,47 @@
+#include "kvm/kvm.h"
+#include "kvm/kvm-cpu.h"
+#include "kvm/arm-pvtime.h"
+
+#define PVTIME_BASE 0x10000000
+
+#define KVM_ARM_VCPU_PVTIME_CTRL	2
+#define   KVM_ARM_VCPU_PVTIME_IPA	0
+
+static int pvtime_init(struct kvm *kvm)
+{
+	char *mem;
+	int size = ALIGN(64 * kvm->cfg.nrcpus, PAGE_SIZE);
+
+	mem = mmap(NULL, size, PROT_RW, MAP_ANON_NORESERVE, -1, 0);
+	if (mem == MAP_FAILED)
+		return -ENOMEM;
+
+	if (kvm__register_dev_mem(kvm, PVTIME_BASE, size, mem))
+		return -EINVAL;
+
+	return 0;
+}
+
+base_init(pvtime_init);
+
+int pvtime_vcpu_init(struct kvm_cpu *vcpu)
+{
+	int err;
+	u64 st_paddr = PVTIME_BASE;
+
+	st_paddr += vcpu->cpu_id * 64;
+
+	struct kvm_device_attr st_base = {
+		.group = KVM_ARM_VCPU_PVTIME_CTRL,
+		.attr = KVM_ARM_VCPU_PVTIME_IPA,
+		.addr = (u64)&st_paddr
+	};
+
+	err = ioctl(vcpu->vcpu_fd, KVM_SET_DEVICE_ATTR, &st_base);
+	if (err) {
+		perror("ioctl st_base failed");
+		return err;
+	}
+
+	return 0;
+}
diff --git a/include/kvm/arm-pvtime.h b/include/kvm/arm-pvtime.h
new file mode 100644
index 0000000..f5db286
--- /dev/null
+++ b/include/kvm/arm-pvtime.h
@@ -0,0 +1,6 @@
+#ifndef ARM_PVTIME_H
+#define ARM_PVTIME_H
+
+int pvtime_vcpu_init(struct kvm_cpu *vcpu);
+
+#endif