| Message ID | 20201027172705.15181-7-alexandru.elisei@arm.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | KVM: arm64: Add Statistical Profiling Extension (SPE) support | expand |
Hi Alex, On 27/10/2020 17:26, Alexandru Elisei wrote: > KVM SPE emulation depends on the configuration option KVM_ARM_SPE and on on > having hardware SPE support on all CPUs. > The host driver must be > compiled-in because we need the SPE interrupt to be enabled; it will be > used to kick us out of the guest when the profiling buffer management > interrupt is asserted by the GIC (for example, when the buffer is full). Great: SPE IRQ very important... > Add a VCPU flag to inform KVM that the guest has SPE enabled. > > It's worth noting that even though the KVM_ARM_SPE config option is gated > by the SPE host driver being compiled-in, we don't actually check that the > driver was loaded successfully when we advertise SPE support for guests. Eh? > That's because we can live with the SPE interrupt being disabled. There is > a delay between when the SPE hardware asserts the interrupt and when the > GIC samples the interrupt line and asserts it to the CPU. If the SPE > interrupt is disabled at the GIC level, this delay will be larger, How does this work? Surely the IRQ needs to be enabled before it can become pending at the CPU to kick us out of the guest... > at most a host timer tick. (Because the timer brings us out of the guest anyway?) Thanks, James
Hi James, On 11/19/20 4:58 PM, James Morse wrote: > Hi Alex, > > On 27/10/2020 17:26, Alexandru Elisei wrote: >> KVM SPE emulation depends on the configuration option KVM_ARM_SPE and on on >> having hardware SPE support on all CPUs. >> The host driver must be >> compiled-in because we need the SPE interrupt to be enabled; it will be >> used to kick us out of the guest when the profiling buffer management >> interrupt is asserted by the GIC (for example, when the buffer is full). > Great: SPE IRQ very important... Within reason. > > >> Add a VCPU flag to inform KVM that the guest has SPE enabled. >> >> It's worth noting that even though the KVM_ARM_SPE config option is gated >> by the SPE host driver being compiled-in, we don't actually check that the >> driver was loaded successfully when we advertise SPE support for guests. > Eh? Yes, this looks half-baked, and probably is, because: 1. I'm not sure I haven't missed anything with my approach to handling the SPE interrupt triggered by the guest (details in the cover letter). The other option would be to use the SPE driver IRQ handler, which makes this moot. 2. The SPE driver probing fails when the host has kpti enabled (the kernel can't profile userspace). In my opinion, this shouldn't affect SPE support for guests, but I didn't want to modify the SPE driver at this stage because of 1. If we agree on my approach to guest SPE interrupt handling, this patch can be improved by at least checking that the SPE driver probed successfully and taking the case where kpti is enabled into consideration. > >> That's because we can live with the SPE interrupt being disabled. There is >> a delay between when the SPE hardware asserts the interrupt and when the >> GIC samples the interrupt line and asserts it to the CPU. If the SPE >> interrupt is disabled at the GIC level, this delay will be larger, > How does this work? Surely the IRQ needs to be enabled before it can become pending at the > CPU to kick us out of the guest... As long as the SPE hardware asserts the buffer management interrupt to the GIC (PMBSR_EL1.S = 1), no profiling is done. If the interrupt is not enabled at the GIC level, then the CPU will not take the interrupt (obviously). But as far as the SPE hardware is concerned, the interrupt is asserted and profiling is disabled. The host checks the PMBSR_EL1.S bit on every VM exit to see if SPE asserts the interrupt and there's no dependency on the GIC asserting the interrupt. The SPE interrupt being disabled at the GIC level is not as bad as it sounds (but it's definitely not ideal) because there will always be a delay between the SPE hardware asserting the interrupt to the GIC and the GIC asserting it to the CPU. Not requiring the interrupt to be enabled at the GIC level makes that delay longer in the case where the host driver failed probing. > > >> at most a host timer tick. > (Because the timer brings us out of the guest anyway?) Yes, once very 4 ms according to the default value for CONFIG_HZ. Thanks, Alex
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index 25d326aecded..43eee197764f 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -406,6 +406,7 @@ struct kvm_vcpu_arch { #define KVM_ARM64_GUEST_HAS_SVE (1 << 5) /* SVE exposed to guest */ #define KVM_ARM64_VCPU_SVE_FINALIZED (1 << 6) /* SVE config completed */ #define KVM_ARM64_GUEST_HAS_PTRAUTH (1 << 7) /* PTRAUTH exposed to guest */ +#define KVM_ARM64_GUEST_HAS_SPE (1 << 8) /* SPE exposed to guest */ #define vcpu_has_sve(vcpu) (system_supports_sve() && \ ((vcpu)->arch.flags & KVM_ARM64_GUEST_HAS_SVE)) @@ -419,6 +420,14 @@ struct kvm_vcpu_arch { #define vcpu_has_ptrauth(vcpu) false #endif +#ifdef CONFIG_KVM_ARM_SPE +#define vcpu_has_spe(vcpu) \ + (cpus_have_final_cap(ARM64_SPE) && \ + ((vcpu)->arch.flags & KVM_ARM64_GUEST_HAS_SPE)) +#else +#define vcpu_has_spe(vcpu) false +#endif + #define vcpu_gp_regs(v) (&(v)->arch.ctxt.regs) /* diff --git a/arch/arm64/kvm/Kconfig b/arch/arm64/kvm/Kconfig index 043756db8f6e..8b35c0b806a7 100644 --- a/arch/arm64/kvm/Kconfig +++ b/arch/arm64/kvm/Kconfig @@ -57,6 +57,14 @@ config KVM_ARM_PMU Adds support for a virtual Performance Monitoring Unit (PMU) in virtual machines. +config KVM_ARM_SPE + bool "Virtual Statistical Profiling Extension (SPE) support" + depends on ARM_SPE_PMU + default y + help + Adds support for a virtual Statistical Profiling Extension (SPE) in + virtual machines. + endif # KVM endif # VIRTUALIZATION diff --git a/include/kvm/arm_spe.h b/include/kvm/arm_spe.h new file mode 100644 index 000000000000..db51ef15bf45 --- /dev/null +++ b/include/kvm/arm_spe.h @@ -0,0 +1,19 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (C) 2019 ARM Ltd. + */ + +#ifndef __ASM_ARM_KVM_SPE_H +#define __ASM_ARM_KVM_SPE_H + +#ifdef CONFIG_KVM_ARM_SPE +static inline bool kvm_arm_supports_spe(void) +{ + return cpus_have_final_cap(ARM64_SPE); +} + +#else +#define kvm_arm_supports_spe() false + +#endif /* CONFIG_KVM_ARM_SPE */ +#endif /* __ASM_ARM_KVM_SPE_H */
KVM SPE emulation depends on the configuration option KVM_ARM_SPE and on on having hardware SPE support on all CPUs. The host driver must be compiled-in because we need the SPE interrupt to be enabled; it will be used to kick us out of the guest when the profiling buffer management interrupt is asserted by the GIC (for example, when the buffer is full). Add a VCPU flag to inform KVM that the guest has SPE enabled. It's worth noting that even though the KVM_ARM_SPE config option is gated by the SPE host driver being compiled-in, we don't actually check that the driver was loaded successfully when we advertise SPE support for guests. That's because we can live with the SPE interrupt being disabled. There is a delay between when the SPE hardware asserts the interrupt and when the GIC samples the interrupt line and asserts it to the CPU. If the SPE interrupt is disabled at the GIC level, this delay will be larger, at most a host timer tick. Signed-off-by: Alexandru Elisei <alexandru.elisei@arm.com> --- arch/arm64/include/asm/kvm_host.h | 9 +++++++++ arch/arm64/kvm/Kconfig | 8 ++++++++ include/kvm/arm_spe.h | 19 +++++++++++++++++++ 3 files changed, 36 insertions(+) create mode 100644 include/kvm/arm_spe.h