| Message ID | 20201027172705.15181-5-alexandru.elisei@arm.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | KVM: arm64: Add Statistical Profiling Extension (SPE) support | expand |
Hi Alex, On 27/10/2020 17:26, Alexandru Elisei wrote: > Detect Statistical Profiling Extension (SPE) support using the cpufeatures > framework. The presence of SPE is reported via the ARM64_SPE capability. > > The feature will be necessary for emulating SPE in KVM, because KVM needs > that all CPUs have SPE hardware to avoid scheduling a VCPU on a CPU without > support. For this reason, the feature type ARM64_CPUCAP_SYSTEM_FEATURE has > been selected to disallow hotplugging a CPU which doesn't support SPE. Can you mention the existing driver in the commit message? Surprisingly it doesn't use cpufeature at all. It looks like arm_spe_pmu_dev_init() goes out of its way to support mismatched systems. (otherwise the significance of the new behaviour isn't clear!) I read it as: the host is fine with mismatched systems, and the existing drivers supports this. But KVM is not. After this patch you can't make the system mismatched 'late'. Thanks, James
Hi James, On 11/19/20 4:58 PM, James Morse wrote: > Hi Alex, > > On 27/10/2020 17:26, Alexandru Elisei wrote: >> Detect Statistical Profiling Extension (SPE) support using the cpufeatures >> framework. The presence of SPE is reported via the ARM64_SPE capability. >> >> The feature will be necessary for emulating SPE in KVM, because KVM needs >> that all CPUs have SPE hardware to avoid scheduling a VCPU on a CPU without >> support. For this reason, the feature type ARM64_CPUCAP_SYSTEM_FEATURE has >> been selected to disallow hotplugging a CPU which doesn't support SPE. > Can you mention the existing driver in the commit message? Surprisingly it doesn't use > cpufeature at all. It looks like arm_spe_pmu_dev_init() goes out of its way to support > mismatched systems. (otherwise the significance of the new behaviour isn't clear!) > > I read it as: the host is fine with mismatched systems, and the existing drivers supports > this. But KVM is not. After this patch you can't make the system mismatched 'late'. That was exactly my intention. Certainly, I'll try to make the commit message clearer by mentioning the SPE driver. Thanks, Alex
On Wed, Dec 02, 2020 at 02:29:31PM +0000, Alexandru Elisei wrote: > On 11/19/20 4:58 PM, James Morse wrote: > > On 27/10/2020 17:26, Alexandru Elisei wrote: > >> Detect Statistical Profiling Extension (SPE) support using the cpufeatures > >> framework. The presence of SPE is reported via the ARM64_SPE capability. > >> > >> The feature will be necessary for emulating SPE in KVM, because KVM needs > >> that all CPUs have SPE hardware to avoid scheduling a VCPU on a CPU without > >> support. For this reason, the feature type ARM64_CPUCAP_SYSTEM_FEATURE has > >> been selected to disallow hotplugging a CPU which doesn't support SPE. > > Can you mention the existing driver in the commit message? Surprisingly it doesn't use > > cpufeature at all. It looks like arm_spe_pmu_dev_init() goes out of its way to support > > mismatched systems. (otherwise the significance of the new behaviour isn't clear!) > > > > I read it as: the host is fine with mismatched systems, and the existing drivers supports > > this. But KVM is not. After this patch you can't make the system mismatched 'late'. > > That was exactly my intention. Certainly, I'll try to make the commit message > clearer by mentioning the SPE driver. Hmm, so are you saying that with this patch applied, a machine where KVM isn't even being used can no longer late-online CPUs without SPE if the boot CPUs had it? If so, then I don't think that's acceptable, unfortunately. As James points out, the current driver is very careful to support big.LITTLE misconfigurations and I don't see why KVM support should change that. Will
Hi Will, On 12/2/20 5:23 PM, Will Deacon wrote: > On Wed, Dec 02, 2020 at 02:29:31PM +0000, Alexandru Elisei wrote: >> On 11/19/20 4:58 PM, James Morse wrote: >>> On 27/10/2020 17:26, Alexandru Elisei wrote: >>>> Detect Statistical Profiling Extension (SPE) support using the cpufeatures >>>> framework. The presence of SPE is reported via the ARM64_SPE capability. >>>> >>>> The feature will be necessary for emulating SPE in KVM, because KVM needs >>>> that all CPUs have SPE hardware to avoid scheduling a VCPU on a CPU without >>>> support. For this reason, the feature type ARM64_CPUCAP_SYSTEM_FEATURE has >>>> been selected to disallow hotplugging a CPU which doesn't support SPE. >>> Can you mention the existing driver in the commit message? Surprisingly it doesn't use >>> cpufeature at all. It looks like arm_spe_pmu_dev_init() goes out of its way to support >>> mismatched systems. (otherwise the significance of the new behaviour isn't clear!) >>> >>> I read it as: the host is fine with mismatched systems, and the existing drivers supports >>> this. But KVM is not. After this patch you can't make the system mismatched 'late'. >> That was exactly my intention. Certainly, I'll try to make the commit message >> clearer by mentioning the SPE driver. > Hmm, so are you saying that with this patch applied, a machine where KVM > isn't even being used can no longer late-online CPUs without SPE if the boot > CPUs had it? If so, then I don't think that's acceptable, unfortunately. Yes, the idea was to prevent hotplugging CPUs that don't have the capability so the kernel won't schedule a SPE-enabled guest on a CPU which doesn't have SPE. > > As James points out, the current driver is very careful to support > big.LITTLE misconfigurations and I don't see why KVM support should change > that. That makes sense, thank you for making it clear from the start that this approach is not the right one. There was a discussion for supporting KVM SPE on heterogeneous systems [1]. I chose to use a capability because the focus for this iteration was to ensure the correctness of SPE emulation, and the capability looked like the easiest way to get KVM SPE up and running for testing. The idea discussed previously [1] was to have userspace configure the VM with a cpumask representing the CPUs the VM is allowed to run on. KVM detects if the VCPU is scheduled on a physical CPU not in the cpumask, and returns from KVM_RUN with an error code. That looks like a good solution to me and generic enough that it can be used for all sorts of mismatched features. I will try to implement it in the next iteration, after I get more feedback on the current series. [1] https://www.spinics.net/lists/arm-kernel/msg778477.html Thanks, Alex
diff --git a/arch/arm64/include/asm/cpucaps.h b/arch/arm64/include/asm/cpucaps.h index 42868dbd29fd..10fd094d9a5b 100644 --- a/arch/arm64/include/asm/cpucaps.h +++ b/arch/arm64/include/asm/cpucaps.h @@ -65,7 +65,8 @@ #define ARM64_HAS_ARMv8_4_TTL 55 #define ARM64_HAS_TLB_RANGE 56 #define ARM64_MTE 57 +#define ARM64_SPE 58 -#define ARM64_NCAPS 58 +#define ARM64_NCAPS 59 #endif /* __ASM_CPUCAPS_H */ diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index dcc165b3fc04..4a0f4dc53824 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -1278,6 +1278,18 @@ has_useable_cnp(const struct arm64_cpu_capabilities *entry, int scope) return has_cpuid_feature(entry, scope); } +static bool __maybe_unused +has_usable_spe(const struct arm64_cpu_capabilities *entry, int scope) +{ + u64 pmbidr; + + if (!has_cpuid_feature(entry, scope)) + return false; + + pmbidr = read_sysreg_s(SYS_PMBIDR_EL1); + return !(pmbidr & BIT(SYS_PMBIDR_EL1_P_SHIFT)); +} + /* * This check is triggered during the early boot before the cpufeature * is initialised. Checking the status on the local CPU allows the boot @@ -2003,6 +2015,18 @@ static const struct arm64_cpu_capabilities arm64_features[] = { .min_field_value = 1, .cpu_enable = cpu_enable_cnp, }, +#endif +#ifdef CONFIG_ARM_SPE_PMU + { + .desc = "Statistical Profiling Extension (SPE)", + .capability = ARM64_SPE, + .type = ARM64_CPUCAP_SYSTEM_FEATURE, + .matches = has_usable_spe, + .sys_reg = SYS_ID_AA64DFR0_EL1, + .sign = FTR_UNSIGNED, + .field_pos = ID_AA64DFR0_PMSVER_SHIFT, + .min_field_value = 1, + }, #endif { .desc = "Speculation barrier (SB)",
Detect Statistical Profiling Extension (SPE) support using the cpufeatures framework. The presence of SPE is reported via the ARM64_SPE capability. The feature will be necessary for emulating SPE in KVM, because KVM needs that all CPUs have SPE hardware to avoid scheduling a VCPU on a CPU without support. For this reason, the feature type ARM64_CPUCAP_SYSTEM_FEATURE has been selected to disallow hotplugging a CPU which doesn't support SPE. Signed-off-by: Alexandru Elisei <alexandru.elisei@arm.com> --- arch/arm64/include/asm/cpucaps.h | 3 ++- arch/arm64/kernel/cpufeature.c | 24 ++++++++++++++++++++++++ 2 files changed, 26 insertions(+), 1 deletion(-)