| Message ID | 20210408131010.1109027-4-maz@kernel.org (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | arm64: Dealing with VHE-only CPUs | expand |
On Thu, Apr 08, 2021 at 02:10:10PM +0100, Marc Zyngier wrote: > CONFIG_ARM64_VHE was introduced with ARMv8.1 (some 7 years ago), > and has been enabled by default for almost all that time. > > Given that newer systems that are VHE capable are finally becoming > available, and that some systems are even incapable of not running VHE, > drop the configuration altogether. > > Anyone willing to stick to non-VHE on VHE hardware for obscure > reasons should use the 'kvm-arm.mode=nvhe' command-line option. > > Suggested-by: Will Deacon <will@kernel.org> > Signed-off-by: Marc Zyngier <maz@kernel.org> > --- > .../admin-guide/kernel-parameters.txt | 3 +-- > arch/arm64/Kconfig | 20 ------------------- > arch/arm64/kernel/cpufeature.c | 4 ---- > arch/arm64/kernel/hyp-stub.S | 2 -- > 4 files changed, 1 insertion(+), 28 deletions(-) Huh, I was really expecting to see ARM64_VHE crop up in some Makefiles, but it doesn't seem to! So: Acked-by: Will Deacon <will@kernel.org> Will
On Thu, 08 Apr 2021 17:59:14 +0100, Will Deacon <will@kernel.org> wrote: > > On Thu, Apr 08, 2021 at 02:10:10PM +0100, Marc Zyngier wrote: > > CONFIG_ARM64_VHE was introduced with ARMv8.1 (some 7 years ago), > > and has been enabled by default for almost all that time. > > > > Given that newer systems that are VHE capable are finally becoming > > available, and that some systems are even incapable of not running VHE, > > drop the configuration altogether. > > > > Anyone willing to stick to non-VHE on VHE hardware for obscure > > reasons should use the 'kvm-arm.mode=nvhe' command-line option. > > > > Suggested-by: Will Deacon <will@kernel.org> > > Signed-off-by: Marc Zyngier <maz@kernel.org> > > --- > > .../admin-guide/kernel-parameters.txt | 3 +-- > > arch/arm64/Kconfig | 20 ------------------- > > arch/arm64/kernel/cpufeature.c | 4 ---- > > arch/arm64/kernel/hyp-stub.S | 2 -- > > 4 files changed, 1 insertion(+), 28 deletions(-) > > Huh, I was really expecting to see ARM64_VHE crop up in some Makefiles, but > it doesn't seem to! So: It was a design decision from the very beginning that all the VHE code would always be compiled in to avoid bitrot. It probably helped weeding out some early bugs. > Acked-by: Will Deacon <will@kernel.org> Thanks, M.
On Thu, Apr 8, 2021 at 3:10 PM Marc Zyngier <maz@kernel.org> wrote: > > CONFIG_ARM64_VHE was introduced with ARMv8.1 (some 7 years ago), > and has been enabled by default for almost all that time. > > Given that newer systems that are VHE capable are finally becoming > available, and that some systems are even incapable of not running VHE, > drop the configuration altogether. > > Anyone willing to stick to non-VHE on VHE hardware for obscure > reasons should use the 'kvm-arm.mode=nvhe' command-line option. Have you considered adding options to do the reverse logic for this and other features, such as making support for the old non-VHE optional at compile time? I understand that so far the rule is (almost) always that an arm64 kernel should run on any Armv8.0-A or higher system regardless of configuration, but the now announced Armv9.0-A definition might be the chance to introduce the concept of a minimum level the way we do on other architectures (e.g. armv6/v6k/v7 or k8/pentium4/core2/atom/generic). The way I can see this working would be to have a single user-visible option that controls whether the kernel supports only Armv9.0-A/Armv8.5-A and assumes all mandatory features of that are present, or it remains as before and supports all implementations back to the first v8. This would help eliminate the runtime detection for not just VHE but also LSE, LPA, PAN, etc. Not sure how significant the cost of any of those are in terms of runtime performance and/or code size, but it would feel nice to be able to build a kernel that can actually rely on sane hardware features even if it will take a few more years before that hardware becomes common enough to actually get some distros ship a kernel that requires v8.5/v9.0. Arnd
On Thu, 08 Apr 2021 22:47:38 +0100, Arnd Bergmann <arnd@kernel.org> wrote: > > On Thu, Apr 8, 2021 at 3:10 PM Marc Zyngier <maz@kernel.org> wrote: > > > > CONFIG_ARM64_VHE was introduced with ARMv8.1 (some 7 years ago), > > and has been enabled by default for almost all that time. > > > > Given that newer systems that are VHE capable are finally becoming > > available, and that some systems are even incapable of not running VHE, > > drop the configuration altogether. > > > > Anyone willing to stick to non-VHE on VHE hardware for obscure > > reasons should use the 'kvm-arm.mode=nvhe' command-line option. > > Have you considered adding options to do the reverse logic for this > and other features, such as making support for the old non-VHE > optional at compile time? This nVHE code is exclusively limited to KVM, because the whole point of VHE is that the EL1 kernel can run at EL2 completely unchanged. Furthermore, nVHE has properties that VHE cannot deliver, such as memory isolation between host and guests. So no, I wouldn't consider make it optional at compile time. Instead, I would consider jettisoning the nVHE EL2 code after init to save a handful of pages when running on a VHE-capable system in VHE mode. > > I understand that so far the rule is (almost) always that an arm64 kernel > should run on any Armv8.0-A or higher system regardless of configuration, > but the now announced Armv9.0-A definition might be the chance to > introduce the concept of a minimum level the way we do on other > architectures (e.g. armv6/v6k/v7 or k8/pentium4/core2/atom/generic). > > The way I can see this working would be to have a single user-visible > option that controls whether the kernel supports only Armv9.0-A/Armv8.5-A > and assumes all mandatory features of that are present, or it remains > as before and supports all implementations back to the first v8. The view that v8.5/v9 is a monolithic setup with a set of mandatory option is unfortunately disconnected from reality. Reality is that an implementer picks and chooses whatever feature set they want irrespective of what the architecture says, and the architecture revision is nothing but an index into the documentation. Features that were mandatory often become optional, and/or are further made unimplementable due to some other extension of the architecture. We also have the extremely common case where a feature that is usable on a host isn't virtualisable, meaning that the kernel gets exposed different feature sets depending on whether it runs bare-metal or not, which puts another nail on the "per-revision feature set" coffin. The ARM architecture is effectively a mix-and-match system, where you choose the feature set you want for a single vertically integrated application, and not a nice linear progression of features that land in bulk. > This would help eliminate the runtime detection for not just VHE but > also LSE, LPA, PAN, etc. Not sure how significant the cost of any of > those are in terms of runtime performance and/or code size, but it > would feel nice to be able to build a kernel that can actually rely > on sane hardware features even if it will take a few more years before > that hardware becomes common enough to actually get some distros > ship a kernel that requires v8.5/v9.0. We went there with 32bit because we were forced to: different ISAs, different page table formats. I would rather keep a single kernel until we get to that point of divergence *or* that we can demonstrate such a significant performance/maintenance improvement that it'd be silly not to do it. The published architecture does not hint at such changes yet, nor has performance overhead been reported due to "feature bloat". The current feature discovery and runtime patching seems adequate and saves us from the bitrot effect that multiple configurations inevitably would trigger. So at the moment, I am strongly in favor of keeping the current "single kernel" approach, and even drop more of the existing configuration symbols (though some have ugly toolchain constraints). Thanks, M.
diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt index 04545725f187..18f8bb3024f4 100644 --- a/Documentation/admin-guide/kernel-parameters.txt +++ b/Documentation/admin-guide/kernel-parameters.txt @@ -2279,8 +2279,7 @@ state is kept private from the host. Not valid if the kernel is running in EL2. - Defaults to VHE/nVHE based on hardware support and - the value of CONFIG_ARM64_VHE. + Defaults to VHE/nVHE based on hardware support. kvm-arm.vgic_v3_group0_trap= [KVM,ARM] Trap guest accesses to GICv3 group-0 diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 5656e7aacd69..e9fbb0b45793 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -1416,19 +1416,6 @@ config ARM64_USE_LSE_ATOMICS built with binutils >= 2.25 in order for the new instructions to be used. -config ARM64_VHE - bool "Enable support for Virtualization Host Extensions (VHE)" - default y - help - Virtualization Host Extensions (VHE) allow the kernel to run - directly at EL2 (instead of EL1) on processors that support - it. This leads to better performance for KVM, as they reduce - the cost of the world switch. - - Selecting this option allows the VHE feature to be detected - at runtime, and does not affect processors that do not - implement this feature. - endmenu menu "ARMv8.2 architectural features" @@ -1684,7 +1671,6 @@ endmenu config ARM64_SVE bool "ARM Scalable Vector Extension support" default y - depends on !KVM || ARM64_VHE help The Scalable Vector Extension (SVE) is an extension to the AArch64 execution state which complements and extends the SIMD functionality @@ -1713,12 +1699,6 @@ config ARM64_SVE booting the kernel. If unsure and you are not observing these symptoms, you should assume that it is safe to say Y. - CPUs that support SVE are architecturally required to support the - Virtualization Host Extensions (VHE), so the kernel makes no - provision for supporting SVE alongside KVM without VHE enabled. - Thus, you will need to enable CONFIG_ARM64_VHE if you want to support - KVM in the same kernel image. - config ARM64_MODULE_PLTS bool "Use PLTs to allow module memory to spill over into vmalloc area" depends on MODULES diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index 6de15deaa912..fa6023ac3548 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -1623,7 +1623,6 @@ int get_cpu_with_amu_feat(void) } #endif -#ifdef CONFIG_ARM64_VHE static bool runs_at_el2(const struct arm64_cpu_capabilities *entry, int __unused) { return is_kernel_in_hyp_mode(); @@ -1642,7 +1641,6 @@ static void cpu_copy_el2regs(const struct arm64_cpu_capabilities *__unused) if (!alternative_is_applied(ARM64_HAS_VIRT_HOST_EXTN)) write_sysreg(read_sysreg(tpidr_el1), tpidr_el2); } -#endif static void cpu_has_fwb(const struct arm64_cpu_capabilities *__unused) { @@ -1845,7 +1843,6 @@ static const struct arm64_cpu_capabilities arm64_features[] = { .type = ARM64_CPUCAP_WEAK_LOCAL_CPU_FEATURE, .matches = has_no_hw_prefetch, }, -#ifdef CONFIG_ARM64_VHE { .desc = "Virtualization Host Extensions", .capability = ARM64_HAS_VIRT_HOST_EXTN, @@ -1853,7 +1850,6 @@ static const struct arm64_cpu_capabilities arm64_features[] = { .matches = runs_at_el2, .cpu_enable = cpu_copy_el2regs, }, -#endif /* CONFIG_ARM64_VHE */ { .desc = "32-bit EL0 Support", .capability = ARM64_HAS_32BIT_EL0, diff --git a/arch/arm64/kernel/hyp-stub.S b/arch/arm64/kernel/hyp-stub.S index 9c9b4d8fc395..74ad3db061d1 100644 --- a/arch/arm64/kernel/hyp-stub.S +++ b/arch/arm64/kernel/hyp-stub.S @@ -224,7 +224,6 @@ SYM_FUNC_END(__hyp_reset_vectors) * Entry point to switch to VHE if deemed capable */ SYM_FUNC_START(switch_to_vhe) -#ifdef CONFIG_ARM64_VHE // Need to have booted at EL2 adr_l x1, __boot_cpu_mode ldr w0, [x1] @@ -240,6 +239,5 @@ SYM_FUNC_START(switch_to_vhe) mov x0, #HVC_VHE_RESTART hvc #0 1: -#endif ret SYM_FUNC_END(switch_to_vhe)
CONFIG_ARM64_VHE was introduced with ARMv8.1 (some 7 years ago), and has been enabled by default for almost all that time. Given that newer systems that are VHE capable are finally becoming available, and that some systems are even incapable of not running VHE, drop the configuration altogether. Anyone willing to stick to non-VHE on VHE hardware for obscure reasons should use the 'kvm-arm.mode=nvhe' command-line option. Suggested-by: Will Deacon <will@kernel.org> Signed-off-by: Marc Zyngier <maz@kernel.org> --- .../admin-guide/kernel-parameters.txt | 3 +-- arch/arm64/Kconfig | 20 ------------------- arch/arm64/kernel/cpufeature.c | 4 ---- arch/arm64/kernel/hyp-stub.S | 2 -- 4 files changed, 1 insertion(+), 28 deletions(-)