| Message ID | 20180917104144.19188-17-suzuki.poulose@arm.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | kvm: arm64: Dynamic IPA and 52bit IPA | expand |
Hi Suzuki, On 9/17/18 12:41 PM, Suzuki K Poulose wrote: > So far we have restricted the IPA size of the VM to the default > value (40bits). Now that we can manage the IPA size per VM and > support dynamic stage2 page tables, we can allow VMs to have > larger IPA. This patch introduces a the maximum IPA size > supported on the host. to be reworded This is decided by the following factors : > > 1) Maximum PARange supported by the CPUs - This can be inferred > from the system wide safe value. > 2) Maximum PA size supported by the host kernel (48 vs 52) > 3) Number of levels in the host page table (as we base our > stage2 tables on the host table helpers). > > Since the stage2 page table code is dependent on the stage1 > page table, we always ensure that : > > Number of Levels at Stage1 >= Number of Levels at Stage2 > > So we limit the IPA to make sure that the above condition > is satisfied. This will affect the following combinations > of VA_BITS and IPA for different page sizes. > > Host configuration | Unsupported IPA ranges > 39bit VA, 4K | [44, 48] > 36bit VA, 16K | [41, 48] > 42bit VA, 64K | [47, 52] > > Supporting the above combinations need independent stage2 > page table manipulation code, which would need substantial > changes. We could purse the solution independently and > switch the page table code once we have it ready. > > Cc: Catalin Marinas <catalin.marinas@arm.com> > Cc: Marc Zyngier <marc.zyngier@arm.com> > Cc: Christoffer Dall <cdall@kernel.org> > Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com> > --- > Changes since V2: > - Restrict the IPA size to limit the number of page table > levels in stage2 to that of stage1 or less. > --- > arch/arm/include/asm/kvm_mmu.h | 2 ++ > arch/arm64/include/asm/kvm_host.h | 2 ++ > arch/arm64/kvm/reset.c | 43 +++++++++++++++++++++++++++++++ > virt/kvm/arm/arm.c | 2 ++ > 4 files changed, 49 insertions(+) > > diff --git a/arch/arm/include/asm/kvm_mmu.h b/arch/arm/include/asm/kvm_mmu.h > index 12ae5fbbcf01..5ad1a54f98dc 100644 > --- a/arch/arm/include/asm/kvm_mmu.h > +++ b/arch/arm/include/asm/kvm_mmu.h > @@ -358,6 +358,8 @@ static inline int hyp_map_aux_data(void) > > #define kvm_phys_to_vttbr(addr) (addr) > > +static inline void kvm_set_ipa_limit(void) {} > + > #endif /* !__ASSEMBLY__ */ > > #endif /* __ARM_KVM_MMU_H__ */ > diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h > index 5ecd457bce7d..f0474061851d 100644 > --- a/arch/arm64/include/asm/kvm_host.h > +++ b/arch/arm64/include/asm/kvm_host.h > @@ -513,6 +513,8 @@ static inline int kvm_arm_have_ssbd(void) > void kvm_vcpu_load_sysregs(struct kvm_vcpu *vcpu); > void kvm_vcpu_put_sysregs(struct kvm_vcpu *vcpu); > > +void kvm_set_ipa_limit(void); > + > #define __KVM_HAVE_ARCH_VM_ALLOC > struct kvm *kvm_arch_alloc_vm(void); > void kvm_arch_free_vm(struct kvm *kvm); > diff --git a/arch/arm64/kvm/reset.c b/arch/arm64/kvm/reset.c > index 51ecf0f7c912..76972b19bdd7 100644 > --- a/arch/arm64/kvm/reset.c > +++ b/arch/arm64/kvm/reset.c > @@ -34,6 +34,9 @@ > #include <asm/kvm_coproc.h> > #include <asm/kvm_mmu.h> > > +/* Maximum phys_shift supported for any VM on this host */ > +static u32 kvm_ipa_limit; > + > /* > * ARMv8 Reset Values > */ > @@ -135,6 +138,46 @@ int kvm_reset_vcpu(struct kvm_vcpu *vcpu) > return kvm_timer_vcpu_reset(vcpu); > } > > +void kvm_set_ipa_limit(void) > +{ > + unsigned int ipa_max, va_max, parange; > + > + parange = read_sanitised_ftr_reg(SYS_ID_AA64MMFR0_EL1) & 0x7; > + ipa_max = id_aa64mmfr0_parange_to_phys_shift(parange); > + > + /* Raise the limit to the default size for backward compatibility */ > + if (ipa_max < KVM_PHYS_SHIFT) { > + WARN_ONCE(1, > + "PARange is %d bits, unsupported configuration!", > + ipa_max); > + ipa_max = KVM_PHYS_SHIFT; I don't really get what does happen in this case. The CPU cannot handle PA up to ipa_max so can the VM run properly? In case it is a showstopper, kvm_set_ipa_limit should return an error, cascaded by init_common_resources. Otherwise the warning message may be reworded. > + } > + > + /* Clamp it to the PA size supported by the kernel */ > + ipa_max = (ipa_max > PHYS_MASK_SHIFT) ? PHYS_MASK_SHIFT : ipa_max; > + /* > + * Since our stage2 table is dependent on the stage1 page table code, > + * we must always honor the following condition: > + * > + * Number of levels in Stage1 >= Number of levels in Stage2. > + * > + * So clamp the ipa limit further down to limit the number of levels. > + * Since we can concatenate upto 16 tables at entry level, we could > + * go upto 4bits above the maximum VA addressible with the current addressable? > + * number of levels. > + */ > + va_max = PGDIR_SHIFT + PAGE_SHIFT - 3; > + va_max += 4; > + > + if (va_max < ipa_max) { > + kvm_info("Limiting IPA limit to %dbytes due to host VA bits limitation\n",> + va_max); > + ipa_max = va_max; you have a trace for this limitation but none for the comparison against PHYS_MASK_SHIFT. > + } > + > + kvm_ipa_limit = ipa_max; > +} > + > /* > * Configure the VTCR_EL2 for this VM. The VTCR value is common > * across all the physical CPUs on the system. We use system wide > diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c > index 43e716bc3f08..631f9a3ad99a 100644 > --- a/virt/kvm/arm/arm.c > +++ b/virt/kvm/arm/arm.c > @@ -1413,6 +1413,8 @@ static int init_common_resources(void) > kvm_vmid_bits = kvm_get_vmid_bits(); > kvm_info("%d-bit VMID\n", kvm_vmid_bits); > > + kvm_set_ipa_limit(); As we have a kvm_info for the supported vmid_bits, may be good to output the max IPA size supported by the host whatever the applied clamps? Thanks Eric > + > return 0; > } > >
On 09/25/2018 10:59 AM, Auger Eric wrote: > Hi Suzuki, > > On 9/17/18 12:41 PM, Suzuki K Poulose wrote: >> So far we have restricted the IPA size of the VM to the default >> value (40bits). Now that we can manage the IPA size per VM and >> support dynamic stage2 page tables, we can allow VMs to have >> larger IPA. This patch introduces a the maximum IPA size >> supported on the host. > to be reworded > This is decided by the following factors : Sure >> diff --git a/arch/arm64/kvm/reset.c b/arch/arm64/kvm/reset.c >> index 51ecf0f7c912..76972b19bdd7 100644 >> --- a/arch/arm64/kvm/reset.c >> +++ b/arch/arm64/kvm/reset.c >> @@ -34,6 +34,9 @@ >> #include <asm/kvm_coproc.h> >> #include <asm/kvm_mmu.h> >> >> +/* Maximum phys_shift supported for any VM on this host */ >> +static u32 kvm_ipa_limit; >> + >> /* >> * ARMv8 Reset Values >> */ >> @@ -135,6 +138,46 @@ int kvm_reset_vcpu(struct kvm_vcpu *vcpu) >> return kvm_timer_vcpu_reset(vcpu); >> } >> >> +void kvm_set_ipa_limit(void) >> +{ >> + unsigned int ipa_max, va_max, parange; >> + >> + parange = read_sanitised_ftr_reg(SYS_ID_AA64MMFR0_EL1) & 0x7; >> + ipa_max = id_aa64mmfr0_parange_to_phys_shift(parange); >> + >> + /* Raise the limit to the default size for backward compatibility */ >> + if (ipa_max < KVM_PHYS_SHIFT) { >> + WARN_ONCE(1, >> + "PARange is %d bits, unsupported configuration!", >> + ipa_max); >> + ipa_max = KVM_PHYS_SHIFT; > I don't really get what does happen in this case. The CPU cannot handle > PA up to ipa_max so can the VM run properly? In case it is a > showstopper, kvm_set_ipa_limit should return an error, cascaded by > init_common_resources. Otherwise the warning message may be reworded. I think this was a warning added to warn against the older Foundation model which had a 36bit PA size. So the VTCR was progammed with a 36bit limit, while the KVM guest was allowed to create 40bit IPA space, though it wouldn't fly well if someone tried to. With this series, I think we may expose the real IPA_MAX (which could be < 40bit) and warn the user if someone tried to create a VM with 40bit IPA (vm_type == 0) and let the call succeed (for the sake of ABI). Marc, Christoffer, Eric Thoughts ? >> + } >> + >> + /* Clamp it to the PA size supported by the kernel */ >> + ipa_max = (ipa_max > PHYS_MASK_SHIFT) ? PHYS_MASK_SHIFT : ipa_max; >> + /* >> + * Since our stage2 table is dependent on the stage1 page table code, >> + * we must always honor the following condition: >> + * >> + * Number of levels in Stage1 >= Number of levels in Stage2. >> + * >> + * So clamp the ipa limit further down to limit the number of levels. >> + * Since we can concatenate upto 16 tables at entry level, we could >> + * go upto 4bits above the maximum VA addressible with the current > addressable? Sure >> + * number of levels. >> + */ >> + va_max = PGDIR_SHIFT + PAGE_SHIFT - 3; >> + va_max += 4; >> + >> + if (va_max < ipa_max) { >> + kvm_info("Limiting IPA limit to %dbytes due to host VA bits limitation\n", >> + va_max); >> + ipa_max = va_max; > you have a trace for this limitation but none for the comparison against > PHYS_MASK_SHIFT. May be I could add a message which only mentions what is the limiting factor kernel VA vs kernel PA support >> + } >> + >> + kvm_ipa_limit = ipa_max; >> +} >> + >> /* >> * Configure the VTCR_EL2 for this VM. The VTCR value is common >> * across all the physical CPUs on the system. We use system wide >> diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c >> index 43e716bc3f08..631f9a3ad99a 100644 >> --- a/virt/kvm/arm/arm.c >> +++ b/virt/kvm/arm/arm.c >> @@ -1413,6 +1413,8 @@ static int init_common_resources(void) >> kvm_vmid_bits = kvm_get_vmid_bits(); >> kvm_info("%d-bit VMID\n", kvm_vmid_bits); >> >> + kvm_set_ipa_limit(); > As we have a kvm_info for the supported vmid_bits, may be good to output > the max IPA size supported by the host whatever the applied clamps? Sure, will do that. Thanks Suzuki
diff --git a/arch/arm/include/asm/kvm_mmu.h b/arch/arm/include/asm/kvm_mmu.h index 12ae5fbbcf01..5ad1a54f98dc 100644 --- a/arch/arm/include/asm/kvm_mmu.h +++ b/arch/arm/include/asm/kvm_mmu.h @@ -358,6 +358,8 @@ static inline int hyp_map_aux_data(void) #define kvm_phys_to_vttbr(addr) (addr) +static inline void kvm_set_ipa_limit(void) {} + #endif /* !__ASSEMBLY__ */ #endif /* __ARM_KVM_MMU_H__ */ diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index 5ecd457bce7d..f0474061851d 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -513,6 +513,8 @@ static inline int kvm_arm_have_ssbd(void) void kvm_vcpu_load_sysregs(struct kvm_vcpu *vcpu); void kvm_vcpu_put_sysregs(struct kvm_vcpu *vcpu); +void kvm_set_ipa_limit(void); + #define __KVM_HAVE_ARCH_VM_ALLOC struct kvm *kvm_arch_alloc_vm(void); void kvm_arch_free_vm(struct kvm *kvm); diff --git a/arch/arm64/kvm/reset.c b/arch/arm64/kvm/reset.c index 51ecf0f7c912..76972b19bdd7 100644 --- a/arch/arm64/kvm/reset.c +++ b/arch/arm64/kvm/reset.c @@ -34,6 +34,9 @@ #include <asm/kvm_coproc.h> #include <asm/kvm_mmu.h> +/* Maximum phys_shift supported for any VM on this host */ +static u32 kvm_ipa_limit; + /* * ARMv8 Reset Values */ @@ -135,6 +138,46 @@ int kvm_reset_vcpu(struct kvm_vcpu *vcpu) return kvm_timer_vcpu_reset(vcpu); } +void kvm_set_ipa_limit(void) +{ + unsigned int ipa_max, va_max, parange; + + parange = read_sanitised_ftr_reg(SYS_ID_AA64MMFR0_EL1) & 0x7; + ipa_max = id_aa64mmfr0_parange_to_phys_shift(parange); + + /* Raise the limit to the default size for backward compatibility */ + if (ipa_max < KVM_PHYS_SHIFT) { + WARN_ONCE(1, + "PARange is %d bits, unsupported configuration!", + ipa_max); + ipa_max = KVM_PHYS_SHIFT; + } + + /* Clamp it to the PA size supported by the kernel */ + ipa_max = (ipa_max > PHYS_MASK_SHIFT) ? PHYS_MASK_SHIFT : ipa_max; + /* + * Since our stage2 table is dependent on the stage1 page table code, + * we must always honor the following condition: + * + * Number of levels in Stage1 >= Number of levels in Stage2. + * + * So clamp the ipa limit further down to limit the number of levels. + * Since we can concatenate upto 16 tables at entry level, we could + * go upto 4bits above the maximum VA addressible with the current + * number of levels. + */ + va_max = PGDIR_SHIFT + PAGE_SHIFT - 3; + va_max += 4; + + if (va_max < ipa_max) { + kvm_info("Limiting IPA limit to %dbytes due to host VA bits limitation\n", + va_max); + ipa_max = va_max; + } + + kvm_ipa_limit = ipa_max; +} + /* * Configure the VTCR_EL2 for this VM. The VTCR value is common * across all the physical CPUs on the system. We use system wide diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c index 43e716bc3f08..631f9a3ad99a 100644 --- a/virt/kvm/arm/arm.c +++ b/virt/kvm/arm/arm.c @@ -1413,6 +1413,8 @@ static int init_common_resources(void) kvm_vmid_bits = kvm_get_vmid_bits(); kvm_info("%d-bit VMID\n", kvm_vmid_bits); + kvm_set_ipa_limit(); + return 0; }
So far we have restricted the IPA size of the VM to the default value (40bits). Now that we can manage the IPA size per VM and support dynamic stage2 page tables, we can allow VMs to have larger IPA. This patch introduces a the maximum IPA size supported on the host. This is decided by the following factors : 1) Maximum PARange supported by the CPUs - This can be inferred from the system wide safe value. 2) Maximum PA size supported by the host kernel (48 vs 52) 3) Number of levels in the host page table (as we base our stage2 tables on the host table helpers). Since the stage2 page table code is dependent on the stage1 page table, we always ensure that : Number of Levels at Stage1 >= Number of Levels at Stage2 So we limit the IPA to make sure that the above condition is satisfied. This will affect the following combinations of VA_BITS and IPA for different page sizes. Host configuration | Unsupported IPA ranges 39bit VA, 4K | [44, 48] 36bit VA, 16K | [41, 48] 42bit VA, 64K | [47, 52] Supporting the above combinations need independent stage2 page table manipulation code, which would need substantial changes. We could purse the solution independently and switch the page table code once we have it ready. Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Marc Zyngier <marc.zyngier@arm.com> Cc: Christoffer Dall <cdall@kernel.org> Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com> --- Changes since V2: - Restrict the IPA size to limit the number of page table levels in stage2 to that of stage1 or less. --- arch/arm/include/asm/kvm_mmu.h | 2 ++ arch/arm64/include/asm/kvm_host.h | 2 ++ arch/arm64/kvm/reset.c | 43 +++++++++++++++++++++++++++++++ virt/kvm/arm/arm.c | 2 ++ 4 files changed, 49 insertions(+)