| Message ID | 20180201175022.25011-1-rkrcmar@redhat.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
On 01/02/2018 12:50, Radim Krčmář wrote: > Guests on new hypersiors might set KVM_ASYNC_PF_DELIVERY_AS_PF_VMEXIT > bit when enabling async_PF, but this bit is reserved on old hypervisors, > which results in a failure upon migration. > > Guests at least expect that KVM_ASYNC_PF_DELIVERY_AS_PF_VMEXIT might not > be present when booting, so we allow userspace to handle migration > compatibility by adding a KVM CPUID flag that determines the presence of > KVM_ASYNC_PF_DELIVERY_AS_PF_VMEXIT. > > Fixes: 52a5c155cf79 ("KVM: async_pf: Let guest support delivery of async_pf from guest mode") > Cc: <stable@vger.kernel.org> > Signed-off-by: Radim Krčmář <rkrcmar@redhat.com> This has to be documented in Documentation/virtual/kvm/cpuid.txt. > diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c > index 4c3103f449a3..c16740a06f0c 100644 > --- a/arch/x86/kvm/x86.c > +++ b/arch/x86/kvm/x86.c > @@ -2139,8 +2139,10 @@ static int kvm_pv_enable_async_pf(struct kvm_vcpu *vcpu, u64 data) > { > gpa_t gpa = data & ~0x3f; > > - /* Bits 3:5 are reserved, Should be zero */ > - if (data & 0x38) > + /* Bits 3:5 are reserved, Should be zero. */ > + if (data & 0x38 || > + (data & KVM_ASYNC_PF_DELIVERY_AS_PF_VMEXIT && > + !guest_kvm_cpuid_has(vcpu, KVM_FEATURE_ASYNC_PF_VMEXIT))) > return 1; > > vcpu->arch.apf.msr_val = data; > This check will break migration if the source guest and host both have the recent kernels which support KVM_ASYNC_PF_DELIVERY_AS_PF_VMEXIT, so I am not sure about it. Otherwise, the patch is okay! Thanks, Paolo
2018-02-01 13:09-0500, Paolo Bonzini: > On 01/02/2018 12:50, Radim Krčmář wrote: > > Guests on new hypersiors might set KVM_ASYNC_PF_DELIVERY_AS_PF_VMEXIT > > bit when enabling async_PF, but this bit is reserved on old hypervisors, > > which results in a failure upon migration. > > > > Guests at least expect that KVM_ASYNC_PF_DELIVERY_AS_PF_VMEXIT might not > > be present when booting, so we allow userspace to handle migration > > compatibility by adding a KVM CPUID flag that determines the presence of > > KVM_ASYNC_PF_DELIVERY_AS_PF_VMEXIT. > > > > Fixes: 52a5c155cf79 ("KVM: async_pf: Let guest support delivery of async_pf from guest mode") > > Cc: <stable@vger.kernel.org> > > Signed-off-by: Radim Krčmář <rkrcmar@redhat.com> > > This has to be documented in Documentation/virtual/kvm/cpuid.txt. Will add, also to the MSR if we agree on v2. > > diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c > > index 4c3103f449a3..c16740a06f0c 100644 > > --- a/arch/x86/kvm/x86.c > > +++ b/arch/x86/kvm/x86.c > > @@ -2139,8 +2139,10 @@ static int kvm_pv_enable_async_pf(struct kvm_vcpu *vcpu, u64 data) > > { > > gpa_t gpa = data & ~0x3f; > > > > - /* Bits 3:5 are reserved, Should be zero */ > > - if (data & 0x38) > > + /* Bits 3:5 are reserved, Should be zero. */ > > + if (data & 0x38 || > > + (data & KVM_ASYNC_PF_DELIVERY_AS_PF_VMEXIT && > > + !guest_kvm_cpuid_has(vcpu, KVM_FEATURE_ASYNC_PF_VMEXIT))) > > return 1; > > > > vcpu->arch.apf.msr_val = data; > > > > This check will break migration if the source guest and host both have > the recent kernels which support KVM_ASYNC_PF_DELIVERY_AS_PF_VMEXIT, so > I am not sure about it. Otherwise, the patch is okay! Good point, breaking forward migration is worse than doing nothing. A compromise solution would be to drop the feature check from the hypervisor. Newer guests would work everywhere and there would be no change to old systems, so v4.13-v4.15 guests could at least upgrade. Slightly better than doing nothing, IMO, thanks.
On 01/02/2018 14:10, Radim Krčmář wrote: >>> >> This check will break migration if the source guest and host both have >> the recent kernels which support KVM_ASYNC_PF_DELIVERY_AS_PF_VMEXIT, so >> I am not sure about it. Otherwise, the patch is okay! > Good point, breaking forward migration is worse than doing nothing. > > A compromise solution would be to drop the feature check from the > hypervisor. Newer guests would work everywhere and there would be no > change to old systems, so v4.13-v4.15 guests could at least upgrade. > > Slightly better than doing nothing, IMO, Yes, I agree. Just removing the guest_kvm_cpuid_has should be fine. Paolo
diff --git a/arch/x86/include/uapi/asm/kvm_para.h b/arch/x86/include/uapi/asm/kvm_para.h index 7a2ade4aa235..6cfa9c8cb7d6 100644 --- a/arch/x86/include/uapi/asm/kvm_para.h +++ b/arch/x86/include/uapi/asm/kvm_para.h @@ -26,6 +26,7 @@ #define KVM_FEATURE_PV_EOI 6 #define KVM_FEATURE_PV_UNHALT 7 #define KVM_FEATURE_PV_TLB_FLUSH 9 +#define KVM_FEATURE_ASYNC_PF_VMEXIT 10 /* The last 8 bits are used to indicate how to interpret the flags field * in pvclock structure. If no bits are set, all flags are ignored. diff --git a/arch/x86/kernel/kvm.c b/arch/x86/kernel/kvm.c index 4e37d1a851a6..971babe964d2 100644 --- a/arch/x86/kernel/kvm.c +++ b/arch/x86/kernel/kvm.c @@ -341,10 +341,10 @@ static void kvm_guest_cpu_init(void) #endif pa |= KVM_ASYNC_PF_ENABLED; - /* Async page fault support for L1 hypervisor is optional */ - if (wrmsr_safe(MSR_KVM_ASYNC_PF_EN, - (pa | KVM_ASYNC_PF_DELIVERY_AS_PF_VMEXIT) & 0xffffffff, pa >> 32) < 0) - wrmsrl(MSR_KVM_ASYNC_PF_EN, pa); + if (kvm_para_has_feature(KVM_FEATURE_ASYNC_PF_VMEXIT)) + pa |= KVM_ASYNC_PF_DELIVERY_AS_PF_VMEXIT; + + wrmsrl(MSR_KVM_ASYNC_PF_EN, pa); __this_cpu_write(apf_reason.enabled, 1); printk(KERN_INFO"KVM setup async PF for cpu %d\n", smp_processor_id()); diff --git a/arch/x86/kvm/cpuid.c b/arch/x86/kvm/cpuid.c index 20e491b94f44..7fc04a176c57 100644 --- a/arch/x86/kvm/cpuid.c +++ b/arch/x86/kvm/cpuid.c @@ -604,7 +604,8 @@ static inline int __do_cpuid_ent(struct kvm_cpuid_entry2 *entry, u32 function, (1 << KVM_FEATURE_PV_EOI) | (1 << KVM_FEATURE_CLOCKSOURCE_STABLE_BIT) | (1 << KVM_FEATURE_PV_UNHALT) | - (1 << KVM_FEATURE_PV_TLB_FLUSH); + (1 << KVM_FEATURE_PV_TLB_FLUSH) | + (1 << KVM_FEATURE_ASYNC_PF_VMEXIT); if (sched_info_on()) entry->eax |= (1 << KVM_FEATURE_STEAL_TIME); diff --git a/arch/x86/kvm/cpuid.h b/arch/x86/kvm/cpuid.h index c2cea6651279..f20731dfe28e 100644 --- a/arch/x86/kvm/cpuid.h +++ b/arch/x86/kvm/cpuid.h @@ -105,6 +105,17 @@ static __always_inline bool guest_cpuid_has(struct kvm_vcpu *vcpu, unsigned x86_ return *reg & bit(x86_feature); } +static inline bool guest_kvm_cpuid_has(struct kvm_vcpu *vcpu, unsigned kvm_feature) +{ + struct kvm_cpuid_entry2 *entry; + + entry = kvm_find_cpuid_entry(vcpu, KVM_CPUID_FEATURES, 0); + if (!entry) + return false; + + return entry->eax & bit(kvm_feature); +} + static __always_inline void guest_cpuid_clear(struct kvm_vcpu *vcpu, unsigned x86_feature) { int *reg; diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 4c3103f449a3..c16740a06f0c 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -2139,8 +2139,10 @@ static int kvm_pv_enable_async_pf(struct kvm_vcpu *vcpu, u64 data) { gpa_t gpa = data & ~0x3f; - /* Bits 3:5 are reserved, Should be zero */ - if (data & 0x38) + /* Bits 3:5 are reserved, Should be zero. */ + if (data & 0x38 || + (data & KVM_ASYNC_PF_DELIVERY_AS_PF_VMEXIT && + !guest_kvm_cpuid_has(vcpu, KVM_FEATURE_ASYNC_PF_VMEXIT))) return 1; vcpu->arch.apf.msr_val = data;
Guests on new hypersiors might set KVM_ASYNC_PF_DELIVERY_AS_PF_VMEXIT bit when enabling async_PF, but this bit is reserved on old hypervisors, which results in a failure upon migration. Guests at least expect that KVM_ASYNC_PF_DELIVERY_AS_PF_VMEXIT might not be present when booting, so we allow userspace to handle migration compatibility by adding a KVM CPUID flag that determines the presence of KVM_ASYNC_PF_DELIVERY_AS_PF_VMEXIT. Fixes: 52a5c155cf79 ("KVM: async_pf: Let guest support delivery of async_pf from guest mode") Cc: <stable@vger.kernel.org> Signed-off-by: Radim Krčmář <rkrcmar@redhat.com> --- arch/x86/include/uapi/asm/kvm_para.h | 1 + arch/x86/kernel/kvm.c | 8 ++++---- arch/x86/kvm/cpuid.c | 3 ++- arch/x86/kvm/cpuid.h | 11 +++++++++++ arch/x86/kvm/x86.c | 6 ++++-- 5 files changed, 22 insertions(+), 7 deletions(-)