| Message ID | 50B60976.7020905@linux.vnet.ibm.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
On Wed, Nov 28, 2012 at 08:54:14PM +0800, Xiao Guangrong wrote: > vmcs->cpu indicates whether it exists on the target cpu, -1 means the vmcs > does not exist on any vcpu > > If vcpu load vmcs with vmcs.cpu = -1, it can be directly added to cpu's percpu > list. The list can be corrupted if the cpu prefetch the vmcs's list before > reading vmcs->cpu. Meanwhile, we should remove vmcs from the list before > making vmcs->vcpu == -1 be visible > > Signed-off-by: Xiao Guangrong <xiaoguangrong@linux.vnet.ibm.com> > --- > arch/x86/kvm/vmx.c | 17 +++++++++++++++++ > 1 files changed, 17 insertions(+), 0 deletions(-) > > diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c > index 29e8f42..6056d88 100644 > --- a/arch/x86/kvm/vmx.c > +++ b/arch/x86/kvm/vmx.c > @@ -1002,6 +1002,15 @@ static void __loaded_vmcs_clear(void *arg) > if (per_cpu(current_vmcs, cpu) == loaded_vmcs->vmcs) > per_cpu(current_vmcs, cpu) = NULL; > list_del(&loaded_vmcs->loaded_vmcss_on_cpu_link); > + > + /* > + * we should ensure updating loaded_vmcs->loaded_vmcss_on_cpu_link > + * is before setting loaded_vmcs->vcpu to -1 which is done in > + * loaded_vmcs_init. Otherwise, other cpu can see vcpu = -1 fist > + * then adds the vmcs into percpu list before it is deleted. > + */ > + smp_wmb(); > + Neither loads nor stores are reordered with like operations (see section 8.2.3.2 of intel's volume 3). This behaviour makes the barrier not necessary. However, i agree access to loaded_vmcs is not obviously safe. I can't tell its safe with vmm_exclusive = 0 (where vcpu->cpu can change at any time). -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On 11/29/2012 08:04 AM, Marcelo Tosatti wrote: > On Wed, Nov 28, 2012 at 08:54:14PM +0800, Xiao Guangrong wrote: >> vmcs->cpu indicates whether it exists on the target cpu, -1 means the vmcs >> does not exist on any vcpu >> >> If vcpu load vmcs with vmcs.cpu = -1, it can be directly added to cpu's percpu >> list. The list can be corrupted if the cpu prefetch the vmcs's list before >> reading vmcs->cpu. Meanwhile, we should remove vmcs from the list before >> making vmcs->vcpu == -1 be visible >> >> Signed-off-by: Xiao Guangrong <xiaoguangrong@linux.vnet.ibm.com> >> --- >> arch/x86/kvm/vmx.c | 17 +++++++++++++++++ >> 1 files changed, 17 insertions(+), 0 deletions(-) >> >> diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c >> index 29e8f42..6056d88 100644 >> --- a/arch/x86/kvm/vmx.c >> +++ b/arch/x86/kvm/vmx.c >> @@ -1002,6 +1002,15 @@ static void __loaded_vmcs_clear(void *arg) >> if (per_cpu(current_vmcs, cpu) == loaded_vmcs->vmcs) >> per_cpu(current_vmcs, cpu) = NULL; >> list_del(&loaded_vmcs->loaded_vmcss_on_cpu_link); >> + >> + /* >> + * we should ensure updating loaded_vmcs->loaded_vmcss_on_cpu_link >> + * is before setting loaded_vmcs->vcpu to -1 which is done in >> + * loaded_vmcs_init. Otherwise, other cpu can see vcpu = -1 fist >> + * then adds the vmcs into percpu list before it is deleted. >> + */ >> + smp_wmb(); >> + > > Neither loads nor stores are reordered with like operations (see > section 8.2.3.2 of intel's volume 3). This behaviour makes the barrier > not necessary. Ouch, yes, you are right. My memory is wrong. It seems only later-read can be reordered with early-write. But if 'read vs read' and 'write vs write' can be guaranteed by CPU, smp_wmb() and smp_rmb() should only be a complier barrier, so i think we can add the barriers to improve the readable and the portable. And anyway, the current code missed complier-barrier. > > However, i agree access to loaded_vmcs is not obviously safe. I can't > tell its safe with vmm_exclusive = 0 (where vcpu->cpu can change at any > time). If vmm_exclusive = 0, the vmcs can removed from percpu list when vcpu is scheduled out. The list is not broken. -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On 11/29/2012 08:04 AM, Marcelo Tosatti wrote: > On Wed, Nov 28, 2012 at 08:54:14PM +0800, Xiao Guangrong wrote: >> vmcs->cpu indicates whether it exists on the target cpu, -1 means the vmcs >> does not exist on any vcpu >> >> If vcpu load vmcs with vmcs.cpu = -1, it can be directly added to cpu's percpu >> list. The list can be corrupted if the cpu prefetch the vmcs's list before >> reading vmcs->cpu. Meanwhile, we should remove vmcs from the list before >> making vmcs->vcpu == -1 be visible >> >> Signed-off-by: Xiao Guangrong <xiaoguangrong@linux.vnet.ibm.com> >> --- >> arch/x86/kvm/vmx.c | 17 +++++++++++++++++ >> 1 files changed, 17 insertions(+), 0 deletions(-) >> >> diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c >> index 29e8f42..6056d88 100644 >> --- a/arch/x86/kvm/vmx.c >> +++ b/arch/x86/kvm/vmx.c >> @@ -1002,6 +1002,15 @@ static void __loaded_vmcs_clear(void *arg) >> if (per_cpu(current_vmcs, cpu) == loaded_vmcs->vmcs) >> per_cpu(current_vmcs, cpu) = NULL; >> list_del(&loaded_vmcs->loaded_vmcss_on_cpu_link); >> + >> + /* >> + * we should ensure updating loaded_vmcs->loaded_vmcss_on_cpu_link >> + * is before setting loaded_vmcs->vcpu to -1 which is done in >> + * loaded_vmcs_init. Otherwise, other cpu can see vcpu = -1 fist >> + * then adds the vmcs into percpu list before it is deleted. >> + */ >> + smp_wmb(); >> + > > Neither loads nor stores are reordered with like operations (see > section 8.2.3.2 of intel's volume 3). This behaviour makes the barrier > not necessary. Ouch, yes, you are right. My memory is wrong. It seems only later-read can be reordered with early-write. But if 'read vs read' and 'write vs write' can be guaranteed by CPU, smp_wmb() and smp_rmb() should act as a complier barrier, so i think we can add the barriers to improve the readable and the portable. And anyway, the current code missed complier-barrier. > > However, i agree access to loaded_vmcs is not obviously safe. I can't > tell its safe with vmm_exclusive = 0 (where vcpu->cpu can change at any > time). If vmm_exclusive = 0, the vmcs can removed from percpu list when vcpu is scheduled out. The list is not broken. -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Wed, Nov 28, 2012 at 08:54:14PM +0800, Xiao Guangrong wrote: > vmcs->cpu indicates whether it exists on the target cpu, -1 means the vmcs > does not exist on any vcpu > > If vcpu load vmcs with vmcs.cpu = -1, it can be directly added to cpu's percpu > list. The list can be corrupted if the cpu prefetch the vmcs's list before > reading vmcs->cpu. Meanwhile, we should remove vmcs from the list before > making vmcs->vcpu == -1 be visible > > Signed-off-by: Xiao Guangrong <xiaoguangrong@linux.vnet.ibm.com> > --- > arch/x86/kvm/vmx.c | 17 +++++++++++++++++ > 1 files changed, 17 insertions(+), 0 deletions(-) Applied, thanks. -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c index 29e8f42..6056d88 100644 --- a/arch/x86/kvm/vmx.c +++ b/arch/x86/kvm/vmx.c @@ -1002,6 +1002,15 @@ static void __loaded_vmcs_clear(void *arg) if (per_cpu(current_vmcs, cpu) == loaded_vmcs->vmcs) per_cpu(current_vmcs, cpu) = NULL; list_del(&loaded_vmcs->loaded_vmcss_on_cpu_link); + + /* + * we should ensure updating loaded_vmcs->loaded_vmcss_on_cpu_link + * is before setting loaded_vmcs->vcpu to -1 which is done in + * loaded_vmcs_init. Otherwise, other cpu can see vcpu = -1 fist + * then adds the vmcs into percpu list before it is deleted. + */ + smp_wmb(); + loaded_vmcs_init(loaded_vmcs); } @@ -1537,6 +1546,14 @@ static void vmx_vcpu_load(struct kvm_vcpu *vcpu, int cpu) kvm_make_request(KVM_REQ_TLB_FLUSH, vcpu); local_irq_disable(); + + /* + * Read loaded_vmcs->cpu should be before fetching + * loaded_vmcs->loaded_vmcss_on_cpu_link. + * See the comments in __loaded_vmcs_clear(). + */ + smp_rmb(); + list_add(&vmx->loaded_vmcs->loaded_vmcss_on_cpu_link, &per_cpu(loaded_vmcss_on_cpu, cpu)); local_irq_enable();
vmcs->cpu indicates whether it exists on the target cpu, -1 means the vmcs does not exist on any vcpu If vcpu load vmcs with vmcs.cpu = -1, it can be directly added to cpu's percpu list. The list can be corrupted if the cpu prefetch the vmcs's list before reading vmcs->cpu. Meanwhile, we should remove vmcs from the list before making vmcs->vcpu == -1 be visible Signed-off-by: Xiao Guangrong <xiaoguangrong@linux.vnet.ibm.com> --- arch/x86/kvm/vmx.c | 17 +++++++++++++++++ 1 files changed, 17 insertions(+), 0 deletions(-)