diff mbox

[PATCHv3,1/1] locking/qspinlock/x86: Avoid test-and-set when PV_DEDICATED is set

Message ID 20171109141732.GA20859@flask (mailing list archive)
State New, archived
Headers show

Commit Message

Radim Krčmář Nov. 9, 2017, 2:17 p.m. UTC
2017-11-09 00:55-0800, Eduardo Valentin:
> Hello,
> 
> On Wed, Nov 08, 2017 at 06:36:52PM +0100, Radim Krčmář wrote:
> > 2017-11-06 12:26-0800, Eduardo Valentin:
> > > Currently, the existing qspinlock implementation will fallback to
> > > test-and-set if the hypervisor has not set the PV_UNHALT flag.
> > > 
> > > This patch gives the opportunity to guest kernels to select
> > > between test-and-set and the regular queueu fair lock implementation
> > > based on the PV_DEDICATED KVM feature flag. When the PV_DEDICATED
> > > flag is not set, the code will still fall back to test-and-set,
> > > but when the PV_DEDICATED flag is set, the code will use
> > > the regular queue spinlock implementation.
> > > 
> > > With this patch, when in autoselect mode, the guest will
> > > use the default spinlock implementation based on host feature
> > > flags as follows:
> > > 
> > > PV_DEDICATED = 1, PV_UNHALT = anything: default is qspinlock
> > > PV_DEDICATED = 0, PV_UNHALT = 1: default is pvqspinlock
> > > PV_DEDICATED = 0, PV_UNHALT = 0: default is tas
> > > 
> > > Cc: Paolo Bonzini <pbonzini@redhat.com>
> > > Cc: "Radim Krčmář" <rkrcmar@redhat.com>
> > > Cc: Jonathan Corbet <corbet@lwn.net>
> > > Cc: Thomas Gleixner <tglx@linutronix.de>
> > > Cc: Ingo Molnar <mingo@redhat.com>
> > > Cc: "H. Peter Anvin" <hpa@zytor.com>
> > > Cc: x86@kernel.org
> > > Cc: Peter Zijlstra <peterz@infradead.org>
> > > Cc: Waiman Long <longman@redhat.com>
> > > Cc: kvm@vger.kernel.org
> > > Cc: linux-doc@vger.kernel.org
> > > Cc: linux-kernel@vger.kernel.org
> > > Cc: Jan H. Schoenherr <jschoenh@amazon.de>
> > > Cc: Anthony Liguori <aliguori@amazon.com>
> > > Suggested-by: Matt Wilson <msw@amazon.com>
> > > Signed-off-by: Eduardo Valentin <eduval@amazon.com>
> > > ---
> > > V3:
> > >  - When PV_DEDICATED is set (1), qspinlock is selected,
> > >    regardless of the value of PV_UNHAULT. Suggested by Paolo Bonzini. 
> > >  - Refreshed on top of tip/master.
> > > V2:
> > >  - rebase on top of tip/master
> > > 
> > >  Documentation/virtual/kvm/cpuid.txt  | 6 ++++++
> > >  arch/x86/include/asm/qspinlock.h     | 4 ++++
> > >  arch/x86/include/uapi/asm/kvm_para.h | 1 +
> > >  arch/x86/kernel/kvm.c                | 2 ++
> > >  4 files changed, 13 insertions(+)
> > > 
> > > diff --git a/Documentation/virtual/kvm/cpuid.txt b/Documentation/virtual/kvm/cpuid.txt
> > > index 3c65feb..117066a 100644
> > > --- a/Documentation/virtual/kvm/cpuid.txt
> > > +++ b/Documentation/virtual/kvm/cpuid.txt
> > > @@ -54,6 +54,12 @@ KVM_FEATURE_PV_UNHALT              ||     7 || guest checks this feature bit
> > >                                     ||       || before enabling paravirtualized
> > >                                     ||       || spinlock support.
> > >  ------------------------------------------------------------------------------
> > > +KVM_FEATURE_PV_DEDICATED           ||     8 || guest checks this feature bit
> > > +                                   ||       || to determine if they run on
> > > +                                   ||       || dedicated vCPUs, allowing opti-
> > > +                                   ||       || mizations such as usage of
> > > +                                   ||       || qspinlocks.
> > > +------------------------------------------------------------------------------
> > >  KVM_FEATURE_CLOCKSOURCE_STABLE_BIT ||    24 || host will warn if no guest-side
> > >                                     ||       || per-cpu warps are expected in
> > >                                     ||       || kvmclock.
> > > diff --git a/arch/x86/include/asm/qspinlock.h b/arch/x86/include/asm/qspinlock.h
> > > index 5e16b5d..de42694 100644
> > > --- a/arch/x86/include/asm/qspinlock.h
> > > +++ b/arch/x86/include/asm/qspinlock.h
> > > @@ -3,6 +3,8 @@
> > >  #define _ASM_X86_QSPINLOCK_H
> > >  
> > >  #include <linux/jump_label.h>
> > > +#include <linux/kvm_para.h>
> > > +
> > >  #include <asm/cpufeature.h>
> > >  #include <asm-generic/qspinlock_types.h>
> > >  #include <asm/paravirt.h>
> > > @@ -58,6 +60,8 @@ static inline bool virt_spin_lock(struct qspinlock *lock)
> > >  	if (!static_branch_likely(&virt_spin_lock_key))
> > >  		return false;
> > >  
> > > +	if (kvm_para_has_feature(KVM_FEATURE_PV_DEDICATED))
> > > +		return false;
> > 
> > Hm, every spinlock slowpath calls cpuid, which causes a VM exit, so I
> > wouldn't expect it to be faster than the existing implementations.
> > (Using the static key would be better.)
> > 
> > How does this patch perform compared to user-forced qspinlock and hybrid
> > pvqspinlock?
> 
> This patch should have same effect as user-forced qspinlock.

This is what I'm doubting, because the patch is adding about two
thousand cycles to every spinlock-taken path.
Doesn't this patch yield better results?


>                                                              However, the key aspect
> here is this patch gives a way for the host to instruct the guest to use qspinlock.
> Even with Longman's patch which allows guest to select the spinlock implementation,
> there should still be the auto-select mode. In such mode, PV_DEDICATED should
> allow the host to get the guest to use qspinlock, without, the guest will fallback
> to tas when PV_UNHALT == 0.

I agree that a flag can be useful for certains setups.

Comments

Eduardo Valentin Nov. 16, 2017, 4:54 a.m. UTC | #1
Hey Radim,

On Thu, Nov 09, 2017 at 03:17:33PM +0100, Radim Krčmář wrote:

<cut>

> 
> This is what I'm doubting, because the patch is adding about two
> thousand cycles to every spinlock-taken path.
> Doesn't this patch yield better results?
> 
> diff --git a/arch/x86/kernel/kvm.c b/arch/x86/kernel/kvm.c
> index 3df743b60c80..d9225e48c11a 100644
> --- a/arch/x86/kernel/kvm.c
> +++ b/arch/x86/kernel/kvm.c
> @@ -676,6 +676,12 @@ void __init kvm_spinlock_init(void)
>  {
>  	if (!kvm_para_available())
>  		return;
> +
> +	if (kvm_para_has_feature(KVM_FEATURE_PV_DEDICATED)) {
> +		static_branch_disable(&virt_spin_lock_key);
> +		return;
> +	}
> +

Yes, the above suggestion is a much better approach. The code has probably changed from the time I wrote the first version. I will refresh with the above suggestion.


>  	/* Does host kernel support KVM_FEATURE_PV_UNHALT? */
>  	if (!kvm_para_has_feature(KVM_FEATURE_PV_UNHALT))
>  		return;
> 
> >                                                              However, the key aspect
> > here is this patch gives a way for the host to instruct the guest to use qspinlock.
> > Even with Longman's patch which allows guest to select the spinlock implementation,
> > there should still be the auto-select mode. In such mode, PV_DEDICATED should
> > allow the host to get the guest to use qspinlock, without, the guest will fallback
> > to tas when PV_UNHALT == 0.
> 
> I agree that a flag can be useful for certains setups.

Cool!

>
Wanpeng Li Nov. 27, 2017, 12:43 a.m. UTC | #2
Hi Eduardo,
2017-11-16 12:54 GMT+08:00 Eduardo Valentin <eduval@amazon.com>:
> Hey Radim,
>
> On Thu, Nov 09, 2017 at 03:17:33PM +0100, Radim Krčmář wrote:
>
> <cut>
>
>>
>> This is what I'm doubting, because the patch is adding about two
>> thousand cycles to every spinlock-taken path.
>> Doesn't this patch yield better results?
>>
>> diff --git a/arch/x86/kernel/kvm.c b/arch/x86/kernel/kvm.c
>> index 3df743b60c80..d9225e48c11a 100644
>> --- a/arch/x86/kernel/kvm.c
>> +++ b/arch/x86/kernel/kvm.c
>> @@ -676,6 +676,12 @@ void __init kvm_spinlock_init(void)
>>  {
>>       if (!kvm_para_available())
>>               return;
>> +
>> +     if (kvm_para_has_feature(KVM_FEATURE_PV_DEDICATED)) {
>> +             static_branch_disable(&virt_spin_lock_key);
>> +             return;
>> +     }
>> +
>
> Yes, the above suggestion is a much better approach. The code has probably changed from the time I wrote the first version. I will refresh with the above suggestion.

Do you mind to send a new version since the merge window is closed?

Regards,
Wanpeng Li

>
>
>>       /* Does host kernel support KVM_FEATURE_PV_UNHALT? */
>>       if (!kvm_para_has_feature(KVM_FEATURE_PV_UNHALT))
>>               return;
>>
>> >                                                              However, the key aspect
>> > here is this patch gives a way for the host to instruct the guest to use qspinlock.
>> > Even with Longman's patch which allows guest to select the spinlock implementation,
>> > there should still be the auto-select mode. In such mode, PV_DEDICATED should
>> > allow the host to get the guest to use qspinlock, without, the guest will fallback
>> > to tas when PV_UNHALT == 0.
>>
>> I agree that a flag can be useful for certains setups.
>
> Cool!
>
>>
>
> --
> All the best,
> Eduardo Valentin
diff mbox

Patch

diff --git a/arch/x86/kernel/kvm.c b/arch/x86/kernel/kvm.c
index 3df743b60c80..d9225e48c11a 100644
--- a/arch/x86/kernel/kvm.c
+++ b/arch/x86/kernel/kvm.c
@@ -676,6 +676,12 @@  void __init kvm_spinlock_init(void)
 {
 	if (!kvm_para_available())
 		return;
+
+	if (kvm_para_has_feature(KVM_FEATURE_PV_DEDICATED)) {
+		static_branch_disable(&virt_spin_lock_key);
+		return;
+	}
+
 	/* Does host kernel support KVM_FEATURE_PV_UNHALT? */
 	if (!kvm_para_has_feature(KVM_FEATURE_PV_UNHALT))
 		return;