KVM: Boosting vCPUs that are delivering interrupts
diff mbox series

Message ID 1562915435-8818-1-git-send-email-wanpengli@tencent.com
State New
Headers show
Series
  • KVM: Boosting vCPUs that are delivering interrupts
Related show

Commit Message

Wanpeng Li July 12, 2019, 7:10 a.m. UTC
From: Wanpeng Li <wanpengli@tencent.com>

Inspired by commit 9cac38dd5d (KVM/s390: Set preempted flag during vcpu wakeup 
and interrupt delivery), except the lock holder, we want to also boost vCPUs 
that are delivering interrupts. Actually most smp_call_function_many calls are 
synchronous ipi calls, the ipi target vCPUs are also good yield candidates. 
This patch sets preempted flag during wakeup and interrupt delivery time.

Testing on 80 HT 2 socket Xeon Skylake server, with 80 vCPUs VM 80GB RAM:
ebizzy -M

            vanilla     boosting    improved
1VM          23000       21232        -9%                      
2VM           2800        8000       180%
3VM           1800        3100        72%

Testing on my Haswell desktop 8 HT, with 8 vCPUs VM 8GB RAM, two VMs, 
one running ebizzy -M, the other running 'stress --cpu 2':

w/ boosting + w/o pv sched yield(vanilla)   

            vanilla     boosting   improved 
   			 1570         4000       55%

w/ boosting + w/ pv sched yield(vanilla)

			vanilla     boosting   improved 
             1844         5157       79%   

w/o boosting, perf top in VM:

 72.33%  [kernel]       [k] smp_call_function_many
  4.22%  [kernel]       [k] call_function_i
  3.71%  [kernel]       [k] async_page_fault

w/ boosting, perf top in VM:

 38.43%  [kernel]       [k] smp_call_function_many
  6.31%  [kernel]       [k] async_page_fault
  6.13%  libc-2.23.so   [.] __memcpy_avx_unaligned
  4.88%  [kernel]       [k] call_function_interrupt

Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Radim Krčmář <rkrcmar@redhat.com>
Cc: Christian Borntraeger <borntraeger@de.ibm.com>
Signed-off-by: Wanpeng Li <wanpengli@tencent.com>
---
 virt/kvm/kvm_main.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

Comments

Christian Borntraeger July 15, 2019, 10:53 a.m. UTC | #1
On 12.07.19 09:10, Wanpeng Li wrote:
> From: Wanpeng Li <wanpengli@tencent.com>
> 
> Inspired by commit 9cac38dd5d (KVM/s390: Set preempted flag during vcpu wakeup 
> and interrupt delivery), except the lock holder, we want to also boost vCPUs 
> that are delivering interrupts. Actually most smp_call_function_many calls are 
> synchronous ipi calls, the ipi target vCPUs are also good yield candidates. 
> This patch sets preempted flag during wakeup and interrupt delivery time.
> 
> Testing on 80 HT 2 socket Xeon Skylake server, with 80 vCPUs VM 80GB RAM:
> ebizzy -M
> 
>             vanilla     boosting    improved
> 1VM          23000       21232        -9%                      
> 2VM           2800        8000       180%
> 3VM           1800        3100        72%
> 
> Testing on my Haswell desktop 8 HT, with 8 vCPUs VM 8GB RAM, two VMs, 
> one running ebizzy -M, the other running 'stress --cpu 2':
> 
> w/ boosting + w/o pv sched yield(vanilla)   
> 
>             vanilla     boosting   improved 
>    			 1570         4000       55%
> 
> w/ boosting + w/ pv sched yield(vanilla)
> 
> 			vanilla     boosting   improved 
>              1844         5157       79%   
> 
> w/o boosting, perf top in VM:
> 
>  72.33%  [kernel]       [k] smp_call_function_many
>   4.22%  [kernel]       [k] call_function_i
>   3.71%  [kernel]       [k] async_page_fault
> 
> w/ boosting, perf top in VM:
> 
>  38.43%  [kernel]       [k] smp_call_function_many
>   6.31%  [kernel]       [k] async_page_fault
>   6.13%  libc-2.23.so   [.] __memcpy_avx_unaligned
>   4.88%  [kernel]       [k] call_function_interrupt
This certainly made sense for s390 so I guess that this also makes sense
for others.
Nnote we (s390) do not use kvm_vcpu_kick, so this should not cause
any issue for s390.


> 
> Cc: Paolo Bonzini <pbonzini@redhat.com>
> Cc: Radim Krčmář <rkrcmar@redhat.com>
> Cc: Christian Borntraeger <borntraeger@de.ibm.com>
> Signed-off-by: Wanpeng Li <wanpengli@tencent.com>
> ---
>  virt/kvm/kvm_main.c | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
> 
> diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
> index b4ab59d..2c46705 100644
> --- a/virt/kvm/kvm_main.c
> +++ b/virt/kvm/kvm_main.c
> @@ -2404,8 +2404,10 @@ void kvm_vcpu_kick(struct kvm_vcpu *vcpu)
>  	int me;
>  	int cpu = vcpu->cpu;
>  
> -	if (kvm_vcpu_wake_up(vcpu))
> +	if (kvm_vcpu_wake_up(vcpu)) {
> +		vcpu->preempted = true;
>  		return;
> +	}
>  
>  	me = get_cpu();
>  	if (cpu != me && (unsigned)cpu < nr_cpu_ids && cpu_online(cpu))
>
Wanpeng Li July 15, 2019, 11:15 a.m. UTC | #2
Cc arm and powerpc people,
On Mon, 15 Jul 2019 at 18:53, Christian Borntraeger
<borntraeger@de.ibm.com> wrote:
>
>
>
> On 12.07.19 09:10, Wanpeng Li wrote:
> > From: Wanpeng Li <wanpengli@tencent.com>
> >
> > Inspired by commit 9cac38dd5d (KVM/s390: Set preempted flag during vcpu wakeup
> > and interrupt delivery), except the lock holder, we want to also boost vCPUs
> > that are delivering interrupts. Actually most smp_call_function_many calls are
> > synchronous ipi calls, the ipi target vCPUs are also good yield candidates.
> > This patch sets preempted flag during wakeup and interrupt delivery time.
> >
> > Testing on 80 HT 2 socket Xeon Skylake server, with 80 vCPUs VM 80GB RAM:
> > ebizzy -M
> >
> >             vanilla     boosting    improved
> > 1VM          23000       21232        -9%
> > 2VM           2800        8000       180%
> > 3VM           1800        3100        72%
> >
> > Testing on my Haswell desktop 8 HT, with 8 vCPUs VM 8GB RAM, two VMs,
> > one running ebizzy -M, the other running 'stress --cpu 2':
> >
> > w/ boosting + w/o pv sched yield(vanilla)
> >
> >             vanilla     boosting   improved
> >                        1570         4000       55%
> >
> > w/ boosting + w/ pv sched yield(vanilla)
> >
> >                       vanilla     boosting   improved
> >              1844         5157       79%
> >
> > w/o boosting, perf top in VM:
> >
> >  72.33%  [kernel]       [k] smp_call_function_many
> >   4.22%  [kernel]       [k] call_function_i
> >   3.71%  [kernel]       [k] async_page_fault
> >
> > w/ boosting, perf top in VM:
> >
> >  38.43%  [kernel]       [k] smp_call_function_many
> >   6.31%  [kernel]       [k] async_page_fault
> >   6.13%  libc-2.23.so   [.] __memcpy_avx_unaligned
> >   4.88%  [kernel]       [k] call_function_interrupt
> This certainly made sense for s390 so I guess that this also makes sense
> for others.
> Nnote we (s390) do not use kvm_vcpu_kick, so this should not cause
> any issue for s390.
>
>
> >
> > Cc: Paolo Bonzini <pbonzini@redhat.com>
> > Cc: Radim Krčmář <rkrcmar@redhat.com>
> > Cc: Christian Borntraeger <borntraeger@de.ibm.com>
> > Signed-off-by: Wanpeng Li <wanpengli@tencent.com>
> > ---
> >  virt/kvm/kvm_main.c | 4 +++-
> >  1 file changed, 3 insertions(+), 1 deletion(-)
> >
> > diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
> > index b4ab59d..2c46705 100644
> > --- a/virt/kvm/kvm_main.c
> > +++ b/virt/kvm/kvm_main.c
> > @@ -2404,8 +2404,10 @@ void kvm_vcpu_kick(struct kvm_vcpu *vcpu)
> >       int me;
> >       int cpu = vcpu->cpu;
> >
> > -     if (kvm_vcpu_wake_up(vcpu))
> > +     if (kvm_vcpu_wake_up(vcpu)) {
> > +             vcpu->preempted = true;
> >               return;
> > +     }
> >
> >       me = get_cpu();
> >       if (cpu != me && (unsigned)cpu < nr_cpu_ids && cpu_online(cpu))
> >
>

Patch
diff mbox series

diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
index b4ab59d..2c46705 100644
--- a/virt/kvm/kvm_main.c
+++ b/virt/kvm/kvm_main.c
@@ -2404,8 +2404,10 @@  void kvm_vcpu_kick(struct kvm_vcpu *vcpu)
 	int me;
 	int cpu = vcpu->cpu;
 
-	if (kvm_vcpu_wake_up(vcpu))
+	if (kvm_vcpu_wake_up(vcpu)) {
+		vcpu->preempted = true;
 		return;
+	}
 
 	me = get_cpu();
 	if (cpu != me && (unsigned)cpu < nr_cpu_ids && cpu_online(cpu))