From patchwork Tue Jul 31 10:48:54 2012 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Nikunj A. Dadhania" X-Patchwork-Id: 1259021 Return-Path: X-Original-To: patchwork-kvm@patchwork.kernel.org Delivered-To: patchwork-process-083081@patchwork1.kernel.org Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by patchwork1.kernel.org (Postfix) with ESMTP id B5CB93FC71 for ; Tue, 31 Jul 2012 10:49:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755750Ab2GaKtj (ORCPT ); Tue, 31 Jul 2012 06:49:39 -0400 Received: from e28smtp04.in.ibm.com ([122.248.162.4]:50611 "EHLO e28smtp04.in.ibm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755743Ab2GaKti (ORCPT ); Tue, 31 Jul 2012 06:49:38 -0400 Received: from /spool/local by e28smtp04.in.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Tue, 31 Jul 2012 16:19:36 +0530 Received: from d28relay02.in.ibm.com (9.184.220.59) by e28smtp04.in.ibm.com (192.168.1.134) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Tue, 31 Jul 2012 16:19:34 +0530 Received: from d28av03.in.ibm.com (d28av03.in.ibm.com [9.184.220.65]) by d28relay02.in.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id q6VAnXTM16318574 for ; Tue, 31 Jul 2012 16:19:34 +0530 Received: from d28av03.in.ibm.com (loopback [127.0.0.1]) by d28av03.in.ibm.com (8.14.4/8.13.1/NCO v10.0 AVout) with ESMTP id q6VAnWX0026667 for ; Tue, 31 Jul 2012 20:49:33 +1000 Received: from abhimanyu.in.ibm.com (abhimanyu.in.ibm.com [9.124.35.147] (may be forged)) by d28av03.in.ibm.com (8.14.4/8.13.1/NCO v10.0 AVin) with ESMTP id q6VAnWGw026664; Tue, 31 Jul 2012 20:49:32 +1000 Subject: [PATCH v3 5/8] KVM Guest: Add paravirt kvm_flush_tlb_others To: peterz@infradead.org, mtosatti@redhat.com, avi@redhat.com From: "Nikunj A. Dadhania" Cc: raghukt@linux.vnet.ibm.com, alex.shi@intel.com, mingo@elte.hu, kvm@vger.kernel.org, hpa@zytor.com Date: Tue, 31 Jul 2012 16:18:54 +0530 Message-ID: <20120731104849.16662.52362.stgit@abhimanyu.in.ibm.com> In-Reply-To: <20120731104312.16662.27889.stgit@abhimanyu.in.ibm.com> References: <20120731104312.16662.27889.stgit@abhimanyu.in.ibm.com> User-Agent: StGit/0.16-2-g0d85 MIME-Version: 1.0 x-cbid: 12073110-5564-0000-0000-000003D127FC Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Nikunj A. Dadhania flush_tlb_others_ipi depends on lot of statics in tlb.c. Replicated the flush_tlb_others_ipi as kvm_flush_tlb_others to further adapt to paravirtualization. Use the vcpu state information inside the kvm_flush_tlb_others to avoid sending ipi to pre-empted vcpus. * Do not send ipi's to offline vcpus and set flush_on_enter flag * For online vcpus: Wait for them to clear the flag The approach was discussed here: https://lkml.org/lkml/2012/2/20/157 v3: * use only one state variable for vcpu-running/flush_on_enter * use cmpxchg to update the state * adapt to Alex Shi's TLB flush optimization v2: * use ACCESS_ONCE so the value is not register cached * Separate HV and Guest code Suggested-by: Peter Zijlstra Signed-off-by: Nikunj A. Dadhania --- Pseudo Algo: ------------ Hypervisor ========== guest_exit() if (!(xchg(state, NOT_IN_GUEST) == SHOULD_FLUSH)) tlb_flush(vcpu); guest_enter() if (!(xchg(state, IN_GUEST) == SHOULD_FLUSH)) tlb_flush(vcpu); Guest ===== flushcpumask = cpumask; for_each_cpu(i, flushmask) { state = vs->state; if(!test_bit(IN_GUEST_MODE, state)) { if (cmpxchg(&vs->state, state, state | (1 << SHOULD_FLUSH)) == SUCCESS) cpumask_clear_cpu(flushmask,i) } } if(!empty(flushmask) smp_call_function_many(f->flushmask, flush_tlb_func) Summary: Author: --- arch/x86/include/asm/tlbflush.h | 11 +++++++++++ arch/x86/kernel/kvm.c | 4 +++- arch/x86/mm/tlb.c | 37 +++++++++++++++++++++++++++++++++++++ 3 files changed, 51 insertions(+), 1 deletions(-) -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html diff --git a/arch/x86/include/asm/tlbflush.h b/arch/x86/include/asm/tlbflush.h index 74a4433..0a343a1 100644 --- a/arch/x86/include/asm/tlbflush.h +++ b/arch/x86/include/asm/tlbflush.h @@ -119,6 +119,13 @@ static inline void native_flush_tlb_others(const struct cpumask *cpumask, { } +static inline void kvm_flush_tlb_others(const struct cpumask *cpumask, + struct mm_struct *mm, + unsigned long start, + unsigned long end) +{ +} + static inline void reset_lazy_tlbstate(void) { } @@ -153,6 +160,10 @@ void native_flush_tlb_others(const struct cpumask *cpumask, struct mm_struct *mm, unsigned long start, unsigned long end); +void kvm_flush_tlb_others(const struct cpumask *cpumask, + struct mm_struct *mm, unsigned long start, + unsigned long end); + #define TLBSTATE_OK 1 #define TLBSTATE_LAZY 2 diff --git a/arch/x86/kernel/kvm.c b/arch/x86/kernel/kvm.c index 37e6599..b538a31 100644 --- a/arch/x86/kernel/kvm.c +++ b/arch/x86/kernel/kvm.c @@ -501,8 +501,10 @@ void __init kvm_guest_init(void) apic_set_eoi_write(kvm_guest_apic_eoi_write); #ifdef CONFIG_PARAVIRT_TLB_FLUSH - if (kvm_para_has_feature(KVM_FEATURE_VCPU_STATE)) + if (kvm_para_has_feature(KVM_FEATURE_VCPU_STATE)) { has_vcpu_state = 1; + pv_mmu_ops.flush_tlb_others = kvm_flush_tlb_others; + } #endif #ifdef CONFIG_SMP diff --git a/arch/x86/mm/tlb.c b/arch/x86/mm/tlb.c index 613cd83..2399013 100644 --- a/arch/x86/mm/tlb.c +++ b/arch/x86/mm/tlb.c @@ -6,6 +6,7 @@ #include #include #include +#include #include #include @@ -119,6 +120,42 @@ static void flush_tlb_func(void *info) } +#ifdef CONFIG_KVM_GUEST + +DECLARE_PER_CPU(struct kvm_vcpu_state, vcpu_state) __aligned(64); + +void kvm_flush_tlb_others(const struct cpumask *cpumask, + struct mm_struct *mm, unsigned long start, + unsigned long end) +{ + struct flush_tlb_info info; + struct kvm_vcpu_state *v_state; + u64 state; + int cpu; + cpumask_t flushmask; + + cpumask_copy(&flushmask, cpumask); + info.flush_mm = mm; + info.flush_start = start; + info.flush_end = end; + /* + * We have to call flush only on online vCPUs. And + * queue flush_on_enter for pre-empted vCPUs + */ + for_each_cpu(cpu, to_cpumask(&flushmask)) { + v_state = &per_cpu(vcpu_state, cpu); + state = v_state->state; + if (!test_bit(KVM_VCPU_STATE_IN_GUEST_MODE, &state)) { + if (cmpxchg(&v_state->state, state, state | 1 << KVM_VCPU_STATE_SHOULD_FLUSH)) + cpumask_clear_cpu(cpu, to_cpumask(&flushmask)); + } + } + + if (!cpumask_empty(to_cpumask(&flushmask))) + smp_call_function_many(&flushmask, flush_tlb_func, &info, 1); +} +#endif /* CONFIG_KVM_GUEST */ + void native_flush_tlb_others(const struct cpumask *cpumask, struct mm_struct *mm, unsigned long start, unsigned long end)