From patchwork Tue Aug 21 11:27:17 2012 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Nikunj A. Dadhania" X-Patchwork-Id: 1354341 Return-Path: X-Original-To: patchwork-kvm@patchwork.kernel.org Delivered-To: patchwork-process-083081@patchwork2.kernel.org Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by patchwork2.kernel.org (Postfix) with ESMTP id 1D633DFB34 for ; Tue, 21 Aug 2012 11:28:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752308Ab2HUL2A (ORCPT ); Tue, 21 Aug 2012 07:28:00 -0400 Received: from e23smtp04.au.ibm.com ([202.81.31.146]:58782 "EHLO e23smtp04.au.ibm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751667Ab2HUL16 (ORCPT ); Tue, 21 Aug 2012 07:27:58 -0400 Received: from /spool/local by e23smtp04.au.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Tue, 21 Aug 2012 21:26:32 +1000 Received: from d23relay05.au.ibm.com (202.81.31.247) by e23smtp04.au.ibm.com (202.81.31.210) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Tue, 21 Aug 2012 21:26:30 +1000 Received: from d23av03.au.ibm.com (d23av03.au.ibm.com [9.190.234.97]) by d23relay05.au.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id q7LBJ4No25166032 for ; Tue, 21 Aug 2012 21:19:04 +1000 Received: from d23av03.au.ibm.com (loopback [127.0.0.1]) by d23av03.au.ibm.com (8.14.4/8.13.1/NCO v10.0 AVout) with ESMTP id q7LBRrup029094 for ; Tue, 21 Aug 2012 21:27:54 +1000 Received: from [9.124.35.230] (abhimanyu.in.ibm.com [9.124.35.230] (may be forged)) by d23av03.au.ibm.com (8.14.4/8.13.1/NCO v10.0 AVin) with ESMTP id q7LBRor2029033; Tue, 21 Aug 2012 21:27:51 +1000 Subject: [PATCH v4 5/8] KVM Guest: Add paravirt kvm_flush_tlb_others To: mtosatti@redhat.com, avi@redhat.com From: "Nikunj A. Dadhania" Cc: raghukt@linux.vnet.ibm.com, alex.shi@intel.com, kvm@vger.kernel.org, stefano.stabellini@eu.citrix.com, peterz@infradead.org, hpa@zytor.com, vsrivatsa@gmail.com, mingo@elte.hu Date: Tue, 21 Aug 2012 16:57:17 +0530 Message-ID: <20120821112649.3512.34920.stgit@abhimanyu> In-Reply-To: <20120821112346.3512.99814.stgit@abhimanyu.in.ibm.com> References: <20120821112346.3512.99814.stgit@abhimanyu.in.ibm.com> User-Agent: StGit/0.16-2-g0d85 MIME-Version: 1.0 x-cbid: 12082111-9264-0000-0000-0000022B99B9 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Nikunj A. Dadhania flush_tlb_others_ipi depends on lot of statics in tlb.c. Replicated the flush_tlb_others_ipi as kvm_flush_tlb_others to further adapt to paravirtualization. Use the vcpu state information inside the kvm_flush_tlb_others to avoid sending ipi to pre-empted vcpus. * Do not send ipi's to offline vcpus and set flush_on_enter flag * For online vcpus: Wait for them to clear the flag The approach was discussed here: https://lkml.org/lkml/2012/2/20/157 v3: * use only one state variable for vcpu-running/flush_on_enter * use cmpxchg to update the state * adapt to Alex Shi's TLB flush optimization v2: * use ACCESS_ONCE so the value is not register cached * Separate HV and Guest code Suggested-by: Peter Zijlstra Signed-off-by: Nikunj A. Dadhania --- Pseudo Algo: ------------ Hypervisor ========== guest_exit() if (!(xchg(state, NOT_IN_GUEST) == SHOULD_FLUSH)) tlb_flush(vcpu); guest_enter() if (!(xchg(state, IN_GUEST) == SHOULD_FLUSH)) tlb_flush(vcpu); Guest ===== flushcpumask = cpumask; for_each_cpu(i, flushmask) { state = vs->state; if(!test_bit(IN_GUEST_MODE, state)) { if (cmpxchg(&vs->state, state, state | (1 << SHOULD_FLUSH)) == SUCCESS) cpumask_clear_cpu(flushmask,i) } } smp_call_function_many(f->flushmask, flush_tlb_func) --- arch/x86/include/asm/tlbflush.h | 11 +++++++++++ arch/x86/kernel/kvm.c | 4 +++- arch/x86/mm/tlb.c | 36 ++++++++++++++++++++++++++++++++++++ 3 files changed, 50 insertions(+), 1 deletions(-) -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html diff --git a/arch/x86/include/asm/tlbflush.h b/arch/x86/include/asm/tlbflush.h index 74a4433..0a343a1 100644 --- a/arch/x86/include/asm/tlbflush.h +++ b/arch/x86/include/asm/tlbflush.h @@ -119,6 +119,13 @@ static inline void native_flush_tlb_others(const struct cpumask *cpumask, { } +static inline void kvm_flush_tlb_others(const struct cpumask *cpumask, + struct mm_struct *mm, + unsigned long start, + unsigned long end) +{ +} + static inline void reset_lazy_tlbstate(void) { } @@ -153,6 +160,10 @@ void native_flush_tlb_others(const struct cpumask *cpumask, struct mm_struct *mm, unsigned long start, unsigned long end); +void kvm_flush_tlb_others(const struct cpumask *cpumask, + struct mm_struct *mm, unsigned long start, + unsigned long end); + #define TLBSTATE_OK 1 #define TLBSTATE_LAZY 2 diff --git a/arch/x86/kernel/kvm.c b/arch/x86/kernel/kvm.c index 37e6599..b538a31 100644 --- a/arch/x86/kernel/kvm.c +++ b/arch/x86/kernel/kvm.c @@ -501,8 +501,10 @@ void __init kvm_guest_init(void) apic_set_eoi_write(kvm_guest_apic_eoi_write); #ifdef CONFIG_PARAVIRT_TLB_FLUSH - if (kvm_para_has_feature(KVM_FEATURE_VCPU_STATE)) + if (kvm_para_has_feature(KVM_FEATURE_VCPU_STATE)) { has_vcpu_state = 1; + pv_mmu_ops.flush_tlb_others = kvm_flush_tlb_others; + } #endif #ifdef CONFIG_SMP diff --git a/arch/x86/mm/tlb.c b/arch/x86/mm/tlb.c index 613cd83..645df99 100644 --- a/arch/x86/mm/tlb.c +++ b/arch/x86/mm/tlb.c @@ -6,6 +6,7 @@ #include #include #include +#include #include #include @@ -119,6 +120,41 @@ static void flush_tlb_func(void *info) } +#ifdef CONFIG_KVM_GUEST + +DECLARE_PER_CPU(struct kvm_vcpu_state, vcpu_state) __aligned(64); + +void kvm_flush_tlb_others(const struct cpumask *cpumask, + struct mm_struct *mm, unsigned long start, + unsigned long end) +{ + struct flush_tlb_info info; + struct kvm_vcpu_state *v_state; + u64 state; + int cpu; + cpumask_t flushmask; + + cpumask_copy(&flushmask, cpumask); + info.flush_mm = mm; + info.flush_start = start; + info.flush_end = end; + /* + * We have to call flush only on online vCPUs. And + * queue flush_on_enter for pre-empted vCPUs + */ + for_each_cpu(cpu, to_cpumask(&flushmask)) { + v_state = &per_cpu(vcpu_state, cpu); + state = v_state->state; + if (!test_bit(KVM_VCPU_STATE_IN_GUEST_MODE, &state)) { + if (cmpxchg(&v_state->state, state, state | 1 << KVM_VCPU_STATE_SHOULD_FLUSH)) + cpumask_clear_cpu(cpu, to_cpumask(&flushmask)); + } + } + + smp_call_function_many(&flushmask, flush_tlb_func, &info, 1); +} +#endif /* CONFIG_KVM_GUEST */ + void native_flush_tlb_others(const struct cpumask *cpumask, struct mm_struct *mm, unsigned long start, unsigned long end)