From patchwork Mon Jul 23 21:17:00 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 10541041 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 24A9A14BC for ; Mon, 23 Jul 2018 21:17:27 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 14F6F284E9 for ; Mon, 23 Jul 2018 21:17:27 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 094CB284FF; Mon, 23 Jul 2018 21:17:27 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A8204284E9 for ; Mon, 23 Jul 2018 21:17:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2388149AbeGWWU2 (ORCPT ); Mon, 23 Jul 2018 18:20:28 -0400 Received: from mail-pl0-f65.google.com ([209.85.160.65]:46107 "EHLO mail-pl0-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2388144AbeGWWU1 (ORCPT ); Mon, 23 Jul 2018 18:20:27 -0400 Received: by mail-pl0-f65.google.com with SMTP id t17-v6so740560ply.13 for ; Mon, 23 Jul 2018 14:17:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :in-reply-to:references; bh=SZHcZhifXOXjtd60Qib7a4jVoj0il8NjZLifmNuGnAM=; b=tEW0/g6YEr5ymzs6xZO1XJ3fN0NOZmxjaTgJE/CupOQhImlV5+C+ILN20CgiSNhucj sHyeBEB4xpp/2CQ8wUUhcecUZSdeYQcDWxc1XmxltPBeCur33quS/UdMVU55rvQtNqtV ht6A8hxdJ2/uwFcK+V6tsS6B+7F6k4m+MaLhcxFaBd6F0Wtum6FfSYLgeCd+eAFqL6p9 jteN5y5qBndEwaB9AR+vVWIRx2xYK7WUILruNfN9WgoVomALGKEGv5et41/5YS4OSxTW itqaRWk2FQOyYVJf1WchFeoRB8DECqiNey88JxwzzaB/x06HCDbPS68J0Xwod4f4VosU Bsag== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:in-reply-to:references; bh=SZHcZhifXOXjtd60Qib7a4jVoj0il8NjZLifmNuGnAM=; b=lYZTRHoCl4NpApIa6D6i8ZKfP1HQFgYRLugJd2kttvkOMkNqH/4BU5LXTBLiZPqoNV PMeWFos5B67I1JiForZZ2h3hbBqxll2OwTOpcB+FyHZiC8sK+OaKRJK0v7vSUZusB//8 QTFW/m8/PFtXNlBpfQkQRzF3kEX/2rj4KfH4Q8URWhhO8JxvO9gBLBSuQzG7ID2dY+Sf XI0kcAOUN5HCfZcjT2/hyevS5Rl2FAlKWtxgtVs20Hl5WLZWMbR0TzUI2LDeda4mfR4Y 3Yg3uapY04WOM1L3x7lSvPmbT2FKb1BseLSkdpodJiTEfszSI5rQqh4D+JRu296Yl5x0 gmLA== X-Gm-Message-State: AOUpUlHmvg40VV4lBfMTHdFJemhLwqmX+4UErhZxvfaelS5gId896nLg hVvN4I/hen0mNUJfWMyiBc/UUuta X-Google-Smtp-Source: AAOMgpckDYM1ddMsH8xfoP+G7ymrT47r1SFWQ6Nvr4k5GXQ6mJ87jXNDq7jd6cVY+nnaxPTMz3RkLg== X-Received: by 2002:a17:902:650c:: with SMTP id b12-v6mr14415010plk.31.1532380643519; Mon, 23 Jul 2018 14:17:23 -0700 (PDT) Received: from localhost ([192.55.54.58]) by smtp.gmail.com with ESMTPSA id e16-v6sm12746205pfn.46.2018.07.23.14.17.22 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 23 Jul 2018 14:17:22 -0700 (PDT) From: Isaku Yamahata To: kvm@vger.kernel.org Cc: pbonzini@redhat.com, rkrcmar@redhat.com, isaku.yamahata@gmail.com, Isaku Yamahata Subject: [RFC PATCH 1/3] kvm/x86: remove KVM_REQ_PENDING_TIMER Date: Mon, 23 Jul 2018 14:17:00 -0700 Message-Id: <1804887df333691553d84946949afd784bf61e9e.1532380136.git.isaku.yamahata@gmail.com> X-Mailer: git-send-email 2.14.1 In-Reply-To: References: In-Reply-To: References: Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Isaku Yamahata Since it doesn't make sense as kvm_cpu_has_pending_timer replaced it, remove dead code. Signed-off-by: Isaku Yamahata --- arch/x86/kvm/lapic.c | 1 - arch/x86/kvm/x86.c | 11 ----------- include/linux/kvm_host.h | 2 +- 3 files changed, 1 insertion(+), 13 deletions(-) diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c index b5cd8465d44f..2c4d19afc76d 100644 --- a/arch/x86/kvm/lapic.c +++ b/arch/x86/kvm/lapic.c @@ -1372,7 +1372,6 @@ static void apic_timer_expired(struct kvm_lapic *apic) return; atomic_inc(&apic->lapic_timer.pending); - kvm_set_pending_timer(vcpu); /* * For x86, the atomic_inc() is serialized, thus diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 2b812b3c5088..3b9165dae5c6 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -1283,16 +1283,6 @@ static void update_pvclock_gtod(struct timekeeper *tk) } #endif -void kvm_set_pending_timer(struct kvm_vcpu *vcpu) -{ - /* - * Note: KVM_REQ_PENDING_TIMER is implicitly checked in - * vcpu_enter_guest. This function is only called from - * the physical CPU that is running vcpu. - */ - kvm_make_request(KVM_REQ_PENDING_TIMER, vcpu); -} - static void kvm_write_wall_clock(struct kvm *kvm, gpa_t wall_clock) { int version; @@ -7592,7 +7582,6 @@ static int vcpu_run(struct kvm_vcpu *vcpu) if (r <= 0) break; - kvm_clear_request(KVM_REQ_PENDING_TIMER, vcpu); if (kvm_cpu_has_pending_timer(vcpu)) kvm_inject_pending_timer_irqs(vcpu); diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 4ee7bc548a83..79877bd2a8c6 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -122,10 +122,10 @@ static inline bool is_error_page(struct page *page) /* * Architecture-independent vcpu->requests bit members * Bits 4-7 are reserved for more arch-independent bits. + * Bit 2 was used before as KVM_REQ_PENDING_TIMER. but not now. */ #define KVM_REQ_TLB_FLUSH (0 | KVM_REQUEST_WAIT | KVM_REQUEST_NO_WAKEUP) #define KVM_REQ_MMU_RELOAD (1 | KVM_REQUEST_WAIT | KVM_REQUEST_NO_WAKEUP) -#define KVM_REQ_PENDING_TIMER 2 #define KVM_REQ_UNHALT 3 #define KVM_REQUEST_ARCH_BASE 8 From patchwork Mon Jul 23 21:17:01 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 10541043 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D2E14112E for ; Mon, 23 Jul 2018 21:17:29 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C263C284E9 for ; Mon, 23 Jul 2018 21:17:29 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B6708284FF; Mon, 23 Jul 2018 21:17:29 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E669A284E9 for ; Mon, 23 Jul 2018 21:17:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2388155AbeGWWUa (ORCPT ); Mon, 23 Jul 2018 18:20:30 -0400 Received: from mail-pg1-f194.google.com ([209.85.215.194]:43696 "EHLO mail-pg1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2388144AbeGWWUa (ORCPT ); Mon, 23 Jul 2018 18:20:30 -0400 Received: by mail-pg1-f194.google.com with SMTP id v13-v6so1245491pgr.10 for ; Mon, 23 Jul 2018 14:17:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :in-reply-to:references; bh=heCGx1NuP6PCHpOP6gioYhq5bpp+LrfZRu0wEXLxuz4=; b=LjdlTWzv8N93iqDvoKt7b4/KE/KM427Bx85RCL+7Lg5faaes0On/93XiNU4if4VSPz 7UE472SQpPRhqYvIQsmQuocJAT9ArWYps34cIasUkDsKBUGaH1PNAAzySvCl5fgbDb6V fcwtLW3/5waJ4ePrb0zf2KLaA42yT9jBkiWz/JkcF2hQdicZFCkKgpuF+SDOjfHpEerm WXHRsAe+Jh800QQRVpNZKkMRX/GurvSpwXwctSNr5G0z20cDaqa7KReRu/auqENQUlj9 o0C4vg62A7ZsJ8pUwZgrxpsMu0vOOyQxj2iSLQ2UaflkxjPnz+89OsXPjZvWhq8etWBt ex5A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:in-reply-to:references; bh=heCGx1NuP6PCHpOP6gioYhq5bpp+LrfZRu0wEXLxuz4=; b=RN2dJCNy7qNd4inGpLmgYWzRKVJc75ZP0VkbqdZqrOFxfKxZ3lyCsFd6LRtJapsw1r VOV0WGL984tYwLVYQqUG297ZHO5pWIKRyzLndf8uQmq00SXxqE8TSpf+br5qt8PDbcs3 lpNqFsQLICmtendP5Qe+zpL3cMzMSfqEQRNalWx/ofxeErK0/iVYVIfRT/0JAzlCH6pM ju5vH4y9IgL/xpG0s8n2aMagq6lMnqIvwFRvdt7JACmlo+nqg/h2bk5AYNf24cL3RodF 2vLAS6OC3bLV/9t4IzLrMIoMQ0GwHE0Nf3larQJYoEFBphJiP6DHhRBLXPvOxKPSWY06 uvKw== X-Gm-Message-State: AOUpUlEr/Qo7OsSNa20UuRohK0l7joiUA0xiUt9zdV7qtHulIITlV1zR oBBdQTLeFY0kRATuuzw1cZxNErSy X-Google-Smtp-Source: AAOMgpdx+evJ8E7vz+pxrwfIY+Y3n4V5GLL7VdktSIIGG7AWT9/O/8DLiRSP7ukObEcEF2EEpJEibg== X-Received: by 2002:a63:3190:: with SMTP id x138-v6mr13573386pgx.60.1532380645532; Mon, 23 Jul 2018 14:17:25 -0700 (PDT) Received: from localhost ([192.55.54.58]) by smtp.gmail.com with ESMTPSA id l13-v6sm11982344pgs.15.2018.07.23.14.17.24 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 23 Jul 2018 14:17:24 -0700 (PDT) From: Isaku Yamahata To: kvm@vger.kernel.org Cc: pbonzini@redhat.com, rkrcmar@redhat.com, isaku.yamahata@gmail.com, Isaku Yamahata Subject: [RFC PATCH 2/3] x86/kvm/vmx: opitmize tsc dealine timer emulation Date: Mon, 23 Jul 2018 14:17:01 -0700 Message-Id: <4f3be35206af83a6e95a98ef6f7448b3899058d2.1532380136.git.isaku.yamahata@gmail.com> X-Mailer: git-send-email 2.14.1 In-Reply-To: References: In-Reply-To: References: Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Isaku Yamahata This patch tries to optimize tsc dealine timer emulation by skipping various checks when possible. This patch implement fast exit handler for - preempt exit handler to emulate tsc deadline timer - wrmsr exit handler to emulate write to tsc deadline The result can be measured by cyclic test and also by ftracing local timer interrupt so that local timer interrupt can be injected faster. Signed-off-by: Isaku Yamahata --- arch/x86/kvm/lapic.c | 17 +---- arch/x86/kvm/lapic.h | 16 +++++ arch/x86/kvm/vmx.c | 148 +++++++++++++++++++++++++++++++++++++++++++ arch/x86/kvm/x86.c | 10 +-- 4 files changed, 171 insertions(+), 20 deletions(-) diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c index 2c4d19afc76d..46e45f9288da 100644 --- a/arch/x86/kvm/lapic.c +++ b/arch/x86/kvm/lapic.c @@ -99,6 +99,7 @@ static inline int __apic_test_and_clear_vector(int vec, void *bitmap) } struct static_key_deferred apic_hw_disabled __read_mostly; +EXPORT_SYMBOL_GPL(apic_hw_disabled); struct static_key_deferred apic_sw_disabled __read_mostly; static inline int apic_enabled(struct kvm_lapic *apic) @@ -292,21 +293,6 @@ static inline int apic_lvt_vector(struct kvm_lapic *apic, int lvt_type) return kvm_lapic_get_reg(apic, lvt_type) & APIC_VECTOR_MASK; } -static inline int apic_lvtt_oneshot(struct kvm_lapic *apic) -{ - return apic->lapic_timer.timer_mode == APIC_LVT_TIMER_ONESHOT; -} - -static inline int apic_lvtt_period(struct kvm_lapic *apic) -{ - return apic->lapic_timer.timer_mode == APIC_LVT_TIMER_PERIODIC; -} - -static inline int apic_lvtt_tscdeadline(struct kvm_lapic *apic) -{ - return apic->lapic_timer.timer_mode == APIC_LVT_TIMER_TSCDEADLINE; -} - static inline int apic_lvt_nmi_mode(u32 lvt_val) { return (lvt_val & (APIC_MODE_MASK | APIC_LVT_MASKED)) == APIC_DM_NMI; @@ -2462,6 +2448,7 @@ void kvm_lapic_sync_to_vapic(struct kvm_vcpu *vcpu) kvm_write_guest_cached(vcpu->kvm, &vcpu->arch.apic->vapic_cache, &data, sizeof(u32)); } +EXPORT_SYMBOL_GPL(kvm_lapic_sync_to_vapic); int kvm_lapic_set_vapic_addr(struct kvm_vcpu *vcpu, gpa_t vapic_addr) { diff --git a/arch/x86/kvm/lapic.h b/arch/x86/kvm/lapic.h index ed0ed39abd36..5ce944a3ed9b 100644 --- a/arch/x86/kvm/lapic.h +++ b/arch/x86/kvm/lapic.h @@ -105,6 +105,7 @@ void kvm_set_lapic_tscdeadline_msr(struct kvm_vcpu *vcpu, u64 data); void kvm_apic_write_nodecode(struct kvm_vcpu *vcpu, u32 offset); void kvm_apic_set_eoi_accelerated(struct kvm_vcpu *vcpu, int vector); +void kvm_update_cr8_intercept(struct kvm_vcpu *vcpu); int kvm_lapic_set_vapic_addr(struct kvm_vcpu *vcpu, gpa_t vapic_addr); void kvm_lapic_sync_from_vapic(struct kvm_vcpu *vcpu); void kvm_lapic_sync_to_vapic(struct kvm_vcpu *vcpu); @@ -234,4 +235,19 @@ static inline enum lapic_mode kvm_apic_mode(u64 apic_base) return apic_base & (MSR_IA32_APICBASE_ENABLE | X2APIC_ENABLE); } +static inline int apic_lvtt_oneshot(struct kvm_lapic *apic) +{ + return apic->lapic_timer.timer_mode == APIC_LVT_TIMER_ONESHOT; +} + +static inline int apic_lvtt_period(struct kvm_lapic *apic) +{ + return apic->lapic_timer.timer_mode == APIC_LVT_TIMER_PERIODIC; +} + +static inline int apic_lvtt_tscdeadline(struct kvm_lapic *apic) +{ + return apic->lapic_timer.timer_mode == APIC_LVT_TIMER_TSCDEADLINE; +} + #endif diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c index e30da9a2430c..08d13febc4b5 100644 --- a/arch/x86/kvm/vmx.c +++ b/arch/x86/kvm/vmx.c @@ -9971,10 +9971,137 @@ static void vmx_arm_hv_timer(struct kvm_vcpu *vcpu) vmcs_write32(VMX_PREEMPTION_TIMER_VALUE, delta_tsc); } +#ifdef CONFIG_X86_64 +static int vmx_set_hv_timer(struct kvm_vcpu *vcpu, u64 guest_deadline_tsc); +static void vmx_cancel_hv_timer(struct kvm_vcpu *vcpu); +#endif + +/* + * handle tsc deadline setting in fast path + */ +static int handle_wrmsr_tscdeadline_fast(struct kvm_vcpu *vcpu, u64 data) +{ + int ret = -EOPNOTSUPP; +#ifdef CONFIG_X86_64 + struct kvm_lapic *apic = vcpu->arch.apic; + + if (!kvm_x86_ops->set_hv_timer || + !lapic_in_kernel(vcpu) || + apic_lvtt_oneshot(apic) || + apic_lvtt_period(apic)) { + goto out; + } + /* simplified version of kvm_set_lapic_tscdeadline_msr(vcpu, msr.data) + * lapic timer advance should be addressed? + */ + atomic_set(&apic->lapic_timer.pending, 0); + apic->lapic_timer.tscdeadline = data; + if (data) { + apic->lapic_timer.hv_timer_in_use = true; + ret = vmx_set_hv_timer(vcpu, data) < 0 ? -1 : 0; + } +out: +#endif + return ret; +} + +static int handle_wrmsr_fast(struct kvm_vcpu *vcpu) +{ + /* + * fast path of handle_wrmsr() -> kvm_set_msr() -> vmx_set_msr() + */ + int ret = -EOPNOTSUPP; + + struct msr_data msr; + u32 ecx = vcpu->arch.regs[VCPU_REGS_RCX]; + u64 data = (vcpu->arch.regs[VCPU_REGS_RAX] & -1u) + | ((u64)(vcpu->arch.regs[VCPU_REGS_RDX] & -1u) << 32); + + msr.data = data; + msr.index = ecx; + msr.host_initiated = false; + + /* simplified version of kvm_set_msr(vcpu, &msr) */ + switch (msr.index) { + case MSR_IA32_TSCDEADLINE: + ret = handle_wrmsr_tscdeadline_fast(vcpu, data); + break; + /* more MSRs to be optimized? */ + default: + break; + } + + if (ret == 0) + ret = kvm_skip_emulated_instruction(vcpu) ? 0 : -EOPNOTSUPP; + return ret; +} + +static int handle_preemption_timer_fast(struct kvm_vcpu *vcpu) +{ + int ret = -EOPNOTSUPP; +#ifdef CONFIG_X86_64 + struct vcpu_vmx *vmx = to_vmx(vcpu); + struct kvm_lapic *apic = vcpu->arch.apic; + struct kvm_timer *ktimer = &apic->lapic_timer; + u32 reg; + int vector; + + /* don't care complicated cases */ + if (!vmx_interrupt_allowed(vcpu) || + vmx->rmode.vm86_active || + !apic_lvtt_tscdeadline(apic) || + !kvm_lapic_hv_timer_in_use(vcpu) || + is_smm(vcpu) || + atomic_read(&apic->lapic_timer.pending) || + !kvm_apic_hw_enabled(apic)) + goto out; + vmx_cancel_hv_timer(vcpu); + ktimer->hv_timer_in_use = false; + ktimer->expired_tscdeadline = ktimer->tscdeadline; + apic->lapic_timer.tscdeadline = 0; + + reg = kvm_lapic_get_reg(apic, APIC_LVTT); + if (reg & APIC_LVT_MASKED) + goto out; + vector = reg & APIC_VECTOR_MASK; + kvm_lapic_set_vector(vector, apic->regs + APIC_TMR); + if (vcpu->arch.apicv_active) { + /* simplified version of + * vmx_deliver_posted_interrupt(vcpu, vector); + */ + pi_test_and_set_pir(vector, &vmx->pi_desc); + pi_test_and_set_on(&vmx->pi_desc); + } else { + kvm_lapic_set_irr(vector, apic); + + if (!vcpu->arch.apicv_active) + kvm_update_cr8_intercept(vcpu); + /* simplified version of vmx_inject_irq() */ + vmcs_write32(VM_ENTRY_INTR_INFO_FIELD, + vector | INTR_INFO_VALID_MASK | + INTR_TYPE_EXT_INTR); + vmx_clear_hlt(vcpu); + } + ret = 0; +out: +#endif + return ret; +} + +static int (*const kvm_vmx_exit_handlers_fast[]) (struct kvm_vcpu *vcpu) = { + [EXIT_REASON_MSR_WRITE] = handle_wrmsr_fast, + [EXIT_REASON_PREEMPTION_TIMER] = handle_preemption_timer_fast, +}; + +static const int kvm_vmx_max_exit_handlers_fast = + ARRAY_SIZE(kvm_vmx_exit_handlers_fast); + + static void __noclone vmx_vcpu_run(struct kvm_vcpu *vcpu) { struct vcpu_vmx *vmx = to_vmx(vcpu); unsigned long cr3, cr4, evmcs_rsp; +continue_vmx_vcpu_run: /* Record the guest's net vcpu time for enforced NMI injections. */ if (unlikely(!enable_vnmi && @@ -10242,6 +10369,27 @@ static void __noclone vmx_vcpu_run(struct kvm_vcpu *vcpu) vmx_complete_atomic_exit(vmx); vmx_recover_nmi_blocking(vmx); + + if (!is_guest_mode(vcpu) && + vmx->exit_reason < kvm_vmx_max_exit_handlers_fast && + kvm_vmx_exit_handlers_fast[vmx->exit_reason]) { + bool idtv_info_valid = + vmx->idt_vectoring_info & VECTORING_INFO_VALID_MASK; + int type = vmx->idt_vectoring_info & VECTORING_INFO_TYPE_MASK; + + if (!idtv_info_valid || + !(type == INTR_TYPE_NMI_INTR || + type == INTR_TYPE_SOFT_EXCEPTION || + type == INTR_TYPE_SOFT_INTR || + type == INTR_TYPE_EXT_INTR)) { + int ret = kvm_vmx_exit_handlers_fast[vmx->exit_reason]( + vcpu); + + if (ret == 0) + goto continue_vmx_vcpu_run; + } + } + vmx_complete_interrupts(vmx); } STACK_FRAME_NON_STANDARD(vmx_vcpu_run); diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 3b9165dae5c6..befd32afd9d7 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -98,7 +98,6 @@ static u64 __read_mostly efer_reserved_bits = ~((u64)EFER_SCE); #define KVM_X2APIC_API_VALID_FLAGS (KVM_X2APIC_API_USE_32BIT_IDS | \ KVM_X2APIC_API_DISABLE_BROADCAST_QUIRK) -static void update_cr8_intercept(struct kvm_vcpu *vcpu); static void process_nmi(struct kvm_vcpu *vcpu); static void enter_smm(struct kvm_vcpu *vcpu); static void __kvm_set_rflags(struct kvm_vcpu *vcpu, unsigned long rflags); @@ -3158,7 +3157,7 @@ static int kvm_vcpu_ioctl_set_lapic(struct kvm_vcpu *vcpu, r = kvm_apic_set_state(vcpu, s); if (r) return r; - update_cr8_intercept(vcpu); + kvm_update_cr8_intercept(vcpu); return 0; } @@ -6775,7 +6774,7 @@ static void post_kvm_run_save(struct kvm_vcpu *vcpu) kvm_vcpu_ready_for_interrupt_injection(vcpu); } -static void update_cr8_intercept(struct kvm_vcpu *vcpu) +void kvm_update_cr8_intercept(struct kvm_vcpu *vcpu) { int max_irr, tpr; @@ -6800,6 +6799,7 @@ static void update_cr8_intercept(struct kvm_vcpu *vcpu) kvm_x86_ops->update_cr8_intercept(vcpu, tpr, max_irr); } +EXPORT_SYMBOL_GPL(kvm_update_cr8_intercept); static int inject_pending_event(struct kvm_vcpu *vcpu, bool req_int_win) { @@ -7372,7 +7372,7 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu) } if (kvm_lapic_enabled(vcpu)) { - update_cr8_intercept(vcpu); + kvm_update_cr8_intercept(vcpu); kvm_lapic_sync_to_vapic(vcpu); } } @@ -8069,7 +8069,7 @@ static int __set_sregs(struct kvm_vcpu *vcpu, struct kvm_sregs *sregs) kvm_set_segment(vcpu, &sregs->tr, VCPU_SREG_TR); kvm_set_segment(vcpu, &sregs->ldt, VCPU_SREG_LDTR); - update_cr8_intercept(vcpu); + kvm_update_cr8_intercept(vcpu); /* Older userspace won't unhalt the vcpu on reset. */ if (kvm_vcpu_is_bsp(vcpu) && kvm_rip_read(vcpu) == 0xfff0 && From patchwork Mon Jul 23 21:17:02 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 10541045 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EE6C614BC for ; Mon, 23 Jul 2018 21:17:30 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DEA5D284E9 for ; Mon, 23 Jul 2018 21:17:30 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D27C9284FF; Mon, 23 Jul 2018 21:17:30 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 85CE7284E9 for ; Mon, 23 Jul 2018 21:17:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2388162AbeGWWUc (ORCPT ); Mon, 23 Jul 2018 18:20:32 -0400 Received: from mail-pl0-f65.google.com ([209.85.160.65]:40424 "EHLO mail-pl0-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2388158AbeGWWUc (ORCPT ); Mon, 23 Jul 2018 18:20:32 -0400 Received: by mail-pl0-f65.google.com with SMTP id s17-v6so745953plp.7 for ; Mon, 23 Jul 2018 14:17:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :in-reply-to:references; bh=7UcEFdWDT7KlFraf+xTG31mc50jqqeE/MliEPJpdoWs=; b=r8vLuvMX9SYCfZKzEJUhpiIgQJ2xsUipwrTpXWvseMISksgdKZgAIReFO7K4DFdYb5 icFfjTnDXozwMh0ago/9rEchbtia3ltF80/d10QzB6mNveo7vPji4zmW2CArcPWrbrgf FVKyKm8A7qiFXW87YmIfVizBLgIc7XOJ/yqbMXnnkSCVp3S0SoNIFzXIRLnUE13CDlWi s3QqjqIyUDu9u+NBgmKvdIZbjixjKGXY15eyJ94rGzs0CCGsGpFmmBdnIJXYeLHSMt9p gg9SVQjmv3lYi2Tw23A8Ex5raJVCDatTKfRJc7I7HcgCcMm94Pl9fDcDbMJaAXdNVJgZ dEAA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:in-reply-to:references; bh=7UcEFdWDT7KlFraf+xTG31mc50jqqeE/MliEPJpdoWs=; b=ZltISkQYkpqypOtn3Cf31RlE5XlDUpUqvO0ixNkyBv6MsVJ8C7QkNlCfzDzmbqyXMg v2+4Qg0aHtTBEFKpOsUcgUJKGXdWX6muVhkjYtQqJ0anwwk7rgCHm2eaj/sXNFH+8DVa gA+FYZIf2l72qU8P2Q7SnGsnZ+ZTcVDlXeJIRC10/XU7eOW/R/gr6kZ0OUxdDr4RpooD q+M/l/cBqDke/0s07xp3HLMnMhcNfT1DACVSI0A6jPFktvMk21MmprW4CwpvHOtX0Tct wMJ7uPaIh1AZBd6fqhEI5PDKxhFdDS9mAZHVWYBm4u0PGHmoq1tC9HlhizIrSZZhint4 QIww== X-Gm-Message-State: AOUpUlHftrAh2o9wyI32y5K159mmgD8woMJ9VpRLpVFtV9MpS1pv0pDA dGu1zYAydtLzSkNEAIdpxFxx2kDw X-Google-Smtp-Source: AAOMgpf98SPG0Crxg+vS+OFaHv16TdcpfmabkubLiSCvp3jc3IwvpatEJlVRft4ay6Iyoeenvty3gQ== X-Received: by 2002:a17:902:724c:: with SMTP id c12-v6mr14021929pll.326.1532380647641; Mon, 23 Jul 2018 14:17:27 -0700 (PDT) Received: from localhost ([192.55.54.58]) by smtp.gmail.com with ESMTPSA id v15-v6sm12913288pff.120.2018.07.23.14.17.26 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 23 Jul 2018 14:17:26 -0700 (PDT) From: Isaku Yamahata To: kvm@vger.kernel.org Cc: pbonzini@redhat.com, rkrcmar@redhat.com, isaku.yamahata@gmail.com, Isaku Yamahata Subject: [RFC PATCH 3/3] x86/kvm/vmx: module parameter to enable/disable fast exit handler Date: Mon, 23 Jul 2018 14:17:02 -0700 Message-Id: X-Mailer: git-send-email 2.14.1 In-Reply-To: References: In-Reply-To: References: Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Isaku Yamahata This is just for convenience for benchmark. Signed-off-by: Isaku Yamahata --- arch/x86/kvm/vmx.c | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c index 08d13febc4b5..a4971b778794 100644 --- a/arch/x86/kvm/vmx.c +++ b/arch/x86/kvm/vmx.c @@ -10096,6 +10096,9 @@ static int (*const kvm_vmx_exit_handlers_fast[]) (struct kvm_vcpu *vcpu) = { static const int kvm_vmx_max_exit_handlers_fast = ARRAY_SIZE(kvm_vmx_exit_handlers_fast); +static bool __ro_after_init enable_fast_exit_handler = true; +module_param_named(fast_exit_handler, enable_fast_exit_handler, bool, 0444); +static DEFINE_STATIC_KEY_TRUE(enable_fast_exit_handler_key); static void __noclone vmx_vcpu_run(struct kvm_vcpu *vcpu) { @@ -10370,7 +10373,8 @@ static void __noclone vmx_vcpu_run(struct kvm_vcpu *vcpu) vmx_complete_atomic_exit(vmx); vmx_recover_nmi_blocking(vmx); - if (!is_guest_mode(vcpu) && + if (static_branch_likely(&enable_fast_exit_handler_key) && + !is_guest_mode(vcpu) && vmx->exit_reason < kvm_vmx_max_exit_handlers_fast && kvm_vmx_exit_handlers_fast[vmx->exit_reason]) { bool idtv_info_valid = @@ -13291,6 +13295,11 @@ static int __init vmx_init(void) { int r; + if (enable_fast_exit_handler) + static_branch_enable(&enable_fast_exit_handler_key); + else + static_branch_disable(&enable_fast_exit_handler_key); + #if IS_ENABLED(CONFIG_HYPERV) /* * Enlightened VMCS usage should be recommended and the host needs