From patchwork Fri Apr 24 06:22:40 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wanpeng Li X-Patchwork-Id: 11507173 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B465592A for ; Fri, 24 Apr 2020 06:23:03 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9C76520704 for ; Fri, 24 Apr 2020 06:23:03 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="r9EXwphq" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726593AbgDXGXA (ORCPT ); Fri, 24 Apr 2020 02:23:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51262 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1726008AbgDXGW7 (ORCPT ); Fri, 24 Apr 2020 02:22:59 -0400 Received: from mail-pf1-x442.google.com (mail-pf1-x442.google.com [IPv6:2607:f8b0:4864:20::442]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A5FB5C09B045; Thu, 23 Apr 2020 23:22:59 -0700 (PDT) Received: by mail-pf1-x442.google.com with SMTP id x2so612902pfx.7; Thu, 23 Apr 2020 23:22:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=UFT0hKfL7iS8UZisqhI7Q6xNbcamwNw/NX1yQQKe78k=; b=r9EXwphqugPnCRsU9hun3rEjYYMPFF5n3SJRgkB06t8M1LLomn2v/KHS7FYZ8QsaTF OZhrIsF24j2K9W3BvcBzWpkJtNptx2ECz0VvKoqPNqW571yWQ2gxidDiGKd90GltpbT3 I/EZLu/35Vf74VFUQ3hWOw2BggmuQXaNuV7nNg+7nghNldbyuqGqm3JkFx93pMoOBnRG kVg3X6koFKGnLYCClKcFJFLoSBpP78pbnKyeOLI+MrkWrs1NCcaQekDcQM35UBP89ncS Zn7IO01wHLBh3q46+r/kwkxK56XEoejS7l+C5IEM6p5L8LqZ2dUOwcii/oAsRVwv0dvS s3Kw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=UFT0hKfL7iS8UZisqhI7Q6xNbcamwNw/NX1yQQKe78k=; b=hzrja8lhh1APcRERjiMLdVltbQ1waWo8pxW+YeKNvPBJqKXnwvzqut8EIrcukj4sFJ EIgIeJx4h4VKXdk/BTwWl6YqVl6zNpPhFNLqZS8X2TFuJY5iFCJ/61KQwdrM+gtdKbQ8 sa2iIwcUqAAJlhTvPbdYolBdJbQJjM4Svol7O9uYakisblVZDQm6YPQoT05uNdvIYezj QKk+OzFEE0/VY9/Lil88D7WCJ5UjA5Ozs/tpUUWHL2K7Z7QjVX3j2OEJSwVW0q1Bww/K d44UNgz8l1aUq90rENueYbWmEHIs9G2T3lOO5jpOrzlDtOonqGD8VM6a5BbVjwLi4Li8 SPSg== X-Gm-Message-State: AGi0PubhdLFATJwf6eZo7uXZwkn2cFM9ztB1bAD5f7q7Vo5TnCplzDCF y2LfON2OTRX8dUy71YnXF88DqRrH X-Google-Smtp-Source: APiQypJxuwuvFp5xaaFdP2nRbicoYJCzqR9KIgDJ6O55/vXmZ012YMWjjUMzMCASJnAKzYj6mBRriw== X-Received: by 2002:a62:7656:: with SMTP id r83mr7763612pfc.71.1587709378973; Thu, 23 Apr 2020 23:22:58 -0700 (PDT) Received: from localhost.localdomain ([103.7.29.6]) by smtp.googlemail.com with ESMTPSA id l30sm3920674pgu.29.2020.04.23.23.22.56 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 23 Apr 2020 23:22:58 -0700 (PDT) From: Wanpeng Li X-Google-Original-From: Wanpeng Li To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: Paolo Bonzini , Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , Haiwei Li Subject: [PATCH v3 1/5] KVM: VMX: Introduce generic fastpath handler Date: Fri, 24 Apr 2020 14:22:40 +0800 Message-Id: <1587709364-19090-2-git-send-email-wanpengli@tencent.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1587709364-19090-1-git-send-email-wanpengli@tencent.com> References: <1587709364-19090-1-git-send-email-wanpengli@tencent.com> Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Wanpeng Li Introduce generic fastpath handler to handle MSR fastpath, VMX-preemption timer fastpath etc. In addition, we can't observe benefit from single target IPI fastpath when APICv is disabled, let's just enable IPI and Timer fastpath when APICv is enabled for now. Tested-by: Haiwei Li Cc: Haiwei Li Signed-off-by: Wanpeng Li --- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/vmx/vmx.c | 25 ++++++++++++++++++++----- 2 files changed, 21 insertions(+), 5 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index f26df2c..6397723 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -188,6 +188,7 @@ enum { enum exit_fastpath_completion { EXIT_FASTPATH_NONE, EXIT_FASTPATH_SKIP_EMUL_INS, + EXIT_FASTPATH_CONT_RUN, }; struct x86_emulate_ctxt; diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 766303b..f1f6638 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -6559,6 +6559,20 @@ void vmx_update_host_rsp(struct vcpu_vmx *vmx, unsigned long host_rsp) } } +static enum exit_fastpath_completion vmx_exit_handlers_fastpath(struct kvm_vcpu *vcpu) +{ + if (!is_guest_mode(vcpu) && vcpu->arch.apicv_active) { + switch (to_vmx(vcpu)->exit_reason) { + case EXIT_REASON_MSR_WRITE: + return handle_fastpath_set_msr_irqoff(vcpu); + default: + return EXIT_FASTPATH_NONE; + } + } + + return EXIT_FASTPATH_NONE; +} + bool __vmx_vcpu_run(struct vcpu_vmx *vmx, unsigned long *regs, bool launched); static enum exit_fastpath_completion vmx_vcpu_run(struct kvm_vcpu *vcpu) @@ -6567,6 +6581,7 @@ static enum exit_fastpath_completion vmx_vcpu_run(struct kvm_vcpu *vcpu) struct vcpu_vmx *vmx = to_vmx(vcpu); unsigned long cr3, cr4; +cont_run: /* Record the guest's net vcpu time for enforced NMI injections. */ if (unlikely(!enable_vnmi && vmx->loaded_vmcs->soft_vnmi_blocked)) @@ -6733,17 +6748,17 @@ static enum exit_fastpath_completion vmx_vcpu_run(struct kvm_vcpu *vcpu) if (unlikely(vmx->exit_reason & VMX_EXIT_REASONS_FAILED_VMENTRY)) return EXIT_FASTPATH_NONE; - if (!is_guest_mode(vcpu) && vmx->exit_reason == EXIT_REASON_MSR_WRITE) - exit_fastpath = handle_fastpath_set_msr_irqoff(vcpu); - else - exit_fastpath = EXIT_FASTPATH_NONE; - vmx->loaded_vmcs->launched = 1; vmx->idt_vectoring_info = vmcs_read32(IDT_VECTORING_INFO_FIELD); vmx_recover_nmi_blocking(vmx); vmx_complete_interrupts(vmx); + exit_fastpath = vmx_exit_handlers_fastpath(vcpu); + /* static call is better with retpolines */ + if (exit_fastpath == EXIT_FASTPATH_CONT_RUN) + goto cont_run; + return exit_fastpath; } From patchwork Fri Apr 24 06:22:41 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wanpeng Li X-Patchwork-Id: 11507181 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0D62715AB for ; Fri, 24 Apr 2020 06:23:24 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E77272084D for ; Fri, 24 Apr 2020 06:23:23 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="iD9AF3OB" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726647AbgDXGXE (ORCPT ); Fri, 24 Apr 2020 02:23:04 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51272 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1725889AbgDXGXD (ORCPT ); Fri, 24 Apr 2020 02:23:03 -0400 Received: from mail-pg1-x544.google.com (mail-pg1-x544.google.com [IPv6:2607:f8b0:4864:20::544]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DA185C09B045; Thu, 23 Apr 2020 23:23:02 -0700 (PDT) Received: by mail-pg1-x544.google.com with SMTP id o10so4160699pgb.6; Thu, 23 Apr 2020 23:23:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=ViOwz5HrCB25jkfYhffFx9rLlViHm455phEHkJXKIN4=; b=iD9AF3OBaONzspPWNY0pchF6vRr8q4e5zkrRpUrCRKJYzZQeZ1zPkiWUxoLV6P/htr jrQ9hnwBPlF++tQYZ8Nqb4fh9kPS4n1Rbvyu3BJ8xqqV1y/c6fFiIDU0eAsuE+9XGG0F 6i3wSH77/ToEurtsNarwBP20GttXP+SmbEMIM6uJzZhDApTiG+RvXQXCOJoLTeXy2ctl ejSd5FrMVT+rGI6np574ejgW6AN7FSvUUmOtqLOB0MAZG02SgOlDMgHOn/YxsWyauN9K ZNzKHF5+59S5QbLVCBU6pYPi+5UXiBAOk+A4ejS9T9FGEPYOY/qdJLNmSqr6PMPxOY2G yySw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=ViOwz5HrCB25jkfYhffFx9rLlViHm455phEHkJXKIN4=; b=CuhySjMjqyibCR9k9qNiRKdPmI5/gDUEMxbm3wS3SyCyW0iDFz5UTXFTbHE5eskATA gXkJpHQ/3xlERY1P5B2vPOOdkPlSUOifkmn3dcAM4RMkWkne0IRsFJPipcHnxMyK2BAs Wku9bvsA6yjmq739Tt5hW10QzreX3rSe+3FdmVZBqndDe6s1jWNz3AjDa3WNCBInKEhN /GcFAeDqDdFQtDFKrwdJwPowlR4FPmo9gpwwoLR5toMrTagDI8ITD/cGyXlmk60gqfC/ ZQ7WTBXvXlmL3fFe7l25v1FrXTwF3rABUI+mrCO2ai/dGTttsXlTE0XQw3HcZ/Q1LOBP orGQ== X-Gm-Message-State: AGi0PuYr489Raoj2wr9kQlByRlNkHAwnkrTcwkB2t1w6K4dDQ7OgJEHy t0hxLnmjzXHvgU02mGxmAk4g9CBI X-Google-Smtp-Source: APiQypLVdC9f0/O/QQJ4vta+2X9cuV9mYMffkIeANatQWpdlP5SeJ6CvlEcEX/fbQVqplVpOpyqfJQ== X-Received: by 2002:a62:2b05:: with SMTP id r5mr7418621pfr.120.1587709382043; Thu, 23 Apr 2020 23:23:02 -0700 (PDT) Received: from localhost.localdomain ([103.7.29.6]) by smtp.googlemail.com with ESMTPSA id l30sm3920674pgu.29.2020.04.23.23.22.59 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 23 Apr 2020 23:23:01 -0700 (PDT) From: Wanpeng Li X-Google-Original-From: Wanpeng Li To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: Paolo Bonzini , Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , Haiwei Li Subject: [PATCH v3 2/5] KVM: X86: Introduce need_cancel_enter_guest helper Date: Fri, 24 Apr 2020 14:22:41 +0800 Message-Id: <1587709364-19090-3-git-send-email-wanpengli@tencent.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1587709364-19090-1-git-send-email-wanpengli@tencent.com> References: <1587709364-19090-1-git-send-email-wanpengli@tencent.com> Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Wanpeng Li Introduce need_cancel_enter_guest() helper, we need to check some conditions before doing CONT_RUN, in addition, it can also catch the case vmexit occurred while another event was being delivered to guest software since vmx_complete_interrupts() adds the request bit. Tested-by: Haiwei Li Cc: Haiwei Li Signed-off-by: Wanpeng Li --- arch/x86/kvm/vmx/vmx.c | 12 +++++++----- arch/x86/kvm/x86.c | 10 ++++++++-- arch/x86/kvm/x86.h | 1 + 3 files changed, 16 insertions(+), 7 deletions(-) diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index f1f6638..5c21027 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -6577,7 +6577,7 @@ bool __vmx_vcpu_run(struct vcpu_vmx *vmx, unsigned long *regs, bool launched); static enum exit_fastpath_completion vmx_vcpu_run(struct kvm_vcpu *vcpu) { - enum exit_fastpath_completion exit_fastpath; + enum exit_fastpath_completion exit_fastpath = EXIT_FASTPATH_NONE; struct vcpu_vmx *vmx = to_vmx(vcpu); unsigned long cr3, cr4; @@ -6754,10 +6754,12 @@ static enum exit_fastpath_completion vmx_vcpu_run(struct kvm_vcpu *vcpu) vmx_recover_nmi_blocking(vmx); vmx_complete_interrupts(vmx); - exit_fastpath = vmx_exit_handlers_fastpath(vcpu); - /* static call is better with retpolines */ - if (exit_fastpath == EXIT_FASTPATH_CONT_RUN) - goto cont_run; + if (!kvm_need_cancel_enter_guest(vcpu)) { + exit_fastpath = vmx_exit_handlers_fastpath(vcpu); + /* static call is better with retpolines */ + if (exit_fastpath == EXIT_FASTPATH_CONT_RUN) + goto cont_run; + } return exit_fastpath; } diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 59958ce..4561104 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -1581,6 +1581,13 @@ int kvm_emulate_wrmsr(struct kvm_vcpu *vcpu) } EXPORT_SYMBOL_GPL(kvm_emulate_wrmsr); +bool kvm_need_cancel_enter_guest(struct kvm_vcpu *vcpu) +{ + return (vcpu->mode == EXITING_GUEST_MODE || kvm_request_pending(vcpu) + || need_resched() || signal_pending(current)); +} +EXPORT_SYMBOL_GPL(kvm_need_cancel_enter_guest); + /* * The fast path for frequent and performance sensitive wrmsr emulation, * i.e. the sending of IPI, sending IPI early in the VM-Exit flow reduces @@ -8373,8 +8380,7 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu) if (kvm_lapic_enabled(vcpu) && vcpu->arch.apicv_active) kvm_x86_ops.sync_pir_to_irr(vcpu); - if (vcpu->mode == EXITING_GUEST_MODE || kvm_request_pending(vcpu) - || need_resched() || signal_pending(current)) { + if (kvm_need_cancel_enter_guest(vcpu)) { vcpu->mode = OUTSIDE_GUEST_MODE; smp_wmb(); local_irq_enable(); diff --git a/arch/x86/kvm/x86.h b/arch/x86/kvm/x86.h index 7b5ed8e..1906e7e 100644 --- a/arch/x86/kvm/x86.h +++ b/arch/x86/kvm/x86.h @@ -364,5 +364,6 @@ static inline bool kvm_dr7_valid(u64 data) void kvm_load_guest_xsave_state(struct kvm_vcpu *vcpu); void kvm_load_host_xsave_state(struct kvm_vcpu *vcpu); u64 kvm_spec_ctrl_valid_bits(struct kvm_vcpu *vcpu); +bool kvm_need_cancel_enter_guest(struct kvm_vcpu *vcpu); #endif From patchwork Fri Apr 24 06:22:42 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wanpeng Li X-Patchwork-Id: 11507175 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 61CAF92A for ; Fri, 24 Apr 2020 06:23:11 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 46F7D20700 for ; Fri, 24 Apr 2020 06:23:11 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="mSBtBjdu" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726671AbgDXGXG (ORCPT ); Fri, 24 Apr 2020 02:23:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51290 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1726665AbgDXGXG (ORCPT ); Fri, 24 Apr 2020 02:23:06 -0400 Received: from mail-pf1-x441.google.com (mail-pf1-x441.google.com [IPv6:2607:f8b0:4864:20::441]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EAFAFC09B047; Thu, 23 Apr 2020 23:23:05 -0700 (PDT) Received: by mail-pf1-x441.google.com with SMTP id 18so3104325pfv.8; Thu, 23 Apr 2020 23:23:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=zsPf//Pz+Ikh6+DsYPyUBP5glvQEMcSomDR2Wz5ze9I=; b=mSBtBjduNX75Q8OPv0HvHykSL62EEwkpC7HS/ITEACsut+AYwVLLBPMn0lKKb6YjVk b0w4uYU1Mi3+aX76+Yldo98XKTAYA/8//z/hk53fOkn6I1fYaLEK3/FKsG25ZzPAwQjB o1aqF3s56c+2xx7kpSXXtpHbJ7bF7uZ39ff7ABCleS+Gycqz5D1INBJxz+yUwRHva7Dg PkN2XR2r8a2SLfAQVIfKdE+6VKwYo9lnGoSwSA19O8nbT6s2TIXM6z1tXnUg/5ERfsX1 Yt973ARfSZDV89nTe8wPdROMoBBTVTIB039HvOmYHpOOpOTkhmC/dflxB6fMeMByKgWY Q84w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=zsPf//Pz+Ikh6+DsYPyUBP5glvQEMcSomDR2Wz5ze9I=; b=bvgNIdGE0Whfgto5Uh9paCCO6/udGxHjvsCp/HOG46LffyVy7Yq3etSaUeGu1pl3kw lgP/DYB7qWDVzwOuqd2rk/a7LGz9cHoA/QNslZZYt+sbjK3Ep8PwM3aFM9qEerqG9m13 58sJyI4TjRD4ZRGWOthWWHNodrP+/HP76g9wtOopAbiPWyguM8wbtnSbu20SchkGEPYc OH/b2jLHXgRS7tTc3jrfJD+a/4jCcZrW9sKvSTSJaoKhLI2pIU+YFJHWbmwJTBp0uBpx fAVMaG2EUxTMbzs/Tfl/6e9LbLsFyicrT+fcN/HBFbMG7EuTm0jdxlZcOUmBmLAp7h9k g7sQ== X-Gm-Message-State: AGi0PuZru2ASqSw8Ze+BrlSnuCbFsyRpy9nY0EBZ1YeEDGE/JTfFgUYg MOpmVvmynoM0Ejc2aLz51UigKYY4 X-Google-Smtp-Source: APiQypJ1ixZkeAqhQDfMzttF2Fw3E7k+vFcjhPVtqlT3SLiCtjcXny5qyc20qxRsSEd27zZLt9KPlQ== X-Received: by 2002:a62:34c1:: with SMTP id b184mr7844399pfa.73.1587709385259; Thu, 23 Apr 2020 23:23:05 -0700 (PDT) Received: from localhost.localdomain ([103.7.29.6]) by smtp.googlemail.com with ESMTPSA id l30sm3920674pgu.29.2020.04.23.23.23.02 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 23 Apr 2020 23:23:04 -0700 (PDT) From: Wanpeng Li X-Google-Original-From: Wanpeng Li To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: Paolo Bonzini , Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , Haiwei Li Subject: [PATCH v3 3/5] KVM: VMX: Optimize posted-interrupt delivery for timer fastpath Date: Fri, 24 Apr 2020 14:22:42 +0800 Message-Id: <1587709364-19090-4-git-send-email-wanpengli@tencent.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1587709364-19090-1-git-send-email-wanpengli@tencent.com> References: <1587709364-19090-1-git-send-email-wanpengli@tencent.com> Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Wanpeng Li Optimizing posted-interrupt delivery especially for the timer fastpath scenario, I observe kvm_x86_ops.deliver_posted_interrupt() has more latency then vmx_sync_pir_to_irr() in the case of timer fastpath scenario, since it needs to wait vmentry, after that it can handle external interrupt, ack the notification vector, read posted-interrupt descriptor etc, it is slower than evaluate and delivery during vmentry immediately approach. Let's skip sending interrupt to notify target pCPU and replace by vmx_sync_pir_to_irr() before each cont_run. Tested-by: Haiwei Li Cc: Haiwei Li Signed-off-by: Wanpeng Li --- arch/x86/kvm/vmx/vmx.c | 9 ++++++--- virt/kvm/kvm_main.c | 1 + 2 files changed, 7 insertions(+), 3 deletions(-) diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 5c21027..d21b66b 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -3909,8 +3909,9 @@ static int vmx_deliver_posted_interrupt(struct kvm_vcpu *vcpu, int vector) if (pi_test_and_set_on(&vmx->pi_desc)) return 0; - if (!kvm_vcpu_trigger_posted_interrupt(vcpu, false)) - kvm_vcpu_kick(vcpu); + if (vcpu != kvm_get_running_vcpu() && + !kvm_vcpu_trigger_posted_interrupt(vcpu, false)) + kvm_vcpu_kick(vcpu); return 0; } @@ -6757,8 +6758,10 @@ static enum exit_fastpath_completion vmx_vcpu_run(struct kvm_vcpu *vcpu) if (!kvm_need_cancel_enter_guest(vcpu)) { exit_fastpath = vmx_exit_handlers_fastpath(vcpu); /* static call is better with retpolines */ - if (exit_fastpath == EXIT_FASTPATH_CONT_RUN) + if (exit_fastpath == EXIT_FASTPATH_CONT_RUN) { + vmx_sync_pir_to_irr(vcpu); goto cont_run; + } } return exit_fastpath; diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index e7436d0..6a289d1 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -4633,6 +4633,7 @@ struct kvm_vcpu *kvm_get_running_vcpu(void) return vcpu; } +EXPORT_SYMBOL_GPL(kvm_get_running_vcpu); /** * kvm_get_running_vcpus - get the per-CPU array of currently running vcpus. From patchwork Fri Apr 24 06:22:43 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wanpeng Li X-Patchwork-Id: 11507179 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 02FB915AB for ; Fri, 24 Apr 2020 06:23:21 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D78C720704 for ; Fri, 24 Apr 2020 06:23:20 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="bVYSpE7r" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726690AbgDXGXL (ORCPT ); Fri, 24 Apr 2020 02:23:11 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51296 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1726051AbgDXGXK (ORCPT ); Fri, 24 Apr 2020 02:23:10 -0400 Received: from mail-pg1-x543.google.com (mail-pg1-x543.google.com [IPv6:2607:f8b0:4864:20::543]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 04F6DC09B045; Thu, 23 Apr 2020 23:23:09 -0700 (PDT) Received: by mail-pg1-x543.google.com with SMTP id t11so4175297pgg.2; Thu, 23 Apr 2020 23:23:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=GikLm+2sMpfTcFbUsqFiKD0m1jTGsA7KiCbME+gQlgI=; b=bVYSpE7r+8u+N+GKZ0wHjRobNnUkmeTTruOsNhrJXRKU/ce1uIhUBU8yEfdQqtUIn7 HkKKkfNe31pYIlafQhks4VjVlRYPECjddshsa+yPInWS+o0w++evATVMY3x/3mryNo4p 7P7IvOTDrvF0QrwcFcR2kIv3iSXe26mrKmFGK2xIuqRvhHc8ZO860RWuifRMrsMu9X+K Y0xfyWwVCEkVi+jdr4YCmRCKdeFQG6uY7wsHS7LXMTrmDnuqAq+zu5arBZjHT3VRS2sJ /KqRseLflH0B5v88S2VKhZFoManN3Pl1QXov3KqPBJ5Zovi/FtFh4TS+oTJI7Igrb2eT HolQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=GikLm+2sMpfTcFbUsqFiKD0m1jTGsA7KiCbME+gQlgI=; b=g/4aAqNIh7NSd11klBDwtRcv0Fqb6WpoLcutyCbdwuZZGF+e2lDlFuO37NFymdqpag R1TyufVmP1hDWT6M+4YTjL1vHmTflH52Kbi47NfsQA66O7mvDFNX5RMiphOKh+E9jfOs XYGkhB/TY+r6nfpdFyqLeChU67GqtZwoFy8YG9HS3ApTDeKlVZO1ev4TW2sghlIogzuH UyhQ9XhaL1Zl/xDe5RY1YDeZ4G1wt8kw8yGLu0vDOgXeZFY1rXSeplhTSYUl8ExqjhBz ljuG7XrShk1nXwD9H4Lwwdrdnf0+U/X9Fw67QJYSvKVLSeFx5+pbJzxASQUhkM0D9drz I5/w== X-Gm-Message-State: AGi0PuYlDVKOna7J1oWm12VfEedK8D/158R89/PcES0cMy+XZpBSxTaK Fs2gdBZUeMhsVxDwI1np8r0SVFs3 X-Google-Smtp-Source: APiQypK/jTCPL327MGXDIrfzXSb3ChDp/4QmJsxtoOJ8rF8Zoo8fgmYSzurNJAVFmSxWlJwuuYlS7A== X-Received: by 2002:a63:5413:: with SMTP id i19mr7985408pgb.213.1587709388225; Thu, 23 Apr 2020 23:23:08 -0700 (PDT) Received: from localhost.localdomain ([103.7.29.6]) by smtp.googlemail.com with ESMTPSA id l30sm3920674pgu.29.2020.04.23.23.23.05 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 23 Apr 2020 23:23:07 -0700 (PDT) From: Wanpeng Li X-Google-Original-From: Wanpeng Li To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: Paolo Bonzini , Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , Haiwei Li Subject: [PATCH v3 4/5] KVM: X86: TSCDEADLINE MSR emulation fastpath Date: Fri, 24 Apr 2020 14:22:43 +0800 Message-Id: <1587709364-19090-5-git-send-email-wanpengli@tencent.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1587709364-19090-1-git-send-email-wanpengli@tencent.com> References: <1587709364-19090-1-git-send-email-wanpengli@tencent.com> Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Wanpeng Li This patch implements tscdealine msr emulation fastpath, after wrmsr tscdeadline vmexit, handle it as soon as possible and vmentry immediately without checking various kvm stuff when possible. Tested-by: Haiwei Li Cc: Haiwei Li Signed-off-by: Wanpeng Li --- arch/x86/kvm/lapic.c | 18 ++++++++++++------ arch/x86/kvm/x86.c | 30 ++++++++++++++++++++++++------ 2 files changed, 36 insertions(+), 12 deletions(-) diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c index 38f7dc9..3589237 100644 --- a/arch/x86/kvm/lapic.c +++ b/arch/x86/kvm/lapic.c @@ -1593,7 +1593,7 @@ static void kvm_apic_inject_pending_timer_irqs(struct kvm_lapic *apic) } } -static void apic_timer_expired(struct kvm_lapic *apic) +static void apic_timer_expired(struct kvm_lapic *apic, bool from_timer_fn) { struct kvm_vcpu *vcpu = apic->vcpu; struct kvm_timer *ktimer = &apic->lapic_timer; @@ -1604,6 +1604,12 @@ static void apic_timer_expired(struct kvm_lapic *apic) if (apic_lvtt_tscdeadline(apic) || ktimer->hv_timer_in_use) ktimer->expired_tscdeadline = ktimer->tscdeadline; + if (!from_timer_fn && vcpu->arch.apicv_active) { + WARN_ON(kvm_get_running_vcpu() != vcpu); + kvm_apic_inject_pending_timer_irqs(apic); + return; + } + if (kvm_use_posted_timer_interrupt(apic->vcpu)) { if (apic->lapic_timer.timer_advance_ns) __kvm_wait_lapic_expire(vcpu); @@ -1643,7 +1649,7 @@ static void start_sw_tscdeadline(struct kvm_lapic *apic) expire = ktime_sub_ns(expire, ktimer->timer_advance_ns); hrtimer_start(&ktimer->timer, expire, HRTIMER_MODE_ABS_HARD); } else - apic_timer_expired(apic); + apic_timer_expired(apic, false); local_irq_restore(flags); } @@ -1751,7 +1757,7 @@ static void start_sw_period(struct kvm_lapic *apic) if (ktime_after(ktime_get(), apic->lapic_timer.target_expiration)) { - apic_timer_expired(apic); + apic_timer_expired(apic, false); if (apic_lvtt_oneshot(apic)) return; @@ -1813,7 +1819,7 @@ static bool start_hv_timer(struct kvm_lapic *apic) if (atomic_read(&ktimer->pending)) { cancel_hv_timer(apic); } else if (expired) { - apic_timer_expired(apic); + apic_timer_expired(apic, false); cancel_hv_timer(apic); } } @@ -1863,7 +1869,7 @@ void kvm_lapic_expired_hv_timer(struct kvm_vcpu *vcpu) goto out; WARN_ON(swait_active(&vcpu->wq)); cancel_hv_timer(apic); - apic_timer_expired(apic); + apic_timer_expired(apic, false); if (apic_lvtt_period(apic) && apic->lapic_timer.period) { advance_periodic_target_expiration(apic); @@ -2369,7 +2375,7 @@ static enum hrtimer_restart apic_timer_fn(struct hrtimer *data) struct kvm_timer *ktimer = container_of(data, struct kvm_timer, timer); struct kvm_lapic *apic = container_of(ktimer, struct kvm_lapic, lapic_timer); - apic_timer_expired(apic); + apic_timer_expired(apic, true); if (lapic_is_periodic(apic)) { advance_periodic_target_expiration(apic); diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 4561104..99061ba 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -1616,27 +1616,45 @@ static int handle_fastpath_set_x2apic_icr_irqoff(struct kvm_vcpu *vcpu, u64 data return 1; } +static int handle_fastpath_set_tscdeadline(struct kvm_vcpu *vcpu, u64 data) +{ + if (!kvm_x86_ops.set_hv_timer || + kvm_mwait_in_guest(vcpu->kvm) || + kvm_can_post_timer_interrupt(vcpu)) + return 1; + + kvm_set_lapic_tscdeadline_msr(vcpu, data); + return 0; +} + enum exit_fastpath_completion handle_fastpath_set_msr_irqoff(struct kvm_vcpu *vcpu) { u32 msr = kvm_rcx_read(vcpu); u64 data; - int ret = 0; + int ret = EXIT_FASTPATH_NONE; switch (msr) { case APIC_BASE_MSR + (APIC_ICR >> 4): data = kvm_read_edx_eax(vcpu); - ret = handle_fastpath_set_x2apic_icr_irqoff(vcpu, data); + if (!handle_fastpath_set_x2apic_icr_irqoff(vcpu, data)) + ret = EXIT_FASTPATH_SKIP_EMUL_INS; + break; + case MSR_IA32_TSCDEADLINE: + data = kvm_read_edx_eax(vcpu); + if (!handle_fastpath_set_tscdeadline(vcpu, data)) + ret = EXIT_FASTPATH_CONT_RUN; break; default: - return EXIT_FASTPATH_NONE; + ret = EXIT_FASTPATH_NONE; } - if (!ret) { + if (ret != EXIT_FASTPATH_NONE) { trace_kvm_msr_write(msr, data); - return EXIT_FASTPATH_SKIP_EMUL_INS; + if (ret == EXIT_FASTPATH_CONT_RUN) + kvm_skip_emulated_instruction(vcpu); } - return EXIT_FASTPATH_NONE; + return ret; } EXPORT_SYMBOL_GPL(handle_fastpath_set_msr_irqoff); From patchwork Fri Apr 24 06:22:44 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wanpeng Li X-Patchwork-Id: 11507177 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B7CB492A for ; Fri, 24 Apr 2020 06:23:15 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9FB6520704 for ; Fri, 24 Apr 2020 06:23:15 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="ALTyvKyQ" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726707AbgDXGXP (ORCPT ); Fri, 24 Apr 2020 02:23:15 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51308 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1726699AbgDXGXN (ORCPT ); Fri, 24 Apr 2020 02:23:13 -0400 Received: from mail-pj1-x1041.google.com (mail-pj1-x1041.google.com [IPv6:2607:f8b0:4864:20::1041]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1A078C09B045; Thu, 23 Apr 2020 23:23:12 -0700 (PDT) Received: by mail-pj1-x1041.google.com with SMTP id fu13so2966738pjb.5; Thu, 23 Apr 2020 23:23:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=MxB/5+Kd2C0pw05dDYhqC6sr311SJsuN/Ux6MEaNAeM=; b=ALTyvKyQUjtlg1598kuRSm4WumDxkjhA8gT+IOrw0J1iqMj9KYEY90As0WJHEcHVWz SamkGK23L+Z9bffWYiSZeLeAxbnhyNhJR+dLizlJvZK68g3vmHopsd4o+qQQXcHlTSdh YsrptaoTIB1yjQd/1JFsnNOWXYiyu/FXkikO7RG0v6ecwBhF7dzayOUPzRT03K5cYgxr zIoFFlYjjq+GnIZ1PUWjzQBOjTcTDVniEbb9vEMqgWyvEYc4lx25ZjNpzthl3tIHQ3K6 uxXvG0nn+9CxfZnNS8HNZSVipbvLF4gi/HiYMMQNyNpDQ4bVFC9/bZOauoHktiqZD7jB lfMQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=MxB/5+Kd2C0pw05dDYhqC6sr311SJsuN/Ux6MEaNAeM=; b=KyEYEkCur6d+JCH/th/ozfnkwCfWLhmQx42wsAyF2cUTI1tnqZj50gxzAzGFqaCl0d zO6kg7F2wiDU+evjjswTXATb6dsVanzsaE/5Z9S2juwrXFra9rC5KwQsDm03pP9beF1i sw8+qN9nvcRsrobMRJQ+v7pC5lq68O8UB7S6AngfvUQ5bbWnZTmWfWCOjxEXaMMRKhYl 5TbkUcFpZ0qbekOXwNvMbClqsTx83ANgeOVNeTkep2O68kZk8njroBwPpP9xIaE2vPkG dnI5O0Ub9trtIhynUKiqTMc708p6hu/qk8q5K8YvuKxDdfGavQp0un4iGEnd0L6DN1Bb KPmg== X-Gm-Message-State: AGi0PuZvh8N8SjnmAYP10PZbuTLOf8MhBxzsBFfBzOMMmpOyLX4eE9A4 7rLfebTZwERVTCZdCXjw2Gg0Kqab X-Google-Smtp-Source: APiQypK/mIzK+C1SxJQBVgxJKD0z5j932b08/NaZVr/QTI123ZiXW+iATgxaMwlHykGir3aFTd+rPA== X-Received: by 2002:a17:90a:db53:: with SMTP id u19mr4771195pjx.41.1587709391365; Thu, 23 Apr 2020 23:23:11 -0700 (PDT) Received: from localhost.localdomain ([103.7.29.6]) by smtp.googlemail.com with ESMTPSA id l30sm3920674pgu.29.2020.04.23.23.23.08 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 23 Apr 2020 23:23:10 -0700 (PDT) From: Wanpeng Li X-Google-Original-From: Wanpeng Li To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: Paolo Bonzini , Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , Haiwei Li Subject: [PATCH v3 5/5] KVM: VMX: Handle preemption timer fastpath Date: Fri, 24 Apr 2020 14:22:44 +0800 Message-Id: <1587709364-19090-6-git-send-email-wanpengli@tencent.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1587709364-19090-1-git-send-email-wanpengli@tencent.com> References: <1587709364-19090-1-git-send-email-wanpengli@tencent.com> Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Wanpeng Li This patch implements handle preemption timer fastpath, after timer fire due to VMX-preemption timer counts down to zero, handle it as soon as possible and vmentry immediately without checking various kvm stuff when possible. Testing on SKX Server. cyclictest in guest(w/o mwait exposed, adaptive advance lapic timer is default -1): 5540.5ns -> 4602ns 17% kvm-unit-test/vmexit.flat: w/o avanced timer: tscdeadline_immed: 2885 -> 2431.25 15.7% tscdeadline: 5668.75 -> 5188.5 8.4% w/ adaptive advance timer default -1: tscdeadline_immed: 2965.25 -> 2520 15.0% tscdeadline: 4663.75 -> 4537 2.7% Tested-by: Haiwei Li Cc: Haiwei Li Signed-off-by: Wanpeng Li --- arch/x86/kvm/vmx/vmx.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index d21b66b..028967a 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -6560,12 +6560,28 @@ void vmx_update_host_rsp(struct vcpu_vmx *vmx, unsigned long host_rsp) } } +static enum exit_fastpath_completion handle_fastpath_preemption_timer(struct kvm_vcpu *vcpu) +{ + struct vcpu_vmx *vmx = to_vmx(vcpu); + + if (!vmx->req_immediate_exit && + !unlikely(vmx->loaded_vmcs->hv_timer_soft_disabled)) { + kvm_lapic_expired_hv_timer(vcpu); + trace_kvm_exit(EXIT_REASON_PREEMPTION_TIMER, vcpu, KVM_ISA_VMX); + return EXIT_FASTPATH_CONT_RUN; + } + + return EXIT_FASTPATH_NONE; +} + static enum exit_fastpath_completion vmx_exit_handlers_fastpath(struct kvm_vcpu *vcpu) { if (!is_guest_mode(vcpu) && vcpu->arch.apicv_active) { switch (to_vmx(vcpu)->exit_reason) { case EXIT_REASON_MSR_WRITE: return handle_fastpath_set_msr_irqoff(vcpu); + case EXIT_REASON_PREEMPTION_TIMER: + return handle_fastpath_preemption_timer(vcpu); default: return EXIT_FASTPATH_NONE; }