From patchwork Thu Mar 26 02:20:00 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wanpeng Li X-Patchwork-Id: 11458991 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3A74781 for ; Thu, 26 Mar 2020 02:20:21 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 1B4212073E for ; Thu, 26 Mar 2020 02:20:20 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="F1LxSMfi" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727733AbgCZCUR (ORCPT ); Wed, 25 Mar 2020 22:20:17 -0400 Received: from mail-pj1-f67.google.com ([209.85.216.67]:34980 "EHLO mail-pj1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727590AbgCZCUR (ORCPT ); Wed, 25 Mar 2020 22:20:17 -0400 Received: by mail-pj1-f67.google.com with SMTP id g9so1805504pjp.0; Wed, 25 Mar 2020 19:20:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=qDBMnluBEIKHHnYAEDKv+uJNUJGj8Te28Rln44SHO8E=; b=F1LxSMfimwN3imJWkTMpv1Ri55oBYP5f5KoUR6cLAlBzGPJ7PeGsjDP/ps2VHuexyk N1hgmnpHVwXNPv7gU5xMpmJ8N0APDSLOc12OUdtaOpWxNEMKrLpNA2fJ+oF9aF8oqL1Z DC8r7yDGm6mXPIOporC8Bb7e+kKM4Job5dMn+9pYU3G/gjeRFiCcvxsUbpw9nx+Oa+U4 bc9lQSgKmMew7NSY0rd6PxTJ0BWkipCoJA3mHdQ2TBdeS3ndNIj/644nizmPJxZLWdPN nyYpp2YTC9vJ4/aIiRRaMkXthP7fIy06FBukde5WhwA5hBIUvkInfiqIZdFy5G5s83Fu m1UQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=qDBMnluBEIKHHnYAEDKv+uJNUJGj8Te28Rln44SHO8E=; b=gaPSxDUjgfZ54TtZxLjrPungrxBAFHVUodN+Ly3np9myaJXm9XsNMJVtEbid+luVxY Js2do5Ampc6XMR/oftQpu8x/ydlIjOt7fyMErsRoTFwbHNb9wm0w1S4GCzwne/lwsB+o mr7DSQi0lfo7+p8XIi3d1l8LnckLAwRrZG1rNJEs4AEz+MniPdN9BXRgClZm8XbZoDhv UtjxRqR0Yor+GGYvxnFLA2sISUKOPsQzl0Fd9IemRto1kTHjZ4F0r68BE0c+olJfvJ1e eCzw1RIuKF/8qRIJVmsQkzyMkGxVgllTKTl7Ocr4wv6wUWP7qM1VthQGaoUXID84zuQM o0fg== X-Gm-Message-State: ANhLgQ0MwtYUKgQFrhp+c8vpXctQefao7QY8K6qTWLgWWnKYbTfgNOqZ SWOEQ/pq+USUidRFrWQM334T34c9 X-Google-Smtp-Source: ADFU+vsDOt+koJGV7OQZXsBtSoSJra3SJ5LO8pyMovhgNC5OYrs0HUtc61ROfkcpYLJHgnUt49SfZQ== X-Received: by 2002:a17:902:70cb:: with SMTP id l11mr6118046plt.127.1585189216114; Wed, 25 Mar 2020 19:20:16 -0700 (PDT) Received: from localhost.localdomain ([103.7.29.6]) by smtp.googlemail.com with ESMTPSA id mq18sm452975pjb.6.2020.03.25.19.20.13 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 25 Mar 2020 19:20:15 -0700 (PDT) From: Wanpeng Li X-Google-Original-From: Wanpeng Li To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: Paolo Bonzini , Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel Subject: [PATCH 1/3] KVM: X86: Delay read msr data iff writes ICR MSR Date: Thu, 26 Mar 2020 10:20:00 +0800 Message-Id: <1585189202-1708-2-git-send-email-wanpengli@tencent.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1585189202-1708-1-git-send-email-wanpengli@tencent.com> References: <1585189202-1708-1-git-send-email-wanpengli@tencent.com> Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Wanpeng Li Delay read msr data until we identify guest accesses ICR MSR to avoid to penalize all other MSR writes. Signed-off-by: Wanpeng Li --- arch/x86/kvm/x86.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 3156e25..9232b15 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -1568,11 +1568,12 @@ static int handle_fastpath_set_x2apic_icr_irqoff(struct kvm_vcpu *vcpu, u64 data enum exit_fastpath_completion handle_fastpath_set_msr_irqoff(struct kvm_vcpu *vcpu) { u32 msr = kvm_rcx_read(vcpu); - u64 data = kvm_read_edx_eax(vcpu); + u64 data; int ret = 0; switch (msr) { case APIC_BASE_MSR + (APIC_ICR >> 4): + data = kvm_read_edx_eax(vcpu); ret = handle_fastpath_set_x2apic_icr_irqoff(vcpu, data); break; default: From patchwork Thu Mar 26 02:20:01 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wanpeng Li X-Patchwork-Id: 11458993 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8ACBD6CA for ; Thu, 26 Mar 2020 02:20:21 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 6A1502073E for ; Thu, 26 Mar 2020 02:20:21 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="m5twDoTV" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727754AbgCZCUU (ORCPT ); Wed, 25 Mar 2020 22:20:20 -0400 Received: from mail-pj1-f68.google.com ([209.85.216.68]:39883 "EHLO mail-pj1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727736AbgCZCUU (ORCPT ); Wed, 25 Mar 2020 22:20:20 -0400 Received: by mail-pj1-f68.google.com with SMTP id z3so1288198pjr.4; Wed, 25 Mar 2020 19:20:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=chlzOeSTKjpX1u/O2+BRSZO0t8ii9bW0UyyBajq5hnM=; b=m5twDoTV9KA/7pPKfQZO9JyQ1ZRYeD+yBTFfg5G2shlnHvj9NoFgm/QP+Bdq1gC59b ypj9t1t6FOTMhptJj+kju/leMjjOnNR3hvfKJMKXYRXJ7VsrVTFDWxkt+SQh/2+4gSOY MD6ZwBO9T3CJidsFAYLSnE9aa9rvG1vVrAhkHYkmLYHQGPbF/5G+eobWXAHfM9olMAXN iUqp7FiAYL5Rjn3Hlx97mT53n73zy8k7ut8hyq8l7E5KjOU6NwOwICP6MR/Bwx4CoDC2 Ko0Pc75nknD0eV8Haz20LWhLVSQpf8jXjC+SpnumFXXOUAB3M2SdQ3qLgxf7OyrNcEQ4 SM3A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=chlzOeSTKjpX1u/O2+BRSZO0t8ii9bW0UyyBajq5hnM=; b=I2Pf2BjwBB03rSQmbVGgC0Pu+q+6CmooTsf5v9D39Sq8x2ktEgG2bskU9MBgsqtdMI TR/vGmbnoaUbKt3sFknloYpfV9O8N8rSRinAku0QNNuu9T0t5TTMDdoAB6g6EYHqB8Kd vu3hfFW3WvaipRCpJBNZZXhIBMXHmHD2kBfKdBbEBTGWVrwqkl5FJQN/iWIMSUuajg5M J4flntxh8MPZaM1ffQ5yAqsn5yNkkv+MD7tnNUB15C/8zinUaOY4kB+GapVLH5Il1D6S ffsFSVLuiKrePUhpKbOdHKCbobKZEi3lfkOJTB4XvPovNuszKUG9PsGNCP1SmKpf21yp cC2w== X-Gm-Message-State: ANhLgQ0aqWIkL9lOnbvUQonTfkkRqk0A41alvsjgydHSgtccz+LFGHyf 5CaZVBWMH/OZs7UDO4kZjNV7wEQY X-Google-Smtp-Source: ADFU+vvdhEnxm4pIe7I3EZfIRn2Gdn0pM0UX4PpkDM/D6S614eJNvjxPhju/qErO5pwntTiWgUvyJQ== X-Received: by 2002:a17:902:d203:: with SMTP id t3mr5912082ply.291.1585189218704; Wed, 25 Mar 2020 19:20:18 -0700 (PDT) Received: from localhost.localdomain ([103.7.29.6]) by smtp.googlemail.com with ESMTPSA id mq18sm452975pjb.6.2020.03.25.19.20.16 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 25 Mar 2020 19:20:18 -0700 (PDT) From: Wanpeng Li X-Google-Original-From: Wanpeng Li To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: Paolo Bonzini , Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel Subject: [PATCH 2/3] KVM: X86: Narrow down the IPI fastpath to single target IPI Date: Thu, 26 Mar 2020 10:20:01 +0800 Message-Id: <1585189202-1708-3-git-send-email-wanpengli@tencent.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1585189202-1708-1-git-send-email-wanpengli@tencent.com> References: <1585189202-1708-1-git-send-email-wanpengli@tencent.com> Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Wanpeng Li The original single target IPI fastpath patch forgot to filter the ICR destination shorthand field. Multicast IPI is not suitable for this feature since wakeup the multiple sleeping vCPUs will extend the interrupt disabled time, it especially worse in the over-subscribe and VM has a little bit more vCPUs scenario. Let's narrow it down to single target IPI. Two VMs, each is 76 vCPUs, one running 'ebizzy -M', the other running cyclictest on all vCPUs, w/ this patch, the avg score of cyclictest can improve more than 5%. (pv tlb, pv ipi, pv sched yield are disabled during testing to avoid the disturb). Signed-off-by: Wanpeng Li --- arch/x86/kvm/x86.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 9232b15..50ef1c5 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -1554,7 +1554,10 @@ EXPORT_SYMBOL_GPL(kvm_emulate_wrmsr); */ static int handle_fastpath_set_x2apic_icr_irqoff(struct kvm_vcpu *vcpu, u64 data) { - if (lapic_in_kernel(vcpu) && apic_x2apic_mode(vcpu->arch.apic) && + if (!lapic_in_kernel(vcpu) || !apic_x2apic_mode(vcpu->arch.apic)) + return 1; + + if (((data & APIC_SHORT_MASK) == APIC_DEST_NOSHORT) && ((data & APIC_DEST_MASK) == APIC_DEST_PHYSICAL) && ((data & APIC_MODE_MASK) == APIC_DM_FIXED)) { From patchwork Thu Mar 26 02:20:02 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wanpeng Li X-Patchwork-Id: 11458995 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1BBF381 for ; Thu, 26 Mar 2020 02:20:28 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id EF4022082F for ; Thu, 26 Mar 2020 02:20:27 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="DbRgiRuq" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727774AbgCZCUZ (ORCPT ); Wed, 25 Mar 2020 22:20:25 -0400 Received: from mail-pg1-f194.google.com ([209.85.215.194]:39337 "EHLO mail-pg1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727736AbgCZCUY (ORCPT ); Wed, 25 Mar 2020 22:20:24 -0400 Received: by mail-pg1-f194.google.com with SMTP id b22so2110596pgb.6; Wed, 25 Mar 2020 19:20:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=1a2sJiVllrGzVZRtd1vAiDbOX8dRnMrn6vYAwgDTFoY=; b=DbRgiRuqIuLcXZ8fLdfUERBz0+EHvUPuP4eUGh98zRY+s74lzBxZl+wjTDeR6/qvi9 cJxZlW+ohAFZAtucAlEAZO/ih6g1/g562R8SwjbeOqI3y/eTODABR3II1Zrgq2awWUlR 7k4TZ1pI9I2vAAvJ6gwsgMtCc2BF95I9oXpyikpwCHlLc/sCNkAb+/BaCw8jcbV4HUuS WfYmrFSpbhJF/Z94GUETmUym/N9h7NwxMkfOjwGO5CZWQrBZUILsd1EPy+cOG66pzGqW R0puReKv5IEi+S6zpWxIVMdVE3wV+jtWIyl8kD+E1oAoAHSXwxnItcPGjeSrOhZ1iv5Q WRzQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=1a2sJiVllrGzVZRtd1vAiDbOX8dRnMrn6vYAwgDTFoY=; b=Oo1hv6mVEQIo7ADc4VsxQXsxDahlf24YY+1l38yKNdBL8eA1PTlPYTHa8mOXqGCjRd FMsnr3/8uPfFy0wzIvDKMTOrltPLmvr9nvd95v+2vJ1GCcf8MbnOLOzX7mEF3kjdrFuq kfHJ2ZByinSzGiHYyOg5VC6WeIDBp+ILQ6NmUbrj9CSHSfszmhgKXhyWBIV8pOCNSLcN 91HbGrkoHuNK/X10oMBhrRV/KeGLoAeTkV9SWx3vHUGwhw6fV5+qCZFCj2Px1fEVpPxZ LGvWHu4WAu91INAKO4XbT0ZCCXtIZYqJrrtJZYxBwEBF4UQ50SvOUpbGLUaAU3roQibr MfXg== X-Gm-Message-State: ANhLgQ11SewBoDCOeAivucupq0+lPVDUnU03sB6TmsS2PrZoJiFWiCup lahrniFSUdeQGMRJRrboxMY/EfR5 X-Google-Smtp-Source: ADFU+vvtRvfPGB6WA9/O2Bc4p+PxKAwq0jLOJDYzp/+tZCYHNS1wG2xmFSno1MPvgsR38xBzUtdXcw== X-Received: by 2002:a63:8c56:: with SMTP id q22mr6179580pgn.154.1585189221433; Wed, 25 Mar 2020 19:20:21 -0700 (PDT) Received: from localhost.localdomain ([103.7.29.6]) by smtp.googlemail.com with ESMTPSA id mq18sm452975pjb.6.2020.03.25.19.20.19 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 25 Mar 2020 19:20:21 -0700 (PDT) From: Wanpeng Li X-Google-Original-From: Wanpeng Li To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: Paolo Bonzini , Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel Subject: [PATCH 3/3] KVM: X86: Micro-optimize IPI fastpath delay Date: Thu, 26 Mar 2020 10:20:02 +0800 Message-Id: <1585189202-1708-4-git-send-email-wanpengli@tencent.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1585189202-1708-1-git-send-email-wanpengli@tencent.com> References: <1585189202-1708-1-git-send-email-wanpengli@tencent.com> Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Wanpeng Li This patch optimizes the virtual IPI fastpath emulation sequence: write ICR2 send virtual IPI read ICR2 write ICR2 send virtual IPI ==> write ICR write ICR We can observe ~0.67% performance improvement for IPI microbenchmark (https://lore.kernel.org/kvm/20171219085010.4081-1-ynorov@caviumnetworks.com/) on Skylake server. Signed-off-by: Wanpeng Li --- arch/x86/kvm/lapic.c | 4 ++-- arch/x86/kvm/lapic.h | 1 + arch/x86/kvm/x86.c | 6 +++++- 3 files changed, 8 insertions(+), 3 deletions(-) diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c index e3099c6..338de38 100644 --- a/arch/x86/kvm/lapic.c +++ b/arch/x86/kvm/lapic.c @@ -1226,7 +1226,7 @@ void kvm_apic_set_eoi_accelerated(struct kvm_vcpu *vcpu, int vector) } EXPORT_SYMBOL_GPL(kvm_apic_set_eoi_accelerated); -static void apic_send_ipi(struct kvm_lapic *apic, u32 icr_low, u32 icr_high) +void kvm_apic_send_ipi(struct kvm_lapic *apic, u32 icr_low, u32 icr_high) { struct kvm_lapic_irq irq; @@ -1940,7 +1940,7 @@ int kvm_lapic_reg_write(struct kvm_lapic *apic, u32 reg, u32 val) case APIC_ICR: /* No delay here, so we always clear the pending bit */ val &= ~(1 << 12); - apic_send_ipi(apic, val, kvm_lapic_get_reg(apic, APIC_ICR2)); + kvm_apic_send_ipi(apic, val, kvm_lapic_get_reg(apic, APIC_ICR2)); kvm_lapic_set_reg(apic, APIC_ICR, val); break; diff --git a/arch/x86/kvm/lapic.h b/arch/x86/kvm/lapic.h index ec6fbfe..bc76860 100644 --- a/arch/x86/kvm/lapic.h +++ b/arch/x86/kvm/lapic.h @@ -95,6 +95,7 @@ void kvm_apic_update_apicv(struct kvm_vcpu *vcpu); bool kvm_irq_delivery_to_apic_fast(struct kvm *kvm, struct kvm_lapic *src, struct kvm_lapic_irq *irq, int *r, struct dest_map *dest_map); +void kvm_apic_send_ipi(struct kvm_lapic *apic, u32 icr_low, u32 icr_high); u64 kvm_get_apic_base(struct kvm_vcpu *vcpu); int kvm_set_apic_base(struct kvm_vcpu *vcpu, struct msr_data *msr_info); diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 50ef1c5..c4bb7d8 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -1561,8 +1561,12 @@ static int handle_fastpath_set_x2apic_icr_irqoff(struct kvm_vcpu *vcpu, u64 data ((data & APIC_DEST_MASK) == APIC_DEST_PHYSICAL) && ((data & APIC_MODE_MASK) == APIC_DM_FIXED)) { + data &= ~(1 << 12); + kvm_apic_send_ipi(vcpu->arch.apic, (u32)data, (u32)(data >> 32)); kvm_lapic_set_reg(vcpu->arch.apic, APIC_ICR2, (u32)(data >> 32)); - return kvm_lapic_reg_write(vcpu->arch.apic, APIC_ICR, (u32)data); + kvm_lapic_set_reg(vcpu->arch.apic, APIC_ICR, (u32)data); + trace_kvm_apic_write(APIC_ICR, (u32)data); + return 0; } return 1;