From patchwork Wed May 12 15:09:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Stamatis, Ilias" X-Patchwork-Id: 12254079 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5F153C2B9F2 for ; Wed, 12 May 2021 15:16:08 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2B5DD61979 for ; Wed, 12 May 2021 15:16:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232307AbhELPRL (ORCPT ); Wed, 12 May 2021 11:17:11 -0400 Received: from smtp-fw-80006.amazon.com ([99.78.197.217]:49139 "EHLO smtp-fw-80006.amazon.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233423AbhELPNY (ORCPT ); Wed, 12 May 2021 11:13:24 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1620832336; x=1652368336; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version; bh=dd+cfBA8UFfC6dBFQo/XyrQ8fEmJ+IwhldFGtGJFtIU=; b=m8EqzTjxxfM+pZFyDTOxUJKnkFgbKCb3k8KJCFLWynWhIABS1mkQVP4b XnHqRUdMV8+sAVWuvmgcLoKUm9ycYxFOv/nKSoarasVINBpSgAJAU9Upl vINoNlli4mnx6ke2u0Bhan0TWG/UBcKJ+4Q6qZ+CR1LXvW7Jd+dFxhuxk c=; X-IronPort-AV: E=Sophos;i="5.82,293,1613433600"; d="scan'208";a="1007669" Received: from pdx4-co-svc-p1-lb2-vlan2.amazon.com (HELO email-inbound-relay-1a-7d76a15f.us-east-1.amazon.com) ([10.25.36.210]) by smtp-border-fw-80006.pdx80.corp.amazon.com with ESMTP; 12 May 2021 15:12:04 +0000 Received: from EX13MTAUEB001.ant.amazon.com (iad12-ws-svc-p26-lb9-vlan2.iad.amazon.com [10.40.163.34]) by email-inbound-relay-1a-7d76a15f.us-east-1.amazon.com (Postfix) with ESMTPS id AE104A24EF; Wed, 12 May 2021 15:12:00 +0000 (UTC) Received: from EX13D08UEB003.ant.amazon.com (10.43.60.11) by EX13MTAUEB001.ant.amazon.com (10.43.60.129) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 12 May 2021 15:11:45 +0000 Received: from EX13MTAUEB002.ant.amazon.com (10.43.60.12) by EX13D08UEB003.ant.amazon.com (10.43.60.11) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 12 May 2021 15:11:45 +0000 Received: from uae075a0dfd4c51.ant.amazon.com (10.106.82.24) by mail-relay.amazon.com (10.43.60.234) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Wed, 12 May 2021 15:11:43 +0000 From: Ilias Stamatis To: , , CC: , , , , , , , , , Subject: [PATCH v2 01/10] math64.h: Add mul_s64_u64_shr() Date: Wed, 12 May 2021 16:09:36 +0100 Message-ID: <20210512150945.4591-2-ilstam@amazon.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210512150945.4591-1-ilstam@amazon.com> References: <20210512150945.4591-1-ilstam@amazon.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org This function is needed for the nested TSC scaling code where we need to multiply the signed TSC offset with the unsigned TSC multiplier. Signed-off-by: Ilias Stamatis --- include/linux/math64.h | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/include/linux/math64.h b/include/linux/math64.h index 66deb1fdc2ef..2928f03d6d46 100644 --- a/include/linux/math64.h +++ b/include/linux/math64.h @@ -3,6 +3,7 @@ #define _LINUX_MATH64_H #include +#include #include #include @@ -234,6 +235,24 @@ static inline u64 mul_u64_u64_shr(u64 a, u64 b, unsigned int shift) #endif +#ifndef mul_s64_u64_shr +static inline u64 mul_s64_u64_shr(s64 a, u64 b, unsigned int shift) +{ + u64 ret; + + /* + * Extract the sign before the multiplication and put it back + * afterwards if needed. + */ + ret = mul_u64_u64_shr(abs(a), b, shift); + + if (a < 0) + ret = -((s64) ret); + + return ret; +} +#endif /* mul_s64_u64_shr */ + #ifndef mul_u64_u32_div static inline u64 mul_u64_u32_div(u64 a, u32 mul, u32 divisor) { From patchwork Wed May 12 15:09:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Stamatis, Ilias" X-Patchwork-Id: 12254081 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 62D85C43617 for ; Wed, 12 May 2021 15:16:12 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2D1B861952 for ; Wed, 12 May 2021 15:16:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232435AbhELPRP (ORCPT ); Wed, 12 May 2021 11:17:15 -0400 Received: from smtp-fw-9103.amazon.com ([207.171.188.200]:48021 "EHLO smtp-fw-9103.amazon.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233422AbhELPN2 (ORCPT ); Wed, 12 May 2021 11:13:28 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1620832340; x=1652368340; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version; bh=VXa53pCFVR+35FCWlxUGm51ip9sDTKeMhg2XuHjq7BY=; b=uSKiY2S8JGY2JP8pzKo1EUbzhO84ZjEvAYUevVqhiifIfvkgA5MAEVl/ Dl/1j6ZnvhulOZAPL4hL8WizhDcEEwiLW95iaH6CwDLmYKWTNULr3/qYX GfBVHzdUh6QUaTwPs9IlCTT2SPw3KIWMoo6/9hhq4mo1qtKnDFxgf8hWc g=; X-IronPort-AV: E=Sophos;i="5.82,293,1613433600"; d="scan'208";a="932812873" Received: from pdx4-co-svc-p1-lb2-vlan3.amazon.com (HELO email-inbound-relay-1a-67b371d8.us-east-1.amazon.com) ([10.25.36.214]) by smtp-border-fw-9103.sea19.amazon.com with ESMTP; 12 May 2021 15:12:12 +0000 Received: from EX13MTAUEB001.ant.amazon.com (iad12-ws-svc-p26-lb9-vlan3.iad.amazon.com [10.40.163.38]) by email-inbound-relay-1a-67b371d8.us-east-1.amazon.com (Postfix) with ESMTPS id A028DA1861; Wed, 12 May 2021 15:12:11 +0000 (UTC) Received: from EX13D08UEB003.ant.amazon.com (10.43.60.11) by EX13MTAUEB001.ant.amazon.com (10.43.60.129) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 12 May 2021 15:12:00 +0000 Received: from EX13MTAUEB002.ant.amazon.com (10.43.60.12) by EX13D08UEB003.ant.amazon.com (10.43.60.11) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 12 May 2021 15:12:00 +0000 Received: from uae075a0dfd4c51.ant.amazon.com (10.106.82.24) by mail-relay.amazon.com (10.43.60.234) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Wed, 12 May 2021 15:11:57 +0000 From: Ilias Stamatis To: , , CC: , , , , , , , , , Subject: [PATCH v2 02/10] KVM: X86: Store L1's TSC scaling ratio in 'struct kvm_vcpu_arch' Date: Wed, 12 May 2021 16:09:37 +0100 Message-ID: <20210512150945.4591-3-ilstam@amazon.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210512150945.4591-1-ilstam@amazon.com> References: <20210512150945.4591-1-ilstam@amazon.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Store L1's scaling ratio in that struct like we already do for L1's TSC offset. This allows for easy save/restore when we enter and then exit the nested guest. Signed-off-by: Ilias Stamatis Reviewed-by: Maxim Levitsky --- arch/x86/include/asm/kvm_host.h | 5 +++-- arch/x86/kvm/x86.c | 6 ++++-- 2 files changed, 7 insertions(+), 4 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 55efbacfc244..7dfc609eacd6 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -707,7 +707,7 @@ struct kvm_vcpu_arch { } st; u64 l1_tsc_offset; - u64 tsc_offset; + u64 tsc_offset; /* current tsc offset */ u64 last_guest_tsc; u64 last_host_tsc; u64 tsc_offset_adjustment; @@ -721,7 +721,8 @@ struct kvm_vcpu_arch { u32 virtual_tsc_khz; s64 ia32_tsc_adjust_msr; u64 msr_ia32_power_ctl; - u64 tsc_scaling_ratio; + u64 l1_tsc_scaling_ratio; + u64 tsc_scaling_ratio; /* current scaling ratio */ atomic_t nmi_queued; /* unprocessed asynchronous NMIs */ unsigned nmi_pending; /* NMI queued after currently running handler */ diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 9b6bca616929..07cf5d7ece38 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -2185,6 +2185,7 @@ static int set_tsc_khz(struct kvm_vcpu *vcpu, u32 user_tsc_khz, bool scale) /* Guest TSC same frequency as host TSC? */ if (!scale) { + vcpu->arch.l1_tsc_scaling_ratio = kvm_default_tsc_scaling_ratio; vcpu->arch.tsc_scaling_ratio = kvm_default_tsc_scaling_ratio; return 0; } @@ -2211,7 +2212,7 @@ static int set_tsc_khz(struct kvm_vcpu *vcpu, u32 user_tsc_khz, bool scale) return -1; } - vcpu->arch.tsc_scaling_ratio = ratio; + vcpu->arch.l1_tsc_scaling_ratio = vcpu->arch.tsc_scaling_ratio = ratio; return 0; } @@ -2223,6 +2224,7 @@ static int kvm_set_tsc_khz(struct kvm_vcpu *vcpu, u32 user_tsc_khz) /* tsc_khz can be zero if TSC calibration fails */ if (user_tsc_khz == 0) { /* set tsc_scaling_ratio to a safe value */ + vcpu->arch.l1_tsc_scaling_ratio = kvm_default_tsc_scaling_ratio; vcpu->arch.tsc_scaling_ratio = kvm_default_tsc_scaling_ratio; return -1; } @@ -2459,7 +2461,7 @@ static inline void adjust_tsc_offset_guest(struct kvm_vcpu *vcpu, static inline void adjust_tsc_offset_host(struct kvm_vcpu *vcpu, s64 adjustment) { - if (vcpu->arch.tsc_scaling_ratio != kvm_default_tsc_scaling_ratio) + if (vcpu->arch.l1_tsc_scaling_ratio != kvm_default_tsc_scaling_ratio) WARN_ON(adjustment < 0); adjustment = kvm_scale_tsc(vcpu, (u64) adjustment); adjust_tsc_offset_guest(vcpu, adjustment); From patchwork Wed May 12 15:09:38 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Stamatis, Ilias" X-Patchwork-Id: 12254087 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E1AE9C43470 for ; Wed, 12 May 2021 15:16:27 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id AEDE861950 for ; Wed, 12 May 2021 15:16:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232939AbhELPRb (ORCPT ); Wed, 12 May 2021 11:17:31 -0400 Received: from smtp-fw-33001.amazon.com ([207.171.190.10]:52922 "EHLO smtp-fw-33001.amazon.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233557AbhELPOJ (ORCPT ); Wed, 12 May 2021 11:14:09 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1620832382; x=1652368382; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version; bh=OwNSwH+TaVkRKL3rLBLITgVEcNInQsK25TDaYcBoxUY=; b=evRULVYRoQLF+asSaZx3o93EuTmHhs9lrWVHA4DCS0/HiJNsy8Lbrj2B EVBqDl8RjxRlSXzREsq1NlUjrdmuUeZQQj5OJ6jC1GltzRo39J/zeFxMd s+QB2yI5pupXaZ7rInnBj6jKZwajNVdLPFg8U8XpciaHv8phppz/gg9CD Y=; X-IronPort-AV: E=Sophos;i="5.82,293,1613433600"; d="scan'208";a="125400032" Received: from pdx4-co-svc-p1-lb2-vlan3.amazon.com (HELO email-inbound-relay-2c-76e0922c.us-west-2.amazon.com) ([10.25.36.214]) by smtp-border-fw-33001.sea14.amazon.com with ESMTP; 12 May 2021 15:12:55 +0000 Received: from EX13MTAUEB002.ant.amazon.com (pdx1-ws-svc-p6-lb9-vlan2.pdx.amazon.com [10.236.137.194]) by email-inbound-relay-2c-76e0922c.us-west-2.amazon.com (Postfix) with ESMTPS id 5C8E2A1C8F; Wed, 12 May 2021 15:12:51 +0000 (UTC) Received: from EX13D08UEB003.ant.amazon.com (10.43.60.11) by EX13MTAUEB002.ant.amazon.com (10.43.60.12) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 12 May 2021 15:12:09 +0000 Received: from EX13MTAUEB002.ant.amazon.com (10.43.60.12) by EX13D08UEB003.ant.amazon.com (10.43.60.11) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 12 May 2021 15:12:10 +0000 Received: from uae075a0dfd4c51.ant.amazon.com (10.106.82.24) by mail-relay.amazon.com (10.43.60.234) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Wed, 12 May 2021 15:12:08 +0000 From: Ilias Stamatis To: , , CC: , , , , , , , , , Subject: [PATCH v2 03/10] KVM: X86: Add kvm_scale_tsc_l1() and kvm_compute_tsc_offset_l1() Date: Wed, 12 May 2021 16:09:38 +0100 Message-ID: <20210512150945.4591-4-ilstam@amazon.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210512150945.4591-1-ilstam@amazon.com> References: <20210512150945.4591-1-ilstam@amazon.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org The existing kvm_scale_tsc() scales the TSC using the current TSC scaling ratio. That used to be the same as L1's scaling ratio but now with nested TSC scaling support it is no longer the case. This patch adds a new kvm_scale_tsc_l1() function that scales the TSC using L1's scaling ratio. The existing kvm_scale_tsc() can still be used for scaling L2 TSC values. Additionally, this patch renames the kvm_compute_tsc_offset() function to kvm_compute_tsc_offset_l1() and has the function treat its TSC argument as an L1 TSC value. All existing code uses this function passing L1 values to it. Signed-off-by: Ilias Stamatis --- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/x86.c | 41 ++++++++++++++++++++++----------- 2 files changed, 29 insertions(+), 13 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 7dfc609eacd6..be59197e5eb7 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1789,6 +1789,7 @@ static inline bool kvm_is_supported_user_return_msr(u32 msr) } u64 kvm_scale_tsc(struct kvm_vcpu *vcpu, u64 tsc); +u64 kvm_scale_tsc_l1(struct kvm_vcpu *vcpu, u64 tsc); u64 kvm_read_l1_tsc(struct kvm_vcpu *vcpu, u64 host_tsc); unsigned long kvm_get_linear_rip(struct kvm_vcpu *vcpu); diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 07cf5d7ece38..84af1af7a2cc 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -2319,18 +2319,30 @@ u64 kvm_scale_tsc(struct kvm_vcpu *vcpu, u64 tsc) } EXPORT_SYMBOL_GPL(kvm_scale_tsc); -static u64 kvm_compute_tsc_offset(struct kvm_vcpu *vcpu, u64 target_tsc) +u64 kvm_scale_tsc_l1(struct kvm_vcpu *vcpu, u64 tsc) +{ + u64 _tsc = tsc; + u64 ratio = vcpu->arch.l1_tsc_scaling_ratio; + + if (ratio != kvm_default_tsc_scaling_ratio) + _tsc = __scale_tsc(ratio, tsc); + + return _tsc; +} +EXPORT_SYMBOL_GPL(kvm_scale_tsc_l1); + +static u64 kvm_compute_tsc_offset_l1(struct kvm_vcpu *vcpu, u64 target_tsc) { u64 tsc; - tsc = kvm_scale_tsc(vcpu, rdtsc()); + tsc = kvm_scale_tsc_l1(vcpu, rdtsc()); return target_tsc - tsc; } u64 kvm_read_l1_tsc(struct kvm_vcpu *vcpu, u64 host_tsc) { - return vcpu->arch.l1_tsc_offset + kvm_scale_tsc(vcpu, host_tsc); + return vcpu->arch.l1_tsc_offset + kvm_scale_tsc_l1(vcpu, host_tsc); } EXPORT_SYMBOL_GPL(kvm_read_l1_tsc); @@ -2363,7 +2375,7 @@ static void kvm_synchronize_tsc(struct kvm_vcpu *vcpu, u64 data) bool synchronizing = false; raw_spin_lock_irqsave(&kvm->arch.tsc_write_lock, flags); - offset = kvm_compute_tsc_offset(vcpu, data); + offset = kvm_compute_tsc_offset_l1(vcpu, data); ns = get_kvmclock_base_ns(); elapsed = ns - kvm->arch.last_tsc_nsec; @@ -2402,7 +2414,7 @@ static void kvm_synchronize_tsc(struct kvm_vcpu *vcpu, u64 data) } else { u64 delta = nsec_to_cycles(vcpu, elapsed); data += delta; - offset = kvm_compute_tsc_offset(vcpu, data); + offset = kvm_compute_tsc_offset_l1(vcpu, data); } matched = true; already_matched = (vcpu->arch.this_tsc_generation == kvm->arch.cur_tsc_generation); @@ -2463,7 +2475,7 @@ static inline void adjust_tsc_offset_host(struct kvm_vcpu *vcpu, s64 adjustment) { if (vcpu->arch.l1_tsc_scaling_ratio != kvm_default_tsc_scaling_ratio) WARN_ON(adjustment < 0); - adjustment = kvm_scale_tsc(vcpu, (u64) adjustment); + adjustment = kvm_scale_tsc_l1(vcpu, (u64) adjustment); adjust_tsc_offset_guest(vcpu, adjustment); } @@ -2846,7 +2858,7 @@ static int kvm_guest_time_update(struct kvm_vcpu *v) /* With all the info we got, fill in the values */ if (kvm_has_tsc_control) - tgt_tsc_khz = kvm_scale_tsc(v, tgt_tsc_khz); + tgt_tsc_khz = kvm_scale_tsc_l1(v, tgt_tsc_khz); if (unlikely(vcpu->hw_tsc_khz != tgt_tsc_khz)) { kvm_get_time_scale(NSEC_PER_SEC, tgt_tsc_khz * 1000LL, @@ -3235,7 +3247,7 @@ int kvm_set_msr_common(struct kvm_vcpu *vcpu, struct msr_data *msr_info) if (msr_info->host_initiated) { kvm_synchronize_tsc(vcpu, data); } else { - u64 adj = kvm_compute_tsc_offset(vcpu, data) - vcpu->arch.l1_tsc_offset; + u64 adj = kvm_compute_tsc_offset_l1(vcpu, data) - vcpu->arch.l1_tsc_offset; adjust_tsc_offset_guest(vcpu, adj); vcpu->arch.ia32_tsc_adjust_msr += adj; } @@ -3537,10 +3549,13 @@ int kvm_get_msr_common(struct kvm_vcpu *vcpu, struct msr_data *msr_info) * return L1's TSC value to ensure backwards-compatible * behavior for migration. */ - u64 tsc_offset = msr_info->host_initiated ? vcpu->arch.l1_tsc_offset : - vcpu->arch.tsc_offset; - - msr_info->data = kvm_scale_tsc(vcpu, rdtsc()) + tsc_offset; + if (msr_info->host_initiated) { + msr_info->data = kvm_scale_tsc_l1(vcpu, rdtsc()) + + vcpu->arch.l1_tsc_offset; + } else { + msr_info->data = kvm_scale_tsc(vcpu, rdtsc()) + + vcpu->arch.tsc_offset; + } break; } case MSR_MTRRcap: @@ -4123,7 +4138,7 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu) mark_tsc_unstable("KVM discovered backwards TSC"); if (kvm_check_tsc_unstable()) { - u64 offset = kvm_compute_tsc_offset(vcpu, + u64 offset = kvm_compute_tsc_offset_l1(vcpu, vcpu->arch.last_guest_tsc); kvm_vcpu_write_tsc_offset(vcpu, offset); vcpu->arch.tsc_catchup = 1; From patchwork Wed May 12 15:09:39 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Stamatis, Ilias" X-Patchwork-Id: 12254085 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5873BC43470 for ; Wed, 12 May 2021 15:16:22 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1BA5661950 for ; Wed, 12 May 2021 15:16:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231783AbhELPRW (ORCPT ); Wed, 12 May 2021 11:17:22 -0400 Received: from smtp-fw-9103.amazon.com ([207.171.188.200]:49050 "EHLO smtp-fw-9103.amazon.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233532AbhELPOI (ORCPT ); Wed, 12 May 2021 11:14:08 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1620832380; x=1652368380; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version; bh=SU+g1QnEecCGtW6dIbozX2en41atljcTC+E2dUOIv3U=; b=msK0IhpfNt6O8BN1hps091DkYQ9A91nrqGCvW3O/aWqVwFsfotI84E59 cShvj38t2bBKrKHa69EYEzBV7AnCdic1Ols43AS1y8oLkkbmopjoGm0Ft RLPw8B6aKSRpRtKaMA2/M82T5CHxCGqoD2K4aRv1+c9oIkrGMNR+wEbxp 0=; X-IronPort-AV: E=Sophos;i="5.82,293,1613433600"; d="scan'208";a="932813016" Received: from pdx4-co-svc-p1-lb2-vlan3.amazon.com (HELO email-inbound-relay-1d-37fd6b3d.us-east-1.amazon.com) ([10.25.36.214]) by smtp-border-fw-9103.sea19.amazon.com with ESMTP; 12 May 2021 15:12:59 +0000 Received: from EX13MTAUEB001.ant.amazon.com (iad12-ws-svc-p26-lb9-vlan2.iad.amazon.com [10.40.163.34]) by email-inbound-relay-1d-37fd6b3d.us-east-1.amazon.com (Postfix) with ESMTPS id DB4DE28039A; Wed, 12 May 2021 15:12:55 +0000 (UTC) Received: from EX13D08UEB003.ant.amazon.com (10.43.60.11) by EX13MTAUEB001.ant.amazon.com (10.43.60.129) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 12 May 2021 15:12:23 +0000 Received: from EX13MTAUEB002.ant.amazon.com (10.43.60.12) by EX13D08UEB003.ant.amazon.com (10.43.60.11) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 12 May 2021 15:12:23 +0000 Received: from uae075a0dfd4c51.ant.amazon.com (10.106.82.24) by mail-relay.amazon.com (10.43.60.234) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Wed, 12 May 2021 15:12:20 +0000 From: Ilias Stamatis To: , , CC: , , , , , , , , , Subject: [PATCH v2 04/10] KVM: VMX: Add a TSC multiplier field in VMCS12 Date: Wed, 12 May 2021 16:09:39 +0100 Message-ID: <20210512150945.4591-5-ilstam@amazon.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210512150945.4591-1-ilstam@amazon.com> References: <20210512150945.4591-1-ilstam@amazon.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org This is required for supporting nested TSC scaling. Signed-off-by: Ilias Stamatis Reviewed-by: Jim Mattson Reviewed-by: Maxim Levitsky --- arch/x86/kvm/vmx/vmcs12.c | 1 + arch/x86/kvm/vmx/vmcs12.h | 4 +++- 2 files changed, 4 insertions(+), 1 deletion(-) diff --git a/arch/x86/kvm/vmx/vmcs12.c b/arch/x86/kvm/vmx/vmcs12.c index 034adb6404dc..d9f5d7c56ae3 100644 --- a/arch/x86/kvm/vmx/vmcs12.c +++ b/arch/x86/kvm/vmx/vmcs12.c @@ -37,6 +37,7 @@ const unsigned short vmcs_field_to_offset_table[] = { FIELD64(VM_ENTRY_MSR_LOAD_ADDR, vm_entry_msr_load_addr), FIELD64(PML_ADDRESS, pml_address), FIELD64(TSC_OFFSET, tsc_offset), + FIELD64(TSC_MULTIPLIER, tsc_multiplier), FIELD64(VIRTUAL_APIC_PAGE_ADDR, virtual_apic_page_addr), FIELD64(APIC_ACCESS_ADDR, apic_access_addr), FIELD64(POSTED_INTR_DESC_ADDR, posted_intr_desc_addr), diff --git a/arch/x86/kvm/vmx/vmcs12.h b/arch/x86/kvm/vmx/vmcs12.h index 13494956d0e9..bb81a23afe89 100644 --- a/arch/x86/kvm/vmx/vmcs12.h +++ b/arch/x86/kvm/vmx/vmcs12.h @@ -70,7 +70,8 @@ struct __packed vmcs12 { u64 eptp_list_address; u64 pml_address; u64 encls_exiting_bitmap; - u64 padding64[2]; /* room for future expansion */ + u64 tsc_multiplier; + u64 padding64[1]; /* room for future expansion */ /* * To allow migration of L1 (complete with its L2 guests) between * machines of different natural widths (32 or 64 bit), we cannot have @@ -258,6 +259,7 @@ static inline void vmx_check_vmcs12_offsets(void) CHECK_OFFSET(eptp_list_address, 304); CHECK_OFFSET(pml_address, 312); CHECK_OFFSET(encls_exiting_bitmap, 320); + CHECK_OFFSET(tsc_multiplier, 328); CHECK_OFFSET(cr0_guest_host_mask, 344); CHECK_OFFSET(cr4_guest_host_mask, 352); CHECK_OFFSET(cr0_read_shadow, 360); From patchwork Wed May 12 15:09:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Stamatis, Ilias" X-Patchwork-Id: 12254083 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C90CAC4360C for ; Wed, 12 May 2021 15:16:15 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 88AD061C38 for ; Wed, 12 May 2021 15:16:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232718AbhELPRT (ORCPT ); Wed, 12 May 2021 11:17:19 -0400 Received: from smtp-fw-6002.amazon.com ([52.95.49.90]:50091 "EHLO smtp-fw-6002.amazon.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233493AbhELPNt (ORCPT ); Wed, 12 May 2021 11:13:49 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1620832361; x=1652368361; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version; bh=kWONKALLAAkYdmcXihmg29EhGTugBqvFORDq7cQCa9M=; b=N4AItl63aD/Mq7z771WX0BfojIAWw1X2qqFKE8QEv3reK1Iv0IDuhq7D rT1e1oVrmj/r+Z8q4bqHzZlLgW3fwXRwDt0AwQzIRzxS3aKSuy4pucXa0 deMukJvCjWmucBg15qyo/CgenU0j4uYhSKKamUH1w6YfGzPH0w+hSB/hA 8=; X-IronPort-AV: E=Sophos;i="5.82,293,1613433600"; d="scan'208";a="111783534" Received: from iad12-co-svc-p1-lb1-vlan2.amazon.com (HELO email-inbound-relay-1e-27fb8269.us-east-1.amazon.com) ([10.43.8.2]) by smtp-border-fw-6002.iad6.amazon.com with ESMTP; 12 May 2021 15:12:36 +0000 Received: from EX13MTAUEB001.ant.amazon.com (iad12-ws-svc-p26-lb9-vlan3.iad.amazon.com [10.40.163.38]) by email-inbound-relay-1e-27fb8269.us-east-1.amazon.com (Postfix) with ESMTPS id EFC89A072C; Wed, 12 May 2021 15:12:32 +0000 (UTC) Received: from EX13D08UEB002.ant.amazon.com (10.43.60.107) by EX13MTAUEB001.ant.amazon.com (10.43.60.96) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 12 May 2021 15:12:32 +0000 Received: from EX13MTAUEB002.ant.amazon.com (10.43.60.12) by EX13D08UEB002.ant.amazon.com (10.43.60.107) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 12 May 2021 15:12:32 +0000 Received: from uae075a0dfd4c51.ant.amazon.com (10.106.82.24) by mail-relay.amazon.com (10.43.60.234) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Wed, 12 May 2021 15:12:30 +0000 From: Ilias Stamatis To: , , CC: , , , , , , , , , Subject: [PATCH v2 05/10] KVM: X86: Add functions for retrieving L2 TSC fields from common code Date: Wed, 12 May 2021 16:09:40 +0100 Message-ID: <20210512150945.4591-6-ilstam@amazon.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210512150945.4591-1-ilstam@amazon.com> References: <20210512150945.4591-1-ilstam@amazon.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org In order to implement as much of the nested TSC scaling logic as possible in common code, we need these vendor callbacks for retrieving the TSC offset and the TSC multiplier that L1 has set for L2. Signed-off-by: Ilias Stamatis --- arch/x86/include/asm/kvm-x86-ops.h | 2 ++ arch/x86/include/asm/kvm_host.h | 2 ++ arch/x86/kvm/svm/svm.c | 14 ++++++++++++++ arch/x86/kvm/vmx/vmx.c | 25 +++++++++++++++++++++++++ 4 files changed, 43 insertions(+) diff --git a/arch/x86/include/asm/kvm-x86-ops.h b/arch/x86/include/asm/kvm-x86-ops.h index 323641097f63..2063616fba1c 100644 --- a/arch/x86/include/asm/kvm-x86-ops.h +++ b/arch/x86/include/asm/kvm-x86-ops.h @@ -87,6 +87,8 @@ KVM_X86_OP(set_identity_map_addr) KVM_X86_OP(get_mt_mask) KVM_X86_OP(load_mmu_pgd) KVM_X86_OP_NULL(has_wbinvd_exit) +KVM_X86_OP(get_l2_tsc_offset) +KVM_X86_OP(get_l2_tsc_multiplier) KVM_X86_OP(write_l1_tsc_offset) KVM_X86_OP(get_exit_info) KVM_X86_OP(check_intercept) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index be59197e5eb7..4c4a3fefff57 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1305,6 +1305,8 @@ struct kvm_x86_ops { bool (*has_wbinvd_exit)(void); + u64 (*get_l2_tsc_offset)(struct kvm_vcpu *vcpu); + u64 (*get_l2_tsc_multiplier)(struct kvm_vcpu *vcpu); /* Returns actual tsc_offset set in active VMCS */ u64 (*write_l1_tsc_offset)(struct kvm_vcpu *vcpu, u64 offset); diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c index dfa351e605de..679b2fc1a3f9 100644 --- a/arch/x86/kvm/svm/svm.c +++ b/arch/x86/kvm/svm/svm.c @@ -1082,6 +1082,18 @@ static void init_sys_seg(struct vmcb_seg *seg, uint32_t type) seg->base = 0; } +static u64 svm_get_l2_tsc_offset(struct kvm_vcpu *vcpu) +{ + struct vcpu_svm *svm = to_svm(vcpu); + + return svm->nested.ctl.tsc_offset; +} + +static u64 svm_get_l2_tsc_multiplier(struct kvm_vcpu *vcpu) +{ + return kvm_default_tsc_scaling_ratio; +} + static u64 svm_write_l1_tsc_offset(struct kvm_vcpu *vcpu, u64 offset) { struct vcpu_svm *svm = to_svm(vcpu); @@ -4526,6 +4538,8 @@ static struct kvm_x86_ops svm_x86_ops __initdata = { .has_wbinvd_exit = svm_has_wbinvd_exit, + .get_l2_tsc_offset = svm_get_l2_tsc_offset, + .get_l2_tsc_multiplier = svm_get_l2_tsc_multiplier, .write_l1_tsc_offset = svm_write_l1_tsc_offset, .load_mmu_pgd = svm_load_mmu_pgd, diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 4bceb5ca3a89..575e13bddda8 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -1787,6 +1787,29 @@ static void setup_msrs(struct vcpu_vmx *vmx) vmx->guest_uret_msrs_loaded = false; } +static u64 vmx_get_l2_tsc_offset(struct kvm_vcpu *vcpu) +{ + struct vmcs12 *vmcs12 = get_vmcs12(vcpu); + u64 offset = 0; + + if (vmcs12->cpu_based_vm_exec_control & CPU_BASED_USE_TSC_OFFSETTING) + offset = vmcs12->tsc_offset; + + return offset; +} + +static u64 vmx_get_l2_tsc_multiplier(struct kvm_vcpu *vcpu) +{ + struct vmcs12 *vmcs12 = get_vmcs12(vcpu); + u64 multiplier = kvm_default_tsc_scaling_ratio; + + if (vmcs12->cpu_based_vm_exec_control & CPU_BASED_USE_TSC_OFFSETTING && + vmcs12->secondary_vm_exec_control & SECONDARY_EXEC_TSC_SCALING) + multiplier = vmcs12->tsc_multiplier; + + return multiplier; +} + static u64 vmx_write_l1_tsc_offset(struct kvm_vcpu *vcpu, u64 offset) { struct vmcs12 *vmcs12 = get_vmcs12(vcpu); @@ -7700,6 +7723,8 @@ static struct kvm_x86_ops vmx_x86_ops __initdata = { .has_wbinvd_exit = cpu_has_vmx_wbinvd_exit, + .get_l2_tsc_offset = vmx_get_l2_tsc_offset, + .get_l2_tsc_multiplier = vmx_get_l2_tsc_multiplier, .write_l1_tsc_offset = vmx_write_l1_tsc_offset, .load_mmu_pgd = vmx_load_mmu_pgd, From patchwork Wed May 12 15:09:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Stamatis, Ilias" X-Patchwork-Id: 12254089 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 00E84C433ED for ; Wed, 12 May 2021 15:16:36 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C3A4761977 for ; Wed, 12 May 2021 15:16:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233057AbhELPRf (ORCPT ); Wed, 12 May 2021 11:17:35 -0400 Received: from smtp-fw-6002.amazon.com ([52.95.49.90]:50864 "EHLO smtp-fw-6002.amazon.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233616AbhELPOP (ORCPT ); Wed, 12 May 2021 11:14:15 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1620832387; x=1652368387; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version; bh=OYeZ+yRdHpDkHcQ3lGwth3XxHtNx/6enPUD/fAb7PFE=; b=n/x++liXBBasste4ewoEpwAvQG5yn79zIKlOp11ZJpN0e7icW31O6XyB t3+50HbOmStjDwMZeJPOg6PLydXIt5AE26zGT9u81CZV+rAROvZr89C/Q mvT7Rw9guf6er7rtvNfjaZiwiKZIwyD5gqtqH8oSCPBS4isf1+h+lRigM s=; X-IronPort-AV: E=Sophos;i="5.82,293,1613433600"; d="scan'208";a="111783658" Received: from iad12-co-svc-p1-lb1-vlan2.amazon.com (HELO email-inbound-relay-2b-81e76b79.us-west-2.amazon.com) ([10.43.8.2]) by smtp-border-fw-6002.iad6.amazon.com with ESMTP; 12 May 2021 15:13:06 +0000 Received: from EX13MTAUEB002.ant.amazon.com (pdx1-ws-svc-p6-lb9-vlan3.pdx.amazon.com [10.236.137.198]) by email-inbound-relay-2b-81e76b79.us-west-2.amazon.com (Postfix) with ESMTPS id 1253AA06D9; Wed, 12 May 2021 15:13:05 +0000 (UTC) Received: from EX13D08UEB002.ant.amazon.com (10.43.60.107) by EX13MTAUEB002.ant.amazon.com (10.43.60.12) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 12 May 2021 15:12:41 +0000 Received: from EX13MTAUEB002.ant.amazon.com (10.43.60.12) by EX13D08UEB002.ant.amazon.com (10.43.60.107) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 12 May 2021 15:12:41 +0000 Received: from uae075a0dfd4c51.ant.amazon.com (10.106.82.24) by mail-relay.amazon.com (10.43.60.234) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Wed, 12 May 2021 15:12:39 +0000 From: Ilias Stamatis To: , , CC: , , , , , , , , , Subject: [PATCH v2 06/10] KVM: X86: Add functions that calculate the 02 TSC offset and multiplier Date: Wed, 12 May 2021 16:09:41 +0100 Message-ID: <20210512150945.4591-7-ilstam@amazon.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210512150945.4591-1-ilstam@amazon.com> References: <20210512150945.4591-1-ilstam@amazon.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org When L2 is entered we need to "merge" the TSC multiplier and TSC offset values of 01 and 12 together. The merging is done using the following equations: offset_02 = ((offset_01 * mult_12) >> shift_bits) + offset_12 mult_02 = (mult_01 * mult_12) >> shift_bits Where shift_bits is kvm_tsc_scaling_ratio_frac_bits. Signed-off-by: Ilias Stamatis --- arch/x86/include/asm/kvm_host.h | 2 ++ arch/x86/kvm/x86.c | 29 +++++++++++++++++++++++++++++ 2 files changed, 31 insertions(+) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 4c4a3fefff57..57a25d8e8b0f 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1793,6 +1793,8 @@ static inline bool kvm_is_supported_user_return_msr(u32 msr) u64 kvm_scale_tsc(struct kvm_vcpu *vcpu, u64 tsc); u64 kvm_scale_tsc_l1(struct kvm_vcpu *vcpu, u64 tsc); u64 kvm_read_l1_tsc(struct kvm_vcpu *vcpu, u64 host_tsc); +void kvm_set_02_tsc_offset(struct kvm_vcpu *vcpu); +void kvm_set_02_tsc_multiplier(struct kvm_vcpu *vcpu); unsigned long kvm_get_linear_rip(struct kvm_vcpu *vcpu); bool kvm_is_linear_rip(struct kvm_vcpu *vcpu, unsigned long linear_rip); diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 84af1af7a2cc..1db6cfc2079f 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -2346,6 +2346,35 @@ u64 kvm_read_l1_tsc(struct kvm_vcpu *vcpu, u64 host_tsc) } EXPORT_SYMBOL_GPL(kvm_read_l1_tsc); +void kvm_set_02_tsc_offset(struct kvm_vcpu *vcpu) +{ + u64 l2_offset = static_call(kvm_x86_get_l2_tsc_offset)(vcpu); + u64 l2_multiplier = static_call(kvm_x86_get_l2_tsc_multiplier)(vcpu); + + if (l2_multiplier != kvm_default_tsc_scaling_ratio) { + vcpu->arch.tsc_offset = mul_s64_u64_shr( + (s64) vcpu->arch.l1_tsc_offset, + l2_multiplier, + kvm_tsc_scaling_ratio_frac_bits); + } + + vcpu->arch.tsc_offset += l2_offset; +} +EXPORT_SYMBOL_GPL(kvm_set_02_tsc_offset); + +void kvm_set_02_tsc_multiplier(struct kvm_vcpu *vcpu) +{ + u64 l2_multiplier = static_call(kvm_x86_get_l2_tsc_multiplier)(vcpu); + + if (l2_multiplier != kvm_default_tsc_scaling_ratio) { + vcpu->arch.tsc_scaling_ratio = mul_u64_u64_shr( + vcpu->arch.l1_tsc_scaling_ratio, + l2_multiplier, + kvm_tsc_scaling_ratio_frac_bits); + } +} +EXPORT_SYMBOL_GPL(kvm_set_02_tsc_multiplier); + static void kvm_vcpu_write_tsc_offset(struct kvm_vcpu *vcpu, u64 offset) { vcpu->arch.l1_tsc_offset = offset; From patchwork Wed May 12 15:09:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Stamatis, Ilias" X-Patchwork-Id: 12254091 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 063F1C433B4 for ; Wed, 12 May 2021 15:16:44 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C7EC761C47 for ; Wed, 12 May 2021 15:16:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233134AbhELPRn (ORCPT ); Wed, 12 May 2021 11:17:43 -0400 Received: from smtp-fw-9103.amazon.com ([207.171.188.200]:49216 "EHLO smtp-fw-9103.amazon.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233617AbhELPOP (ORCPT ); Wed, 12 May 2021 11:14:15 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1620832387; x=1652368387; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version; bh=dCPeFV5B3yxfSMAfc1TaVPUCDD3dKVYzbYjdG1JDI2I=; b=fdTOJWlhcEM8wq86RmQXoWSozflwxcgjRBTXAzgVsvt8j/EsEU9VH41Q DhkNK83EW4O5cDDftU1M+7pn0SFT7gsPYUZqFBvuxMOrfeEcG2qs0Ki2P oZf1+xWvbSPgXzxyEBiP6zpkiQseqw1jlWN1IaNTugfTBh4EmGtsZxCw4 0=; X-IronPort-AV: E=Sophos;i="5.82,293,1613433600"; d="scan'208";a="932813036" Received: from pdx4-co-svc-p1-lb2-vlan3.amazon.com (HELO email-inbound-relay-1e-c7f73527.us-east-1.amazon.com) ([10.25.36.214]) by smtp-border-fw-9103.sea19.amazon.com with ESMTP; 12 May 2021 15:13:05 +0000 Received: from EX13MTAUEB001.ant.amazon.com (iad12-ws-svc-p26-lb9-vlan3.iad.amazon.com [10.40.163.38]) by email-inbound-relay-1e-c7f73527.us-east-1.amazon.com (Postfix) with ESMTPS id 40E49BA432; Wed, 12 May 2021 15:13:02 +0000 (UTC) Received: from EX13D08UEB002.ant.amazon.com (10.43.60.107) by EX13MTAUEB001.ant.amazon.com (10.43.60.96) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 12 May 2021 15:12:51 +0000 Received: from EX13MTAUEB002.ant.amazon.com (10.43.60.12) by EX13D08UEB002.ant.amazon.com (10.43.60.107) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 12 May 2021 15:12:51 +0000 Received: from uae075a0dfd4c51.ant.amazon.com (10.106.82.24) by mail-relay.amazon.com (10.43.60.234) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Wed, 12 May 2021 15:12:49 +0000 From: Ilias Stamatis To: , , CC: , , , , , , , , , Subject: [PATCH v2 07/10] KVM: X86: Move write_l1_tsc_offset() logic to common code and rename it Date: Wed, 12 May 2021 16:09:42 +0100 Message-ID: <20210512150945.4591-8-ilstam@amazon.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210512150945.4591-1-ilstam@amazon.com> References: <20210512150945.4591-1-ilstam@amazon.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org The write_l1_tsc_offset() callback has a misleading name. It does not set L1's TSC offset, it rather updates the current TSC offset which might be different if a nested guest is executing. Additionally, both the vmx and svm implementations use the same logic for calculating the current TSC before writing it to hardware. This patch renames the function and moves the common logic to the caller. The vmx/svm-specific code now merely sets the given offset to the corresponding hardware structure. Signed-off-by: Ilias Stamatis --- arch/x86/include/asm/kvm-x86-ops.h | 2 +- arch/x86/include/asm/kvm_host.h | 3 +-- arch/x86/kvm/svm/svm.c | 21 ++++----------------- arch/x86/kvm/vmx/vmx.c | 23 +++-------------------- arch/x86/kvm/x86.c | 17 ++++++++++++++++- 5 files changed, 25 insertions(+), 41 deletions(-) diff --git a/arch/x86/include/asm/kvm-x86-ops.h b/arch/x86/include/asm/kvm-x86-ops.h index 2063616fba1c..029c9615378f 100644 --- a/arch/x86/include/asm/kvm-x86-ops.h +++ b/arch/x86/include/asm/kvm-x86-ops.h @@ -89,7 +89,7 @@ KVM_X86_OP(load_mmu_pgd) KVM_X86_OP_NULL(has_wbinvd_exit) KVM_X86_OP(get_l2_tsc_offset) KVM_X86_OP(get_l2_tsc_multiplier) -KVM_X86_OP(write_l1_tsc_offset) +KVM_X86_OP(write_tsc_offset) KVM_X86_OP(get_exit_info) KVM_X86_OP(check_intercept) KVM_X86_OP(handle_exit_irqoff) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 57a25d8e8b0f..61cf201c001a 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1307,8 +1307,7 @@ struct kvm_x86_ops { u64 (*get_l2_tsc_offset)(struct kvm_vcpu *vcpu); u64 (*get_l2_tsc_multiplier)(struct kvm_vcpu *vcpu); - /* Returns actual tsc_offset set in active VMCS */ - u64 (*write_l1_tsc_offset)(struct kvm_vcpu *vcpu, u64 offset); + void (*write_tsc_offset)(struct kvm_vcpu *vcpu, u64 offset); /* * Retrieve somewhat arbitrary exit information. Intended to be used diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c index 679b2fc1a3f9..b18f60463073 100644 --- a/arch/x86/kvm/svm/svm.c +++ b/arch/x86/kvm/svm/svm.c @@ -1094,26 +1094,13 @@ static u64 svm_get_l2_tsc_multiplier(struct kvm_vcpu *vcpu) return kvm_default_tsc_scaling_ratio; } -static u64 svm_write_l1_tsc_offset(struct kvm_vcpu *vcpu, u64 offset) +static void svm_write_tsc_offset(struct kvm_vcpu *vcpu, u64 offset) { struct vcpu_svm *svm = to_svm(vcpu); - u64 g_tsc_offset = 0; - - if (is_guest_mode(vcpu)) { - /* Write L1's TSC offset. */ - g_tsc_offset = svm->vmcb->control.tsc_offset - - svm->vmcb01.ptr->control.tsc_offset; - svm->vmcb01.ptr->control.tsc_offset = offset; - } - - trace_kvm_write_tsc_offset(vcpu->vcpu_id, - svm->vmcb->control.tsc_offset - g_tsc_offset, - offset); - - svm->vmcb->control.tsc_offset = offset + g_tsc_offset; + svm->vmcb01.ptr->control.tsc_offset = vcpu->arch.l1_tsc_offset; + svm->vmcb->control.tsc_offset = offset; vmcb_mark_dirty(svm->vmcb, VMCB_INTERCEPTS); - return svm->vmcb->control.tsc_offset; } /* Evaluate instruction intercepts that depend on guest CPUID features. */ @@ -4540,7 +4527,7 @@ static struct kvm_x86_ops svm_x86_ops __initdata = { .get_l2_tsc_offset = svm_get_l2_tsc_offset, .get_l2_tsc_multiplier = svm_get_l2_tsc_multiplier, - .write_l1_tsc_offset = svm_write_l1_tsc_offset, + .write_tsc_offset = svm_write_tsc_offset, .load_mmu_pgd = svm_load_mmu_pgd, diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 575e13bddda8..3c4eb14a1e86 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -1810,26 +1810,9 @@ static u64 vmx_get_l2_tsc_multiplier(struct kvm_vcpu *vcpu) return multiplier; } -static u64 vmx_write_l1_tsc_offset(struct kvm_vcpu *vcpu, u64 offset) +static void vmx_write_tsc_offset(struct kvm_vcpu *vcpu, u64 offset) { - struct vmcs12 *vmcs12 = get_vmcs12(vcpu); - u64 g_tsc_offset = 0; - - /* - * We're here if L1 chose not to trap WRMSR to TSC. According - * to the spec, this should set L1's TSC; The offset that L1 - * set for L2 remains unchanged, and still needs to be added - * to the newly set TSC to get L2's TSC. - */ - if (is_guest_mode(vcpu) && - (vmcs12->cpu_based_vm_exec_control & CPU_BASED_USE_TSC_OFFSETTING)) - g_tsc_offset = vmcs12->tsc_offset; - - trace_kvm_write_tsc_offset(vcpu->vcpu_id, - vcpu->arch.tsc_offset - g_tsc_offset, - offset); - vmcs_write64(TSC_OFFSET, offset + g_tsc_offset); - return offset + g_tsc_offset; + vmcs_write64(TSC_OFFSET, offset); } /* @@ -7725,7 +7708,7 @@ static struct kvm_x86_ops vmx_x86_ops __initdata = { .get_l2_tsc_offset = vmx_get_l2_tsc_offset, .get_l2_tsc_multiplier = vmx_get_l2_tsc_multiplier, - .write_l1_tsc_offset = vmx_write_l1_tsc_offset, + .write_tsc_offset = vmx_write_tsc_offset, .load_mmu_pgd = vmx_load_mmu_pgd, diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 1db6cfc2079f..f3ba1be4d5b9 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -2377,8 +2377,23 @@ EXPORT_SYMBOL_GPL(kvm_set_02_tsc_multiplier); static void kvm_vcpu_write_tsc_offset(struct kvm_vcpu *vcpu, u64 offset) { + trace_kvm_write_tsc_offset(vcpu->vcpu_id, + vcpu->arch.l1_tsc_offset, + offset); + vcpu->arch.l1_tsc_offset = offset; - vcpu->arch.tsc_offset = static_call(kvm_x86_write_l1_tsc_offset)(vcpu, offset); + vcpu->arch.tsc_offset = offset; + + if (is_guest_mode(vcpu)) { + /* + * We're here if L1 chose not to trap WRMSR to TSC and + * according to the spec this should set L1's TSC (as opposed + * to setting L1's offset for L2). + */ + kvm_set_02_tsc_offset(vcpu); + } + + static_call(kvm_x86_write_tsc_offset)(vcpu, vcpu->arch.tsc_offset); } static inline bool kvm_check_tsc_unstable(void) From patchwork Wed May 12 15:09:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Stamatis, Ilias" X-Patchwork-Id: 12254093 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 450CFC43460 for ; Wed, 12 May 2021 15:16:44 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 13EC16197F for ; Wed, 12 May 2021 15:16:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233196AbhELPRo (ORCPT ); Wed, 12 May 2021 11:17:44 -0400 Received: from smtp-fw-33001.amazon.com ([207.171.190.10]:53424 "EHLO smtp-fw-33001.amazon.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233692AbhELPO2 (ORCPT ); Wed, 12 May 2021 11:14:28 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1620832400; x=1652368400; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version; bh=aODMbDqUW++xg0p7BAJBzog0uWqHgdYh2MyWroPFvxU=; b=C1PbLD+djI8nNqGrx1eTlpZuV4CSgPTuWEVQ3lrONP0R9FEMhnKfrsJn GaV1/9xV4xWJ8ogOemX7Y1ug/CEApTqnJoVBIK2o9UcgNjOGH9mvf3TAH 9MPjE8dGBpmp+C2Th/1Z3CEp7PsGusda20Mg5lowqRrkHjTsD9JvX8H9y Y=; X-IronPort-AV: E=Sophos;i="5.82,293,1613433600"; d="scan'208";a="125400149" Received: from pdx4-co-svc-p1-lb2-vlan3.amazon.com (HELO email-inbound-relay-1d-2c665b5d.us-east-1.amazon.com) ([10.25.36.214]) by smtp-border-fw-33001.sea14.amazon.com with ESMTP; 12 May 2021 15:13:19 +0000 Received: from EX13MTAUEB001.ant.amazon.com (iad12-ws-svc-p26-lb9-vlan3.iad.amazon.com [10.40.163.38]) by email-inbound-relay-1d-2c665b5d.us-east-1.amazon.com (Postfix) with ESMTPS id C0FB6A1D3B; Wed, 12 May 2021 15:13:15 +0000 (UTC) Received: from EX13D08UEB002.ant.amazon.com (10.43.60.107) by EX13MTAUEB001.ant.amazon.com (10.43.60.96) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 12 May 2021 15:13:05 +0000 Received: from EX13MTAUEB002.ant.amazon.com (10.43.60.12) by EX13D08UEB002.ant.amazon.com (10.43.60.107) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 12 May 2021 15:13:05 +0000 Received: from uae075a0dfd4c51.ant.amazon.com (10.106.82.24) by mail-relay.amazon.com (10.43.60.234) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Wed, 12 May 2021 15:13:03 +0000 From: Ilias Stamatis To: , , CC: , , , , , , , , , Subject: [PATCH v2 08/10] KVM: VMX: Set the TSC offset and multiplier on nested entry and exit Date: Wed, 12 May 2021 16:09:43 +0100 Message-ID: <20210512150945.4591-9-ilstam@amazon.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210512150945.4591-1-ilstam@amazon.com> References: <20210512150945.4591-1-ilstam@amazon.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Now that nested TSC scaling is supported we need to calculate the correct 02 values for both the offset and the multiplier using the corresponding functions. On L2's exit the L1 values are restored. Signed-off-by: Ilias Stamatis --- arch/x86/kvm/vmx/nested.c | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c index 6058a65a6ede..f1dff1ebaccb 100644 --- a/arch/x86/kvm/vmx/nested.c +++ b/arch/x86/kvm/vmx/nested.c @@ -3354,8 +3354,9 @@ enum nvmx_vmentry_status nested_vmx_enter_non_root_mode(struct kvm_vcpu *vcpu, } enter_guest_mode(vcpu); - if (vmcs12->cpu_based_vm_exec_control & CPU_BASED_USE_TSC_OFFSETTING) - vcpu->arch.tsc_offset += vmcs12->tsc_offset; + + kvm_set_02_tsc_offset(vcpu); + kvm_set_02_tsc_multiplier(vcpu); if (prepare_vmcs02(vcpu, vmcs12, &entry_failure_code)) { exit_reason.basic = EXIT_REASON_INVALID_STATE; @@ -4463,8 +4464,12 @@ void nested_vmx_vmexit(struct kvm_vcpu *vcpu, u32 vm_exit_reason, if (nested_cpu_has_preemption_timer(vmcs12)) hrtimer_cancel(&to_vmx(vcpu)->nested.preemption_timer); - if (vmcs12->cpu_based_vm_exec_control & CPU_BASED_USE_TSC_OFFSETTING) - vcpu->arch.tsc_offset -= vmcs12->tsc_offset; + if (vmcs12->cpu_based_vm_exec_control & CPU_BASED_USE_TSC_OFFSETTING) { + vcpu->arch.tsc_offset = vcpu->arch.l1_tsc_offset; + + if (vmcs12->secondary_vm_exec_control & SECONDARY_EXEC_TSC_SCALING) + vcpu->arch.tsc_scaling_ratio = vcpu->arch.l1_tsc_scaling_ratio; + } if (likely(!vmx->fail)) { sync_vmcs02_to_vmcs12(vcpu, vmcs12); From patchwork Wed May 12 15:09:44 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Stamatis, Ilias" X-Patchwork-Id: 12254095 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 41D06C433B4 for ; Wed, 12 May 2021 15:16:47 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0C67761C47 for ; Wed, 12 May 2021 15:16:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233263AbhELPRt (ORCPT ); Wed, 12 May 2021 11:17:49 -0400 Received: from smtp-fw-80007.amazon.com ([99.78.197.218]:13419 "EHLO smtp-fw-80007.amazon.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233758AbhELPOo (ORCPT ); Wed, 12 May 2021 11:14:44 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1620832417; x=1652368417; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version; bh=oMhJxrjs2Es+3529yXwQICPY1v/fvMiXw1lk6BUiGrw=; b=ewQH27FlzuHxE0FFpiB4EiqJBIMP7B2kPDH0SI4bLUfbebuKsF3G8OFn opHigdIfThhBVoFLpK1gfx93PThNiTktioKMxfURapY5Zf0rIi5AQfkSp bSAgkRGghNTtVMcjqr9ysWGgo72tPuEK9LHxKsqzXJklUXGTKg4Jq2FB9 Y=; X-IronPort-AV: E=Sophos;i="5.82,293,1613433600"; d="scan'208";a="984968" Received: from pdx4-co-svc-p1-lb2-vlan3.amazon.com (HELO email-inbound-relay-1a-807d4a99.us-east-1.amazon.com) ([10.25.36.214]) by smtp-border-fw-80007.pdx80.corp.amazon.com with ESMTP; 12 May 2021 15:13:29 +0000 Received: from EX13MTAUEB001.ant.amazon.com (iad12-ws-svc-p26-lb9-vlan2.iad.amazon.com [10.40.163.34]) by email-inbound-relay-1a-807d4a99.us-east-1.amazon.com (Postfix) with ESMTPS id 5D4A6A1F8E; Wed, 12 May 2021 15:13:25 +0000 (UTC) Received: from EX13D08UEB003.ant.amazon.com (10.43.60.11) by EX13MTAUEB001.ant.amazon.com (10.43.60.129) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 12 May 2021 15:13:15 +0000 Received: from EX13MTAUEB002.ant.amazon.com (10.43.60.12) by EX13D08UEB003.ant.amazon.com (10.43.60.11) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 12 May 2021 15:13:15 +0000 Received: from uae075a0dfd4c51.ant.amazon.com (10.106.82.24) by mail-relay.amazon.com (10.43.60.234) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Wed, 12 May 2021 15:13:13 +0000 From: Ilias Stamatis To: , , CC: , , , , , , , , , Subject: [PATCH v2 09/10] KVM: VMX: Expose TSC scaling to L2 Date: Wed, 12 May 2021 16:09:44 +0100 Message-ID: <20210512150945.4591-10-ilstam@amazon.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210512150945.4591-1-ilstam@amazon.com> References: <20210512150945.4591-1-ilstam@amazon.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Expose the TSC scaling feature to nested guests. Signed-off-by: Ilias Stamatis Reviewed-by: Maxim Levitsky --- arch/x86/kvm/vmx/nested.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c index f1dff1ebaccb..d431e17dbc5b 100644 --- a/arch/x86/kvm/vmx/nested.c +++ b/arch/x86/kvm/vmx/nested.c @@ -2277,7 +2277,8 @@ static void prepare_vmcs02_early(struct vcpu_vmx *vmx, struct vmcs12 *vmcs12) SECONDARY_EXEC_ENABLE_USR_WAIT_PAUSE | SECONDARY_EXEC_VIRTUAL_INTR_DELIVERY | SECONDARY_EXEC_APIC_REGISTER_VIRT | - SECONDARY_EXEC_ENABLE_VMFUNC); + SECONDARY_EXEC_ENABLE_VMFUNC | + SECONDARY_EXEC_TSC_SCALING); if (nested_cpu_has(vmcs12, CPU_BASED_ACTIVATE_SECONDARY_CONTROLS)) exec_control |= vmcs12->secondary_vm_exec_control; @@ -6479,7 +6480,8 @@ void nested_vmx_setup_ctls_msrs(struct nested_vmx_msrs *msrs, u32 ept_caps) SECONDARY_EXEC_RDRAND_EXITING | SECONDARY_EXEC_ENABLE_INVPCID | SECONDARY_EXEC_RDSEED_EXITING | - SECONDARY_EXEC_XSAVES; + SECONDARY_EXEC_XSAVES | + SECONDARY_EXEC_TSC_SCALING; /* * We can emulate "VMCS shadowing," even if the hardware From patchwork Wed May 12 15:09:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Stamatis, Ilias" X-Patchwork-Id: 12254097 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 87596C43611 for ; Wed, 12 May 2021 15:16:47 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5495A61C44 for ; Wed, 12 May 2021 15:16:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233279AbhELPRv (ORCPT ); Wed, 12 May 2021 11:17:51 -0400 Received: from smtp-fw-4101.amazon.com ([72.21.198.25]:25570 "EHLO smtp-fw-4101.amazon.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233820AbhELPPI (ORCPT ); Wed, 12 May 2021 11:15:08 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1620832441; x=1652368441; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version; bh=uGO4mCIQH4xEGCcMeem8ddAwwkcjIQxlPW+s+OxhsrI=; b=HcElgUvq2Cgy4KyLhz8Dtqf6CIG4LcmI769gUfW+69bNvq/6URr7ouIX djZZbYHeuAIDT7vCYQsXxxf3bmDNVV/OaX/+ZjtcpVG1fQQ9InITSrhNu cwsV4aK5cCJIKWvTx0u778GzBy/1eZ9c5VpW6T0ic6uqITOiQPHaK4Qne c=; X-IronPort-AV: E=Sophos;i="5.82,293,1613433600"; d="scan'208";a="107340911" Received: from iad12-co-svc-p1-lb1-vlan3.amazon.com (HELO email-inbound-relay-1e-c7c08562.us-east-1.amazon.com) ([10.43.8.6]) by smtp-border-fw-4101.iad4.amazon.com with ESMTP; 12 May 2021 15:13:51 +0000 Received: from EX13MTAUEB001.ant.amazon.com (iad12-ws-svc-p26-lb9-vlan2.iad.amazon.com [10.40.163.34]) by email-inbound-relay-1e-c7c08562.us-east-1.amazon.com (Postfix) with ESMTPS id 6B0612402B9; Wed, 12 May 2021 15:13:47 +0000 (UTC) Received: from EX13D08UEB003.ant.amazon.com (10.43.60.11) by EX13MTAUEB001.ant.amazon.com (10.43.60.129) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 12 May 2021 15:13:35 +0000 Received: from EX13MTAUEB002.ant.amazon.com (10.43.60.12) by EX13D08UEB003.ant.amazon.com (10.43.60.11) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 12 May 2021 15:13:34 +0000 Received: from uae075a0dfd4c51.ant.amazon.com (10.106.82.24) by mail-relay.amazon.com (10.43.60.234) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Wed, 12 May 2021 15:13:32 +0000 From: Ilias Stamatis To: , , CC: , , , , , , , , , Subject: [PATCH v2 10/10] KVM: selftests: x86: Add vmx_nested_tsc_scaling_test Date: Wed, 12 May 2021 16:09:45 +0100 Message-ID: <20210512150945.4591-11-ilstam@amazon.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210512150945.4591-1-ilstam@amazon.com> References: <20210512150945.4591-1-ilstam@amazon.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Test that nested TSC scaling works as expected with both L1 and L2 scaled. Signed-off-by: Ilias Stamatis --- tools/testing/selftests/kvm/.gitignore | 1 + tools/testing/selftests/kvm/Makefile | 1 + .../kvm/x86_64/vmx_nested_tsc_scaling_test.c | 242 ++++++++++++++++++ 3 files changed, 244 insertions(+) create mode 100644 tools/testing/selftests/kvm/x86_64/vmx_nested_tsc_scaling_test.c diff --git a/tools/testing/selftests/kvm/.gitignore b/tools/testing/selftests/kvm/.gitignore index bd83158e0e0b..cc02022f9951 100644 --- a/tools/testing/selftests/kvm/.gitignore +++ b/tools/testing/selftests/kvm/.gitignore @@ -29,6 +29,7 @@ /x86_64/vmx_preemption_timer_test /x86_64/vmx_set_nested_state_test /x86_64/vmx_tsc_adjust_test +/x86_64/vmx_nested_tsc_scaling_test /x86_64/xapic_ipi_test /x86_64/xen_shinfo_test /x86_64/xen_vmcall_test diff --git a/tools/testing/selftests/kvm/Makefile b/tools/testing/selftests/kvm/Makefile index e439d027939d..1078240b1313 100644 --- a/tools/testing/selftests/kvm/Makefile +++ b/tools/testing/selftests/kvm/Makefile @@ -60,6 +60,7 @@ TEST_GEN_PROGS_x86_64 += x86_64/vmx_close_while_nested_test TEST_GEN_PROGS_x86_64 += x86_64/vmx_dirty_log_test TEST_GEN_PROGS_x86_64 += x86_64/vmx_set_nested_state_test TEST_GEN_PROGS_x86_64 += x86_64/vmx_tsc_adjust_test +TEST_GEN_PROGS_x86_64 += x86_64/vmx_nested_tsc_scaling_test TEST_GEN_PROGS_x86_64 += x86_64/xapic_ipi_test TEST_GEN_PROGS_x86_64 += x86_64/xss_msr_test TEST_GEN_PROGS_x86_64 += x86_64/debug_regs diff --git a/tools/testing/selftests/kvm/x86_64/vmx_nested_tsc_scaling_test.c b/tools/testing/selftests/kvm/x86_64/vmx_nested_tsc_scaling_test.c new file mode 100644 index 000000000000..f4a8f426281a --- /dev/null +++ b/tools/testing/selftests/kvm/x86_64/vmx_nested_tsc_scaling_test.c @@ -0,0 +1,242 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * vmx_nested_tsc_scaling_test + * + * Copyright (C) 2021 Amazon.com, Inc. or its affiliates. + * + * This test case verifies that nested TSC scaling behaves as expected when + * both L1 and L2 are scaled using different ratios. For this test we scale + * L1 down and scale L2 up. + */ + +#include + +#include "kvm_util.h" +#include "vmx.h" +#include "kselftest.h" + + +#define VCPU_ID 0 + +/* L2 is scaled up (from L1's perspective) by this factor */ +#define L2_SCALE_FACTOR 4ULL + +#define TSC_OFFSET_L2 ((uint64_t) -33125236320908) +#define TSC_MULTIPLIER_L2 (L2_SCALE_FACTOR << 48) + +#define L2_GUEST_STACK_SIZE 64 + +enum { USLEEP, UCHECK_L1, UCHECK_L2 }; +#define GUEST_SLEEP(sec) ucall(UCALL_SYNC, 2, USLEEP, sec) +#define GUEST_CHECK(level, freq) ucall(UCALL_SYNC, 2, level, freq) + + +/* + * This function checks whether the "actual" TSC frequency of a guest matches + * its expected frequency. In order to account for delays in taking the TSC + * measurements, a difference of 1% between the actual and the expected value + * is tolerated. + */ +static void compare_tsc_freq(uint64_t actual, uint64_t expected) +{ + uint64_t tolerance, thresh_low, thresh_high; + + tolerance = expected / 100; + thresh_low = expected - tolerance; + thresh_high = expected + tolerance; + + TEST_ASSERT(thresh_low < actual, + "TSC freq is expected to be between %"PRIu64" and %"PRIu64 + " but it actually is %"PRIu64, + thresh_low, thresh_high, actual); + TEST_ASSERT(thresh_high > actual, + "TSC freq is expected to be between %"PRIu64" and %"PRIu64 + " but it actually is %"PRIu64, + thresh_low, thresh_high, actual); +} + +static void check_tsc_freq(int level) +{ + uint64_t tsc_start, tsc_end, tsc_freq; + + /* + * Reading the TSC twice with about a second's difference should give + * us an approximation of the TSC frequency from the guest's + * perspective. Now, this won't be completely accurate, but it should + * be good enough for the purposes of this test. + */ + tsc_start = rdmsr(MSR_IA32_TSC); + GUEST_SLEEP(1); + tsc_end = rdmsr(MSR_IA32_TSC); + + tsc_freq = tsc_end - tsc_start; + + GUEST_CHECK(level, tsc_freq); +} + +static void l2_guest_code(void) +{ + check_tsc_freq(UCHECK_L2); + + /* exit to L1 */ + __asm__ __volatile__("vmcall"); +} + +static void l1_guest_code(struct vmx_pages *vmx_pages) +{ + unsigned long l2_guest_stack[L2_GUEST_STACK_SIZE]; + uint32_t control; + + /* check that L1's frequency looks alright before launching L2 */ + check_tsc_freq(UCHECK_L1); + + GUEST_ASSERT(prepare_for_vmx_operation(vmx_pages)); + GUEST_ASSERT(load_vmcs(vmx_pages)); + + /* prepare the VMCS for L2 execution */ + prepare_vmcs(vmx_pages, l2_guest_code, &l2_guest_stack[L2_GUEST_STACK_SIZE]); + + /* enable TSC offsetting and TSC scaling for L2 */ + control = vmreadz(CPU_BASED_VM_EXEC_CONTROL); + control |= CPU_BASED_USE_MSR_BITMAPS | CPU_BASED_USE_TSC_OFFSETTING; + vmwrite(CPU_BASED_VM_EXEC_CONTROL, control); + + control = vmreadz(SECONDARY_VM_EXEC_CONTROL); + control |= SECONDARY_EXEC_TSC_SCALING; + vmwrite(SECONDARY_VM_EXEC_CONTROL, control); + + vmwrite(TSC_OFFSET, TSC_OFFSET_L2); + vmwrite(TSC_MULTIPLIER, TSC_MULTIPLIER_L2); + vmwrite(TSC_MULTIPLIER_HIGH, TSC_MULTIPLIER_L2 >> 32); + + /* launch L2 */ + GUEST_ASSERT(!vmlaunch()); + GUEST_ASSERT(vmreadz(VM_EXIT_REASON) == EXIT_REASON_VMCALL); + + /* check that L1's frequency still looks good */ + check_tsc_freq(UCHECK_L1); + + GUEST_DONE(); +} + +static void tsc_scaling_check_supported(void) +{ + if (!kvm_check_cap(KVM_CAP_TSC_CONTROL)) { + print_skip("TSC scaling not supported by the HW"); + exit(KSFT_SKIP); + } +} + +static void stable_tsc_check_supported(void) +{ + FILE *fp; + char buf[4]; + + fp = fopen("/sys/devices/system/clocksource/clocksource0/current_clocksource", "r"); + if (fp == NULL) + goto skip_test; + + if (fgets(buf, sizeof(buf), fp) == NULL) + goto skip_test; + + if (strncmp(buf, "tsc", sizeof(buf))) + goto skip_test; + + return; +skip_test: + print_skip("TSC is not stable"); + exit(KSFT_SKIP); +} + +int main(int argc, char *argv[]) +{ + struct kvm_vm *vm; + vm_vaddr_t vmx_pages_gva; + + uint64_t tsc_start, tsc_end; + uint64_t tsc_khz; + uint64_t l1_scale_factor; + uint64_t l0_tsc_freq = 0; + uint64_t l1_tsc_freq = 0; + uint64_t l2_tsc_freq = 0; + + nested_vmx_check_supported(); + tsc_scaling_check_supported(); + stable_tsc_check_supported(); + + /* + * We set L1's scale factor to be a random number from 2 to 10. + * Ideally we would do the same for L2's factor but that one is + * referenced by both main() and l1_guest_code() and using a global + * variable does not work. + */ + srand(time(NULL)); + l1_scale_factor = (rand() % 9) + 2; + printf("L1's scale factor is: %"PRIu64"\n", l1_scale_factor); + printf("L2's scale factor is: %llu\n", L2_SCALE_FACTOR); + + tsc_start = rdtsc(); + sleep(1); + tsc_end = rdtsc(); + + l0_tsc_freq = tsc_end - tsc_start; + printf("real TSC frequency is around: %"PRIu64"\n", l0_tsc_freq); + + vm = vm_create_default(VCPU_ID, 0, (void *) l1_guest_code); + vcpu_alloc_vmx(vm, &vmx_pages_gva); + vcpu_args_set(vm, VCPU_ID, 1, vmx_pages_gva); + + tsc_khz = _vcpu_ioctl(vm, VCPU_ID, KVM_GET_TSC_KHZ, NULL); + TEST_ASSERT(tsc_khz != -1, "vcpu ioctl KVM_GET_TSC_KHZ failed"); + + /* scale down L1's TSC frequency */ + vcpu_ioctl(vm, VCPU_ID, KVM_SET_TSC_KHZ, + (void *) (tsc_khz / l1_scale_factor)); + + for (;;) { + volatile struct kvm_run *run = vcpu_state(vm, VCPU_ID); + struct ucall uc; + + vcpu_run(vm, VCPU_ID); + TEST_ASSERT(run->exit_reason == KVM_EXIT_IO, + "Got exit_reason other than KVM_EXIT_IO: %u (%s)\n", + run->exit_reason, + exit_reason_str(run->exit_reason)); + + switch (get_ucall(vm, VCPU_ID, &uc)) { + case UCALL_ABORT: + TEST_FAIL("%s", (const char *) uc.args[0]); + case UCALL_SYNC: + switch (uc.args[0]) { + case USLEEP: + sleep(uc.args[1]); + break; + case UCHECK_L1: + l1_tsc_freq = uc.args[1]; + printf("L1's TSC frequency is around: %"PRIu64 + "\n", l1_tsc_freq); + + compare_tsc_freq(l1_tsc_freq, + l0_tsc_freq / l1_scale_factor); + break; + case UCHECK_L2: + l2_tsc_freq = uc.args[1]; + printf("L2's TSC frequency is around: %"PRIu64 + "\n", l2_tsc_freq); + + compare_tsc_freq(l2_tsc_freq, + l1_tsc_freq * L2_SCALE_FACTOR); + break; + } + break; + case UCALL_DONE: + goto done; + default: + TEST_FAIL("Unknown ucall %lu", uc.cmd); + } + } + +done: + kvm_vm_free(vm); + return 0; +}