From patchwork Fri Jan 20 01:11:17 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marcelo Tosatti X-Patchwork-Id: 13109142 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C424DC677F1 for ; Fri, 20 Jan 2023 04:51:15 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pIjMC-0005iV-Vr; Thu, 19 Jan 2023 23:50:29 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pIjM7-0005h7-G2 for qemu-devel@nongnu.org; Thu, 19 Jan 2023 23:50:25 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pIjM5-0005WS-4N for qemu-devel@nongnu.org; Thu, 19 Jan 2023 23:50:23 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1674190218; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=QCQ/rJ9OcUhFGJfEZRwZPT6ketogptymaLeWaE9tJ3Y=; b=V3bYoWmMd99RF/wp1eimQsxV/F5nnSrCAArn/PYgXhwe76MsmcUK2uC0XJChtyJFqWsFax 7S+9zoIs4YqB6mUK5+vQ1m1zemtFTlNHwGVwGXRhKKJ5wt4X3jC5f1TuM9txw3PLnbA2Hr g2rfUoqZh46WkVVCyb/mtvRkAncGzlw= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-346-1motH31IPoihUWJxAXRf3A-1; Thu, 19 Jan 2023 23:48:50 -0500 X-MC-Unique: 1motH31IPoihUWJxAXRf3A-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 16BAD1C0512B; Fri, 20 Jan 2023 04:48:50 +0000 (UTC) Received: from tpad.localdomain (ovpn-112-2.gru2.redhat.com [10.97.112.2]) by smtp.corp.redhat.com (Postfix) with ESMTPS id E2AF840C6EC4; Fri, 20 Jan 2023 04:48:49 +0000 (UTC) Received: by tpad.localdomain (Postfix, from userid 1000) id 002204099C362; Thu, 19 Jan 2023 22:15:17 -0300 (-03) Message-ID: <20230120011412.530500343@redhat.com> User-Agent: quilt/0.67 Date: Thu, 19 Jan 2023 22:11:17 -0300 From: Marcelo Tosatti To: qemu-devel@nongnu.org Cc: Paolo Bonzini , kvm@vger.kernel.org, Marcelo Tosatti Subject: [PATCH 1/2] linux-headers: sync KVM_CLOCK_CORRECT_TSC_SHIFT flag References: <20230120011116.134437211@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.2 Received-SPF: pass client-ip=170.10.133.124; envelope-from=mtosatti@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sync new KVM_CLOCK_CORRECT_TSC_SHIFT from upstream Linux headers. Signed-off-by: Marcelo Tosatti Index: qemu/linux-headers/linux/kvm.h =================================================================== --- qemu.orig/linux-headers/linux/kvm.h +++ qemu/linux-headers/linux/kvm.h @@ -1300,6 +1300,9 @@ struct kvm_irqfd { #define KVM_CLOCK_TSC_STABLE 2 #define KVM_CLOCK_REALTIME (1 << 2) #define KVM_CLOCK_HOST_TSC (1 << 3) +/* whether tsc_shift as seen by the guest matches guest visible TSC */ +/* This is true since commit 78db6a5037965429c04d708281f35a6e5562d31b */ +#define KVM_CLOCK_CORRECT_TSC_SHIFT (1 << 4) struct kvm_clock_data { __u64 clock; From patchwork Fri Jan 20 01:11:18 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marcelo Tosatti X-Patchwork-Id: 13109143 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 75DF3C38159 for ; Fri, 20 Jan 2023 04:51:24 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pIjMm-0005sh-Vm; Thu, 19 Jan 2023 23:51:05 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pIjMl-0005sR-NP for qemu-devel@nongnu.org; Thu, 19 Jan 2023 23:51:03 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pIjMk-0005g2-1J for qemu-devel@nongnu.org; Thu, 19 Jan 2023 23:51:03 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1674190261; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: references:references; bh=+LRagk6PL+VCh4WXT1/1N+QScv/LuUh/E9vGhjKrpng=; b=dHjAbNUTBt/NKfRA7QxqAY9WD7oXli+peNBCFiVdnXvktRnF7TSf61L2FmW5s/vPHyXX++ 6Niv79r14ogUB1ku/s4+GLgZtwO7+3Tg7nbMRoL0A+8ZD2yokCSWlTru5MERwZdv5r4YIz hfCMewkCdg0n2qg+A/1tGBC3eAMA40A= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-346-Pxq9fQkDMlqlGtbUtn9NVA-1; Thu, 19 Jan 2023 23:48:50 -0500 X-MC-Unique: Pxq9fQkDMlqlGtbUtn9NVA-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 407E485CBE1; Fri, 20 Jan 2023 04:48:50 +0000 (UTC) Received: from tpad.localdomain (ovpn-112-2.gru2.redhat.com [10.97.112.2]) by smtp.corp.redhat.com (Postfix) with ESMTPS id E2C711121315; Fri, 20 Jan 2023 04:48:49 +0000 (UTC) Received: by tpad.localdomain (Postfix, from userid 1000) id 0A2DE4099C363; Thu, 19 Jan 2023 22:15:17 -0300 (-03) Message-ID: <20230120011412.558538345@redhat.com> User-Agent: quilt/0.67 Date: Thu, 19 Jan 2023 22:11:18 -0300 From: Marcelo Tosatti To: qemu-devel@nongnu.org Cc: Paolo Bonzini , kvm@vger.kernel.org, Marcelo Tosatti Subject: [PATCH 2/2] hw/i386/kvm/clock.c: read kvmclock from guest memory if !correct_tsc_shift References: <20230120011116.134437211@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.3 Received-SPF: pass client-ip=170.10.133.124; envelope-from=mtosatti@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Before kernel commit 78db6a5037965429c04d708281f35a6e5562d31b, kvm_guest_time_update() would use vcpu->virtual_tsc_khz to calculate tsc_shift value in the vcpus pvclock structure written to guest memory. For those kernels, if vcpu->virtual_tsc_khz != tsc_khz (which can be the case when guest state is restored via migration, or if tsc-khz option is passed to QEMU), and TSC scaling is not enabled (which happens if the difference between the frequency requested via KVM_SET_TSC_KHZ and the host TSC KHZ is smaller than 250ppm), then there can be a difference between what KVM_GET_CLOCK would return and what the guest reads as kvmclock value. The effect is that the guest sees a jump in kvmclock value (either forwards or backwards) in such case. To fix incoming migration from pre-78db6a5037965 hosts, read kvmclock value from guest memory. Unless the KVM_CLOCK_CORRECT_TSC_SHIFT bit indicates that the value retrieved by KVM_GET_CLOCK on the source is safe to be used. Signed-off-by: Marcelo Tosatti Index: qemu/hw/i386/kvm/clock.c =================================================================== --- qemu.orig/hw/i386/kvm/clock.c +++ qemu/hw/i386/kvm/clock.c @@ -50,6 +50,16 @@ struct KVMClockState { /* whether the 'clock' value was obtained in a host with * reliable KVM_GET_CLOCK */ bool clock_is_reliable; + + /* whether machine type supports correct_tsc_shift */ + bool mach_use_correct_tsc_shift; + + /* + * whether the 'clock' value was obtained in a host + * that computes correct tsc_shift field (the one + * written to guest memory) + */ + bool clock_correct_tsc_shift; }; struct pvclock_vcpu_time_info { @@ -150,6 +160,8 @@ static void kvm_update_clock(KVMClockSta * read from memory */ s->clock_is_reliable = kvm_has_adjust_clock_stable(); + + s->clock_correct_tsc_shift = kvm_has_correct_tsc_shift(); } static void do_kvmclock_ctrl(CPUState *cpu, run_on_cpu_data data) @@ -176,7 +188,7 @@ static void kvmclock_vm_state_change(voi * If the host where s->clock was read did not support reliable * KVM_GET_CLOCK, read kvmclock value from memory. */ - if (!s->clock_is_reliable) { + if (!s->clock_is_reliable || !s->clock_correct_tsc_shift) { uint64_t pvclock_via_mem = kvmclock_current_nsec(s); /* We can't rely on the saved clock value, just discard it */ if (pvclock_via_mem) { @@ -252,14 +264,40 @@ static const VMStateDescription kvmclock }; /* + * Sending clock_correct_tsc_shift=true means that the destination + * can use VMSTATE_UINT64(clock, KVMClockState) value, + * instead of reading from guest memory. + */ +static bool kvmclock_clock_correct_tsc_shift_needed(void *opaque) +{ + KVMClockState *s = opaque; + + return s->mach_use_correct_tsc_shift; +} + +static const VMStateDescription kvmclock_correct_tsc_shift = { + .name = "kvmclock/clock_correct_tsc_shift", + .version_id = 1, + .minimum_version_id = 1, + .needed = kvmclock_clock_correct_tsc_shift_needed, + .fields = (VMStateField[]) { + VMSTATE_BOOL(clock_correct_tsc_shift, KVMClockState), + VMSTATE_END_OF_LIST() + } +}; + +/* * When migrating, assume the source has an unreliable - * KVM_GET_CLOCK unless told otherwise. + * KVM_GET_CLOCK (and computes tsc shift + * in guest memory using vcpu->virtual_tsc_khz), + * unless told otherwise. */ static int kvmclock_pre_load(void *opaque) { KVMClockState *s = opaque; s->clock_is_reliable = false; + s->clock_correct_tsc_shift = false; return 0; } @@ -301,6 +339,7 @@ static const VMStateDescription kvmclock }, .subsections = (const VMStateDescription * []) { &kvmclock_reliable_get_clock, + &kvmclock_correct_tsc_shift, NULL } }; @@ -308,6 +347,8 @@ static const VMStateDescription kvmclock static Property kvmclock_properties[] = { DEFINE_PROP_BOOL("x-mach-use-reliable-get-clock", KVMClockState, mach_use_reliable_get_clock, true), + DEFINE_PROP_BOOL("x-mach-use-correct-tsc-shift", KVMClockState, + mach_use_correct_tsc_shift, true), DEFINE_PROP_END_OF_LIST(), }; Index: qemu/target/i386/kvm/kvm.c =================================================================== --- qemu.orig/target/i386/kvm/kvm.c +++ qemu/target/i386/kvm/kvm.c @@ -164,6 +164,13 @@ bool kvm_has_adjust_clock_stable(void) return (ret & KVM_CLOCK_TSC_STABLE); } +bool kvm_has_correct_tsc_shift(void) +{ + int ret = kvm_check_extension(kvm_state, KVM_CAP_ADJUST_CLOCK); + + return ret & KVM_CLOCK_CORRECT_TSC_SHIFT; +} + bool kvm_has_adjust_clock(void) { return kvm_check_extension(kvm_state, KVM_CAP_ADJUST_CLOCK); Index: qemu/target/i386/kvm/kvm_i386.h =================================================================== --- qemu.orig/target/i386/kvm/kvm_i386.h +++ qemu/target/i386/kvm/kvm_i386.h @@ -35,6 +35,7 @@ bool kvm_has_smm(void); bool kvm_has_adjust_clock(void); bool kvm_has_adjust_clock_stable(void); +bool kvm_has_correct_tsc_shift(void); bool kvm_has_exception_payload(void); void kvm_synchronize_all_tsc(void); void kvm_arch_reset_vcpu(X86CPU *cs);