From patchwork Thu Sep 24 03:29:01 2009 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zachary Amsden X-Patchwork-Id: 49736 Received: from vger.kernel.org (vger.kernel.org [209.132.176.167]) by demeter.kernel.org (8.14.2/8.14.2) with ESMTP id n8O3WlLr024591 for ; Thu, 24 Sep 2009 03:32:48 GMT Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753060AbZIXDcU (ORCPT ); Wed, 23 Sep 2009 23:32:20 -0400 Received: (majordomo@vger.kernel.org) by vger.kernel.org id S1752715AbZIXDcT (ORCPT ); Wed, 23 Sep 2009 23:32:19 -0400 Received: from mx1.redhat.com ([209.132.183.28]:17240 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752712AbZIXDbW (ORCPT ); Wed, 23 Sep 2009 23:31:22 -0400 Received: from int-mx03.intmail.prod.int.phx2.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.16]) by mx1.redhat.com (8.13.8/8.13.8) with ESMTP id n8O3VQSq031141; Wed, 23 Sep 2009 23:31:26 -0400 Received: from localhost.localdomain (vpn-12-112.rdu.redhat.com [10.11.12.112]) by int-mx03.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id n8O3VLPu019943; Wed, 23 Sep 2009 23:31:24 -0400 From: Zachary Amsden To: kvm@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Zachary Amsden , Avi Kivity , Marcelo Tosatti Subject: [PATCH: kvm 2/6] Kill the confusing tsc_ref_khz and ref_freq variables. Date: Wed, 23 Sep 2009 17:29:01 -1000 Message-Id: <1253762945-5750-2-git-send-email-zamsden@redhat.com> In-Reply-To: <1253762945-5750-1-git-send-email-zamsden@redhat.com> References: <1253762945-5750-1-git-send-email-zamsden@redhat.com> Organization: Frobozz Magic Timekeeping Company X-Scanned-By: MIMEDefang 2.67 on 10.5.11.16 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org They are globals, not clearly protected by any ordering or locking, and vulnerable to various startup races. Instead, for variable TSC machines, register the cpufreq notifier and get the TSC frequency directly from the cpufreq machinery. Not only is it always right, it is also perfectly accurate, as no error prone measurement is required. On such machines, also detect the frequency when bringing a new CPU online; it isn't clear what frequency it will start with, and it may not correspond to the reference. Signed-off-by: Zachary Amsden --- arch/x86/kvm/x86.c | 38 ++++++++++++++++++++++++++++---------- 1 files changed, 28 insertions(+), 10 deletions(-) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 15d2ace..35082dd 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -650,6 +650,19 @@ static void kvm_set_time_scale(uint32_t tsc_khz, struct pvclock_vcpu_time_info * static DEFINE_PER_CPU(unsigned long, cpu_tsc_khz); +static inline void kvm_get_cpu_khz(int cpu) +{ + unsigned int khz = cpufreq_get(cpu); + if (khz <= 0) { + static int warn = 0; + if (warn++ < 10) + printk(KERN_ERR "kvm: could not get frequency for CPU " + "%d. Timers may be inaccurate\n", cpu); + khz = tsc_khz; + } + per_cpu(cpu_tsc_khz, cpu) = khz; +} + static void kvm_write_guest_time(struct kvm_vcpu *v) { struct timespec ts; @@ -3061,9 +3074,6 @@ static void bounce_off(void *info) /* nothing */ } -static unsigned int ref_freq; -static unsigned long tsc_khz_ref; - static int kvmclock_cpufreq_notifier(struct notifier_block *nb, unsigned long val, void *data) { @@ -3071,15 +3081,15 @@ static int kvmclock_cpufreq_notifier(struct notifier_block *nb, unsigned long va struct kvm *kvm; struct kvm_vcpu *vcpu; int i, send_ipi = 0; - - if (!ref_freq) - ref_freq = freq->old; + unsigned long old_khz; if (val == CPUFREQ_PRECHANGE && freq->old > freq->new) return 0; if (val == CPUFREQ_POSTCHANGE && freq->old < freq->new) return 0; - per_cpu(cpu_tsc_khz, freq->cpu) = cpufreq_scale(tsc_khz_ref, ref_freq, freq->new); + old_khz = per_cpu(cpu_tsc_khz, freq->cpu); + per_cpu(cpu_tsc_khz, freq->cpu) = cpufreq_scale(old_khz, freq->old, + freq->new); spin_lock(&kvm_lock); list_for_each_entry(kvm, &vm_list, vm_list) { @@ -3120,12 +3130,18 @@ static void kvm_timer_init(void) { int cpu; - for_each_possible_cpu(cpu) - per_cpu(cpu_tsc_khz, cpu) = tsc_khz; if (!boot_cpu_has(X86_FEATURE_CONSTANT_TSC)) { - tsc_khz_ref = tsc_khz; cpufreq_register_notifier(&kvmclock_cpufreq_notifier_block, CPUFREQ_TRANSITION_NOTIFIER); + for_each_online_cpu(cpu) + kvm_get_cpu_khz(cpu); + } else { + for_each_possible_cpu(cpu) + per_cpu(cpu_tsc_khz, cpu) = tsc_khz; + } + for_each_possible_cpu(cpu) { + printk(KERN_DEBUG "kvm: cpu %d = %ld khz\n", + cpu, per_cpu(cpu_tsc_khz, cpu)); } } @@ -4698,6 +4714,8 @@ int kvm_arch_vcpu_reset(struct kvm_vcpu *vcpu) int kvm_arch_hardware_enable(void *garbage) { + if (!boot_cpu_has(X86_FEATURE_CONSTANT_TSC)) + kvm_get_cpu_khz(raw_smp_processor_id()); return kvm_x86_ops->hardware_enable(garbage); }