From patchwork Mon May 6 18:35:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Reinette Chatre X-Patchwork-Id: 13655890 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.9]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8C1B013E8BA; Mon, 6 May 2024 18:36:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.9 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715020586; cv=none; b=qFO32/aZMUZqMDs4edVxmS2GSWEdqThQ1N09idqoeQb4FDnxSJjG2TR+sfWNaGVkU97cVGzAY2LPl5tKTGXeWNIfMh3f3i8naBjFkaTZjCd1GJszwwn7J72YMYVW68oJd1dMHWis6LfcQB5g8QqRqVWoKBUboZRZFR24jrklQUI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715020586; c=relaxed/simple; bh=JCMdLmx7hIlCtOWjHewphvGoWCSoaO1qN1Iyo0V0rtg=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=P/VM63ooTNJDRfxb1QtVGalRRE364RlInmva6CWASj0b2REvMEi3mlAo5zRcijC0GeZ75CKk/ZOBtOGJAslF+n/yzBf29IdbF7s+yQtBh0K2Y4wfmCcK8+3DyBX1T1/aGJmqlo/O/AK9jKxOYimzUZJFTKqx/EosK4wDrjhPKGU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=Eb6HrA0+; arc=none smtp.client-ip=192.198.163.9 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="Eb6HrA0+" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1715020585; x=1746556585; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=JCMdLmx7hIlCtOWjHewphvGoWCSoaO1qN1Iyo0V0rtg=; b=Eb6HrA0+IVyp2sJhunKku6pJQbRR8xrl6Zgg7zCxqAwO5heq2Acf35QT wHnNLZ3LCkXhoIVTSlKu7agTZW95+2DvlaloTIAiCwJ1XeMya/FrS0CGc FlYUEu6Rrt48ogIfaq3CUkO/TmyodNYDyAxeJyjYelKjmJ5SdMa+EyMxv ymPfugqjKoqzAIQP9bvOEWP6ksEpdi7MQ3Z9PDbnxsID3pa/21J5SdbRg JQVBXhAaANKSnlxdALmVsaq4Fgll4CBlkOiO3IKP9Awih/tXxCyG/1DkC BegSd5YHSZZbjiE1MShVUyaSDTz5yAygonYBDPGd55v1hVdVj3jdPOb+E w==; X-CSE-ConnectionGUID: N5Fd4vAISRSm7VWeB3aOkA== X-CSE-MsgGUID: aKYfLx+TTbuy/Lgx8zaQ9w== X-IronPort-AV: E=McAfee;i="6600,9927,11065"; a="21455743" X-IronPort-AV: E=Sophos;i="6.07,259,1708416000"; d="scan'208";a="21455743" Received: from fmviesa008.fm.intel.com ([10.60.135.148]) by fmvoesa103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 May 2024 11:36:10 -0700 X-CSE-ConnectionGUID: Dl22qp/lSiOB0TYBDYy6EA== X-CSE-MsgGUID: bStAaS47SbybSrCM0oz2qw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.07,259,1708416000"; d="scan'208";a="28237805" Received: from rchatre-ws.ostc.intel.com ([10.54.69.144]) by fmviesa008-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 May 2024 11:36:09 -0700 From: Reinette Chatre To: isaku.yamahata@intel.com, pbonzini@redhat.com, erdemaktas@google.com, vkuznets@redhat.com, seanjc@google.com, vannapurve@google.com, jmattson@google.com, mlevitsk@redhat.com, xiaoyao.li@intel.com, chao.gao@intel.com, rick.p.edgecombe@intel.com, yuan.yao@intel.com Cc: reinette.chatre@intel.com, kvm@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH V6 1/4] KVM: x86: hyper-v: Calculate APIC bus frequency for Hyper-V Date: Mon, 6 May 2024 11:35:55 -0700 Message-Id: X-Mailer: git-send-email 2.34.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata Remove APIC_BUS_FREQUENCY and calculate it based on nanoseconds per APIC bus cycle. APIC_BUS_FREQUENCY is used only for HV_X64_MSR_APIC_FREQUENCY. The MSR is not frequently read, calculate it every time. There are two constants related to the APIC bus frequency: APIC_BUS_FREQUENCY and APIC_BUS_CYCLE_NS. Only one value is required because one can be calculated from the other: APIC_BUS_CYCLES_NS = 1000 * 1000 * 1000 / APIC_BUS_FREQUENCY. Remove APIC_BUS_FREQUENCY and instead calculate it when needed. This prepares for support of configurable APIC bus frequency by requiring to change only a single variable. Suggested-by: Maxim Levitsky Signed-off-by: Isaku Yamahata Reviewed-by: Maxim Levitsky Reviewed-by: Xiaoyao Li Reviewed-by: Rick Edgecombe [reinette: rework changelog] Signed-off-by: Reinette Chatre --- Changes v5: - Add Rick's Reviewed-by tag. Changes v4: - Modify subject to match custom: "KVM: x86/hyperv:" -> "KVM: x86: hyper-v:" and use standard capitalization for Hyper-V ("hyper-v" -> "Hyper-V"). - Rework changelog to remove pronouns ("we"). - Change logic in commit log: "APIC bus cycles per NS" -> "nanoseconds per APIC bus cycle". - Fix typos. (Maxim and Xiaoyao) - Add Maxim and Xiaoyao's Reviewed-by tags. - Add Maxim's "Suggested-by". Changes v3: - Newly added according to Maxim Levitsky suggestion. --- arch/x86/kvm/hyperv.c | 2 +- arch/x86/kvm/lapic.h | 1 - 2 files changed, 1 insertion(+), 2 deletions(-) diff --git a/arch/x86/kvm/hyperv.c b/arch/x86/kvm/hyperv.c index 8a47f8541eab..1030701db967 100644 --- a/arch/x86/kvm/hyperv.c +++ b/arch/x86/kvm/hyperv.c @@ -1737,7 +1737,7 @@ static int kvm_hv_get_msr(struct kvm_vcpu *vcpu, u32 msr, u64 *pdata, data = (u64)vcpu->arch.virtual_tsc_khz * 1000; break; case HV_X64_MSR_APIC_FREQUENCY: - data = APIC_BUS_FREQUENCY; + data = div64_u64(1000000000ULL, APIC_BUS_CYCLE_NS); break; default: kvm_pr_unimpl_rdmsr(vcpu, msr); diff --git a/arch/x86/kvm/lapic.h b/arch/x86/kvm/lapic.h index 0a0ea4b5dd8c..a20cb006b6c8 100644 --- a/arch/x86/kvm/lapic.h +++ b/arch/x86/kvm/lapic.h @@ -17,7 +17,6 @@ #define APIC_DEST_MASK 0x800 #define APIC_BUS_CYCLE_NS 1 -#define APIC_BUS_FREQUENCY (1000000000ULL / APIC_BUS_CYCLE_NS) #define APIC_BROADCAST 0xFF #define X2APIC_BROADCAST 0xFFFFFFFFul From patchwork Mon May 6 18:35:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Reinette Chatre X-Patchwork-Id: 13655891 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.9]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 771F6158A07; Mon, 6 May 2024 18:36:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.9 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715020587; cv=none; b=ag64HB9vuS4VhJfNf1PMHhHu62AUB7kf12Y32SMs5ZPwHGpVW91lVtfBVV3o2v36Aea9YcYSn4o3v7DfIktomOwWnfyyLkQPsvwhGU6Md7icEpdAFxBY9B3a3ZOyUgYi2Vzoi3A3pECDYbt4sPVWFIQOeROfPVJyIssiPeT20fw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715020587; c=relaxed/simple; bh=aLUvoztgCdvkMpFl2d8hvMMLL/FRvtTxcIQ0ugO2uR0=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version:Content-Type; b=L1hVGuQURVvgoGbr1fnQgr1RbEuygmJj/Apsg0YrpRFnZYmjEfFMOLgEeAYHPciU2sF3nMAF20yot7jBqQ7KdCncIkGDA5PQEcVtqYD7oJXq+QQTb/fKOk35QYLZq0tgU80q/OJEZAlUNDxyf1bsI/TWHKWQDQLxfgHruIyf/So= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=A3mnZ5BR; arc=none smtp.client-ip=192.198.163.9 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="A3mnZ5BR" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1715020586; x=1746556586; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=aLUvoztgCdvkMpFl2d8hvMMLL/FRvtTxcIQ0ugO2uR0=; b=A3mnZ5BR9t+euNKNg3m/4zoaG4ib99wTupD95Q9pIilKemsrPkoTC0Ry SzGwWNm+qlDHAk+v5muhob3BBpxMc5BZKuHs6rAk5zusXt2H12O7o0Jpa /+N1dnGwkw7VnINdzVrHeQifq9oxVDQNVvOfBxF9ZDW/JzbtH5jjaXkvx Mb2V0WLndDnJUAQA5B6uP8Yna6YaOP8znbQWFFA8exrNsP8CmYNJa0hHT vQUTHmT/ChZXB9x0M11YEwdrztc89Li6uscEQx/FYQxOyxSkQaA0aov6+ /8FNF6zZZIYxh9yRKCv5IJBKGJxELFkwIOhQh+l4df9Lo+VoiQjkbDIkK g==; X-CSE-ConnectionGUID: q0GeNK5uTUiya47dv8Qv8g== X-CSE-MsgGUID: NqGuVTysTP2/nlG9As3eoQ== X-IronPort-AV: E=McAfee;i="6600,9927,11065"; a="21455746" X-IronPort-AV: E=Sophos;i="6.07,259,1708416000"; d="scan'208";a="21455746" Received: from fmviesa008.fm.intel.com ([10.60.135.148]) by fmvoesa103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 May 2024 11:36:10 -0700 X-CSE-ConnectionGUID: q31vWtKTRXWl5KWf7MUbEQ== X-CSE-MsgGUID: l5L7qQaYRMK3Vm8yqeCU2w== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.07,259,1708416000"; d="scan'208";a="28237809" Received: from rchatre-ws.ostc.intel.com ([10.54.69.144]) by fmviesa008-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 May 2024 11:36:10 -0700 From: Reinette Chatre To: isaku.yamahata@intel.com, pbonzini@redhat.com, erdemaktas@google.com, vkuznets@redhat.com, seanjc@google.com, vannapurve@google.com, jmattson@google.com, mlevitsk@redhat.com, xiaoyao.li@intel.com, chao.gao@intel.com, rick.p.edgecombe@intel.com, yuan.yao@intel.com Cc: reinette.chatre@intel.com, kvm@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH V6 2/4] KVM: x86: Make nsec per APIC bus cycle a VM variable Date: Mon, 6 May 2024 11:35:56 -0700 Message-Id: <51b8612bf2942d06b8d879aaede17b35b2d1814c.1715017765.git.reinette.chatre@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata Introduce the VM variable "nanoseconds per APIC bus cycle" in preparation to make the APIC bus frequency configurable. The TDX architecture hard-codes the core crystal clock frequency to 25MHz and mandates exposing it via CPUID leaf 0x15. The TDX architecture does not allow the VMM to override the value. In addition, per Intel SDM: "The APIC timer frequency will be the processor’s bus clock or core crystal clock frequency (when TSC/core crystal clock ratio is enumerated in CPUID leaf 0x15) divided by the value specified in the divide configuration register." The resulting 25MHz APIC bus frequency conflicts with the KVM hardcoded APIC bus frequency of 1GHz. Introduce the VM variable "nanoseconds per APIC bus cycle" to prepare for allowing userspace to tell KVM to use the frequency that TDX mandates instead of the default 1Ghz. Doing so ensures that the guest doesn't have a conflicting view of the APIC bus frequency. Signed-off-by: Isaku Yamahata Reviewed-by: Maxim Levitsky Reviewed-by: Rick Edgecombe Reviewed-by: Xiaoyao Li [reinette: rework changelog] Signed-off-by: Reinette Chatre --- Changes v6: - Add Xiaoyao Li's Reviewed-by tag. Changes v5: - Add Rick's Reviewed-by tag. Changes v4: - Reword changelog to address comments related to "bus clock" vs "core crystal clock" frequency. (Xiaoyao) - Typo in changelog ("APIC APIC" -> "APIC"). - Change logic "APIC bus cycles per nsec" -> "nanoseconds per APIC bus cycle". Changes V3: - Update commit message. - Dropped apic_bus_frequency according to Maxim Levitsky. Changes v2: - No change. --- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/hyperv.c | 3 ++- arch/x86/kvm/lapic.c | 6 ++++-- arch/x86/kvm/lapic.h | 2 +- arch/x86/kvm/x86.c | 1 + 5 files changed, 9 insertions(+), 4 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 1d13e3cd1dc5..f2735582c7e0 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1358,6 +1358,7 @@ struct kvm_arch { u32 default_tsc_khz; bool user_set_tsc; + u64 apic_bus_cycle_ns; seqcount_raw_spinlock_t pvclock_sc; bool use_master_clock; diff --git a/arch/x86/kvm/hyperv.c b/arch/x86/kvm/hyperv.c index 1030701db967..5c31e715d2ad 100644 --- a/arch/x86/kvm/hyperv.c +++ b/arch/x86/kvm/hyperv.c @@ -1737,7 +1737,8 @@ static int kvm_hv_get_msr(struct kvm_vcpu *vcpu, u32 msr, u64 *pdata, data = (u64)vcpu->arch.virtual_tsc_khz * 1000; break; case HV_X64_MSR_APIC_FREQUENCY: - data = div64_u64(1000000000ULL, APIC_BUS_CYCLE_NS); + data = div64_u64(1000000000ULL, + vcpu->kvm->arch.apic_bus_cycle_ns); break; default: kvm_pr_unimpl_rdmsr(vcpu, msr); diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c index cf37586f0466..3e66a0a95999 100644 --- a/arch/x86/kvm/lapic.c +++ b/arch/x86/kvm/lapic.c @@ -1547,7 +1547,8 @@ static u32 apic_get_tmcct(struct kvm_lapic *apic) remaining = 0; ns = mod_64(ktime_to_ns(remaining), apic->lapic_timer.period); - return div64_u64(ns, (APIC_BUS_CYCLE_NS * apic->divide_count)); + return div64_u64(ns, (apic->vcpu->kvm->arch.apic_bus_cycle_ns * + apic->divide_count)); } static void __report_tpr_access(struct kvm_lapic *apic, bool write) @@ -1965,7 +1966,8 @@ static void start_sw_tscdeadline(struct kvm_lapic *apic) static inline u64 tmict_to_ns(struct kvm_lapic *apic, u32 tmict) { - return (u64)tmict * APIC_BUS_CYCLE_NS * (u64)apic->divide_count; + return (u64)tmict * apic->vcpu->kvm->arch.apic_bus_cycle_ns * + (u64)apic->divide_count; } static void update_target_expiration(struct kvm_lapic *apic, uint32_t old_divisor) diff --git a/arch/x86/kvm/lapic.h b/arch/x86/kvm/lapic.h index a20cb006b6c8..51e09f5a7fc5 100644 --- a/arch/x86/kvm/lapic.h +++ b/arch/x86/kvm/lapic.h @@ -16,7 +16,7 @@ #define APIC_DEST_NOSHORT 0x0 #define APIC_DEST_MASK 0x800 -#define APIC_BUS_CYCLE_NS 1 +#define APIC_BUS_CYCLE_NS_DEFAULT 1 #define APIC_BROADCAST 0xFF #define X2APIC_BROADCAST 0xFFFFFFFFul diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index b389129d59a9..bc3c63e58488 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -12623,6 +12623,7 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type) raw_spin_unlock_irqrestore(&kvm->arch.tsc_write_lock, flags); kvm->arch.default_tsc_khz = max_tsc_khz ? : tsc_khz; + kvm->arch.apic_bus_cycle_ns = APIC_BUS_CYCLE_NS_DEFAULT; kvm->arch.guest_can_read_msr_platform_info = true; kvm->arch.enable_pmu = enable_pmu; From patchwork Mon May 6 18:35:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Reinette Chatre X-Patchwork-Id: 13655892 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.9]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 08271158D84; Mon, 6 May 2024 18:36:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.9 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715020587; cv=none; b=AcUU6JSIFTWPR9HdF9KpaqqBjxk6wYW3VlEF5PvODin1L9LozB3ta6KhTZ1vnnvLG4ZU9mTDGEOcZCXRp+eblm9G39lLhrKL6zIeZjN5TWDZffsuXNRQv0sxhcpbO64bE8HbsnfdAR9ijryOsHvC2cKyfiaJaaIn5OJH3TDPBs8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715020587; c=relaxed/simple; bh=0MgHNgq04PAuwzWmHgs3NlDZm14HBu1aJPkRBeg0xmM=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version:Content-Type; b=KFCNBMJGNhHy9+RHdDoO6o1adc4VUzlvj+VLJQFb/uWOspxzqrb9ZN0rFJfOVb5KkNbEzOJA2Rf2/0o947i1t+4AI1p2cVYMJqcMA7XZbWWDiPXXYY6gAMpI79Djcf22WAQdMHS9+34r6VfYVqXkxK7tBpSuvVN1VQrCV8ZF6nI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=L/Ns84LR; arc=none smtp.client-ip=192.198.163.9 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="L/Ns84LR" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1715020586; x=1746556586; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=0MgHNgq04PAuwzWmHgs3NlDZm14HBu1aJPkRBeg0xmM=; b=L/Ns84LRK9wEJ7TIcKNZOFA+7F9AykE+JkfCJL6HcLqNEW5b0ZqdoLra WchU9VsHLhCz2T4TvEoQ5tapE/5XAqbL8kwQt8jgvZjzbePTEjZ8sKLiq DdB7NEXl+o2Gg+uVDCR7mucqp9mWg09rXIFbKLSkibFGZOxo4yCtyOcz+ GlEEiBafFhnc4i5sTRHcLliv9jhUc9SDCNWIWh5CEt/w5XidPzpfTnCLK FWZw52b+QFk1j9Fg+qs/N8OM33KQ/hsH2LhGvMFyyG/Uh4tTvY1vCm5rZ oZmUuTSHIzkQPYzJOhGIm+lsBBSGTVwI3h3v7S8nBewSihqkLwMrZeF1K A==; X-CSE-ConnectionGUID: HRYMwKGVRl6PBy1QlFZLNw== X-CSE-MsgGUID: 10EUrWGzRSu/p9HtBiGzCw== X-IronPort-AV: E=McAfee;i="6600,9927,11065"; a="21455752" X-IronPort-AV: E=Sophos;i="6.07,259,1708416000"; d="scan'208";a="21455752" Received: from fmviesa008.fm.intel.com ([10.60.135.148]) by fmvoesa103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 May 2024 11:36:10 -0700 X-CSE-ConnectionGUID: zYu8lbR1RA6/HudOC8wTSA== X-CSE-MsgGUID: opfrFNdRS4+NGuKtkK5jqA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.07,259,1708416000"; d="scan'208";a="28237812" Received: from rchatre-ws.ostc.intel.com ([10.54.69.144]) by fmviesa008-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 May 2024 11:36:10 -0700 From: Reinette Chatre To: isaku.yamahata@intel.com, pbonzini@redhat.com, erdemaktas@google.com, vkuznets@redhat.com, seanjc@google.com, vannapurve@google.com, jmattson@google.com, mlevitsk@redhat.com, xiaoyao.li@intel.com, chao.gao@intel.com, rick.p.edgecombe@intel.com, yuan.yao@intel.com Cc: reinette.chatre@intel.com, kvm@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH V6 3/4] KVM: x86: Add a capability to configure bus frequency for APIC timer Date: Mon, 6 May 2024 11:35:57 -0700 Message-Id: <2b69deffadfd5f52355eee26c45a8d97e75d4f8c.1715017765.git.reinette.chatre@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata Add KVM_CAP_X86_APIC_BUS_CYCLES_NS capability to configure the APIC bus clock frequency for APIC timer emulation. Allow KVM_ENABLE_CAPABILITY(KVM_CAP_X86_APIC_BUS_CYCLES_NS) to set the frequency in nanoseconds. When using this capability, the user space VMM should configure CPUID leaf 0x15 to advertise the frequency. Vishal reported that the TDX guest kernel expects a 25MHz APIC bus frequency but ends up getting interrupts at a significantly higher rate. The TDX architecture hard-codes the core crystal clock frequency to 25MHz and mandates exposing it via CPUID leaf 0x15. The TDX architecture does not allow the VMM to override the value. In addition, per Intel SDM: "The APIC timer frequency will be the processor’s bus clock or core crystal clock frequency (when TSC/core crystal clock ratio is enumerated in CPUID leaf 0x15) divided by the value specified in the divide configuration register." The resulting 25MHz APIC bus frequency conflicts with the KVM hardcoded APIC bus frequency of 1GHz. The KVM doesn't enumerate CPUID leaf 0x15 to the guest unless the user space VMM sets it using KVM_SET_CPUID. If the CPUID leaf 0x15 is enumerated, the guest kernel uses it as the APIC bus frequency. If not, the guest kernel measures the frequency based on other known timers like the ACPI timer or the legacy PIT. As reported by Vishal the TDX guest kernel expects a 25MHz timer frequency but gets timer interrupt more frequently due to the 1GHz frequency used by KVM. To ensure that the guest doesn't have a conflicting view of the APIC bus frequency, allow the userspace to tell KVM to use the same frequency that TDX mandates instead of the default 1Ghz. There are several options to address this: 1. Make the KVM able to configure APIC bus frequency (this series). Pro: It resembles the existing hardware. The recent Intel CPUs adapts 25MHz. Con: Require the VMM to emulate the APIC timer at 25MHz. 2. Make the TDX architecture enumerate CPUID leaf 0x15 to configurable frequency or not enumerate it. Pro: Any APIC bus frequency is allowed. Con: Deviates from TDX architecture. 3. Make the TDX guest kernel use 1GHz when it's running on KVM. Con: The kernel ignores CPUID leaf 0x15. 4. Change CPUID leaf 0x15 under TDX to report the crystal clock frequency as 1 GHz. Pro: This has been the virtual APIC frequency for KVM guests for 13 years. Pro: This requires changing only one hard-coded constant in TDX. Con: It doesn't work with other VMMs as TDX isn't specific to KVM. Con: Core crystal clock frequency is also used to calculate TSC frequency. Con: If it is configured to value different from hardware, it will break the correctness of INTEL-PT Mini Time Count (MTC) packets in TDs. Reported-by: Vishal Annapurve Closes: https://lore.kernel.org/lkml/20231006011255.4163884-1-vannapurve@google.com/ Signed-off-by: Isaku Yamahata Reviewed-by: Rick Edgecombe Reviewed-by: Yuan Yao Reviewed-by: Xiaoyao Li Co-developed-by: Reinette Chatre Signed-off-by: Reinette Chatre --- Changes v6: - Add Yuan Yao's Reviewed-by tag. - Add Xiaoyao Li's Reviewed-by tag. Changes v5: - Rename capability KVM_CAP_X86_APIC_BUS_FREQUENCY -> KVM_CAP_X86_APIC_BUS_CYCLES_NS. (Xiaoyao Li) - Add Rick's Reviewed-by tag. Changes v4: - Rework implementation following Sean's guidance in: https://lore.kernel.org/all/ZdjzIgS6EAeCsUue@google.com/ - Reword con #2 to acknowledge feedback. (Sean) - Add the "con" information from Xiaoyao during earlier review of v2. - Rework changelog to address comments related to "bus clock" vs "core crystal clock" frequency. (Xiaoyao) - Drop snippet about impact on TSC deadline timer emulation. (Maxim) - Drop Maxim Levitsky's "Reviewed-by" tag due to many changes to patch since tag received. - Switch "Subject:" to match custom "KVM: X86:" -> "KVM: x86:" Changes v3: - Added reviewed-by Maxim Levitsky. - Minor update of the commit message. Changes v2: - Add check if vcpu isn't created. - Add check if lapic chip is in-kernel emulation. - Fix build error for i386. - Add document to api.rst. - Typo in the commit message. --- Documentation/virt/kvm/api.rst | 17 +++++++++++++++++ arch/x86/kvm/x86.c | 27 +++++++++++++++++++++++++++ include/uapi/linux/kvm.h | 1 + 3 files changed, 45 insertions(+) diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst index c6a57cebb65c..7f6d4b5e3a53 100644 --- a/Documentation/virt/kvm/api.rst +++ b/Documentation/virt/kvm/api.rst @@ -8063,6 +8063,23 @@ error/annotated fault. See KVM_EXIT_MEMORY_FAULT for more information. +7.35 KVM_CAP_X86_APIC_BUS_CYCLES_NS +----------------------------------- + +:Architectures: x86 +:Target: VM +:Parameters: args[0] is the desired APIC bus clock rate, in nanoseconds +:Returns: 0 on success, -EINVAL if args[0] contains an invalid value for the + frequency or if any vCPUs have been created, -ENXIO if a virtual + local APIC has not been created using KVM_CREATE_IRQCHIP. + +This capability sets VM's APIC bus clock frequency, used by KVM's in-kernel +virtual APIC when emulating APIC timers. KVM's default value can be retrieved +by KVM_CHECK_EXTENSION. + +Note: Userspace is responsible for correctly configuring CPUID 0x15, a.k.a. the +core crystal clock frequency, if a non-zero CPUID 0x15 is exposed to the guest. + 8. Other capabilities. ====================== diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index bc3c63e58488..f560811d658b 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -4712,6 +4712,9 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) case KVM_CAP_MEMORY_FAULT_INFO: r = 1; break; + case KVM_CAP_X86_APIC_BUS_CYCLES_NS: + r = APIC_BUS_CYCLE_NS_DEFAULT; + break; case KVM_CAP_EXIT_HYPERCALL: r = KVM_EXIT_HYPERCALL_VALID_MASK; break; @@ -6752,6 +6755,30 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm, } mutex_unlock(&kvm->lock); break; + case KVM_CAP_X86_APIC_BUS_CYCLES_NS: { + u64 bus_cycle_ns = cap->args[0]; + u64 unused; + + r = -EINVAL; + /* + * Guard against overflow in tmict_to_ns(). 128 is the highest + * divide value that can be programmed in APIC_TDCR. + */ + if (!bus_cycle_ns || + check_mul_overflow((u64)U32_MAX * 128, bus_cycle_ns, &unused)) + break; + + r = 0; + mutex_lock(&kvm->lock); + if (!irqchip_in_kernel(kvm)) + r = -ENXIO; + else if (kvm->created_vcpus) + r = -EINVAL; + else + kvm->arch.apic_bus_cycle_ns = bus_cycle_ns; + mutex_unlock(&kvm->lock); + break; + } default: r = -EINVAL; break; diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h index 2190adbe3002..6a4d9432ab11 100644 --- a/include/uapi/linux/kvm.h +++ b/include/uapi/linux/kvm.h @@ -917,6 +917,7 @@ struct kvm_enable_cap { #define KVM_CAP_MEMORY_ATTRIBUTES 233 #define KVM_CAP_GUEST_MEMFD 234 #define KVM_CAP_VM_TYPES 235 +#define KVM_CAP_X86_APIC_BUS_CYCLES_NS 236 struct kvm_irq_routing_irqchip { __u32 irqchip; From patchwork Mon May 6 18:35:58 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Reinette Chatre X-Patchwork-Id: 13655893 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.9]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B0B35158DC4; Mon, 6 May 2024 18:36:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.9 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715020588; cv=none; b=QovhV4FyhtzRq2e5G5sVGFAjAVwn3yvcLmFs/5uLpADbapudChK7UR5EYqg3Ao96lT7qTsO2F8UcdjKsyy0dQXTuPKEx4wTkvhFI08dDbxZb3Sbr9pHof0lCWPDLvtIaYDGAhCMeQG5p64CBDoInnf6q275FC4K1/MRq47IWLx0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715020588; c=relaxed/simple; bh=+ihVkDEpUcnmzb/O05pN4MdzfyRmDXtshitm8PzX3qw=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=kE+qHZKtt4ZhrB+eBMGs7PeHEDPH/rMjkaFaXd09iCXSMgdJT7HokRXw5p62fL//V6sogJU04s08PSA99ebj/2C32cnw4bec3FiLvzXo5L+iWBaxsDFsS9HB13FL5Yrcjc9v+wmtQ7absBm4ane+lymo4ARB3IzK6FbFm3Xzch0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=ly9yZ3VF; arc=none smtp.client-ip=192.198.163.9 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="ly9yZ3VF" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1715020587; x=1746556587; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=+ihVkDEpUcnmzb/O05pN4MdzfyRmDXtshitm8PzX3qw=; b=ly9yZ3VF2hBxg241uO8E4r76K7/Voru+5P9NA5+X6vXNQ6d2JsdZwViQ TMplP6zCM314TH/XHwUmyTt5Xhfk5t4T2rAHZy3pIS91SY5h0VRVhAyMO 70FWzSzafeyGVmeUSjYQt2m5leFtkuS8LH7NSVI9i2SLSTsE8tBOKSmx+ M8rcTk+KwCoghd5aA5Z7PCw8OCAXM6fGeGCySM1AeL/Bqd9wAS+P6SNsu G5Jey11eiDvSNa0aUsPxSInDB4oaQZGJbKmnB1YEkqkS8xNun+yApyWrl OqTkWGRT1UxoExynSUtGvzPCdeNcAGO3bU6YcHwoNL8Ym3SzyMOXH7Aul A==; X-CSE-ConnectionGUID: tAH24pjiQDKLHFYkRqtUaQ== X-CSE-MsgGUID: C/U9IQm+TbO4Dwkt+nvYcw== X-IronPort-AV: E=McAfee;i="6600,9927,11065"; a="21455756" X-IronPort-AV: E=Sophos;i="6.07,259,1708416000"; d="scan'208";a="21455756" Received: from fmviesa008.fm.intel.com ([10.60.135.148]) by fmvoesa103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 May 2024 11:36:10 -0700 X-CSE-ConnectionGUID: 28MLq4amRrmBCBVBu2azjQ== X-CSE-MsgGUID: 7spJvhH4T9Sp5VixDIx9sA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.07,259,1708416000"; d="scan'208";a="28237815" Received: from rchatre-ws.ostc.intel.com ([10.54.69.144]) by fmviesa008-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 May 2024 11:36:10 -0700 From: Reinette Chatre To: isaku.yamahata@intel.com, pbonzini@redhat.com, erdemaktas@google.com, vkuznets@redhat.com, seanjc@google.com, vannapurve@google.com, jmattson@google.com, mlevitsk@redhat.com, xiaoyao.li@intel.com, chao.gao@intel.com, rick.p.edgecombe@intel.com, yuan.yao@intel.com Cc: reinette.chatre@intel.com, kvm@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH V6 4/4] KVM: selftests: Add test for configure of x86 APIC bus frequency Date: Mon, 6 May 2024 11:35:58 -0700 Message-Id: X-Mailer: git-send-email 2.34.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata Test if the APIC bus clock frequency is the expected configured value. Set APIC timer's initial count to the maximum value and busy wait for 100 msec (any value is okay) with TSC value. Read the APIC timer's "current count" to calculate the actual APIC bus clock frequency based on TSC frequency. Suggested-by: Sean Christopherson Signed-off-by: Isaku Yamahata Reviewed-by: Maxim Levitsky Co-developed-by: Reinette Chatre Signed-off-by: Reinette Chatre --- Changes v6: - Use vm_create() wrapper instead of open coding it. (Zide) - Improve grammar of test description. (Zide) Changes v5: - Update to new name of capability KVM_CAP_X86_APIC_BUS_FREQUENCY -> KVM_CAP_X86_APIC_BUS_CYCLES_NS. (Xiaoyao Li) Changes v4: - Rework changelog. - Add Sean's "Suggested-by" to acknowledge guidance received in https://lore.kernel.org/all/ZU0BASXWcck85r90@google.com/ - Add copyright. - Add test description to file header. - Consistent capitalization for acronyms. - Rebase to kvm-x86/next. - Update to v4 change of providing bus clock rate in nanoseconds. - Add a "TEST_REQUIRE()" for the new capability so that the test can work on kernels that do not support the new capability. - Address checkpatch warnings and use tabs instead of spaces in header file to match existing code. Changes v3: - Use 1.5GHz instead of 1GHz as frequency. Changes v2: - Newly added. --- tools/testing/selftests/kvm/Makefile | 1 + .../selftests/kvm/include/x86_64/apic.h | 7 + .../kvm/x86_64/apic_bus_clock_test.c | 166 ++++++++++++++++++ 3 files changed, 174 insertions(+) create mode 100644 tools/testing/selftests/kvm/x86_64/apic_bus_clock_test.c diff --git a/tools/testing/selftests/kvm/Makefile b/tools/testing/selftests/kvm/Makefile index 6de9994971c9..52039d7c8dd5 100644 --- a/tools/testing/selftests/kvm/Makefile +++ b/tools/testing/selftests/kvm/Makefile @@ -111,6 +111,7 @@ TEST_GEN_PROGS_x86_64 += x86_64/vmx_invalid_nested_guest_state TEST_GEN_PROGS_x86_64 += x86_64/vmx_set_nested_state_test TEST_GEN_PROGS_x86_64 += x86_64/vmx_tsc_adjust_test TEST_GEN_PROGS_x86_64 += x86_64/vmx_nested_tsc_scaling_test +TEST_GEN_PROGS_x86_64 += x86_64/apic_bus_clock_test TEST_GEN_PROGS_x86_64 += x86_64/xapic_ipi_test TEST_GEN_PROGS_x86_64 += x86_64/xapic_state_test TEST_GEN_PROGS_x86_64 += x86_64/xcr0_cpuid_test diff --git a/tools/testing/selftests/kvm/include/x86_64/apic.h b/tools/testing/selftests/kvm/include/x86_64/apic.h index bed316fdecd5..b0d2fc62e172 100644 --- a/tools/testing/selftests/kvm/include/x86_64/apic.h +++ b/tools/testing/selftests/kvm/include/x86_64/apic.h @@ -60,6 +60,13 @@ #define APIC_VECTOR_MASK 0x000FF #define APIC_ICR2 0x310 #define SET_APIC_DEST_FIELD(x) ((x) << 24) +#define APIC_LVT0 0x350 +#define APIC_LVT_TIMER_ONESHOT (0 << 17) +#define APIC_LVT_TIMER_PERIODIC (1 << 17) +#define APIC_LVT_TIMER_TSCDEADLINE (2 << 17) +#define APIC_TMICT 0x380 +#define APIC_TMCCT 0x390 +#define APIC_TDCR 0x3E0 void apic_disable(void); void xapic_enable(void); diff --git a/tools/testing/selftests/kvm/x86_64/apic_bus_clock_test.c b/tools/testing/selftests/kvm/x86_64/apic_bus_clock_test.c new file mode 100644 index 000000000000..56eb686144c6 --- /dev/null +++ b/tools/testing/selftests/kvm/x86_64/apic_bus_clock_test.c @@ -0,0 +1,166 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Test configure of APIC bus frequency. + * + * Copyright (c) 2024 Intel Corporation + * + * To verify if the APIC bus frequency can be configured this, test starts + * by setting the TSC frequency in KVM, and then: + * For every APIC timer frequency supported: + * * In the guest: + * * * Start the APIC timer by programming the APIC TMICT (initial count + * register) to the largest value possible to guarantee that it will + * not expire during the test, + * * * Wait for a known duration based on previously set TSC frequency, + * * * Stop the timer and read the APIC TMCCT (current count) register to + * determine the count at that time (TMCCT is loaded from TMICT when + * TMICT is programmed and then starts counting down). + * * In the host: + * * * Determine if the APIC counts close to configured APIC bus frequency + * while taking into account how the APIC timer frequency was modified + * using the APIC TDCR (divide configuration register). + */ +#define _GNU_SOURCE /* for program_invocation_short_name */ + +#include "apic.h" +#include "test_util.h" + +/* + * Pick one convenient value, 1.5GHz. No special meaning and different from + * the default value, 1GHz. + */ +#define TSC_HZ (1500 * 1000 * 1000ULL) + +/* Wait for 100 msec, not too long, not too short value. */ +#define LOOP_MSEC 100ULL +#define TSC_WAIT_DELTA (TSC_HZ / 1000 * LOOP_MSEC) + +/* + * Pick a typical value, 25MHz. Different enough from the default value, 1GHz. + */ +#define APIC_BUS_CLOCK_FREQ (25 * 1000 * 1000ULL) + +static void guest_code(void) +{ + /* + * Possible TDCR values and its divide count. Used to modify APIC + * timer frequency. + */ + struct { + u32 tdcr; + u32 divide_count; + } tdcrs[] = { + {0x0, 2}, + {0x1, 4}, + {0x2, 8}, + {0x3, 16}, + {0x8, 32}, + {0x9, 64}, + {0xa, 128}, + {0xb, 1}, + }; + + u32 tmict, tmcct; + u64 tsc0, tsc1; + int i; + + asm volatile("cli"); + + xapic_enable(); + + /* + * Setup one-shot timer. The vector does not matter because the + * interrupt does not fire. + */ + xapic_write_reg(APIC_LVT0, APIC_LVT_TIMER_ONESHOT); + + for (i = 0; i < ARRAY_SIZE(tdcrs); i++) { + xapic_write_reg(APIC_TDCR, tdcrs[i].tdcr); + + /* Set the largest value to not trigger the interrupt. */ + tmict = ~0; + xapic_write_reg(APIC_TMICT, tmict); + + /* Busy wait for LOOP_MSEC */ + tsc0 = rdtsc(); + tsc1 = tsc0; + while (tsc1 - tsc0 < TSC_WAIT_DELTA) + tsc1 = rdtsc(); + + /* Read APIC timer and TSC */ + tmcct = xapic_read_reg(APIC_TMCCT); + tsc1 = rdtsc(); + + /* Stop timer */ + xapic_write_reg(APIC_TMICT, 0); + + /* Report it. */ + GUEST_SYNC_ARGS(tdcrs[i].divide_count, tmict - tmcct, + tsc1 - tsc0, 0, 0); + } + + GUEST_DONE(); +} + +void test_apic_bus_clock(struct kvm_vcpu *vcpu) +{ + bool done = false; + struct ucall uc; + + while (!done) { + vcpu_run(vcpu); + TEST_ASSERT_KVM_EXIT_REASON(vcpu, KVM_EXIT_IO); + + switch (get_ucall(vcpu, &uc)) { + case UCALL_DONE: + done = true; + break; + case UCALL_ABORT: + REPORT_GUEST_ASSERT(uc); + break; + case UCALL_SYNC: { + u32 divide_counter = uc.args[1]; + u32 apic_cycles = uc.args[2]; + u64 tsc_cycles = uc.args[3]; + u64 freq; + + TEST_ASSERT(tsc_cycles > 0, + "TSC cycles must not be zero."); + + /* Allow 1% slack. */ + freq = apic_cycles * divide_counter * TSC_HZ / tsc_cycles; + TEST_ASSERT(freq < APIC_BUS_CLOCK_FREQ * 101 / 100, + "APIC bus clock frequency is too large"); + TEST_ASSERT(freq > APIC_BUS_CLOCK_FREQ * 99 / 100, + "APIC bus clock frequency is too small"); + break; + } + default: + TEST_FAIL("Unknown ucall %lu", uc.cmd); + break; + } + } +} + +int main(int argc, char *argv[]) +{ + struct kvm_vcpu *vcpu; + struct kvm_vm *vm; + + TEST_REQUIRE(kvm_has_cap(KVM_CAP_X86_APIC_BUS_CYCLES_NS)); + + vm = vm_create(1); + vm_ioctl(vm, KVM_SET_TSC_KHZ, (void *)(TSC_HZ / 1000)); + /* + * KVM_CAP_X86_APIC_BUS_CYCLES_NS expects APIC bus clock rate in + * nanoseconds and requires that no vCPU is created. + */ + vm_enable_cap(vm, KVM_CAP_X86_APIC_BUS_CYCLES_NS, + NSEC_PER_SEC / APIC_BUS_CLOCK_FREQ); + vcpu = vm_vcpu_add(vm, 0, guest_code); + + virt_pg_map(vm, APIC_DEFAULT_GPA, APIC_DEFAULT_GPA); + + test_apic_bus_clock(vcpu); + kvm_vm_free(vm); +}