From patchwork Fri Jan 20 12:20:30 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marcelo Tosatti X-Patchwork-Id: 9528199 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 01BD860459 for ; Fri, 20 Jan 2017 12:28:28 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E6DB628616 for ; Fri, 20 Jan 2017 12:28:27 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id DBB5C2861F; Fri, 20 Jan 2017 12:28:27 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5A01D28616 for ; Fri, 20 Jan 2017 12:28:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752089AbdATM1n (ORCPT ); Fri, 20 Jan 2017 07:27:43 -0500 Received: from mx1.redhat.com ([209.132.183.28]:51780 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751961AbdATM1k (ORCPT ); Fri, 20 Jan 2017 07:27:40 -0500 Received: from int-mx09.intmail.prod.int.phx2.redhat.com (int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id C60BE7FE89; Fri, 20 Jan 2017 12:27:40 +0000 (UTC) Received: from amt.cnet (vpn1-4-171.gru2.redhat.com [10.97.4.171]) by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v0KCRd4F031441; Fri, 20 Jan 2017 07:27:40 -0500 Received: from amt.cnet (localhost [127.0.0.1]) by amt.cnet (Postfix) with ESMTP id F387110090A; Fri, 20 Jan 2017 10:27:07 -0200 (BRST) Received: (from marcelo@localhost) by amt.cnet (8.14.7/8.14.7/Submit) id v0KCR7hV020331; Fri, 20 Jan 2017 10:27:07 -0200 Message-Id: <20170120122503.842086637@redhat.com> User-Agent: quilt/0.60-1 Date: Fri, 20 Jan 2017 10:20:30 -0200 From: Marcelo Tosatti To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Paolo Bonzini , Radim Krcmar , Richard Cochran , Miroslav Lichvar , Marcelo Tosatti Subject: [patch 5/5] PTP: add kvm PTP driver References: <20170120122025.665985919@redhat.com> Content-Disposition: inline; filename=kvm-ptpdriver X-Scanned-By: MIMEDefang 2.68 on 10.5.11.22 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.27]); Fri, 20 Jan 2017 12:27:40 +0000 (UTC) Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Add a driver with gettime method returning hosts realtime clock. This allows Chrony to synchronize host and guest clocks with high precision (see results below). chronyc> sources MS Name/IP address Stratum Poll Reach LastRx Last sample --- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html =============================================================================== #* PHC0 0 3 377 4 +162ns[ -683ns] +/- 11ns To configure Chronyd to use PHC refclock, add the following line to its configuration file: refclock PHC /dev/ptpX poll 3 dpoll -2 offset 0 Where /dev/ptpX is the kvmclock PTP clock. Signed-off-by: Marcelo Tosatti --- drivers/ptp/Kconfig | 12 ++ drivers/ptp/Makefile | 1 drivers/ptp/ptp_kvm.c | 213 ++++++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 226 insertions(+) v2: check for kvmclock (Radim) initialize global variables before device registration (Radim) v3: use cross timestamps callback (Paolo, Miroslav, Radim) Index: kvm-ptpdriver/drivers/ptp/Kconfig =================================================================== --- kvm-ptpdriver.orig/drivers/ptp/Kconfig 2017-01-20 10:03:44.458489214 -0200 +++ kvm-ptpdriver/drivers/ptp/Kconfig 2017-01-20 10:04:26.912597433 -0200 @@ -90,4 +90,16 @@ To compile this driver as a module, choose M here: the module will be called ptp_pch. +config PTP_1588_CLOCK_KVM + tristate "KVM virtual PTP clock" + depends on PTP_1588_CLOCK + depends on KVM_GUEST + default y + help + This driver adds support for using kvm infrastructure as a PTP + clock. This clock is only useful if you are using KVM guests. + + To compile this driver as a module, choose M here: the module + will be called ptp_kvm. + endmenu Index: kvm-ptpdriver/drivers/ptp/Makefile =================================================================== --- kvm-ptpdriver.orig/drivers/ptp/Makefile 2017-01-20 10:03:44.458489214 -0200 +++ kvm-ptpdriver/drivers/ptp/Makefile 2017-01-20 10:04:26.913597436 -0200 @@ -6,3 +6,4 @@ obj-$(CONFIG_PTP_1588_CLOCK) += ptp.o obj-$(CONFIG_PTP_1588_CLOCK_IXP46X) += ptp_ixp46x.o obj-$(CONFIG_PTP_1588_CLOCK_PCH) += ptp_pch.o +obj-$(CONFIG_PTP_1588_CLOCK_KVM) += ptp_kvm.o Index: kvm-ptpdriver/drivers/ptp/ptp_kvm.c =================================================================== --- /dev/null 1970-01-01 00:00:00.000000000 +0000 +++ kvm-ptpdriver/drivers/ptp/ptp_kvm.c 2017-01-20 10:19:20.555311672 -0200 @@ -0,0 +1,213 @@ +/* + * Virtual PTP 1588 clock for use with KVM guests + * + * Copyright (C) 2017 Red Hat Inc. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + */ +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include + +struct kvm_ptp_clock { + struct ptp_clock *ptp_clock; + struct ptp_clock_info caps; +}; + +DEFINE_SPINLOCK(kvm_ptp_lock); + +static struct pvclock_vsyscall_time_info *hv_clock; + +static struct kvm_clock_offset clock_off; +static phys_addr_t clock_off_gpa; + +/* + * system_counterval.cycles: kvmclock value com TSC do host. + * system_counterval.cs: kvmclock clocksource. + * device_time: host realtime clock. + * + */ +static int ptp_kvm_get_time_fn(ktime_t *device_time, + struct system_counterval_t *system_counter, + void *ctx) +{ + unsigned long ret; + struct timespec64 tspec; + unsigned version; + u8 flags; + int cpu; + struct pvclock_vcpu_time_info *src; + + preempt_disable_notrace(); + cpu = smp_processor_id(); + src = &hv_clock[cpu].pvti; + + spin_lock(&kvm_ptp_lock); + + do { + /* + * We are measuring the delay between + * kvm_hypercall and rdtsc using TSC, + * and converting that delta to + * tsc_to_system_mul and tsc_shift + * So any changes to tsc_to_system_mul + * and tsc_shift in this region + * invalidate the measurement. + */ + version = pvclock_read_begin(src); + + ret = kvm_hypercall2(KVM_HC_CLOCK_PAIRING, + clock_off_gpa, + KVM_CLOCK_PAIRING_WALLCLOCK); + if (ret != 0) { + pr_err("clock offset hypercall ret %lu\n", ret); + spin_unlock(&kvm_ptp_lock); + preempt_enable_notrace(); + return -EOPNOTSUPP; + } + + tspec.tv_sec = clock_off.sec; + tspec.tv_nsec = clock_off.nsec; + ret = __pvclock_read_cycles(src, clock_off.tsc); + flags = src->flags; + } while (pvclock_read_retry(src, version)); + + preempt_enable_notrace(); + + system_counter->cycles = ret; + system_counter->cs = get_kvmclock_cs(); + + tspec.tv_nsec = tspec.tv_nsec; + + *device_time = timespec_to_ktime(tspec); + + spin_unlock(&kvm_ptp_lock); + + return 0; +} + +static int ptp_kvm_getcrosststamp(struct ptp_clock_info *ptp, + struct system_device_crosststamp *xtstamp) +{ + return get_device_system_crosststamp(ptp_kvm_get_time_fn, NULL, + NULL, xtstamp); +} + +/* + * PTP clock operations + */ + +static int ptp_kvm_adjfreq(struct ptp_clock_info *ptp, s32 ppb) +{ + return -EOPNOTSUPP; +} + +static int ptp_kvm_adjtime(struct ptp_clock_info *ptp, s64 delta) +{ + return -EOPNOTSUPP; +} + +static int ptp_kvm_gettime(struct ptp_clock_info *ptp, struct timespec64 *ts) +{ + unsigned long ret; + struct timespec64 tspec; + + spin_lock(&kvm_ptp_lock); + + ret = kvm_hypercall2(KVM_HC_CLOCK_PAIRING, + clock_off_gpa, + KVM_CLOCK_PAIRING_WALLCLOCK); + if (ret != 0) { + pr_err("clock offset hypercall ret %lu\n", ret); + spin_unlock(&kvm_ptp_lock); + return -EOPNOTSUPP; + } + + tspec.tv_sec = clock_off.sec; + tspec.tv_nsec = clock_off.nsec; + spin_unlock(&kvm_ptp_lock); + + memcpy(ts, &tspec, sizeof(struct timespec64)); + + return 0; +} + +static int ptp_kvm_settime(struct ptp_clock_info *ptp, + const struct timespec64 *ts) +{ + return -EOPNOTSUPP; +} + +static int ptp_kvm_enable(struct ptp_clock_info *ptp, + struct ptp_clock_request *rq, int on) +{ + return -EOPNOTSUPP; +} + +static struct ptp_clock_info ptp_kvm_caps = { + .owner = THIS_MODULE, + .name = "KVM virtual PTP", + .max_adj = 0, + .n_ext_ts = 0, + .n_pins = 0, + .pps = 0, + .adjfreq = ptp_kvm_adjfreq, + .adjtime = ptp_kvm_adjtime, + .gettime64 = ptp_kvm_gettime, + .settime64 = ptp_kvm_settime, + .enable = ptp_kvm_enable, + .getcrosststamp = ptp_kvm_getcrosststamp, + .emulate_ptp_sys_offset_mean = true, +}; + +/* module operations */ + +static struct kvm_ptp_clock kvm_ptp_clock; + +static void __exit ptp_kvm_exit(void) +{ + ptp_clock_unregister(kvm_ptp_clock.ptp_clock); +} + +static int __init ptp_kvm_init(void) +{ + clock_off_gpa = slow_virt_to_phys(&clock_off); + hv_clock = pvclock_pvti_cpu0_va(); + + if (!hv_clock) + return -ENODEV; + + kvm_ptp_clock.caps = ptp_kvm_caps; + + kvm_ptp_clock.ptp_clock = ptp_clock_register(&kvm_ptp_clock.caps, NULL); + + if (IS_ERR(kvm_ptp_clock.ptp_clock)) + return PTR_ERR(kvm_ptp_clock.ptp_clock); + + return 0; +} + +module_init(ptp_kvm_init); +module_exit(ptp_kvm_exit); + +MODULE_AUTHOR("Marcelo Tosatti "); +MODULE_DESCRIPTION("PTP clock using KVMCLOCK"); +MODULE_LICENSE("GPL");