From patchwork Fri Jun 19 13:01:19 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jianyong Wu X-Patchwork-Id: 11614179 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3C93613A0 for ; Fri, 19 Jun 2020 13:04:30 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id EC1982080C for ; Fri, 19 Jun 2020 13:04:29 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="p0fGDK+/" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EC1982080C Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=Si+s8TjvULsLmfTQWAHVxyp7vvx8cebEtZ8tVXmu4JE=; b=p0fGDK+/RtuqHlyqsLY+6kkzvB ycrHGygAT3dJ4de7qJiv+4PnxplFO2dB+Dm5Z9l07T+H6EnGBQEEOHRF4gXy9bPSxL1v5UQ5TZjbQ IW7jgiGifsdmMmE/Zpyn+d5lS6vo6YDpi0CtVQ04ealDyD3bX4LTlXBB9DTkqEy95jq9r344dNu8B QOnKazhNdsbxznfQ/VibQLBcW9IbrUQJOFgMSLR94K5nSSTuVpxGKfIDlF18THXMxMsMCCCC8sHH8 +gNbWODZt5HWFmCaFDOf9QoJGFMjqUa4fYXwxcPS1TTjgrVRCw+7TetL8f8C1PyFt2AJ4PLB35D7c 3i7gNL2g==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1jmGgw-0006YE-Ek; Fri, 19 Jun 2020 13:04:22 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1jmGfB-0004ys-TX for linux-arm-kernel@lists.infradead.org; Fri, 19 Jun 2020 13:02:35 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 57F4A101E; Fri, 19 Jun 2020 06:02:33 -0700 (PDT) Received: from entos-d05.shanghai.arm.com (entos-d05.shanghai.arm.com [10.169.40.35]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 29FCB3F6CF; Fri, 19 Jun 2020 06:02:26 -0700 (PDT) From: Jianyong Wu To: netdev@vger.kernel.org, yangbo.lu@nxp.com, john.stultz@linaro.org, tglx@linutronix.de, pbonzini@redhat.com, sean.j.christopherson@intel.com, maz@kernel.org, richardcochran@gmail.com, Mark.Rutland@arm.com, will@kernel.org, suzuki.poulose@arm.com, steven.price@arm.com Subject: [PATCH v13 8/9] ptp: arm64: Enable ptp_kvm for arm64 Date: Fri, 19 Jun 2020 21:01:19 +0800 Message-Id: <20200619130120.40556-9-jianyong.wu@arm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200619130120.40556-1-jianyong.wu@arm.com> References: <20200619130120.40556-1-jianyong.wu@arm.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200619_060234_045176_A1BB02A6 X-CRM114-Status: GOOD ( 18.38 ) X-Spam-Score: -2.3 (--) X-Spam-Report: SpamAssassin version 3.4.4 on bombadil.infradead.org summary: Content analysis details: (-2.3 points) pts rule name description ---- ---------------------- -------------------------------------------------- -2.3 RCVD_IN_DNSWL_MED RBL: Sender listed at https://www.dnswl.org/, medium trust [217.140.110.172 listed in list.dnswl.org] 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: justin.he@arm.com, Wei.Chen@arm.com, kvm@vger.kernel.org, Steve.Capper@arm.com, jianyong.wu@arm.com, linux-kernel@vger.kernel.org, Kaly.Xin@arm.com, nd@arm.com, kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org Currently, there is no mechanism to keep time sync between guest and host in arm64 virtualization environment. Time in guest will drift compared with host after boot up as they may both use third party time sources to correct their time respectively. The time deviation will be in order of milliseconds. But in some scenarios,like in cloud envirenment, we ask for higher time precision. kvm ptp clock, which choose the host clock source as a reference clock to sync time between guest and host, has been adopted by x86 which makes the time sync order from milliseconds to nanoseconds. This patch enables kvm ptp clock for arm64 and improve clock sync precison significantly. Test result comparisons between with kvm ptp clock and without it in arm64 are as follows. This test derived from the result of command 'chronyc sources'. we should take more care of the last sample column which shows the offset between the local clock and the source at the last measurement. no kvm ptp in guest: MS Name/IP address Stratum Poll Reach LastRx Last sample ======================================================================== ^* dns1.synet.edu.cn 2 6 377 13 +1040us[+1581us] +/- 21ms ^* dns1.synet.edu.cn 2 6 377 21 +1040us[+1581us] +/- 21ms ^* dns1.synet.edu.cn 2 6 377 29 +1040us[+1581us] +/- 21ms ^* dns1.synet.edu.cn 2 6 377 37 +1040us[+1581us] +/- 21ms ^* dns1.synet.edu.cn 2 6 377 45 +1040us[+1581us] +/- 21ms ^* dns1.synet.edu.cn 2 6 377 53 +1040us[+1581us] +/- 21ms ^* dns1.synet.edu.cn 2 6 377 61 +1040us[+1581us] +/- 21ms ^* dns1.synet.edu.cn 2 6 377 4 -130us[ +796us] +/- 21ms ^* dns1.synet.edu.cn 2 6 377 12 -130us[ +796us] +/- 21ms ^* dns1.synet.edu.cn 2 6 377 20 -130us[ +796us] +/- 21ms in host: MS Name/IP address Stratum Poll Reach LastRx Last sample ======================================================================== ^* 120.25.115.20 2 7 377 72 -470us[ -603us] +/- 18ms ^* 120.25.115.20 2 7 377 92 -470us[ -603us] +/- 18ms ^* 120.25.115.20 2 7 377 112 -470us[ -603us] +/- 18ms ^* 120.25.115.20 2 7 377 2 +872ns[-6808ns] +/- 17ms ^* 120.25.115.20 2 7 377 22 +872ns[-6808ns] +/- 17ms ^* 120.25.115.20 2 7 377 43 +872ns[-6808ns] +/- 17ms ^* 120.25.115.20 2 7 377 63 +872ns[-6808ns] +/- 17ms ^* 120.25.115.20 2 7 377 83 +872ns[-6808ns] +/- 17ms ^* 120.25.115.20 2 7 377 103 +872ns[-6808ns] +/- 17ms ^* 120.25.115.20 2 7 377 123 +872ns[-6808ns] +/- 17ms The dns1.synet.edu.cn is the network reference clock for guest and 120.25.115.20 is the network reference clock for host. we can't get the clock error between guest and host directly, but a roughly estimated value will be in order of hundreds of us to ms. with kvm ptp in guest: chrony has been disabled in host to remove the disturb by network clock. MS Name/IP address Stratum Poll Reach LastRx Last sample ======================================================================== * PHC0 0 3 377 8 -7ns[ +1ns] +/- 3ns * PHC0 0 3 377 8 +1ns[ +16ns] +/- 3ns * PHC0 0 3 377 6 -4ns[ -0ns] +/- 6ns * PHC0 0 3 377 6 -8ns[ -12ns] +/- 5ns * PHC0 0 3 377 5 +2ns[ +4ns] +/- 4ns * PHC0 0 3 377 13 +2ns[ +4ns] +/- 4ns * PHC0 0 3 377 12 -4ns[ -6ns] +/- 4ns * PHC0 0 3 377 11 -8ns[ -11ns] +/- 6ns * PHC0 0 3 377 10 -14ns[ -20ns] +/- 4ns * PHC0 0 3 377 8 +4ns[ +5ns] +/- 4ns The PHC0 is the ptp clock which choose the host clock as its source clock. So we can see that the clock difference between host and guest is in order of ns. Signed-off-by: Jianyong Wu --- drivers/clocksource/arm_arch_timer.c | 24 +++++++++++++ drivers/ptp/Kconfig | 2 +- drivers/ptp/ptp_kvm_arm64.c | 53 ++++++++++++++++++++++++++++ 3 files changed, 78 insertions(+), 1 deletion(-) create mode 100644 drivers/ptp/ptp_kvm_arm64.c diff --git a/drivers/clocksource/arm_arch_timer.c b/drivers/clocksource/arm_arch_timer.c index edc5be0ae324..69d30b407cd9 100644 --- a/drivers/clocksource/arm_arch_timer.c +++ b/drivers/clocksource/arm_arch_timer.c @@ -1639,3 +1639,27 @@ static int __init arch_timer_acpi_init(struct acpi_table_header *table) } TIMER_ACPI_DECLARE(arch_timer, ACPI_SIG_GTDT, arch_timer_acpi_init); #endif + +#if IS_ENABLED(CONFIG_PTP_1588_CLOCK_KVM) +#include +int kvm_arch_ptp_get_crosststamp(unsigned long *cycle, struct timespec64 *ts, + struct clocksource **cs) +{ + struct arm_smccc_res hvc_res; + ktime_t ktime_overall; + + /* Currently, our guest will always use the virtual counter */ + arm_smccc_1_1_invoke(ARM_SMCCC_VENDOR_HYP_KVM_PTP_FUNC_ID, + ARM_PTP_VIRT_COUNTER, &hvc_res); + if ((int)(hvc_res.a0) < 0) + return -EOPNOTSUPP; + + ktime_overall = (long long)hvc_res.a0 << 32 | hvc_res.a1; + *ts = ktime_to_timespec64(ktime_overall); + *cycle = (long long)hvc_res.a2 << 32 | hvc_res.a3; + *cs = &clocksource_counter; + + return 0; +} +EXPORT_SYMBOL_GPL(kvm_arch_ptp_get_crosststamp); +#endif diff --git a/drivers/ptp/Kconfig b/drivers/ptp/Kconfig index 942f72d8151d..127e96f14f89 100644 --- a/drivers/ptp/Kconfig +++ b/drivers/ptp/Kconfig @@ -106,7 +106,7 @@ config PTP_1588_CLOCK_PCH config PTP_1588_CLOCK_KVM tristate "KVM virtual PTP clock" depends on PTP_1588_CLOCK - depends on KVM_GUEST && X86 + depends on KVM_GUEST && X86 || ARM64 && ARM_ARCH_TIMER && ARM_PSCI_FW default y help This driver adds support for using kvm infrastructure as a PTP diff --git a/drivers/ptp/ptp_kvm_arm64.c b/drivers/ptp/ptp_kvm_arm64.c new file mode 100644 index 000000000000..704a7252b119 --- /dev/null +++ b/drivers/ptp/ptp_kvm_arm64.c @@ -0,0 +1,53 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Virtual PTP 1588 clock for use with KVM guests + * Copyright (C) 2019 ARM Ltd. + * All Rights Reserved + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +int kvm_arch_ptp_init(void) +{ + struct arm_smccc_res hvc_res; + + arm_smccc_1_1_invoke(ARM_SMCCC_VENDOR_HYP_KVM_FEATURES_FUNC_ID, + &hvc_res); + if (!(hvc_res.a0 | BIT(ARM_SMCCC_KVM_FUNC_KVM_PTP))) + return -EOPNOTSUPP; + + return 0; +} + +int kvm_arch_ptp_get_clock_generic(struct timespec64 *ts, + struct arm_smccc_res *hvc_res) +{ + ktime_t ktime_overall; + + arm_smccc_1_1_invoke(ARM_SMCCC_VENDOR_HYP_KVM_PTP_FUNC_ID, + hvc_res); + if ((int)(hvc_res->a0) < 0) + return -EOPNOTSUPP; + + ktime_overall = (long long)hvc_res->a0 << 32 | hvc_res->a1; + *ts = ktime_to_timespec64(ktime_overall); + + return 0; +} + +int kvm_arch_ptp_get_clock(struct timespec64 *ts) +{ + struct arm_smccc_res hvc_res; + + kvm_arch_ptp_get_clock_generic(ts, &hvc_res); + + return 0; +}