From patchwork Fri May 22 08:37:13 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jianyong Wu X-Patchwork-Id: 11565037 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 16FBF13B1 for ; Fri, 22 May 2020 08:37:47 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0564020870 for ; Fri, 22 May 2020 08:37:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729586AbgEVIhl (ORCPT ); Fri, 22 May 2020 04:37:41 -0400 Received: from foss.arm.com ([217.140.110.172]:59234 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728959AbgEVIhk (ORCPT ); Fri, 22 May 2020 04:37:40 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 6DFB455D; Fri, 22 May 2020 01:37:39 -0700 (PDT) Received: from entos-d05.shanghai.arm.com (entos-d05.shanghai.arm.com [10.169.40.35]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id B55613F52E; Fri, 22 May 2020 01:37:32 -0700 (PDT) From: Jianyong Wu To: netdev@vger.kernel.org, yangbo.lu@nxp.com, john.stultz@linaro.org, tglx@linutronix.de, pbonzini@redhat.com, sean.j.christopherson@intel.com, maz@kernel.org, richardcochran@gmail.com, Mark.Rutland@arm.com, will@kernel.org, suzuki.poulose@arm.com, steven.price@arm.com Cc: linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, kvm@vger.kernel.org, Steve.Capper@arm.com, Kaly.Xin@arm.com, justin.he@arm.com, Wei.Chen@arm.com, jianyong.wu@arm.com, nd@arm.com Subject: [RFC PATCH v12 0/11] Enable ptp_kvm for arm64 Date: Fri, 22 May 2020 16:37:13 +0800 Message-Id: <20200522083724.38182-1-jianyong.wu@arm.com> X-Mailer: git-send-email 2.17.1 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Currently, we offen use ntp (sync time with remote network clock) to sync time in VM. But the precision of ntp is subject to network delay so it's difficult to sync time in a high precision. kvm virtual ptp clock (ptp_kvm) offers another way to sync time in VM, as the remote clock locates in the host instead of remote network clock. It targets to sync time between guest and host in virtualization environment and in this way, we can keep the time of all the VMs running in the same host in sync. In general, the delay of communication between host and guest is quiet small, so ptp_kvm can offer time sync precision up to in order of nanosecond. Please keep in mind that ptp_kvm just limits itself to be a channel which transmit the remote clock from host to guest and leaves the time sync jobs to an application, eg. chrony, in usersapce in VM. How ptp_kvm works: After ptp_kvm initialized, there will be a new device node under /dev called ptp%d. A guest userspace service, like chrony, can use this device to get host walltime, sometimes also counter cycle, which depends on the service it calls. Then this guest userspace service can use those data to do the time sync for guest. here is a rough sketch to show how kvm ptp works. |----------------------------| |--------------------------| | guest userspace | | host | |ioctl -> /dev/ptp%d | | | | ^ | | | | |----------------------------| | | | | | guest kernel | | | | | V (get host walltime/counter cycle) | | kvm ptp API (hypercall)- -|- - - - - - - - - ->hypercall service | | <- - - - - - - - - - - - | |----------------------------| |--------------------------| 1. time sync service in guest userspace call ptp device using ioctl. 2. guest kernel ptp_kvm API get this request then invoke hypercall to request host walltime/counter cycle to host kernel. 3. ptp_kvm host hypercall service response to the request and send back data 4. ptp copy the data to userspace. This ptp_kvm implemetation focuses itself to step 2 and 3 and step 2 works in guest comparing step 3 works in host kernel. change log: from v11 to v12: (1) rebase code on 5.7_rc6 and rebase 2 patches from Will Decon including 1/11 and 2/11. as these patches introduce discover mechanism of vendor smccc service. (2) rebase ptp_kvm hypercall service from standard smccc to vendor smccc and add ptp_kvm to vendor smccc service discover mechanism. (3) add detail of why we need ptp_kvm and how ptp_kvm works in cover letter. from v10 to v11: (1) rebase code on 5.7_rc2. (2) remove support for arm32, as kvm support for arm32 will be removed [1] (3) add error report in ptp_kvm initialization. from v11 to v10: (1) change code base to v5.5. (2) enable ptp_kvm both for arm32 and arm64. (3) let user choose which of virtual counter or physical counter should return when using crosstimestamp mode of ptp_kvm for arm/arm64. (4) extend input argument for getcrosstimestamp API. from v8 to v9: (1) move ptp_kvm.h to driver/ptp/ (2) replace license declaration of ptp_kvm.h the same with other header files in the same directory. from v7 to v8: (1) separate adding clocksource id for arm_arch_counter as a single patch. (2) update commit message for patch 4/8. (3) refine patch 7/8 and patch 8/8 to make them more independent. from v6 to v7: (1) include the omitted clocksource_id.h in last version. (2) reorder the header file in patch. (3) refine some words in commit message to make it more impersonal. from v5 to v6: (1) apply Mark's patch[4] to get SMCCC conduit. (2) add mechanism to recognize current clocksource by add clocksouce_id value into struct clocksource instead of method in patch-v5. (3) rename kvm_arch_ptp_get_clock_fn into kvm_arch_ptp_get_crosststamp. from v4 to v5: (1) remove hvc delay compensasion as it should leave to userspace. (2) check current clocksource in hvc call service. (3) expose current clocksource by adding it to system_time_snapshot. (4) add helper to check if clocksource is arm_arch_counter. (5) rename kvm_ptp.c to ptp_kvm_common.c from v3 to v4: (1) fix clocksource of ptp_kvm to arch_sys_counter. (2) move kvm_arch_ptp_get_clock_fn into arm_arch_timer.c (3) subtract cntvoff before return cycles from host. (4) use ktime_get_snapshot instead of getnstimeofday and get_current_counterval to return time and counter value. (5) split ktime and counter into two 32-bit block respectively to avoid Y2038-safe issue. (6) set time compensation to device time as half of the delay of hvc call. (7) add ARM_ARCH_TIMER as dependency of ptp_kvm for arm64. from v2 to v3: (1) fix some issues in commit log. (2) add some receivers in send list. from v1 to v2: (1) move arch-specific code from arch/ to driver/ptp/ (2) offer mechanism to inform userspace if ptp_kvm service is available. (3) separate ptp_kvm code for arm64 into hypervisor part and guest part. (4) add API to expose monotonic clock and counter value. (5) refine code: remove no necessary part and reconsitution. [1] https://patchwork.kernel.org/cover/11373351/ Jianyong Wu (8): psci: export psci conduit get helper. ptp: Reorganize ptp_kvm modules to make it arch-independent. clocksource: Add clocksource id for arm arch counter psci: Add hypercall service for ptp_kvm. ptp: arm64: Enable ptp_kvm for arm/arm64 ptp: extend input argument for getcrosstimestamp API arm64: add mechanism to let user choose which counter to return arm64: Add kvm capability check extension for ptp_kvm Thomas Gleixner (1): time: Add mechanism to recognize clocksource in time_get_snapshot drivers/clocksource/arm_arch_timer.c | 33 ++++++++ drivers/firmware/psci/psci.c | 1 + drivers/net/ethernet/intel/e1000e/ptp.c | 3 +- drivers/ptp/Kconfig | 2 +- drivers/ptp/Makefile | 1 + drivers/ptp/ptp_chardev.c | 8 +- drivers/ptp/ptp_kvm.h | 11 +++ drivers/ptp/ptp_kvm_arm64.c | 53 ++++++++++++ drivers/ptp/{ptp_kvm.c => ptp_kvm_common.c} | 85 ++++++-------------- drivers/ptp/ptp_kvm_x86.c | 89 +++++++++++++++++++++ include/linux/arm-smccc.h | 21 +++++ include/linux/clocksource.h | 6 ++ include/linux/clocksource_ids.h | 12 +++ include/linux/ptp_clock_kernel.h | 3 +- include/linux/timekeeping.h | 12 +-- include/uapi/linux/kvm.h | 1 + include/uapi/linux/ptp_clock.h | 4 +- kernel/time/clocksource.c | 3 + kernel/time/timekeeping.c | 1 + virt/kvm/arm/arm.c | 1 + virt/kvm/arm/hypercalls.c | 44 +++++++++- 21 files changed, 322 insertions(+), 72 deletions(-) create mode 100644 drivers/ptp/ptp_kvm.h create mode 100644 drivers/ptp/ptp_kvm_arm64.c rename drivers/ptp/{ptp_kvm.c => ptp_kvm_common.c} (60%) create mode 100644 drivers/ptp/ptp_kvm_x86.c create mode 100644 include/linux/clocksource_ids.h