From patchwork Tue Jun 8 21:47:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oliver Upton X-Patchwork-Id: 12308201 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 61B4AC47082 for ; Tue, 8 Jun 2021 21:48:00 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 39B4C613B9 for ; Tue, 8 Jun 2021 21:48:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234426AbhFHVtw (ORCPT ); Tue, 8 Jun 2021 17:49:52 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43978 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234230AbhFHVtv (ORCPT ); Tue, 8 Jun 2021 17:49:51 -0400 Received: from mail-il1-x14a.google.com (mail-il1-x14a.google.com [IPv6:2607:f8b0:4864:20::14a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 631ABC061574 for ; Tue, 8 Jun 2021 14:47:58 -0700 (PDT) Received: by mail-il1-x14a.google.com with SMTP id g12-20020a056e021a2cb02901dfc46878d8so16255659ile.4 for ; Tue, 08 Jun 2021 14:47:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=4A17gJzJ1kAenjV8HCdX+pKLR+NA8dR0yX/rB5Wyus4=; b=sZ03P3gmdWaLshxczUdD9Od3bFJUln8rI4ImTn8BDyJzHtddp3THf5COkhfLMdj55E n9sjgUCKQgP4l6XRXdCHfCyOxu9hvHY16sqhT5/R6MnHcwV+HuxNq1NlyJj3uadUFc4n H7r4IxlGZHiAuvHOQ1gg1tFGgdhyKWQakRbEG+U2ckcGSCRIM5nYO7UbFtMxfpZwvKAE +fjddPP9+5BdMz7lTndnpx7znrI/Msz/7n33UV3eAkPwFaiEb5kqY7CxJfGm7Mkw6Xqm aKbHQ34Eby8DGH6OlizugwpfJIrMiZjv4GfocR3ndPv29968ap/g4kMYOR2TTuB1RcY2 SR7g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=4A17gJzJ1kAenjV8HCdX+pKLR+NA8dR0yX/rB5Wyus4=; b=KiO41peRyuNimke/cMf8O8OMUja9smA6/qUDSFtk+6gqqyJPMK7ceowOCSBuDB+C69 MFEDBhguDwpZ4W5xcr30nzZ7bxvYA8Ha5aqp2xoxbldOmb6HA5y792OhF6ZEUMQ9Aebw Z79n8Oa2iaKAZwHkVxLSFLDcAG3Ni+YVP31tDitwO6qhXTQhB7BJxA7SK+PpheQGBVbA uTR0cLs7wHyFEt1IK2gHbOZo1QuJ/o0t5sHtDB8PWknKD66YRYeQGmLAWdn7PijFdNpp 1UA2GD9/yFSK06YgDnxFYax3o6pyZuGB6DLCjBs/IEMwUu5MJqbK6XRb0j0tKMLH6u23 fLkA== X-Gm-Message-State: AOAM531lWaMvl8hYoeCscm2uqD6tpf4q8ziNr+4r8Z3Yy5z0ys/QuKM8 jwXc40M63D+at27JID+E6APOXvH/ro1jez1MwOboObLKc+QbkWm21faFsUniq6NT2ygl8DD2Tv/ pBvRylwupUYS5kbzTjL4+Z1gqRlWavsvNuuhp4paP/Y+Hd6ImcxaXlob22Q== X-Google-Smtp-Source: ABdhPJxnS6SA33thGYVUqbqETQx/fbnXtTuCiMbrOX/ip3kv7XfbzDxbKzCuHLNRXwCPmcgEWAfR4WhvG1c= X-Received: from oupton.c.googlers.com ([fda3:e722:ac3:10:2b:ff92:c0a8:404]) (user=oupton job=sendgmr) by 2002:a5d:9842:: with SMTP id p2mr20519082ios.132.1623188877580; Tue, 08 Jun 2021 14:47:57 -0700 (PDT) Date: Tue, 8 Jun 2021 21:47:33 +0000 In-Reply-To: <20210608214742.1897483-1-oupton@google.com> Message-Id: <20210608214742.1897483-2-oupton@google.com> Mime-Version: 1.0 References: <20210608214742.1897483-1-oupton@google.com> X-Mailer: git-send-email 2.32.0.rc1.229.g3e70b5a671-goog Subject: [PATCH 01/10] KVM: Introduce KVM_{GET,SET}_SYSTEM_COUNTER_STATE ioctls From: Oliver Upton To: kvm@vger.kernel.org, kvmarm@lists.cs.columbia.edu Cc: Paolo Bonzini , Sean Christopherson , Marc Zyngier , Peter Shier , Jim Mattson , David Matlack , Ricardo Koller , Jing Zhang , Raghavendra Rao Anata , Oliver Upton Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Correctly migrating a guest's view of the system counter is challenging. To date, KVM has utilized value-based interfaces (e.g. guest value for IA32_TSC msr on x86, CNTVCT_EL0 on arm64) for migrating guest counter state. Restoring counters by value is problematic, especially for operators who intend to have the guest's counters account for elapsed time during live migration. Reason being, the guest counters _still_ increment even after calculating an appropriate value to restore a guest's counter on the target machine. Furthermore, maintaining the phase relationship between vCPU counters is impossible with a value-based approach. The only hope for maintaining the phase relationship of counters is to restore them by offset. Introduce a new pair of vcpu ioctls, KVM_GET_SYSTEM_COUNTER_STATE and KVM_SET_SYSTEM_COUNTER_STATE, that aim to do just that. Each implementing architecture will define its own counter state structure, allowing for flexibility with ISAs that may have multiple counters (like arm64). Reviewed-by: Jing Zhang Reviewed-by: David Matlack Reviewed-by: Jim Mattson Reviewed-by: Peter Shier Reviewed-by: Ricardo Koller Signed-off-by: Oliver Upton --- include/uapi/linux/kvm.h | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h index 3fd9a7e9d90c..562650c14e39 100644 --- a/include/uapi/linux/kvm.h +++ b/include/uapi/linux/kvm.h @@ -1082,6 +1082,7 @@ struct kvm_ppc_resize_hpt { #define KVM_CAP_SGX_ATTRIBUTE 196 #define KVM_CAP_VM_COPY_ENC_CONTEXT_FROM 197 #define KVM_CAP_PTP_KVM 198 +#define KVM_CAP_SYSTEM_COUNTER_STATE 199 #ifdef KVM_CAP_IRQ_ROUTING @@ -1645,6 +1646,10 @@ struct kvm_xen_vcpu_attr { #define KVM_XEN_VCPU_ATTR_TYPE_RUNSTATE_DATA 0x4 #define KVM_XEN_VCPU_ATTR_TYPE_RUNSTATE_ADJUST 0x5 +/* Available with KVM_CAP_SYSTEM_COUNTER_STATE */ +#define KVM_GET_SYSTEM_COUNTER_STATE _IOWR(KVMIO, 0xcc, struct kvm_system_counter_state) +#define KVM_SET_SYSTEM_COUNTER_STATE _IOW(KVMIO, 0xcd, struct kvm_system_counter_state) + /* Secure Encrypted Virtualization command */ enum sev_cmd_id { /* Guest initialization commands */ From patchwork Tue Jun 8 21:47:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oliver Upton X-Patchwork-Id: 12308203 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 894BEC47082 for ; Tue, 8 Jun 2021 21:48:06 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6DFA0613B9 for ; Tue, 8 Jun 2021 21:48:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234430AbhFHVt7 (ORCPT ); Tue, 8 Jun 2021 17:49:59 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44000 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234064AbhFHVt6 (ORCPT ); Tue, 8 Jun 2021 17:49:58 -0400 Received: from mail-qv1-xf4a.google.com (mail-qv1-xf4a.google.com [IPv6:2607:f8b0:4864:20::f4a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DD02EC061574 for ; Tue, 8 Jun 2021 14:48:04 -0700 (PDT) Received: by mail-qv1-xf4a.google.com with SMTP id ce7-20020a05621403c7b0290238a4eac5c4so1678541qvb.13 for ; Tue, 08 Jun 2021 14:48:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=Ca/qZaiknXj3Iq9ZLm3iYeH+fivlx7sz5NKgsuoyOb4=; b=FucUvvJFCa05/fx+2iKbRyVJpDUoSdRufaojOSwRTffpsdVFvbdf/J9dXzxOAmTCGK m0Z3BztkAnyYMsLiTRVsqHq1OXRbF6kIZkTtKPpx0QwTYLaF+dswbCCEq+ZWNjJmQ+m0 uuI4gku9wm37sKM0u7PbfqYSD4ol8UUT2aRAA4uMCbaM41+UXNi8wmOZvO42vej34B6G Z1N1DHhgMcBAjElxU0oU0lWQd7Isg8O1ff3iAQDBJ02sumAPfBUMYOpMXVjWfl1GWUJ1 ZxEZ3e7RP+EBvHIHvVRd1m0jn113P0YM7qrkBwukO4eMSu/bqDqfpm+/NKlijZ+MWW63 zYrA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=Ca/qZaiknXj3Iq9ZLm3iYeH+fivlx7sz5NKgsuoyOb4=; b=eYGvYToibIuxMp/GuwaMsihk+wL4nfXABTBMSSHBkWETsXNLALsyDmBWG9EwHEwrpk xjD4Urqe37hOgb8jkkbATsaJQDdW46HySp6mdKInr2n1ovatL6FmlNjzELaHBDF6TlA3 qZR7lTBpq8IIipuFhxPDvbzn0NkS7DcIebqA3npZpLJiEj3JT7Byi3tYCvxbrfSx0l3l 2nA+S8zILF7ZhaYFKuQBt0QoD5j3r/l9evc2wHxkxPRaDoVied4t3+fB9qwc95l6k/w0 lWVpWz3HzO1YNrbCSbIrcptTDH22H67yIo5NzroXU7GpR80XzlY9qRCrulJNDRg0YYbH lRDQ== X-Gm-Message-State: AOAM5333LrchCL63Dxnc3HJCzXHDD6wn+CJDvNTq65YQxNaFMHzCcc+E HXPaLlX/jx+FxMnNqEOTfcV8Xk1zuUmp5eFtGfEYw6CX1EOtm6CvUDwUbauijyy6Jtkr3/MwMQs D/aE04MUbuUdNHItyXl+PSpF7Vr2KMWmE7taJA5g6tTzOG/gxFZeQfaCFzw== X-Google-Smtp-Source: ABdhPJyOt0TfQ9WWojPwzmuw0t6yU1SRclBJiOl4Nkj1pFOEutp+3uVpbmEDpDFTcOwrVm5OIBfDBOZQmlY= X-Received: from oupton.c.googlers.com ([fda3:e722:ac3:10:2b:ff92:c0a8:404]) (user=oupton job=sendgmr) by 2002:ad4:4241:: with SMTP id l1mr2485084qvq.2.1623188883854; Tue, 08 Jun 2021 14:48:03 -0700 (PDT) Date: Tue, 8 Jun 2021 21:47:34 +0000 In-Reply-To: <20210608214742.1897483-1-oupton@google.com> Message-Id: <20210608214742.1897483-3-oupton@google.com> Mime-Version: 1.0 References: <20210608214742.1897483-1-oupton@google.com> X-Mailer: git-send-email 2.32.0.rc1.229.g3e70b5a671-goog Subject: [PATCH 02/10] KVM: arm64: Implement initial support for KVM_CAP_SYSTEM_COUNTER_STATE From: Oliver Upton To: kvm@vger.kernel.org, kvmarm@lists.cs.columbia.edu Cc: Paolo Bonzini , Sean Christopherson , Marc Zyngier , Peter Shier , Jim Mattson , David Matlack , Ricardo Koller , Jing Zhang , Raghavendra Rao Anata , Oliver Upton Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org ARMv8 provides for a virtual counter-timer offset that is added to guest views of the virtual counter-timer (CNTVOFF_EL2). To date, KVM has not provided userspace with any perception of this, and instead affords a value-based scheme of migrating the virtual counter-timer by directly reading/writing the guest's CNTVCT_EL0. This is problematic because counters continue to elapse while the register is being written, meaning it is possible for drift to sneak in to the guest's time scale. This is exacerbated by the fact that KVM will calculate an appropriate CNTVOFF_EL2 every time the register is written, which will be broadcast to all virtual CPUs. The only possible way to avoid causing guest time to drift is to restore counter-timers by offset. Implement initial support for KVM_{GET,SET}_SYSTEM_COUNTER_STATE ioctls to migrate the value of CNTVOFF_EL2. These ioctls yield precise control of the virtual counter-timers to userspace, allowing it to define its own heuristics for managing vCPU offsets. Reviewed-by: Jim Mattson Reviewed-by: Jing Zhang Reviewed-by: Peter Shier Signed-off-by: Oliver Upton --- arch/arm64/include/asm/kvm_host.h | 5 +++++ arch/arm64/include/uapi/asm/kvm.h | 10 ++++++++++ arch/arm64/kvm/arch_timer.c | 22 ++++++++++++++++++++++ arch/arm64/kvm/arm.c | 25 +++++++++++++++++++++++++ 4 files changed, 62 insertions(+) diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index 7cd7d5c8c4bc..31107d5e61af 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -781,4 +781,9 @@ void __init kvm_hyp_reserve(void); static inline void kvm_hyp_reserve(void) { } #endif +int kvm_arm_vcpu_get_system_counter_state(struct kvm_vcpu *vcpu, + struct kvm_system_counter_state *state); +int kvm_arm_vcpu_set_system_counter_state(struct kvm_vcpu *vcpu, + struct kvm_system_counter_state *state); + #endif /* __ARM64_KVM_HOST_H__ */ diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h index 24223adae150..d3987089c524 100644 --- a/arch/arm64/include/uapi/asm/kvm.h +++ b/arch/arm64/include/uapi/asm/kvm.h @@ -184,6 +184,16 @@ struct kvm_vcpu_events { __u32 reserved[12]; }; +/* for KVM_{GET,SET}_SYSTEM_COUNTER_STATE */ +struct kvm_system_counter_state { + /* indicates what fields are valid in the structure */ + __u32 flags; + __u32 pad; + /* guest counter-timer offset, relative to host cntpct_el0 */ + __u64 cntvoff; + __u64 rsvd[7]; +}; + /* If you need to interpret the index values, here is the key: */ #define KVM_REG_ARM_COPROC_MASK 0x000000000FFF0000 #define KVM_REG_ARM_COPROC_SHIFT 16 diff --git a/arch/arm64/kvm/arch_timer.c b/arch/arm64/kvm/arch_timer.c index 74e0699661e9..955a7a183362 100644 --- a/arch/arm64/kvm/arch_timer.c +++ b/arch/arm64/kvm/arch_timer.c @@ -1259,3 +1259,25 @@ int kvm_arm_timer_has_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr) return -ENXIO; } + +int kvm_arm_vcpu_get_system_counter_state(struct kvm_vcpu *vcpu, + struct kvm_system_counter_state *state) +{ + if (state->flags) + return -EINVAL; + + state->cntvoff = timer_get_offset(vcpu_vtimer(vcpu)); + + return 0; +} + +int kvm_arm_vcpu_set_system_counter_state(struct kvm_vcpu *vcpu, + struct kvm_system_counter_state *state) +{ + if (state->flags) + return -EINVAL; + + timer_set_offset(vcpu_vtimer(vcpu), state->cntvoff); + + return 0; +} diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index 1126eae27400..b78ffb4db9dd 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -207,6 +207,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) case KVM_CAP_SET_GUEST_DEBUG: case KVM_CAP_VCPU_ATTRIBUTES: case KVM_CAP_PTP_KVM: + case KVM_CAP_SYSTEM_COUNTER_STATE: r = 1; break; case KVM_CAP_SET_GUEST_DEBUG2: @@ -1273,6 +1274,30 @@ long kvm_arch_vcpu_ioctl(struct file *filp, return kvm_arm_vcpu_finalize(vcpu, what); } + case KVM_GET_SYSTEM_COUNTER_STATE: { + struct kvm_system_counter_state state; + + if (copy_from_user(&state, argp, sizeof(state))) + return -EFAULT; + + r = kvm_arm_vcpu_get_system_counter_state(vcpu, &state); + if (r) + break; + + if (copy_to_user(argp, &state, sizeof(state))) + return -EFAULT; + + break; + } + case KVM_SET_SYSTEM_COUNTER_STATE: { + struct kvm_system_counter_state state; + + if (copy_from_user(&state, argp, sizeof(state))) + return -EFAULT; + + r = kvm_arm_vcpu_set_system_counter_state(vcpu, &state); + break; + } default: r = -EINVAL; } From patchwork Tue Jun 8 21:47:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oliver Upton X-Patchwork-Id: 12308219 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5351AC4743F for ; Tue, 8 Jun 2021 21:49:12 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 38A7061278 for ; Tue, 8 Jun 2021 21:49:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234778AbhFHVvE (ORCPT ); Tue, 8 Jun 2021 17:51:04 -0400 Received: from mail-qk1-f202.google.com ([209.85.222.202]:50960 "EHLO mail-qk1-f202.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234727AbhFHVvD (ORCPT ); Tue, 8 Jun 2021 17:51:03 -0400 Received: by mail-qk1-f202.google.com with SMTP id n3-20020a378b030000b02903a624ca95adso15925944qkd.17 for ; Tue, 08 Jun 2021 14:49:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=p0iXV74EOKOaRQh7Ns0atnVcsDIwc2nUiv6QHU1ihI8=; b=IsOx3E0B1JEQxQJinZeEfDucbClCYwUTsO/vTYI2y5XpQnzMpN6YRzuuHdgKIYRXXM XOF4MGwTGoUXk8eucXgEmq9FGfuYsXmJrfog4fE5WXSKk9MKPgfEux5jIQmZ3HP1PEc9 pivjPO8RaqyBwXKY/q1wf200QbNBJ9H5/05RnmbBzTtQYBRwP7C5Q+Ik+XuhpCFNTIO6 InSW4Le6iK2rTy+FwpGweaGXAf7EpWA8WuNL9RMTu4uIDnJGKcHiw/aCoaZeAKZEV/A7 L5j43e64FDVmYM1cF+zlnL0furxW2Mnf+0GA0WZtP7E8X18qBt1bwdeuNk+DbLeQXrU5 eKZw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=p0iXV74EOKOaRQh7Ns0atnVcsDIwc2nUiv6QHU1ihI8=; b=VA6KDyEhb1HeQp41R4EjjkTIqZOrxBTfu9ZhKJEosNPfEZktJxg9TS3bPFgqVCzPQP Q9qYWzNDHijbWjchg5HMJCl+qvtWuphO27Z85nGW/foodyWqXGFr979s3KNyDh4Kys1j Pcx7nat6rPZKOuii6KNcotIo/cZeLmlY+bJdwAaf1s4sFDCHtjYYIyM1dI79kws40hME bJ0Wn5AePT2LFFJQoh6uzvTahaqS+aLXaAx180Ru0W5Rqt8Sd16t6Fn3k7u5ExN6+Nk+ euXmj7kzgaiIVOecJlyK+KS5TJ3wRgR0XLRM0TzbDfPbpBsOVcJlZ3HVXngxBei6yxrg AMmg== X-Gm-Message-State: AOAM530aUbSMDRiTznFkwiPJd3+SVuPVO+Be/vOcmP4cOJCFI6db3OH/ nfnQWTwQS4BMv3i2XivoyoXazKqdBMs9fOVr2v9coE1C9ZQJwgIyHIrB3zYvWLDoExxBEuxEQTx g6zuZzY3k4xHAASbh9KwQzUKlzbZJ9AfGDrSjfnRPUuksiBFgPpbLNgrIqg== X-Google-Smtp-Source: ABdhPJyDITlUaxD6OVnwR4PjuvS3tM8g/BFPOp78Wn37AtDQoat5Mk9tWPfrM3ukQg1KRwHPERtgtoaZTiY= X-Received: from oupton.c.googlers.com ([fda3:e722:ac3:10:2b:ff92:c0a8:404]) (user=oupton job=sendgmr) by 2002:a0c:b911:: with SMTP id u17mr2201627qvf.14.1623188889690; Tue, 08 Jun 2021 14:48:09 -0700 (PDT) Date: Tue, 8 Jun 2021 21:47:35 +0000 In-Reply-To: <20210608214742.1897483-1-oupton@google.com> Message-Id: <20210608214742.1897483-4-oupton@google.com> Mime-Version: 1.0 References: <20210608214742.1897483-1-oupton@google.com> X-Mailer: git-send-email 2.32.0.rc1.229.g3e70b5a671-goog Subject: [PATCH 03/10] selftests: KVM: Introduce system_counter_state_test From: Oliver Upton To: kvm@vger.kernel.org, kvmarm@lists.cs.columbia.edu Cc: Paolo Bonzini , Sean Christopherson , Marc Zyngier , Peter Shier , Jim Mattson , David Matlack , Ricardo Koller , Jing Zhang , Raghavendra Rao Anata , Oliver Upton Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Test that the KVM_{GET,SET}_SYSTEM_COUNTER_STATE ioctls correctly configure the guest's view of the virtual counter-timer (CNTVCT_EL0). Reviewed-by: Jing Zhang Reviewed-by: Jim Mattson Reviewed-by: David Matlack Reviewed-by: Peter Shier Reviewed-by: Ricardo Koller Signed-off-by: Oliver Upton --- tools/testing/selftests/kvm/.gitignore | 1 + tools/testing/selftests/kvm/Makefile | 1 + .../selftests/kvm/include/aarch64/processor.h | 24 +++ .../selftests/kvm/system_counter_state_test.c | 199 ++++++++++++++++++ 4 files changed, 225 insertions(+) create mode 100644 tools/testing/selftests/kvm/system_counter_state_test.c diff --git a/tools/testing/selftests/kvm/.gitignore b/tools/testing/selftests/kvm/.gitignore index bd83158e0e0b..1a5782d8a0d4 100644 --- a/tools/testing/selftests/kvm/.gitignore +++ b/tools/testing/selftests/kvm/.gitignore @@ -34,6 +34,7 @@ /x86_64/xen_vmcall_test /x86_64/xss_msr_test /x86_64/vmx_pmu_msrs_test +/system_counter_state_test /demand_paging_test /dirty_log_test /dirty_log_perf_test diff --git a/tools/testing/selftests/kvm/Makefile b/tools/testing/selftests/kvm/Makefile index e439d027939d..b14f16dc954a 100644 --- a/tools/testing/selftests/kvm/Makefile +++ b/tools/testing/selftests/kvm/Makefile @@ -80,6 +80,7 @@ TEST_GEN_PROGS_x86_64 += steal_time TEST_GEN_PROGS_aarch64 += aarch64/get-reg-list TEST_GEN_PROGS_aarch64 += aarch64/get-reg-list-sve TEST_GEN_PROGS_aarch64 += aarch64/vgic_init +TEST_GEN_PROGS_aarch64 += system_counter_state_test TEST_GEN_PROGS_aarch64 += demand_paging_test TEST_GEN_PROGS_aarch64 += dirty_log_test TEST_GEN_PROGS_aarch64 += dirty_log_perf_test diff --git a/tools/testing/selftests/kvm/include/aarch64/processor.h b/tools/testing/selftests/kvm/include/aarch64/processor.h index b7fa0c8551db..48c964ce62ff 100644 --- a/tools/testing/selftests/kvm/include/aarch64/processor.h +++ b/tools/testing/selftests/kvm/include/aarch64/processor.h @@ -52,6 +52,30 @@ static inline void set_reg(struct kvm_vm *vm, uint32_t vcpuid, uint64_t id, uint vcpu_ioctl(vm, vcpuid, KVM_SET_ONE_REG, ®); } +static inline uint64_t read_cntpct_ordered(void) +{ + uint64_t r; + + asm volatile("isb\n\t" + "mrs %0, cntpct_el0\n\t" + "isb\n\t" + : "=r"(r)); + + return r; +} + +static inline uint64_t read_cntvct_ordered(void) +{ + uint64_t r; + + asm volatile("isb\n\t" + "mrs %0, cntvct_el0\n\t" + "isb\n\t" + : "=r"(r)); + + return r; +} + void aarch64_vcpu_setup(struct kvm_vm *vm, int vcpuid, struct kvm_vcpu_init *init); void aarch64_vcpu_add_default(struct kvm_vm *vm, uint32_t vcpuid, struct kvm_vcpu_init *init, void *guest_code); diff --git a/tools/testing/selftests/kvm/system_counter_state_test.c b/tools/testing/selftests/kvm/system_counter_state_test.c new file mode 100644 index 000000000000..059971f6cb87 --- /dev/null +++ b/tools/testing/selftests/kvm/system_counter_state_test.c @@ -0,0 +1,199 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * system_counter_state_test.c -- suite of tests for correctness of + * KVM_{GET,SET}_SYSTEM_COUNTER_STATE ioctls. + * + * Copyright (c) 2021, Google LLC. + */ +#define _GNU_SOURCE +#include +#include +#include + +#include "kvm_util.h" +#include "processor.h" +#include "test_util.h" + +#define VCPU_ID 0 + +#ifdef __aarch64__ + +enum counter { + VIRTUAL, + PHYSICAL, +}; + +static char *counter_name(enum counter counter) +{ + switch (counter) { + case VIRTUAL: + return "virtual"; + case PHYSICAL: + return "physical"; + default: + TEST_ASSERT(false, "unrecognized counter: %d", counter); + } + + /* never reached */ + return NULL; +} + +struct system_counter_state_test { + enum counter counter; + struct kvm_system_counter_state state; +}; + +static struct system_counter_state_test test_cases[] = { + { + .counter = VIRTUAL, + .state = { + .cntvoff = 0 + } + }, + { + .counter = VIRTUAL, + .state = { + .cntvoff = 1000000 + } + }, + { + .counter = VIRTUAL, + .state = { + .cntvoff = -1 + } + }, +}; + +static void pr_test(struct system_counter_state_test *test) +{ + pr_info("counter: %s, cntvoff: %lld\n", counter_name(test->counter), test->state.cntvoff); +} + +static struct kvm_system_counter_state * +get_system_counter_state(struct system_counter_state_test *test) +{ + return &test->state; +} + +/* + * Reads the guest counter-timer under test. + */ +static uint64_t guest_read_counter(struct system_counter_state_test *test) +{ + switch (test->counter) { + case PHYSICAL: + return read_cntpct_ordered(); + case VIRTUAL: + return read_cntvct_ordered(); + default: + GUEST_ASSERT(0); + } + + /* never reached */ + return -1; +} + +/* + * Reads the host physical counter-timer and transforms it into a guest value + * according to the kvm_system_counter_state structure. + */ +static uint64_t host_read_guest_counter(struct system_counter_state_test *test) +{ + uint64_t r; + + r = read_cntvct_ordered(); + switch (test->counter) { + case VIRTUAL: + r -= test->state.cntvoff; + break; + default: + TEST_ASSERT(false, "unrecognized counter: %d", test->counter); + } + + return r; +} + +#else +#error test not implemented for architecture being built! +#endif + +static void guest_main(void) +{ + int i; + + for (i = 0; i < ARRAY_SIZE(test_cases); i++) { + struct system_counter_state_test *test = &test_cases[i]; + + GUEST_SYNC(guest_read_counter(test)); + } + + GUEST_DONE(); +} + +static bool enter_guest(struct kvm_vm *vm, uint64_t *guest_counter) +{ + struct ucall uc; + + vcpu_ioctl(vm, VCPU_ID, KVM_RUN, NULL); + + switch (get_ucall(vm, VCPU_ID, &uc)) { + case UCALL_DONE: + return true; + case UCALL_SYNC: + if (guest_counter) + *guest_counter = uc.args[1]; + break; + case UCALL_ABORT: + TEST_ASSERT(false, "%s at %s:%ld", (const char *)uc.args[0], + __FILE__, uc.args[1]); + break; + default: + TEST_ASSERT(false, "unexpected exit: %s", + exit_reason_str(vcpu_state(vm, VCPU_ID)->exit_reason)); + break; + } + + /* more work to do in the guest */ + return false; +} + +static void run_tests(struct kvm_vm *vm) +{ + int i; + + for (i = 0; i < ARRAY_SIZE(test_cases); i++) { + struct system_counter_state_test *test = &test_cases[i]; + uint64_t start, end, obs; + + pr_info("test %d: ", i); + pr_test(test); + + vcpu_ioctl(vm, VCPU_ID, KVM_SET_SYSTEM_COUNTER_STATE, + get_system_counter_state(test)); + + start = host_read_guest_counter(test); + TEST_ASSERT(!enter_guest(vm, &obs), "guest completed unexpectedly"); + end = host_read_guest_counter(test); + + TEST_ASSERT(start < obs && obs < end, + "guest counter value (%ld) outside expected bounds: (%ld, %ld)", + obs, start, end); + } + + TEST_ASSERT(enter_guest(vm, NULL), "guest didn't run to completion"); +} + +int main(void) +{ + struct kvm_vm *vm; + + if (!kvm_check_cap(KVM_CAP_SYSTEM_COUNTER_STATE)) { + print_skip("KVM_CAP_SYSTEM_COUNTER_STATE not supported"); + exit(KSFT_SKIP); + } + + vm = vm_create_default(0, 0, guest_main); + ucall_init(vm, NULL); + run_tests(vm); + kvm_vm_free(vm); +} From patchwork Tue Jun 8 21:47:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oliver Upton X-Patchwork-Id: 12308207 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4A15CC4743D for ; Tue, 8 Jun 2021 21:48:29 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1F91961278 for ; Tue, 8 Jun 2021 21:48:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234605AbhFHVuV (ORCPT ); Tue, 8 Jun 2021 17:50:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44042 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234064AbhFHVuV (ORCPT ); Tue, 8 Jun 2021 17:50:21 -0400 Received: from mail-yb1-xb4a.google.com (mail-yb1-xb4a.google.com [IPv6:2607:f8b0:4864:20::b4a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 90C91C061574 for ; Tue, 8 Jun 2021 14:48:15 -0700 (PDT) Received: by mail-yb1-xb4a.google.com with SMTP id v28-20020a25fc1c0000b0290547fac9371fso6251273ybd.14 for ; Tue, 08 Jun 2021 14:48:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=jq/+BcyiAwtDYy8ThiCBLyJ5XHWSBXaMbOIVg/4GVn8=; b=ckg85lAIZmxZUdlRzWIhhAAZPBFTL/aTLpCURVBL7uHEuwqOX3JpYbHy9HaqxL4Sxk kyJfoHbNbjYa25k4xE4ZcoWNEm+xHOVn2T25WNbTZwuTYvkJngvHV3psTiIQI0dyHjEv 3dYmyHYF1988m9d9TNyHVt+Xc+eVFMTlmDqbnstCfX/jej21f6y9fr1RISeEQ3tEMbzj BHuvoJ/MoF+kgt/iAr00V7FbpAB9uUuTv8XmPE2YkHGb3AghryYgDg6jdzdcKm9MVWoQ Wve0yu6vB4RJOnFydwlIqFTxHelNRU1hzhatay/6Hqw9HWcnZTiIk9vstjboyEkTORuk wa/Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=jq/+BcyiAwtDYy8ThiCBLyJ5XHWSBXaMbOIVg/4GVn8=; b=EM8fg3Fa0Iy7/pMRyG02Vts3qabIUFHviobzRzxgK5fi34cZaoETiqSli/sWIRpiKP ZP56Ed8w6GEaWxkVReffEb7FWieohMJMPq5WS5scObof7qUyCOczIY+higL/+GlEPq11 FNZ0vDbGx3UgSR/YHbmtd6BYRo8lbDAuaw4fi52pel6YCoSnQ6kR9RDB1YMgRA1dYmxX WFWcWjnKWo4Ku3NJDB12+UJ4Kfz6xYAJDLOFgkGiodFurVKGzzRIB2mVwEZ+aC8Sk8Oh kp6JdHgMa5EhvrObPBLB9hLnRyPcGoaAAW3ZDplxYC8fy4o/qXx7eodXqeqUYAJBrxDM eGtQ== X-Gm-Message-State: AOAM532HZbQN1veMe5icIv6pMk43zb1xUCkMZcuioIb5Ge0SR0L82g8O cia4BXgp043YPUIHMIYsp/T5mmAYhDU5FCNZHQXU1dv4yk2NtAw/8KQpcjgInwpHHzvP5YCBidt 0LLV510jtnVql4EwYe49aXSvkO0P6kLE6qUDyryjYWPeZTB7nVurMlWG0HA== X-Google-Smtp-Source: ABdhPJzajH43W3nB1u/QnWpuQy2rJw6clRNnWk4ESe01ais70W6jI3PIWW8SpRyxIKWNG40paVp6mpQJOVc= X-Received: from oupton.c.googlers.com ([fda3:e722:ac3:10:2b:ff92:c0a8:404]) (user=oupton job=sendgmr) by 2002:a5b:303:: with SMTP id j3mr33062599ybp.433.1623188894702; Tue, 08 Jun 2021 14:48:14 -0700 (PDT) Date: Tue, 8 Jun 2021 21:47:36 +0000 In-Reply-To: <20210608214742.1897483-1-oupton@google.com> Message-Id: <20210608214742.1897483-5-oupton@google.com> Mime-Version: 1.0 References: <20210608214742.1897483-1-oupton@google.com> X-Mailer: git-send-email 2.32.0.rc1.229.g3e70b5a671-goog Subject: [PATCH 04/10] KVM: arm64: Add userspace control of the guest's physical counter From: Oliver Upton To: kvm@vger.kernel.org, kvmarm@lists.cs.columbia.edu Cc: Paolo Bonzini , Sean Christopherson , Marc Zyngier , Peter Shier , Jim Mattson , David Matlack , Ricardo Koller , Jing Zhang , Raghavendra Rao Anata , Oliver Upton Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org ARMv8.6 adds an extension to the architecture providing hypervisors with more extensive controls of the guest's counters. A particularly interesting control is CNTPOFF_EL2, a fixed offset subtracted from the physical counter value to derive the guest's value. VMMs that live migrate their guests may be particularly interested in this feature in order to provide a consistent view of the physical counter across live migrations. In the interim, KVM can emulate this behavior by simply enabling traps on CNTPCT_EL0 and subtracting an offset. Add a new field to kvm_system_counter_state allowing a VMM to request an offset to the physical counter. If this offset is nonzero, enable traps on CNTPCT_EL0. Emulate guest reads to the register in the fast path to keep counter reads reasonably performant, avoiding a full exit from the guest. Reviewed-by: Peter Shier Reviewed-by: Ricardo Koller Signed-off-by: Oliver Upton --- arch/arm64/include/asm/kvm_host.h | 1 + arch/arm64/include/asm/sysreg.h | 1 + arch/arm64/include/uapi/asm/kvm.h | 9 +++- arch/arm64/kvm/arch_timer.c | 66 +++++++++++++++++++++++-- arch/arm64/kvm/hyp/include/hyp/switch.h | 31 ++++++++++++ arch/arm64/kvm/hyp/nvhe/timer-sr.c | 16 ++++-- 6 files changed, 117 insertions(+), 7 deletions(-) diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index 31107d5e61af..a3abafcea328 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -200,6 +200,7 @@ enum vcpu_sysreg { SP_EL1, SPSR_EL1, + CNTPOFF_EL2, CNTVOFF_EL2, CNTV_CVAL_EL0, CNTV_CTL_EL0, diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h index 65d15700a168..193da426690a 100644 --- a/arch/arm64/include/asm/sysreg.h +++ b/arch/arm64/include/asm/sysreg.h @@ -505,6 +505,7 @@ #define SYS_AMEVCNTR0_MEM_STALL SYS_AMEVCNTR0_EL0(3) #define SYS_CNTFRQ_EL0 sys_reg(3, 3, 14, 0, 0) +#define SYS_CNTPCT_EL0 sys_reg(3, 3, 14, 0, 1) #define SYS_CNTP_TVAL_EL0 sys_reg(3, 3, 14, 2, 0) #define SYS_CNTP_CTL_EL0 sys_reg(3, 3, 14, 2, 1) diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h index d3987089c524..ee709e2f0292 100644 --- a/arch/arm64/include/uapi/asm/kvm.h +++ b/arch/arm64/include/uapi/asm/kvm.h @@ -184,6 +184,8 @@ struct kvm_vcpu_events { __u32 reserved[12]; }; +#define KVM_SYSTEM_COUNTER_STATE_PHYS_OFFSET (1ul << 0) + /* for KVM_{GET,SET}_SYSTEM_COUNTER_STATE */ struct kvm_system_counter_state { /* indicates what fields are valid in the structure */ @@ -191,7 +193,12 @@ struct kvm_system_counter_state { __u32 pad; /* guest counter-timer offset, relative to host cntpct_el0 */ __u64 cntvoff; - __u64 rsvd[7]; + /* + * Guest physical counter-timer offset, relative to host cntpct_el0. + * Valid when KVM_SYSTEM_COUNTER_STATE_PHYS_OFFSET is set. + */ + __u64 cntpoff; + __u64 rsvd[6]; }; /* If you need to interpret the index values, here is the key: */ diff --git a/arch/arm64/kvm/arch_timer.c b/arch/arm64/kvm/arch_timer.c index 955a7a183362..a74642d1515f 100644 --- a/arch/arm64/kvm/arch_timer.c +++ b/arch/arm64/kvm/arch_timer.c @@ -50,6 +50,7 @@ static void kvm_arm_timer_write(struct kvm_vcpu *vcpu, static u64 kvm_arm_timer_read(struct kvm_vcpu *vcpu, struct arch_timer_context *timer, enum kvm_arch_timer_regs treg); +static bool kvm_timer_emulation_required(struct arch_timer_context *ctx); u32 timer_get_ctl(struct arch_timer_context *ctxt) { @@ -86,6 +87,8 @@ static u64 timer_get_offset(struct arch_timer_context *ctxt) struct kvm_vcpu *vcpu = ctxt->vcpu; switch(arch_timer_ctx_index(ctxt)) { + case TIMER_PTIMER: + return __vcpu_sys_reg(vcpu, CNTPOFF_EL2); case TIMER_VTIMER: return __vcpu_sys_reg(vcpu, CNTVOFF_EL2); default: @@ -130,6 +133,9 @@ static void timer_set_offset(struct arch_timer_context *ctxt, u64 offset) struct kvm_vcpu *vcpu = ctxt->vcpu; switch(arch_timer_ctx_index(ctxt)) { + case TIMER_PTIMER: + __vcpu_sys_reg(vcpu, CNTPOFF_EL2) = offset; + break; case TIMER_VTIMER: __vcpu_sys_reg(vcpu, CNTVOFF_EL2) = offset; break; @@ -145,7 +151,7 @@ u64 kvm_phys_timer_read(void) static void get_timer_map(struct kvm_vcpu *vcpu, struct timer_map *map) { - if (has_vhe()) { + if (has_vhe() && !kvm_timer_emulation_required(vcpu_ptimer(vcpu))) { map->direct_vtimer = vcpu_vtimer(vcpu); map->direct_ptimer = vcpu_ptimer(vcpu); map->emul_ptimer = NULL; @@ -746,6 +752,30 @@ int kvm_timer_vcpu_reset(struct kvm_vcpu *vcpu) return 0; } +bool kvm_timer_emulation_required(struct arch_timer_context *ctx) +{ + int idx = arch_timer_ctx_index(ctx); + + switch (idx) { + /* + * hardware doesn't support offsetting of the physical counter/timer. + * If offsetting is requested, enable emulation of the physical + * counter/timer. + */ + case TIMER_PTIMER: + return timer_get_offset(ctx); + /* + * Conversely, hardware does support offsetting of the virtual + * counter/timer. + */ + case TIMER_VTIMER: + return false; + default: + WARN_ON(1); + return false; + } +} + /* Make the updates of cntvoff for all vtimer contexts atomic */ static void update_vtimer_cntvoff(struct kvm_vcpu *vcpu, u64 cntvoff) { @@ -1184,6 +1214,24 @@ void kvm_timer_init_vhe(void) write_sysreg(val, cnthctl_el2); } +static void kvm_timer_update_traps_vhe(struct kvm_vcpu *vcpu) +{ + u32 cnthctl_shift = 10; + u64 val; + + if (!kvm_timer_emulation_required(vcpu_ptimer(vcpu))) + return; + + /* + * We must trap accesses to the physical counter/timer to emulate the + * nonzero offset. + */ + val = read_sysreg(cnthctl_el2); + val &= ~(CNTHCTL_EL1PCEN << cnthctl_shift); + val &= ~(CNTHCTL_EL1PCTEN << cnthctl_shift); + write_sysreg(val, cnthctl_el2); +} + static void set_timer_irqs(struct kvm *kvm, int vtimer_irq, int ptimer_irq) { struct kvm_vcpu *vcpu; @@ -1260,24 +1308,36 @@ int kvm_arm_timer_has_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr) return -ENXIO; } +#define KVM_SYSTEM_COUNTER_STATE_VALID_FLAG_BITS \ + (KVM_SYSTEM_COUNTER_STATE_PHYS_OFFSET) + int kvm_arm_vcpu_get_system_counter_state(struct kvm_vcpu *vcpu, struct kvm_system_counter_state *state) { - if (state->flags) + if (state->flags & ~KVM_SYSTEM_COUNTER_STATE_VALID_FLAG_BITS) return -EINVAL; state->cntvoff = timer_get_offset(vcpu_vtimer(vcpu)); + if (state->flags & KVM_SYSTEM_COUNTER_STATE_PHYS_OFFSET) + state->cntpoff = timer_get_offset(vcpu_ptimer(vcpu)); + return 0; } int kvm_arm_vcpu_set_system_counter_state(struct kvm_vcpu *vcpu, struct kvm_system_counter_state *state) { - if (state->flags) + if (state->flags & ~KVM_SYSTEM_COUNTER_STATE_VALID_FLAG_BITS) return -EINVAL; timer_set_offset(vcpu_vtimer(vcpu), state->cntvoff); + if (state->flags & KVM_SYSTEM_COUNTER_STATE_PHYS_OFFSET) + timer_set_offset(vcpu_ptimer(vcpu), state->cntpoff); + + if (has_vhe()) + kvm_timer_update_traps_vhe(vcpu); + return 0; } diff --git a/arch/arm64/kvm/hyp/include/hyp/switch.h b/arch/arm64/kvm/hyp/include/hyp/switch.h index e4a2f295a394..12ada31e12e2 100644 --- a/arch/arm64/kvm/hyp/include/hyp/switch.h +++ b/arch/arm64/kvm/hyp/include/hyp/switch.h @@ -287,6 +287,30 @@ static inline bool __hyp_handle_fpsimd(struct kvm_vcpu *vcpu) return true; } +static inline u64 __hyp_read_cntpct(struct kvm_vcpu *vcpu) +{ + return read_sysreg(cntpct_el0) - __vcpu_sys_reg(vcpu, CNTPOFF_EL2); +} + +static inline bool __hyp_handle_counter(struct kvm_vcpu *vcpu) +{ + u32 sysreg = esr_sys64_to_sysreg(kvm_vcpu_get_esr(vcpu)); + int rt = kvm_vcpu_sys_get_rt(vcpu); + u64 rv; + + switch (sysreg) { + case SYS_CNTPCT_EL0: + rv = __hyp_read_cntpct(vcpu); + break; + default: + return false; + } + + vcpu_set_reg(vcpu, rt, rv); + __kvm_skip_instr(vcpu); + return true; +} + static inline bool handle_tx2_tvm(struct kvm_vcpu *vcpu) { u32 sysreg = esr_sys64_to_sysreg(kvm_vcpu_get_esr(vcpu)); @@ -439,6 +463,13 @@ static inline bool fixup_guest_exit(struct kvm_vcpu *vcpu, u64 *exit_code) if (*exit_code != ARM_EXCEPTION_TRAP) goto exit; + /* + * We trap acesses to the physical counter value register (CNTPCT_EL0) + * if userspace has requested a physical counter offset. + */ + if (__hyp_handle_counter(vcpu)) + goto guest; + if (cpus_have_final_cap(ARM64_WORKAROUND_CAVIUM_TX2_219_TVM) && kvm_vcpu_trap_get_class(vcpu) == ESR_ELx_EC_SYS64 && handle_tx2_tvm(vcpu)) diff --git a/arch/arm64/kvm/hyp/nvhe/timer-sr.c b/arch/arm64/kvm/hyp/nvhe/timer-sr.c index 9072e71693ba..1b8e6e47a4ea 100644 --- a/arch/arm64/kvm/hyp/nvhe/timer-sr.c +++ b/arch/arm64/kvm/hyp/nvhe/timer-sr.c @@ -35,14 +35,24 @@ void __timer_disable_traps(struct kvm_vcpu *vcpu) */ void __timer_enable_traps(struct kvm_vcpu *vcpu) { - u64 val; + u64 val, cntpoff; + + cntpoff = __vcpu_sys_reg(vcpu, CNTPOFF_EL2); /* * Disallow physical timer access for the guest - * Physical counter access is allowed */ val = read_sysreg(cnthctl_el2); val &= ~CNTHCTL_EL1PCEN; - val |= CNTHCTL_EL1PCTEN; + + /* + * Disallow physical counter access for the guest if offsetting is + * requested. + */ + if (cntpoff) + val &= ~CNTHCTL_EL1PCTEN; + else + val |= CNTHCTL_EL1PCTEN; + write_sysreg(val, cnthctl_el2); } From patchwork Tue Jun 8 21:47:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oliver Upton X-Patchwork-Id: 12308221 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8A7BBC47082 for ; Tue, 8 Jun 2021 21:49:36 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 55608613AE for ; Tue, 8 Jun 2021 21:49:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234768AbhFHVv3 (ORCPT ); Tue, 8 Jun 2021 17:51:29 -0400 Received: from mail-yb1-f201.google.com ([209.85.219.201]:55996 "EHLO mail-yb1-f201.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234752AbhFHVv2 (ORCPT ); Tue, 8 Jun 2021 17:51:28 -0400 Received: by mail-yb1-f201.google.com with SMTP id m194-20020a2526cb0000b02905375d41acd7so28707370ybm.22 for ; Tue, 08 Jun 2021 14:49:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=qXkSbOYb2IC3X2Tl5AKB9URDdm1XI6BUL14Xin2r7eg=; b=F9podaJObKk/txBttzcQCUMN3bmiUkX6RaROZiUt4OyZK4vESkbOf4/II5yKFotHVQ alHKMpjv3ojhTmGJfKR0t8Pyc3bB+pnSBywsc97WcltB+In8AABdaquDC6LG0mdDRJni uH97XjL/yCGjxozIBw5JWghGNQXCuEsCu9BK04G/h9CcBweO7u09GJQdL0rl/obTCjUz dgck6LWM7TZyL78hf1fLz0pprF8fjd7LbZBlbaBcNXq8Uxv6B5cctb1GH046oxmztpJ3 xwTLXOmKNCU36JKeHjq8101RrY8QnGF03HpMmuFmQFoASYx3qaG24I2W95/80lYrhHBd YrpQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=qXkSbOYb2IC3X2Tl5AKB9URDdm1XI6BUL14Xin2r7eg=; b=s9nGEF8md3WR53Hkw7n3AoydFDu4L+bCEsNt4MF+DclS2JHY5ye8lFgNyQQTsrZjeg ZfQxyNwJT7xsPrE665d/j6cJLqvj7gWgAed3Dap5twKy15nBNkDuV+IczNVjU68JWX/6 Ka8GCn569LAhclfDq2NmqbVAw4juJW7nz0cCwa9++IzMrH3v4uH+OASDh4wSAq70ER15 M38wu8RopNkAcbOy0JQ4WjYZ+2dn4RxkL0v87T9NrmlXSQ43SdO0bVM7OD2nOptRZI5X hR+Wa/jQl+5XetE1urAp5+tDcyhoevTgKLg7E641ZWAulGJZz+OWgrp4KW7/OnmPf8JB jkLQ== X-Gm-Message-State: AOAM532YS3iadrCBZCODjRnCZTZ+MFlJzU80pAdsyfWJcQLBGxHUM+mb Q22+nmBsWZmyZGQip06+B5ENO8iIbcm1z1CNcXjyIoHyTZSz46QAslAs8UYDLrvDTb1LRl1Arh2 m0QbI6QjDS8A2hOTm53yBDXHnQibJxwIUobicKn39r6FMxw5BdI3MuFyl/g== X-Google-Smtp-Source: ABdhPJxckUBKZ1i6dYMHyxVrr1ezG9WhmAhLTg+a5+kWk0J+vJAtGDvqW3iEOXVnlBNIdoUd9FtMux/VlVo= X-Received: from oupton.c.googlers.com ([fda3:e722:ac3:10:2b:ff92:c0a8:404]) (user=oupton job=sendgmr) by 2002:a25:9bc4:: with SMTP id w4mr32208884ybo.168.1623188899905; Tue, 08 Jun 2021 14:48:19 -0700 (PDT) Date: Tue, 8 Jun 2021 21:47:37 +0000 In-Reply-To: <20210608214742.1897483-1-oupton@google.com> Message-Id: <20210608214742.1897483-6-oupton@google.com> Mime-Version: 1.0 References: <20210608214742.1897483-1-oupton@google.com> X-Mailer: git-send-email 2.32.0.rc1.229.g3e70b5a671-goog Subject: [PATCH 05/10] selftests: KVM: Add test cases for physical counter offsetting From: Oliver Upton To: kvm@vger.kernel.org, kvmarm@lists.cs.columbia.edu Cc: Paolo Bonzini , Sean Christopherson , Marc Zyngier , Peter Shier , Jim Mattson , David Matlack , Ricardo Koller , Jing Zhang , Raghavendra Rao Anata , Oliver Upton Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Add support for physical counter offsetting to counter_state_test. Assert that guest reads of the physical counter are within the expected bounds. Reviewed-by: Peter Shier Reviewed-by: Ricardo Koller Signed-off-by: Oliver Upton --- .../selftests/kvm/system_counter_state_test.c | 27 ++++++++++++++++++- 1 file changed, 26 insertions(+), 1 deletion(-) diff --git a/tools/testing/selftests/kvm/system_counter_state_test.c b/tools/testing/selftests/kvm/system_counter_state_test.c index 059971f6cb87..f537eb7b928c 100644 --- a/tools/testing/selftests/kvm/system_counter_state_test.c +++ b/tools/testing/selftests/kvm/system_counter_state_test.c @@ -62,11 +62,34 @@ static struct system_counter_state_test test_cases[] = { .cntvoff = -1 } }, + { + .counter = PHYSICAL, + .state = { + .flags = KVM_SYSTEM_COUNTER_STATE_PHYS_OFFSET, + .cntpoff = 0 + } + }, + { + .counter = PHYSICAL, + .state = { + .flags = KVM_SYSTEM_COUNTER_STATE_PHYS_OFFSET, + .cntpoff = 1000000 + } + }, + { + .counter = PHYSICAL, + .state = { + .flags = KVM_SYSTEM_COUNTER_STATE_PHYS_OFFSET, + .cntpoff = -1 + } + }, }; static void pr_test(struct system_counter_state_test *test) { - pr_info("counter: %s, cntvoff: %lld\n", counter_name(test->counter), test->state.cntvoff); + pr_info("counter: %s, cntvoff: %lld, cntpoff: %lld\n", + counter_name(test->counter), test->state.cntvoff, + test->state.cntpoff); } static struct kvm_system_counter_state * @@ -103,6 +126,8 @@ static uint64_t host_read_guest_counter(struct system_counter_state_test *test) r = read_cntvct_ordered(); switch (test->counter) { + case PHYSICAL: + r -= test->state.cntpoff; case VIRTUAL: r -= test->state.cntvoff; break; From patchwork Tue Jun 8 21:47:38 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oliver Upton X-Patchwork-Id: 12308205 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3CC75C47082 for ; Tue, 8 Jun 2021 21:48:27 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1ECF0613B6 for ; Tue, 8 Jun 2021 21:48:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234501AbhFHVuT (ORCPT ); Tue, 8 Jun 2021 17:50:19 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44080 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234064AbhFHVuT (ORCPT ); Tue, 8 Jun 2021 17:50:19 -0400 Received: from mail-il1-x149.google.com (mail-il1-x149.google.com [IPv6:2607:f8b0:4864:20::149]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C56CDC061787 for ; Tue, 8 Jun 2021 14:48:25 -0700 (PDT) Received: by mail-il1-x149.google.com with SMTP id g14-20020a926b0e0000b02901bb2deb9d71so16221876ilc.6 for ; Tue, 08 Jun 2021 14:48:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=VJQxNxTePlfcf3dKFi1wZt07MNFX3dohgtzce5ctPa8=; b=iT2LSgpulAl5w53RB3CBjKdDjKgT1rjoyLj64YQHdNGhEeD+nIa19wz3vII3s98KAG VLJPGgncFNePZwCtyquAJe14uE17M/dcFUyntRJiXhmzOwvp2vbPHqx3DoPsiSS0/aBu 5pzm4FEvMZM5TCzuwrH+bZI6lFUwJX5SivK+C47q1Y8PyupQC20hP5IXgdkuHMiAtNAX t5556xcRrXS++W4ZoifRWftNJ1ADUQPup1hQnFmAnDChj2/0q861ilHIX3w6zW64Ab9B g7xo34HbT5lS+qnq8ITh+Na97cPNDjempspB8bduXonZle5P1Rp/9s5V3gyuR4B67T2A ishg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=VJQxNxTePlfcf3dKFi1wZt07MNFX3dohgtzce5ctPa8=; b=f3olOQLl4QGRtikQeBlvDapwJuhypyP/ET65a0Fn6G8G6gGgsJEGYVg2oVzsC/8jUw 61jrgvFr7tV4PI6YXLMVkpQjE4dGhgwPLw6ZA8TVmqlhe/8OW/3miobU0gQXaUOG5zMI 4x9UV+/Pe4rWI3BdeS1F7D4cVAmeFXQ+viukjM9P67pXlMzieV/VtiPW677pAnh4lt3y fNz2JQPkP9NxfCYRnQ4sVmtgK+c5NyRUXbzaf8/gwaKdXTp48DoTYq460lD8WTcFG20W myC2irAUmKeJnbddnHRUQxWhUatAQQdTUoHCTh7QLab7CkZ+wFbPHPWRwTsopvui+pYu 2mgA== X-Gm-Message-State: AOAM530A0+OIZu+k4b4+SlOqlUEf8It67YMN3tqWQEmApwFhx5wozzd8 eIor9mk2XqQcLTwz93WAfOOXvkPW8k7pM/MxTbMQt4nf/3rS/x8BBrayaog+nXVw5OJSlAkMv8m Nx0PAwJIyTF3KzpJgX4QJCg5nr3cOHSaKKOKm9RQ55pGT6iyEGIPHrSQ4iQ== X-Google-Smtp-Source: ABdhPJzBTB2xENzPFDzDvoxEae14J3frnbRBetzaDnCBIYPPtwPfiwJKr/up2FrtPuroAUjo6qPOLg+MpDg= X-Received: from oupton.c.googlers.com ([fda3:e722:ac3:10:2b:ff92:c0a8:404]) (user=oupton job=sendgmr) by 2002:a02:9542:: with SMTP id y60mr22663989jah.134.1623188905120; Tue, 08 Jun 2021 14:48:25 -0700 (PDT) Date: Tue, 8 Jun 2021 21:47:38 +0000 In-Reply-To: <20210608214742.1897483-1-oupton@google.com> Message-Id: <20210608214742.1897483-7-oupton@google.com> Mime-Version: 1.0 References: <20210608214742.1897483-1-oupton@google.com> X-Mailer: git-send-email 2.32.0.rc1.229.g3e70b5a671-goog Subject: [PATCH 06/10] selftests: KVM: Add counter emulation benchmark From: Oliver Upton To: kvm@vger.kernel.org, kvmarm@lists.cs.columbia.edu Cc: Paolo Bonzini , Sean Christopherson , Marc Zyngier , Peter Shier , Jim Mattson , David Matlack , Ricardo Koller , Jing Zhang , Raghavendra Rao Anata , Oliver Upton Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Add a test case for counter emulation on arm64. A side effect of how KVM handles physical counter offsetting on non-ECV systems is that the virtual counter will always hit hardware and the physical could be emulated. Force emulation by writing a nonzero offset to the physical counter and compare the elapsed cycles to a direct read of the hardware register. Reviewed-by: Ricardo Koller Signed-off-by: Oliver Upton --- tools/testing/selftests/kvm/.gitignore | 1 + tools/testing/selftests/kvm/Makefile | 1 + .../kvm/aarch64/counter_emulation_benchmark.c | 209 ++++++++++++++++++ 3 files changed, 211 insertions(+) create mode 100644 tools/testing/selftests/kvm/aarch64/counter_emulation_benchmark.c diff --git a/tools/testing/selftests/kvm/.gitignore b/tools/testing/selftests/kvm/.gitignore index 1a5782d8a0d4..ac6a7d17d04a 100644 --- a/tools/testing/selftests/kvm/.gitignore +++ b/tools/testing/selftests/kvm/.gitignore @@ -1,4 +1,5 @@ # SPDX-License-Identifier: GPL-2.0-only +/aarch64/counter_emulation_benchmark /aarch64/get-reg-list /aarch64/get-reg-list-sve /aarch64/vgic_init diff --git a/tools/testing/selftests/kvm/Makefile b/tools/testing/selftests/kvm/Makefile index b14f16dc954a..c2e5a7d877b1 100644 --- a/tools/testing/selftests/kvm/Makefile +++ b/tools/testing/selftests/kvm/Makefile @@ -77,6 +77,7 @@ TEST_GEN_PROGS_x86_64 += memslot_modification_stress_test TEST_GEN_PROGS_x86_64 += set_memory_region_test TEST_GEN_PROGS_x86_64 += steal_time +TEST_GEN_PROGS_aarch64 += aarch64/counter_emulation_benchmark TEST_GEN_PROGS_aarch64 += aarch64/get-reg-list TEST_GEN_PROGS_aarch64 += aarch64/get-reg-list-sve TEST_GEN_PROGS_aarch64 += aarch64/vgic_init diff --git a/tools/testing/selftests/kvm/aarch64/counter_emulation_benchmark.c b/tools/testing/selftests/kvm/aarch64/counter_emulation_benchmark.c new file mode 100644 index 000000000000..c403e0762200 --- /dev/null +++ b/tools/testing/selftests/kvm/aarch64/counter_emulation_benchmark.c @@ -0,0 +1,209 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * counter_emulation_benchmark.c -- test to measure the effects of counter + * emulation on guest reads of the physical counter. + * + * Copyright (c) 2021, Google LLC. + */ + +#define _GNU_SOURCE +#include +#include +#include +#include +#include +#include + +#include "kvm_util.h" +#include "processor.h" +#include "test_util.h" + +#define VCPU_ID 0 + +static struct counter_values { + uint64_t cntvct_start; + uint64_t cntpct; + uint64_t cntvct_end; +} counter_values; + +static uint64_t nr_iterations = 1000; + +static void do_test(void) +{ + /* + * Open-coded approach instead of using helper methods to keep a tight + * interval around the physical counter read. + */ + asm volatile("isb\n\t" + "mrs %[cntvct_start], cntvct_el0\n\t" + "isb\n\t" + "mrs %[cntpct], cntpct_el0\n\t" + "isb\n\t" + "mrs %[cntvct_end], cntvct_el0\n\t" + "isb\n\t" + : [cntvct_start] "=r"(counter_values.cntvct_start), + [cntpct] "=r"(counter_values.cntpct), + [cntvct_end] "=r"(counter_values.cntvct_end)); +} + +static void guest_main(void) +{ + int i; + + for (i = 0; i < nr_iterations; i++) { + do_test(); + GUEST_SYNC(i); + } + + for (i = 0; i < nr_iterations; i++) { + do_test(); + GUEST_SYNC(i); + } + + GUEST_DONE(); +} + +static bool enter_guest(struct kvm_vm *vm) +{ + struct ucall uc; + + vcpu_ioctl(vm, VCPU_ID, KVM_RUN, NULL); + + switch (get_ucall(vm, VCPU_ID, &uc)) { + case UCALL_DONE: + return true; + case UCALL_SYNC: + break; + case UCALL_ABORT: + TEST_ASSERT(false, "%s at %s:%ld", (const char *)uc.args[0], + __FILE__, uc.args[1]); + break; + default: + TEST_ASSERT(false, "unexpected exit: %s", + exit_reason_str(vcpu_state(vm, VCPU_ID)->exit_reason)); + break; + } + + /* more work to do in the guest */ + return false; +} + +static double counter_frequency(void) +{ + uint32_t freq; + + asm volatile("mrs %0, cntfrq_el0" + : "=r" (freq)); + + return freq / 1000000.0; +} + +static void log_csv(FILE *csv, bool trapped) +{ + double freq = counter_frequency(); + + fprintf(csv, "%s,%.02f,%lu,%lu,%lu\n", + trapped ? "true" : "false", freq, + counter_values.cntvct_start, + counter_values.cntpct, + counter_values.cntvct_end); +} + +static double run_loop(struct kvm_vm *vm, FILE *csv, bool trapped) +{ + double avg = 0; + int i; + + for (i = 0; i < nr_iterations; i++) { + uint64_t delta; + + TEST_ASSERT(!enter_guest(vm), "guest exited unexpectedly"); + sync_global_from_guest(vm, counter_values); + + if (csv) + log_csv(csv, trapped); + + delta = counter_values.cntvct_end - counter_values.cntvct_start; + avg = ((avg * i) + delta) / (i + 1); + } + + return avg; +} + +static void run_tests(struct kvm_vm *vm, FILE *csv) +{ + struct kvm_system_counter_state state = {0}; + double avg_trapped, avg_native, freq; + + freq = counter_frequency(); + + if (csv) + fputs("trapped,freq_mhz,cntvct_start,cntpct,cntvct_end\n", csv); + + /* no physical offsetting; kvm allows reads of cntpct_el0 */ + vcpu_ioctl(vm, VCPU_ID, KVM_SET_SYSTEM_COUNTER_STATE, &state); + avg_native = run_loop(vm, csv, false); + + /* force emulation of the physical counter */ + state.flags = KVM_SYSTEM_COUNTER_STATE_PHYS_OFFSET; + state.cntpoff = 1; + vcpu_ioctl(vm, VCPU_ID, KVM_SET_SYSTEM_COUNTER_STATE, &state); + avg_trapped = run_loop(vm, csv, true); + + TEST_ASSERT(enter_guest(vm), "guest didn't run to completion"); + pr_info("%lu iterations: average cycles (@%.02fMHz) native: %.02f, trapped: %.02f\n", + nr_iterations, freq, avg_native, avg_trapped); +} + +static void usage(const char *program_name) +{ + fprintf(stderr, + "Usage: %s [-h] [-o csv_file] [-n iterations]\n" + " -h prints this message\n" + " -n number of test iterations (default: %lu)\n" + " -o csv file to write data\n", + program_name, nr_iterations); +} + +int main(int argc, char **argv) +{ + struct kvm_vm *vm; + FILE *csv = NULL; + int opt; + + if (!kvm_check_cap(KVM_CAP_SYSTEM_COUNTER_STATE)) { + print_skip("KVM_CAP_SYSTEM_COUNTER_STATE not supported"); + exit(KSFT_SKIP); + } + + while ((opt = getopt(argc, argv, "hn:o:")) != -1) { + switch (opt) { + case 'o': + csv = fopen(optarg, "w"); + if (!csv) { + fprintf(stderr, "failed to open file '%s': %d\n", + optarg, errno); + exit(1); + } + break; + case 'n': + nr_iterations = strtoul(optarg, NULL, 0); + break; + default: + fprintf(stderr, "unrecognized option: '-%c'\n", opt); + /* fallthrough */ + case 'h': + usage(argv[0]); + exit(1); + } + } + + vm = vm_create_default(0, 0, guest_main); + sync_global_to_guest(vm, nr_iterations); + ucall_init(vm, NULL); + run_tests(vm, csv); + kvm_vm_free(vm); + + if (csv) + fclose(csv); +} From patchwork Tue Jun 8 21:47:39 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oliver Upton X-Patchwork-Id: 12308209 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C7280C47082 for ; Tue, 8 Jun 2021 21:48:32 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B05CB613AD for ; Tue, 8 Jun 2021 21:48:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234643AbhFHVuY (ORCPT ); Tue, 8 Jun 2021 17:50:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44102 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234625AbhFHVuY (ORCPT ); Tue, 8 Jun 2021 17:50:24 -0400 Received: from mail-il1-x14a.google.com (mail-il1-x14a.google.com [IPv6:2607:f8b0:4864:20::14a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 95B5EC061787 for ; Tue, 8 Jun 2021 14:48:30 -0700 (PDT) Received: by mail-il1-x14a.google.com with SMTP id q4-20020a056e020784b02901e2ee9a8333so15585500ils.20 for ; Tue, 08 Jun 2021 14:48:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=V441yi9NFbr1zLEPRX6rZzogwZk2O2NHoTW1SdrwQz0=; b=RrEhuyAGvy7n4Z230SQn5erzHDHZV14ZiRzmyU7qS7EtGyzBvorh9yADusm1lCinf/ WVcuWVkzvDNwzqZAfWXDokqI5lAT+Qs7Y0+ft7CRYbGTxH/xvSrOlvlekCxl20Uz0STp QSOnuZuirGHWp1OGH03LM2g5MCiLL5AcwOTW+vd81Ldcs7Xmo/AQ4+CBbBXK72Ld9jux R5MNTEX7cgzJP3n1Vdics1aqGJIC+TH/pxKzYVW5KlTSaWYqAxcFQjRqL56CblAapIZA UwicB9ZHCZb0o8lAB5BOgkXwbCmtd4X/TdMZmml6vemqAthBn+741qYe/AxuYycR2/v8 5Cnw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=V441yi9NFbr1zLEPRX6rZzogwZk2O2NHoTW1SdrwQz0=; b=OWHcmq3y8suI3BTNu+3E3iwZ0Fuk9WPPlCSob2fd+kP3Rpb+dRFdOKbb58dAEUuuhn GaeJ+153f5UcxLpsbdj6C8V8Klct5ShvPKqCDvSGl3tdgKb5zDdkdBRVnAT83gusQXPt VSHoY1TfyWxGJCbdo0+rKGqafz91HbhAoO6ayhVH+zd4Huw1vULc4ltkwu47mSM8ZQ1g VMcgt5e14vzwUnToCnOKRUTuOmcZZXvQpHLkC9dBxbvJJ8/FhoRfrgSD7tE+IjAz3gbc XKXNLuzrt5JeM7zq06bTB1k2WPyUGUyPSdNxsluhCKq7qfeQ5/0/2bWxOiR/UzKoGbsU K1IQ== X-Gm-Message-State: AOAM531tTxjv0kefxh/CdKdmFGwhxD4iwCqg4zoUW/cg3V2UwPT5AlVm P3ifqu22oNjdDoJu0Nsz5AlnIE/pzDS06XAa0+4ch83+EdWmqVhTSiWByXXQfnEKVllp7HBAVVu y3Az6PkEH80geBkVxemeBEEGN/XC3/XhF0T9aLDHp0adze5dBvGw6qzYe/g== X-Google-Smtp-Source: ABdhPJx2UsCEL2axJZTGIvmQZ2dIVVT3dzGmnONSw4l/0DJG9Hp/Proa7NFVaaB024/4wdx+UhYYYjniirQ= X-Received: from oupton.c.googlers.com ([fda3:e722:ac3:10:2b:ff92:c0a8:404]) (user=oupton job=sendgmr) by 2002:a05:6638:16d6:: with SMTP id g22mr12795895jat.142.1623188909951; Tue, 08 Jun 2021 14:48:29 -0700 (PDT) Date: Tue, 8 Jun 2021 21:47:39 +0000 In-Reply-To: <20210608214742.1897483-1-oupton@google.com> Message-Id: <20210608214742.1897483-8-oupton@google.com> Mime-Version: 1.0 References: <20210608214742.1897483-1-oupton@google.com> X-Mailer: git-send-email 2.32.0.rc1.229.g3e70b5a671-goog Subject: [PATCH 07/10] KVM: x86: Refactor tsc synchronization code From: Oliver Upton To: kvm@vger.kernel.org, kvmarm@lists.cs.columbia.edu Cc: Paolo Bonzini , Sean Christopherson , Marc Zyngier , Peter Shier , Jim Mattson , David Matlack , Ricardo Koller , Jing Zhang , Raghavendra Rao Anata , Oliver Upton Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Refactor kvm_synchronize_tsc to make a new function that allows callers to specify TSC parameters (offset, value, nanoseconds, etc.) explicitly for the sake of participating in TSC synchronization. This changes the locking semantics around TSC writes. Writes to the TSC will now take the pvclock gtod lock while holding the tsc write lock, whereas before these locks were disjoint. Reviewed-by: David Matlack Signed-off-by: Oliver Upton --- Documentation/virt/kvm/locking.rst | 11 +++ arch/x86/kvm/x86.c | 106 +++++++++++++++++------------ 2 files changed, 74 insertions(+), 43 deletions(-) diff --git a/Documentation/virt/kvm/locking.rst b/Documentation/virt/kvm/locking.rst index 1fc860c007a3..d21e0e480d63 100644 --- a/Documentation/virt/kvm/locking.rst +++ b/Documentation/virt/kvm/locking.rst @@ -25,6 +25,9 @@ On x86: holding kvm->arch.mmu_lock (typically with ``read_lock``, otherwise there's no need to take kvm->arch.tdp_mmu_pages_lock at all). +- kvm->arch.tsc_write_lock is taken outside + kvm->arch.pvclock_gtod_sync_lock + Everything else is a leaf: no other lock is taken inside the critical sections. @@ -211,6 +214,14 @@ time it will be set using the Dirty tracking mechanism described above. :Comment: 'raw' because hardware enabling/disabling must be atomic /wrt migration. +:Name: kvm_arch::pvclock_gtod_sync_lock +:Type: raw_spinlock_t +:Arch: x86 +:Protects: kvm_arch::{cur_tsc_generation,cur_tsc_nsec,cur_tsc_write, + cur_tsc_offset,nr_vcpus_matched_tsc} +:Comment: 'raw' because updating the kvm master clock must not be + preempted. + :Name: kvm_arch::tsc_write_lock :Type: raw_spinlock :Arch: x86 diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 9b6bca616929..61069995a592 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -2351,13 +2351,73 @@ static inline bool kvm_check_tsc_unstable(void) return check_tsc_unstable(); } +/* + * Infers attempts to synchronize the guest's tsc from host writes. Sets the + * offset for the vcpu and tracks the TSC matching generation that the vcpu + * participates in. + * + * Must hold kvm->arch.tsc_write_lock to call this function. + */ +static void __kvm_synchronize_tsc(struct kvm_vcpu *vcpu, u64 offset, u64 tsc, + u64 ns, bool matched) +{ + struct kvm *kvm = vcpu->kvm; + bool already_matched; + unsigned long flags; + + lockdep_assert_held(&kvm->arch.tsc_write_lock); + + already_matched = + (vcpu->arch.this_tsc_generation == kvm->arch.cur_tsc_generation); + + /* + * We track the most recent recorded KHZ, write and time to + * allow the matching interval to be extended at each write. + */ + kvm->arch.last_tsc_nsec = ns; + kvm->arch.last_tsc_write = tsc; + kvm->arch.last_tsc_khz = vcpu->arch.virtual_tsc_khz; + + vcpu->arch.last_guest_tsc = tsc; + + /* Keep track of which generation this VCPU has synchronized to */ + vcpu->arch.this_tsc_generation = kvm->arch.cur_tsc_generation; + vcpu->arch.this_tsc_nsec = kvm->arch.cur_tsc_nsec; + vcpu->arch.this_tsc_write = kvm->arch.cur_tsc_write; + + kvm_vcpu_write_tsc_offset(vcpu, offset); + + spin_lock_irqsave(&kvm->arch.pvclock_gtod_sync_lock, flags); + if (!matched) { + /* + * We split periods of matched TSC writes into generations. + * For each generation, we track the original measured + * nanosecond time, offset, and write, so if TSCs are in + * sync, we can match exact offset, and if not, we can match + * exact software computation in compute_guest_tsc() + * + * These values are tracked in kvm->arch.cur_xxx variables. + */ + kvm->arch.nr_vcpus_matched_tsc = 0; + kvm->arch.cur_tsc_generation++; + kvm->arch.cur_tsc_nsec = ns; + kvm->arch.cur_tsc_write = tsc; + kvm->arch.cur_tsc_offset = offset; + matched = false; + } else if (!already_matched) { + kvm->arch.nr_vcpus_matched_tsc++; + } + + kvm_track_tsc_matching(vcpu); + spin_unlock_irqrestore(&kvm->arch.pvclock_gtod_sync_lock, flags); +} + static void kvm_synchronize_tsc(struct kvm_vcpu *vcpu, u64 data) { struct kvm *kvm = vcpu->kvm; u64 offset, ns, elapsed; unsigned long flags; - bool matched; - bool already_matched; + bool matched = false; bool synchronizing = false; raw_spin_lock_irqsave(&kvm->arch.tsc_write_lock, flags); @@ -2403,51 +2463,11 @@ static void kvm_synchronize_tsc(struct kvm_vcpu *vcpu, u64 data) offset = kvm_compute_tsc_offset(vcpu, data); } matched = true; - already_matched = (vcpu->arch.this_tsc_generation == kvm->arch.cur_tsc_generation); - } else { - /* - * We split periods of matched TSC writes into generations. - * For each generation, we track the original measured - * nanosecond time, offset, and write, so if TSCs are in - * sync, we can match exact offset, and if not, we can match - * exact software computation in compute_guest_tsc() - * - * These values are tracked in kvm->arch.cur_xxx variables. - */ - kvm->arch.cur_tsc_generation++; - kvm->arch.cur_tsc_nsec = ns; - kvm->arch.cur_tsc_write = data; - kvm->arch.cur_tsc_offset = offset; - matched = false; } - /* - * We also track th most recent recorded KHZ, write and time to - * allow the matching interval to be extended at each write. - */ - kvm->arch.last_tsc_nsec = ns; - kvm->arch.last_tsc_write = data; - kvm->arch.last_tsc_khz = vcpu->arch.virtual_tsc_khz; - - vcpu->arch.last_guest_tsc = data; + __kvm_synchronize_tsc(vcpu, offset, data, ns, matched); - /* Keep track of which generation this VCPU has synchronized to */ - vcpu->arch.this_tsc_generation = kvm->arch.cur_tsc_generation; - vcpu->arch.this_tsc_nsec = kvm->arch.cur_tsc_nsec; - vcpu->arch.this_tsc_write = kvm->arch.cur_tsc_write; - - kvm_vcpu_write_tsc_offset(vcpu, offset); raw_spin_unlock_irqrestore(&kvm->arch.tsc_write_lock, flags); - - spin_lock_irqsave(&kvm->arch.pvclock_gtod_sync_lock, flags); - if (!matched) { - kvm->arch.nr_vcpus_matched_tsc = 0; - } else if (!already_matched) { - kvm->arch.nr_vcpus_matched_tsc++; - } - - kvm_track_tsc_matching(vcpu); - spin_unlock_irqrestore(&kvm->arch.pvclock_gtod_sync_lock, flags); } static inline void adjust_tsc_offset_guest(struct kvm_vcpu *vcpu, From patchwork Tue Jun 8 21:47:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oliver Upton X-Patchwork-Id: 12308211 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 67D10C47082 for ; Tue, 8 Jun 2021 21:48:48 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4C755613B6 for ; Tue, 8 Jun 2021 21:48:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234628AbhFHVuk (ORCPT ); Tue, 8 Jun 2021 17:50:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44126 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234364AbhFHVuk (ORCPT ); Tue, 8 Jun 2021 17:50:40 -0400 Received: from mail-yb1-xb49.google.com (mail-yb1-xb49.google.com [IPv6:2607:f8b0:4864:20::b49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7113CC061574 for ; Tue, 8 Jun 2021 14:48:36 -0700 (PDT) Received: by mail-yb1-xb49.google.com with SMTP id y135-20020a25dc8d0000b029054716974cceso8534691ybe.12 for ; Tue, 08 Jun 2021 14:48:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=9ZoMqiE2Ix8XEX2FQzL2zr/Zx9M+aXu+vadqrJ/vnTw=; b=XxSvqxLou4MF4BxFLH0qp6z9iCNGB8fRfuEiXIEK/ImcP+Dl6aGVZQ5neE5XszmYpf Z57i2uiMF8yel+7eV37wtdnG8M+hdfascVdwEH1WYmGcPN8QoNJiDlaG+FyzYMr3kiqO c0Nb/Gu8xmsdxc91iXouuzftZF2YxXLyjsItZH3CM8YiZpDmOD7FdrE2CcLHgywd+ly3 dH9dHva14YiQp5i7HqLJYokKiSnxup62jn0HPgQ2I3DLB7MmCHTahF3hAEDiu21v8BYK Oo1o3VM6/vOCNcYUphNsg+YXWJ87ZEzGQBl1ty+19uEJbmOyslLtCfALWev7VrQCxOZa /xgQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=9ZoMqiE2Ix8XEX2FQzL2zr/Zx9M+aXu+vadqrJ/vnTw=; b=h96kaMecugq0PfFfDRnclzWFcRRlZx/ztnb2VanNZ9orgyCFntdB1K6+HTWKf3xLeE l/3ELwDUbDtQrmDlWGLZIsbo6fxlZy98Ab+zsnc7GhnU+i8WrDegGYaYX54HqTDucotp F/zRwBWIj3C662vFBfC2t1WGWonOmfxgXUTku5AdZt8nnOMR0/DgiiHzqWOA8gfPfOo5 tkUlzZeZHgkq5ePC3boodR0LZnQorgXWHs6nSDCp5zorQXFfQtjthrPnack0CaGDLAvG MND8nm5sZx9iyL/7juXeYZ/FE+TefBWdqJ3kpvQEDEWBgW+XAYUFCWP9h9KxcIMQ06CT r0wQ== X-Gm-Message-State: AOAM530VungHg6wyBYEJnYLszSE99sifY7u+LLAFXvAOlmPC9c9ZLyPN CsnFxJzIBMyipFgmn1qmUl+rRXyT5Qz8xx3UOuXCBqkUJGo1Qc8Mev0UuD+W6Wvhd4qoCCC8+PR dh6luGrgKlFkuhf5hSLYE+LO1M25UdImlzp2Bpn8P3FfZtNL8ribmJDNN8Q== X-Google-Smtp-Source: ABdhPJxLd52vbcGMLWWE2gQyW873IX70SEaG+rVIXWmdXwk5b+dvVT+6qrinH7iDe569lig2y1MvZyvExDc= X-Received: from oupton.c.googlers.com ([fda3:e722:ac3:10:2b:ff92:c0a8:404]) (user=oupton job=sendgmr) by 2002:a25:dbc4:: with SMTP id g187mr35049365ybf.142.1623188915485; Tue, 08 Jun 2021 14:48:35 -0700 (PDT) Date: Tue, 8 Jun 2021 21:47:40 +0000 In-Reply-To: <20210608214742.1897483-1-oupton@google.com> Message-Id: <20210608214742.1897483-9-oupton@google.com> Mime-Version: 1.0 References: <20210608214742.1897483-1-oupton@google.com> X-Mailer: git-send-email 2.32.0.rc1.229.g3e70b5a671-goog Subject: [PATCH 08/10] KVM: x86: Implement KVM_CAP_SYSTEM_COUNTER_STATE From: Oliver Upton To: kvm@vger.kernel.org, kvmarm@lists.cs.columbia.edu Cc: Paolo Bonzini , Sean Christopherson , Marc Zyngier , Peter Shier , Jim Mattson , David Matlack , Ricardo Koller , Jing Zhang , Raghavendra Rao Anata , Oliver Upton Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org To date, VMM-directed TSC synchronization and migration has been messy. KVM has some baked-in heuristics around TSC writes to infer if the VMM is attempting to synchronize. This is problematic, as it depends on the host writing to the guest's TSC within 1 second of the last write. A much cleaner approach to configuring the guest's views of the TSC is to simply migrate the TSC offset for every vCPU. Offsets are idempotent, and thus are not subject to change depending on when the VMM actually reads the values from KVM. The VMM can then read the TSC once to capture the instant at which the guest's TSCs are paused. Implement the KVM_{GET,SET}_SYSTEM_COUNTER_STATE ioctls and advertise KVM_CAP_SYSTEM_COUNTER_STATE to userspace. Reviewed-by: David Matlack Signed-off-by: Oliver Upton --- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/include/uapi/asm/kvm.h | 8 ++++ arch/x86/kvm/x86.c | 70 +++++++++++++++++++++++++++++++++ 3 files changed, 79 insertions(+) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 55efbacfc244..8768173f614c 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1018,6 +1018,7 @@ struct kvm_arch { u64 last_tsc_nsec; u64 last_tsc_write; u32 last_tsc_khz; + u64 last_tsc_offset; u64 cur_tsc_nsec; u64 cur_tsc_write; u64 cur_tsc_offset; diff --git a/arch/x86/include/uapi/asm/kvm.h b/arch/x86/include/uapi/asm/kvm.h index 0662f644aad9..60ad6b9ebcd6 100644 --- a/arch/x86/include/uapi/asm/kvm.h +++ b/arch/x86/include/uapi/asm/kvm.h @@ -490,4 +490,12 @@ struct kvm_pmu_event_filter { #define KVM_PMU_EVENT_ALLOW 0 #define KVM_PMU_EVENT_DENY 1 +/* for KVM_CAP_SYSTEM_COUNTER_STATE */ +struct kvm_system_counter_state { + __u32 flags; + __u32 pad; + __u64 tsc_offset; + __u64 rsvd[6]; +}; + #endif /* _ASM_X86_KVM_H */ diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 61069995a592..bb3ecb5cd548 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -2332,6 +2332,11 @@ u64 kvm_read_l1_tsc(struct kvm_vcpu *vcpu, u64 host_tsc) } EXPORT_SYMBOL_GPL(kvm_read_l1_tsc); +static u64 kvm_vcpu_read_tsc_offset(struct kvm_vcpu *vcpu) +{ + return vcpu->arch.l1_tsc_offset; +} + static void kvm_vcpu_write_tsc_offset(struct kvm_vcpu *vcpu, u64 offset) { vcpu->arch.l1_tsc_offset = offset; @@ -2377,6 +2382,7 @@ static void __kvm_synchronize_tsc(struct kvm_vcpu *vcpu, u64 offset, u64 tsc, kvm->arch.last_tsc_nsec = ns; kvm->arch.last_tsc_write = tsc; kvm->arch.last_tsc_khz = vcpu->arch.virtual_tsc_khz; + kvm->arch.last_tsc_offset = offset; vcpu->arch.last_guest_tsc = tsc; @@ -2485,6 +2491,44 @@ static inline void adjust_tsc_offset_host(struct kvm_vcpu *vcpu, s64 adjustment) adjust_tsc_offset_guest(vcpu, adjustment); } +static int kvm_vcpu_get_system_counter_state(struct kvm_vcpu *vcpu, + struct kvm_system_counter_state *state) +{ + if (state->flags) + return -EINVAL; + + state->tsc_offset = kvm_vcpu_read_tsc_offset(vcpu); + return 0; +} + +static int kvm_vcpu_set_system_counter_state(struct kvm_vcpu *vcpu, + struct kvm_system_counter_state *state) +{ + struct kvm *kvm = vcpu->kvm; + u64 offset, tsc, ns; + unsigned long flags; + bool matched; + + if (state->flags) + return -EINVAL; + + offset = state->tsc_offset; + + raw_spin_lock_irqsave(&kvm->arch.tsc_write_lock, flags); + + matched = (vcpu->arch.virtual_tsc_khz && + kvm->arch.last_tsc_khz == vcpu->arch.virtual_tsc_khz && + kvm->arch.last_tsc_offset == offset); + + tsc = kvm_scale_tsc(vcpu, rdtsc()) + offset; + ns = get_kvmclock_base_ns(); + + __kvm_synchronize_tsc(vcpu, offset, tsc, ns, matched); + raw_spin_unlock_irqrestore(&kvm->arch.tsc_write_lock, flags); + + return 0; +} + #ifdef CONFIG_X86_64 static u64 read_tsc(void) @@ -3912,6 +3956,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) case KVM_CAP_SGX_ATTRIBUTE: #endif case KVM_CAP_VM_COPY_ENC_CONTEXT_FROM: + case KVM_CAP_SYSTEM_COUNTER_STATE: r = 1; break; case KVM_CAP_SET_GUEST_DEBUG2: @@ -5200,6 +5245,31 @@ long kvm_arch_vcpu_ioctl(struct file *filp, break; } #endif + case KVM_GET_SYSTEM_COUNTER_STATE: { + struct kvm_system_counter_state state; + + r = -EFAULT; + if (copy_from_user(&state, argp, sizeof(state))) + goto out; + + r = kvm_vcpu_get_system_counter_state(vcpu, &state); + if (r) + goto out; + if (copy_to_user(argp, &state, sizeof(state))) + r = -EFAULT; + + break; + } + case KVM_SET_SYSTEM_COUNTER_STATE: { + struct kvm_system_counter_state state; + + r = -EFAULT; + if (copy_from_user(&state, argp, sizeof(state))) + goto out; + + r = kvm_vcpu_set_system_counter_state(vcpu, &state); + break; + } default: r = -EINVAL; } From patchwork Tue Jun 8 21:47:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oliver Upton X-Patchwork-Id: 12308213 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6490AC47082 for ; Tue, 8 Jun 2021 21:49:07 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4576161278 for ; Tue, 8 Jun 2021 21:49:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234742AbhFHVu6 (ORCPT ); Tue, 8 Jun 2021 17:50:58 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44152 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234749AbhFHVur (ORCPT ); Tue, 8 Jun 2021 17:50:47 -0400 Received: from mail-qt1-x84a.google.com (mail-qt1-x84a.google.com [IPv6:2607:f8b0:4864:20::84a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 71EC1C061787 for ; Tue, 8 Jun 2021 14:48:42 -0700 (PDT) Received: by mail-qt1-x84a.google.com with SMTP id w1-20020ac87a610000b02902433332a0easo10224826qtt.0 for ; Tue, 08 Jun 2021 14:48:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=67DO9q/Kcpbvyzcr1mvzFYjXWQVENF8Dem27uU+5fps=; b=QRW0mdlazXmqwpTtBWy7gVO2Vsqzm0otOCoz8nuttjFrKwU3ub0Iu5e1zlq3zfhm47 kr/+jX+ynlTMJpj9DSXjzfaGXc1LPOGlTQo9pGayczFTLEFXJIhXNukAq9GRVvrtuPGo AfSqLIZMes0c/emJ+lFgjrAdAqb+4+9OKXMHn7gpXK3/gnJQRjCRb+7ysIivX6tx64jN IN2WmWEXpAQs9SpbLHd4Yqdey4MDGBMRiXtWAZZciVWk9YrRMhk8PZjlnBWeCQnxoeZi tJq6j6DbTuOySOcAvcOcgyGFruy5fiBQtOu8l7qfv4mg3UgD+JEz1yeJ6oyHfCsa1eyR uHWA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=67DO9q/Kcpbvyzcr1mvzFYjXWQVENF8Dem27uU+5fps=; b=ZiDizrYGsKrOavl2x/uv+XrnAxoU9I+L2mi6LaZZ6CbZ4iI4tutBVZa+31hEUkrJ62 DNa+lYXgTXqJbzucxc1n3dGXLTw6wCZXw6iCj2rMPVl12SnmshSv64L/SYz/6DUY/gkn c40wkMWxW7jneB0l607C2cPbaCrlCxYWmfkXjN3AVDrbI4pFgTNXKpRPg58sE6cLuK2M G84UOK1MOkN6F5lSlNnS71rmX1tdF+lOJLycuwX5Qs4krwTHMFNF26v9bYi2WwwzB31w 4cW9rz/hEpuDT99QpkSZKfkbaGyQNuWmG4vdxs4vuqbG+Jh18+h8k8Q0Qv3E2SuSztdU nVIA== X-Gm-Message-State: AOAM533rjepCWhO9l6nE1THhTq1HDW/iqKndCCBPwjMsSBXUmIYFv4s9 gx9lSu612KxwtA5qj4zxutDyUZoiIIAwOCm3vn8ebgQOKN5je2DC7VOGMdItw06ZpzuYtAqq4kw jb3IS/0c5hlP6VTGcoZOXwSKRkXn5ZysosL5Exqu6cX3Uy/qcjQo2GM6eRg== X-Google-Smtp-Source: ABdhPJyg1Wey+rmDumCpN1DSiKtE3tz1DCSq/H8NM9+D8wklMqy8ucTb5VDFwusjuwissOIoFQe/WoKO0aI= X-Received: from oupton.c.googlers.com ([fda3:e722:ac3:10:2b:ff92:c0a8:404]) (user=oupton job=sendgmr) by 2002:ad4:4863:: with SMTP id u3mr2132928qvy.7.1623188921468; Tue, 08 Jun 2021 14:48:41 -0700 (PDT) Date: Tue, 8 Jun 2021 21:47:41 +0000 In-Reply-To: <20210608214742.1897483-1-oupton@google.com> Message-Id: <20210608214742.1897483-10-oupton@google.com> Mime-Version: 1.0 References: <20210608214742.1897483-1-oupton@google.com> X-Mailer: git-send-email 2.32.0.rc1.229.g3e70b5a671-goog Subject: [PATCH 09/10] selftests: KVM: Add support for x86 to system_counter_state_test From: Oliver Upton To: kvm@vger.kernel.org, kvmarm@lists.cs.columbia.edu Cc: Paolo Bonzini , Sean Christopherson , Marc Zyngier , Peter Shier , Jim Mattson , David Matlack , Ricardo Koller , Jing Zhang , Raghavendra Rao Anata , Oliver Upton Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Test that userspace manipulation of the tsc offset (via KVM_{GET,SET}_SYSTEM_COUNTER_STATE) results in the expected guest view of the TSC. Reviewed-by: David Matlack Signed-off-by: Oliver Upton --- tools/testing/selftests/kvm/Makefile | 1 + .../selftests/kvm/system_counter_state_test.c | 32 +++++++++++++++++++ 2 files changed, 33 insertions(+) diff --git a/tools/testing/selftests/kvm/Makefile b/tools/testing/selftests/kvm/Makefile index c2e5a7d877b1..28207474c79c 100644 --- a/tools/testing/selftests/kvm/Makefile +++ b/tools/testing/selftests/kvm/Makefile @@ -76,6 +76,7 @@ TEST_GEN_PROGS_x86_64 += kvm_page_table_test TEST_GEN_PROGS_x86_64 += memslot_modification_stress_test TEST_GEN_PROGS_x86_64 += set_memory_region_test TEST_GEN_PROGS_x86_64 += steal_time +TEST_GEN_PROGS_x86_64 += system_counter_state_test TEST_GEN_PROGS_aarch64 += aarch64/counter_emulation_benchmark TEST_GEN_PROGS_aarch64 += aarch64/get-reg-list diff --git a/tools/testing/selftests/kvm/system_counter_state_test.c b/tools/testing/selftests/kvm/system_counter_state_test.c index f537eb7b928c..9dcbc95bba3f 100644 --- a/tools/testing/selftests/kvm/system_counter_state_test.c +++ b/tools/testing/selftests/kvm/system_counter_state_test.c @@ -138,6 +138,38 @@ static uint64_t host_read_guest_counter(struct system_counter_state_test *test) return r; } +#elif __x86_64__ + +/* no additional information required beyond the counter state. */ +#define system_counter_state_test kvm_system_counter_state + +static struct system_counter_state_test test_cases[] = { + { .tsc_offset = 0 }, + { .tsc_offset = 1000000 }, + { .tsc_offset = -1 }, +}; + +static void pr_test(struct system_counter_state_test *test) +{ + printf("tsc_offset: %lld\n", test->tsc_offset); +} + +static struct kvm_system_counter_state * +get_system_counter_state(struct system_counter_state_test *test) +{ + return test; +} + +static uint64_t guest_read_counter(struct system_counter_state_test *test) +{ + return rdtsc(); +} + +static uint64_t host_read_guest_counter(struct system_counter_state_test *test) +{ + return rdtsc() + test->tsc_offset; +} + #else #error test not implemented for architecture being built! #endif From patchwork Tue Jun 8 21:47:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oliver Upton X-Patchwork-Id: 12308215 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 39F64C4743F for ; Tue, 8 Jun 2021 21:49:09 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2630E613AD for ; Tue, 8 Jun 2021 21:49:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234746AbhFHVvB (ORCPT ); Tue, 8 Jun 2021 17:51:01 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44166 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234779AbhFHVu5 (ORCPT ); Tue, 8 Jun 2021 17:50:57 -0400 Received: from mail-il1-x149.google.com (mail-il1-x149.google.com [IPv6:2607:f8b0:4864:20::149]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 154FFC06178B for ; Tue, 8 Jun 2021 14:48:46 -0700 (PDT) Received: by mail-il1-x149.google.com with SMTP id h5-20020a92c2650000b02901ed6e8897a2so4339635ild.23 for ; Tue, 08 Jun 2021 14:48:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=ZMiQ0RX43+2xU8fhkPaV9SCdEkQtx107I/DVKk+Rtz0=; b=vojVCLN1HHEauEqNOm/PIY0zLcZtycXgG4Yq1qZkQn1i6fCilmFgAbGo7EwMIWp9xD uouwKiRPFHRRJoGtI1aiOdDm+b3jc3qG8B2qFpcPFjcuk29VM4ivu3QJMGHK1t5qbJkv JGwhnLezKG9L2wbLcVZbk2Dd3Uk8maRdkYMoYOhO6oUClxho6+yxHwdxNkR6zvXzPQB/ A4RD377h8wBYdTz/O9LeZhod9DfUUZrQ3kKjgHRzSqbcXZEp0H7vIkysawdBHb476m8V FILJ9G4mdM8C5+WVDoNhmN8vBGw/vMCx32d3LKmJkv10sQWDTM7ieN3DlfgKKdyxFO4s df5w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=ZMiQ0RX43+2xU8fhkPaV9SCdEkQtx107I/DVKk+Rtz0=; b=WO603bkGG3rcXmh8E3/Np1h9Sw9VyadiVGREqQcXvax2CW/STJLuz0tYwMeRtfxgvF W+Fz5UUrE7WUQjIVkCRhkZvLAGoj7IXT4OEJnlmxmTcsjQmRq1A61X2Znfi+cn/trXjW tQeZxnhJHtWUO6etI5D37BLLb5nPD++aXp7xVUFP60NEn06835CYdl3j8ksZ4gX2UK76 kNVpdkmfP4Hiu9G1/iX73ORCdTgqxMPCQoW6tYD563SLIO09c/6PgLugu2Vw/GjKNxBd Jtl3XNhUnU83N5KXDgt6GRq22TcfD3p/ncyOR6fMyiqxiNMAtJsy/zBSOf7RrGHaFouS vhAg== X-Gm-Message-State: AOAM530L5q5KNjK8yvb/22NCEl/Unr34+DySBuE4y0gW1op8u7zZ7MuI BSZ6bXHztx48BReUxnTdxJQoTn/Lko3nuLWLi0Tdh6euCTOBeQeYzndGcsRSf1p4jBI5ZP/uMLz aG9WbSnaaKzukacdb/PPXoscgCTnjlvtCsE08eu74mI/PGNFSSjm4eljYuQ== X-Google-Smtp-Source: ABdhPJyR8V1HzhwDWkDHKme3jZntuYhIwMcdFAlEYI5sseau6in9wS93gGWc94zbz+SRkO7moO/Rzr28Otw= X-Received: from oupton.c.googlers.com ([fda3:e722:ac3:10:2b:ff92:c0a8:404]) (user=oupton job=sendgmr) by 2002:a05:6e02:1b82:: with SMTP id h2mr20761162ili.7.1623188925465; Tue, 08 Jun 2021 14:48:45 -0700 (PDT) Date: Tue, 8 Jun 2021 21:47:42 +0000 In-Reply-To: <20210608214742.1897483-1-oupton@google.com> Message-Id: <20210608214742.1897483-11-oupton@google.com> Mime-Version: 1.0 References: <20210608214742.1897483-1-oupton@google.com> X-Mailer: git-send-email 2.32.0.rc1.229.g3e70b5a671-goog Subject: [PATCH 10/10] Documentation: KVM: Document KVM_{GET,SET}_SYSTEM_COUNTER_STATE ioctls From: Oliver Upton To: kvm@vger.kernel.org, kvmarm@lists.cs.columbia.edu Cc: Paolo Bonzini , Sean Christopherson , Marc Zyngier , Peter Shier , Jim Mattson , David Matlack , Ricardo Koller , Jing Zhang , Raghavendra Rao Anata , Oliver Upton Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Reviewed-by: David Matlack Reviewed-by: Peter Shier Reviewed-by: Ricardo Koller Signed-off-by: Oliver Upton --- Documentation/virt/kvm/api.rst | 98 ++++++++++++++++++++++++++++++++++ 1 file changed, 98 insertions(+) diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst index 7fcb2fd38f42..85654748156a 100644 --- a/Documentation/virt/kvm/api.rst +++ b/Documentation/virt/kvm/api.rst @@ -5034,6 +5034,104 @@ see KVM_XEN_VCPU_SET_ATTR above. The KVM_XEN_VCPU_ATTR_TYPE_RUNSTATE_ADJUST type may not be used with the KVM_XEN_VCPU_GET_ATTR ioctl. +4.130 KVM_GET_SYSTEM_COUNTER_STATE +--------------------------- + +:Capability: KVM_CAP_SYSTEM_COUNTER_STATE +:Architectures: arm64, x86 +:Type: vcpu ioctl +:Parameters: struct kvm_system_counter_state +:Returns: 0 on success, < 0 on error + +Allows the vCPU counter state to be read. Each architecture defines +its own kvm_system_counter_state structure depending on the backing hardware +controls used for the guest's counter. + +ARM64 + +:: + + struct kvm_system_counter_state { + /* indicates what fields are valid in the structure */ + __u32 flags; + +Enumerates what fields are valid in the kvm_system_counter_state structure. +Userspace should set this field to indicate what fields it wants the kernel +to populate. + +:: + + __u32 pad; + /* + * Guest physical counter-timer offset, relative to host cntpct_el0. + * Valid when KVM_SYSTEM_COUNTER_STATE_PHYS_OFFSET is set. + */ + __u64 cntvoff; + +Offset for the guest virtual counter-timer, as it relates to the host's +physical counter-timer (CNTPCT_EL0). This field is populated when the +KVM_SYSTEM_COUNTER_STATE_PHYS_OFFSET bit is set in the flags field. + +:: + + /* guest physical counter-timer offset, relative to host cntpct_el0 */ + __u64 cntpoff; + +Offset for the guest physical counter-timer, as it relates to the host's +physical counter-timer (CNTPCT_EL0). + +:: + + __u64 rsvd[5]; + }; + +x86 + +:: + + struct kvm_system_counter_state { + __u32 flags; + +Enumerates what fields are valid in the kvm_system_counter_state structure. +Currently, the structure has not been extended, so there are no valid flag +bits. This field should then be set to zero. + +:: + + __u32 pad; + __u64 tsc_offset; + +Offset for the guest TSC, as it relates to the host's TSC. + +:: + + __u64 rsvd[6]; + }; + +4.131 KVM_SET_SYSTEM_COUNTER_STATE +--------------------------- + +:Capability: KVM_CAP_SYSTEM_COUNTER_STATE +:Architectures: arm64, x86 +:Type: vcpu ioctl +:Parameters: struct kvm_system_counter_state +:Returns: 0 on success, < 0 on error. + +Allows the vCPU counter state to be written. For more details on the +structure, see KVM_GET_SYSTEM_COUNTER_STATE above. + +ARM64 + +VMMs should either use this ioctl *OR* directly write to the vCPU's +CNTVCT_EL0 register. Mixing both methods of restoring counter state +can cause drift between virtual CPUs. + +x86 + +VMMs should either use this ioctl *OR* directly write to the vCPU's +IA32_TSC register. Mixing both methods of restoring TSC state can +cause drift between virtual CPUs. + 5. The kvm_run structure ========================