From patchwork Mon Apr 26 09:06:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hikaru Nishida X-Patchwork-Id: 12224013 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D466EC433B4 for ; Mon, 26 Apr 2021 09:10:15 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A7A0A61139 for ; Mon, 26 Apr 2021 09:10:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232524AbhDZJK4 (ORCPT ); Mon, 26 Apr 2021 05:10:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59946 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232718AbhDZJJQ (ORCPT ); Mon, 26 Apr 2021 05:09:16 -0400 Received: from mail-pl1-x62f.google.com (mail-pl1-x62f.google.com [IPv6:2607:f8b0:4864:20::62f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DD2A1C061763 for ; Mon, 26 Apr 2021 02:08:26 -0700 (PDT) Received: by mail-pl1-x62f.google.com with SMTP id s15so177053plg.6 for ; Mon, 26 Apr 2021 02:08:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=V50YyMbdbg4kQSCC5l/2b7FBtbZFhZcyEXS5Kc3/Kqk=; b=Za4K1kqevYO48etJVHFvsJSz5Lpt2B7otONeFbwl8U91tfs2k0ZWOgpaTPJvoqt87r GnZRGMhsbds1i7v30bAxXk5gcYXkOtmC8hBem5xAO+Hi8iW5e62iU0AlU5brTo4SNSyn JDC40NZFnx4VaEdPeYrHKVQeMEDa1Q9y1dXiI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=V50YyMbdbg4kQSCC5l/2b7FBtbZFhZcyEXS5Kc3/Kqk=; b=jJ5u9cT1w4/L2svZYb8KFvgFIkkbWUX7piJ0xeIUBRkiXvo63gjRVJFb30DKZctpIo RrE7o2Fp3/5kCWIXJ+EJiZz2sLuahm08Pfrz+hBPh2AypgkpO2oqc6wJ3WjRLIPxLmPz H82zhDO40FyemdKkpdrMUmFXZ1EVz4xbnO0f9i/OHcdsaLcblaHk4DgNTIcXe0NpPxsg bc3TI3vU2hIgLD324QjbwwF36xsiROimAq2CoPZ4FWcUKSHc+iv0xOKssl8DJDl/eDKH Nm1YFrJCTUqnNJKZRBC6nLz4q4QxdA44lj1NhQXLJ9zNaRZSFFF3pj+dAv2kkbMzrPQz kWJw== X-Gm-Message-State: AOAM532oBPv1JH7IFCMWJyry90tpDgNeenK61mSG3A3V+31XpCkoX5Bo avfq7hg4L7JsEt0iClxAG/VxNDomxTtyvQ== X-Google-Smtp-Source: ABdhPJyQGYzq+3+Y2biONIrwkE7niPYsewV7PhLNOyYleKr/TJht+Nwnjq940Dao4JQSvBYmyTDSFQ== X-Received: by 2002:a17:902:d2c3:b029:ec:b1ce:c488 with SMTP id n3-20020a170902d2c3b02900ecb1cec488mr17905356plc.4.1619428106144; Mon, 26 Apr 2021 02:08:26 -0700 (PDT) Received: from localhost (160.131.236.35.bc.googleusercontent.com. [35.236.131.160]) by smtp.gmail.com with UTF8SMTPSA id m2sm10679322pgv.87.2021.04.26.02.08.24 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 26 Apr 2021 02:08:25 -0700 (PDT) From: Hikaru Nishida To: kvm@vger.kernel.org Cc: suleiman@google.com, Hikaru Nishida , Jonathan Corbet , Paolo Bonzini , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [RFC PATCH 1/6] x86/kvm: Reserve KVM_FEATURE_HOST_SUSPEND_TIME and MSR_KVM_HOST_SUSPEND_TIME Date: Mon, 26 Apr 2021 18:06:40 +0900 Message-Id: <20210426090644.2218834-2-hikalium@chromium.org> X-Mailer: git-send-email 2.31.1.498.g6c1eba8ee3d-goog In-Reply-To: <20210426090644.2218834-1-hikalium@chromium.org> References: <20210426090644.2218834-1-hikalium@chromium.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org No functional change; just add documentation for KVM_FEATURE_HOST_SUSPEND_TIME and its corresponding MSR_KVM_HOST_SUSPEND_TIME to support virtual suspend timing injection in later patches. Signed-off-by: Hikaru Nishida --- Documentation/virt/kvm/cpuid.rst | 3 +++ Documentation/virt/kvm/msr.rst | 29 +++++++++++++++++++++++++++++ 2 files changed, 32 insertions(+) diff --git a/Documentation/virt/kvm/cpuid.rst b/Documentation/virt/kvm/cpuid.rst index cf62162d4be2..c7cb581b9a9b 100644 --- a/Documentation/virt/kvm/cpuid.rst +++ b/Documentation/virt/kvm/cpuid.rst @@ -96,6 +96,9 @@ KVM_FEATURE_MSI_EXT_DEST_ID 15 guest checks this feature bit before using extended destination ID bits in MSI address bits 11-5. +KVM_FEATURE_HOST_SUSPEND_TIME 16 host suspend time information + is available at msr 0x4b564d08. + KVM_FEATURE_CLOCKSOURCE_STABLE_BIT 24 host will warn if no guest-side per-cpu warps are expected in kvmclock diff --git a/Documentation/virt/kvm/msr.rst b/Documentation/virt/kvm/msr.rst index e37a14c323d2..de96743245c9 100644 --- a/Documentation/virt/kvm/msr.rst +++ b/Documentation/virt/kvm/msr.rst @@ -376,3 +376,32 @@ data: write '1' to bit 0 of the MSR, this causes the host to re-scan its queue and check if there are more notifications pending. The MSR is available if KVM_FEATURE_ASYNC_PF_INT is present in CPUID. + +MSR_KVM_HOST_SUSPEND_TIME: + 0x4b564d08 + +data: + 8-byte alignment physical address of a memory area which must be + in guest RAM, plus an enable bit in bit 0. This memory is expected to + hold a copy of the following structure:: + + struct kvm_host_suspend_time { + __u64 suspend_time_ns; + }; + + whose data will be filled in by the hypervisor. + If the guest register this structure through the MSR write, the host + will stop all the clocks including TSCs observed by the guest during + the host's suspension and report the duration of suspend through this + structure. Fields have the following meanings: + + host_suspend_time_ns: + Total number of nanoseconds passed during the host's suspend + while the VM is running. This value will be increasing + monotonically. + + Note that although MSRs are per-CPU entities, the effect of this + particular MSR is global. + + Availability of this MSR must be checked via bit 16 in 0x4000001 cpuid + leaf prior to usage. From patchwork Mon Apr 26 09:06:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hikaru Nishida X-Patchwork-Id: 12224011 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 83EACC433ED for ; Mon, 26 Apr 2021 09:10:15 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 35A2C6102A for ; Mon, 26 Apr 2021 09:10:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232078AbhDZJKz (ORCPT ); Mon, 26 Apr 2021 05:10:55 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59950 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232524AbhDZJJQ (ORCPT ); Mon, 26 Apr 2021 05:09:16 -0400 Received: from mail-pj1-x102f.google.com (mail-pj1-x102f.google.com [IPv6:2607:f8b0:4864:20::102f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1DCAFC06138C for ; Mon, 26 Apr 2021 02:08:31 -0700 (PDT) Received: by mail-pj1-x102f.google.com with SMTP id u14-20020a17090a1f0eb029014e38011b09so4895731pja.5 for ; Mon, 26 Apr 2021 02:08:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=CyW+WjVffqnD9VFGsqpSFk2yAUyd3GC+azDgWQuCGt8=; b=NfvuDJlz/4axwtjejO6Bhm8ZUOSpcdBl5CqsZzYNOleZcxJvwfRh2sth2duBAfJN1Q HTeUGen/ixs8Ww2Wzn+BxsyYRO2WwerMCpnUZ1W8V00BPJXr8gTd/r6QsQkrxdH19DnF wSiNGd7vmSRRV2YuJA6mT293nZ+w9XrrFUaU0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=CyW+WjVffqnD9VFGsqpSFk2yAUyd3GC+azDgWQuCGt8=; b=ADllKI633r982KPuOwK/h14DAjsDazIyDkBtL3N06aydq7JGH6Jzi89h3oBXc9Nmls G8eo+QgIeMiyrkW6jqzAuBuEAJHRZY1ExYWT1epeJT8BeagTupjDCzvEoVrlyRuxbBP6 l4ZCSUJSL+H4ngmB5BKCYhlwS2ctrgTJ4bXXVcFUOtL7sUUmL3xCaJ+Zw2F+406S0P3Z kmR+1xjZQAk/4Z8oD82ZhcptKLlFiYT6rgqfEhyyZLWJzhcCs1/hZWIOY1w4dawVqTgW d8WUiPkmswAVQ2ZJA7cEYmCTloFj7bq02GkLLOvK6vWxTQS0Jt98A47U+/pOrTNJsVNb e2TA== X-Gm-Message-State: AOAM530T3jHHNaHv/rv2YWgM0+CcXcVu3LFATFjSrQK1EuuGAGFNdqCl 15g5VQc3QtaiJb8T/5f7nlDVftDvReqRyw== X-Google-Smtp-Source: ABdhPJw3ORFptR2N7NrnhHVAT4v5iW/nbwpLcjFHss3abkqyk5Qm0Xs2DBLzRql6Q6CPi6PKDjFl2g== X-Received: by 2002:a17:90a:a61:: with SMTP id o88mr7915546pjo.8.1619428110375; Mon, 26 Apr 2021 02:08:30 -0700 (PDT) Received: from localhost (160.131.236.35.bc.googleusercontent.com. [35.236.131.160]) by smtp.gmail.com with UTF8SMTPSA id x2sm10358014pfx.41.2021.04.26.02.08.27 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 26 Apr 2021 02:08:30 -0700 (PDT) From: Hikaru Nishida To: kvm@vger.kernel.org Cc: suleiman@google.com, Hikaru Nishida , Borislav Petkov , "H. Peter Anvin" , Ingo Molnar , Jim Mattson , Joerg Roedel , John Stultz , Paolo Bonzini , Sean Christopherson , Stephen Boyd , Thomas Gleixner , Vitaly Kuznetsov , Wanpeng Li , linux-kernel@vger.kernel.org, x86@kernel.org Subject: [RFC PATCH 2/6] x86/kvm: Add a struct and constants for virtual suspend time injection Date: Mon, 26 Apr 2021 18:06:41 +0900 Message-Id: <20210426090644.2218834-3-hikalium@chromium.org> X-Mailer: git-send-email 2.31.1.498.g6c1eba8ee3d-goog In-Reply-To: <20210426090644.2218834-1-hikalium@chromium.org> References: <20210426090644.2218834-1-hikalium@chromium.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org This patch adds definitions that are needed by both host and guest to implement virtual suspend time injection. This patch also adds #include in kernel/time/timekeeping.c to make necesarily functions which will be introduced in later patches available. Signed-off-by: Hikaru Nishida --- arch/x86/include/uapi/asm/kvm_para.h | 6 ++++++ kernel/time/timekeeping.c | 1 + 2 files changed, 7 insertions(+) diff --git a/arch/x86/include/uapi/asm/kvm_para.h b/arch/x86/include/uapi/asm/kvm_para.h index 950afebfba88..13788b01094f 100644 --- a/arch/x86/include/uapi/asm/kvm_para.h +++ b/arch/x86/include/uapi/asm/kvm_para.h @@ -33,6 +33,7 @@ #define KVM_FEATURE_PV_SCHED_YIELD 13 #define KVM_FEATURE_ASYNC_PF_INT 14 #define KVM_FEATURE_MSI_EXT_DEST_ID 15 +#define KVM_FEATURE_HOST_SUSPEND_TIME 16 #define KVM_HINTS_REALTIME 0 @@ -54,6 +55,7 @@ #define MSR_KVM_POLL_CONTROL 0x4b564d05 #define MSR_KVM_ASYNC_PF_INT 0x4b564d06 #define MSR_KVM_ASYNC_PF_ACK 0x4b564d07 +#define MSR_KVM_HOST_SUSPEND_TIME 0x4b564d08 struct kvm_steal_time { __u64 steal; @@ -64,6 +66,10 @@ struct kvm_steal_time { __u32 pad[11]; }; +struct kvm_host_suspend_time { + __u64 suspend_time_ns; +}; + #define KVM_VCPU_PREEMPTED (1 << 0) #define KVM_VCPU_FLUSH_TLB (1 << 1) diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c index 6aee5768c86f..ff0304de7de9 100644 --- a/kernel/time/timekeeping.c +++ b/kernel/time/timekeeping.c @@ -22,6 +22,7 @@ #include #include #include +#include #include "tick-internal.h" #include "ntp_internal.h" From patchwork Mon Apr 26 09:06:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hikaru Nishida X-Patchwork-Id: 12224003 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 32EF6C43461 for ; Mon, 26 Apr 2021 09:09:07 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 050306102A for ; Mon, 26 Apr 2021 09:09:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232711AbhDZJJq (ORCPT ); Mon, 26 Apr 2021 05:09:46 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59962 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232477AbhDZJJQ (ORCPT ); Mon, 26 Apr 2021 05:09:16 -0400 Received: from mail-pg1-x52b.google.com (mail-pg1-x52b.google.com [IPv6:2607:f8b0:4864:20::52b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 593DCC061756 for ; Mon, 26 Apr 2021 02:08:35 -0700 (PDT) Received: by mail-pg1-x52b.google.com with SMTP id b17so2632356pgh.7 for ; Mon, 26 Apr 2021 02:08:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Ze/d40V9r4jheKthRdtDHEg05bx8WqXy096MFG21BNk=; b=ZEW/ctj1rqE5BI7PZyNweAk4hCGwOn1XeB1UK3EInZuZ+KOoZj+FZniUs0eawFNTb3 piyAhNUq9Pt8NKjOGRjvXBUPiX9qxN9duDSUduOf6L6+VgKTaOlJcE4N0Fwb0qughoZE OAeozLRUZfPsZt//DGi8umsGUs0nsnU1cT4gM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Ze/d40V9r4jheKthRdtDHEg05bx8WqXy096MFG21BNk=; b=YDWeJO7eNFy32mBkURs1Jl6Fcyn8MoDOJvEIITlLz3LXVLNx1V076mehGRg9JVOkgk 1jcJ07tbUrDYDMqe25hD6Q1qhPLP5QVrWSvtx6nXf25jBXJbx1W7E0H9jJCSFX2AXTi4 wxJYsFVy0iKdtfGGj+utRTYpauOcs+zRB8OIUwspauOlAWdkMit453+vnGomlJdwbpdP cmkSsM02sQ3hiBtLkhTBNadX0pgnwlKU2zUaGyG1cQ6Plqi5BPvuckTvX91lNStjfzc9 FOPWz72r5OE4h2Z7P9Et2cXgEvTuLVPAoOXD51FUiWlnLVilHemKHrFL4kbqWZnPoSl4 /2Lg== X-Gm-Message-State: AOAM531xVEx8aMH5wtjtWI3+8GLX857HDlK/F1ovNtQaniCWaGEdW4qD dZvjKPOHmVe3meFv6bxvQ5RQher4vtD73A== X-Google-Smtp-Source: ABdhPJyaxGiniEDS09BITbtGyjkgjq743HWx/3/pq/SXK/DWtL+tSMT3yuzIQODND9+YCVlEcaq/bg== X-Received: by 2002:a63:a47:: with SMTP id z7mr15779744pgk.350.1619428114657; Mon, 26 Apr 2021 02:08:34 -0700 (PDT) Received: from localhost (160.131.236.35.bc.googleusercontent.com. [35.236.131.160]) by smtp.gmail.com with UTF8SMTPSA id 22sm14650977pjl.31.2021.04.26.02.08.32 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 26 Apr 2021 02:08:34 -0700 (PDT) From: Hikaru Nishida To: kvm@vger.kernel.org Cc: suleiman@google.com, Hikaru Nishida , Borislav Petkov , "H. Peter Anvin" , Ingo Molnar , Jim Mattson , Joerg Roedel , Paolo Bonzini , Sean Christopherson , Thomas Gleixner , Vitaly Kuznetsov , Wanpeng Li , linux-kernel@vger.kernel.org, x86@kernel.org Subject: [RFC PATCH 3/6] x86/kvm: Add CONFIG_KVM_VIRT_SUSPEND_TIMING Date: Mon, 26 Apr 2021 18:06:42 +0900 Message-Id: <20210426090644.2218834-4-hikalium@chromium.org> X-Mailer: git-send-email 2.31.1.498.g6c1eba8ee3d-goog In-Reply-To: <20210426090644.2218834-1-hikalium@chromium.org> References: <20210426090644.2218834-1-hikalium@chromium.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org The config option can be used to enable virtual suspend time injection support on kvm hosts. Signed-off-by: Hikaru Nishida --- arch/x86/kvm/Kconfig | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/arch/x86/kvm/Kconfig b/arch/x86/kvm/Kconfig index a788d5120d4d..6cb6795726a2 100644 --- a/arch/x86/kvm/Kconfig +++ b/arch/x86/kvm/Kconfig @@ -119,4 +119,17 @@ config KVM_MMU_AUDIT This option adds a R/W kVM module parameter 'mmu_audit', which allows auditing of KVM MMU events at runtime. +config KVM_VIRT_SUSPEND_TIMING + bool "Virtual suspend time injection" + depends on KVM=y + default n + help + This option makes the host's suspension reflected on the guest's clocks. + In other words, guest's CLOCK_MONOTONIC will stop and + CLOCK_BOOTTIME keeps running during the host's suspension. + This feature will only be effective when both guest and host enable + this option. + + If unsure, say N. + endif # VIRTUALIZATION From patchwork Mon Apr 26 09:06:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hikaru Nishida X-Patchwork-Id: 12224009 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AF007C433B4 for ; Mon, 26 Apr 2021 09:09:14 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 490626117A for ; Mon, 26 Apr 2021 09:09:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232228AbhDZJJx (ORCPT ); Mon, 26 Apr 2021 05:09:53 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59950 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232730AbhDZJJj (ORCPT ); Mon, 26 Apr 2021 05:09:39 -0400 Received: from mail-pl1-x62c.google.com (mail-pl1-x62c.google.com [IPv6:2607:f8b0:4864:20::62c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9861FC061756 for ; Mon, 26 Apr 2021 02:08:39 -0700 (PDT) Received: by mail-pl1-x62c.google.com with SMTP id e9so11614647plj.2 for ; Mon, 26 Apr 2021 02:08:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=WigoIf1bDqjv1l/y8T/BSRs1tlh6BcVRQlFS9r21Wyk=; b=O37T04+5AWwgCdxukdUFl/2gYu9CiWxGS9LJj2bmIfz11o3AEhBymw8dLTScUiiFfg 5N9kMbA4aHkAF7PMguRJNstyKo+ems6yhsi677RizCS28zD/GA/otyQrgnFILOAcN2VY DHj7SQgr7OuOQwwvme2+lKu2D0KkO9hxui4wM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=WigoIf1bDqjv1l/y8T/BSRs1tlh6BcVRQlFS9r21Wyk=; b=rfPMo+BHOyxpS6MCZLWQ5b8ZNLv4WUL1/Xj1GwqThLohsyJ8FiqCyg/S2+OCX0Px/g 39LP+cxiw9X5kYk6mgG3BfPlL2IWu83p4amR8AifiH+RQWHOsATqTPe8FCuU4AmMbiBh a3b+9vHdFAPiq4ypQCDd7GIbaWvJ5LV5jd4uIAridbs83VomVLq8fccdr/QgciE+4HWv MFPjg4pIK37/Ho1weIVxbPwBdQ3+yyooGweMyGY8WRn/qLxBI07HjgvM+jDrSOGLxomZ qJvisza4FTjmUOumKIRdNwQJ5qbTs7xRtTByGNGAJPiELg+rEORmZxWaPH/R5tXaMBfl IWxQ== X-Gm-Message-State: AOAM5333Uc2nJdXOZh0i7eF3UvzMylHLOYFhBuj4wYozSWnPOIfDINUA 8H2fZqIkzD+CjUpvvHIU/DLwqyEHBhMH9Q== X-Google-Smtp-Source: ABdhPJxZbgs+YGZOf9JouVrQNr4cdFdLfOz0VCnniBcKCMqWwTJ/+Za99oAbn5f027WdoSCC1NWIhQ== X-Received: by 2002:a17:90b:224d:: with SMTP id hk13mr20590501pjb.218.1619428118872; Mon, 26 Apr 2021 02:08:38 -0700 (PDT) Received: from localhost (160.131.236.35.bc.googleusercontent.com. [35.236.131.160]) by smtp.gmail.com with UTF8SMTPSA id d12sm11920666pjj.50.2021.04.26.02.08.35 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 26 Apr 2021 02:08:38 -0700 (PDT) From: Hikaru Nishida To: kvm@vger.kernel.org Cc: suleiman@google.com, Hikaru Nishida , Borislav Petkov , "H. Peter Anvin" , Ingo Molnar , Jim Mattson , Joerg Roedel , John Stultz , Paolo Bonzini , Sean Christopherson , Stephen Boyd , Thomas Gleixner , Vitaly Kuznetsov , Wanpeng Li , linux-kernel@vger.kernel.org, x86@kernel.org Subject: [RFC PATCH 4/6] x86/kvm: Add a host side support for virtual suspend time injection Date: Mon, 26 Apr 2021 18:06:43 +0900 Message-Id: <20210426090644.2218834-5-hikalium@chromium.org> X-Mailer: git-send-email 2.31.1.498.g6c1eba8ee3d-goog In-Reply-To: <20210426090644.2218834-1-hikalium@chromium.org> References: <20210426090644.2218834-1-hikalium@chromium.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org This patch implements virtual suspend time injection support for kvm hosts. If this functionality is enabled and the guest requests it, the host will stop all the clocks observed by the guest during the host's suspension and report the duration of suspend to the guest through struct kvm_host_suspend_time to give a chance to adjust CLOCK_BOOTTIME to the guest. This mechanism can be used to align the guest's clock behavior to the hosts' ones. Signed-off-by: Hikaru Nishida --- arch/x86/include/asm/kvm_host.h | 5 ++ arch/x86/kvm/cpuid.c | 4 ++ arch/x86/kvm/x86.c | 89 ++++++++++++++++++++++++++++++++- include/linux/kvm_host.h | 7 +++ kernel/time/timekeeping.c | 3 ++ 5 files changed, 107 insertions(+), 1 deletion(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 3768819693e5..6584adaab3bf 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -994,6 +994,11 @@ struct kvm_arch { gpa_t wall_clock; +#ifdef CONFIG_KVM_VIRT_SUSPEND_TIMING + struct gfn_to_hva_cache suspend_time; + bool suspend_time_injection_enabled; +#endif /* KVM_VIRT_SUSPEND_TIMING */ + bool mwait_in_guest; bool hlt_in_guest; bool pause_in_guest; diff --git a/arch/x86/kvm/cpuid.c b/arch/x86/kvm/cpuid.c index 6bd2f8b830e4..62224b7bd7f9 100644 --- a/arch/x86/kvm/cpuid.c +++ b/arch/x86/kvm/cpuid.c @@ -814,6 +814,10 @@ static inline int __do_cpuid_func(struct kvm_cpuid_array *array, u32 function) (1 << KVM_FEATURE_PV_SCHED_YIELD) | (1 << KVM_FEATURE_ASYNC_PF_INT); +#ifdef CONFIG_KVM_VIRT_SUSPEND_TIMING + entry->eax |= (1 << KVM_FEATURE_HOST_SUSPEND_TIME); +#endif + if (sched_info_on()) entry->eax |= (1 << KVM_FEATURE_STEAL_TIME); diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index eca63625aee4..d919f771ce31 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -1285,6 +1285,9 @@ static const u32 emulated_msrs_all[] = { MSR_KVM_ASYNC_PF_EN, MSR_KVM_STEAL_TIME, MSR_KVM_PV_EOI_EN, MSR_KVM_ASYNC_PF_INT, MSR_KVM_ASYNC_PF_ACK, +#ifdef CONFIG_KVM_VIRT_SUSPEND_TIMING + MSR_KVM_HOST_SUSPEND_TIME, +#endif MSR_IA32_TSC_ADJUST, MSR_IA32_TSCDEADLINE, @@ -2020,6 +2023,20 @@ static void kvm_write_system_time(struct kvm_vcpu *vcpu, gpa_t system_time, return; } +#ifdef CONFIG_KVM_VIRT_SUSPEND_TIMING +static void kvm_write_suspend_time(struct kvm *kvm, bool updated) +{ + struct kvm_arch *ka = &kvm->arch; + struct kvm_host_suspend_time st; + + if (!ka->suspend_time_injection_enabled) + return; + + st.suspend_time_ns = kvm->suspend_time_ns; + kvm_write_guest_cached(kvm, &ka->suspend_time, &st, sizeof(st)); +} +#endif + static uint32_t div_frac(uint32_t dividend, uint32_t divisor) { do_shl32_div32(dividend, divisor); @@ -2653,6 +2670,9 @@ static void kvm_setup_pvclock_page(struct kvm_vcpu *v, if (vcpu->pvclock_set_guest_stopped_request) { vcpu->hv_clock.flags |= PVCLOCK_GUEST_STOPPED; +#ifdef CONFIG_KVM_VIRT_SUSPEND_TIMING + kvm_write_suspend_time(v->kvm, true); +#endif vcpu->pvclock_set_guest_stopped_request = false; } @@ -3229,7 +3249,23 @@ int kvm_set_msr_common(struct kvm_vcpu *vcpu, struct msr_data *msr_info) vcpu->arch.msr_kvm_poll_control = data; break; - +#ifdef CONFIG_KVM_VIRT_SUSPEND_TIMING + case MSR_KVM_HOST_SUSPEND_TIME: + if (!guest_pv_has(vcpu, KVM_FEATURE_HOST_SUSPEND_TIME)) + return 1; + if (!(data & KVM_MSR_ENABLED)) { + vcpu->kvm->arch.suspend_time_injection_enabled = false; + break; + } + if (kvm_gfn_to_hva_cache_init(vcpu->kvm, + &vcpu->kvm->arch.suspend_time, data & ~KVM_MSR_ENABLED, + sizeof(struct kvm_host_suspend_time))) { + return 1; + } + vcpu->kvm->arch.suspend_time_injection_enabled = true; + kvm_write_suspend_time(vcpu->kvm, false); + break; +#endif case MSR_IA32_MCG_CTL: case MSR_IA32_MCG_STATUS: case MSR_IA32_MC0_CTL ... MSR_IA32_MCx_CTL(KVM_MAX_MCE_BANKS) - 1: @@ -3535,6 +3571,15 @@ int kvm_get_msr_common(struct kvm_vcpu *vcpu, struct msr_data *msr_info) msr_info->data = vcpu->arch.msr_kvm_poll_control; break; +#ifdef CONFIG_KVM_VIRT_SUSPEND_TIMING + case MSR_KVM_HOST_SUSPEND_TIME: + if (!guest_pv_has(vcpu, KVM_FEATURE_HOST_SUSPEND_TIME)) + return 1; + msr_info->data = vcpu->kvm->arch.suspend_time.gpa; + if (vcpu->kvm->arch.suspend_time_injection_enabled) + msr_info->data |= KVM_MSR_ENABLED; + break; +#endif case MSR_IA32_P5_MC_ADDR: case MSR_IA32_P5_MC_TYPE: case MSR_IA32_MCG_CAP: @@ -11723,6 +11768,48 @@ int kvm_sev_es_string_io(struct kvm_vcpu *vcpu, unsigned int size, } EXPORT_SYMBOL_GPL(kvm_sev_es_string_io); +#ifdef CONFIG_KVM_VIRT_SUSPEND_TIMING +void kvm_arch_timekeeping_inject_sleeptime(const struct timespec64 *delta) +{ + struct kvm_vcpu *vcpu; + u64 suspend_time_ns; + struct kvm *kvm; + s64 adj; + int i; + + suspend_time_ns = timespec64_to_ns(delta); + adj = tsc_khz * (suspend_time_ns / 1000000); + /* + * Adjust TSCs on all vcpus as if they are stopped during a suspend. + * doing a similar thing in kvm_arch_hardware_enable(). + */ + mutex_lock(&kvm_lock); + list_for_each_entry(kvm, &vm_list, vm_list) { + if (!kvm->arch.suspend_time_injection_enabled) + continue; + + kvm_for_each_vcpu(i, vcpu, kvm) { + /* + * Adjustment here is always smaller than the gap + * observed by the guest, so subtracting the value + * here never rewinds the observed TSC in guests. + * This adjustment will be applied on the next + * kvm_arch_vcpu_load(). + */ + vcpu->arch.tsc_offset_adjustment -= adj; + } + /* + * Move the offset of kvm_clock here as if it is stopped + * during the suspension. + */ + kvm->arch.kvmclock_offset -= suspend_time_ns; + /* suspend_time is accumulated per VM. */ + kvm->suspend_time_ns += suspend_time_ns; + } + mutex_unlock(&kvm_lock); +} +#endif + EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_entry); EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_exit); EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_fast_mmio); diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 1b65e7204344..e077b9a960fc 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -520,6 +520,9 @@ struct kvm { pid_t userspace_pid; unsigned int max_halt_poll_ns; u32 dirty_ring_size; +#ifdef CONFIG_KVM_VIRT_SUSPEND_TIMING + u64 suspend_time_ns; +#endif }; #define kvm_err(fmt, ...) \ @@ -1522,4 +1525,8 @@ static inline void kvm_handle_signal_exit(struct kvm_vcpu *vcpu) /* Max number of entries allowed for each kvm dirty ring */ #define KVM_DIRTY_RING_MAX_ENTRIES 65536 +#ifdef CONFIG_KVM_VIRT_SUSPEND_TIMING +void kvm_arch_timekeeping_inject_sleeptime(const struct timespec64 *delta); +#endif /* CONFIG_KVM_VIRT_SUSPEND_TIMING */ + #endif diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c index ff0304de7de9..342a032ad552 100644 --- a/kernel/time/timekeeping.c +++ b/kernel/time/timekeeping.c @@ -1786,6 +1786,9 @@ void timekeeping_resume(void) if (inject_sleeptime) { suspend_timing_needed = false; __timekeeping_inject_sleeptime(tk, &ts_delta); +#ifdef CONFIG_KVM_VIRT_SUSPEND_TIMING + kvm_arch_timekeeping_inject_sleeptime(&ts_delta); +#endif } /* Re-base the last cycle value */ From patchwork Mon Apr 26 09:06:44 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hikaru Nishida X-Patchwork-Id: 12224005 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 10C19C43462 for ; Mon, 26 Apr 2021 09:09:08 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id CD71861153 for ; Mon, 26 Apr 2021 09:09:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232504AbhDZJJr (ORCPT ); Mon, 26 Apr 2021 05:09:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59946 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232728AbhDZJJY (ORCPT ); Mon, 26 Apr 2021 05:09:24 -0400 Received: from mail-pj1-x1034.google.com (mail-pj1-x1034.google.com [IPv6:2607:f8b0:4864:20::1034]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A2EC3C061760 for ; Mon, 26 Apr 2021 02:08:42 -0700 (PDT) Received: by mail-pj1-x1034.google.com with SMTP id f2-20020a17090a4a82b02900c67bf8dc69so4809015pjh.1 for ; Mon, 26 Apr 2021 02:08:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=TUB8ZW50W1LH/QwrQax9EjDH6gcEGV3rni30bl3X4NY=; b=nMLVxO6CIiy/xoC7MTIJbp07rnSHaLHlZQaXqT9LsKQyH9vwAf5oAote5EctuSpRnV ZNk0sQSWlXUvYSx+7YcGU6axtY+S79n1oE2UhCNrFJOcSEL4YsgyWEJGtbdGKLJkLgkI QennNi2LcO+vB/r1UTYKCXEQNpal4XjwjKvKQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=TUB8ZW50W1LH/QwrQax9EjDH6gcEGV3rni30bl3X4NY=; b=sZN0euoZbJ4ZxYHoe0w89AIai0rryeYKxVlinilmxUJHni8SrOsC2haUCL91hUtGrg ppNqedAQFZb7ujLXcDLc+gDaZm41Bp9Ig/93l02I3wfJ7/NY575iWAnm7Ur2KyyiBgEE tlaiz8Ql16/H7Z7EfQLz64hEwDbx3tHUFgU+sdxCyqwIzqVSYMfAbSeKrwConl6YuRe7 1tZ5OniSdAwHU7IaGQfiCcRD82HV4s+vcXJ6MNZluVrj8KkLbn4Qaz8bz4JvdYs2GWml I6m72KEKOHRqalqwbq2IWstU6Ngb2+MbC0zB8YeOe8H+Kbs0xwZni2ZScGeIYi7Na/Z0 hktw== X-Gm-Message-State: AOAM530H/tTB1OA23uvfvTk2k5SgFthPi7cdsPsVvUXMHK8ayVmpg3Cd d5dMw2AXnlWN5cVn6f2NI0YFAI1oC1W77g== X-Google-Smtp-Source: ABdhPJxz8tmuXwED9CTvIhb+i/UFSl3au8tJ/pNDTPQq4thJ6fB38kVQUMNqaZ8W+CGIK8/FjIcZJA== X-Received: by 2002:a17:90a:c8:: with SMTP id v8mr21678604pjd.18.1619428121944; Mon, 26 Apr 2021 02:08:41 -0700 (PDT) Received: from localhost (160.131.236.35.bc.googleusercontent.com. [35.236.131.160]) by smtp.gmail.com with UTF8SMTPSA id j29sm10602171pgl.30.2021.04.26.02.08.40 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 26 Apr 2021 02:08:41 -0700 (PDT) From: Hikaru Nishida To: kvm@vger.kernel.org Cc: suleiman@google.com, Hikaru Nishida , Borislav Petkov , "H. Peter Anvin" , Ingo Molnar , Thomas Gleixner , linux-kernel@vger.kernel.org, x86@kernel.org Subject: [RFC PATCH 5/6] x86/kvm: Add CONFIG_KVM_VIRT_SUSPEND_TIMING_GUEST Date: Mon, 26 Apr 2021 18:06:44 +0900 Message-Id: <20210426090644.2218834-6-hikalium@chromium.org> X-Mailer: git-send-email 2.31.1.498.g6c1eba8ee3d-goog In-Reply-To: <20210426090644.2218834-1-hikalium@chromium.org> References: <20210426090644.2218834-1-hikalium@chromium.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org The config option can be used to enable virtual suspend time injection support on kvm guests. Signed-off-by: Hikaru Nishida --- arch/x86/Kconfig | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index 2792879d398e..fac06534c30a 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -825,6 +825,19 @@ config KVM_GUEST underlying device model, the host provides the guest with timing infrastructure such as time of day, and system time +config KVM_VIRT_SUSPEND_TIMING_GUEST + bool "Virtual suspend time injection (guest side)" + depends on KVM_GUEST + default n + help + This option makes the host's suspension reflected on the guest's clocks. + In other words, guest's CLOCK_MONOTONIC will stop and + CLOCK_BOOTTIME keeps running during the host's suspension. + This feature will only be effective when both guest and host enable + this option. + + If unsure, say N. + config ARCH_CPUIDLE_HALTPOLL def_bool n prompt "Disable host haltpoll when loading haltpoll driver" From patchwork Mon Apr 26 09:06:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hikaru Nishida X-Patchwork-Id: 12224007 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E82B8C433ED for ; Mon, 26 Apr 2021 09:09:12 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B1BE861153 for ; Mon, 26 Apr 2021 09:09:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232506AbhDZJJs (ORCPT ); Mon, 26 Apr 2021 05:09:48 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60046 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232760AbhDZJJl (ORCPT ); Mon, 26 Apr 2021 05:09:41 -0400 Received: from mail-pj1-x1034.google.com (mail-pj1-x1034.google.com [IPv6:2607:f8b0:4864:20::1034]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2DB6AC061763 for ; Mon, 26 Apr 2021 02:08:47 -0700 (PDT) Received: by mail-pj1-x1034.google.com with SMTP id h14-20020a17090aea8eb02901553e1cc649so2681213pjz.0 for ; Mon, 26 Apr 2021 02:08:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=FyptFyp3SyGnFZnAfWqIco1W5xYNar4z+brwR/7K1Qs=; b=Dwj2TQhSHanr+DcnkGMAn4cCQdrEqWE2GA5gf5cA/g0o5BfK70YUus85kYUeUaMSW9 8VawpLzkbE0lkJdot8kmkSIMyW5JtDR7rzvl7kkygWLyXwzey3gfz47HFFW51aql0GwY G8gek6QXOKddMzftArLWvsRYJdrtIUodSyLvI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=FyptFyp3SyGnFZnAfWqIco1W5xYNar4z+brwR/7K1Qs=; b=WCMh9BwuaS3uQbj9N/Z52SWSSWoifM9hhFEGD4YGEdomcIxmzZ0+B+oYH8GMH9VDUm KZmdMYwdPBlEcTlzGHIO3Gfi2c4TIqt3QwO4c381NJxl9QX9MVsLDRaIoFRCrOrH207g GpqQMbS2DblpMGOTDoWbg2nPEjQRSOCHdPebiTPq2itwogx3yoE99zNafcHZvgS0LCef ZTEmrVDZ3mWutyu/fFUSurUKt1LZs4lHz2WsOFwzPKmkc+3FKlU/7SddHu2KHJpg7EWp w7zaa0qtWzn5h/iSezN0yoQ3nqeVD9ClICbCD/zGbvZjMjfuxep0QJOG4jd8V5oDPN8e 46Cw== X-Gm-Message-State: AOAM531+nKaXsnAkEYwQ5wl/b1stQ8TM8zpvpW9776OX5EZVVTtYMAzI sWUkOYR8dsoLY+DXPFs2iNV+sOs1VUB5LQ== X-Google-Smtp-Source: ABdhPJxZMCEPxK7rD0Opc39txVircSohOFkFpiPkj5pJ5scEvSMNlFtubplHz81Ev36IJZ8c3+uMaw== X-Received: by 2002:a17:90a:e643:: with SMTP id ep3mr9330612pjb.194.1619428126486; Mon, 26 Apr 2021 02:08:46 -0700 (PDT) Received: from localhost (160.131.236.35.bc.googleusercontent.com. [35.236.131.160]) by smtp.gmail.com with UTF8SMTPSA id i11sm10716676pfo.183.2021.04.26.02.08.43 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 26 Apr 2021 02:08:46 -0700 (PDT) From: Hikaru Nishida To: kvm@vger.kernel.org Cc: suleiman@google.com, Hikaru Nishida , Borislav Petkov , "H. Peter Anvin" , Ingo Molnar , Jim Mattson , Joerg Roedel , John Stultz , Paolo Bonzini , Sean Christopherson , Stephen Boyd , Thomas Gleixner , Vitaly Kuznetsov , Wanpeng Li , linux-kernel@vger.kernel.org, x86@kernel.org Subject: [RFC PATCH 6/6] x86/kvm: Add a guest side support for virtual suspend time injection Date: Mon, 26 Apr 2021 18:06:45 +0900 Message-Id: <20210426090644.2218834-7-hikalium@chromium.org> X-Mailer: git-send-email 2.31.1.498.g6c1eba8ee3d-goog In-Reply-To: <20210426090644.2218834-1-hikalium@chromium.org> References: <20210426090644.2218834-1-hikalium@chromium.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org This patch implements virtual suspend time injection support for kvm guests. If this functionality is enabled and the host supports KVM_FEATURE_HOST_SUSPEND_TIME, the guest will register struct kvm_host_suspend_time through MSR to monitor how much time we spend during the host's suspension. If the duration is increased, the guest will advance CLOCK_BOOTTIME to match with the host's suspension. Signed-off-by: Hikaru Nishida --- arch/x86/include/asm/kvm_para.h | 9 +++++++++ arch/x86/kernel/kvmclock.c | 25 +++++++++++++++++++++++++ include/linux/timekeeper_internal.h | 4 ++++ kernel/time/timekeeping.c | 27 +++++++++++++++++++++++++++ 4 files changed, 65 insertions(+) diff --git a/arch/x86/include/asm/kvm_para.h b/arch/x86/include/asm/kvm_para.h index 338119852512..e552efa931a8 100644 --- a/arch/x86/include/asm/kvm_para.h +++ b/arch/x86/include/asm/kvm_para.h @@ -18,6 +18,15 @@ static inline bool kvm_check_and_clear_guest_paused(void) } #endif /* CONFIG_KVM_GUEST */ +#ifdef CONFIG_KVM_VIRT_SUSPEND_TIMING_GUEST +u64 kvm_get_suspend_time(void); +#else +static inline u64 kvm_get_suspend_time(void) +{ + return 0; +} +#endif /* CONFIG_KVM_VIRT_SUSPEND_TIMING_GUEST */ + #define KVM_HYPERCALL \ ALTERNATIVE("vmcall", "vmmcall", X86_FEATURE_VMMCALL) diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c index 1fc0962c89c0..630460beb05a 100644 --- a/arch/x86/kernel/kvmclock.c +++ b/arch/x86/kernel/kvmclock.c @@ -49,6 +49,9 @@ early_param("no-kvmclock-vsyscall", parse_no_kvmclock_vsyscall); static struct pvclock_vsyscall_time_info hv_clock_boot[HVC_BOOT_ARRAY_SIZE] __bss_decrypted __aligned(PAGE_SIZE); +#ifdef CONFIG_KVM_VIRT_SUSPEND_TIMING_GUEST +static struct kvm_host_suspend_time suspend_time __bss_decrypted; +#endif static struct pvclock_wall_clock wall_clock __bss_decrypted; static DEFINE_PER_CPU(struct pvclock_vsyscall_time_info *, hv_clock_per_cpu); static struct pvclock_vsyscall_time_info *hvclock_mem; @@ -164,6 +167,20 @@ static int kvm_cs_enable(struct clocksource *cs) return 0; } +#ifdef CONFIG_KVM_VIRT_SUSPEND_TIMING_GUEST +/* + * kvm_get_suspend_time_and_clear_request - This function + * checks how much suspend time is injected by the host. + * A returned value is a total time passed during the guest in a suspend + * state while this guest is running. (Not a duration of the last host + * suspend.) + */ +u64 kvm_get_suspend_time(void) +{ + return suspend_time.suspend_time_ns; +} +#endif /* CONFIG_KVM_VIRT_SUSPEND_TIMING_GUEST */ + struct clocksource kvm_clock = { .name = "kvm-clock", .read = kvm_clock_get_cycles, @@ -324,6 +341,14 @@ void __init kvmclock_init(void) return; } +#ifdef CONFIG_KVM_VIRT_SUSPEND_TIMING_GUEST + if (kvm_para_has_feature(KVM_FEATURE_HOST_SUSPEND_TIME)) { + /* Register the suspend time structure */ + wrmsrl(MSR_KVM_HOST_SUSPEND_TIME, + slow_virt_to_phys(&suspend_time) | KVM_MSR_ENABLED); + } +#endif + if (cpuhp_setup_state(CPUHP_BP_PREPARE_DYN, "kvmclock:setup_percpu", kvmclock_setup_percpu, NULL) < 0) { return; diff --git a/include/linux/timekeeper_internal.h b/include/linux/timekeeper_internal.h index 84ff2844df2a..a5fd515f0a9d 100644 --- a/include/linux/timekeeper_internal.h +++ b/include/linux/timekeeper_internal.h @@ -124,6 +124,10 @@ struct timekeeper { u32 ntp_err_mult; /* Flag used to avoid updating NTP twice with same second */ u32 skip_second_overflow; +#ifdef CONFIG_KVM_VIRT_SUSPEND_TIMING_GUEST + /* suspend_time_injected keeps the duration injected through kvm */ + u64 suspend_time_injected; +#endif #ifdef CONFIG_DEBUG_TIMEKEEPING long last_warning; /* diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c index 342a032ad552..6b89e0d42596 100644 --- a/kernel/time/timekeeping.c +++ b/kernel/time/timekeeping.c @@ -2114,6 +2114,29 @@ static u64 logarithmic_accumulation(struct timekeeper *tk, u64 offset, return offset; } +#ifdef CONFIG_KVM_VIRT_SUSPEND_TIMING_GUEST +/* + * timekeeping_inject_suspend_time - Inject virtual suspend time + * if it is requested by kvm host. + * This function should be called under holding timekeeper_lock and + * only from timekeeping_advance(). + */ +static void timekeeping_inject_virtual_suspend_time(struct timekeeper *tk) +{ + struct timespec64 delta; + u64 suspend_time; + + suspend_time = kvm_get_suspend_time(); + if (suspend_time <= tk->suspend_time_injected) { + /* Sufficient amount of suspend time is already injected. */ + return; + } + delta = ns_to_timespec64(suspend_time - tk->suspend_time_injected); + __timekeeping_inject_sleeptime(tk, &delta); + tk->suspend_time_injected = suspend_time; +} +#endif + /* * timekeeping_advance - Updates the timekeeper to the current time and * current NTP tick length @@ -2143,6 +2166,10 @@ static void timekeeping_advance(enum timekeeping_adv_mode mode) /* Do some additional sanity checking */ timekeeping_check_update(tk, offset); +#ifdef CONFIG_KVM_VIRT_SUSPEND_TIMING_GUEST + timekeeping_inject_virtual_suspend_time(tk); +#endif + /* * With NO_HZ we may have to accumulate many cycle_intervals * (think "ticks") worth of time at once. To do this efficiently,