From patchwork Fri Sep 14 00:38:26 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Liran Alon X-Patchwork-Id: 10600211 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C1F0B14D6 for ; Fri, 14 Sep 2018 00:39:19 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3A5812B268 for ; Fri, 14 Sep 2018 00:39:19 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2E3572B2A3; Fri, 14 Sep 2018 00:39:19 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EA8152B268 for ; Fri, 14 Sep 2018 00:39:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726969AbeINFvH (ORCPT ); Fri, 14 Sep 2018 01:51:07 -0400 Received: from userp2120.oracle.com ([156.151.31.85]:49282 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726873AbeINFvH (ORCPT ); Fri, 14 Sep 2018 01:51:07 -0400 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.22/8.16.0.22) with SMTP id w8E0YYke186588; Fri, 14 Sep 2018 00:38:54 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=corp-2018-07-02; bh=UGFyisa0N7LYS/aIlb1S/6VxUjz957q4kytLrlcB2oA=; b=nJ4otqbxz4YypR3eNZ+EjFJQ8yGOZJw5KO+p9b7Whg8BzZaWrFdN9rDe+k9abrVn9ViP L5wTuYT/MD2nmof6qKmilzQCzRFyyMFxY7s66yy2XtJSHR2R23y5jL8bUic+QWMN1PJV f5RQX7TPTFVXsLmYlaNcJ625VM6n8zLKOHGfM9d5KGbQR0iAnzxzlmoTkB3EMPnPqytD xhpirwLjPLbiAOaUzHkBcDkOSvGMMIQUr+w9uSdDZYyJBz7WoltO+ez8kvbwI47KJ+Tb DAfVQTSUhXsYeniuZS0RkFX5Yj0DoBcUFPRONcHwZ9UVy3Z3Y/xuYhrenwYhLuamm8IB xw== Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) by userp2120.oracle.com with ESMTP id 2mc72r499t-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 14 Sep 2018 00:38:54 +0000 Received: from aserv0122.oracle.com (aserv0122.oracle.com [141.146.126.236]) by aserv0022.oracle.com (8.14.4/8.14.4) with ESMTP id w8E0crVa028443 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 14 Sep 2018 00:38:53 GMT Received: from abhmp0002.oracle.com (abhmp0002.oracle.com [141.146.116.8]) by aserv0122.oracle.com (8.14.4/8.14.4) with ESMTP id w8E0cr94014145; Fri, 14 Sep 2018 00:38:53 GMT Received: from spark.ravello.local (/213.57.127.2) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Fri, 14 Sep 2018 00:38:52 +0000 From: Liran Alon To: qemu-devel@nongnu.org, rth@twiddle.net, ehabkost@redhat.com Cc: pbonzini@redhat.com, mtosatti@redhat.com, kvm@vger.kernel.org, jmattson@google.com, idan.brown@oracle.com, Liran Alon Subject: [PATCH 1/2] i386: Compile CPUX86State xsave_buf only when support KVM or HVF Date: Fri, 14 Sep 2018 03:38:26 +0300 Message-Id: <20180914003827.124570-2-liran.alon@oracle.com> X-Mailer: git-send-email 2.16.1 In-Reply-To: <20180914003827.124570-1-liran.alon@oracle.com> References: <20180914003827.124570-1-liran.alon@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9015 signatures=668708 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=975 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1807170000 definitions=main-1809140005 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP While at it, also rename var to indicate it is not used only in KVM. Reviewed-by: Nikita Leshchenko Reviewed-by: Patrick Colp Signed-off-by: Liran Alon --- target/i386/cpu.h | 4 +++- target/i386/hvf/README.md | 2 +- target/i386/hvf/hvf.c | 2 +- target/i386/hvf/x86hvf.c | 4 ++-- target/i386/kvm.c | 6 +++--- 5 files changed, 10 insertions(+), 8 deletions(-) diff --git a/target/i386/cpu.h b/target/i386/cpu.h index b572a8e4aa41..6e4c2b02f947 100644 --- a/target/i386/cpu.h +++ b/target/i386/cpu.h @@ -1327,7 +1327,9 @@ typedef struct CPUX86State { bool tsc_valid; int64_t tsc_khz; int64_t user_tsc_khz; /* for sanity check only */ - void *kvm_xsave_buf; +#if defined(CONFIG_KVM) || defined(CONFIG_HVF) + void *xsave_buf; +#endif #if defined(CONFIG_HVF) HVFX86EmulatorState *hvf_emul; #endif diff --git a/target/i386/hvf/README.md b/target/i386/hvf/README.md index 0d27a0d52b58..2d33477aca50 100644 --- a/target/i386/hvf/README.md +++ b/target/i386/hvf/README.md @@ -2,6 +2,6 @@ These sources (and ../hvf-all.c) are adapted from Veertu Inc's vdhh (Veertu Desktop Hosted Hypervisor) (last known location: https://github.com/veertuinc/vdhh) with some minor changes, the most significant of which were: -1. Adapt to our current QEMU's `CPUState` structure and `address_space_rw` API; many struct members have been moved around (emulated x86 state, kvm_xsave_buf) due to historical differences + QEMU needing to handle more emulation targets. +1. Adapt to our current QEMU's `CPUState` structure and `address_space_rw` API; many struct members have been moved around (emulated x86 state, xsave_buf) due to historical differences + QEMU needing to handle more emulation targets. 2. Removal of `apic_page` and hyperv-related functionality. 3. More relaxed use of `qemu_mutex_lock_iothread`. diff --git a/target/i386/hvf/hvf.c b/target/i386/hvf/hvf.c index df69e6d0a7af..5db167df98e6 100644 --- a/target/i386/hvf/hvf.c +++ b/target/i386/hvf/hvf.c @@ -587,7 +587,7 @@ int hvf_init_vcpu(CPUState *cpu) hvf_reset_vcpu(cpu); x86cpu = X86_CPU(cpu); - x86cpu->env.kvm_xsave_buf = qemu_memalign(4096, 4096); + x86cpu->env.xsave_buf = qemu_memalign(4096, 4096); hv_vcpu_enable_native_msr(cpu->hvf_fd, MSR_STAR, 1); hv_vcpu_enable_native_msr(cpu->hvf_fd, MSR_LSTAR, 1); diff --git a/target/i386/hvf/x86hvf.c b/target/i386/hvf/x86hvf.c index 6c88939b968b..df8e946fbcde 100644 --- a/target/i386/hvf/x86hvf.c +++ b/target/i386/hvf/x86hvf.c @@ -75,7 +75,7 @@ void hvf_put_xsave(CPUState *cpu_state) struct X86XSaveArea *xsave; - xsave = X86_CPU(cpu_state)->env.kvm_xsave_buf; + xsave = X86_CPU(cpu_state)->env.xsave_buf; x86_cpu_xsave_all_areas(X86_CPU(cpu_state), xsave); @@ -163,7 +163,7 @@ void hvf_get_xsave(CPUState *cpu_state) { struct X86XSaveArea *xsave; - xsave = X86_CPU(cpu_state)->env.kvm_xsave_buf; + xsave = X86_CPU(cpu_state)->env.xsave_buf; if (hv_vcpu_read_fpstate(cpu_state->hvf_fd, (void*)xsave, 4096)) { abort(); diff --git a/target/i386/kvm.c b/target/i386/kvm.c index 0b2a07d3a47b..c1cd8c461fe4 100644 --- a/target/i386/kvm.c +++ b/target/i386/kvm.c @@ -1189,7 +1189,7 @@ int kvm_arch_init_vcpu(CPUState *cs) } if (has_xsave) { - env->kvm_xsave_buf = qemu_memalign(4096, sizeof(struct kvm_xsave)); + env->xsave_buf = qemu_memalign(4096, sizeof(struct kvm_xsave)); } cpu->kvm_msr_buf = g_malloc0(MSR_BUF_SIZE); @@ -1639,7 +1639,7 @@ ASSERT_OFFSET(XSAVE_PKRU, pkru_state); static int kvm_put_xsave(X86CPU *cpu) { CPUX86State *env = &cpu->env; - X86XSaveArea *xsave = env->kvm_xsave_buf; + X86XSaveArea *xsave = env->xsave_buf; if (!has_xsave) { return kvm_put_fpu(cpu); @@ -2081,7 +2081,7 @@ static int kvm_get_fpu(X86CPU *cpu) static int kvm_get_xsave(X86CPU *cpu) { CPUX86State *env = &cpu->env; - X86XSaveArea *xsave = env->kvm_xsave_buf; + X86XSaveArea *xsave = env->xsave_buf; int ret; if (!has_xsave) { From patchwork Fri Sep 14 00:38:27 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Liran Alon X-Patchwork-Id: 10600213 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A241E14D6 for ; Fri, 14 Sep 2018 00:39:21 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 19A7B2B26F for ; Fri, 14 Sep 2018 00:39:21 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 0DBDD2B2A8; Fri, 14 Sep 2018 00:39:21 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6B30D2B26F for ; Fri, 14 Sep 2018 00:39:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728240AbeINFvK (ORCPT ); Fri, 14 Sep 2018 01:51:10 -0400 Received: from userp2120.oracle.com ([156.151.31.85]:49292 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726873AbeINFvK (ORCPT ); Fri, 14 Sep 2018 01:51:10 -0400 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.22/8.16.0.22) with SMTP id w8E0YskP186661; Fri, 14 Sep 2018 00:38:59 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references; s=corp-2018-07-02; bh=+hb18Qg1cYqSxVQyYrhmKewiNHw8IEYlndWbis8j4mA=; b=Dm7O1FYzeicUsT5mdodcBU91C7jCM8PkI+Xv6ovgh8GODrYi5GroolYHkophFpHSaloL Pei37hm+fbs5vn5EGWDw1PYwar45H+nswmcUrz3evay0sw9+lpKxj1veIM/sYEkJ1vc8 za3pyBfTkLboFGfOTGEpzUckrPoq2iuc3dngenLIESndyfvviFzoIVXpkU7vCtKmB2x4 9ndGsih26dNIam4fbVr2Vj1c8AlszfpoS9kNkvQwc3k0As1Ysj8jGXxIBuUf+PwFBzBv o9GB0lgg5sD6Z4dyTaQOYu4FC6SG/6HRqmVfm8/Yfr1RIA5b+/ESYK4Io49caOhfL1II DA== Received: from aserv0021.oracle.com (aserv0021.oracle.com [141.146.126.233]) by userp2120.oracle.com with ESMTP id 2mc72r49a5-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 14 Sep 2018 00:38:59 +0000 Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by aserv0021.oracle.com (8.14.4/8.14.4) with ESMTP id w8E0cwLZ016409 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 14 Sep 2018 00:38:58 GMT Received: from abhmp0002.oracle.com (abhmp0002.oracle.com [141.146.116.8]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id w8E0cvkb020292; Fri, 14 Sep 2018 00:38:57 GMT Received: from spark.ravello.local (/213.57.127.2) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Fri, 14 Sep 2018 00:38:57 +0000 From: Liran Alon To: qemu-devel@nongnu.org, rth@twiddle.net, ehabkost@redhat.com Cc: pbonzini@redhat.com, mtosatti@redhat.com, kvm@vger.kernel.org, jmattson@google.com, idan.brown@oracle.com, Liran Alon Subject: [PATCH 2/2] KVM: i386: Add support for save and restore nested state Date: Fri, 14 Sep 2018 03:38:27 +0300 Message-Id: <20180914003827.124570-3-liran.alon@oracle.com> X-Mailer: git-send-email 2.16.1 In-Reply-To: <20180914003827.124570-1-liran.alon@oracle.com> References: <20180914003827.124570-1-liran.alon@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9015 signatures=668708 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1807170000 definitions=main-1809140005 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Kernel commit 8fcc4b5923af ("kvm: nVMX: Introduce KVM_CAP_NESTED_STATE") introduced new IOCTLs to extract and restore KVM internal state used to run a VM that is in VMX operation. Utilize these IOCTLs to add support of migration of VMs which are running nested hypervisors. Reviewed-by: Nikita Leshchenko Reviewed-by: Patrick Colp Signed-off-by: Liran Alon --- accel/kvm/kvm-all.c | 14 +++++++++++++ include/sysemu/kvm.h | 1 + target/i386/cpu.h | 2 ++ target/i386/kvm.c | 58 +++++++++++++++++++++++++++++++++++++++++++++++++++ target/i386/machine.c | 38 +++++++++++++++++++++++++++++++++ 5 files changed, 113 insertions(+) diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c index de12f78eb8e4..3f0d0dd741b9 100644 --- a/accel/kvm/kvm-all.c +++ b/accel/kvm/kvm-all.c @@ -87,6 +87,7 @@ struct KVMState #ifdef KVM_CAP_SET_GUEST_DEBUG struct kvm_sw_breakpoint_head kvm_sw_breakpoints; #endif + uint32_t nested_state_len; int many_ioeventfds; int intx_set_mask; bool sync_mmu; @@ -1628,6 +1629,14 @@ static int kvm_init(MachineState *ms) s->debugregs = kvm_check_extension(s, KVM_CAP_DEBUGREGS); #endif + ret = kvm_check_extension(s, KVM_CAP_NESTED_STATE); + if (ret < 0) { + fprintf(stderr, "kvm failed to get size of nested state (%d)", + ret); + goto err; + } + s->nested_state_len = (uint32_t)ret; + #ifdef KVM_CAP_IRQ_ROUTING kvm_direct_msi_allowed = (kvm_check_extension(s, KVM_CAP_SIGNAL_MSI) > 0); #endif @@ -2187,6 +2196,11 @@ int kvm_has_debugregs(void) return kvm_state->debugregs; } +uint32_t kvm_nested_state_length(void) +{ + return kvm_state->nested_state_len; +} + int kvm_has_many_ioeventfds(void) { if (!kvm_enabled()) { diff --git a/include/sysemu/kvm.h b/include/sysemu/kvm.h index 0b64b8e06786..17dadaf57f40 100644 --- a/include/sysemu/kvm.h +++ b/include/sysemu/kvm.h @@ -210,6 +210,7 @@ bool kvm_has_sync_mmu(void); int kvm_has_vcpu_events(void); int kvm_has_robust_singlestep(void); int kvm_has_debugregs(void); +uint32_t kvm_nested_state_length(void); int kvm_has_pit_state2(void); int kvm_has_many_ioeventfds(void); int kvm_has_gsi_routing(void); diff --git a/target/i386/cpu.h b/target/i386/cpu.h index 6e4c2b02f947..3b97b5b280f0 100644 --- a/target/i386/cpu.h +++ b/target/i386/cpu.h @@ -1330,6 +1330,8 @@ typedef struct CPUX86State { #if defined(CONFIG_KVM) || defined(CONFIG_HVF) void *xsave_buf; #endif + struct kvm_nested_state *nested_state; + uint32_t nested_state_len; /* needed for migration */ #if defined(CONFIG_HVF) HVFX86EmulatorState *hvf_emul; #endif diff --git a/target/i386/kvm.c b/target/i386/kvm.c index c1cd8c461fe4..8160ab55909a 100644 --- a/target/i386/kvm.c +++ b/target/i386/kvm.c @@ -1191,6 +1191,22 @@ int kvm_arch_init_vcpu(CPUState *cs) if (has_xsave) { env->xsave_buf = qemu_memalign(4096, sizeof(struct kvm_xsave)); } + + env->nested_state_len = kvm_nested_state_length(); + if (env->nested_state_len > 0) { + uint32_t min_nested_state_len = + offsetof(struct kvm_nested_state, size) + sizeof(uint32_t); + + /* + * Verify nested state length cover at least the size + * field of struct kvm_nested_state + */ + assert(env->nested_state_len >= min_nested_state_len); + + env->nested_state = g_malloc0(env->nested_state_len); + env->nested_state->size = env->nested_state_len; + } + cpu->kvm_msr_buf = g_malloc0(MSR_BUF_SIZE); if (!(env->features[FEAT_8000_0001_EDX] & CPUID_EXT2_RDTSCP)) { @@ -2867,6 +2883,39 @@ static int kvm_get_debugregs(X86CPU *cpu) return 0; } +static int kvm_put_nested_state(X86CPU *cpu) +{ + CPUX86State *env = &cpu->env; + + if (kvm_nested_state_length() == 0) { + return 0; + } + + assert(env->nested_state->size <= env->nested_state_len); + return kvm_vcpu_ioctl(CPU(cpu), KVM_SET_NESTED_STATE, env->nested_state); +} + +static int kvm_get_nested_state(X86CPU *cpu) +{ + CPUX86State *env = &cpu->env; + + if (kvm_nested_state_length() == 0) { + return 0; + } + + + /* + * It is possible that migration restored a smaller size into + * nested_state->size than what our kernel support. + * We preserve migration origin nested_state->size for + * call to KVM_SET_NESTED_STATE but wish that our next call + * to KVM_GET_NESTED_STATE will use max size our kernel support. + */ + env->nested_state->size = env->nested_state_len; + + return kvm_vcpu_ioctl(CPU(cpu), KVM_GET_NESTED_STATE, env->nested_state); +} + int kvm_arch_put_registers(CPUState *cpu, int level) { X86CPU *x86_cpu = X86_CPU(cpu); @@ -2874,6 +2923,11 @@ int kvm_arch_put_registers(CPUState *cpu, int level) assert(cpu_is_stopped(cpu) || qemu_cpu_is_self(cpu)); + ret = kvm_put_nested_state(x86_cpu); + if (ret < 0) { + return ret; + } + if (level >= KVM_PUT_RESET_STATE) { ret = kvm_put_msr_feature_control(x86_cpu); if (ret < 0) { @@ -2989,6 +3043,10 @@ int kvm_arch_get_registers(CPUState *cs) if (ret < 0) { goto out; } + ret = kvm_get_nested_state(cpu); + if (ret < 0) { + goto out; + } ret = 0; out: cpu_sync_bndcs_hflags(&cpu->env); diff --git a/target/i386/machine.c b/target/i386/machine.c index 084c2c73a8f7..394953de8d1c 100644 --- a/target/i386/machine.c +++ b/target/i386/machine.c @@ -842,6 +842,43 @@ static const VMStateDescription vmstate_tsc_khz = { } }; +static int nested_state_post_load(void *opaque, int version_id) +{ + X86CPU *cpu = opaque; + CPUX86State *env = &cpu->env; + + /* + * Verify that the size specified in given struct is set + * to no more than the size that our kernel support + */ + if (env->nested_state->size > env->nested_state_len) { + return -EINVAL; + } + + return 0; +} + +static bool nested_state_needed(void *opaque) +{ + X86CPU *cpu = opaque; + CPUX86State *env = &cpu->env; + return (env->nested_state_len > 0); +} + +static const VMStateDescription vmstate_nested_state = { + .name = "cpu/nested_state", + .version_id = 1, + .minimum_version_id = 1, + .post_load = nested_state_post_load, + .needed = nested_state_needed, + .fields = (VMStateField[]) { + VMSTATE_VBUFFER_UINT32(env.nested_state, X86CPU, + 0, NULL, + env.nested_state_len), + VMSTATE_END_OF_LIST() + } +}; + static bool mcg_ext_ctl_needed(void *opaque) { X86CPU *cpu = opaque; @@ -1080,6 +1117,7 @@ VMStateDescription vmstate_x86_cpu = { &vmstate_msr_intel_pt, &vmstate_msr_virt_ssbd, &vmstate_svm_npt, + &vmstate_nested_state, NULL } };