[v8,026/103] KVM: TDX: allocate/free TDX vcpu structure

Message ID	2347416df71bf525bd716a036e92e21a2e2a7520.1659854790.git.isaku.yamahata@intel.com (mailing list archive)
State	New, archived
Headers	show Return-Path: <kvm-owner@kernel.org> From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini <pbonzini@redhat.com>, erdemaktas@google.com, Sean Christopherson <seanjc@google.com>, Sagi Shahar <sagis@google.com> Subject: [PATCH v8 026/103] KVM: TDX: allocate/free TDX vcpu structure Date: Sun, 7 Aug 2022 15:01:11 -0700 Message-Id: <2347416df71bf525bd716a036e92e21a2e2a7520.1659854790.git.isaku.yamahata@intel.com> In-Reply-To: <cover.1659854790.git.isaku.yamahata@intel.com> References: <cover.1659854790.git.isaku.yamahata@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Precedence: bulk
Series	KVM TDX basic feature support \| expand [v8,000/103] KVM TDX basic feature support [v8,001/103] KVM: x86: Move check_processor_compatibility from init ops to runtime ops [v8,002/103] Partially revert "KVM: Pass kvm_init()'s opaque param to additional arch funcs" [v8,003/103] KVM: Refactor CPU compatibility check on module initialization [v8,004/103] KVM: VMX: Move out vmx_x86_ops to 'main.c' to wrap VMX and TDX [v8,005/103] KVM: x86: Refactor KVM VMX module init/exit functions [v8,006/103] KVM: Enable hardware before doing arch VM initialization [v8,007/103] KVM: TDX: Add placeholders for TDX VM/vcpu structure [v8,008/103] x86/virt/tdx: Add a helper function to return system wide info about TDX module [v8,009/103] KVM: TDX: Initialize the TDX module when loading the KVM intel kernel module [v8,010/103] KVM: x86: Introduce vm_type to differentiate default VMs from confidential VMs [v8,011/103] KVM: TDX: Make TDX VM type supported [v8,012/103,MARKER] The start of TDX KVM patch series: TDX architectural definitions [v8,013/103] KVM: TDX: Define TDX architectural definitions [v8,014/103] KVM: TDX: Add TDX "architectural" error codes [v8,015/103] KVM: TDX: Add C wrapper functions for SEAMCALLs to the TDX module [v8,016/103] KVM: TDX: Add helper functions to print TDX SEAMCALL error [v8,017/103,MARKER] The start of TDX KVM patch series: TD VM creation/destruction [v8,018/103] KVM: TDX: Stub in tdx.h with structs, accessors, and VMCS helpers [v8,019/103] x86/cpu: Add helper functions to allocate/free TDX private host key id [v8,020/103] KVM: TDX: create/destroy VM structure [v8,021/103] KVM: TDX: x86: Add ioctl to get TDX systemwide parameters [v8,022/103] KVM: TDX: Add place holder for TDX VM specific mem_enc_op ioctl [v8,023/103] KVM: TDX: initialize VM with TDX specific parameters [v8,024/103] KVM: TDX: Make pmu_intel.c ignore guest TD case [v8,025/103,MARKER] The start of TDX KVM patch series: TD vcpu creation/destruction [v8,026/103] KVM: TDX: allocate/free TDX vcpu structure [v8,027/103] KVM: TDX: Do TDX specific vcpu initialization [v8,028/103,MARKER] The start of TDX KVM patch series: KVM MMU GPA shared bits [v8,029/103] KVM: x86/mmu: introduce config for PRIVATE KVM MMU [v8,030/103] KVM: x86/mmu: Add address conversion functions for TDX shared bit of GPA [v8,031/103,MARKER] The start of TDX KVM patch series: KVM TDP refactoring for TDX [v8,032/103] KVM: x86/mmu: Allow non-zero value for non-present SPTE [v8,033/103] KVM: x86/mmu: Track shadow MMIO value/mask on a per-VM basis [v8,034/103] KVM: x86/mmu: Disallow fast page fault on private GPA [v8,035/103] KVM: x86/mmu: Allow per-VM override of the TDP max page level [v8,036/103] KVM: VMX: Introduce test mode related to EPT violation VE [v8,037/103,MARKER] The start of TDX KVM patch series: KVM TDP MMU hooks [v8,038/103] KVM: x86/tdp_mmu: refactor kvm_tdp_mmu_map() [v8,039/103] KVM: x86/tdp_mmu: Init role member of struct kvm_mmu_page at allocation [v8,040/103] KVM: x86/mmu: Require TDP MMU for TDX [v8,041/103] KVM: x86/mmu: Add a new is_private member for union kvm_mmu_page_role [v8,042/103] KVM: x86/mmu: Add a private pointer to struct kvm_mmu_page [v8,043/103] KVM: x86/tdp_mmu: Don't zap private pages for unsupported cases [v8,044/103] KVM: x86/tdp_mmu: Support TDX private mapping for TDP MMU [v8,045/103,MARKER] The start of TDX KVM patch series: TDX EPT violation [v8,046/103] KVM: x86/mmu: Disallow dirty logging for x86 TDX [v8,047/103] KVM: x86/tdp_mmu: Ignore unsupported mmu operation on private GFNs [v8,048/103] KVM: VMX: Split out guts of EPT violation to common/exposed function [v8,049/103] KVM: VMX: Move setting of EPT MMU masks to common VT-x code [v8,050/103] KVM: TDX: Add load_mmu_pgd method for TDX [v8,051/103] KVM: TDX: don't request KVM_REQ_APIC_PAGE_RELOAD [v8,052/103] KVM: x86/VMX: introduce vmx tlb_remote_flush and tlb_remote_flush_with_range [v8,053/103] KVM: TDX: TDP MMU TDX support [v8,054/103,MARKER] The start of TDX KVM patch series: KVM TDP MMU MapGPA [v8,055/103] KVM: Add functions to track whether GFN is private or shared [v8,056/103] KVM: x86/mmu: Let vcpu re-try when faulting page type conflict [v8,057/103] KVM: x86/mmu: Introduce kvm_mmu_map_tdp_page() for use by TDX [v8,058/103] KVM: x86/tdp_mmu: implement MapGPA hypercall for TDX [v8,059/103,MARKER] The start of TDX KVM patch series: TD finalization [v8,060/103] KVM: TDX: Create initial guest memory [v8,061/103] KVM: TDX: Finalize VM initialization [v8,062/103,MARKER] The start of TDX KVM patch series: TD vcpu enter/exit [v8,063/103] KVM: TDX: Add helper assembly function to TDX vcpu [v8,064/103] KVM: TDX: Implement TDX vcpu enter/exit path [v8,065/103] KVM: TDX: vcpu_run: save/restore host state(host kernel gs) [v8,066/103] KVM: TDX: restore host xsave state when exit from the guest TD [v8,067/103] KVM: x86: Allow to update cached values in kvm_user_return_msrs w/o wrmsr [v8,068/103] KVM: TDX: restore user ret MSRs [v8,069/103,MARKER] The start of TDX KVM patch series: TD vcpu exits/interrupts/hypercalls [v8,070/103] KVM: TDX: complete interrupts after tdexit [v8,071/103] KVM: TDX: restore debug store when TD exit [v8,072/103] KVM: TDX: handle vcpu migration over logical processor [v8,073/103] KVM: x86: Add a switch_db_regs flag to handle TDX's auto-switched behavior [v8,074/103] KVM: TDX: Add support for find pending IRQ in a protected local APIC [v8,075/103] KVM: x86: Assume timer IRQ was injected if APIC state is proteced [v8,076/103] KVM: TDX: remove use of struct vcpu_vmx from posted_interrupt.c [v8,077/103] KVM: TDX: Implement interrupt injection [v8,078/103] KVM: TDX: Implements vcpu request_immediate_exit [v8,079/103] KVM: TDX: Implement methods to inject NMI [v8,080/103] KVM: VMX: Modify NMI and INTR handlers to take intr_info as function argument [v8,081/103] KVM: VMX: Move NMI/exception handler to common helper [v8,082/103] KVM: x86: Split core of hypercall emulation to helper function [v8,083/103] KVM: TDX: Add a place holder to handle TDX VM exit [v8,084/103] KVM: TDX: Retry seamcall when TDX_OPERAND_BUSY with operand SEPT [v8,085/103] KVM: TDX: handle EXIT_REASON_OTHER_SMI [v8,086/103] KVM: TDX: handle ept violation/misconfig exit [v8,087/103] KVM: TDX: handle EXCEPTION_NMI and EXTERNAL_INTERRUPT [v8,088/103] KVM: TDX: Add a place holder for handler of TDX hypercalls (TDG.VP.VMCALL) [v8,089/103] KVM: TDX: handle KVM hypercall with TDG.VP.VMCALL [v8,090/103] KVM: TDX: Handle TDX PV CPUID hypercall [v8,091/103] KVM: TDX: Handle TDX PV HLT hypercall [v8,092/103] KVM: TDX: Handle TDX PV port io hypercall [v8,093/103] KVM: TDX: Handle TDX PV MMIO hypercall [v8,094/103] KVM: TDX: Implement callbacks for MSR operations for TDX [v8,095/103] KVM: TDX: Handle TDX PV rdmsr/wrmsr hypercall [v8,096/103] KVM: TDX: Handle TDX PV report fatal error hypercall [v8,097/103] KVM: TDX: Handle TDX PV map_gpa hypercall [v8,098/103] KVM: TDX: Handle TDG.VP.VMCALL<GetTdVmCallInfo> hypercall [v8,099/103] KVM: TDX: Silently discard SMI request [v8,100/103] KVM: TDX: Silently ignore INIT/SIPI [v8,101/103] KVM: TDX: Add methods to ignore accesses to CPU state [v8,102/103] Documentation/virt/kvm: Document on Trust Domain Extensions(TDX) [v8,103/103] KVM: x86: design documentation on TDX support of x86 KVM TDP MMU

Message ID

2347416df71bf525bd716a036e92e21a2e2a7520.1659854790.git.isaku.yamahata@intel.com (mailing list archive)

State

New, archived

Headers

From: isaku.yamahata@intel.com
To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org
Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com,
        Paolo Bonzini <pbonzini@redhat.com>, erdemaktas@google.com,
        Sean Christopherson <seanjc@google.com>,
        Sagi Shahar <sagis@google.com>
Subject: [PATCH v8 026/103] KVM: TDX: allocate/free TDX vcpu structure
Date: Sun,  7 Aug 2022 15:01:11 -0700
Message-Id: 
 <2347416df71bf525bd716a036e92e21a2e2a7520.1659854790.git.isaku.yamahata@intel.com>
In-Reply-To: <cover.1659854790.git.isaku.yamahata@intel.com>
References: <cover.1659854790.git.isaku.yamahata@intel.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Precedence: bulk

Series

KVM TDX basic feature support | expand

Commit Message

Isaku Yamahata Aug. 7, 2022, 10:01 p.m. UTC

From: Isaku Yamahata <isaku.yamahata@intel.com>

The next step of TDX guest creation is to create vcpu.  Allocate TDX vcpu
structures, initialize it.  Allocate pages of TDX vcpu for the TDX module.

In the case of the conventional case, cpuid is empty at the initialization.
and cpuid is configured after the vcpu initialization.  Because TDX
supports only X2APIC mode, cpuid is forcibly initialized to support X2APIC
on the vcpu initialization.

Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
---
 arch/x86/kvm/vmx/main.c    |  40 +++++++++--
 arch/x86/kvm/vmx/tdx.c     | 135 +++++++++++++++++++++++++++++++++++++
 arch/x86/kvm/vmx/x86_ops.h |   8 +++
 3 files changed, 179 insertions(+), 4 deletions(-)

Comments

Binbin Wu Aug. 30, 2022, 3:20 a.m. UTC | #1

On 2022/8/8 6:01, isaku.yamahata@intel.com wrote:
> From: Isaku Yamahata <isaku.yamahata@intel.com>
>
> The next step of TDX guest creation is to create vcpu.  Allocate TDX vcpu
> structures, initialize it.  Allocate pages of TDX vcpu for the TDX module.
>
> In the case of the conventional case, cpuid is empty at the initialization.
> and cpuid is configured after the vcpu initialization.  Because TDX
> supports only X2APIC mode, cpuid is forcibly initialized to support X2APIC
> on the vcpu initialization.
>
> Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
> ---
>   arch/x86/kvm/vmx/main.c    |  40 +++++++++--
>   arch/x86/kvm/vmx/tdx.c     | 135 +++++++++++++++++++++++++++++++++++++
>   arch/x86/kvm/vmx/x86_ops.h |   8 +++
>   3 files changed, 179 insertions(+), 4 deletions(-)
>
> diff --git a/arch/x86/kvm/vmx/main.c b/arch/x86/kvm/vmx/main.c
> index 067f5de56c53..4f4ed4ad65a7 100644
> --- a/arch/x86/kvm/vmx/main.c
> +++ b/arch/x86/kvm/vmx/main.c
> @@ -73,6 +73,38 @@ static void vt_vm_free(struct kvm *kvm)
>   		return tdx_vm_free(kvm);
>   }
>   
> +static int vt_vcpu_precreate(struct kvm *kvm)
> +{
> +	if (is_td(kvm))
> +		return 0;
> +
> +	return vmx_vcpu_precreate(kvm);
> +}
> +
> +static int vt_vcpu_create(struct kvm_vcpu *vcpu)
> +{
> +	if (is_td_vcpu(vcpu))
> +		return tdx_vcpu_create(vcpu);
> +
> +	return vmx_vcpu_create(vcpu);
> +}
> +
> +static void vt_vcpu_free(struct kvm_vcpu *vcpu)
> +{
> +	if (is_td_vcpu(vcpu))
> +		return tdx_vcpu_free(vcpu);
> +
> +	return vmx_vcpu_free(vcpu);
> +}
> +
> +static void vt_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event)
> +{
> +	if (is_td_vcpu(vcpu))
> +		return tdx_vcpu_reset(vcpu, init_event);
> +
> +	return vmx_vcpu_reset(vcpu, init_event);
> +}
> +
>   static int vt_mem_enc_ioctl(struct kvm *kvm, void __user *argp)
>   {
>   	if (!is_td(kvm))
> @@ -98,10 +130,10 @@ struct kvm_x86_ops vt_x86_ops __initdata = {
>   	.vm_destroy = vt_vm_destroy,
>   	.vm_free = vt_vm_free,
>   
> -	.vcpu_precreate = vmx_vcpu_precreate,
> -	.vcpu_create = vmx_vcpu_create,
> -	.vcpu_free = vmx_vcpu_free,
> -	.vcpu_reset = vmx_vcpu_reset,
> +	.vcpu_precreate = vt_vcpu_precreate,
> +	.vcpu_create = vt_vcpu_create,
> +	.vcpu_free = vt_vcpu_free,
> +	.vcpu_reset = vt_vcpu_reset,
>   
>   	.prepare_switch_to_guest = vmx_prepare_switch_to_guest,
>   	.vcpu_load = vmx_vcpu_load,
> diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c
> index dcd2f460275e..ee682a65b233 100644
> --- a/arch/x86/kvm/vmx/tdx.c
> +++ b/arch/x86/kvm/vmx/tdx.c
> @@ -6,6 +6,7 @@
>   #include "capabilities.h"
>   #include "x86_ops.h"
>   #include "tdx.h"
> +#include "x86.h"
>   
>   #undef pr_fmt
>   #define pr_fmt(fmt) "tdx: " fmt
> @@ -47,6 +48,11 @@ static __always_inline hpa_t set_hkid_to_hpa(hpa_t pa, u16 hkid)
>   	return pa | ((hpa_t)hkid << boot_cpu_data.x86_phys_bits);
>   }
>   
> +static inline bool is_td_vcpu_created(struct vcpu_tdx *tdx)
> +{
> +	return tdx->tdvpr.added;
> +}
> +
>   static inline bool is_td_created(struct kvm_tdx *kvm_tdx)
>   {
>   	return kvm_tdx->tdr.added;
> @@ -378,6 +384,135 @@ int tdx_vm_init(struct kvm *kvm)
>   	return ret;
>   }
>   
> +int tdx_vcpu_create(struct kvm_vcpu *vcpu)
> +{
> +	struct vcpu_tdx *tdx = to_tdx(vcpu);
> +	int ret, i;
> +
> +	/* TDX only supports x2APIC, which requires an in-kernel local APIC. */
> +	if (!vcpu->arch.apic)
> +		return -EINVAL;
> +
> +	fpstate_set_confidential(&vcpu->arch.guest_fpu);
> +
> +	ret = tdx_alloc_td_page(&tdx->tdvpr);
> +	if (ret)
> +		return ret;
> +
> +	tdx->tdvpx = kcalloc(tdx_caps.tdvpx_nr_pages, sizeof(*tdx->tdvpx),
> +			GFP_KERNEL_ACCOUNT);
> +	if (!tdx->tdvpx) {
> +		ret = -ENOMEM;
> +		goto free_tdvpr;
> +	}
> +	for (i = 0; i < tdx_caps.tdvpx_nr_pages; i++) {
> +		ret = tdx_alloc_td_page(&tdx->tdvpx[i]);
> +		if (ret)
> +			goto free_tdvpx;
> +	}
> +
> +	vcpu->arch.efer = EFER_SCE | EFER_LME | EFER_LMA | EFER_NX;
> +
> +	vcpu->arch.cr0_guest_owned_bits = -1ul;
> +	vcpu->arch.cr4_guest_owned_bits = -1ul;
> +
> +	vcpu->arch.tsc_offset = to_kvm_tdx(vcpu->kvm)->tsc_offset;
> +	vcpu->arch.l1_tsc_offset = vcpu->arch.tsc_offset;
> +	vcpu->arch.guest_state_protected =
> +		!(to_kvm_tdx(vcpu->kvm)->attributes & TDX_TD_ATTRIBUTE_DEBUG);
> +
> +	return 0;
> +
> +free_tdvpx:
> +	/* @i points at the TDVPX page that failed allocation. */
> +	for (--i; i >= 0; i--)
> +		free_page(tdx->tdvpx[i].va);
> +	kfree(tdx->tdvpx);
> +free_tdvpr:
> +	free_page(tdx->tdvpr.va);
> +
> +	return ret;
> +}
> +
> +void tdx_vcpu_free(struct kvm_vcpu *vcpu)
> +{
> +	struct vcpu_tdx *tdx = to_tdx(vcpu);
> +	int i;
> +
> +	/* Can't reclaim or free pages if teardown failed. */
> +	if (is_hkid_assigned(to_kvm_tdx(vcpu->kvm)))
> +		return;
> +
> +	for (i = 0; i < tdx_caps.tdvpx_nr_pages; i++)
> +		tdx_reclaim_td_page(&tdx->tdvpx[i]);

What if some of the pages failed to be reclaimed?


> +	kfree(tdx->tdvpx);
> +	tdx_reclaim_td_page(&tdx->tdvpr);
> +}
> +
> +void tdx_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event)
> +{
> +	struct kvm_tdx *kvm_tdx = to_kvm_tdx(vcpu->kvm);
> +	struct vcpu_tdx *tdx = to_tdx(vcpu);
> +	struct msr_data apic_base_msr;
> +	u64 err;
> +	int i;
> +
> +	/* TDX doesn't support INIT event. */
> +	if (WARN_ON(init_event))
> +		goto td_bugged;
> +	if (WARN_ON(is_td_vcpu_created(tdx)))
> +		goto td_bugged;
> +
> +	err = tdh_vp_create(kvm_tdx->tdr.pa, tdx->tdvpr.pa);
> +	if (WARN_ON_ONCE(err)) {
> +		pr_tdx_error(TDH_VP_CREATE, err, NULL);
> +		goto td_bugged;
> +	}
> +	tdx_mark_td_page_added(&tdx->tdvpr);
> +
> +	for (i = 0; i < tdx_caps.tdvpx_nr_pages; i++) {
> +		err = tdh_vp_addcx(tdx->tdvpr.pa, tdx->tdvpx[i].pa);
> +		if (WARN_ON_ONCE(err)) {
> +			pr_tdx_error(TDH_VP_ADDCX, err, NULL);
> +			goto td_bugged;
> +		}
> +		tdx_mark_td_page_added(&tdx->tdvpx[i]);
> +	}
> +
> +	if (!vcpu->arch.cpuid_entries) {
> +		/*
> +		 * On cpu creation, cpuid entry is blank.  Forcibly enable
> +		 * X2APIC feature to allow X2APIC.
> +		 */
> +		struct kvm_cpuid_entry2 *e;
> +
> +		e = kvmalloc_array(1, sizeof(*e), GFP_KERNEL_ACCOUNT);
> +		*e  = (struct kvm_cpuid_entry2) {
> +			.function = 1,	/* Features for X2APIC */
> +			.index = 0,
> +			.eax = 0,
> +			.ebx = 0,
> +			.ecx = 1ULL << 21,	/* X2APIC */
> +			.edx = 0,
> +		};

Just out of curiosity, where will this info be used afterwards?



> +		vcpu->arch.cpuid_entries = e;
> +		vcpu->arch.cpuid_nent = 1;
> +	}
> +	apic_base_msr.data = APIC_DEFAULT_PHYS_BASE | LAPIC_MODE_X2APIC;
> +	if (kvm_vcpu_is_reset_bsp(vcpu))
> +		apic_base_msr.data |= MSR_IA32_APICBASE_BSP;
> +	apic_base_msr.host_initiated = true;
> +	if (WARN_ON(kvm_set_apic_base(vcpu, &apic_base_msr)))
> +		goto td_bugged;
> +
> +	vcpu->arch.mp_state = KVM_MP_STATE_RUNNABLE;
> +
> +	return;
> +
> +td_bugged:
> +	vcpu->kvm->vm_bugged = true;
> +}
> +
>   int tdx_dev_ioctl(void __user *argp)
>   {
>   	struct kvm_tdx_capabilities __user *user_caps;
> diff --git a/arch/x86/kvm/vmx/x86_ops.h b/arch/x86/kvm/vmx/x86_ops.h
> index f0fe40c7ac34..b98bbcd9ef42 100644
> --- a/arch/x86/kvm/vmx/x86_ops.h
> +++ b/arch/x86/kvm/vmx/x86_ops.h
> @@ -138,6 +138,10 @@ int tdx_vm_init(struct kvm *kvm);
>   void tdx_mmu_release_hkid(struct kvm *kvm);
>   void tdx_vm_free(struct kvm *kvm);
>   
> +int tdx_vcpu_create(struct kvm_vcpu *vcpu);
> +void tdx_vcpu_free(struct kvm_vcpu *vcpu);
> +void tdx_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event);
> +
>   int tdx_vm_ioctl(struct kvm *kvm, void __user *argp);
>   #else
>   static inline int tdx_hardware_setup(struct kvm_x86_ops *x86_ops) { return 0; }
> @@ -150,6 +154,10 @@ static inline void tdx_mmu_release_hkid(struct kvm *kvm) {}
>   static inline void tdx_flush_shadow_all_private(struct kvm *kvm) {}
>   static inline void tdx_vm_free(struct kvm *kvm) {}
>   
> +static inline int tdx_vcpu_create(struct kvm_vcpu *vcpu) { return -EOPNOTSUPP; }
> +static inline void tdx_vcpu_free(struct kvm_vcpu *vcpu) {}
> +static inline void tdx_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event) {}
> +
>   static inline int tdx_vm_ioctl(struct kvm *kvm, void __user *argp) { return -EOPNOTSUPP; }
>   #endif
>

diff --git a/arch/x86/kvm/vmx/main.c b/arch/x86/kvm/vmx/main.c
index 067f5de56c53..4f4ed4ad65a7 100644
--- a/arch/x86/kvm/vmx/main.c
+++ b/arch/x86/kvm/vmx/main.c
@@ -73,6 +73,38 @@  static void vt_vm_free(struct kvm *kvm)
 		return tdx_vm_free(kvm);
 }
 
+static int vt_vcpu_precreate(struct kvm *kvm)
+{
+	if (is_td(kvm))
+		return 0;
+
+	return vmx_vcpu_precreate(kvm);
+}
+
+static int vt_vcpu_create(struct kvm_vcpu *vcpu)
+{
+	if (is_td_vcpu(vcpu))
+		return tdx_vcpu_create(vcpu);
+
+	return vmx_vcpu_create(vcpu);
+}
+
+static void vt_vcpu_free(struct kvm_vcpu *vcpu)
+{
+	if (is_td_vcpu(vcpu))
+		return tdx_vcpu_free(vcpu);
+
+	return vmx_vcpu_free(vcpu);
+}
+
+static void vt_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event)
+{
+	if (is_td_vcpu(vcpu))
+		return tdx_vcpu_reset(vcpu, init_event);
+
+	return vmx_vcpu_reset(vcpu, init_event);
+}
+
 static int vt_mem_enc_ioctl(struct kvm *kvm, void __user *argp)
 {
 	if (!is_td(kvm))
@@ -98,10 +130,10 @@  struct kvm_x86_ops vt_x86_ops __initdata = {
 	.vm_destroy = vt_vm_destroy,
 	.vm_free = vt_vm_free,
 
-	.vcpu_precreate = vmx_vcpu_precreate,
-	.vcpu_create = vmx_vcpu_create,
-	.vcpu_free = vmx_vcpu_free,
-	.vcpu_reset = vmx_vcpu_reset,
+	.vcpu_precreate = vt_vcpu_precreate,
+	.vcpu_create = vt_vcpu_create,
+	.vcpu_free = vt_vcpu_free,
+	.vcpu_reset = vt_vcpu_reset,
 
 	.prepare_switch_to_guest = vmx_prepare_switch_to_guest,
 	.vcpu_load = vmx_vcpu_load,
diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c
index dcd2f460275e..ee682a65b233 100644
--- a/arch/x86/kvm/vmx/tdx.c
+++ b/arch/x86/kvm/vmx/tdx.c
@@ -6,6 +6,7 @@ 
 #include "capabilities.h"
 #include "x86_ops.h"
 #include "tdx.h"
+#include "x86.h"
 
 #undef pr_fmt
 #define pr_fmt(fmt) "tdx: " fmt
@@ -47,6 +48,11 @@  static __always_inline hpa_t set_hkid_to_hpa(hpa_t pa, u16 hkid)
 	return pa | ((hpa_t)hkid << boot_cpu_data.x86_phys_bits);
 }
 
+static inline bool is_td_vcpu_created(struct vcpu_tdx *tdx)
+{
+	return tdx->tdvpr.added;
+}
+
 static inline bool is_td_created(struct kvm_tdx *kvm_tdx)
 {
 	return kvm_tdx->tdr.added;
@@ -378,6 +384,135 @@  int tdx_vm_init(struct kvm *kvm)
 	return ret;
 }
 
+int tdx_vcpu_create(struct kvm_vcpu *vcpu)
+{
+	struct vcpu_tdx *tdx = to_tdx(vcpu);
+	int ret, i;
+
+	/* TDX only supports x2APIC, which requires an in-kernel local APIC. */
+	if (!vcpu->arch.apic)
+		return -EINVAL;
+
+	fpstate_set_confidential(&vcpu->arch.guest_fpu);
+
+	ret = tdx_alloc_td_page(&tdx->tdvpr);
+	if (ret)
+		return ret;
+
+	tdx->tdvpx = kcalloc(tdx_caps.tdvpx_nr_pages, sizeof(*tdx->tdvpx),
+			GFP_KERNEL_ACCOUNT);
+	if (!tdx->tdvpx) {
+		ret = -ENOMEM;
+		goto free_tdvpr;
+	}
+	for (i = 0; i < tdx_caps.tdvpx_nr_pages; i++) {
+		ret = tdx_alloc_td_page(&tdx->tdvpx[i]);
+		if (ret)
+			goto free_tdvpx;
+	}
+
+	vcpu->arch.efer = EFER_SCE | EFER_LME | EFER_LMA | EFER_NX;
+
+	vcpu->arch.cr0_guest_owned_bits = -1ul;
+	vcpu->arch.cr4_guest_owned_bits = -1ul;
+
+	vcpu->arch.tsc_offset = to_kvm_tdx(vcpu->kvm)->tsc_offset;
+	vcpu->arch.l1_tsc_offset = vcpu->arch.tsc_offset;
+	vcpu->arch.guest_state_protected =
+		!(to_kvm_tdx(vcpu->kvm)->attributes & TDX_TD_ATTRIBUTE_DEBUG);
+
+	return 0;
+
+free_tdvpx:
+	/* @i points at the TDVPX page that failed allocation. */
+	for (--i; i >= 0; i--)
+		free_page(tdx->tdvpx[i].va);
+	kfree(tdx->tdvpx);
+free_tdvpr:
+	free_page(tdx->tdvpr.va);
+
+	return ret;
+}
+
+void tdx_vcpu_free(struct kvm_vcpu *vcpu)
+{
+	struct vcpu_tdx *tdx = to_tdx(vcpu);
+	int i;
+
+	/* Can't reclaim or free pages if teardown failed. */
+	if (is_hkid_assigned(to_kvm_tdx(vcpu->kvm)))
+		return;
+
+	for (i = 0; i < tdx_caps.tdvpx_nr_pages; i++)
+		tdx_reclaim_td_page(&tdx->tdvpx[i]);
+	kfree(tdx->tdvpx);
+	tdx_reclaim_td_page(&tdx->tdvpr);
+}
+
+void tdx_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event)
+{
+	struct kvm_tdx *kvm_tdx = to_kvm_tdx(vcpu->kvm);
+	struct vcpu_tdx *tdx = to_tdx(vcpu);
+	struct msr_data apic_base_msr;
+	u64 err;
+	int i;
+
+	/* TDX doesn't support INIT event. */
+	if (WARN_ON(init_event))
+		goto td_bugged;
+	if (WARN_ON(is_td_vcpu_created(tdx)))
+		goto td_bugged;
+
+	err = tdh_vp_create(kvm_tdx->tdr.pa, tdx->tdvpr.pa);
+	if (WARN_ON_ONCE(err)) {
+		pr_tdx_error(TDH_VP_CREATE, err, NULL);
+		goto td_bugged;
+	}
+	tdx_mark_td_page_added(&tdx->tdvpr);
+
+	for (i = 0; i < tdx_caps.tdvpx_nr_pages; i++) {
+		err = tdh_vp_addcx(tdx->tdvpr.pa, tdx->tdvpx[i].pa);
+		if (WARN_ON_ONCE(err)) {
+			pr_tdx_error(TDH_VP_ADDCX, err, NULL);
+			goto td_bugged;
+		}
+		tdx_mark_td_page_added(&tdx->tdvpx[i]);
+	}
+
+	if (!vcpu->arch.cpuid_entries) {
+		/*
+		 * On cpu creation, cpuid entry is blank.  Forcibly enable
+		 * X2APIC feature to allow X2APIC.
+		 */
+		struct kvm_cpuid_entry2 *e;
+
+		e = kvmalloc_array(1, sizeof(*e), GFP_KERNEL_ACCOUNT);
+		*e  = (struct kvm_cpuid_entry2) {
+			.function = 1,	/* Features for X2APIC */
+			.index = 0,
+			.eax = 0,
+			.ebx = 0,
+			.ecx = 1ULL << 21,	/* X2APIC */
+			.edx = 0,
+		};
+		vcpu->arch.cpuid_entries = e;
+		vcpu->arch.cpuid_nent = 1;
+	}
+	apic_base_msr.data = APIC_DEFAULT_PHYS_BASE | LAPIC_MODE_X2APIC;
+	if (kvm_vcpu_is_reset_bsp(vcpu))
+		apic_base_msr.data |= MSR_IA32_APICBASE_BSP;
+	apic_base_msr.host_initiated = true;
+	if (WARN_ON(kvm_set_apic_base(vcpu, &apic_base_msr)))
+		goto td_bugged;
+
+	vcpu->arch.mp_state = KVM_MP_STATE_RUNNABLE;
+
+	return;
+
+td_bugged:
+	vcpu->kvm->vm_bugged = true;
+}
+
 int tdx_dev_ioctl(void __user *argp)
 {
 	struct kvm_tdx_capabilities __user *user_caps;
diff --git a/arch/x86/kvm/vmx/x86_ops.h b/arch/x86/kvm/vmx/x86_ops.h
index f0fe40c7ac34..b98bbcd9ef42 100644
--- a/arch/x86/kvm/vmx/x86_ops.h
+++ b/arch/x86/kvm/vmx/x86_ops.h
@@ -138,6 +138,10 @@  int tdx_vm_init(struct kvm *kvm);
 void tdx_mmu_release_hkid(struct kvm *kvm);
 void tdx_vm_free(struct kvm *kvm);
 
+int tdx_vcpu_create(struct kvm_vcpu *vcpu);
+void tdx_vcpu_free(struct kvm_vcpu *vcpu);
+void tdx_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event);
+
 int tdx_vm_ioctl(struct kvm *kvm, void __user *argp);
 #else
 static inline int tdx_hardware_setup(struct kvm_x86_ops *x86_ops) { return 0; }
@@ -150,6 +154,10 @@  static inline void tdx_mmu_release_hkid(struct kvm *kvm) {}
 static inline void tdx_flush_shadow_all_private(struct kvm *kvm) {}
 static inline void tdx_vm_free(struct kvm *kvm) {}
 
+static inline int tdx_vcpu_create(struct kvm_vcpu *vcpu) { return -EOPNOTSUPP; }
+static inline void tdx_vcpu_free(struct kvm_vcpu *vcpu) {}
+static inline void tdx_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event) {}
+
 static inline int tdx_vm_ioctl(struct kvm *kvm, void __user *argp) { return -EOPNOTSUPP; }
 #endif

[v8,026/103] KVM: TDX: allocate/free TDX vcpu structure

Commit Message

Comments

Patch