From patchwork Mon Jun 26 14:12:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Huang, Kai" X-Patchwork-Id: 13292976 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BB181EB64DA for ; Mon, 26 Jun 2023 14:14:24 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 341AA8D0002; Mon, 26 Jun 2023 10:14:24 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2F1B78D0001; Mon, 26 Jun 2023 10:14:24 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 193AF8D0002; Mon, 26 Jun 2023 10:14:24 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 0CE8D8D0001 for ; Mon, 26 Jun 2023 10:14:24 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id E0A631C7D79 for ; Mon, 26 Jun 2023 14:14:23 +0000 (UTC) X-FDA: 80945094006.06.24D87ED Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by imf09.hostedemail.com (Postfix) with ESMTP id A126514002E for ; Mon, 26 Jun 2023 14:14:20 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=HiVFAgpc; spf=pass (imf09.hostedemail.com: domain of kai.huang@intel.com designates 134.134.136.126 as permitted sender) smtp.mailfrom=kai.huang@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1687788861; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=O0zV14ARBZH4CT6JqnymnoFIn2R1e9PMRCYcNEm2LPs=; b=HiRWwc6E1m/aARia9fPGCcqrSr2eV3JgmU/X5t89JXLEMRT1aHd04c7iNwH8sCoGad01x1 o76AjabXRBU61eMaFNQXVtTjyY6g68b0KNhg/dZGBaja1lMKsk7VN0BM4AKGNqmyVLZjrk 7BLxDN2CxQcJEOO+AaO5LWFcQ/pTwNA= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1687788861; a=rsa-sha256; cv=none; b=ThqdMHiANb8J3R7OFr+HoppjOELbpvjd/wQXxNlTv9O+nWOYBp/TRSMj5gWoj8RcR7yqzQ lp1O3MT4acvgm059wF1t5n4vyv361X8eNtmjou4DqYMnVwOBH1GCbIyIzJEHiaHmLRn28H vSgAKhHqhD5v0aZAdxPvhGayVqj0U/0= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=HiVFAgpc; spf=pass (imf09.hostedemail.com: domain of kai.huang@intel.com designates 134.134.136.126 as permitted sender) smtp.mailfrom=kai.huang@intel.com; dmarc=pass (policy=none) header.from=intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1687788860; x=1719324860; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=djNSftBcYLpF7pO3r41BBoH2hINTuguq/oJSItu9PeQ=; b=HiVFAgpcYGofIxZoCuVQW36dU83EYKMtQF9UE8eZwQsrhKqS9maVj1cw 43H4+s3kn/H27vugWeHmJVWXzUUmdCv42VM72lmY4Toy8MBnK21OOSQeD OSgUKr9LU5nyYeHjSubDqNxjcOFNmcnQ+QRHoWSW1Ifr0Z/yqL4+mDe5v rFXbZabnqklYnHQjyg5fetRxBGuoWLtzBrnQVSTWCWZKaLK2qgfvPpbKx 7xNjy9qkxx7a88Ru2EPKXtWT6cK6DYsNhGYMwypdLuGAsFozI8MBIIIcz 3DNEDhVCwMhmIzo9NeHL5gHLPwMGPhGBp+91tGx3kEUTZquOCNEGI81O7 g==; X-IronPort-AV: E=McAfee;i="6600,9927,10753"; a="346033700" X-IronPort-AV: E=Sophos;i="6.01,159,1684825200"; d="scan'208";a="346033700" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Jun 2023 07:14:20 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10753"; a="890292322" X-IronPort-AV: E=Sophos;i="6.01,159,1684825200"; d="scan'208";a="890292322" Received: from smithau-mobl1.amr.corp.intel.com (HELO khuang2-desk.gar.corp.intel.com) ([10.213.179.223]) by orsmga005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Jun 2023 07:14:13 -0700 From: Kai Huang To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: linux-mm@kvack.org, x86@kernel.org, dave.hansen@intel.com, kirill.shutemov@linux.intel.com, tony.luck@intel.com, peterz@infradead.org, tglx@linutronix.de, bp@alien8.de, mingo@redhat.com, hpa@zytor.com, seanjc@google.com, pbonzini@redhat.com, david@redhat.com, dan.j.williams@intel.com, rafael.j.wysocki@intel.com, ashok.raj@intel.com, reinette.chatre@intel.com, len.brown@intel.com, ak@linux.intel.com, isaku.yamahata@intel.com, ying.huang@intel.com, chao.gao@intel.com, sathyanarayanan.kuppuswamy@linux.intel.com, nik.borisov@suse.com, bagasdotme@gmail.com, sagis@google.com, imammedo@redhat.com, kai.huang@intel.com Subject: [PATCH v12 07/22] x86/virt/tdx: Add skeleton to enable TDX on demand Date: Tue, 27 Jun 2023 02:12:37 +1200 Message-Id: <104d324cd68b12e14722ee5d85a660cccccd8892.1687784645.git.kai.huang@intel.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Queue-Id: A126514002E X-Rspam-User: X-Stat-Signature: rtybcu71pj94t48aq7nibx3emqraryto X-Rspamd-Server: rspam03 X-HE-Tag: 1687788860-464659 X-HE-Meta: U2FsdGVkX1+MEWCVLkVWM18F8we7mXDhmf5pdKKCC3GH6xrF3TEdjO3w7JGQmxxgCW/c0b63lmY/6dO0tgpa62JqdSRAzs3kSeLqGABEmY8XUPdutS5zN+NGLyhZWoG8oAJj/m3qPJ+zuewkSfUtF4LqV+vWYSrulZw9Se7OACcAewvAgMSK7mJ23CExSRF9GIbFfa4GbF0ORvBukLVF91+zicXnC8nZR3SlISPFeuPoHL8efZzZzaIUYqAhFNObVPiNJegjJjfs+2qiCRJGUvnN7F2zM6erLkIBPMrKwreSY2ryvlXRUtCuIBApv1Q6wUyx0PhOI47wdcqVZNOG1BQbNHoCZP99FYGGPzW0DEfB9bnW1Rso+UHTOyCW7/m3PwtRF+bi/aZscm2y6ZrwT6FQAAaH2TPoRPA5RVCBMlmivdk0k+DklYKfLIdXYZj49+9OEn6bZnRtD6iXHuqXaapz5Txzt6y6E4UQSm6wEki9y36FWtUoBjuXvRyLYdneo410Xp4Ih+3at1izXojnx8nzA3o05tx5KSdZRPF/DJcOLNSdGKbmtzUYMh4ALlpMlSau+cCeoq+BaeJDlcf2JlCGXyt4jblYIKdjMiUAP0ZDMqnbDLdwDz3HH+M67zO6fsnCz4kqxkpTZfi3Era+plvtKY5eovt/k438CbqzrSPQjU6Ir7EX9l6pxnd46GPif1cQ97VlyvLpyLkEuJ3t1hYv3VFoT8KRr+R3QUdHr4AxKb8Y6m/7S+xgEoPWgZWf+EZGzxChZL6JdrxMQ+uQBB/fDQrbcg9Ft2mlgilqe4RBUQYD0p/K/0nnkNfmgQLndPMblkCS5vJT7JHnpdtXDuSFxx4S4oCtrVRfFaqlQl/gXcHQ73zCoQ5c05Pyp3eI9CUGBS1iN72/Az1YtycGcaHzDLyKaqb7kSsz9Q+APUMKc1Mn0RlxcSK+swlQxu/ExEIJikYYXxWM++Npbrh SLOue7zv 0JQheysea82uIWbR4lD0sm6MTUoz+vlOKyEXTC4ECpzMT+HClObjAQLFjccQ4dHpWIpDB3MDD15BcnFtfpkCaZTdpwyF0yZAvl1dGvN7lEIOMYWeqh+cwh09luV2fhXHX+dvU4zYB8VsLOetsPPt81XXdj+0FYFQMHpo7 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: To enable TDX the kernel needs to initialize TDX from two perspectives: 1) Do a set of SEAMCALLs to initialize the TDX module to make it ready to create and run TDX guests; 2) Do the per-cpu initialization SEAMCALL on one logical cpu before the kernel wants to make any other SEAMCALLs on that cpu (including those involved during module initialization and running TDX guests). The TDX module can be initialized only once in its lifetime. Instead of always initializing it at boot time, this implementation chooses an "on demand" approach to initialize TDX until there is a real need (e.g when requested by KVM). This approach has below pros: 1) It avoids consuming the memory that must be allocated by kernel and given to the TDX module as metadata (~1/256th of the TDX-usable memory), and also saves the CPU cycles of initializing the TDX module (and the metadata) when TDX is not used at all. 2) The TDX module design allows it to be updated while the system is running. The update procedure shares quite a few steps with this "on demand" initialization mechanism. The hope is that much of "on demand" mechanism can be shared with a future "update" mechanism. A boot-time TDX module implementation would not be able to share much code with the update mechanism. 3) Making SEAMCALL requires VMX to be enabled. Currently, only the KVM code mucks with VMX enabling. If the TDX module were to be initialized separately from KVM (like at boot), the boot code would need to be taught how to muck with VMX enabling and KVM would need to be taught how to cope with that. Making KVM itself responsible for TDX initialization lets the rest of the kernel stay blissfully unaware of VMX. Similar to module initialization, also make the per-cpu initialization "on demand" as it also depends on VMX being enabled. Add two functions, tdx_enable() and tdx_cpu_enable(), to enable the TDX module and enable TDX on local cpu respectively. For now tdx_enable() is a placeholder. The TODO list will be pared down as functionality is added. Export both tdx_cpu_enable() and tdx_enable() for KVM use. In tdx_enable() use a state machine protected by mutex to make sure the initialization will only be done once, as tdx_enable() can be called multiple times (i.e. KVM module can be reloaded) and may be called concurrently by other kernel components in the future. The per-cpu initialization on each cpu can only be done once during the module's life time. Use a per-cpu variable to track its status to make sure it is only done once in tdx_cpu_enable(). Also, a SEAMCALL to do TDX module global initialization must be done once on any logical cpu before any per-cpu initialization SEAMCALL. Do it inside tdx_cpu_enable() too (if hasn't been done). tdx_enable() can potentially invoke SEAMCALLs on any online cpus. The per-cpu initialization must be done before those SEAMCALLs are invoked on some cpu. To keep things simple, in tdx_cpu_enable(), always do the per-cpu initialization regardless of whether the TDX module has been initialized or not. And in tdx_enable(), don't call tdx_cpu_enable() but assume the caller has disabled CPU hotplug, done VMXON and tdx_cpu_enable() on all online cpus before calling tdx_enable(). Signed-off-by: Kai Huang --- v11 -> v12: - Simplified TDX module global init and lp init status tracking (David). - Added comment around try_init_module_global() for using raw_spin_lock() (Dave). - Added one sentence to changelog to explain why to expose tdx_enable() and tdx_cpu_enable() (Dave). - Simplifed comments around tdx_enable() and tdx_cpu_enable() to use lockdep_assert_*() instead. (Dave) - Removed redundent "TDX" in error message (Dave). v10 -> v11: - Return -NODEV instead of -EINVAL when CONFIG_INTEL_TDX_HOST is off. - Return the actual error code for tdx_enable() instead of -EINVAL. - Added Isaku's Reviewed-by. v9 -> v10: - Merged the patch to handle per-cpu initialization to this patch to tell the story better. - Changed how to handle the per-cpu initialization to only provide a tdx_cpu_enable() function to let the user of TDX to do it when the user wants to run TDX code on a certain cpu. - Changed tdx_enable() to not call cpus_read_lock() explicitly, but call lockdep_assert_cpus_held() to assume the caller has done that. - Improved comments around tdx_enable() and tdx_cpu_enable(). - Improved changelog to tell the story better accordingly. v8 -> v9: - Removed detailed TODO list in the changelog (Dave). - Added back steps to do module global initialization and per-cpu initialization in the TODO list comment. - Moved the 'enum tdx_module_status_t' from tdx.c to local tdx.h v7 -> v8: - Refined changelog (Dave). - Removed "all BIOS-enabled cpus" related code (Peter/Thomas/Dave). - Add a "TODO list" comment in init_tdx_module() to list all steps of initializing the TDX Module to tell the story (Dave). - Made tdx_enable() unverisally return -EINVAL, and removed nonsense comments (Dave). - Simplified __tdx_enable() to only handle success or failure. - TDX_MODULE_SHUTDOWN -> TDX_MODULE_ERROR - Removed TDX_MODULE_NONE (not loaded) as it is not necessary. - Improved comments (Dave). - Pointed out 'tdx_module_status' is software thing (Dave). v6 -> v7: - No change. v5 -> v6: - Added code to set status to TDX_MODULE_NONE if TDX module is not loaded (Chao) - Added Chao's Reviewed-by. - Improved comments around cpus_read_lock(). - v3->v5 (no feedback on v4): - Removed the check that SEAMRR and TDX KeyID have been detected on all present cpus. - Removed tdx_detect(). - Added num_online_cpus() to MADT-enabled CPUs check within the CPU hotplug lock and return early with error message. - Improved dmesg printing for TDX module detection and initialization. --- arch/x86/include/asm/tdx.h | 4 + arch/x86/virt/vmx/tdx/tdx.c | 162 ++++++++++++++++++++++++++++++++++++ arch/x86/virt/vmx/tdx/tdx.h | 13 +++ 3 files changed, 179 insertions(+) diff --git a/arch/x86/include/asm/tdx.h b/arch/x86/include/asm/tdx.h index 4dfe2e794411..d8226a50c58c 100644 --- a/arch/x86/include/asm/tdx.h +++ b/arch/x86/include/asm/tdx.h @@ -97,8 +97,12 @@ static inline long tdx_kvm_hypercall(unsigned int nr, unsigned long p1, #ifdef CONFIG_INTEL_TDX_HOST bool platform_tdx_enabled(void); +int tdx_cpu_enable(void); +int tdx_enable(void); #else /* !CONFIG_INTEL_TDX_HOST */ static inline bool platform_tdx_enabled(void) { return false; } +static inline int tdx_cpu_enable(void) { return -ENODEV; } +static inline int tdx_enable(void) { return -ENODEV; } #endif /* CONFIG_INTEL_TDX_HOST */ #endif /* !__ASSEMBLY__ */ diff --git a/arch/x86/virt/vmx/tdx/tdx.c b/arch/x86/virt/vmx/tdx/tdx.c index 141d12376c4d..29ca18f66d61 100644 --- a/arch/x86/virt/vmx/tdx/tdx.c +++ b/arch/x86/virt/vmx/tdx/tdx.c @@ -13,6 +13,10 @@ #include #include #include +#include +#include +#include +#include #include #include #include @@ -23,6 +27,13 @@ static u32 tdx_global_keyid __ro_after_init; static u32 tdx_guest_keyid_start __ro_after_init; static u32 tdx_nr_guest_keyids __ro_after_init; +static bool tdx_global_initialized; +static DEFINE_RAW_SPINLOCK(tdx_global_init_lock); +static DEFINE_PER_CPU(bool, tdx_lp_initialized); + +static enum tdx_module_status_t tdx_module_status; +static DEFINE_MUTEX(tdx_module_lock); + /* * Wrapper of __seamcall() to convert SEAMCALL leaf function error code * to kernel error code. @seamcall_ret and @out contain the SEAMCALL @@ -74,6 +85,157 @@ static int __always_unused seamcall(u64 fn, u64 rcx, u64 rdx, u64 r8, u64 r9, } } +/* + * Do the module global initialization if not done yet. + * It's always called with interrupts and preemption disabled. + */ +static int try_init_module_global(void) +{ + unsigned long flags; + int ret; + + /* + * The TDX module global initialization only needs to be done + * once on any cpu. + */ + raw_spin_lock_irqsave(&tdx_global_init_lock, flags); + + if (tdx_global_initialized) { + ret = 0; + goto out; + } + + /* All '0's are just unused parameters. */ + ret = seamcall(TDH_SYS_INIT, 0, 0, 0, 0, NULL, NULL); + if (!ret) + tdx_global_initialized = true; +out: + raw_spin_unlock_irqrestore(&tdx_global_init_lock, flags); + + return ret; +} + +/** + * tdx_cpu_enable - Enable TDX on local cpu + * + * Do one-time TDX module per-cpu initialization SEAMCALL (and TDX module + * global initialization SEAMCALL if not done) on local cpu to make this + * cpu be ready to run any other SEAMCALLs. + * + * Call this function with preemption disabled. + * + * Return 0 on success, otherwise errors. + */ +int tdx_cpu_enable(void) +{ + int ret; + + if (!platform_tdx_enabled()) + return -ENODEV; + + lockdep_assert_preemption_disabled(); + + /* Already done */ + if (__this_cpu_read(tdx_lp_initialized)) + return 0; + + /* + * The TDX module global initialization is the very first step + * to enable TDX. Need to do it first (if hasn't been done) + * before the per-cpu initialization. + */ + ret = try_init_module_global(); + if (ret) + return ret; + + /* All '0's are just unused parameters */ + ret = seamcall(TDH_SYS_LP_INIT, 0, 0, 0, 0, NULL, NULL); + if (ret) + return ret; + + __this_cpu_write(tdx_lp_initialized, true); + + return 0; +} +EXPORT_SYMBOL_GPL(tdx_cpu_enable); + +static int init_tdx_module(void) +{ + /* + * TODO: + * + * - Get TDX module information and TDX-capable memory regions. + * - Build the list of TDX-usable memory regions. + * - Construct a list of "TD Memory Regions" (TDMRs) to cover + * all TDX-usable memory regions. + * - Configure the TDMRs and the global KeyID to the TDX module. + * - Configure the global KeyID on all packages. + * - Initialize all TDMRs. + * + * Return error before all steps are done. + */ + return -EINVAL; +} + +static int __tdx_enable(void) +{ + int ret; + + ret = init_tdx_module(); + if (ret) { + pr_err("module initialization failed (%d)\n", ret); + tdx_module_status = TDX_MODULE_ERROR; + return ret; + } + + pr_info("module initialized.\n"); + tdx_module_status = TDX_MODULE_INITIALIZED; + + return 0; +} + +/** + * tdx_enable - Enable TDX module to make it ready to run TDX guests + * + * This function assumes the caller has: 1) held read lock of CPU hotplug + * lock to prevent any new cpu from becoming online; 2) done both VMXON + * and tdx_cpu_enable() on all online cpus. + * + * This function can be called in parallel by multiple callers. + * + * Return 0 if TDX is enabled successfully, otherwise error. + */ +int tdx_enable(void) +{ + int ret; + + if (!platform_tdx_enabled()) + return -ENODEV; + + lockdep_assert_cpus_held(); + + mutex_lock(&tdx_module_lock); + + switch (tdx_module_status) { + case TDX_MODULE_UNKNOWN: + ret = __tdx_enable(); + break; + case TDX_MODULE_INITIALIZED: + /* Already initialized, great, tell the caller. */ + ret = 0; + break; + default: + /* Failed to initialize in the previous attempts */ + ret = -EINVAL; + break; + } + + mutex_unlock(&tdx_module_lock); + + return ret; +} +EXPORT_SYMBOL_GPL(tdx_enable); + static int __init record_keyid_partitioning(u32 *tdx_keyid_start, u32 *nr_tdx_keyids) { diff --git a/arch/x86/virt/vmx/tdx/tdx.h b/arch/x86/virt/vmx/tdx/tdx.h index 55dbb1b8c971..9fb46033c852 100644 --- a/arch/x86/virt/vmx/tdx/tdx.h +++ b/arch/x86/virt/vmx/tdx/tdx.h @@ -16,11 +16,24 @@ */ #define TDX_RND_NO_ENTROPY 0x8000020300000000ULL +/* + * TDX module SEAMCALL leaf functions + */ +#define TDH_SYS_INIT 33 +#define TDH_SYS_LP_INIT 35 + /* * Do not put any hardware-defined TDX structure representations below * this comment! */ +/* Kernel defined TDX module status during module initialization. */ +enum tdx_module_status_t { + TDX_MODULE_UNKNOWN, + TDX_MODULE_INITIALIZED, + TDX_MODULE_ERROR +}; + struct tdx_module_output; u64 __seamcall(u64 fn, u64 rcx, u64 rdx, u64 r8, u64 r9, struct tdx_module_output *out);