From patchwork Mon Feb 26 08:25:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571429 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6A43A1DA3F; Mon, 26 Feb 2024 08:27:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.16 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936057; cv=none; b=DBRIxIiCZ/xZa3c+dbSNWWcCO+SNPCkNKv9yyzNCgdHFJe/K/LSxbP4nHngBpl9K69LjJ7xyd34wbxkL2H+thDQ+e2gyAwqsCjEqmFfmPLWo3HaOyeQweIvryIHazXsY1V+T9q7k4a0EyaKW4X8FxBjChc9pNU1JnL05kq7se3k= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936057; c=relaxed/simple; bh=w4lY8P57h5k6ODIw2ePbR3iDGqUeK20stwm2wWMGTNQ=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=SGd9ir5hsXH8d/PcqEDgpwjgLW08nK9feJhBcrpRMEBezT4xvYIOke3PDh57lmTKJgYVChkZ0IImTGLqTBi/trivctplD5gHuJZwIjvfEFrFgCHwfe5yeziXY5ckIMzhOEyCKT3S0Tgzf6sCdWQpDN6u4l9t4K/EWx1h1cfK9EQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=HzQYTdDz; arc=none smtp.client-ip=198.175.65.16 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="HzQYTdDz" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936056; x=1740472056; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=w4lY8P57h5k6ODIw2ePbR3iDGqUeK20stwm2wWMGTNQ=; b=HzQYTdDz1vU34CU77d53Ae1ZlSJLHE/pHMokY5oREH05Tr2Rq1WkHf5W b6n3rpcLCFsIYFAFKMSlTqlqz7vkOwM1L3ag1Tqa8cfgfUuJ9ZTM/PvNQ gVkLr+DCLi/T4ylRUOI+T30jCGVjgD3MJTOgBV2JBVrOmOxx7zeV1xVeM Ud2mBuwTJPx7dTVOORBjRTnYQ6/J5STDpQITkXCFKaJqr3RPQvP+EGczE EvNFDl64H+p8z7LuiS0vcOxBx3/W100ax6Mc+lqtj7AjTWuf8gBAZ2EbY Hdjz7Wa40zBOIOOknW9GdVKvqVF88mdTytMSjaMSQm9liAeD/3rRQECiT Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="3340694" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="3340694" Received: from orviesa006.jf.intel.com ([10.64.159.146]) by orvoesa108.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:27:34 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="7020034" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa006-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:27:33 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com, "Kirill A . Shutemov" Subject: [PATCH v19 001/130] x86/virt/tdx: Rename _offset to _member for TD_SYSINFO_MAP() macro Date: Mon, 26 Feb 2024 00:25:03 -0800 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Kai Huang TD_SYSINFO_MAP() macro actually takes the member of the 'struct tdx_tdmr_sysinfo' as the second argument and uses the offsetof() to calculate the offset for that member. Rename the macro argument _offset to _member to reflect this. Signed-off-by: Kai Huang Reviewed-by: Kirill A. Shutemov Signed-off-by: Isaku Yamahata Reviewed-by: Binbin Wu --- arch/x86/virt/vmx/tdx/tdx.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/arch/x86/virt/vmx/tdx/tdx.c b/arch/x86/virt/vmx/tdx/tdx.c index 4d6826a76f78..2aee64d2f27f 100644 --- a/arch/x86/virt/vmx/tdx/tdx.c +++ b/arch/x86/virt/vmx/tdx/tdx.c @@ -297,9 +297,9 @@ struct field_mapping { int offset; }; -#define TD_SYSINFO_MAP(_field_id, _offset) \ +#define TD_SYSINFO_MAP(_field_id, _member) \ { .field_id = MD_FIELD_ID_##_field_id, \ - .offset = offsetof(struct tdx_tdmr_sysinfo, _offset) } + .offset = offsetof(struct tdx_tdmr_sysinfo, _member) } /* Map TD_SYSINFO fields into 'struct tdx_tdmr_sysinfo': */ static const struct field_mapping fields[] = { From patchwork Mon Feb 26 08:25:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571430 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BB65A1EB2F; Mon, 26 Feb 2024 08:27:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.16 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936058; cv=none; b=fP4Q+C1/35TZw9WkK2RD4byUsovURWrhXXY85z0qANhqWPVAaTuQoODT0nU8apMujA8GdF1wesx50yj8fT+ZYrNFYZuFlIAor6TXciVoB2useCFj7GRoz5AjbOj9jUxkl684DLp1eI5cX1mcsbReuynPe6h0rjt3YHIlx/OI0aE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936058; c=relaxed/simple; bh=0J/36UWM7sYbzkdjbiDRhB/xvJEm5o7DRTxpZXmcMqs=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=MFsxZSLcfUOf1j1mNkzKxx6KmpvDyU892xHad/gzX/gNbiib7alEt/Mcik31xknpxjF4bMBtCJd90dHn5M+myFDc2BGbF7YC1SYk0yNHP28bytg5iA92MkXX0HSOq3/a5Vzh4HCGGxMBtpv9iRO+BsSTt8kY1kSIKNj79Y2LfG4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=KIqmk+t0; arc=none smtp.client-ip=198.175.65.16 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="KIqmk+t0" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936057; x=1740472057; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=0J/36UWM7sYbzkdjbiDRhB/xvJEm5o7DRTxpZXmcMqs=; b=KIqmk+t0TW7TD3wjNXBD4A2qWIPN5CTtTAPGGlMag+/aTqey57vd274L HOMlcwdA3jDk6aLmNuuxx79VmD7TR9fDth0SEwAiXIjS48OfCoNLpaOFA g68RUnf6YmQGfeNFXGRBjvfu8Yw3C1kwEUoDHcv6r1Ob6GpZ+2JkiPI1K rBzWxTAsonK8HMwk7xe7SjLRSQD65vr75UJDicPWO6AT4qFgBHVmmMIaA t+w5gW746SNCi0sn1YUFa+tsw4FPqX0xo5cbgkanmy3N7Lc7jjIcG0urs rZKb8nQCzJgLvtKwyLEFQesREONB2WHTcVUWnsU/NgdVuuKxA+kFifirM w==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="3340698" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="3340698" Received: from orviesa006.jf.intel.com ([10.64.159.146]) by orvoesa108.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:27:35 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="7020045" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa006-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:27:34 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 002/130] x86/virt/tdx: Move TDMR metadata fields map table to local variable Date: Mon, 26 Feb 2024 00:25:04 -0800 Message-Id: <44d9530187b4b0b1c05e150fa73fe22ab54fc911.1708933498.git.isaku.yamahata@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Kai Huang The kernel reads all TDMR related global metadata fields based on a table which maps the metadata fields to the corresponding members of 'struct tdx_tdmr_sysinfo'. Currently this table is a static variable. But this table is only used by the function which reads these metadata fields and becomes useless after reading is done. Change the table to function local variable. Signed-off-by: Kai Huang Signed-off-by: Isaku Yamahata Reviewed-by: Binbin Wu Reviewed-by: Kirill A. Shutemov --- arch/x86/virt/vmx/tdx/tdx.c | 17 ++++++++--------- 1 file changed, 8 insertions(+), 9 deletions(-) diff --git a/arch/x86/virt/vmx/tdx/tdx.c b/arch/x86/virt/vmx/tdx/tdx.c index 2aee64d2f27f..cdcb3332bc5d 100644 --- a/arch/x86/virt/vmx/tdx/tdx.c +++ b/arch/x86/virt/vmx/tdx/tdx.c @@ -301,17 +301,16 @@ struct field_mapping { { .field_id = MD_FIELD_ID_##_field_id, \ .offset = offsetof(struct tdx_tdmr_sysinfo, _member) } -/* Map TD_SYSINFO fields into 'struct tdx_tdmr_sysinfo': */ -static const struct field_mapping fields[] = { - TD_SYSINFO_MAP(MAX_TDMRS, max_tdmrs), - TD_SYSINFO_MAP(MAX_RESERVED_PER_TDMR, max_reserved_per_tdmr), - TD_SYSINFO_MAP(PAMT_4K_ENTRY_SIZE, pamt_entry_size[TDX_PS_4K]), - TD_SYSINFO_MAP(PAMT_2M_ENTRY_SIZE, pamt_entry_size[TDX_PS_2M]), - TD_SYSINFO_MAP(PAMT_1G_ENTRY_SIZE, pamt_entry_size[TDX_PS_1G]), -}; - static int get_tdx_tdmr_sysinfo(struct tdx_tdmr_sysinfo *tdmr_sysinfo) { + /* Map TD_SYSINFO fields into 'struct tdx_tdmr_sysinfo': */ + const struct field_mapping fields[] = { + TD_SYSINFO_MAP(MAX_TDMRS, max_tdmrs), + TD_SYSINFO_MAP(MAX_RESERVED_PER_TDMR, max_reserved_per_tdmr), + TD_SYSINFO_MAP(PAMT_4K_ENTRY_SIZE, pamt_entry_size[TDX_PS_4K]), + TD_SYSINFO_MAP(PAMT_2M_ENTRY_SIZE, pamt_entry_size[TDX_PS_2M]), + TD_SYSINFO_MAP(PAMT_1G_ENTRY_SIZE, pamt_entry_size[TDX_PS_1G]), + }; int ret; int i; From patchwork Mon Feb 26 08:25:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571432 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 529801EB5B; Mon, 26 Feb 2024 08:27:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.16 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936060; cv=none; b=tfLTV6l7Pxt3gb9+dWhzIYuniN0f2+V+Sl2fUD8i6j4LQnXU0vmRPxTeK8Rdim88dX3Uku/hG6N801R7ejZmYP27KTpKwxnTGkaQBSaZnOoWAcncq4ZjUrWxwBgQkc3wAKQfz6wsxGr+l+jWvAvf1gE8o1bwtE8aK4VIT3gIGOM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936060; c=relaxed/simple; bh=Ipwvsm7XmA6ghvLHLEgf7+RqG0jtiV2UhCk6lJOoMpU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=QRHUuKBmxinpY3egpJ8uNXmwoe7bWr3RuzCFDN2GHlaHvjrDy4lXm+bnhGrfXXtmj3x/29JgRKt1lgf4p9cv3g0s4mmdnxktwhxelA1C7FbxLY/9EZsKp4MDOCl2GYWqfp/hs8J79wIq8D4bU6mEX5H6dLEF3l1TYaFgeJmxYxw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=BYFQ0Z3s; arc=none smtp.client-ip=198.175.65.16 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="BYFQ0Z3s" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936058; x=1740472058; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Ipwvsm7XmA6ghvLHLEgf7+RqG0jtiV2UhCk6lJOoMpU=; b=BYFQ0Z3stVG43PiQHxUs2lAS3NHq02vxRFzADqa1Fk92Sq25HoWKOwIO pTiSqKjwch9jzABzydlFcQMHZPLwslA1bFm3jl/ovKvQfb0yOcKErqtoC 25PEtDECxF0pYdCZQ9OqllCaVuRaFnheEVMGxQJE/HNzKMSjPdvL7AqZ2 j7Ko1gqTWG/HHwgh0qX5Hjjrbt3shxcWkLG0ebhtbDfPdfbcfoehBtkp0 OGOVvO3t39wfMXjg+/xQoqWoZ0kbjHNkQTkHEpStTSHuPSRFXVOXvYiQz 3g5JIPURH0MHinG5kZlFKH+OzDdhsDD4xNEIr7giQc+NZY/epLcsF58zx Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="3340702" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="3340702" Received: from orviesa006.jf.intel.com ([10.64.159.146]) by orvoesa108.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:27:36 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="7020055" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa006-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:27:35 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com, "Kirill A . Shutemov" Subject: [PATCH v19 003/130] x86/virt/tdx: Unbind global metadata read with 'struct tdx_tdmr_sysinfo' Date: Mon, 26 Feb 2024 00:25:05 -0800 Message-Id: <96c21cc1d283cf59ecba003cd5a19bfbce83675d.1708933498.git.isaku.yamahata@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Kai Huang For now the kernel only reads TDMR related global metadata fields for module initialization, and the metadata read code only works with the 'struct tdx_tdmr_sysinfo'. KVM will need to read a bunch of non-TDMR related metadata to create and run TDX guests. It's essential to provide a generic metadata read infrastructure which is not bound to any specific structure. To start providing such infrastructure, unbound the metadata read with the 'struct tdx_tdmr_sysinfo'. Signed-off-by: Kai Huang Reviewed-by: Kirill A. Shutemov Signed-off-by: Isaku Yamahata Reviewed-by: Binbin Wu --- arch/x86/virt/vmx/tdx/tdx.c | 25 ++++++++++++++----------- 1 file changed, 14 insertions(+), 11 deletions(-) diff --git a/arch/x86/virt/vmx/tdx/tdx.c b/arch/x86/virt/vmx/tdx/tdx.c index cdcb3332bc5d..eb208da4ff63 100644 --- a/arch/x86/virt/vmx/tdx/tdx.c +++ b/arch/x86/virt/vmx/tdx/tdx.c @@ -273,9 +273,9 @@ static int read_sys_metadata_field(u64 field_id, u64 *data) static int read_sys_metadata_field16(u64 field_id, int offset, - struct tdx_tdmr_sysinfo *ts) + void *stbuf) { - u16 *ts_member = ((void *)ts) + offset; + u16 *st_member = stbuf + offset; u64 tmp; int ret; @@ -287,7 +287,7 @@ static int read_sys_metadata_field16(u64 field_id, if (ret) return ret; - *ts_member = tmp; + *st_member = tmp; return 0; } @@ -297,19 +297,22 @@ struct field_mapping { int offset; }; -#define TD_SYSINFO_MAP(_field_id, _member) \ - { .field_id = MD_FIELD_ID_##_field_id, \ - .offset = offsetof(struct tdx_tdmr_sysinfo, _member) } +#define TD_SYSINFO_MAP(_field_id, _struct, _member) \ + { .field_id = MD_FIELD_ID_##_field_id, \ + .offset = offsetof(_struct, _member) } + +#define TD_SYSINFO_MAP_TDMR_INFO(_field_id, _member) \ + TD_SYSINFO_MAP(_field_id, struct tdx_tdmr_sysinfo, _member) static int get_tdx_tdmr_sysinfo(struct tdx_tdmr_sysinfo *tdmr_sysinfo) { /* Map TD_SYSINFO fields into 'struct tdx_tdmr_sysinfo': */ const struct field_mapping fields[] = { - TD_SYSINFO_MAP(MAX_TDMRS, max_tdmrs), - TD_SYSINFO_MAP(MAX_RESERVED_PER_TDMR, max_reserved_per_tdmr), - TD_SYSINFO_MAP(PAMT_4K_ENTRY_SIZE, pamt_entry_size[TDX_PS_4K]), - TD_SYSINFO_MAP(PAMT_2M_ENTRY_SIZE, pamt_entry_size[TDX_PS_2M]), - TD_SYSINFO_MAP(PAMT_1G_ENTRY_SIZE, pamt_entry_size[TDX_PS_1G]), + TD_SYSINFO_MAP_TDMR_INFO(MAX_TDMRS, max_tdmrs), + TD_SYSINFO_MAP_TDMR_INFO(MAX_RESERVED_PER_TDMR, max_reserved_per_tdmr), + TD_SYSINFO_MAP_TDMR_INFO(PAMT_4K_ENTRY_SIZE, pamt_entry_size[TDX_PS_4K]), + TD_SYSINFO_MAP_TDMR_INFO(PAMT_2M_ENTRY_SIZE, pamt_entry_size[TDX_PS_2M]), + TD_SYSINFO_MAP_TDMR_INFO(PAMT_1G_ENTRY_SIZE, pamt_entry_size[TDX_PS_1G]), }; int ret; int i; From patchwork Mon Feb 26 08:25:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571431 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 56FA21EEEA; Mon, 26 Feb 2024 08:27:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.16 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936059; cv=none; b=RnY0MUTQ3ByxnOodU7zpqQgf2QiRp7QOEPudp1/oZw6J7nsIzwPRtz8vUOvBxCUT2UO0xQW2ZgAKNaAm1lN5bAwZSPTWDe4TmlrNnVRG7uBeuX+prxGbqz/+dODUOihq7r9LmPhD8wyubrudEn1gvWkEsdcL0lHLdLNlCbn9upo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936059; c=relaxed/simple; bh=CebvYlBsSGTBw3oWiiDipBdbdj7zNc7xmDvoSKzO9SY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=JS9FcnuUvGyRZfwsPeXetmgvmiLlZga9L7jB8Cn6XIjuSQR1Kktro874CUiURDTbWSVvJnIe0prWENNiOT4KcUX3b3HiRjCnsUwsNUEblV4KT7xSmStMRpMkk9RrRTcZmT3q7sfjlwF07EPeJGbxqLhyArTaaSeTxATgWpwLRwo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=FSWBBbRM; arc=none smtp.client-ip=198.175.65.16 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="FSWBBbRM" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936058; x=1740472058; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=CebvYlBsSGTBw3oWiiDipBdbdj7zNc7xmDvoSKzO9SY=; b=FSWBBbRMLitF5IqDCT7dvd9kY8U3cOk5mSm+KlP/+IBlD5S5/WA4Ufq2 7IoeQmbplalzOzAGD9DNgaAIlWiEpzpazXmojkSyIOFrHvhPZq8Gb9cdB mf+cC7s3WUb5o5fKvAubsmFOodvkajGblTDp0+ylfnLGKKVj6SWH0z5KI BH0CJyx0FvCA01KdNqxe3a0Z6/4Wa3kUeACww5FiKmP92slxUE1M6cqog vN5sWFWMMjn+QSvytCTWFgWsjBG3RlvSO/Vbs8h2jBWesARNSR4cF0iOF HDpcYQGLKmKAx5rNgsdy+kriIEByvSmYNQfs9n+bbaI5Z052JrGHTPhy6 w==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="3340707" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="3340707" Received: from orviesa006.jf.intel.com ([10.64.159.146]) by orvoesa108.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:27:37 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="7020066" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa006-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:27:36 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 004/130] x86/virt/tdx: Support global metadata read for all element sizes Date: Mon, 26 Feb 2024 00:25:06 -0800 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Kai Huang For now the kernel only reads TDMR related global metadata fields for module initialization. All these fields are 16-bits, and the kernel only supports reading 16-bits fields. KVM will need to read a bunch of non-TDMR related metadata to create and run TDX guests. It's essential to provide a generic metadata read infrastructure which supports reading all 8/16/32/64 bits element sizes. Extend the metadata read to support reading all these element sizes. Signed-off-by: Kai Huang Signed-off-by: Isaku Yamahata --- arch/x86/virt/vmx/tdx/tdx.c | 59 +++++++++++++++++++++++++------------ arch/x86/virt/vmx/tdx/tdx.h | 2 -- 2 files changed, 40 insertions(+), 21 deletions(-) diff --git a/arch/x86/virt/vmx/tdx/tdx.c b/arch/x86/virt/vmx/tdx/tdx.c index eb208da4ff63..a19adc898df6 100644 --- a/arch/x86/virt/vmx/tdx/tdx.c +++ b/arch/x86/virt/vmx/tdx/tdx.c @@ -271,23 +271,35 @@ static int read_sys_metadata_field(u64 field_id, u64 *data) return 0; } -static int read_sys_metadata_field16(u64 field_id, - int offset, - void *stbuf) +/* Return the metadata field element size in bytes */ +static int get_metadata_field_bytes(u64 field_id) { - u16 *st_member = stbuf + offset; + /* + * TDX supports 8/16/32/64 bits metadata field element sizes. + * TDX module determines the metadata element size based on the + * "element size code" encoded in the field ID (see the comment + * of MD_FIELD_ID_ELE_SIZE_CODE macro for specific encodings). + */ + return 1 << MD_FIELD_ID_ELE_SIZE_CODE(field_id); +} + +static int stbuf_read_sys_metadata_field(u64 field_id, + int offset, + int bytes, + void *stbuf) +{ + void *st_member = stbuf + offset; u64 tmp; int ret; - if (WARN_ON_ONCE(MD_FIELD_ID_ELE_SIZE_CODE(field_id) != - MD_FIELD_ID_ELE_SIZE_16BIT)) + if (WARN_ON_ONCE(get_metadata_field_bytes(field_id) != bytes)) return -EINVAL; ret = read_sys_metadata_field(field_id, &tmp); if (ret) return ret; - *st_member = tmp; + memcpy(st_member, &tmp, bytes); return 0; } @@ -295,11 +307,30 @@ static int read_sys_metadata_field16(u64 field_id, struct field_mapping { u64 field_id; int offset; + int size; }; #define TD_SYSINFO_MAP(_field_id, _struct, _member) \ { .field_id = MD_FIELD_ID_##_field_id, \ - .offset = offsetof(_struct, _member) } + .offset = offsetof(_struct, _member), \ + .size = sizeof(typeof(((_struct *)0)->_member)) } + +static int read_sys_metadata(struct field_mapping *fields, int nr_fields, + void *stbuf) +{ + int i, ret; + + for (i = 0; i < nr_fields; i++) { + ret = stbuf_read_sys_metadata_field(fields[i].field_id, + fields[i].offset, + fields[i].size, + stbuf); + if (ret) + return ret; + } + + return 0; +} #define TD_SYSINFO_MAP_TDMR_INFO(_field_id, _member) \ TD_SYSINFO_MAP(_field_id, struct tdx_tdmr_sysinfo, _member) @@ -314,19 +345,9 @@ static int get_tdx_tdmr_sysinfo(struct tdx_tdmr_sysinfo *tdmr_sysinfo) TD_SYSINFO_MAP_TDMR_INFO(PAMT_2M_ENTRY_SIZE, pamt_entry_size[TDX_PS_2M]), TD_SYSINFO_MAP_TDMR_INFO(PAMT_1G_ENTRY_SIZE, pamt_entry_size[TDX_PS_1G]), }; - int ret; - int i; /* Populate 'tdmr_sysinfo' fields using the mapping structure above: */ - for (i = 0; i < ARRAY_SIZE(fields); i++) { - ret = read_sys_metadata_field16(fields[i].field_id, - fields[i].offset, - tdmr_sysinfo); - if (ret) - return ret; - } - - return 0; + return read_sys_metadata(fields, ARRAY_SIZE(fields), tdmr_sysinfo); } /* Calculate the actual TDMR size */ diff --git a/arch/x86/virt/vmx/tdx/tdx.h b/arch/x86/virt/vmx/tdx/tdx.h index b701f69485d3..4c32c8bf156a 100644 --- a/arch/x86/virt/vmx/tdx/tdx.h +++ b/arch/x86/virt/vmx/tdx/tdx.h @@ -53,8 +53,6 @@ #define MD_FIELD_ID_ELE_SIZE_CODE(_field_id) \ (((_field_id) & GENMASK_ULL(33, 32)) >> 32) -#define MD_FIELD_ID_ELE_SIZE_16BIT 1 - struct tdmr_reserved_area { u64 offset; u64 size; From patchwork Mon Feb 26 08:25:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571433 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EE17E200D5; Mon, 26 Feb 2024 08:27:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.16 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936061; cv=none; b=bI4h32bjIgFic4tDx1KVV61iBhjj02hS9gziswhSyUm3wwwjFTQ+Qaxcqmvn5Bhlsyj5O2LN2hsffaH6mEiqLJw2lE2fSXDIwJpVLWUXOIj42PwI5oHPx/hzFVwyq8GEiLN25nSy602fUJUF58X3JuDT1ESbASxfL/Pi89rj/ic= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936061; c=relaxed/simple; bh=WwPAQVyS6ydkrI/ZnLxTBH2KaaNIW1B/7NwF7C9kOnA=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=ejQDQOvzpC71nXy5fgQ4sykAI+OJz71tt6WJBrguC3AfD6hEhIYwXw936ZBf/EIBLVrC/3ODdjxE30/ANoFfHE306L6XDKaEWxBM3ZqnscFKBIcT9hEUSAEU8vIrnr6oKsj7B3Vb72X/D+zKudUnvmY7PAPID4tU7S09XSWq0U0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=CKXMFwPT; arc=none smtp.client-ip=198.175.65.16 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="CKXMFwPT" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936059; x=1740472059; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=WwPAQVyS6ydkrI/ZnLxTBH2KaaNIW1B/7NwF7C9kOnA=; b=CKXMFwPTaQE5LrCyCQ+FoTc1WCsogcaL0w592cJI5tB8J3YHi2BOgBdl sDlsZW3gdXvh0uOPAgPhjqpPXcNoLau6iFSUWgkn2G1CfuKdC8Byy+/Q0 08tk/kTRx34+R46XAyoRb4DREEs3KRW3j+gK3cVwvqoHF+1MB2LHZ5OMS iRavtE2IIEAM2Cj/53NU0SKXXV0fNYgERdi94UXVnzqtEKP3lczsDkqmE WCunFC5uQFweQQ2BAHvegdEM3d8eHhygY1TcKofwbYUkRCDVjhjM/5lLo v2COmKXTsPf/nnAxEwecCTnpMtpt21T8+wYg3FTYzQn3/2AUhj+coy6wt A==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="3340712" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="3340712" Received: from orviesa006.jf.intel.com ([10.64.159.146]) by orvoesa108.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:27:38 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="7020075" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa006-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:27:37 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com, "Kirill A . Shutemov" Subject: [PATCH v19 005/130] x86/virt/tdx: Export global metadata read infrastructure Date: Mon, 26 Feb 2024 00:25:07 -0800 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Kai Huang KVM will need to read a bunch of non-TDMR related metadata to create and run TDX guests. Export the metadata read infrastructure for KVM to use. Specifically, export two helpers: 1) The helper which reads multiple metadata fields to a buffer of a structure based on the "field ID -> structure member" mapping table. 2) The low level helper which just reads a given field ID. The two helpers cover cases when the user wants to cache a bunch of metadata fields to a certain structure and when the user just wants to query a specific metadata field on demand. They are enough for KVM to use (and also should be enough for other potential users). Signed-off-by: Kai Huang Reviewed-by: Kirill A. Shutemov Signed-off-by: Isaku Yamahata Reviewed-by: Binbin Wu --- arch/x86/include/asm/tdx.h | 22 ++++++++++++++++++++++ arch/x86/virt/vmx/tdx/tdx.c | 25 ++++++++----------------- 2 files changed, 30 insertions(+), 17 deletions(-) diff --git a/arch/x86/include/asm/tdx.h b/arch/x86/include/asm/tdx.h index eba178996d84..709b9483f9e4 100644 --- a/arch/x86/include/asm/tdx.h +++ b/arch/x86/include/asm/tdx.h @@ -116,6 +116,28 @@ static inline u64 sc_retry(sc_func_t func, u64 fn, int tdx_cpu_enable(void); int tdx_enable(void); const char *tdx_dump_mce_info(struct mce *m); + +struct tdx_metadata_field_mapping { + u64 field_id; + int offset; + int size; +}; + +#define TD_SYSINFO_MAP(_field_id, _struct, _member) \ + { .field_id = MD_FIELD_ID_##_field_id, \ + .offset = offsetof(_struct, _member), \ + .size = sizeof(typeof(((_struct *)0)->_member)) } + +/* + * Read multiple global metadata fields to a buffer of a structure + * based on the "field ID -> structure member" mapping table. + */ +int tdx_sys_metadata_read(const struct tdx_metadata_field_mapping *fields, + int nr_fields, void *stbuf); + +/* Read a single global metadata field */ +int tdx_sys_metadata_field_read(u64 field_id, u64 *data); + #else static inline void tdx_init(void) { } static inline int tdx_cpu_enable(void) { return -ENODEV; } diff --git a/arch/x86/virt/vmx/tdx/tdx.c b/arch/x86/virt/vmx/tdx/tdx.c index a19adc898df6..dc21310776ab 100644 --- a/arch/x86/virt/vmx/tdx/tdx.c +++ b/arch/x86/virt/vmx/tdx/tdx.c @@ -251,7 +251,7 @@ static int build_tdx_memlist(struct list_head *tmb_list) return ret; } -static int read_sys_metadata_field(u64 field_id, u64 *data) +int tdx_sys_metadata_field_read(u64 field_id, u64 *data) { struct tdx_module_args args = {}; int ret; @@ -270,6 +270,7 @@ static int read_sys_metadata_field(u64 field_id, u64 *data) return 0; } +EXPORT_SYMBOL_GPL(tdx_sys_metadata_field_read); /* Return the metadata field element size in bytes */ static int get_metadata_field_bytes(u64 field_id) @@ -295,7 +296,7 @@ static int stbuf_read_sys_metadata_field(u64 field_id, if (WARN_ON_ONCE(get_metadata_field_bytes(field_id) != bytes)) return -EINVAL; - ret = read_sys_metadata_field(field_id, &tmp); + ret = tdx_sys_metadata_field_read(field_id, &tmp); if (ret) return ret; @@ -304,19 +305,8 @@ static int stbuf_read_sys_metadata_field(u64 field_id, return 0; } -struct field_mapping { - u64 field_id; - int offset; - int size; -}; - -#define TD_SYSINFO_MAP(_field_id, _struct, _member) \ - { .field_id = MD_FIELD_ID_##_field_id, \ - .offset = offsetof(_struct, _member), \ - .size = sizeof(typeof(((_struct *)0)->_member)) } - -static int read_sys_metadata(struct field_mapping *fields, int nr_fields, - void *stbuf) +int tdx_sys_metadata_read(const struct tdx_metadata_field_mapping *fields, + int nr_fields, void *stbuf) { int i, ret; @@ -331,6 +321,7 @@ static int read_sys_metadata(struct field_mapping *fields, int nr_fields, return 0; } +EXPORT_SYMBOL_GPL(tdx_sys_metadata_read); #define TD_SYSINFO_MAP_TDMR_INFO(_field_id, _member) \ TD_SYSINFO_MAP(_field_id, struct tdx_tdmr_sysinfo, _member) @@ -338,7 +329,7 @@ static int read_sys_metadata(struct field_mapping *fields, int nr_fields, static int get_tdx_tdmr_sysinfo(struct tdx_tdmr_sysinfo *tdmr_sysinfo) { /* Map TD_SYSINFO fields into 'struct tdx_tdmr_sysinfo': */ - const struct field_mapping fields[] = { + const struct tdx_metadata_field_mapping fields[] = { TD_SYSINFO_MAP_TDMR_INFO(MAX_TDMRS, max_tdmrs), TD_SYSINFO_MAP_TDMR_INFO(MAX_RESERVED_PER_TDMR, max_reserved_per_tdmr), TD_SYSINFO_MAP_TDMR_INFO(PAMT_4K_ENTRY_SIZE, pamt_entry_size[TDX_PS_4K]), @@ -347,7 +338,7 @@ static int get_tdx_tdmr_sysinfo(struct tdx_tdmr_sysinfo *tdmr_sysinfo) }; /* Populate 'tdmr_sysinfo' fields using the mapping structure above: */ - return read_sys_metadata(fields, ARRAY_SIZE(fields), tdmr_sysinfo); + return tdx_sys_metadata_read(fields, ARRAY_SIZE(fields), tdmr_sysinfo); } /* Calculate the actual TDMR size */ From patchwork Mon Feb 26 08:25:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571434 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 770DD208AD; Mon, 26 Feb 2024 08:27:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.16 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936061; cv=none; b=TvKFBMS8gAanr5Loqh81RTnJcDCORJTSqNhR0TV9r76xkSpZVU9zJoypK6htLEQhA1+MHkG7T743oKJxsIowx+e3ebfRmnsOtaDX7MtWUWi4dIGV0+OrhcR5Cm5JaBVqME8ShgGQSJqVw+iuk7bESYT66gDa763Lh0VFBPvmS3M= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936061; c=relaxed/simple; bh=JAOPGYcP6NR00jGXljrBqWwP9YIKeIJlQyj51gG7x+s=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=AkJqxFpVph5xcN3ImwLvKNV82WOs+Nt4SUWQ2VSyNgLqP8V0JQC3+2OlUcaYipePCEdydc1Wifit2p5hqaHUC9OlWvhpZYKZUatGexQVNQTJ9ayOeacFH5vaKuPTNxGUX2pZsLnH/uRCpr4kLx8M8OyK5n6+k0+GSlaMD4/Py0g= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=JIn5nBfu; arc=none smtp.client-ip=198.175.65.16 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="JIn5nBfu" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936060; x=1740472060; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=JAOPGYcP6NR00jGXljrBqWwP9YIKeIJlQyj51gG7x+s=; b=JIn5nBfu5PewrwzahWS7hgWb5eb7LyKk1wWSMwEPl32jYCn05mbXtZ8d uEXeOQTlQatHAjgkWjC61e6R6HloEt4R0VfDUlFw2hvcWBajLHI8pazVt cP4PN0EGRZcVm/JZNrHmyPZCMBka/LGFsfIM5C3JMtLdmjJrH6E+Paxhx ZI1EugzO1HqqnCIwPGJC1M+/KccOZHKThG7ydXtls0tfn9svDKspr2OFi HWsit3nKZbWu0FzCDzeK4CW4dTNIS2cOeeIwtx5eBtrmwGYl18Qe77gRP umG3PdJup3m2IZrDdadjJ0w+syN2076bTFVGkOR+ZN0yGybh3udmaNHro A==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="3340716" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="3340716" Received: from orviesa006.jf.intel.com ([10.64.159.146]) by orvoesa108.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:27:39 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="7020085" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa006-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:27:38 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 006/130] x86/virt/tdx: Export TDX KeyID information Date: Mon, 26 Feb 2024 00:25:08 -0800 Message-Id: <0e77a714e4ee2936218bf7a0ae6811f7bae27432.1708933498.git.isaku.yamahata@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Kai Huang Each TDX guest must be protected by its own unique TDX KeyID. KVM will need to tell the TDX module the unique KeyID for a TDX guest when KVM creates it. Export the TDX KeyID range that can be used by TDX guests for KVM to use. KVM can then manage these KeyIDs and assign one for each TDX guest when it is created. Each TDX guest has a root control structure called "Trust Domain Root" (TDR). Unlike the rest of the TDX guest, the TDR is protected by the TDX global KeyID. When tearing down the TDR, KVM will need to pass the TDX global KeyID explicitly to the TDX module to flush cache associated to the TDR. Also export the TDX global KeyID for KVM to tear down the TDR. Signed-off-by: Kai Huang Signed-off-by: Isaku Yamahata --- arch/x86/include/asm/tdx.h | 5 +++++ arch/x86/virt/vmx/tdx/tdx.c | 11 ++++++++--- 2 files changed, 13 insertions(+), 3 deletions(-) diff --git a/arch/x86/include/asm/tdx.h b/arch/x86/include/asm/tdx.h index 709b9483f9e4..16be3a1e4916 100644 --- a/arch/x86/include/asm/tdx.h +++ b/arch/x86/include/asm/tdx.h @@ -88,6 +88,11 @@ static inline long tdx_kvm_hypercall(unsigned int nr, unsigned long p1, #endif /* CONFIG_INTEL_TDX_GUEST && CONFIG_KVM_GUEST */ #ifdef CONFIG_INTEL_TDX_HOST + +extern u32 tdx_global_keyid; +extern u32 tdx_guest_keyid_start; +extern u32 tdx_nr_guest_keyids; + u64 __seamcall(u64 fn, struct tdx_module_args *args); u64 __seamcall_ret(u64 fn, struct tdx_module_args *args); u64 __seamcall_saved_ret(u64 fn, struct tdx_module_args *args); diff --git a/arch/x86/virt/vmx/tdx/tdx.c b/arch/x86/virt/vmx/tdx/tdx.c index dc21310776ab..d2b8f079a637 100644 --- a/arch/x86/virt/vmx/tdx/tdx.c +++ b/arch/x86/virt/vmx/tdx/tdx.c @@ -39,9 +39,14 @@ #include #include "tdx.h" -static u32 tdx_global_keyid __ro_after_init; -static u32 tdx_guest_keyid_start __ro_after_init; -static u32 tdx_nr_guest_keyids __ro_after_init; +u32 tdx_global_keyid __ro_after_init; +EXPORT_SYMBOL_GPL(tdx_global_keyid); + +u32 tdx_guest_keyid_start __ro_after_init; +EXPORT_SYMBOL_GPL(tdx_guest_keyid_start); + +u32 tdx_nr_guest_keyids __ro_after_init; +EXPORT_SYMBOL_GPL(tdx_nr_guest_keyids); static DEFINE_PER_CPU(bool, tdx_lp_initialized); From patchwork Mon Feb 26 08:25:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571435 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 064DA219FD; Mon, 26 Feb 2024 08:27:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.7 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936062; cv=none; b=QFT12y6QG2nHYsGQhNv/fbiwqqSQA/1mR4aq+lXeyjfSVOFE8aAZGIpOlR9YgHWUBukXKA7k1NYqn+Y//A0GiRlhZ5KxuWdFwQKneCvooJq5jvJIU8apStEMHGcvPFEMqglthbtJGPXFjrip6Athm1gHwO9+nkpW/FlsS5acDIg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936062; c=relaxed/simple; bh=jlfMA54rdGG+IPOKGOPLVKDQtiL3/PSTjptpBZUxEnw=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=N2DpACiIFUnOrKHlPyLMt2r1o+1vtoi62te3UUrVY7L5kukc0krhHyMBnYAqxVqyx5LIQWNilUkoK42/APWXbyL1/yM61XqZevPtWUwScA4hk7b6MHarnrEgN+TifgttvPjND1rQSu/vy5XqHRLOlC4XRSQ6sq/zcVQFVu4EpV4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=VwLS2EQl; arc=none smtp.client-ip=192.198.163.7 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="VwLS2EQl" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936060; x=1740472060; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=jlfMA54rdGG+IPOKGOPLVKDQtiL3/PSTjptpBZUxEnw=; b=VwLS2EQlknHA1ShFnNsgAuvgmOM3dSUKMujoqfDPG1+ILG+vLryL5QXf aOkpbIG3vqUhTpqQwgszKeN7EoE6xGuyS3zZksIEpz4JWQQdu1T/p564I ooiqkIcWMs6wYlJ8rW9sQln++97hv34Eps4WB9Ys7afkQKG/VMuTOtrNe ssYa3kTCm5ydEyk99ywfAVo36P1I7At8I72tM61ecpoi+MHhEgNkkkmlJ RYQs2xRHsiFuiyCyvqaXs0vbYmJbYxx0gemOUrd2TVf4OruT2rIPoY9dQ +6TvoaG/WgpyY+pWTncVoI8uteUXc7V+1rS3RJ2U2lEcRon0JgRWZbX2W Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="28631452" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="28631452" Received: from fmviesa007.fm.intel.com ([10.60.135.147]) by fmvoesa101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:27:39 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6474307" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by fmviesa007-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:27:39 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 007/130] x86/virt/tdx: Export SEAMCALL functions Date: Mon, 26 Feb 2024 00:25:09 -0800 Message-Id: <8f64043a6c393c017347bf8954d92b84b58603ec.1708933498.git.isaku.yamahata@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Kai Huang KVM will need to make SEAMCALLs to create and run TDX guests. Export SEAMCALL functions for KVM to use. Also add declaration of SEAMCALL functions to to support CONFIG_MODVERSIONS=y. Signed-off-by: Kai Huang Signed-off-by: Isaku Yamahata --- arch/x86/include/asm/asm-prototypes.h | 1 + arch/x86/virt/vmx/tdx/seamcall.S | 4 ++++ 2 files changed, 5 insertions(+) diff --git a/arch/x86/include/asm/asm-prototypes.h b/arch/x86/include/asm/asm-prototypes.h index b1a98fa38828..0ec572ad75f1 100644 --- a/arch/x86/include/asm/asm-prototypes.h +++ b/arch/x86/include/asm/asm-prototypes.h @@ -13,6 +13,7 @@ #include #include #include +#include #ifndef CONFIG_X86_CMPXCHG64 extern void cmpxchg8b_emu(void); diff --git a/arch/x86/virt/vmx/tdx/seamcall.S b/arch/x86/virt/vmx/tdx/seamcall.S index 5b1f2286aea9..e32cf82ed47e 100644 --- a/arch/x86/virt/vmx/tdx/seamcall.S +++ b/arch/x86/virt/vmx/tdx/seamcall.S @@ -1,5 +1,6 @@ /* SPDX-License-Identifier: GPL-2.0 */ #include +#include #include #include "tdxcall.S" @@ -21,6 +22,7 @@ SYM_FUNC_START(__seamcall) TDX_MODULE_CALL host=1 SYM_FUNC_END(__seamcall) +EXPORT_SYMBOL_GPL(__seamcall); /* * __seamcall_ret() - Host-side interface functions to SEAM software @@ -40,6 +42,7 @@ SYM_FUNC_END(__seamcall) SYM_FUNC_START(__seamcall_ret) TDX_MODULE_CALL host=1 ret=1 SYM_FUNC_END(__seamcall_ret) +EXPORT_SYMBOL_GPL(__seamcall_ret); /* * __seamcall_saved_ret() - Host-side interface functions to SEAM software @@ -59,3 +62,4 @@ SYM_FUNC_END(__seamcall_ret) SYM_FUNC_START(__seamcall_saved_ret) TDX_MODULE_CALL host=1 ret=1 saved=1 SYM_FUNC_END(__seamcall_saved_ret) +EXPORT_SYMBOL_GPL(__seamcall_saved_ret); From patchwork Mon Feb 26 08:25:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571436 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3C2A72576D; Mon, 26 Feb 2024 08:27:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.7 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936063; cv=none; b=O+AE/ZpQ1xouLjRqgScAyXtpI20QTAl9//cEnfXY8Th+5vplYerD8GqjvQin4HFJ0JMpqKrKCGtv5/yfAt+rwXHs3UfDkMRsi58CMjOATvjwC1gnSDOeG5dRphW/Mc23XfSdWU0RqkC3fC7DyRvUy2mbV8AzReUB5P1GI28pn9c= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936063; c=relaxed/simple; bh=xIS0IEpU4jLV8PiHi+HchMYIeNnm0fFPNPHfNFUxmaM=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=UEhwOwvOd/UgI7Hs3zJ7AFzlYt/bM+/NJnzqRqJgFx0EiELstCuGiGtGTXNIL6Fn/fpSmDSt9B2SjkOS27lKvyG7etXX5qXR2plQ/R6dPuZZrQDrAj4Yk4FZlYyh65TuUoW0sEPamtiDQq2y4aoMk9eSdzjODDfqcQ5rW6ociD8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=WGwzpJXw; arc=none smtp.client-ip=192.198.163.7 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="WGwzpJXw" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936061; x=1740472061; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=xIS0IEpU4jLV8PiHi+HchMYIeNnm0fFPNPHfNFUxmaM=; b=WGwzpJXwqFrom4jDm3BFVTCqX7uiDXqoq3mIwbVakbW6l008gXz/RNix BjCZzENr5zbRYZSarOCuEeMpH2DMa70c66bMVCLEn8xReOwqwqo6xJHAF R9NUK9bug+GKGKhcT9xQNyUJZgj6QWQtONP91mCBUyaA9q4WqcP0T4bRq CwhXLzMyizBc+12weAZyofPl98hrEDGzrkld3wb2utTQrYbDsWFEXlUXo S4fUrNx8zIYnHXDiyJFl19/OJ8dGHuSQWQ8uvruMzWg0yw+/g78iHc0mX d4I9nzkgEaULD9vjkPmfnAVAkV151qVeAw3+TPff69H+5M5NejDwLSsU9 w==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="28631456" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="28631456" Received: from fmviesa007.fm.intel.com ([10.60.135.147]) by fmvoesa101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:27:40 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6474311" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by fmviesa007-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:27:39 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 008/130] x86/tdx: Warning with 32bit build shift-count-overflow Date: Mon, 26 Feb 2024 00:25:10 -0800 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata This patch fixes the following warnings. In file included from arch/x86/kernel/asm-offsets.c:22: arch/x86/include/asm/tdx.h:92:87: warning: shift count >= width of type [-Wshift-count-overflow] arch/x86/include/asm/tdx.h:20:21: note: expanded from macro 'TDX_ERROR' #define TDX_ERROR _BITUL(63) ^~~~~~~~~~ Also consistently use ULL for TDX_SEAMCALL_VMFAILINVALID. Fixes: 527a534c7326 ("x86/tdx: Provide common base for SEAMCALL and TDCALL C wrappers") Signed-off-by: Isaku Yamahata --- arch/x86/include/asm/tdx.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/arch/x86/include/asm/tdx.h b/arch/x86/include/asm/tdx.h index 16be3a1e4916..1e9dcdf9912b 100644 --- a/arch/x86/include/asm/tdx.h +++ b/arch/x86/include/asm/tdx.h @@ -17,9 +17,9 @@ * Bits 47:40 == 0xFF indicate Reserved status code class that never used by * TDX module. */ -#define TDX_ERROR _BITUL(63) +#define TDX_ERROR _BITULL(63) #define TDX_SW_ERROR (TDX_ERROR | GENMASK_ULL(47, 40)) -#define TDX_SEAMCALL_VMFAILINVALID (TDX_SW_ERROR | _UL(0xFFFF0000)) +#define TDX_SEAMCALL_VMFAILINVALID (TDX_SW_ERROR | _ULL(0xFFFF0000)) #define TDX_SEAMCALL_GP (TDX_SW_ERROR | X86_TRAP_GP) #define TDX_SEAMCALL_UD (TDX_SW_ERROR | X86_TRAP_UD) From patchwork Mon Feb 26 08:25:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571437 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4B2833399F; Mon, 26 Feb 2024 08:27:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.7 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936064; cv=none; b=LFqCZsoEDvOEruwIaPm7Q7+tbZXTjJdT97+0oVBGdyiDmOzg18Y2yvd0uTlW1Otl+6Naxm28HPCO45Filkbv93wBgZ3CMvr67SmWIkjS4i2HOZbLl0x1HCmPvdf393Yyw33a+QJfo2Yl0edqdaRguaestetLk5IooeQRhQW60u8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936064; c=relaxed/simple; bh=eTShvvspjk0PvYC7qMm7VTWj6uInBYa2cEf1cy02sbw=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=pqsRv15flTfQjO+II747ujtr2QSQaNh/5sZjXOylDgJEKbW2FXgi9zO0uWZkRI8S4lv1HodaI61+H4YpCdhaOGx5wMpE2HFyUR1cwH0OPOc6GFRcHxZes7bKgHCNIO+iVf4JQ68GMAZJyfRTiQTm0lGyIoDN/HRWqA6FqfiW+rc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=BNhxpvNl; arc=none smtp.client-ip=192.198.163.7 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="BNhxpvNl" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936062; x=1740472062; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=eTShvvspjk0PvYC7qMm7VTWj6uInBYa2cEf1cy02sbw=; b=BNhxpvNlISeVeAN8JOQRIlxodbDlIwBSPD+7CKQ5T2FMbOn0+u2SVKrx Ew1372MycZrT3Ax2cVSPDel1OYtCv6TVD954/A+nmuZEZqg1TQ3ZiPQm6 Gf202aKtVHkPscp3jNI7lIO1k4vIS1E3VaxRddiAmgHqGWzsD+yMKYV4l a09w/zeZ401Sq8weILT/ulNXcic796C1uaRF++vzCmn/RwozkeRdQ7Rx9 dmxuaCNMXueM7hi3qWGeKmemuFJm0XyZJ3Hv92UKt9kOnAJQAXaCtIKC3 Jbn+H32qMeOWNoiWlycfe8cUnDJCx3YVQ1U1P+YCe86r5alhMzl8YTJ8x Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="28631460" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="28631460" Received: from fmviesa007.fm.intel.com ([10.60.135.147]) by fmvoesa101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:27:40 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6474316" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by fmviesa007-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:27:39 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com, Michael Roth Subject: [PATCH v19 009/130] KVM: x86: Add gmem hook for determining max NPT mapping level Date: Mon, 26 Feb 2024 00:25:11 -0800 Message-Id: <7ef616293093a94809625432f7855aa7da958492.1708933498.git.isaku.yamahata@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Michael Roth In the case of SEV-SNP, whether or not a 2MB page can be mapped via a 2MB mapping in the guest's nested page table depends on whether or not any subpages within the range have already been initialized as private in the RMP table. The existing mixed-attribute tracking in KVM is insufficient here, for instance: - gmem allocates 2MB page - guest issues PVALIDATE on 2MB page - guest later converts a subpage to shared - SNP host code issues PSMASH to split 2MB RMP mapping to 4K - KVM MMU splits NPT mapping to 4K At this point there are no mixed attributes, and KVM would normally allow for 2MB NPT mappings again, but this is actually not allowed because the RMP table mappings are 4K and cannot be promoted on the hypervisor side, so the NPT mappings must still be limited to 4K to match this. Add a hook to determine the max NPT mapping size in situations like this. Signed-off-by: Michael Roth Link: https://lore.kernel.org/r/20231230172351.574091-31-michael.roth@amd.com Signed-off-by: Isaku Yamahata --- arch/x86/include/asm/kvm-x86-ops.h | 1 + arch/x86/include/asm/kvm_host.h | 2 ++ arch/x86/kvm/mmu/mmu.c | 12 ++++++++++-- 3 files changed, 13 insertions(+), 2 deletions(-) diff --git a/arch/x86/include/asm/kvm-x86-ops.h b/arch/x86/include/asm/kvm-x86-ops.h index 378ed944b849..156832f01ebe 100644 --- a/arch/x86/include/asm/kvm-x86-ops.h +++ b/arch/x86/include/asm/kvm-x86-ops.h @@ -138,6 +138,7 @@ KVM_X86_OP(complete_emulated_msr) KVM_X86_OP(vcpu_deliver_sipi_vector) KVM_X86_OP_OPTIONAL_RET0(vcpu_get_apicv_inhibit_reasons); KVM_X86_OP_OPTIONAL(get_untagged_addr) +KVM_X86_OP_OPTIONAL_RET0(gmem_max_level) #undef KVM_X86_OP #undef KVM_X86_OP_OPTIONAL diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index d271ba20a0b2..d15f5b4b1656 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1796,6 +1796,8 @@ struct kvm_x86_ops { unsigned long (*vcpu_get_apicv_inhibit_reasons)(struct kvm_vcpu *vcpu); gva_t (*get_untagged_addr)(struct kvm_vcpu *vcpu, gva_t gva, unsigned int flags); + + int (*gmem_max_level)(struct kvm *kvm, kvm_pfn_t pfn, gfn_t gfn, u8 *max_level); }; struct kvm_x86_nested_ops { diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 2d6cdeab1f8a..1e5e12d2707d 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4308,6 +4308,7 @@ static int kvm_faultin_pfn_private(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault) { int max_order, r; + u8 max_level; if (!kvm_slot_can_be_private(fault->slot)) { kvm_mmu_prepare_memory_fault_exit(vcpu, fault); @@ -4321,8 +4322,15 @@ static int kvm_faultin_pfn_private(struct kvm_vcpu *vcpu, return r; } - fault->max_level = min(kvm_max_level_for_order(max_order), - fault->max_level); + max_level = kvm_max_level_for_order(max_order); + r = static_call(kvm_x86_gmem_max_level)(vcpu->kvm, fault->pfn, + fault->gfn, &max_level); + if (r) { + kvm_release_pfn_clean(fault->pfn); + return r; + } + + fault->max_level = min(max_level, fault->max_level); fault->map_writable = !(fault->slot->flags & KVM_MEM_READONLY); return RET_PF_CONTINUE; From patchwork Mon Feb 26 08:25:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571438 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6971B38397; Mon, 26 Feb 2024 08:27:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.7 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936064; cv=none; b=pF5J3HEbuQPP9h7Z1CVwgogwrkdjufFplX5oDUtJPirTYnki3/iwu8F5X+Ug9mq2drsFoEL0HH3VlDwNUmLUWwF85/PsqD7Tv2KgyVmN3KJc9SEbu8qw/SPyHrrqKPztaU57QosBZpU+QVfHJno96ZpTfs69J85lTehxhJQp+Dc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936064; c=relaxed/simple; bh=Fyink57/ejQGrZ8tQRVicXDKz0HYEbCM9W9KphhTHVI=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=r+DAekreflbmQuWO8yoyLk1OGY6bXHq6LZU+FLepVVTXet2AuGjK7AzJFkiXUJw/dq2pbukz43+6HHRPHFynT24qUNoIYMOJsZmT+nXLuyxa26e9MqNYwEnBrV16UhHkjkMr07KNsVjS3oA0nycPVouZLLKEZ6ZtiuHfWmZu7lI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=V2gLD7oE; arc=none smtp.client-ip=192.198.163.7 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="V2gLD7oE" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936062; x=1740472062; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Fyink57/ejQGrZ8tQRVicXDKz0HYEbCM9W9KphhTHVI=; b=V2gLD7oE5oysIguiRSLUZOD6qMydjEB0hYWDSop+WMEMQdy66UUyj3Og Ddvy+dJpwOQfb1TPxHWE9G+HL+SJPqsvlvD5eFYnvJnNCFIPtALV663wF 6+pgklIv5Cbu6/TEo1e6OLGIT2rGzyr5yu6OZYwE/cU5V5Pwkjre8tpoC u1BmttzhJkmVGgP7524WF1N0LN4n2qtaZdDX39VLgZfd0OwKzZKnxI3zP crLkVGhKhHysOYXYjriP37u8j8KW6aAzy/GsVMXkVb0GTAL3YnZLZmwhu TSl7E/CYjvN8lrYWjVT8QGKOwrzZYH+J4U4IM8c+DTzxoxKBCDF9OOQay Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="28631466" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="28631466" Received: from fmviesa007.fm.intel.com ([10.60.135.147]) by fmvoesa101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:27:40 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6474321" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by fmviesa007-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:27:40 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 010/130] KVM: x86: Pass is_private to gmem hook of gmem_max_level Date: Mon, 26 Feb 2024 00:25:12 -0800 Message-Id: <8108cc2b8ff01ec22de68f0d0758ef0671db43fc.1708933498.git.isaku.yamahata@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata TDX wants to know the faulting address is shared or private so that the max level is limited by Secure-EPT or not. Because fault->gfn doesn't include shared bit, gfn doesn't tell if the faulting address is shared or not. Pass is_private for TDX case. TDX logic will be if (!is_private) return 0; else return PG_LEVEL_4K. Signed-off-by: Isaku Yamahata --- arch/x86/include/asm/kvm_host.h | 3 ++- arch/x86/kvm/mmu/mmu.c | 3 ++- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index d15f5b4b1656..57ce89fc2740 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1797,7 +1797,8 @@ struct kvm_x86_ops { gva_t (*get_untagged_addr)(struct kvm_vcpu *vcpu, gva_t gva, unsigned int flags); - int (*gmem_max_level)(struct kvm *kvm, kvm_pfn_t pfn, gfn_t gfn, u8 *max_level); + int (*gmem_max_level)(struct kvm *kvm, kvm_pfn_t pfn, gfn_t gfn, + bool is_private, u8 *max_level); }; struct kvm_x86_nested_ops { diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 1e5e12d2707d..22db1a9f528a 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4324,7 +4324,8 @@ static int kvm_faultin_pfn_private(struct kvm_vcpu *vcpu, max_level = kvm_max_level_for_order(max_order); r = static_call(kvm_x86_gmem_max_level)(vcpu->kvm, fault->pfn, - fault->gfn, &max_level); + fault->gfn, fault->is_private, + &max_level); if (r) { kvm_release_pfn_clean(fault->pfn); return r; From patchwork Mon Feb 26 08:25:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571439 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7BE5A47A76; Mon, 26 Feb 2024 08:27:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.7 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936065; cv=none; b=Wz3RT4olsqdH1Y/11c/B77RNtTCWvsyB6Nqfv/QSUbRjGpLTvLt0m0by0/BXoKggd9OsV1TCTrMUjzu5ye2NXo1uJtl7zmu98uREew/MCcxM1YMYcc1OgtQ2O32huOwLEjUgvP7Qmnac6lY0SyfJSLczghWE85rjWv6pyD8uyoY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936065; c=relaxed/simple; bh=UFjZ9FiwgdrptLt8Ffkz8jHayts9vWGjhO2Osf/SglA=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=iQHIuGJPKICl4XmL77nAvHeZ5yx7ZZY1rlt+0PC7lr2dLBugyvZndjAbi0vSQ4UOiWB5lnFlG0rEGmlQB3hHOX2lH5vU9DJBuqiMVofA33LdPb5EsY2FVkgO7lXMOeqsCvXoYJgiMlZhQYcNDZb/2NiHWxnA5VDRYBGIQsyIrNQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=kwXRcev7; arc=none smtp.client-ip=192.198.163.7 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="kwXRcev7" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936063; x=1740472063; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=UFjZ9FiwgdrptLt8Ffkz8jHayts9vWGjhO2Osf/SglA=; b=kwXRcev7ZOZJFKoN8FDhMXJQ0YapgcuJ95znKjW0+7Y5sNk362koTeut VVBBBiFqZayI+Safg9VuhuhD53EtecmpGmq6EXgoi/ORyUhnxguatn6nU dYUv1/TGiCiUXCPphlyKvba1OOcwjfd5MdGFVjg1JHJo7dRsewlU5dh0b IiEtdzkf87kR6SYIAOb46xWxnvAnWNmX9btuFMSQA7YaWNY4456IRPamJ ZgVtMZzMlLWTB+simjjENvdTEYaY0G/aQIN42ItwwcWZxB7HTHhDFzdA5 Bhrx166ycI0lQuv4jiVIH6eVJhwaygHnIactwyc/dG/8KOGcRLta/SW73 w==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="28631470" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="28631470" Received: from fmviesa007.fm.intel.com ([10.60.135.147]) by fmvoesa101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:27:41 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6474327" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by fmviesa007-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:27:40 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 011/130] KVM: Add new members to struct kvm_gfn_range to operate on Date: Mon, 26 Feb 2024 00:25:13 -0800 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata Add new members to strut kvm_gfn_range to indicate which mapping (private-vs-shared) to operate on. only_private and only_shared. Update mmu notifier, set memory attributes ioctl or KVM gmem callback to initialize them. It was premature for set_memory_attributes ioctl to call kvm_unmap_gfn_range(). Instead, let kvm_arch_ste_memory_attributes() handle it and add a new x86 vendor callback to react to memory attribute change. [1] - If it's from the mmu notifier, zap shared pages only - If it's from the KVM gmem, zap private pages only - If setting memory attributes, vendor callback checks new attributes and make decisions. SNP would do nothing and handle it later with gmem callback TDX callback would do as follows. When it converts pages to shared, zap private pages only. When it converts pages to private, zap shared pages only. TDX needs to know which mapping to operate on. Shared-EPT vs. Secure-EPT. The following sequence to convert the GPA to private doesn't work for TDX because the page can already be private. 1) Update memory attributes to private in memory attributes xarray 2) Zap the GPA range irrespective of private-or-shared. Even if the page is already private, zap the entry. 3) EPT violation on the GPA 4) Populate the GPA as private The page is zeroed, and the guest has to accept the page again. In step 2, TDX wants to zap only shared pages and skip private ones. [1] https://lore.kernel.org/all/ZJX0hk+KpQP0KUyB@google.com/ Suggested-by: Sean Christopherson Signed-off-by: Isaku Yamahata --- Changes v18: - rebased to kvm-next Changes v2 -> v3: - Drop the KVM_GFN_RANGE flags - Updated struct kvm_gfn_range - Change kvm_arch_set_memory_attributes() to return bool for flush - Added set_memory_attributes x86 op for vendor backends - Refined commit message to describe TDX care concretely Changes v1 -> v2: - consolidate KVM_GFN_RANGE_FLAGS_GMEM_{PUNCH_HOLE, RELEASE} into KVM_GFN_RANGE_FLAGS_GMEM. - Update the commit message to describe TDX more. Drop SEV_SNP. Signed-off-by: Isaku Yamahata --- include/linux/kvm_host.h | 2 ++ virt/kvm/guest_memfd.c | 3 +++ virt/kvm/kvm_main.c | 17 +++++++++++++++++ 3 files changed, 22 insertions(+) diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 7e7fd25b09b3..0520cd8d03cc 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -264,6 +264,8 @@ struct kvm_gfn_range { gfn_t start; gfn_t end; union kvm_mmu_notifier_arg arg; + bool only_private; + bool only_shared; bool may_block; }; bool kvm_unmap_gfn_range(struct kvm *kvm, struct kvm_gfn_range *range); diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c index 0f4e0cf4f158..3830d50b9b67 100644 --- a/virt/kvm/guest_memfd.c +++ b/virt/kvm/guest_memfd.c @@ -64,6 +64,9 @@ static void kvm_gmem_invalidate_begin(struct kvm_gmem *gmem, pgoff_t start, .end = slot->base_gfn + min(pgoff + slot->npages, end) - pgoff, .slot = slot, .may_block = true, + /* guest memfd is relevant to only private mappings. */ + .only_private = true, + .only_shared = false, }; if (!found_memslot) { diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 10bfc88a69f7..0349e1f241d1 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -634,6 +634,12 @@ static __always_inline kvm_mn_ret_t __kvm_handle_hva_range(struct kvm *kvm, */ gfn_range.arg = range->arg; gfn_range.may_block = range->may_block; + /* + * HVA-based notifications aren't relevant to private + * mappings as they don't have a userspace mapping. + */ + gfn_range.only_private = false; + gfn_range.only_shared = true; /* * {gfn(page) | page intersects with [hva_start, hva_end)} = @@ -2486,6 +2492,16 @@ static __always_inline void kvm_handle_gfn_range(struct kvm *kvm, gfn_range.arg = range->arg; gfn_range.may_block = range->may_block; + /* + * If/when KVM supports more attributes beyond private .vs shared, this + * _could_ set only_{private,shared} appropriately if the entire target + * range already has the desired private vs. shared state (it's unclear + * if that is a net win). For now, KVM reaches this point if and only + * if the private flag is being toggled, i.e. all mappings are in play. + */ + gfn_range.only_private = false; + gfn_range.only_shared = false; + for (i = 0; i < kvm_arch_nr_memslot_as_ids(kvm); i++) { slots = __kvm_memslots(kvm, i); @@ -2542,6 +2558,7 @@ static int kvm_vm_set_mem_attributes(struct kvm *kvm, gfn_t start, gfn_t end, struct kvm_mmu_notifier_range pre_set_range = { .start = start, .end = end, + .arg.attributes = attributes, .handler = kvm_pre_set_memory_attributes, .on_lock = kvm_mmu_invalidate_begin, .flush_on_ret = true, From patchwork Mon Feb 26 08:25:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571440 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id ADFD84D13F; Mon, 26 Feb 2024 08:27:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.7 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936067; cv=none; b=cuMTIgxm7wjTuKgTCtbHuZm60Bsszl/ydhcpsvxh9urWlQgczLEpU4N4rwQfxKSoMRXSYWPpDBYZucB/YGuIVY1kWE/WaN/+cj2plOcBSy9tehfnQgipO9Avdm+fNAZWTyj6dcvdY2ZFvcjn0ww6QC74KVPwHGWv1zTntUeQyb0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936067; c=relaxed/simple; bh=fCJNKDBsJ1N8mTGUepr1GXYGBkdZ1JmNWyjOeM03/hk=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=sRGfqMWeQUkNJCedbFiYZFPhT3AUfYBNxYHX40KvnCqsco9QZv97C4rKOcTcvZH/SssHfkCY5O/99C2uakwYGNNQxh4DrI2tVDuPjwpuez8IG1N/N+4o8VY923an00glBQbwFSl1yn9dhRVinvyWKYt8s3kochxC6GQEZHosQuE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=kmACF3An; arc=none smtp.client-ip=192.198.163.7 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="kmACF3An" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936065; x=1740472065; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=fCJNKDBsJ1N8mTGUepr1GXYGBkdZ1JmNWyjOeM03/hk=; b=kmACF3AnTgRi2uLqSma9rlfPeecxSZEdpKIr6Y3+QAxexDBsPWfDp2BT at8yfjlsNGCvCvISSbM38LUSsC5J6Uxyn0rTQ3gWw6ff4yzd+DbfEpNaD XMUoy+LA/g7gg0GO2Gvw+QT/TkHcFeo8iX4QZYSGrPo8GMhWXyAP7q4t6 siIkQ9gcrdhLriROr/o7l11Zxia6aFAtlr3DPMecJbx0ZgL+etdT0MTxW EQGnZVe+4cyYyvLcKGCKmuX37Y7db7k8Qs8+Jx1uueEKwrGtqMcMauS8o 9ZsK4H5JYbFcsLjvRoRa+1X5GzX3PseAmUzjh6HlaNKCnJvsObDClKJjn g==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="28631474" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="28631474" Received: from fmviesa007.fm.intel.com ([10.60.135.147]) by fmvoesa101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:27:41 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6474330" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by fmviesa007-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:27:41 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 012/130] KVM: x86/mmu: Pass around full 64-bit error code for the KVM page fault Date: Mon, 26 Feb 2024 00:25:14 -0800 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata Because the full 64-bit error code, or more info about the fault, for the KVM page fault will be needed for protected VM, TDX and SEV-SNP, update kvm_mmu_do_page_fault() to accept the 64-bit value so it can pass it to the callbacks. The upper 32 bits of error code are discarded at kvm_mmu_page_fault() by lower_32_bits(). Now it's passed down as full 64 bits. Currently two hardware defined bits, PFERR_GUEST_FINAL_MASK and PFERR_GUEST_PAGE_MASK, and one software defined bit, PFERR_IMPLICIT_ACCESS, is defined. PFERR_IMPLICIT_ACCESS: commit 4f4aa80e3b88 ("KVM: X86: Handle implicit supervisor access with SMAP") introduced a software defined bit PFERR_IMPLICIT_ACCESS at bit 48 to indicate implicit access for SMAP with instruction emulator. Concretely emulator_read_std() and emulator_write_std() set the bit. permission_fault() checks the bit as smap implicit access. The vendor page fault handler shouldn't pass the bit to kvm_mmu_page_fault(). PFERR_GUEST_FINAL_MASK and PFERR_GUEST_PAGE_MASK: commit 147277540bbc ("kvm: svm: Add support for additional SVM NPF error codes") introduced them to optimize the nested page fault handling. Other code path doesn't use the bits. Those two bits can be safely passed down without functionality change. The accesses of fault->error_code are as follows - FNAME(page_fault): PFERR_IMPLICIT_ACCESS shouldn't be passed down. PFERR_GUEST_FINAL_MASK and PFERR_GUEST_PAGE_MASK aren't used. - kvm_mmu_page_fault(): explicit mask with PFERR_RSVD_MASK, and PFERR_NESTED_GUEST_PAGE is used outside of the masking upper 32 bits. - mmutrace: change u32 -> u64 No functional change is intended. This is a preparation to pass on more info with page fault error code. Signed-off-by: Isaku Yamahata --- Changes v3 -> v4: - Dropped debug print part as it was deleted in the kvm-x86-next Changes v2 -> v3: - Make depends on a patch to clear PFERR_IMPLICIT_ACCESS - drop clearing the upper 32 bit, instead just pass whole 64 bits - update commit message to mention about PFERR_IMPLICIT_ACCESS and PFERR_NESTED_GUEST_PAGE Changes v1 -> v2: - no change Signed-off-by: Isaku Yamahata --- arch/x86/kvm/mmu/mmu.c | 3 +-- arch/x86/kvm/mmu/mmu_internal.h | 4 ++-- arch/x86/kvm/mmu/mmutrace.h | 2 +- 3 files changed, 4 insertions(+), 5 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 22db1a9f528a..ccdbff3d85ec 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -5822,8 +5822,7 @@ int noinline kvm_mmu_page_fault(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa, u64 err } if (r == RET_PF_INVALID) { - r = kvm_mmu_do_page_fault(vcpu, cr2_or_gpa, - lower_32_bits(error_code), false, + r = kvm_mmu_do_page_fault(vcpu, cr2_or_gpa, error_code, false, &emulation_type); if (KVM_BUG_ON(r == RET_PF_INVALID, vcpu->kvm)) return -EIO; diff --git a/arch/x86/kvm/mmu/mmu_internal.h b/arch/x86/kvm/mmu/mmu_internal.h index 0669a8a668ca..21f55e8b4dc6 100644 --- a/arch/x86/kvm/mmu/mmu_internal.h +++ b/arch/x86/kvm/mmu/mmu_internal.h @@ -190,7 +190,7 @@ static inline bool is_nx_huge_page_enabled(struct kvm *kvm) struct kvm_page_fault { /* arguments to kvm_mmu_do_page_fault. */ const gpa_t addr; - const u32 error_code; + const u64 error_code; const bool prefetch; /* Derived from error_code. */ @@ -280,7 +280,7 @@ enum { }; static inline int kvm_mmu_do_page_fault(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa, - u32 err, bool prefetch, int *emulation_type) + u64 err, bool prefetch, int *emulation_type) { struct kvm_page_fault fault = { .addr = cr2_or_gpa, diff --git a/arch/x86/kvm/mmu/mmutrace.h b/arch/x86/kvm/mmu/mmutrace.h index ae86820cef69..195d98bc8de8 100644 --- a/arch/x86/kvm/mmu/mmutrace.h +++ b/arch/x86/kvm/mmu/mmutrace.h @@ -260,7 +260,7 @@ TRACE_EVENT( TP_STRUCT__entry( __field(int, vcpu_id) __field(gpa_t, cr2_or_gpa) - __field(u32, error_code) + __field(u64, error_code) __field(u64 *, sptep) __field(u64, old_spte) __field(u64, new_spte) From patchwork Mon Feb 26 08:25:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571441 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DD9B34D5AC; Mon, 26 Feb 2024 08:27:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.7 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936067; cv=none; b=TZGovrxgZOPpOv6kVgjPiLzyxBESOY+c9k33t96o16g7e9eNXR94JPVR3c+4HYBXvi6xFI8PzQJXuwB/RkeTOwbqccobs2llSK6oYvXw1McWvNrDYttEdIuFmaO6+GbpG2nBJ/b2oqlhC9lj5nUPPaIPxa2hhLIqFYic8UE3qUE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936067; c=relaxed/simple; bh=Awuik1MUAYsAnCDFjrgL7JE4Wn0+9r53tlCToVb0+AQ=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=lGwLZK92Q6brMjmv9i4DlwaOc6f0Gu0vdYkgdU+SsHdVoNiY/wg4Da9exw0BRvCdI7vkXwCj1LnayJ5vTF6SzTR0QAyWNJxZWTyH/upjynFQhgEJ508cnjF+8nNNUZDTPpDVQxW54tBTCRcfOqHMaStdHmPRnGg+TGyFYY26Bfk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=eAoVIEZx; arc=none smtp.client-ip=192.198.163.7 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="eAoVIEZx" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936065; x=1740472065; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Awuik1MUAYsAnCDFjrgL7JE4Wn0+9r53tlCToVb0+AQ=; b=eAoVIEZxkRvjJIiWiAjOYqVVD3dbQan1SyYsXU/oayX73Q7UkJCaqowq w+MQh7ugGL7JQrCjHlNDxa4K7pUDD1IRKvx1AySEi2MxHLKF9IsLcSv7r bDPNbjaOJoWJjpuzcTDHgmqUQuCFYNxA5rGQKIdwCHaQrm82tr6ndXGA/ Ypc9pWeRcOm8EGtoNW7zj+s5/KYWMeHWYG4HNaRc7MXFyNaphDSqBk458 /3eWQsZNXh2iZ9XBacwmX3c9KGIycHiigo/hh+CGjnQ5NDhhNvLplFGEv Sde9fvbECUJAwNksMpp2gYCmnSJOw3BBIEqySnsq+Dcc/Lu12ooDxLdrI A==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="28631479" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="28631479" Received: from fmviesa007.fm.intel.com ([10.60.135.147]) by fmvoesa101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:27:42 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6474333" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by fmviesa007-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:27:42 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 013/130] KVM: x86: Use PFERR_GUEST_ENC_MASK to indicate fault is private Date: Mon, 26 Feb 2024 00:25:15 -0800 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata SEV-SNP defines PFERR_GUEST_ENC_MASK (bit 32) in page-fault error bits to represent the guest page is encrypted. Use the bit to designate that the page fault is private and that it requires looking up memory attributes. The vendor kvm page fault handler should set PFERR_GUEST_ENC_MASK bit based on their fault information. It may or may not use the hardware value directly or parse the hardware value to set the bit. For KVM_X86_SW_PROTECTED_VM, ask memory attributes for the fault privateness. For async page fault, carry the bit and use it for kvm page fault handler. Signed-off-by: Isaku Yamahata --- Changes v4 -> v5: - Eliminate kvm_is_fault_private() by open code the function - Make async page fault handler to carry is_private bit Changes v3 -> v4: - rename back struct kvm_page_fault::private => is_private - catch up rename: KVM_X86_PROTECTED_VM => KVM_X86_SW_PROTECTED_VM Changes v2 -> v3: - Revive PFERR_GUEST_ENC_MASK - rename struct kvm_page_fault::is_private => private - Add check KVM_X86_PROTECTED_VM Changes v1 -> v2: - Introduced fault type and replaced is_private with fault_type. - Add kvm_get_fault_type() to encapsulate the difference. Signed-off-by: Isaku Yamahata --- arch/x86/include/asm/kvm_host.h | 3 +++ arch/x86/kvm/mmu/mmu.c | 24 +++++++++++++++++------- arch/x86/kvm/mmu/mmu_internal.h | 2 +- 3 files changed, 21 insertions(+), 8 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 57ce89fc2740..28314e7d546c 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -264,6 +264,7 @@ enum x86_intercept_stage; #define PFERR_SGX_BIT 15 #define PFERR_GUEST_FINAL_BIT 32 #define PFERR_GUEST_PAGE_BIT 33 +#define PFERR_GUEST_ENC_BIT 34 #define PFERR_IMPLICIT_ACCESS_BIT 48 #define PFERR_PRESENT_MASK BIT(PFERR_PRESENT_BIT) @@ -275,6 +276,7 @@ enum x86_intercept_stage; #define PFERR_SGX_MASK BIT(PFERR_SGX_BIT) #define PFERR_GUEST_FINAL_MASK BIT_ULL(PFERR_GUEST_FINAL_BIT) #define PFERR_GUEST_PAGE_MASK BIT_ULL(PFERR_GUEST_PAGE_BIT) +#define PFERR_GUEST_ENC_MASK BIT_ULL(PFERR_GUEST_ENC_BIT) #define PFERR_IMPLICIT_ACCESS BIT_ULL(PFERR_IMPLICIT_ACCESS_BIT) #define PFERR_NESTED_GUEST_PAGE (PFERR_GUEST_PAGE_MASK | \ @@ -1836,6 +1838,7 @@ struct kvm_arch_async_pf { gfn_t gfn; unsigned long cr3; bool direct_map; + u64 error_code; }; extern u32 __read_mostly kvm_nr_uret_msrs; diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index ccdbff3d85ec..61674d6b17aa 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4246,18 +4246,19 @@ static u32 alloc_apf_token(struct kvm_vcpu *vcpu) return (vcpu->arch.apf.id++ << 12) | vcpu->vcpu_id; } -static bool kvm_arch_setup_async_pf(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa, - gfn_t gfn) +static bool kvm_arch_setup_async_pf(struct kvm_vcpu *vcpu, + struct kvm_page_fault *fault) { struct kvm_arch_async_pf arch; arch.token = alloc_apf_token(vcpu); - arch.gfn = gfn; + arch.gfn = fault->gfn; arch.direct_map = vcpu->arch.mmu->root_role.direct; arch.cr3 = kvm_mmu_get_guest_pgd(vcpu, vcpu->arch.mmu); + arch.error_code = fault->error_code & PFERR_GUEST_ENC_MASK; - return kvm_setup_async_pf(vcpu, cr2_or_gpa, - kvm_vcpu_gfn_to_hva(vcpu, gfn), &arch); + return kvm_setup_async_pf(vcpu, fault->addr, + kvm_vcpu_gfn_to_hva(vcpu, fault->gfn), &arch); } void kvm_arch_async_page_ready(struct kvm_vcpu *vcpu, struct kvm_async_pf *work) @@ -4276,7 +4277,8 @@ void kvm_arch_async_page_ready(struct kvm_vcpu *vcpu, struct kvm_async_pf *work) work->arch.cr3 != kvm_mmu_get_guest_pgd(vcpu, vcpu->arch.mmu)) return; - kvm_mmu_do_page_fault(vcpu, work->cr2_or_gpa, 0, true, NULL); + kvm_mmu_do_page_fault(vcpu, work->cr2_or_gpa, work->arch.error_code, + true, NULL); } static inline u8 kvm_max_level_for_order(int order) @@ -4390,7 +4392,7 @@ static int __kvm_faultin_pfn(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault trace_kvm_async_pf_repeated_fault(fault->addr, fault->gfn); kvm_make_request(KVM_REQ_APF_HALT, vcpu); return RET_PF_RETRY; - } else if (kvm_arch_setup_async_pf(vcpu, fault->addr, fault->gfn)) { + } else if (kvm_arch_setup_async_pf(vcpu, fault)) { return RET_PF_RETRY; } } @@ -5814,6 +5816,14 @@ int noinline kvm_mmu_page_fault(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa, u64 err if (WARN_ON_ONCE(!VALID_PAGE(vcpu->arch.mmu->root.hpa))) return RET_PF_RETRY; + /* + * This is racy with updating memory attributes with mmu_seq. If we + * hit a race, it would result in retrying page fault. + */ + if (vcpu->kvm->arch.vm_type == KVM_X86_SW_PROTECTED_VM && + kvm_mem_is_private(vcpu->kvm, gpa_to_gfn(cr2_or_gpa))) + error_code |= PFERR_GUEST_ENC_MASK; + r = RET_PF_INVALID; if (unlikely(error_code & PFERR_RSVD_MASK)) { r = handle_mmio_page_fault(vcpu, cr2_or_gpa, direct); diff --git a/arch/x86/kvm/mmu/mmu_internal.h b/arch/x86/kvm/mmu/mmu_internal.h index 21f55e8b4dc6..0443bfcf5d9c 100644 --- a/arch/x86/kvm/mmu/mmu_internal.h +++ b/arch/x86/kvm/mmu/mmu_internal.h @@ -292,13 +292,13 @@ static inline int kvm_mmu_do_page_fault(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa, .user = err & PFERR_USER_MASK, .prefetch = prefetch, .is_tdp = likely(vcpu->arch.mmu->page_fault == kvm_tdp_page_fault), + .is_private = err & PFERR_GUEST_ENC_MASK, .nx_huge_page_workaround_enabled = is_nx_huge_page_enabled(vcpu->kvm), .max_level = KVM_MAX_HUGEPAGE_LEVEL, .req_level = PG_LEVEL_4K, .goal_level = PG_LEVEL_4K, - .is_private = kvm_mem_is_private(vcpu->kvm, cr2_or_gpa >> PAGE_SHIFT), }; int r; From patchwork Mon Feb 26 08:25:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571442 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0C8094EB44; Mon, 26 Feb 2024 08:27:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.7 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936068; cv=none; b=VsML1MoBy9enias2gfd/IjpZH7nSXT3z23KlfTU2RlolotPDSRWC9o8USP6AWvmM09BRQi19oT7eMWYpebM8PtOUgcOYictKBGpHxC77WIFBolmT+r9J4vPsl2s5XjYDBFSFVxc8aLlUSaGtI4Y/5HgK5O3HGE8dbtSSB1j69mw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936068; c=relaxed/simple; bh=M0yJ/NQ+t9AuoodsI2KxRlq7XjG9s/CzpOfeudWoVdA=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=FgzyjHfi1rwO3GH9RYQTgVQ9VjYT5h6TwttAbErwBW4rU1O/vfWvsYECbpN17cqN+zo9tAXcWvTngE06Z/j0kshK+WBwXqd1tvuG/QyBCZq40b2hUgsY+QIhro7a8lb3g3WGUrFcxwzFJOM2yIuMZFnWGuVFEI8kJlVvMDLJUWs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=mVKnxvyW; arc=none smtp.client-ip=192.198.163.7 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="mVKnxvyW" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936066; x=1740472066; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=M0yJ/NQ+t9AuoodsI2KxRlq7XjG9s/CzpOfeudWoVdA=; b=mVKnxvyWLyCTuYqw6QzSOlsLS4RJcLMSBEMRkMLagLkUBLdbJr87FoZB cH/+NsbqdjS3IpYfraICpHRg0s61YJbACYaof/lIv8NrzNGwvHz0X0GQL OdsGInhlC/jT6lby7Qi2GWoYJWj9gNFsGZeGuAtf4fEXX8FI8GVJbfbIM t87czq2tcqvbJR7GW9K/k8D51JKy6QoeoOnuhr/oAOfNUmhHL8WpQY5hP 0JUDzH9p86hUBbYJp6RRYpL/b7SdOvEY/QRNAAPBZ3mRoNLjvkv0WpysU M2IzyQ+08BJMPcEKwTlY7RSqtyZUivjazlF9kKAQyzI5K6eJ0rNzJRLy+ A==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="28631485" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="28631485" Received: from fmviesa007.fm.intel.com ([10.60.135.147]) by fmvoesa101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:27:42 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6474338" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by fmviesa007-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:27:42 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 014/130] KVM: Add KVM vcpu ioctl to pre-populate guest memory Date: Mon, 26 Feb 2024 00:25:16 -0800 Message-Id: <8b7380f1b02f8e3995f18bebb085e43165d5d682.1708933498.git.isaku.yamahata@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata Add new ioctl KVM_MEMORY_MAPPING in the kvm common code. It iterates on the memory range and call arch specific function. Add stub function as weak symbol. Signed-off-by: Isaku Yamahata --- v19: - newly added --- include/linux/kvm_host.h | 4 +++ include/uapi/linux/kvm.h | 10 ++++++ virt/kvm/kvm_main.c | 67 ++++++++++++++++++++++++++++++++++++++++ 3 files changed, 81 insertions(+) diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 0520cd8d03cc..eeaf4e73317c 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -2389,4 +2389,8 @@ static inline int kvm_gmem_get_pfn(struct kvm *kvm, } #endif /* CONFIG_KVM_PRIVATE_MEM */ +void kvm_arch_vcpu_pre_memory_mapping(struct kvm_vcpu *vcpu); +int kvm_arch_vcpu_memory_mapping(struct kvm_vcpu *vcpu, + struct kvm_memory_mapping *mapping); + #endif diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h index c3308536482b..5e2b28934aa9 100644 --- a/include/uapi/linux/kvm.h +++ b/include/uapi/linux/kvm.h @@ -1155,6 +1155,7 @@ struct kvm_ppc_resize_hpt { #define KVM_CAP_MEMORY_ATTRIBUTES 233 #define KVM_CAP_GUEST_MEMFD 234 #define KVM_CAP_VM_TYPES 235 +#define KVM_CAP_MEMORY_MAPPING 236 #ifdef KVM_CAP_IRQ_ROUTING @@ -2227,4 +2228,13 @@ struct kvm_create_guest_memfd { __u64 reserved[6]; }; +#define KVM_MEMORY_MAPPING _IOWR(KVMIO, 0xd5, struct kvm_memory_mapping) + +struct kvm_memory_mapping { + __u64 base_gfn; + __u64 nr_pages; + __u64 flags; + __u64 source; +}; + #endif /* __LINUX_KVM_H */ diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 0349e1f241d1..2f0a8e28795e 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -4409,6 +4409,62 @@ static int kvm_vcpu_ioctl_get_stats_fd(struct kvm_vcpu *vcpu) return fd; } +__weak void kvm_arch_vcpu_pre_memory_mapping(struct kvm_vcpu *vcpu) +{ +} + +__weak int kvm_arch_vcpu_memory_mapping(struct kvm_vcpu *vcpu, + struct kvm_memory_mapping *mapping) +{ + return -EOPNOTSUPP; +} + +static int kvm_vcpu_memory_mapping(struct kvm_vcpu *vcpu, + struct kvm_memory_mapping *mapping) +{ + bool added = false; + int idx, r = 0; + + /* flags isn't used yet. */ + if (mapping->flags) + return -EINVAL; + + /* Sanity check */ + if (!IS_ALIGNED(mapping->source, PAGE_SIZE) || + !mapping->nr_pages || + mapping->nr_pages & GENMASK_ULL(63, 63 - PAGE_SHIFT) || + mapping->base_gfn + mapping->nr_pages <= mapping->base_gfn) + return -EINVAL; + + vcpu_load(vcpu); + idx = srcu_read_lock(&vcpu->kvm->srcu); + kvm_arch_vcpu_pre_memory_mapping(vcpu); + + while (mapping->nr_pages) { + if (signal_pending(current)) { + r = -ERESTARTSYS; + break; + } + + if (need_resched()) + cond_resched(); + + r = kvm_arch_vcpu_memory_mapping(vcpu, mapping); + if (r) + break; + + added = true; + } + + srcu_read_unlock(&vcpu->kvm->srcu, idx); + vcpu_put(vcpu); + + if (added && mapping->nr_pages > 0) + r = -EAGAIN; + + return r; +} + static long kvm_vcpu_ioctl(struct file *filp, unsigned int ioctl, unsigned long arg) { @@ -4610,6 +4666,17 @@ static long kvm_vcpu_ioctl(struct file *filp, r = kvm_vcpu_ioctl_get_stats_fd(vcpu); break; } + case KVM_MEMORY_MAPPING: { + struct kvm_memory_mapping mapping; + + r = -EFAULT; + if (copy_from_user(&mapping, argp, sizeof(mapping))) + break; + r = kvm_vcpu_memory_mapping(vcpu, &mapping); + if (copy_to_user(argp, &mapping, sizeof(mapping))) + r = -EFAULT; + break; + } default: r = kvm_arch_vcpu_ioctl(filp, ioctl, arg); } From patchwork Mon Feb 26 08:25:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571443 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3C93A4F8A0; Mon, 26 Feb 2024 08:27:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.7 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936069; cv=none; b=dSqOeekH0NEBwzgZOKN4coGrYUFSB1k2fRXKU+88KopL6y2ykgVPV8GJUVAoq9ZswQ+0wxcsUuNe4+yFmtIl1IEh39OVvDB+cK9y/D6j+naWCboMBzijx1HTJxIeK2ICjWb9FbVs0SajJj9mEcJkxiXbE1HN7FD2+4Sq3Pv9kpU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936069; c=relaxed/simple; bh=yEg4v544YC7e+UFPcUmTBeIBNiztK9idn0tOEopCeIQ=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=RO8fhbNt8HbNSiAxWg8n3kylaNFvAuTD2V1rnRG7HWpzYdoOwtQf+6glab7e1vxfs1YHLjL+Kp3FIICltAYwZO5MPBfn86IJomtRMirm5cZyziBcFwYV0eFU4dXcYHyh0/XNVA7HhmBHWngpaYhHny1eOrNP8OU3BSJkt3tkFgw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=IDx2P+CM; arc=none smtp.client-ip=192.198.163.7 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="IDx2P+CM" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936067; x=1740472067; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=yEg4v544YC7e+UFPcUmTBeIBNiztK9idn0tOEopCeIQ=; b=IDx2P+CMzhL++faYjGDve/xKCONFD+CCHyUyPO+Jh+lhNV/Z0Ey5CmwU QhZDfTe7TgjTin7CWgOjc0k7Gn9gWqsRZV2eBQcqV/6ZuzRBmHuzWv+yl oePP2kAzD6idPWo2OX0LhduJARg5SoDS5CwsdYgw2C7GntHxlLA1oWYvZ g2GUwaOX9nfPje50nI86tMs5FPxJ82ae/K5GX2YsyeYRop9tXMhukE6+G z1wDQhq84wOZa/OrzSgIQh1CGmdCDolbO8UKO9EZPcRBezHgDdj9G76aq QFGGa0ltqNQRf285gFeUSfhNX86fSPdAgAejIC5nHqBz+jkqMsHq8b71f g==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="28631491" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="28631491" Received: from fmviesa007.fm.intel.com ([10.60.135.147]) by fmvoesa101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:27:43 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6474342" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by fmviesa007-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:27:42 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 015/130] KVM: Document KVM_MEMORY_MAPPING ioctl Date: Mon, 26 Feb 2024 00:25:17 -0800 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata Adds documentation of KVM_MEMORY_MAPPING ioctl. Signed-off-by: Isaku Yamahata --- v19: - newly added --- Documentation/virt/kvm/api.rst | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst index 3ec0b7a455a0..667dc58f7d2f 100644 --- a/Documentation/virt/kvm/api.rst +++ b/Documentation/virt/kvm/api.rst @@ -6323,6 +6323,34 @@ a single guest_memfd file, but the bound ranges must not overlap). See KVM_SET_USER_MEMORY_REGION2 for additional details. +4.143 KVM_MEMORY_MAPPING +------------------------ + +:Capability: KVM_CAP_MEMORY_MAPPING +:Architectures: none +:Type: vcpu ioctl +:Parameters: struct kvm_memory_mapping(in/out) +:Returns: 0 on success, <0 on error + +KVM_MEMORY_MAPPING populates guest memory without running vcpu. + +:: + + struct kvm_memory_mapping { + __u64 base_gfn; + __u64 nr_pages; + __u64 flags; + __u64 source; + }; + +KVM_MEMORY_MAPPING populates guest memory in the underlying mapping. If source +is not zero and it's supported (it depends on underlying technology), the guest +memory content is populated with the source. If nr_pages is large, it may +return -EAGAIN and the values (base_gfn and nr_pages. source if not zero) are +updated to point the remaining range. + +The "flags" field is reserved for future extensions and must be '0'. + 5. The kvm_run structure ======================== From patchwork Mon Feb 26 08:25:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571444 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9AB6E50A68; Mon, 26 Feb 2024 08:27:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.7 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936069; cv=none; b=mqaolWkBKHTkWGBS+VHTcwPlXd+AckwWLGYaMSq7GynlsmAzwthbQkyeQ8zMoheXfNdBycCNoq6hhAy3YNS+NDSghzNXFqKpBBI4gwEEbVJyQSUCQXDmOQ3g6YE04EWdc4kkfR2BXoR5MRuEcb5/Fr3au1CxQl9ETk3yZOexXr4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936069; c=relaxed/simple; bh=GqHoLR8nZGd0nBBjdIPWykKQiMpqYqCouXVMjthiwa8=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=JFePOq7ChUDEcekOi3iewSjsOsB5n3RBaGvhXt/N1z2ECwFvAS0SAQ8qKbqaCmrpOn3cjE6O1zPClBJ5xCpk4MGD1/yz0FHDsE72+oGGgv5gCX+LRUeGuoEpTHQGy992blD8dv5vDkiic5S/ZecaMYgDg43iFXuZD+8wWA8INOA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=hffJbi5x; arc=none smtp.client-ip=192.198.163.7 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="hffJbi5x" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936068; x=1740472068; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=GqHoLR8nZGd0nBBjdIPWykKQiMpqYqCouXVMjthiwa8=; b=hffJbi5x2K/2siNEXQ1FVjo2ysr0bPi9K2Df481tpULsmHv/EWg0J7S6 tuRzhg0Eo8pz2mc5ql5oAA+cBb5fYGVbnNH9gTINHMM/Ze/DYURmmC9vT /LtPuP/HhaNLfORLkycSUunGQDFGpaM60hQNTccnGaCPT8zPgyCoDbn7I j2pZqcgAx3eatwrrPrMNTVcIuodzzOgBVG50Z1ZGzter9ZuOpCKjs3NjR mR2v33sblytVbMxOmvc1WsW5jY20vMS36sHnDxWTkLdU2LVN16I8Cas9G WsyakamYJ1yPI3XqY+tzT33WhsIwRarQYj/POhsZpubc2286ZiXc4vD8j w==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="28631493" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="28631493" Received: from fmviesa007.fm.intel.com ([10.60.135.147]) by fmvoesa101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:27:43 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6474345" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by fmviesa007-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:27:43 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com, Sean Christopherson Subject: [PATCH v19 016/130] KVM: x86/mmu: Introduce kvm_mmu_map_tdp_page() for use by TDX Date: Mon, 26 Feb 2024 00:25:18 -0800 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Sean Christopherson Introduce a helper to directly (pun intended) fault-in a TDP page without having to go through the full page fault path. This allows TDX to get the resulting pfn and also allows the RET_PF_* enums to stay in mmu.c where they belong. Signed-off-by: Sean Christopherson Signed-off-by: Isaku Yamahata --- v19: - Move up for KVM_MEMORY_MAPPING. - Add goal_level for the caller to know how many pages are mapped. v14 -> v15: - Remove loop in kvm_mmu_map_tdp_page() and return error code based on RET_FP_xxx value to avoid potential infinite loop. The caller should loop on -EAGAIN instead now. Signed-off-by: Isaku Yamahata --- arch/x86/kvm/mmu.h | 3 +++ arch/x86/kvm/mmu/mmu.c | 58 ++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 61 insertions(+) diff --git a/arch/x86/kvm/mmu.h b/arch/x86/kvm/mmu.h index 60f21bb4c27b..d96c93a25b3b 100644 --- a/arch/x86/kvm/mmu.h +++ b/arch/x86/kvm/mmu.h @@ -183,6 +183,9 @@ static inline void kvm_mmu_refresh_passthrough_bits(struct kvm_vcpu *vcpu, __kvm_mmu_refresh_passthrough_bits(vcpu, mmu); } +int kvm_mmu_map_tdp_page(struct kvm_vcpu *vcpu, gpa_t gpa, u64 error_code, + u8 max_level, u8 *goal_level); + /* * Check if a given access (described through the I/D, W/R and U/S bits of a * page fault error code pfec) causes a permission fault with the given PTE diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 61674d6b17aa..ca0c91f14063 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4615,6 +4615,64 @@ int kvm_tdp_page_fault(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault) return direct_page_fault(vcpu, fault); } +int kvm_mmu_map_tdp_page(struct kvm_vcpu *vcpu, gpa_t gpa, u64 error_code, + u8 max_level, u8 *goal_level) +{ + int r; + struct kvm_page_fault fault = (struct kvm_page_fault) { + .addr = gpa, + .error_code = error_code, + .exec = error_code & PFERR_FETCH_MASK, + .write = error_code & PFERR_WRITE_MASK, + .present = error_code & PFERR_PRESENT_MASK, + .rsvd = error_code & PFERR_RSVD_MASK, + .user = error_code & PFERR_USER_MASK, + .prefetch = false, + .is_tdp = true, + .is_private = error_code & PFERR_GUEST_ENC_MASK, + .nx_huge_page_workaround_enabled = is_nx_huge_page_enabled(vcpu->kvm), + }; + + WARN_ON_ONCE(!vcpu->arch.mmu->root_role.direct); + fault.slot = kvm_vcpu_gfn_to_memslot(vcpu, fault.gfn); + + r = mmu_topup_memory_caches(vcpu, false); + if (r) + return r; + + fault.max_level = max_level; + fault.req_level = PG_LEVEL_4K; + fault.goal_level = PG_LEVEL_4K; + +#ifdef CONFIG_X86_64 + if (tdp_mmu_enabled) + r = kvm_tdp_mmu_page_fault(vcpu, &fault); + else +#endif + r = direct_page_fault(vcpu, &fault); + + if (is_error_noslot_pfn(fault.pfn) || vcpu->kvm->vm_bugged) + return -EFAULT; + + switch (r) { + case RET_PF_RETRY: + return -EAGAIN; + + case RET_PF_FIXED: + case RET_PF_SPURIOUS: + if (goal_level) + *goal_level = fault.goal_level; + return 0; + + case RET_PF_CONTINUE: + case RET_PF_EMULATE: + case RET_PF_INVALID: + default: + return -EIO; + } +} +EXPORT_SYMBOL_GPL(kvm_mmu_map_tdp_page); + static void nonpaging_init_context(struct kvm_mmu *context) { context->page_fault = nonpaging_page_fault; From patchwork Mon Feb 26 08:25:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571445 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7010951C4C; Mon, 26 Feb 2024 08:27:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.7 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936070; cv=none; b=WYQTY+BDKmL9DtW3m6Ui+zw9b0+77douwVH3Bwc97w4YarW0/ke7fOf0MWFwFtdMJP+LvqzVM1XvbNQuOd2x7h3DeiEp4o5pqRvtvv1Jx88SRnV0QGeSMNDK6fTx9PGH1Zc/WjivMTkllaEFuSMsOOT2vO45GiRdMjTNosuKHg4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936070; c=relaxed/simple; bh=o8YPZiq6LeeZQGwQQQuIE5wqxkP0AxPJNMylilPEjus=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=POK9SGWXNt5Pwsg8gV1AZkU3XtTILmPlmFpI5BibedFfnK1JgWj7qyIDW9FAyMC9JjODoEGmUe+cF0zj+ajF5vJNGbtzMzWM1wzv7OrNxyCaFTgAfFLpuLVSqwRv574d0gCiPNaLHUASTiaUGniGbXPwwczEw3+ju6hsnOrMilk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=aj5qsrUs; arc=none smtp.client-ip=192.198.163.7 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="aj5qsrUs" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936068; x=1740472068; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=o8YPZiq6LeeZQGwQQQuIE5wqxkP0AxPJNMylilPEjus=; b=aj5qsrUsb0uB8pdy59w+SY8cx0gRe8kKqrf8ZYPVrwbz9hnCAOmktJmO kTXuh3VyIqc1hgpzb5sDU+hAtP9mDnWEoS21FYufyDUPq6bcjHOtYONl3 +QQFzsm/1KWbp/HOMtQ+pw03csWPVaFzC1K/+VFdho9uz5KzmzYdchFi1 QjS3i8FKtjYafwk86Rk44pHbvVk6SIeQuOVYsG6f6nLnQCJwsItL5XuUq 4XUVnRSGWuJhL46IFluOpZQAQIT12Exjg/KHVmDmtpYslZboM1c8tBFJJ mn1MpLOQX3+gMUclX71+AKnrIItX5lpK9hNJtacZUUQ6Nm12yUZ4PQ7Ge w==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="28631497" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="28631497" Received: from fmviesa007.fm.intel.com ([10.60.135.147]) by fmvoesa101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:27:43 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6474348" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by fmviesa007-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:27:43 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 017/130] KVM: x86: Implement kvm_arch_{, pre_}vcpu_memory_mapping() Date: Mon, 26 Feb 2024 00:25:19 -0800 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata Wire KVM_MEMORY_MAPPING ioctl to kvm_mmu_map_tdp_page() to populate guest memory. Signed-off-by: Isaku Yamahata --- v19: - newly added --- arch/x86/kvm/x86.c | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 48a61d283406..03dab4266172 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -4663,6 +4663,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) case KVM_CAP_VM_DISABLE_NX_HUGE_PAGES: case KVM_CAP_IRQFD_RESAMPLE: case KVM_CAP_MEMORY_FAULT_INFO: + case KVM_CAP_MEMORY_MAPPING: r = 1; break; case KVM_CAP_EXIT_HYPERCALL: @@ -5801,6 +5802,31 @@ static int kvm_vcpu_ioctl_enable_cap(struct kvm_vcpu *vcpu, } } +void kvm_arch_vcpu_pre_memory_mapping(struct kvm_vcpu *vcpu) +{ + kvm_mmu_reload(vcpu); +} + +int kvm_arch_vcpu_memory_mapping(struct kvm_vcpu *vcpu, + struct kvm_memory_mapping *mapping) +{ + u8 max_level = KVM_MAX_HUGEPAGE_LEVEL; + u64 error_code = PFERR_WRITE_MASK; + u8 goal_level = PG_LEVEL_4K; + int r; + + r = kvm_mmu_map_tdp_page(vcpu, gfn_to_gpa(mapping->base_gfn), error_code, + max_level, &goal_level); + if (r) + return r; + + if (mapping->source) + mapping->source += KVM_HPAGE_SIZE(goal_level); + mapping->base_gfn += KVM_PAGES_PER_HPAGE(goal_level); + mapping->nr_pages -= KVM_PAGES_PER_HPAGE(goal_level); + return r; +} + long kvm_arch_vcpu_ioctl(struct file *filp, unsigned int ioctl, unsigned long arg) { From patchwork Mon Feb 26 08:25:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571446 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7D8E452F86; Mon, 26 Feb 2024 08:27:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.7 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936071; cv=none; b=R6Aa9mkw1JpWzjiUe2uXibZkGAZPmyve59edPLs7j4I6G5YtZpK3Ek+xN2suoonvQsmS90JoZXcAjuPETJr2qnGCEYopEzkQoyr1JWvIdjUbuDthDAHkYXt4hDyaxQz4DmYm+IPMiLUr9n9w0skKA8nj2F9Fj7L6I9kuPMmgCGY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936071; c=relaxed/simple; bh=gBqTh0BqhczjXbWb42Bx3PUV8vhAtv81RcTY6rA0AfQ=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Lcw8/OpnX710CDmEjSFFsHdxh5RbdrZqdMxmgjjGf43kXpqjME2oRa9QzM7f5Z0EMYKc//QbjybRu5oW6HeA4U7hG2eqUDJ8AKmOv9P1SSO6S5XB4iB1LL2ZrUUvmHtxCnnLfxAPQZ7M8a7xOje6lkfvVJtfYUHC33oFoyGOpUs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=bMOlfcli; arc=none smtp.client-ip=192.198.163.7 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="bMOlfcli" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936069; x=1740472069; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=gBqTh0BqhczjXbWb42Bx3PUV8vhAtv81RcTY6rA0AfQ=; b=bMOlfcliAAiavUwFuqQ/8EYVSX9Bw85QWmgmZtz2Y0YLLhWdHzxLE4Ty bVSo+sXPebCbjIRlH8Xc+Qcz6bocvFRS2HQwJXGSxMCNVH0WUYCiJ3REL 0c8BeK/TlmcAnCb6YeWARBEcuV1MFk8uLPBWQkhLqI/TDz1VQ8d1QyQFp XQ1SirVMLA3O5gs7Wg42zdOv02vTTwJDRtSMZ9H9APN3Yx9kECtnMXEFc 1FAr7OpZ4iHNZY3x78/ErLUX3ASOQU9sFWSRlm6rdsoZvorA+R35TDJvz OWhfgSWBaYKJtAZiDBSXPiv/LVlED8zvlVWDhoukWNWCe1f11UqDjLMNh w==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="28631502" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="28631502" Received: from fmviesa007.fm.intel.com ([10.60.135.147]) by fmvoesa101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:27:44 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6474353" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by fmviesa007-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:27:44 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com, Chao Gao Subject: [PATCH v19 018/130] KVM: x86/mmu: Assume guest MMIOs are shared Date: Mon, 26 Feb 2024 00:25:20 -0800 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Chao Gao TODO: Drop this patch once the common patch is merged. When memory slot isn't found for kvm page fault, handle it as MMIO. The guest of TDX_VM, SNP_VM, or SW_PROTECTED_VM don't necessarily convert the virtual MMIO range to shared before accessing it. When the guest tries to access the virtual device's MMIO without any private/shared conversion, An NPT fault or EPT violation is raised first to find private-shared mismatch. Don't raise KVM_EXIT_MEMORY_FAULT, fall back to KVM_PFN_NOLSLOT. Signed-off-by: Chao Gao Signed-off-by: Isaku Yamahata --- v19: - Added slot check before kvm_faultin_private_pfn() - updated comment - rewrite the commit message --- arch/x86/kvm/mmu/mmu.c | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index ca0c91f14063..c45252ed2ffd 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -4342,6 +4342,7 @@ static int kvm_faultin_pfn_private(struct kvm_vcpu *vcpu, static int __kvm_faultin_pfn(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault) { struct kvm_memory_slot *slot = fault->slot; + bool force_mmio; bool async; /* @@ -4371,12 +4372,21 @@ static int __kvm_faultin_pfn(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault return RET_PF_EMULATE; } - if (fault->is_private != kvm_mem_is_private(vcpu->kvm, fault->gfn)) { + /* + * !fault->slot means MMIO for SNP and TDX. Don't require explicit GPA + * conversion for MMIO because MMIO is assigned at the boot time. Fall + * to !is_private case to get pfn = KVM_PFN_NOSLOT. + */ + force_mmio = !slot && + vcpu->kvm->arch.vm_type != KVM_X86_DEFAULT_VM && + vcpu->kvm->arch.vm_type != KVM_X86_SW_PROTECTED_VM; + if (!force_mmio && + fault->is_private != kvm_mem_is_private(vcpu->kvm, fault->gfn)) { kvm_mmu_prepare_memory_fault_exit(vcpu, fault); return -EFAULT; } - if (fault->is_private) + if (!force_mmio && fault->is_private) return kvm_faultin_pfn_private(vcpu, fault); async = false; From patchwork Mon Feb 26 08:25:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571447 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D43405336F; Mon, 26 Feb 2024 08:27:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.7 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936071; cv=none; b=lWpKHc41WVS1/5/G5AvF7Q0qivTBBv3ohdUYjP1Gx5csA7oZXflpuYJVa5tolBM1L70sDVgJ7dD63KWGQ+vHoo+prmmqtkHLrfUtOQqbHdl5jZHxVDvHtjEPVba2Bix6tt47ymttmmqMC3N/ESX5SXCUef43+dp/uc+jjmVxOdA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936071; c=relaxed/simple; bh=qKBirrBY3+MoQoDpbImPbRXsqS9NNbeEormb4888U70=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=X1vwIRd+1+y+3h40XrcQhspSvTdUZjB+pNZU1VM1zZKoNz+ZCAwL1atJ7eQGZ2ldpJzla0nTYata7CI8i/uShxUJnvKMmrEWaKe0bguc3wbHOlBufJW3f0/ZVqilN3iViQcw/qf0S6vRRhUAUdZczYx+jr9Y66gd8/H01cVnBf8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=eGJhJmfZ; arc=none smtp.client-ip=192.198.163.7 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="eGJhJmfZ" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936070; x=1740472070; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=qKBirrBY3+MoQoDpbImPbRXsqS9NNbeEormb4888U70=; b=eGJhJmfZRHZRK592g1XLGwrkp7EqGydS49l0gxJ81LAb2nsnBqcAumUV 1ihZ1k8xBuEdTFvhIUotzeMV3jjJhAe/+jcUNPQGszFLWRb8ou9nGNYCc yB1uBNWIYGPef7BmEzPDl+R3hJEb4w7jx9ZuUx+JFHQy229RxspPH02sy BskrFFbkaARPY++u7l9SlXnzBd6Alsb5nNJWEdRMG/QMeGaRWJds4Z572 VphbKuxkZTaA/Q4ddzN/EWFnD+1vMSuUuYiLJ2CmJGT0G1ahL8ptB/zHk VxsEB4nsnIHjEY7flFqqmsZ83pluEkUgs10X+919mjy8OuqPPhzRSa7+S w==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="28631506" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="28631506" Received: from fmviesa007.fm.intel.com ([10.60.135.147]) by fmvoesa101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:27:44 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6474357" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by fmviesa007-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:27:44 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 019/130] KVM: x86: Add is_vm_type_supported callback Date: Mon, 26 Feb 2024 00:25:21 -0800 Message-Id: <6712a8a18abb033b1c32b9b6579ac297e3b00ab6.1708933498.git.isaku.yamahata@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata For SEV_SNP and TDX, allow the backend can override the supported vm type. Add KVM_X86_SNP_VM and KVM_X86_TDX_VM to reserve the bit. Signed-off-by: Isaku Yamahata --- v19: - Mention KVM_X86_SNP_VM to the commit message v18: - include into TDX KVM patch series v18 Changes v3 -> v4: - Added KVM_X86_SNP_VM Changes v2 -> v3: - no change - didn't bother to rename KVM_X86_PROTECTED_VM to KVM_X86_SW_PROTECTED_VM Changes v1 -> v2 - no change Signed-off-by: Isaku Yamahata --- arch/x86/include/asm/kvm-x86-ops.h | 1 + arch/x86/include/asm/kvm_host.h | 1 + arch/x86/include/uapi/asm/kvm.h | 2 ++ arch/x86/kvm/svm/svm.c | 7 +++++++ arch/x86/kvm/vmx/vmx.c | 7 +++++++ arch/x86/kvm/x86.c | 12 +++++++++++- arch/x86/kvm/x86.h | 2 ++ 7 files changed, 31 insertions(+), 1 deletion(-) diff --git a/arch/x86/include/asm/kvm-x86-ops.h b/arch/x86/include/asm/kvm-x86-ops.h index 156832f01ebe..8be71a5c5c87 100644 --- a/arch/x86/include/asm/kvm-x86-ops.h +++ b/arch/x86/include/asm/kvm-x86-ops.h @@ -20,6 +20,7 @@ KVM_X86_OP(hardware_disable) KVM_X86_OP(hardware_unsetup) KVM_X86_OP(has_emulated_msr) KVM_X86_OP(vcpu_after_set_cpuid) +KVM_X86_OP(is_vm_type_supported) KVM_X86_OP(vm_init) KVM_X86_OP_OPTIONAL(vm_destroy) KVM_X86_OP_OPTIONAL_RET0(vcpu_precreate) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 28314e7d546c..37cda8aa07b6 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1603,6 +1603,7 @@ struct kvm_x86_ops { bool (*has_emulated_msr)(struct kvm *kvm, u32 index); void (*vcpu_after_set_cpuid)(struct kvm_vcpu *vcpu); + bool (*is_vm_type_supported)(unsigned long vm_type); unsigned int vm_size; int (*vm_init)(struct kvm *kvm); void (*vm_destroy)(struct kvm *kvm); diff --git a/arch/x86/include/uapi/asm/kvm.h b/arch/x86/include/uapi/asm/kvm.h index a448d0964fc0..aa7a56a47564 100644 --- a/arch/x86/include/uapi/asm/kvm.h +++ b/arch/x86/include/uapi/asm/kvm.h @@ -564,5 +564,7 @@ struct kvm_pmu_event_filter { #define KVM_X86_DEFAULT_VM 0 #define KVM_X86_SW_PROTECTED_VM 1 +#define KVM_X86_TDX_VM 2 +#define KVM_X86_SNP_VM 3 #endif /* _ASM_X86_KVM_H */ diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c index e90b429c84f1..f76dd52d29ba 100644 --- a/arch/x86/kvm/svm/svm.c +++ b/arch/x86/kvm/svm/svm.c @@ -4886,6 +4886,12 @@ static void svm_vm_destroy(struct kvm *kvm) sev_vm_destroy(kvm); } +static bool svm_is_vm_type_supported(unsigned long type) +{ + /* FIXME: Check if CPU is capable of SEV-SNP. */ + return __kvm_is_vm_type_supported(type); +} + static int svm_vm_init(struct kvm *kvm) { if (!pause_filter_count || !pause_filter_thresh) @@ -4914,6 +4920,7 @@ static struct kvm_x86_ops svm_x86_ops __initdata = { .vcpu_free = svm_vcpu_free, .vcpu_reset = svm_vcpu_reset, + .is_vm_type_supported = svm_is_vm_type_supported, .vm_size = sizeof(struct kvm_svm), .vm_init = svm_vm_init, .vm_destroy = svm_vm_destroy, diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 1111d9d08903..fca3457dd050 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -7541,6 +7541,12 @@ static int vmx_vcpu_create(struct kvm_vcpu *vcpu) return err; } +static bool vmx_is_vm_type_supported(unsigned long type) +{ + /* TODO: Check if TDX is supported. */ + return __kvm_is_vm_type_supported(type); +} + #define L1TF_MSG_SMT "L1TF CPU bug present and SMT on, data leak possible. See CVE-2018-3646 and https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/l1tf.html for details.\n" #define L1TF_MSG_L1D "L1TF CPU bug present and virtualization mitigation disabled, data leak possible. See CVE-2018-3646 and https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/l1tf.html for details.\n" @@ -8263,6 +8269,7 @@ static struct kvm_x86_ops vmx_x86_ops __initdata = { .hardware_disable = vmx_hardware_disable, .has_emulated_msr = vmx_has_emulated_msr, + .is_vm_type_supported = vmx_is_vm_type_supported, .vm_size = sizeof(struct kvm_vmx), .vm_init = vmx_vm_init, .vm_destroy = vmx_vm_destroy, diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 03dab4266172..442b356e4939 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -4576,12 +4576,18 @@ static int kvm_ioctl_get_supported_hv_cpuid(struct kvm_vcpu *vcpu, } #endif -static bool kvm_is_vm_type_supported(unsigned long type) +bool __kvm_is_vm_type_supported(unsigned long type) { return type == KVM_X86_DEFAULT_VM || (type == KVM_X86_SW_PROTECTED_VM && IS_ENABLED(CONFIG_KVM_SW_PROTECTED_VM) && tdp_enabled); } +EXPORT_SYMBOL_GPL(__kvm_is_vm_type_supported); + +static bool kvm_is_vm_type_supported(unsigned long type) +{ + return static_call(kvm_x86_is_vm_type_supported)(type); +} int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) { @@ -4784,6 +4790,10 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) r = BIT(KVM_X86_DEFAULT_VM); if (kvm_is_vm_type_supported(KVM_X86_SW_PROTECTED_VM)) r |= BIT(KVM_X86_SW_PROTECTED_VM); + if (kvm_is_vm_type_supported(KVM_X86_TDX_VM)) + r |= BIT(KVM_X86_TDX_VM); + if (kvm_is_vm_type_supported(KVM_X86_SNP_VM)) + r |= BIT(KVM_X86_SNP_VM); break; default: break; diff --git a/arch/x86/kvm/x86.h b/arch/x86/kvm/x86.h index 2f7e19166658..4e40c23d66ed 100644 --- a/arch/x86/kvm/x86.h +++ b/arch/x86/kvm/x86.h @@ -9,6 +9,8 @@ #include "kvm_cache_regs.h" #include "kvm_emulate.h" +bool __kvm_is_vm_type_supported(unsigned long type); + struct kvm_caps { /* control of guest tsc rate supported? */ bool has_tsc_control; From patchwork Mon Feb 26 08:25:22 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571450 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 90BC8535AE; Mon, 26 Feb 2024 08:27:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.7 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936074; cv=none; b=Gk57UDJEqUTIoItCUjVk746NwtWKzkDPLAm4vVPO3yQQFK61WdUkdlqFeBsYX4pRDYxGP+T7YdlJysC+mwlspThaecl5GjPjVSj4hWDAztYeY7DX2kRHv2vncUOYEELRjwz8o4xfqaAF3aY6Z1lyAjx7U74lHSNfeGTzF555fRs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936074; c=relaxed/simple; bh=7isqSr8GNC9ok033+WjZij4Zo7dj0pCzaexgcAwRjNg=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=jfBvfYmouzaETqF+8iHAC3ULzGqaKkcl4O+ZzSlBSvl18Zwa3vwQf3FOQA94LbJdHwVZvLs5fHQ7cVzUjewxJ+m4vVjDOqSMb1WpsYV6VuUAZz/ZKEfUqDD3eERD/oX5+rnTB+xjjGxCRNEbFpKqRnQCEBHrSDFwA6Yx6f5PDws= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=H5MZQpsL; arc=none smtp.client-ip=192.198.163.7 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="H5MZQpsL" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936071; x=1740472071; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=7isqSr8GNC9ok033+WjZij4Zo7dj0pCzaexgcAwRjNg=; b=H5MZQpsLW6XiX2uGwwN5ApovELs7GEvYY5XfIdGbaTy0fD9+35EuNUU+ tIh/zHPOydoEcMao5aeZxIynvYRzuw8UeNtwshB2bzY7Gnaupmh7uZPQh 00WSc9XXH8T/G1X2nYc79m0+Ma5w2MycSIaWN0b50n6nuSz5KV8GaSA3S eu0VuUbEgnVCv9u1QHQvl613bPjQVKxx5EaBoDimxfPtK64VscwxhlJKR JHpZOzj1jKbQ+uMt3QH4o44S0TguNXRMliWNksXeMBM2nHQxe2K0VlGlw jIA3a9MbXzvIuf/H95NpVa/OGKqqMQeQcnnTtLz5JnJu9Vmsq5x9bgIVV g==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="28631510" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="28631510" Received: from fmviesa007.fm.intel.com ([10.60.135.147]) by fmvoesa101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:27:45 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6474361" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by fmviesa007-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:27:44 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com, Sean Christopherson , Xiaoyao Li , Binbin Wu , Yuan Yao Subject: [PATCH v19 020/130] KVM: VMX: Move out vmx_x86_ops to 'main.c' to wrap VMX and TDX Date: Mon, 26 Feb 2024 00:25:22 -0800 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Sean Christopherson KVM accesses Virtual Machine Control Structure (VMCS) with VMX instructions to operate on VM. TDX doesn't allow VMM to operate VMCS directly. Instead, TDX has its own data structures, and TDX SEAMCALL APIs for VMM to indirectly operate those data structures. This means we must have a TDX version of kvm_x86_ops. The existing global struct kvm_x86_ops already defines an interface which fits with TDX. But kvm_x86_ops is system-wide, not per-VM structure. To allow VMX to coexist with TDs, the kvm_x86_ops callbacks will have wrappers "if (tdx) tdx_op() else vmx_op()" to switch VMX or TDX at run time. To split the runtime switch, the VMX implementation, and the TDX implementation, add main.c, and move out the vmx_x86_ops hooks in preparation for adding TDX, which can coexist with VMX, i.e. KVM can run both VMs and TDs. Use 'vt' for the naming scheme as a nod to VT-x and as a concatenation of VmxTdx. The current code looks as follows. In vmx.c static vmx_op() { ... } static struct kvm_x86_ops vmx_x86_ops = { .op = vmx_op, initialization code The eventually converted code will look like In vmx.c, keep the VMX operations. vmx_op() { ... } VMX initialization In tdx.c, define the TDX operations. tdx_op() { ... } TDX initialization In x86_ops.h, declare the VMX and TDX operations. vmx_op(); tdx_op(); In main.c, define common wrappers for VMX and TDX. static vt_ops() { if (tdx) tdx_ops() else vmx_ops() } static struct kvm_x86_ops vt_x86_ops = { .op = vt_op, initialization to call VMX and TDX initialization Opportunistically, fix the name inconsistency from vmx_create_vcpu() and vmx_free_vcpu() to vmx_vcpu_create() and vmx_vcpu_free(). Co-developed-by: Xiaoyao Li Signed-off-by: Xiaoyao Li Signed-off-by: Sean Christopherson Signed-off-by: Isaku Yamahata Reviewed-by: Paolo Bonzini Reviewed-by: Binbin Wu Reviewed-by: Yuan Yao Reviewed-by: Xiaoyao Li --- v19: - Move down the declaration of vmx_hardware_setup() in x86_ops.h. Xiaoyao v18: - Add Reviewed-by: Binbin Wu - fix indent alignments pointed by Binbin Wu Signed-off-by: Isaku Yamahata --- arch/x86/kvm/Makefile | 2 +- arch/x86/kvm/vmx/main.c | 169 +++++++++++++++++ arch/x86/kvm/vmx/vmx.c | 378 ++++++++++--------------------------- arch/x86/kvm/vmx/x86_ops.h | 124 ++++++++++++ 4 files changed, 397 insertions(+), 276 deletions(-) create mode 100644 arch/x86/kvm/vmx/main.c create mode 100644 arch/x86/kvm/vmx/x86_ops.h diff --git a/arch/x86/kvm/Makefile b/arch/x86/kvm/Makefile index 475b5fa917a6..274df24b647f 100644 --- a/arch/x86/kvm/Makefile +++ b/arch/x86/kvm/Makefile @@ -20,7 +20,7 @@ kvm-$(CONFIG_KVM_XEN) += xen.o kvm-$(CONFIG_KVM_SMM) += smm.o kvm-intel-y += vmx/vmx.o vmx/vmenter.o vmx/pmu_intel.o vmx/vmcs12.o \ - vmx/nested.o vmx/posted_intr.o + vmx/nested.o vmx/posted_intr.o vmx/main.o kvm-intel-$(CONFIG_X86_SGX_KVM) += vmx/sgx.o kvm-intel-$(CONFIG_KVM_HYPERV) += vmx/hyperv.o vmx/hyperv_evmcs.o diff --git a/arch/x86/kvm/vmx/main.c b/arch/x86/kvm/vmx/main.c new file mode 100644 index 000000000000..eeb7a43b271d --- /dev/null +++ b/arch/x86/kvm/vmx/main.c @@ -0,0 +1,169 @@ +// SPDX-License-Identifier: GPL-2.0 +#include + +#include "x86_ops.h" +#include "vmx.h" +#include "nested.h" +#include "pmu.h" + +#define VMX_REQUIRED_APICV_INHIBITS \ + (BIT(APICV_INHIBIT_REASON_DISABLE)| \ + BIT(APICV_INHIBIT_REASON_ABSENT) | \ + BIT(APICV_INHIBIT_REASON_HYPERV) | \ + BIT(APICV_INHIBIT_REASON_BLOCKIRQ) | \ + BIT(APICV_INHIBIT_REASON_PHYSICAL_ID_ALIASED) | \ + BIT(APICV_INHIBIT_REASON_APIC_ID_MODIFIED) | \ + BIT(APICV_INHIBIT_REASON_APIC_BASE_MODIFIED)) + +struct kvm_x86_ops vt_x86_ops __initdata = { + .name = KBUILD_MODNAME, + + .check_processor_compatibility = vmx_check_processor_compat, + + .hardware_unsetup = vmx_hardware_unsetup, + + .hardware_enable = vmx_hardware_enable, + .hardware_disable = vmx_hardware_disable, + .has_emulated_msr = vmx_has_emulated_msr, + + .is_vm_type_supported = vmx_is_vm_type_supported, + .vm_size = sizeof(struct kvm_vmx), + .vm_init = vmx_vm_init, + .vm_destroy = vmx_vm_destroy, + + .vcpu_precreate = vmx_vcpu_precreate, + .vcpu_create = vmx_vcpu_create, + .vcpu_free = vmx_vcpu_free, + .vcpu_reset = vmx_vcpu_reset, + + .prepare_switch_to_guest = vmx_prepare_switch_to_guest, + .vcpu_load = vmx_vcpu_load, + .vcpu_put = vmx_vcpu_put, + + .update_exception_bitmap = vmx_update_exception_bitmap, + .get_msr_feature = vmx_get_msr_feature, + .get_msr = vmx_get_msr, + .set_msr = vmx_set_msr, + .get_segment_base = vmx_get_segment_base, + .get_segment = vmx_get_segment, + .set_segment = vmx_set_segment, + .get_cpl = vmx_get_cpl, + .get_cs_db_l_bits = vmx_get_cs_db_l_bits, + .is_valid_cr0 = vmx_is_valid_cr0, + .set_cr0 = vmx_set_cr0, + .is_valid_cr4 = vmx_is_valid_cr4, + .set_cr4 = vmx_set_cr4, + .set_efer = vmx_set_efer, + .get_idt = vmx_get_idt, + .set_idt = vmx_set_idt, + .get_gdt = vmx_get_gdt, + .set_gdt = vmx_set_gdt, + .set_dr7 = vmx_set_dr7, + .sync_dirty_debug_regs = vmx_sync_dirty_debug_regs, + .cache_reg = vmx_cache_reg, + .get_rflags = vmx_get_rflags, + .set_rflags = vmx_set_rflags, + .get_if_flag = vmx_get_if_flag, + + .flush_tlb_all = vmx_flush_tlb_all, + .flush_tlb_current = vmx_flush_tlb_current, + .flush_tlb_gva = vmx_flush_tlb_gva, + .flush_tlb_guest = vmx_flush_tlb_guest, + + .vcpu_pre_run = vmx_vcpu_pre_run, + .vcpu_run = vmx_vcpu_run, + .handle_exit = vmx_handle_exit, + .skip_emulated_instruction = vmx_skip_emulated_instruction, + .update_emulated_instruction = vmx_update_emulated_instruction, + .set_interrupt_shadow = vmx_set_interrupt_shadow, + .get_interrupt_shadow = vmx_get_interrupt_shadow, + .patch_hypercall = vmx_patch_hypercall, + .inject_irq = vmx_inject_irq, + .inject_nmi = vmx_inject_nmi, + .inject_exception = vmx_inject_exception, + .cancel_injection = vmx_cancel_injection, + .interrupt_allowed = vmx_interrupt_allowed, + .nmi_allowed = vmx_nmi_allowed, + .get_nmi_mask = vmx_get_nmi_mask, + .set_nmi_mask = vmx_set_nmi_mask, + .enable_nmi_window = vmx_enable_nmi_window, + .enable_irq_window = vmx_enable_irq_window, + .update_cr8_intercept = vmx_update_cr8_intercept, + .set_virtual_apic_mode = vmx_set_virtual_apic_mode, + .set_apic_access_page_addr = vmx_set_apic_access_page_addr, + .refresh_apicv_exec_ctrl = vmx_refresh_apicv_exec_ctrl, + .load_eoi_exitmap = vmx_load_eoi_exitmap, + .apicv_pre_state_restore = vmx_apicv_pre_state_restore, + .required_apicv_inhibits = VMX_REQUIRED_APICV_INHIBITS, + .hwapic_irr_update = vmx_hwapic_irr_update, + .hwapic_isr_update = vmx_hwapic_isr_update, + .guest_apic_has_interrupt = vmx_guest_apic_has_interrupt, + .sync_pir_to_irr = vmx_sync_pir_to_irr, + .deliver_interrupt = vmx_deliver_interrupt, + .dy_apicv_has_pending_interrupt = pi_has_pending_interrupt, + + .set_tss_addr = vmx_set_tss_addr, + .set_identity_map_addr = vmx_set_identity_map_addr, + .get_mt_mask = vmx_get_mt_mask, + + .get_exit_info = vmx_get_exit_info, + + .vcpu_after_set_cpuid = vmx_vcpu_after_set_cpuid, + + .has_wbinvd_exit = cpu_has_vmx_wbinvd_exit, + + .get_l2_tsc_offset = vmx_get_l2_tsc_offset, + .get_l2_tsc_multiplier = vmx_get_l2_tsc_multiplier, + .write_tsc_offset = vmx_write_tsc_offset, + .write_tsc_multiplier = vmx_write_tsc_multiplier, + + .load_mmu_pgd = vmx_load_mmu_pgd, + + .check_intercept = vmx_check_intercept, + .handle_exit_irqoff = vmx_handle_exit_irqoff, + + .request_immediate_exit = vmx_request_immediate_exit, + + .sched_in = vmx_sched_in, + + .cpu_dirty_log_size = PML_ENTITY_NUM, + .update_cpu_dirty_logging = vmx_update_cpu_dirty_logging, + + .nested_ops = &vmx_nested_ops, + + .pi_update_irte = vmx_pi_update_irte, + .pi_start_assignment = vmx_pi_start_assignment, + +#ifdef CONFIG_X86_64 + .set_hv_timer = vmx_set_hv_timer, + .cancel_hv_timer = vmx_cancel_hv_timer, +#endif + + .setup_mce = vmx_setup_mce, + +#ifdef CONFIG_KVM_SMM + .smi_allowed = vmx_smi_allowed, + .enter_smm = vmx_enter_smm, + .leave_smm = vmx_leave_smm, + .enable_smi_window = vmx_enable_smi_window, +#endif + + .check_emulate_instruction = vmx_check_emulate_instruction, + .apic_init_signal_blocked = vmx_apic_init_signal_blocked, + .migrate_timers = vmx_migrate_timers, + + .msr_filter_changed = vmx_msr_filter_changed, + .complete_emulated_msr = kvm_complete_insn_gp, + + .vcpu_deliver_sipi_vector = kvm_vcpu_deliver_sipi_vector, + + .get_untagged_addr = vmx_get_untagged_addr, +}; + +struct kvm_x86_init_ops vt_init_ops __initdata = { + .hardware_setup = vmx_hardware_setup, + .handle_intel_pt_intr = NULL, + + .runtime_ops = &vt_x86_ops, + .pmu_ops = &intel_pmu_ops, +}; diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index fca3457dd050..434f5aaef030 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -65,6 +65,7 @@ #include "vmcs12.h" #include "vmx.h" #include "x86.h" +#include "x86_ops.h" #include "smm.h" #include "vmx_onhyperv.h" @@ -516,8 +517,6 @@ static inline void vmx_segment_cache_clear(struct vcpu_vmx *vmx) static unsigned long host_idt_base; #if IS_ENABLED(CONFIG_HYPERV) -static struct kvm_x86_ops vmx_x86_ops __initdata; - static bool __read_mostly enlightened_vmcs = true; module_param(enlightened_vmcs, bool, 0444); @@ -567,9 +566,8 @@ static __init void hv_init_evmcs(void) } if (ms_hyperv.nested_features & HV_X64_NESTED_DIRECT_FLUSH) - vmx_x86_ops.enable_l2_tlb_flush + vt_x86_ops.enable_l2_tlb_flush = hv_enable_l2_tlb_flush; - } else { enlightened_vmcs = false; } @@ -1474,7 +1472,7 @@ void vmx_vcpu_load_vmcs(struct kvm_vcpu *vcpu, int cpu, * Switches to specified vcpu, until a matching vcpu_put(), but assumes * vcpu mutex is already taken. */ -static void vmx_vcpu_load(struct kvm_vcpu *vcpu, int cpu) +void vmx_vcpu_load(struct kvm_vcpu *vcpu, int cpu) { struct vcpu_vmx *vmx = to_vmx(vcpu); @@ -1485,7 +1483,7 @@ static void vmx_vcpu_load(struct kvm_vcpu *vcpu, int cpu) vmx->host_debugctlmsr = get_debugctlmsr(); } -static void vmx_vcpu_put(struct kvm_vcpu *vcpu) +void vmx_vcpu_put(struct kvm_vcpu *vcpu) { vmx_vcpu_pi_put(vcpu); @@ -1544,7 +1542,7 @@ void vmx_set_rflags(struct kvm_vcpu *vcpu, unsigned long rflags) vmx->emulation_required = vmx_emulation_required(vcpu); } -static bool vmx_get_if_flag(struct kvm_vcpu *vcpu) +bool vmx_get_if_flag(struct kvm_vcpu *vcpu) { return vmx_get_rflags(vcpu) & X86_EFLAGS_IF; } @@ -1650,8 +1648,8 @@ static int vmx_rtit_ctl_check(struct kvm_vcpu *vcpu, u64 data) return 0; } -static int vmx_check_emulate_instruction(struct kvm_vcpu *vcpu, int emul_type, - void *insn, int insn_len) +int vmx_check_emulate_instruction(struct kvm_vcpu *vcpu, int emul_type, + void *insn, int insn_len) { /* * Emulation of instructions in SGX enclaves is impossible as RIP does @@ -1735,7 +1733,7 @@ static int skip_emulated_instruction(struct kvm_vcpu *vcpu) * Recognizes a pending MTF VM-exit and records the nested state for later * delivery. */ -static void vmx_update_emulated_instruction(struct kvm_vcpu *vcpu) +void vmx_update_emulated_instruction(struct kvm_vcpu *vcpu) { struct vmcs12 *vmcs12 = get_vmcs12(vcpu); struct vcpu_vmx *vmx = to_vmx(vcpu); @@ -1766,7 +1764,7 @@ static void vmx_update_emulated_instruction(struct kvm_vcpu *vcpu) } } -static int vmx_skip_emulated_instruction(struct kvm_vcpu *vcpu) +int vmx_skip_emulated_instruction(struct kvm_vcpu *vcpu) { vmx_update_emulated_instruction(vcpu); return skip_emulated_instruction(vcpu); @@ -1785,7 +1783,7 @@ static void vmx_clear_hlt(struct kvm_vcpu *vcpu) vmcs_write32(GUEST_ACTIVITY_STATE, GUEST_ACTIVITY_ACTIVE); } -static void vmx_inject_exception(struct kvm_vcpu *vcpu) +void vmx_inject_exception(struct kvm_vcpu *vcpu) { struct kvm_queued_exception *ex = &vcpu->arch.exception; u32 intr_info = ex->vector | INTR_INFO_VALID_MASK; @@ -1906,12 +1904,12 @@ u64 vmx_get_l2_tsc_multiplier(struct kvm_vcpu *vcpu) return kvm_caps.default_tsc_scaling_ratio; } -static void vmx_write_tsc_offset(struct kvm_vcpu *vcpu) +void vmx_write_tsc_offset(struct kvm_vcpu *vcpu) { vmcs_write64(TSC_OFFSET, vcpu->arch.tsc_offset); } -static void vmx_write_tsc_multiplier(struct kvm_vcpu *vcpu) +void vmx_write_tsc_multiplier(struct kvm_vcpu *vcpu) { vmcs_write64(TSC_MULTIPLIER, vcpu->arch.tsc_scaling_ratio); } @@ -1954,7 +1952,7 @@ static inline bool is_vmx_feature_control_msr_valid(struct vcpu_vmx *vmx, return !(msr->data & ~valid_bits); } -static int vmx_get_msr_feature(struct kvm_msr_entry *msr) +int vmx_get_msr_feature(struct kvm_msr_entry *msr) { switch (msr->index) { case KVM_FIRST_EMULATED_VMX_MSR ... KVM_LAST_EMULATED_VMX_MSR: @@ -1971,7 +1969,7 @@ static int vmx_get_msr_feature(struct kvm_msr_entry *msr) * Returns 0 on success, non-0 otherwise. * Assumes vcpu_load() was already called. */ -static int vmx_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info) +int vmx_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info) { struct vcpu_vmx *vmx = to_vmx(vcpu); struct vmx_uret_msr *msr; @@ -2152,7 +2150,7 @@ static u64 vmx_get_supported_debugctl(struct kvm_vcpu *vcpu, bool host_initiated * Returns 0 on success, non-0 otherwise. * Assumes vcpu_load() was already called. */ -static int vmx_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info) +int vmx_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info) { struct vcpu_vmx *vmx = to_vmx(vcpu); struct vmx_uret_msr *msr; @@ -2455,7 +2453,7 @@ static int vmx_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info) return ret; } -static void vmx_cache_reg(struct kvm_vcpu *vcpu, enum kvm_reg reg) +void vmx_cache_reg(struct kvm_vcpu *vcpu, enum kvm_reg reg) { unsigned long guest_owned_bits; @@ -2756,7 +2754,7 @@ static bool kvm_is_vmx_supported(void) return supported; } -static int vmx_check_processor_compat(void) +int vmx_check_processor_compat(void) { int cpu = raw_smp_processor_id(); struct vmcs_config vmcs_conf; @@ -2798,7 +2796,7 @@ static int kvm_cpu_vmxon(u64 vmxon_pointer) return -EFAULT; } -static int vmx_hardware_enable(void) +int vmx_hardware_enable(void) { int cpu = raw_smp_processor_id(); u64 phys_addr = __pa(per_cpu(vmxarea, cpu)); @@ -2838,7 +2836,7 @@ static void vmclear_local_loaded_vmcss(void) __loaded_vmcs_clear(v); } -static void vmx_hardware_disable(void) +void vmx_hardware_disable(void) { vmclear_local_loaded_vmcss(); @@ -3152,7 +3150,7 @@ static void exit_lmode(struct kvm_vcpu *vcpu) #endif -static void vmx_flush_tlb_all(struct kvm_vcpu *vcpu) +void vmx_flush_tlb_all(struct kvm_vcpu *vcpu) { struct vcpu_vmx *vmx = to_vmx(vcpu); @@ -3182,7 +3180,7 @@ static inline int vmx_get_current_vpid(struct kvm_vcpu *vcpu) return to_vmx(vcpu)->vpid; } -static void vmx_flush_tlb_current(struct kvm_vcpu *vcpu) +void vmx_flush_tlb_current(struct kvm_vcpu *vcpu) { struct kvm_mmu *mmu = vcpu->arch.mmu; u64 root_hpa = mmu->root.hpa; @@ -3198,7 +3196,7 @@ static void vmx_flush_tlb_current(struct kvm_vcpu *vcpu) vpid_sync_context(vmx_get_current_vpid(vcpu)); } -static void vmx_flush_tlb_gva(struct kvm_vcpu *vcpu, gva_t addr) +void vmx_flush_tlb_gva(struct kvm_vcpu *vcpu, gva_t addr) { /* * vpid_sync_vcpu_addr() is a nop if vpid==0, see the comment in @@ -3207,7 +3205,7 @@ static void vmx_flush_tlb_gva(struct kvm_vcpu *vcpu, gva_t addr) vpid_sync_vcpu_addr(vmx_get_current_vpid(vcpu), addr); } -static void vmx_flush_tlb_guest(struct kvm_vcpu *vcpu) +void vmx_flush_tlb_guest(struct kvm_vcpu *vcpu) { /* * vpid_sync_context() is a nop if vpid==0, e.g. if enable_vpid==0 or a @@ -3252,7 +3250,7 @@ void ept_save_pdptrs(struct kvm_vcpu *vcpu) #define CR3_EXITING_BITS (CPU_BASED_CR3_LOAD_EXITING | \ CPU_BASED_CR3_STORE_EXITING) -static bool vmx_is_valid_cr0(struct kvm_vcpu *vcpu, unsigned long cr0) +bool vmx_is_valid_cr0(struct kvm_vcpu *vcpu, unsigned long cr0) { if (is_guest_mode(vcpu)) return nested_guest_cr0_valid(vcpu, cr0); @@ -3373,8 +3371,7 @@ u64 construct_eptp(struct kvm_vcpu *vcpu, hpa_t root_hpa, int root_level) return eptp; } -static void vmx_load_mmu_pgd(struct kvm_vcpu *vcpu, hpa_t root_hpa, - int root_level) +void vmx_load_mmu_pgd(struct kvm_vcpu *vcpu, hpa_t root_hpa, int root_level) { struct kvm *kvm = vcpu->kvm; bool update_guest_cr3 = true; @@ -3403,8 +3400,7 @@ static void vmx_load_mmu_pgd(struct kvm_vcpu *vcpu, hpa_t root_hpa, vmcs_writel(GUEST_CR3, guest_cr3); } - -static bool vmx_is_valid_cr4(struct kvm_vcpu *vcpu, unsigned long cr4) +bool vmx_is_valid_cr4(struct kvm_vcpu *vcpu, unsigned long cr4) { /* * We operate under the default treatment of SMM, so VMX cannot be @@ -3520,7 +3516,7 @@ void vmx_get_segment(struct kvm_vcpu *vcpu, struct kvm_segment *var, int seg) var->g = (ar >> 15) & 1; } -static u64 vmx_get_segment_base(struct kvm_vcpu *vcpu, int seg) +u64 vmx_get_segment_base(struct kvm_vcpu *vcpu, int seg) { struct kvm_segment s; @@ -3597,14 +3593,14 @@ void __vmx_set_segment(struct kvm_vcpu *vcpu, struct kvm_segment *var, int seg) vmcs_write32(sf->ar_bytes, vmx_segment_access_rights(var)); } -static void vmx_set_segment(struct kvm_vcpu *vcpu, struct kvm_segment *var, int seg) +void vmx_set_segment(struct kvm_vcpu *vcpu, struct kvm_segment *var, int seg) { __vmx_set_segment(vcpu, var, seg); to_vmx(vcpu)->emulation_required = vmx_emulation_required(vcpu); } -static void vmx_get_cs_db_l_bits(struct kvm_vcpu *vcpu, int *db, int *l) +void vmx_get_cs_db_l_bits(struct kvm_vcpu *vcpu, int *db, int *l) { u32 ar = vmx_read_guest_seg_ar(to_vmx(vcpu), VCPU_SREG_CS); @@ -3612,25 +3608,25 @@ static void vmx_get_cs_db_l_bits(struct kvm_vcpu *vcpu, int *db, int *l) *l = (ar >> 13) & 1; } -static void vmx_get_idt(struct kvm_vcpu *vcpu, struct desc_ptr *dt) +void vmx_get_idt(struct kvm_vcpu *vcpu, struct desc_ptr *dt) { dt->size = vmcs_read32(GUEST_IDTR_LIMIT); dt->address = vmcs_readl(GUEST_IDTR_BASE); } -static void vmx_set_idt(struct kvm_vcpu *vcpu, struct desc_ptr *dt) +void vmx_set_idt(struct kvm_vcpu *vcpu, struct desc_ptr *dt) { vmcs_write32(GUEST_IDTR_LIMIT, dt->size); vmcs_writel(GUEST_IDTR_BASE, dt->address); } -static void vmx_get_gdt(struct kvm_vcpu *vcpu, struct desc_ptr *dt) +void vmx_get_gdt(struct kvm_vcpu *vcpu, struct desc_ptr *dt) { dt->size = vmcs_read32(GUEST_GDTR_LIMIT); dt->address = vmcs_readl(GUEST_GDTR_BASE); } -static void vmx_set_gdt(struct kvm_vcpu *vcpu, struct desc_ptr *dt) +void vmx_set_gdt(struct kvm_vcpu *vcpu, struct desc_ptr *dt) { vmcs_write32(GUEST_GDTR_LIMIT, dt->size); vmcs_writel(GUEST_GDTR_BASE, dt->address); @@ -4102,7 +4098,7 @@ void pt_update_intercept_for_msr(struct kvm_vcpu *vcpu) } } -static bool vmx_guest_apic_has_interrupt(struct kvm_vcpu *vcpu) +bool vmx_guest_apic_has_interrupt(struct kvm_vcpu *vcpu) { struct vcpu_vmx *vmx = to_vmx(vcpu); void *vapic_page; @@ -4122,7 +4118,7 @@ static bool vmx_guest_apic_has_interrupt(struct kvm_vcpu *vcpu) return ((rvi & 0xf0) > (vppr & 0xf0)); } -static void vmx_msr_filter_changed(struct kvm_vcpu *vcpu) +void vmx_msr_filter_changed(struct kvm_vcpu *vcpu) { struct vcpu_vmx *vmx = to_vmx(vcpu); u32 i; @@ -4263,8 +4259,8 @@ static int vmx_deliver_posted_interrupt(struct kvm_vcpu *vcpu, int vector) return 0; } -static void vmx_deliver_interrupt(struct kvm_lapic *apic, int delivery_mode, - int trig_mode, int vector) +void vmx_deliver_interrupt(struct kvm_lapic *apic, int delivery_mode, + int trig_mode, int vector) { struct kvm_vcpu *vcpu = apic->vcpu; @@ -4426,7 +4422,7 @@ static u32 vmx_vmexit_ctrl(void) ~(VM_EXIT_LOAD_IA32_PERF_GLOBAL_CTRL | VM_EXIT_LOAD_IA32_EFER); } -static void vmx_refresh_apicv_exec_ctrl(struct kvm_vcpu *vcpu) +void vmx_refresh_apicv_exec_ctrl(struct kvm_vcpu *vcpu) { struct vcpu_vmx *vmx = to_vmx(vcpu); @@ -4690,7 +4686,7 @@ static int vmx_alloc_ipiv_pid_table(struct kvm *kvm) return 0; } -static int vmx_vcpu_precreate(struct kvm *kvm) +int vmx_vcpu_precreate(struct kvm *kvm) { return vmx_alloc_ipiv_pid_table(kvm); } @@ -4845,7 +4841,7 @@ static void __vmx_vcpu_reset(struct kvm_vcpu *vcpu) vmx->pi_desc.sn = 1; } -static void vmx_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event) +void vmx_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event) { struct vcpu_vmx *vmx = to_vmx(vcpu); @@ -4904,12 +4900,12 @@ static void vmx_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event) vmx_update_fb_clear_dis(vcpu, vmx); } -static void vmx_enable_irq_window(struct kvm_vcpu *vcpu) +void vmx_enable_irq_window(struct kvm_vcpu *vcpu) { exec_controls_setbit(to_vmx(vcpu), CPU_BASED_INTR_WINDOW_EXITING); } -static void vmx_enable_nmi_window(struct kvm_vcpu *vcpu) +void vmx_enable_nmi_window(struct kvm_vcpu *vcpu) { if (!enable_vnmi || vmcs_read32(GUEST_INTERRUPTIBILITY_INFO) & GUEST_INTR_STATE_STI) { @@ -4920,7 +4916,7 @@ static void vmx_enable_nmi_window(struct kvm_vcpu *vcpu) exec_controls_setbit(to_vmx(vcpu), CPU_BASED_NMI_WINDOW_EXITING); } -static void vmx_inject_irq(struct kvm_vcpu *vcpu, bool reinjected) +void vmx_inject_irq(struct kvm_vcpu *vcpu, bool reinjected) { struct vcpu_vmx *vmx = to_vmx(vcpu); uint32_t intr; @@ -4948,7 +4944,7 @@ static void vmx_inject_irq(struct kvm_vcpu *vcpu, bool reinjected) vmx_clear_hlt(vcpu); } -static void vmx_inject_nmi(struct kvm_vcpu *vcpu) +void vmx_inject_nmi(struct kvm_vcpu *vcpu) { struct vcpu_vmx *vmx = to_vmx(vcpu); @@ -5026,7 +5022,7 @@ bool vmx_nmi_blocked(struct kvm_vcpu *vcpu) GUEST_INTR_STATE_NMI)); } -static int vmx_nmi_allowed(struct kvm_vcpu *vcpu, bool for_injection) +int vmx_nmi_allowed(struct kvm_vcpu *vcpu, bool for_injection) { if (to_vmx(vcpu)->nested.nested_run_pending) return -EBUSY; @@ -5048,7 +5044,7 @@ bool vmx_interrupt_blocked(struct kvm_vcpu *vcpu) (GUEST_INTR_STATE_STI | GUEST_INTR_STATE_MOV_SS)); } -static int vmx_interrupt_allowed(struct kvm_vcpu *vcpu, bool for_injection) +int vmx_interrupt_allowed(struct kvm_vcpu *vcpu, bool for_injection) { if (to_vmx(vcpu)->nested.nested_run_pending) return -EBUSY; @@ -5063,7 +5059,7 @@ static int vmx_interrupt_allowed(struct kvm_vcpu *vcpu, bool for_injection) return !vmx_interrupt_blocked(vcpu); } -static int vmx_set_tss_addr(struct kvm *kvm, unsigned int addr) +int vmx_set_tss_addr(struct kvm *kvm, unsigned int addr) { void __user *ret; @@ -5083,7 +5079,7 @@ static int vmx_set_tss_addr(struct kvm *kvm, unsigned int addr) return init_rmode_tss(kvm, ret); } -static int vmx_set_identity_map_addr(struct kvm *kvm, u64 ident_addr) +int vmx_set_identity_map_addr(struct kvm *kvm, u64 ident_addr) { to_kvm_vmx(kvm)->ept_identity_map_addr = ident_addr; return 0; @@ -5369,8 +5365,7 @@ static int handle_io(struct kvm_vcpu *vcpu) return kvm_fast_pio(vcpu, size, port, in); } -static void -vmx_patch_hypercall(struct kvm_vcpu *vcpu, unsigned char *hypercall) +void vmx_patch_hypercall(struct kvm_vcpu *vcpu, unsigned char *hypercall) { /* * Patch in the VMCALL instruction: @@ -5579,7 +5574,7 @@ static int handle_dr(struct kvm_vcpu *vcpu) return kvm_complete_insn_gp(vcpu, err); } -static void vmx_sync_dirty_debug_regs(struct kvm_vcpu *vcpu) +void vmx_sync_dirty_debug_regs(struct kvm_vcpu *vcpu) { get_debugreg(vcpu->arch.db[0], 0); get_debugreg(vcpu->arch.db[1], 1); @@ -5598,7 +5593,7 @@ static void vmx_sync_dirty_debug_regs(struct kvm_vcpu *vcpu) set_debugreg(DR6_RESERVED, 6); } -static void vmx_set_dr7(struct kvm_vcpu *vcpu, unsigned long val) +void vmx_set_dr7(struct kvm_vcpu *vcpu, unsigned long val) { vmcs_writel(GUEST_DR7, val); } @@ -5869,7 +5864,7 @@ static int handle_invalid_guest_state(struct kvm_vcpu *vcpu) return 1; } -static int vmx_vcpu_pre_run(struct kvm_vcpu *vcpu) +int vmx_vcpu_pre_run(struct kvm_vcpu *vcpu) { if (vmx_emulation_required_with_pending_exception(vcpu)) { kvm_prepare_emulation_failure_exit(vcpu); @@ -6133,9 +6128,8 @@ static int (*kvm_vmx_exit_handlers[])(struct kvm_vcpu *vcpu) = { static const int kvm_vmx_max_exit_handlers = ARRAY_SIZE(kvm_vmx_exit_handlers); -static void vmx_get_exit_info(struct kvm_vcpu *vcpu, u32 *reason, - u64 *info1, u64 *info2, - u32 *intr_info, u32 *error_code) +void vmx_get_exit_info(struct kvm_vcpu *vcpu, u32 *reason, + u64 *info1, u64 *info2, u32 *intr_info, u32 *error_code) { struct vcpu_vmx *vmx = to_vmx(vcpu); @@ -6578,7 +6572,7 @@ static int __vmx_handle_exit(struct kvm_vcpu *vcpu, fastpath_t exit_fastpath) return 0; } -static int vmx_handle_exit(struct kvm_vcpu *vcpu, fastpath_t exit_fastpath) +int vmx_handle_exit(struct kvm_vcpu *vcpu, fastpath_t exit_fastpath) { int ret = __vmx_handle_exit(vcpu, exit_fastpath); @@ -6666,7 +6660,7 @@ static noinstr void vmx_l1d_flush(struct kvm_vcpu *vcpu) : "eax", "ebx", "ecx", "edx"); } -static void vmx_update_cr8_intercept(struct kvm_vcpu *vcpu, int tpr, int irr) +void vmx_update_cr8_intercept(struct kvm_vcpu *vcpu, int tpr, int irr) { struct vmcs12 *vmcs12 = get_vmcs12(vcpu); int tpr_threshold; @@ -6736,7 +6730,7 @@ void vmx_set_virtual_apic_mode(struct kvm_vcpu *vcpu) vmx_update_msr_bitmap_x2apic(vcpu); } -static void vmx_set_apic_access_page_addr(struct kvm_vcpu *vcpu) +void vmx_set_apic_access_page_addr(struct kvm_vcpu *vcpu) { const gfn_t gfn = APIC_DEFAULT_PHYS_BASE >> PAGE_SHIFT; struct kvm *kvm = vcpu->kvm; @@ -6805,7 +6799,7 @@ static void vmx_set_apic_access_page_addr(struct kvm_vcpu *vcpu) kvm_release_pfn_clean(pfn); } -static void vmx_hwapic_isr_update(int max_isr) +void vmx_hwapic_isr_update(int max_isr) { u16 status; u8 old; @@ -6839,7 +6833,7 @@ static void vmx_set_rvi(int vector) } } -static void vmx_hwapic_irr_update(struct kvm_vcpu *vcpu, int max_irr) +void vmx_hwapic_irr_update(struct kvm_vcpu *vcpu, int max_irr) { /* * When running L2, updating RVI is only relevant when @@ -6853,7 +6847,7 @@ static void vmx_hwapic_irr_update(struct kvm_vcpu *vcpu, int max_irr) vmx_set_rvi(max_irr); } -static int vmx_sync_pir_to_irr(struct kvm_vcpu *vcpu) +int vmx_sync_pir_to_irr(struct kvm_vcpu *vcpu) { struct vcpu_vmx *vmx = to_vmx(vcpu); int max_irr; @@ -6899,7 +6893,7 @@ static int vmx_sync_pir_to_irr(struct kvm_vcpu *vcpu) return max_irr; } -static void vmx_load_eoi_exitmap(struct kvm_vcpu *vcpu, u64 *eoi_exit_bitmap) +void vmx_load_eoi_exitmap(struct kvm_vcpu *vcpu, u64 *eoi_exit_bitmap) { if (!kvm_vcpu_apicv_active(vcpu)) return; @@ -6910,7 +6904,7 @@ static void vmx_load_eoi_exitmap(struct kvm_vcpu *vcpu, u64 *eoi_exit_bitmap) vmcs_write64(EOI_EXIT_BITMAP3, eoi_exit_bitmap[3]); } -static void vmx_apicv_pre_state_restore(struct kvm_vcpu *vcpu) +void vmx_apicv_pre_state_restore(struct kvm_vcpu *vcpu) { struct vcpu_vmx *vmx = to_vmx(vcpu); @@ -6973,7 +6967,7 @@ static void handle_external_interrupt_irqoff(struct kvm_vcpu *vcpu) vcpu->arch.at_instruction_boundary = true; } -static void vmx_handle_exit_irqoff(struct kvm_vcpu *vcpu) +void vmx_handle_exit_irqoff(struct kvm_vcpu *vcpu) { struct vcpu_vmx *vmx = to_vmx(vcpu); @@ -6990,7 +6984,7 @@ static void vmx_handle_exit_irqoff(struct kvm_vcpu *vcpu) * The kvm parameter can be NULL (module initialization, or invocation before * VM creation). Be sure to check the kvm parameter before using it. */ -static bool vmx_has_emulated_msr(struct kvm *kvm, u32 index) +bool vmx_has_emulated_msr(struct kvm *kvm, u32 index) { switch (index) { case MSR_IA32_SMBASE: @@ -7113,7 +7107,7 @@ static void vmx_complete_interrupts(struct vcpu_vmx *vmx) IDT_VECTORING_ERROR_CODE); } -static void vmx_cancel_injection(struct kvm_vcpu *vcpu) +void vmx_cancel_injection(struct kvm_vcpu *vcpu) { __vmx_complete_interrupts(vcpu, vmcs_read32(VM_ENTRY_INTR_INFO_FIELD), @@ -7268,7 +7262,7 @@ static noinstr void vmx_vcpu_enter_exit(struct kvm_vcpu *vcpu, guest_state_exit_irqoff(); } -static fastpath_t vmx_vcpu_run(struct kvm_vcpu *vcpu) +fastpath_t vmx_vcpu_run(struct kvm_vcpu *vcpu) { struct vcpu_vmx *vmx = to_vmx(vcpu); unsigned long cr3, cr4; @@ -7424,7 +7418,7 @@ static fastpath_t vmx_vcpu_run(struct kvm_vcpu *vcpu) return vmx_exit_handlers_fastpath(vcpu); } -static void vmx_vcpu_free(struct kvm_vcpu *vcpu) +void vmx_vcpu_free(struct kvm_vcpu *vcpu) { struct vcpu_vmx *vmx = to_vmx(vcpu); @@ -7435,7 +7429,7 @@ static void vmx_vcpu_free(struct kvm_vcpu *vcpu) free_loaded_vmcs(vmx->loaded_vmcs); } -static int vmx_vcpu_create(struct kvm_vcpu *vcpu) +int vmx_vcpu_create(struct kvm_vcpu *vcpu) { struct vmx_uret_msr *tsx_ctrl; struct vcpu_vmx *vmx; @@ -7541,7 +7535,7 @@ static int vmx_vcpu_create(struct kvm_vcpu *vcpu) return err; } -static bool vmx_is_vm_type_supported(unsigned long type) +bool vmx_is_vm_type_supported(unsigned long type) { /* TODO: Check if TDX is supported. */ return __kvm_is_vm_type_supported(type); @@ -7550,7 +7544,7 @@ static bool vmx_is_vm_type_supported(unsigned long type) #define L1TF_MSG_SMT "L1TF CPU bug present and SMT on, data leak possible. See CVE-2018-3646 and https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/l1tf.html for details.\n" #define L1TF_MSG_L1D "L1TF CPU bug present and virtualization mitigation disabled, data leak possible. See CVE-2018-3646 and https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/l1tf.html for details.\n" -static int vmx_vm_init(struct kvm *kvm) +int vmx_vm_init(struct kvm *kvm) { if (!ple_gap) kvm->arch.pause_in_guest = true; @@ -7581,7 +7575,7 @@ static int vmx_vm_init(struct kvm *kvm) return 0; } -static u8 vmx_get_mt_mask(struct kvm_vcpu *vcpu, gfn_t gfn, bool is_mmio) +u8 vmx_get_mt_mask(struct kvm_vcpu *vcpu, gfn_t gfn, bool is_mmio) { /* We wanted to honor guest CD/MTRR/PAT, but doing so could result in * memory aliases with conflicting memory types and sometimes MCEs. @@ -7753,7 +7747,7 @@ static void update_intel_pt_cfg(struct kvm_vcpu *vcpu) vmx->pt_desc.ctl_bitmask &= ~(0xfULL << (32 + i * 4)); } -static void vmx_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu) +void vmx_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu) { struct vcpu_vmx *vmx = to_vmx(vcpu); @@ -7907,7 +7901,7 @@ static __init void vmx_set_cpu_caps(void) kvm_cpu_cap_check_and_set(X86_FEATURE_WAITPKG); } -static void vmx_request_immediate_exit(struct kvm_vcpu *vcpu) +void vmx_request_immediate_exit(struct kvm_vcpu *vcpu) { to_vmx(vcpu)->req_immediate_exit = true; } @@ -7946,10 +7940,10 @@ static int vmx_check_intercept_io(struct kvm_vcpu *vcpu, return intercept ? X86EMUL_UNHANDLEABLE : X86EMUL_CONTINUE; } -static int vmx_check_intercept(struct kvm_vcpu *vcpu, - struct x86_instruction_info *info, - enum x86_intercept_stage stage, - struct x86_exception *exception) +int vmx_check_intercept(struct kvm_vcpu *vcpu, + struct x86_instruction_info *info, + enum x86_intercept_stage stage, + struct x86_exception *exception) { struct vmcs12 *vmcs12 = get_vmcs12(vcpu); @@ -8029,8 +8023,8 @@ static inline int u64_shl_div_u64(u64 a, unsigned int shift, return 0; } -static int vmx_set_hv_timer(struct kvm_vcpu *vcpu, u64 guest_deadline_tsc, - bool *expired) +int vmx_set_hv_timer(struct kvm_vcpu *vcpu, u64 guest_deadline_tsc, + bool *expired) { struct vcpu_vmx *vmx; u64 tscl, guest_tscl, delta_tsc, lapic_timer_advance_cycles; @@ -8069,13 +8063,13 @@ static int vmx_set_hv_timer(struct kvm_vcpu *vcpu, u64 guest_deadline_tsc, return 0; } -static void vmx_cancel_hv_timer(struct kvm_vcpu *vcpu) +void vmx_cancel_hv_timer(struct kvm_vcpu *vcpu) { to_vmx(vcpu)->hv_deadline_tsc = -1; } #endif -static void vmx_sched_in(struct kvm_vcpu *vcpu, int cpu) +void vmx_sched_in(struct kvm_vcpu *vcpu, int cpu) { if (!kvm_pause_in_guest(vcpu->kvm)) shrink_ple_window(vcpu); @@ -8104,7 +8098,7 @@ void vmx_update_cpu_dirty_logging(struct kvm_vcpu *vcpu) secondary_exec_controls_clearbit(vmx, SECONDARY_EXEC_ENABLE_PML); } -static void vmx_setup_mce(struct kvm_vcpu *vcpu) +void vmx_setup_mce(struct kvm_vcpu *vcpu) { if (vcpu->arch.mcg_cap & MCG_LMCE_P) to_vmx(vcpu)->msr_ia32_feature_control_valid_bits |= @@ -8115,7 +8109,7 @@ static void vmx_setup_mce(struct kvm_vcpu *vcpu) } #ifdef CONFIG_KVM_SMM -static int vmx_smi_allowed(struct kvm_vcpu *vcpu, bool for_injection) +int vmx_smi_allowed(struct kvm_vcpu *vcpu, bool for_injection) { /* we need a nested vmexit to enter SMM, postpone if run is pending */ if (to_vmx(vcpu)->nested.nested_run_pending) @@ -8123,7 +8117,7 @@ static int vmx_smi_allowed(struct kvm_vcpu *vcpu, bool for_injection) return !is_smm(vcpu); } -static int vmx_enter_smm(struct kvm_vcpu *vcpu, union kvm_smram *smram) +int vmx_enter_smm(struct kvm_vcpu *vcpu, union kvm_smram *smram) { struct vcpu_vmx *vmx = to_vmx(vcpu); @@ -8144,7 +8138,7 @@ static int vmx_enter_smm(struct kvm_vcpu *vcpu, union kvm_smram *smram) return 0; } -static int vmx_leave_smm(struct kvm_vcpu *vcpu, const union kvm_smram *smram) +int vmx_leave_smm(struct kvm_vcpu *vcpu, const union kvm_smram *smram) { struct vcpu_vmx *vmx = to_vmx(vcpu); int ret; @@ -8165,18 +8159,18 @@ static int vmx_leave_smm(struct kvm_vcpu *vcpu, const union kvm_smram *smram) return 0; } -static void vmx_enable_smi_window(struct kvm_vcpu *vcpu) +void vmx_enable_smi_window(struct kvm_vcpu *vcpu) { /* RSM will cause a vmexit anyway. */ } #endif -static bool vmx_apic_init_signal_blocked(struct kvm_vcpu *vcpu) +bool vmx_apic_init_signal_blocked(struct kvm_vcpu *vcpu) { return to_vmx(vcpu)->nested.vmxon && !is_guest_mode(vcpu); } -static void vmx_migrate_timers(struct kvm_vcpu *vcpu) +void vmx_migrate_timers(struct kvm_vcpu *vcpu) { if (is_guest_mode(vcpu)) { struct hrtimer *timer = &to_vmx(vcpu)->nested.preemption_timer; @@ -8186,7 +8180,7 @@ static void vmx_migrate_timers(struct kvm_vcpu *vcpu) } } -static void vmx_hardware_unsetup(void) +void vmx_hardware_unsetup(void) { kvm_set_posted_intr_wakeup_handler(NULL); @@ -8196,18 +8190,7 @@ static void vmx_hardware_unsetup(void) free_kvm_area(); } -#define VMX_REQUIRED_APICV_INHIBITS \ -( \ - BIT(APICV_INHIBIT_REASON_DISABLE)| \ - BIT(APICV_INHIBIT_REASON_ABSENT) | \ - BIT(APICV_INHIBIT_REASON_HYPERV) | \ - BIT(APICV_INHIBIT_REASON_BLOCKIRQ) | \ - BIT(APICV_INHIBIT_REASON_PHYSICAL_ID_ALIASED) | \ - BIT(APICV_INHIBIT_REASON_APIC_ID_MODIFIED) | \ - BIT(APICV_INHIBIT_REASON_APIC_BASE_MODIFIED) \ -) - -static void vmx_vm_destroy(struct kvm *kvm) +void vmx_vm_destroy(struct kvm *kvm) { struct kvm_vmx *kvm_vmx = to_kvm_vmx(kvm); @@ -8258,151 +8241,6 @@ gva_t vmx_get_untagged_addr(struct kvm_vcpu *vcpu, gva_t gva, unsigned int flags return (sign_extend64(gva, lam_bit) & ~BIT_ULL(63)) | (gva & BIT_ULL(63)); } -static struct kvm_x86_ops vmx_x86_ops __initdata = { - .name = KBUILD_MODNAME, - - .check_processor_compatibility = vmx_check_processor_compat, - - .hardware_unsetup = vmx_hardware_unsetup, - - .hardware_enable = vmx_hardware_enable, - .hardware_disable = vmx_hardware_disable, - .has_emulated_msr = vmx_has_emulated_msr, - - .is_vm_type_supported = vmx_is_vm_type_supported, - .vm_size = sizeof(struct kvm_vmx), - .vm_init = vmx_vm_init, - .vm_destroy = vmx_vm_destroy, - - .vcpu_precreate = vmx_vcpu_precreate, - .vcpu_create = vmx_vcpu_create, - .vcpu_free = vmx_vcpu_free, - .vcpu_reset = vmx_vcpu_reset, - - .prepare_switch_to_guest = vmx_prepare_switch_to_guest, - .vcpu_load = vmx_vcpu_load, - .vcpu_put = vmx_vcpu_put, - - .update_exception_bitmap = vmx_update_exception_bitmap, - .get_msr_feature = vmx_get_msr_feature, - .get_msr = vmx_get_msr, - .set_msr = vmx_set_msr, - .get_segment_base = vmx_get_segment_base, - .get_segment = vmx_get_segment, - .set_segment = vmx_set_segment, - .get_cpl = vmx_get_cpl, - .get_cs_db_l_bits = vmx_get_cs_db_l_bits, - .is_valid_cr0 = vmx_is_valid_cr0, - .set_cr0 = vmx_set_cr0, - .is_valid_cr4 = vmx_is_valid_cr4, - .set_cr4 = vmx_set_cr4, - .set_efer = vmx_set_efer, - .get_idt = vmx_get_idt, - .set_idt = vmx_set_idt, - .get_gdt = vmx_get_gdt, - .set_gdt = vmx_set_gdt, - .set_dr7 = vmx_set_dr7, - .sync_dirty_debug_regs = vmx_sync_dirty_debug_regs, - .cache_reg = vmx_cache_reg, - .get_rflags = vmx_get_rflags, - .set_rflags = vmx_set_rflags, - .get_if_flag = vmx_get_if_flag, - - .flush_tlb_all = vmx_flush_tlb_all, - .flush_tlb_current = vmx_flush_tlb_current, - .flush_tlb_gva = vmx_flush_tlb_gva, - .flush_tlb_guest = vmx_flush_tlb_guest, - - .vcpu_pre_run = vmx_vcpu_pre_run, - .vcpu_run = vmx_vcpu_run, - .handle_exit = vmx_handle_exit, - .skip_emulated_instruction = vmx_skip_emulated_instruction, - .update_emulated_instruction = vmx_update_emulated_instruction, - .set_interrupt_shadow = vmx_set_interrupt_shadow, - .get_interrupt_shadow = vmx_get_interrupt_shadow, - .patch_hypercall = vmx_patch_hypercall, - .inject_irq = vmx_inject_irq, - .inject_nmi = vmx_inject_nmi, - .inject_exception = vmx_inject_exception, - .cancel_injection = vmx_cancel_injection, - .interrupt_allowed = vmx_interrupt_allowed, - .nmi_allowed = vmx_nmi_allowed, - .get_nmi_mask = vmx_get_nmi_mask, - .set_nmi_mask = vmx_set_nmi_mask, - .enable_nmi_window = vmx_enable_nmi_window, - .enable_irq_window = vmx_enable_irq_window, - .update_cr8_intercept = vmx_update_cr8_intercept, - .set_virtual_apic_mode = vmx_set_virtual_apic_mode, - .set_apic_access_page_addr = vmx_set_apic_access_page_addr, - .refresh_apicv_exec_ctrl = vmx_refresh_apicv_exec_ctrl, - .load_eoi_exitmap = vmx_load_eoi_exitmap, - .apicv_pre_state_restore = vmx_apicv_pre_state_restore, - .required_apicv_inhibits = VMX_REQUIRED_APICV_INHIBITS, - .hwapic_irr_update = vmx_hwapic_irr_update, - .hwapic_isr_update = vmx_hwapic_isr_update, - .guest_apic_has_interrupt = vmx_guest_apic_has_interrupt, - .sync_pir_to_irr = vmx_sync_pir_to_irr, - .deliver_interrupt = vmx_deliver_interrupt, - .dy_apicv_has_pending_interrupt = pi_has_pending_interrupt, - - .set_tss_addr = vmx_set_tss_addr, - .set_identity_map_addr = vmx_set_identity_map_addr, - .get_mt_mask = vmx_get_mt_mask, - - .get_exit_info = vmx_get_exit_info, - - .vcpu_after_set_cpuid = vmx_vcpu_after_set_cpuid, - - .has_wbinvd_exit = cpu_has_vmx_wbinvd_exit, - - .get_l2_tsc_offset = vmx_get_l2_tsc_offset, - .get_l2_tsc_multiplier = vmx_get_l2_tsc_multiplier, - .write_tsc_offset = vmx_write_tsc_offset, - .write_tsc_multiplier = vmx_write_tsc_multiplier, - - .load_mmu_pgd = vmx_load_mmu_pgd, - - .check_intercept = vmx_check_intercept, - .handle_exit_irqoff = vmx_handle_exit_irqoff, - - .request_immediate_exit = vmx_request_immediate_exit, - - .sched_in = vmx_sched_in, - - .cpu_dirty_log_size = PML_ENTITY_NUM, - .update_cpu_dirty_logging = vmx_update_cpu_dirty_logging, - - .nested_ops = &vmx_nested_ops, - - .pi_update_irte = vmx_pi_update_irte, - .pi_start_assignment = vmx_pi_start_assignment, - -#ifdef CONFIG_X86_64 - .set_hv_timer = vmx_set_hv_timer, - .cancel_hv_timer = vmx_cancel_hv_timer, -#endif - - .setup_mce = vmx_setup_mce, - -#ifdef CONFIG_KVM_SMM - .smi_allowed = vmx_smi_allowed, - .enter_smm = vmx_enter_smm, - .leave_smm = vmx_leave_smm, - .enable_smi_window = vmx_enable_smi_window, -#endif - - .check_emulate_instruction = vmx_check_emulate_instruction, - .apic_init_signal_blocked = vmx_apic_init_signal_blocked, - .migrate_timers = vmx_migrate_timers, - - .msr_filter_changed = vmx_msr_filter_changed, - .complete_emulated_msr = kvm_complete_insn_gp, - - .vcpu_deliver_sipi_vector = kvm_vcpu_deliver_sipi_vector, - - .get_untagged_addr = vmx_get_untagged_addr, -}; - static unsigned int vmx_handle_intel_pt_intr(void) { struct kvm_vcpu *vcpu = kvm_get_running_vcpu(); @@ -8468,9 +8306,7 @@ static void __init vmx_setup_me_spte_mask(void) kvm_mmu_set_me_spte_mask(0, me_mask); } -static struct kvm_x86_init_ops vmx_init_ops __initdata; - -static __init int hardware_setup(void) +__init int vmx_hardware_setup(void) { unsigned long host_bndcfgs; struct desc_ptr dt; @@ -8539,16 +8375,16 @@ static __init int hardware_setup(void) * using the APIC_ACCESS_ADDR VMCS field. */ if (!flexpriority_enabled) - vmx_x86_ops.set_apic_access_page_addr = NULL; + vt_x86_ops.set_apic_access_page_addr = NULL; if (!cpu_has_vmx_tpr_shadow()) - vmx_x86_ops.update_cr8_intercept = NULL; + vt_x86_ops.update_cr8_intercept = NULL; #if IS_ENABLED(CONFIG_HYPERV) if (ms_hyperv.nested_features & HV_X64_NESTED_GUEST_MAPPING_FLUSH && enable_ept) { - vmx_x86_ops.flush_remote_tlbs = hv_flush_remote_tlbs; - vmx_x86_ops.flush_remote_tlbs_range = hv_flush_remote_tlbs_range; + vt_x86_ops.flush_remote_tlbs = hv_flush_remote_tlbs; + vt_x86_ops.flush_remote_tlbs_range = hv_flush_remote_tlbs_range; } #endif @@ -8563,7 +8399,7 @@ static __init int hardware_setup(void) if (!cpu_has_vmx_apicv()) enable_apicv = 0; if (!enable_apicv) - vmx_x86_ops.sync_pir_to_irr = NULL; + vt_x86_ops.sync_pir_to_irr = NULL; if (!enable_apicv || !cpu_has_vmx_ipiv()) enable_ipiv = false; @@ -8599,7 +8435,7 @@ static __init int hardware_setup(void) enable_pml = 0; if (!enable_pml) - vmx_x86_ops.cpu_dirty_log_size = 0; + vt_x86_ops.cpu_dirty_log_size = 0; if (!cpu_has_vmx_preemption_timer()) enable_preemption_timer = false; @@ -8624,9 +8460,9 @@ static __init int hardware_setup(void) } if (!enable_preemption_timer) { - vmx_x86_ops.set_hv_timer = NULL; - vmx_x86_ops.cancel_hv_timer = NULL; - vmx_x86_ops.request_immediate_exit = __kvm_request_immediate_exit; + vt_x86_ops.set_hv_timer = NULL; + vt_x86_ops.cancel_hv_timer = NULL; + vt_x86_ops.request_immediate_exit = __kvm_request_immediate_exit; } kvm_caps.supported_mce_cap |= MCG_LMCE_P; @@ -8637,9 +8473,9 @@ static __init int hardware_setup(void) if (!enable_ept || !enable_pmu || !cpu_has_vmx_intel_pt()) pt_mode = PT_MODE_SYSTEM; if (pt_mode == PT_MODE_HOST_GUEST) - vmx_init_ops.handle_intel_pt_intr = vmx_handle_intel_pt_intr; + vt_init_ops.handle_intel_pt_intr = vmx_handle_intel_pt_intr; else - vmx_init_ops.handle_intel_pt_intr = NULL; + vt_init_ops.handle_intel_pt_intr = NULL; setup_default_sgx_lepubkeyhash(); @@ -8662,14 +8498,6 @@ static __init int hardware_setup(void) return r; } -static struct kvm_x86_init_ops vmx_init_ops __initdata = { - .hardware_setup = hardware_setup, - .handle_intel_pt_intr = NULL, - - .runtime_ops = &vmx_x86_ops, - .pmu_ops = &intel_pmu_ops, -}; - static void vmx_cleanup_l1d_flush(void) { if (vmx_l1d_flush_pages) { @@ -8711,7 +8539,7 @@ static int __init vmx_init(void) */ hv_init_evmcs(); - r = kvm_x86_vendor_init(&vmx_init_ops); + r = kvm_x86_vendor_init(&vt_init_ops); if (r) return r; diff --git a/arch/x86/kvm/vmx/x86_ops.h b/arch/x86/kvm/vmx/x86_ops.h new file mode 100644 index 000000000000..2f8b6c43fe0f --- /dev/null +++ b/arch/x86/kvm/vmx/x86_ops.h @@ -0,0 +1,124 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef __KVM_X86_VMX_X86_OPS_H +#define __KVM_X86_VMX_X86_OPS_H + +#include + +#include "x86.h" + +extern struct kvm_x86_ops vt_x86_ops __initdata; +extern struct kvm_x86_init_ops vt_init_ops __initdata; + +__init int vmx_hardware_setup(void); +void vmx_hardware_unsetup(void); +int vmx_check_processor_compat(void); +int vmx_hardware_enable(void); +void vmx_hardware_disable(void); +bool vmx_is_vm_type_supported(unsigned long type); +int vmx_vm_init(struct kvm *kvm); +void vmx_vm_destroy(struct kvm *kvm); +int vmx_vcpu_precreate(struct kvm *kvm); +int vmx_vcpu_create(struct kvm_vcpu *vcpu); +int vmx_vcpu_pre_run(struct kvm_vcpu *vcpu); +fastpath_t vmx_vcpu_run(struct kvm_vcpu *vcpu); +void vmx_vcpu_free(struct kvm_vcpu *vcpu); +void vmx_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event); +void vmx_vcpu_load(struct kvm_vcpu *vcpu, int cpu); +void vmx_vcpu_put(struct kvm_vcpu *vcpu); +int vmx_handle_exit(struct kvm_vcpu *vcpu, fastpath_t exit_fastpath); +void vmx_handle_exit_irqoff(struct kvm_vcpu *vcpu); +int vmx_skip_emulated_instruction(struct kvm_vcpu *vcpu); +void vmx_update_emulated_instruction(struct kvm_vcpu *vcpu); +int vmx_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info); +#ifdef CONFIG_KVM_SMM +int vmx_smi_allowed(struct kvm_vcpu *vcpu, bool for_injection); +int vmx_enter_smm(struct kvm_vcpu *vcpu, union kvm_smram *smram); +int vmx_leave_smm(struct kvm_vcpu *vcpu, const union kvm_smram *smram); +void vmx_enable_smi_window(struct kvm_vcpu *vcpu); +#endif +int vmx_check_emulate_instruction(struct kvm_vcpu *vcpu, int emul_type, + void *insn, int insn_len); +int vmx_check_intercept(struct kvm_vcpu *vcpu, + struct x86_instruction_info *info, + enum x86_intercept_stage stage, + struct x86_exception *exception); +bool vmx_apic_init_signal_blocked(struct kvm_vcpu *vcpu); +void vmx_migrate_timers(struct kvm_vcpu *vcpu); +void vmx_set_virtual_apic_mode(struct kvm_vcpu *vcpu); +void vmx_apicv_pre_state_restore(struct kvm_vcpu *vcpu); +bool vmx_check_apicv_inhibit_reasons(enum kvm_apicv_inhibit reason); +void vmx_hwapic_irr_update(struct kvm_vcpu *vcpu, int max_irr); +void vmx_hwapic_isr_update(int max_isr); +bool vmx_guest_apic_has_interrupt(struct kvm_vcpu *vcpu); +int vmx_sync_pir_to_irr(struct kvm_vcpu *vcpu); +void vmx_deliver_interrupt(struct kvm_lapic *apic, int delivery_mode, + int trig_mode, int vector); +void vmx_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu); +bool vmx_has_emulated_msr(struct kvm *kvm, u32 index); +void vmx_msr_filter_changed(struct kvm_vcpu *vcpu); +void vmx_prepare_switch_to_guest(struct kvm_vcpu *vcpu); +void vmx_update_exception_bitmap(struct kvm_vcpu *vcpu); +int vmx_get_msr_feature(struct kvm_msr_entry *msr); +int vmx_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info); +u64 vmx_get_segment_base(struct kvm_vcpu *vcpu, int seg); +void vmx_get_segment(struct kvm_vcpu *vcpu, struct kvm_segment *var, int seg); +void vmx_set_segment(struct kvm_vcpu *vcpu, struct kvm_segment *var, int seg); +int vmx_get_cpl(struct kvm_vcpu *vcpu); +void vmx_get_cs_db_l_bits(struct kvm_vcpu *vcpu, int *db, int *l); +bool vmx_is_valid_cr0(struct kvm_vcpu *vcpu, unsigned long cr0); +void vmx_set_cr0(struct kvm_vcpu *vcpu, unsigned long cr0); +void vmx_load_mmu_pgd(struct kvm_vcpu *vcpu, hpa_t root_hpa, int root_level); +void vmx_set_cr4(struct kvm_vcpu *vcpu, unsigned long cr4); +bool vmx_is_valid_cr4(struct kvm_vcpu *vcpu, unsigned long cr4); +int vmx_set_efer(struct kvm_vcpu *vcpu, u64 efer); +void vmx_get_idt(struct kvm_vcpu *vcpu, struct desc_ptr *dt); +void vmx_set_idt(struct kvm_vcpu *vcpu, struct desc_ptr *dt); +void vmx_get_gdt(struct kvm_vcpu *vcpu, struct desc_ptr *dt); +void vmx_set_gdt(struct kvm_vcpu *vcpu, struct desc_ptr *dt); +void vmx_set_dr7(struct kvm_vcpu *vcpu, unsigned long val); +void vmx_sync_dirty_debug_regs(struct kvm_vcpu *vcpu); +void vmx_cache_reg(struct kvm_vcpu *vcpu, enum kvm_reg reg); +unsigned long vmx_get_rflags(struct kvm_vcpu *vcpu); +void vmx_set_rflags(struct kvm_vcpu *vcpu, unsigned long rflags); +bool vmx_get_if_flag(struct kvm_vcpu *vcpu); +void vmx_flush_tlb_all(struct kvm_vcpu *vcpu); +void vmx_flush_tlb_current(struct kvm_vcpu *vcpu); +void vmx_flush_tlb_gva(struct kvm_vcpu *vcpu, gva_t addr); +void vmx_flush_tlb_guest(struct kvm_vcpu *vcpu); +void vmx_set_interrupt_shadow(struct kvm_vcpu *vcpu, int mask); +u32 vmx_get_interrupt_shadow(struct kvm_vcpu *vcpu); +void vmx_patch_hypercall(struct kvm_vcpu *vcpu, unsigned char *hypercall); +void vmx_inject_irq(struct kvm_vcpu *vcpu, bool reinjected); +void vmx_inject_nmi(struct kvm_vcpu *vcpu); +void vmx_inject_exception(struct kvm_vcpu *vcpu); +void vmx_cancel_injection(struct kvm_vcpu *vcpu); +int vmx_interrupt_allowed(struct kvm_vcpu *vcpu, bool for_injection); +int vmx_nmi_allowed(struct kvm_vcpu *vcpu, bool for_injection); +bool vmx_get_nmi_mask(struct kvm_vcpu *vcpu); +void vmx_set_nmi_mask(struct kvm_vcpu *vcpu, bool masked); +void vmx_enable_nmi_window(struct kvm_vcpu *vcpu); +void vmx_enable_irq_window(struct kvm_vcpu *vcpu); +void vmx_update_cr8_intercept(struct kvm_vcpu *vcpu, int tpr, int irr); +void vmx_set_apic_access_page_addr(struct kvm_vcpu *vcpu); +void vmx_refresh_apicv_exec_ctrl(struct kvm_vcpu *vcpu); +void vmx_load_eoi_exitmap(struct kvm_vcpu *vcpu, u64 *eoi_exit_bitmap); +int vmx_set_tss_addr(struct kvm *kvm, unsigned int addr); +int vmx_set_identity_map_addr(struct kvm *kvm, u64 ident_addr); +u8 vmx_get_mt_mask(struct kvm_vcpu *vcpu, gfn_t gfn, bool is_mmio); +void vmx_get_exit_info(struct kvm_vcpu *vcpu, u32 *reason, + u64 *info1, u64 *info2, u32 *intr_info, u32 *error_code); +u64 vmx_get_l2_tsc_offset(struct kvm_vcpu *vcpu); +u64 vmx_get_l2_tsc_multiplier(struct kvm_vcpu *vcpu); +void vmx_write_tsc_offset(struct kvm_vcpu *vcpu); +void vmx_write_tsc_multiplier(struct kvm_vcpu *vcpu); +void vmx_request_immediate_exit(struct kvm_vcpu *vcpu); +void vmx_sched_in(struct kvm_vcpu *vcpu, int cpu); +void vmx_update_cpu_dirty_logging(struct kvm_vcpu *vcpu); +#ifdef CONFIG_X86_64 +int vmx_set_hv_timer(struct kvm_vcpu *vcpu, u64 guest_deadline_tsc, + bool *expired); +void vmx_cancel_hv_timer(struct kvm_vcpu *vcpu); +#endif +void vmx_setup_mce(struct kvm_vcpu *vcpu); + +#endif /* __KVM_X86_VMX_X86_OPS_H */ From patchwork Mon Feb 26 08:25:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571448 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8A92353E2D; Mon, 26 Feb 2024 08:27:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.7 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936073; cv=none; b=kXVgyIhMKh6BqkAOzyYYUQPxLB2g/6jhBAlhNr+HfUWmDzmOn8H8b++ZZfCNY+oIETTlRuiD/TS0UiRq7iGgm3lsJPOvzt1yR+dVu0W3t1jU0DE9kcdx7dX7Xze7KNVl3pn/Kn6JJrXTYGi6M5XJiKislCodLYjHf5wxaC03D8s= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936073; c=relaxed/simple; bh=ZOimDrQPGotChcrBFIpUTI6niKrsT5WJoO34ZaH7XfY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=hFrL8eztccLKdBRwFiiRuZnFULM0vU17xim/0XTPi/sx3pHtBHxUX23Btm0SPpOEE4x2K/7eIwp29Hc/fDfrxtEXuDHA4XX4w8bWqR1Fhb1m2P0HG1/6B11JmZ06Jp1YKvJb3w5TCDP+WzdJUFsRT8hJLdx9a1MZ1skiFh921LQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=ny3P4K5p; arc=none smtp.client-ip=192.198.163.7 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="ny3P4K5p" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936072; x=1740472072; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=ZOimDrQPGotChcrBFIpUTI6niKrsT5WJoO34ZaH7XfY=; b=ny3P4K5ptdLSYeHrYKkyoDcCZLnl+stGQM7hmahZPtx8heecEZzu4Ygt TVoNfUPbVaQ/56W+sT5768MyLSuOQ0HnJUxkXqtHAcZ9xw6AvMnaT7s22 3HJDJWx/H2zx1zhnfGBc5OEBgOvX1VbLxZaHhwzChhfeFjBAOE+YUUjA+ zZW46mzw8Ozgb2FPkyVAJJg1qrxPILDhHHXi1oFm2MfZ2fiHQlNE9qUTk weeJANb3ZU9q0kXZ9KnVKIHcSa5JNryOuIeHTofCV+0IJ0zB26tg+wZ7m ZifpJN5tsOcOKRaHTPnBNUNofQgNTwg9RBl+scyHGjHvaHPuM7Lc5vm1Q w==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="28631515" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="28631515" Received: from fmviesa007.fm.intel.com ([10.60.135.147]) by fmvoesa101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:27:45 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6474365" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by fmviesa007-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:27:45 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com, Yuan Yao Subject: [PATCH v19 021/130] KVM: x86/vmx: initialize loaded_vmcss_on_cpu in vmx_init() Date: Mon, 26 Feb 2024 00:25:23 -0800 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata vmx_hardware_disable() accesses loaded_vmcss_on_cpu via hardware_disable_all(). To allow hardware_enable/disable_all() before kvm_init(), initialize it in before kvm_x86_vendor_init() in vmx_init() so that tdx module initialization, hardware_setup method, can reference the variable. Signed-off-by: Isaku Yamahata Reviewed-by: Yuan Yao Reviewed-by: Binbin Wu --- v19: - Fix the subject to match the patch by Yuan v18: - Move the vmcss_on_cpu initialization from vmx_hardware_setup() to early point of vmx_init() by Binbin Signed-off-by: Isaku Yamahata --- arch/x86/kvm/vmx/vmx.c | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 434f5aaef030..8af0668e4dca 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -8539,6 +8539,10 @@ static int __init vmx_init(void) */ hv_init_evmcs(); + /* vmx_hardware_disable() accesses loaded_vmcss_on_cpu. */ + for_each_possible_cpu(cpu) + INIT_LIST_HEAD(&per_cpu(loaded_vmcss_on_cpu, cpu)); + r = kvm_x86_vendor_init(&vt_init_ops); if (r) return r; @@ -8554,11 +8558,8 @@ static int __init vmx_init(void) if (r) goto err_l1d_flush; - for_each_possible_cpu(cpu) { - INIT_LIST_HEAD(&per_cpu(loaded_vmcss_on_cpu, cpu)); - + for_each_possible_cpu(cpu) pi_init_cpu(cpu); - } cpu_emergency_register_virt_callback(vmx_emergency_disable); From patchwork Mon Feb 26 08:25:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571449 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0C4F45467C; Mon, 26 Feb 2024 08:27:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.7 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936074; cv=none; b=Tv41SgtyLme1ztiU6fwVCvawETCR2a+9hjPSIaE9MsXTkcHskioRs/uZTGc1x1JbhM43hWOTRR1ehnNnHlaSdtF39fn7JTjaXpQyL1PRd0Du/pf87ED3knfHnnpiXB07C5jj6vpSBWFm9ZVv028g9GmkOk6TxjwMKQ/uOMM3E+w= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936074; c=relaxed/simple; bh=nSKv+OYoU4BrKki36nr1+daFvMh+aOLZupHMr2KpcIc=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=XpJOAJoLmSscFFipdAdCxSVIcfjfSXTzxfFaty8qUfAByfhOdNszauN5PaTb6lYI4A3i+Q1ZsCSYp4pnQ6feQVOfxNwEd0oISO6l7Z0NqPbckMUXmvKdPWNxmdWfeHwtf5kKCzjYJwvSrJ5wofOUzPx5n+wP3rOp+kc93hBSOV0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=hRetNvSv; arc=none smtp.client-ip=192.198.163.7 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="hRetNvSv" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936072; x=1740472072; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=nSKv+OYoU4BrKki36nr1+daFvMh+aOLZupHMr2KpcIc=; b=hRetNvSvntCrIuJCDrAC7oirED+4yw9HWMevKYLTtvBMJqgPjUpO+QjL WIllc7ChBI0loQUYWq98Onbh3LU/uQ2UwSqyqFZrz4iPSkK8bKQUyQtAy HeDi/DaLyBj6+W8YCZJs7FPmPsgUy5TNbYqFM+QROELJDNkjUXG0mqduk gsZgMjNLCLLcbibqxcezsI3CoF/fB+gLhJAHq1hVfYPxYnZzTsOHenzML bjNc76+HaGqxjl4P+LhLi90LKQ4ngOf5M/rVUR0ixQayHyOq2nZLF3rPW 41npLHSUMoO+f4widNIov/kNN762jM8dxx8N1XK8QS4vaRHLyn3JKSCob A==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="28631528" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="28631528" Received: from fmviesa007.fm.intel.com ([10.60.135.147]) by fmvoesa101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:27:50 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6474397" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by fmviesa007-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:27:50 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 022/130] KVM: x86/vmx: Refactor KVM VMX module init/exit functions Date: Mon, 26 Feb 2024 00:25:24 -0800 Message-Id: <11d5ae6a1102a50b0e773fc7efd949bb0bd2b776.1708933498.git.isaku.yamahata@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata Currently, KVM VMX module initialization/exit functions are a single function each. Refactor KVM VMX module initialization functions into KVM common part and VMX part so that TDX specific part can be added cleanly. Opportunistically refactor module exit function as well. The current module initialization flow is, 0.) Check if VMX is supported, 1.) hyper-v specific initialization, 2.) system-wide x86 specific and vendor specific initialization, 3.) Final VMX specific system-wide initialization, 4.) calculate the sizes of VMX kvm structure and VMX vcpu structure, 5.) report those sizes to the KVM common layer and KVM common initialization Refactor the KVM VMX module initialization function into functions with a wrapper function to separate VMX logic in vmx.c from a file, main.c, common among VMX and TDX. Introduce a wrapper function for vmx_init(). The KVM architecture common layer allocates struct kvm with reported size for architecture-specific code. The KVM VMX module defines its structure as struct vmx_kvm { struct kvm; VMX specific members;} and uses it as struct vmx kvm. Similar for vcpu structure. TDX KVM patches will define TDX specific kvm and vcpu structures. The current module exit function is also a single function, a combination of VMX specific logic and common KVM logic. Refactor it into VMX specific logic and KVM common logic. This is just refactoring to keep the VMX specific logic in vmx.c from main.c. Signed-off-by: Isaku Yamahata Reviewed-by: Yin Fengwei --- v19: - Eliminate the unnecessary churn with vmx_hardware_setup() by Xiaoyao v18: - Move loaded_vmcss_on_cpu initialization to vt_init() before kvm_x86_vendor_init(). - added __init to an empty stub fucntion, hv_init_evmcs(). Signed-off-by: Isaku Yamahata --- arch/x86/kvm/vmx/main.c | 54 ++++++++++++++++++++++++++++++++++ arch/x86/kvm/vmx/vmx.c | 60 +++++--------------------------------- arch/x86/kvm/vmx/x86_ops.h | 14 +++++++++ 3 files changed, 75 insertions(+), 53 deletions(-) diff --git a/arch/x86/kvm/vmx/main.c b/arch/x86/kvm/vmx/main.c index eeb7a43b271d..18cecf12c7c8 100644 --- a/arch/x86/kvm/vmx/main.c +++ b/arch/x86/kvm/vmx/main.c @@ -167,3 +167,57 @@ struct kvm_x86_init_ops vt_init_ops __initdata = { .runtime_ops = &vt_x86_ops, .pmu_ops = &intel_pmu_ops, }; + +static int __init vt_init(void) +{ + unsigned int vcpu_size, vcpu_align; + int cpu, r; + + if (!kvm_is_vmx_supported()) + return -EOPNOTSUPP; + + /* + * Note, hv_init_evmcs() touches only VMX knobs, i.e. there's nothing + * to unwind if a later step fails. + */ + hv_init_evmcs(); + + /* vmx_hardware_disable() accesses loaded_vmcss_on_cpu. */ + for_each_possible_cpu(cpu) + INIT_LIST_HEAD(&per_cpu(loaded_vmcss_on_cpu, cpu)); + + r = kvm_x86_vendor_init(&vt_init_ops); + if (r) + return r; + + r = vmx_init(); + if (r) + goto err_vmx_init; + + /* + * Common KVM initialization _must_ come last, after this, /dev/kvm is + * exposed to userspace! + */ + vcpu_size = sizeof(struct vcpu_vmx); + vcpu_align = __alignof__(struct vcpu_vmx); + r = kvm_init(vcpu_size, vcpu_align, THIS_MODULE); + if (r) + goto err_kvm_init; + + return 0; + +err_kvm_init: + vmx_exit(); +err_vmx_init: + kvm_x86_vendor_exit(); + return r; +} +module_init(vt_init); + +static void vt_exit(void) +{ + kvm_exit(); + kvm_x86_vendor_exit(); + vmx_exit(); +} +module_exit(vt_exit); diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 8af0668e4dca..2fb1cd2e28a2 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -477,7 +477,7 @@ DEFINE_PER_CPU(struct vmcs *, current_vmcs); * We maintain a per-CPU linked-list of VMCS loaded on that CPU. This is needed * when a CPU is brought down, and we need to VMCLEAR all VMCSs loaded on it. */ -static DEFINE_PER_CPU(struct list_head, loaded_vmcss_on_cpu); +DEFINE_PER_CPU(struct list_head, loaded_vmcss_on_cpu); static DECLARE_BITMAP(vmx_vpid_bitmap, VMX_NR_VPIDS); static DEFINE_SPINLOCK(vmx_vpid_lock); @@ -537,7 +537,7 @@ static int hv_enable_l2_tlb_flush(struct kvm_vcpu *vcpu) return 0; } -static __init void hv_init_evmcs(void) +__init void hv_init_evmcs(void) { int cpu; @@ -573,7 +573,7 @@ static __init void hv_init_evmcs(void) } } -static void hv_reset_evmcs(void) +void hv_reset_evmcs(void) { struct hv_vp_assist_page *vp_ap; @@ -597,10 +597,6 @@ static void hv_reset_evmcs(void) vp_ap->current_nested_vmcs = 0; vp_ap->enlighten_vmentry = 0; } - -#else /* IS_ENABLED(CONFIG_HYPERV) */ -static void hv_init_evmcs(void) {} -static void hv_reset_evmcs(void) {} #endif /* IS_ENABLED(CONFIG_HYPERV) */ /* @@ -2743,7 +2739,7 @@ static bool __kvm_is_vmx_supported(void) return true; } -static bool kvm_is_vmx_supported(void) +bool kvm_is_vmx_supported(void) { bool supported; @@ -8508,7 +8504,7 @@ static void vmx_cleanup_l1d_flush(void) l1tf_vmx_mitigation = VMENTER_L1D_FLUSH_AUTO; } -static void __vmx_exit(void) +void vmx_exit(void) { allow_smaller_maxphyaddr = false; @@ -8517,36 +8513,10 @@ static void __vmx_exit(void) vmx_cleanup_l1d_flush(); } -static void vmx_exit(void) -{ - kvm_exit(); - kvm_x86_vendor_exit(); - - __vmx_exit(); -} -module_exit(vmx_exit); - -static int __init vmx_init(void) +int __init vmx_init(void) { int r, cpu; - if (!kvm_is_vmx_supported()) - return -EOPNOTSUPP; - - /* - * Note, hv_init_evmcs() touches only VMX knobs, i.e. there's nothing - * to unwind if a later step fails. - */ - hv_init_evmcs(); - - /* vmx_hardware_disable() accesses loaded_vmcss_on_cpu. */ - for_each_possible_cpu(cpu) - INIT_LIST_HEAD(&per_cpu(loaded_vmcss_on_cpu, cpu)); - - r = kvm_x86_vendor_init(&vt_init_ops); - if (r) - return r; - /* * Must be called after common x86 init so enable_ept is properly set * up. Hand the parameter mitigation value in which was stored in @@ -8556,7 +8526,7 @@ static int __init vmx_init(void) */ r = vmx_setup_l1d_flush(vmentry_l1d_flush_param); if (r) - goto err_l1d_flush; + return r; for_each_possible_cpu(cpu) pi_init_cpu(cpu); @@ -8573,21 +8543,5 @@ static int __init vmx_init(void) if (!enable_ept) allow_smaller_maxphyaddr = true; - /* - * Common KVM initialization _must_ come last, after this, /dev/kvm is - * exposed to userspace! - */ - r = kvm_init(sizeof(struct vcpu_vmx), __alignof__(struct vcpu_vmx), - THIS_MODULE); - if (r) - goto err_kvm_init; - return 0; - -err_kvm_init: - __vmx_exit(); -err_l1d_flush: - kvm_x86_vendor_exit(); - return r; } -module_init(vmx_init); diff --git a/arch/x86/kvm/vmx/x86_ops.h b/arch/x86/kvm/vmx/x86_ops.h index 2f8b6c43fe0f..b936388853ab 100644 --- a/arch/x86/kvm/vmx/x86_ops.h +++ b/arch/x86/kvm/vmx/x86_ops.h @@ -6,6 +6,20 @@ #include "x86.h" +#if IS_ENABLED(CONFIG_HYPERV) +__init void hv_init_evmcs(void); +void hv_reset_evmcs(void); +#else /* IS_ENABLED(CONFIG_HYPERV) */ +static inline __init void hv_init_evmcs(void) {} +static inline void hv_reset_evmcs(void) {} +#endif /* IS_ENABLED(CONFIG_HYPERV) */ + +DECLARE_PER_CPU(struct list_head, loaded_vmcss_on_cpu); + +bool kvm_is_vmx_supported(void); +int __init vmx_init(void); +void vmx_exit(void); + extern struct kvm_x86_ops vt_x86_ops __initdata; extern struct kvm_x86_init_ops vt_init_ops __initdata; From patchwork Mon Feb 26 08:25:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571451 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3964B54760; Mon, 26 Feb 2024 08:27:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.13 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936075; cv=none; b=rtvOsN3eoTcekuJ/cUckUc5zV+756TVq0yeCU7neXjnCBFSxjxAuBrWlbl/LtgfiZuvn6RQmzmJeZ2FvkxNr5OvE88pgxoSOuSTbSIfV/DD7fzQqR9dGwbXLlaMBPAbLiU02eNRb3mGX7E+apiKufl8f8a78GhzRfRfPS+y9X9s= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936075; c=relaxed/simple; bh=O0sWvFfgIW927Ww+UNgjiWxMLfLIE4xsplwGoeS5rss=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=hKJhWlO4IKMzH57omB8vVQvUU9VBR3dn0diouN6pdl7qB2laNMoxg3UJPkT2TYwu7ydf3l0wCPpykVIVR3wbU/fOR198dIF2DZt6+KggiXewxc7E9MKwNjgBU7m5/K0PLWPQFOZ61M4c349w3xrLuYRrbUXYR0FPb6WqcrloMiI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=Hzyl8gCv; arc=none smtp.client-ip=192.198.163.13 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="Hzyl8gCv" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936072; x=1740472072; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=O0sWvFfgIW927Ww+UNgjiWxMLfLIE4xsplwGoeS5rss=; b=Hzyl8gCvpIY4cxae7ng9qWu2z4RQLUI/4umtGR/m2G7cZMId8OW4osC0 ff4VXcRb20wSJSAz+9r6TWiZNNheRyaaG3MsPSxlsjx9wK4zXhEYRVXy3 5qp9f4jZwwW88QQOR02wEqvTD2AxJq+NIxlC4BqJnCVBW4a9BNlnPMydB RhSwB5PmFQ+sNHE6/DpNDridg1Hz7iZPkVNQ6xJwCyXEX5BRN/hiXO4v8 Ozbb8Sx101C642iE5Pq24q+gWxDsHeGkhT4XVn++qVaQ5GLmqZvSehqT9 l63Vre1W9MXNhQq8TjN3SvHfUFiazMLdrEl6az6L1uYAder1LsGcupEEX g==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="6155236" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6155236" Received: from orviesa009.jf.intel.com ([10.64.159.149]) by fmvoesa107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:27:51 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6615437" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa009-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:27:51 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 023/130] KVM: TDX: Initialize the TDX module when loading the KVM intel kernel module Date: Mon, 26 Feb 2024 00:25:25 -0800 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata TDX requires several initialization steps for KVM to create guest TDs. Detect CPU feature, enable VMX (TDX is based on VMX) on all online CPUs, detect the TDX module availability, initialize it and disable VMX. To enable/disable VMX on all online CPUs, utilize vmx_hardware_enable/disable(). The method also initializes each CPU for TDX. TDX requires calling a TDX initialization function per logical processor (LP) before the LP uses TDX. When the CPU is becoming online, call the TDX LP initialization API. If it fails to initialize TDX, refuse CPU online for simplicity instead of TDX avoiding the failed LP. There are several options on when to initialize the TDX module. A.) kernel module loading time, B.) the first guest TD creation time. A.) was chosen. With B.), a user may hit an error of the TDX initialization when trying to create the first guest TD. The machine that fails to initialize the TDX module can't boot any guest TD further. Such failure is undesirable and a surprise because the user expects that the machine can accommodate guest TD, but not. So A.) is better than B.). Introduce a module parameter, kvm_intel.tdx, to explicitly enable TDX KVM support. It's off by default to keep the same behavior for those who don't use TDX. Implement hardware_setup method to detect TDX feature of CPU and initialize TDX module. Suggested-by: Sean Christopherson Signed-off-by: Isaku Yamahata --- v19: - fixed vt_hardware_enable() to use vmx_hardware_enable() - renamed vmx_tdx_enabled => tdx_enabled - renamed vmx_tdx_on() => tdx_on() v18: - Added comment in vt_hardware_enable() by Binbin. Signed-off-by: Isaku Yamahata --- arch/x86/kvm/Makefile | 1 + arch/x86/kvm/vmx/main.c | 19 ++++++++- arch/x86/kvm/vmx/tdx.c | 84 ++++++++++++++++++++++++++++++++++++++ arch/x86/kvm/vmx/x86_ops.h | 6 +++ 4 files changed, 109 insertions(+), 1 deletion(-) create mode 100644 arch/x86/kvm/vmx/tdx.c diff --git a/arch/x86/kvm/Makefile b/arch/x86/kvm/Makefile index 274df24b647f..5b85ef84b2e9 100644 --- a/arch/x86/kvm/Makefile +++ b/arch/x86/kvm/Makefile @@ -24,6 +24,7 @@ kvm-intel-y += vmx/vmx.o vmx/vmenter.o vmx/pmu_intel.o vmx/vmcs12.o \ kvm-intel-$(CONFIG_X86_SGX_KVM) += vmx/sgx.o kvm-intel-$(CONFIG_KVM_HYPERV) += vmx/hyperv.o vmx/hyperv_evmcs.o +kvm-intel-$(CONFIG_INTEL_TDX_HOST) += vmx/tdx.o kvm-amd-y += svm/svm.o svm/vmenter.o svm/pmu.o svm/nested.o svm/avic.o \ svm/sev.o diff --git a/arch/x86/kvm/vmx/main.c b/arch/x86/kvm/vmx/main.c index 18cecf12c7c8..18aef6e23aab 100644 --- a/arch/x86/kvm/vmx/main.c +++ b/arch/x86/kvm/vmx/main.c @@ -6,6 +6,22 @@ #include "nested.h" #include "pmu.h" +static bool enable_tdx __ro_after_init; +module_param_named(tdx, enable_tdx, bool, 0444); + +static __init int vt_hardware_setup(void) +{ + int ret; + + ret = vmx_hardware_setup(); + if (ret) + return ret; + + enable_tdx = enable_tdx && !tdx_hardware_setup(&vt_x86_ops); + + return 0; +} + #define VMX_REQUIRED_APICV_INHIBITS \ (BIT(APICV_INHIBIT_REASON_DISABLE)| \ BIT(APICV_INHIBIT_REASON_ABSENT) | \ @@ -22,6 +38,7 @@ struct kvm_x86_ops vt_x86_ops __initdata = { .hardware_unsetup = vmx_hardware_unsetup, + /* TDX cpu enablement is done by tdx_hardware_setup(). */ .hardware_enable = vmx_hardware_enable, .hardware_disable = vmx_hardware_disable, .has_emulated_msr = vmx_has_emulated_msr, @@ -161,7 +178,7 @@ struct kvm_x86_ops vt_x86_ops __initdata = { }; struct kvm_x86_init_ops vt_init_ops __initdata = { - .hardware_setup = vmx_hardware_setup, + .hardware_setup = vt_hardware_setup, .handle_intel_pt_intr = NULL, .runtime_ops = &vt_x86_ops, diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c new file mode 100644 index 000000000000..43c504fb4fed --- /dev/null +++ b/arch/x86/kvm/vmx/tdx.c @@ -0,0 +1,84 @@ +// SPDX-License-Identifier: GPL-2.0 +#include + +#include + +#include "capabilities.h" +#include "x86_ops.h" +#include "x86.h" + +#undef pr_fmt +#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt + +static int __init tdx_module_setup(void) +{ + int ret; + + ret = tdx_enable(); + if (ret) { + pr_info("Failed to initialize TDX module.\n"); + return ret; + } + + return 0; +} + +struct tdx_enabled { + cpumask_var_t enabled; + atomic_t err; +}; + +static void __init tdx_on(void *_enable) +{ + struct tdx_enabled *enable = _enable; + int r; + + r = vmx_hardware_enable(); + if (!r) { + cpumask_set_cpu(smp_processor_id(), enable->enabled); + r = tdx_cpu_enable(); + } + if (r) + atomic_set(&enable->err, r); +} + +static void __init vmx_off(void *_enabled) +{ + cpumask_var_t *enabled = (cpumask_var_t *)_enabled; + + if (cpumask_test_cpu(smp_processor_id(), *enabled)) + vmx_hardware_disable(); +} + +int __init tdx_hardware_setup(struct kvm_x86_ops *x86_ops) +{ + struct tdx_enabled enable = { + .err = ATOMIC_INIT(0), + }; + int r = 0; + + if (!enable_ept) { + pr_warn("Cannot enable TDX with EPT disabled\n"); + return -EINVAL; + } + + if (!zalloc_cpumask_var(&enable.enabled, GFP_KERNEL)) { + r = -ENOMEM; + goto out; + } + + /* tdx_enable() in tdx_module_setup() requires cpus lock. */ + cpus_read_lock(); + on_each_cpu(tdx_on, &enable, true); /* TDX requires vmxon. */ + r = atomic_read(&enable.err); + if (!r) + r = tdx_module_setup(); + else + r = -EIO; + on_each_cpu(vmx_off, &enable.enabled, true); + cpus_read_unlock(); + free_cpumask_var(enable.enabled); + +out: + return r; +} diff --git a/arch/x86/kvm/vmx/x86_ops.h b/arch/x86/kvm/vmx/x86_ops.h index b936388853ab..346289a2a01c 100644 --- a/arch/x86/kvm/vmx/x86_ops.h +++ b/arch/x86/kvm/vmx/x86_ops.h @@ -135,4 +135,10 @@ void vmx_cancel_hv_timer(struct kvm_vcpu *vcpu); #endif void vmx_setup_mce(struct kvm_vcpu *vcpu); +#ifdef CONFIG_INTEL_TDX_HOST +int __init tdx_hardware_setup(struct kvm_x86_ops *x86_ops); +#else +static inline int tdx_hardware_setup(struct kvm_x86_ops *x86_ops) { return -EOPNOTSUPP; } +#endif + #endif /* __KVM_X86_VMX_X86_OPS_H */ From patchwork Mon Feb 26 08:25:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571452 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8B3F854FB0; Mon, 26 Feb 2024 08:27:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.13 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936075; cv=none; b=qxu6Wn0FRrHDAzyqugHYK91UN33euYL0OzqmpeU+08sJEqL0Rad62248IX/3NZceuUNbP8gbn7d8b4YcKvctLKqnNv5AxJ9hH66hR2b86GiXiw5xKBfhcreJOVJlrDNhgW6YDkGY6ZegRYg01yzdcPsFYjaSkdQzKNEKANQ/MiE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936075; c=relaxed/simple; bh=t8ZVY21QvgWkqCaFflB5Ky+24TTmKzW3GvfNfpp1CgI=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=cXAhaZxf+PLqiz8v2NulK7WOuUaJKNMpg+RBSwO7lG9I3RQvS6A1oyn1tAq8m8wviLE6NqfGElCuvmpP0fTiNzQU4odndlOH6YV4AwJS/LKpyMvXtAzD7ktjiSMz7/qs/c/FRMCZXlV/ZDFupCyj10oGnCN7QkN9dESEfZPEfC8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=WC0e7zAt; arc=none smtp.client-ip=192.198.163.13 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="WC0e7zAt" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936073; x=1740472073; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=t8ZVY21QvgWkqCaFflB5Ky+24TTmKzW3GvfNfpp1CgI=; b=WC0e7zAtCJC0BzdkTeCrcqmVEQpLEw0reBZ//ZZVZqHzn2JE6KMzHY9r E57vixUr4CbFcem13VbJqGzdtnyZM9NkO1LlQd1fIN9jZbbhEvM/Y/y+J JOlIghoTFh8Vhp4tbOuYoiQ9nbkRV6NXnro8s81v+PryHlYD7SMuDvoYR mGjm55NxZ/v04dkypIbQkcYAmDEJFJ6TYOJVTLSxDbYscqkxsebPjdm7v zGs8wrO4qvF1Mx1+ormZWDzwvZjp4UEiWaLPEADyGjozkw8bxaiCOlbaR C462iROSAB05aU5xX14HNvNzsM7bDt5b8zTrFKi0/eI1y7mpzQHsW/tFF A==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="6155242" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6155242" Received: from orviesa009.jf.intel.com ([10.64.159.149]) by fmvoesa107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:27:52 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6615452" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa009-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:27:51 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 024/130] KVM: TDX: Add placeholders for TDX VM/vcpu structure Date: Mon, 26 Feb 2024 00:25:26 -0800 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata Add placeholders TDX VM/vcpu structure that overlays with VMX VM/vcpu structures. Initialize VM structure size and vcpu size/align so that x86 KVM common code knows those size irrespective of VMX or TDX. Those structures will be populated as guest creation logic develops. Add helper functions to check if the VM is guest TD and add conversion functions between KVM VM/VCPU and TDX VM/VCPU. Signed-off-by: Isaku Yamahata --- v19: - correctly update ops.vm_size, vcpu_size and, vcpu_align by Xiaoyao v14 -> v15: - use KVM_X86_TDX_VM Signed-off-by: Isaku Yamahata --- arch/x86/kvm/vmx/main.c | 14 ++++++++++++ arch/x86/kvm/vmx/tdx.c | 1 + arch/x86/kvm/vmx/tdx.h | 50 +++++++++++++++++++++++++++++++++++++++++ 3 files changed, 65 insertions(+) create mode 100644 arch/x86/kvm/vmx/tdx.h diff --git a/arch/x86/kvm/vmx/main.c b/arch/x86/kvm/vmx/main.c index 18aef6e23aab..e11edbd19e7c 100644 --- a/arch/x86/kvm/vmx/main.c +++ b/arch/x86/kvm/vmx/main.c @@ -5,6 +5,7 @@ #include "vmx.h" #include "nested.h" #include "pmu.h" +#include "tdx.h" static bool enable_tdx __ro_after_init; module_param_named(tdx, enable_tdx, bool, 0444); @@ -18,6 +19,9 @@ static __init int vt_hardware_setup(void) return ret; enable_tdx = enable_tdx && !tdx_hardware_setup(&vt_x86_ops); + if (enable_tdx) + vt_x86_ops.vm_size = max_t(unsigned int, vt_x86_ops.vm_size, + sizeof(struct kvm_tdx)); return 0; } @@ -215,8 +219,18 @@ static int __init vt_init(void) * Common KVM initialization _must_ come last, after this, /dev/kvm is * exposed to userspace! */ + /* + * kvm_x86_ops is updated with vt_x86_ops. vt_x86_ops.vm_size must + * be set before kvm_x86_vendor_init(). + */ vcpu_size = sizeof(struct vcpu_vmx); vcpu_align = __alignof__(struct vcpu_vmx); + if (enable_tdx) { + vcpu_size = max_t(unsigned int, vcpu_size, + sizeof(struct vcpu_tdx)); + vcpu_align = max_t(unsigned int, vcpu_align, + __alignof__(struct vcpu_tdx)); + } r = kvm_init(vcpu_size, vcpu_align, THIS_MODULE); if (r) goto err_kvm_init; diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c index 43c504fb4fed..14ef0ccd8f1a 100644 --- a/arch/x86/kvm/vmx/tdx.c +++ b/arch/x86/kvm/vmx/tdx.c @@ -6,6 +6,7 @@ #include "capabilities.h" #include "x86_ops.h" #include "x86.h" +#include "tdx.h" #undef pr_fmt #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt diff --git a/arch/x86/kvm/vmx/tdx.h b/arch/x86/kvm/vmx/tdx.h new file mode 100644 index 000000000000..473013265bd8 --- /dev/null +++ b/arch/x86/kvm/vmx/tdx.h @@ -0,0 +1,50 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef __KVM_X86_TDX_H +#define __KVM_X86_TDX_H + +#ifdef CONFIG_INTEL_TDX_HOST +struct kvm_tdx { + struct kvm kvm; + /* TDX specific members follow. */ +}; + +struct vcpu_tdx { + struct kvm_vcpu vcpu; + /* TDX specific members follow. */ +}; + +static inline bool is_td(struct kvm *kvm) +{ + return kvm->arch.vm_type == KVM_X86_TDX_VM; +} + +static inline bool is_td_vcpu(struct kvm_vcpu *vcpu) +{ + return is_td(vcpu->kvm); +} + +static inline struct kvm_tdx *to_kvm_tdx(struct kvm *kvm) +{ + return container_of(kvm, struct kvm_tdx, kvm); +} + +static inline struct vcpu_tdx *to_tdx(struct kvm_vcpu *vcpu) +{ + return container_of(vcpu, struct vcpu_tdx, vcpu); +} +#else +struct kvm_tdx { + struct kvm kvm; +}; + +struct vcpu_tdx { + struct kvm_vcpu vcpu; +}; + +static inline bool is_td(struct kvm *kvm) { return false; } +static inline bool is_td_vcpu(struct kvm_vcpu *vcpu) { return false; } +static inline struct kvm_tdx *to_kvm_tdx(struct kvm *kvm) { return NULL; } +static inline struct vcpu_tdx *to_tdx(struct kvm_vcpu *vcpu) { return NULL; } +#endif /* CONFIG_INTEL_TDX_HOST */ + +#endif /* __KVM_X86_TDX_H */ From patchwork Mon Feb 26 08:25:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571453 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A00CF55C06; Mon, 26 Feb 2024 08:27:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.13 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936076; cv=none; b=hI2QR7UQ5YXnjYu/BMo7Mphc+/UhHqSYd0hQ4f/nvp3yG+L23QO11mGtwuYZDptd9/CGI/kH7oRQtmG1lruObwssEKEykteuPlOohfYEMN1itqR4d1SPlQjfZFr02il+HzOpo85i8SYi00oy6WwzvobqBK/NooDjhLVp1AV1Wd0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936076; c=relaxed/simple; bh=KEs01LJAB8yu/AF31xmX6BSRiT45bPjmbTfdFCoFSQE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=NxzOwr4op9Hf2jXL1rKMRRFfnXE6+EZ+ObTgYOl6RTMkOT3ZXWbuBp+Gal5uxdTCset0umXIu8gigkkmBwbRB5qeJ1C4c87GSku/ldfE9ub1pui3B+/j1Dm3ntTNLXWCALfhevx2MybHekeYS9E5UkrLnDNwOtG3Aq51drxrbBQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=XoV082Fl; arc=none smtp.client-ip=192.198.163.13 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="XoV082Fl" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936074; x=1740472074; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=KEs01LJAB8yu/AF31xmX6BSRiT45bPjmbTfdFCoFSQE=; b=XoV082FlBVs+Aml+pIb9qZKHq3YPDo48mbpqGK/hWbfVPpjVs1ZDS1x3 HKYaaYBpTTU1iKC4AkyPlaOI+zVfBpTcMRUVDr0aKKtMpcm4AhsS4njIW aQSpX13WiHJ/wKthOnToL8kRkhxudQJq3yqGRiN5OSPuQrOJcMZBHCW1y IPi+JaQzY6lHX+kMm0Zy3karRrP3d3w5GkSbPfwdF6slp8A7rd6zC6j5t wjhzwmggNjhAYRxf6/tLydDpv8Fd9pen6AW+CkaB5nTonuS3BbZQGESf0 VtN7rIoZDJMvsZCoNTQB+LxgfUUCq6XEK/TvKyFURE002TmIN2+Z/jnIz Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="6155246" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6155246" Received: from orviesa009.jf.intel.com ([10.64.159.149]) by fmvoesa107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:27:52 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6615467" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa009-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:27:52 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 025/130] KVM: TDX: Make TDX VM type supported Date: Mon, 26 Feb 2024 00:25:27 -0800 Message-Id: <5159c2b6a23560e9d8252c1311dd91d328e58871.1708933498.git.isaku.yamahata@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata NOTE: This patch is in position of the patch series for developers to be able to test codes during the middle of the patch series although this patch series doesn't provide functional features until the all the patches of this patch series. When merging this patch series, this patch can be moved to the end. As first step TDX VM support, return that TDX VM type supported to device model, e.g. qemu. The callback to create guest TD is vm_init callback for KVM_CREATE_VM. Signed-off-by: Isaku Yamahata --- arch/x86/kvm/vmx/main.c | 18 ++++++++++++++++-- arch/x86/kvm/vmx/tdx.c | 6 ++++++ arch/x86/kvm/vmx/vmx.c | 6 ------ arch/x86/kvm/vmx/x86_ops.h | 3 ++- 4 files changed, 24 insertions(+), 9 deletions(-) diff --git a/arch/x86/kvm/vmx/main.c b/arch/x86/kvm/vmx/main.c index e11edbd19e7c..fa19682b366c 100644 --- a/arch/x86/kvm/vmx/main.c +++ b/arch/x86/kvm/vmx/main.c @@ -10,6 +10,12 @@ static bool enable_tdx __ro_after_init; module_param_named(tdx, enable_tdx, bool, 0444); +static bool vt_is_vm_type_supported(unsigned long type) +{ + return __kvm_is_vm_type_supported(type) || + (enable_tdx && tdx_is_vm_type_supported(type)); +} + static __init int vt_hardware_setup(void) { int ret; @@ -26,6 +32,14 @@ static __init int vt_hardware_setup(void) return 0; } +static int vt_vm_init(struct kvm *kvm) +{ + if (is_td(kvm)) + return -EOPNOTSUPP; /* Not ready to create guest TD yet. */ + + return vmx_vm_init(kvm); +} + #define VMX_REQUIRED_APICV_INHIBITS \ (BIT(APICV_INHIBIT_REASON_DISABLE)| \ BIT(APICV_INHIBIT_REASON_ABSENT) | \ @@ -47,9 +61,9 @@ struct kvm_x86_ops vt_x86_ops __initdata = { .hardware_disable = vmx_hardware_disable, .has_emulated_msr = vmx_has_emulated_msr, - .is_vm_type_supported = vmx_is_vm_type_supported, + .is_vm_type_supported = vt_is_vm_type_supported, .vm_size = sizeof(struct kvm_vmx), - .vm_init = vmx_vm_init, + .vm_init = vt_vm_init, .vm_destroy = vmx_vm_destroy, .vcpu_precreate = vmx_vcpu_precreate, diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c index 14ef0ccd8f1a..a7e096fd8361 100644 --- a/arch/x86/kvm/vmx/tdx.c +++ b/arch/x86/kvm/vmx/tdx.c @@ -24,6 +24,12 @@ static int __init tdx_module_setup(void) return 0; } +bool tdx_is_vm_type_supported(unsigned long type) +{ + /* enable_tdx check is done by the caller. */ + return type == KVM_X86_TDX_VM; +} + struct tdx_enabled { cpumask_var_t enabled; atomic_t err; diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 2fb1cd2e28a2..d928acc15d0f 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -7531,12 +7531,6 @@ int vmx_vcpu_create(struct kvm_vcpu *vcpu) return err; } -bool vmx_is_vm_type_supported(unsigned long type) -{ - /* TODO: Check if TDX is supported. */ - return __kvm_is_vm_type_supported(type); -} - #define L1TF_MSG_SMT "L1TF CPU bug present and SMT on, data leak possible. See CVE-2018-3646 and https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/l1tf.html for details.\n" #define L1TF_MSG_L1D "L1TF CPU bug present and virtualization mitigation disabled, data leak possible. See CVE-2018-3646 and https://www.kernel.org/doc/html/latest/admin-guide/hw-vuln/l1tf.html for details.\n" diff --git a/arch/x86/kvm/vmx/x86_ops.h b/arch/x86/kvm/vmx/x86_ops.h index 346289a2a01c..f4da88a228d0 100644 --- a/arch/x86/kvm/vmx/x86_ops.h +++ b/arch/x86/kvm/vmx/x86_ops.h @@ -28,7 +28,6 @@ void vmx_hardware_unsetup(void); int vmx_check_processor_compat(void); int vmx_hardware_enable(void); void vmx_hardware_disable(void); -bool vmx_is_vm_type_supported(unsigned long type); int vmx_vm_init(struct kvm *kvm); void vmx_vm_destroy(struct kvm *kvm); int vmx_vcpu_precreate(struct kvm *kvm); @@ -137,8 +136,10 @@ void vmx_setup_mce(struct kvm_vcpu *vcpu); #ifdef CONFIG_INTEL_TDX_HOST int __init tdx_hardware_setup(struct kvm_x86_ops *x86_ops); +bool tdx_is_vm_type_supported(unsigned long type); #else static inline int tdx_hardware_setup(struct kvm_x86_ops *x86_ops) { return -EOPNOTSUPP; } +static inline bool tdx_is_vm_type_supported(unsigned long type) { return false; } #endif #endif /* __KVM_X86_VMX_X86_OPS_H */ From patchwork Mon Feb 26 08:25:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571454 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A2F2F55E5F; Mon, 26 Feb 2024 08:27:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.13 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936077; cv=none; b=M5W4RcYXV60xWe5YpqXvvLaj4J7PXXBhFCwMbdwwEOtwpBXy1ehiicmd9Uy8zpiHMqL+ueKq4FSFPz/iOW/qZXgUu7YD92Dx+eOkrqQMU654i+xGMnSKzTwoZti/GMLCYSkmc1jvvupirjEE+yO3tyqxnOlr7uK0uRaF5rpfoFY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936077; c=relaxed/simple; bh=gZq/xU5HPMr+7bL2m3UNHQGfAGkvgf+PBtN9vzhZEYs=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=EScouAOd+8TsQOh7x/H3efLGNy+kPmse+jcWNBxldS/h6/S1KEXpZBACopXYjnGeNIl1KrSJgEV3kQvJzPupQdIOu4LCiWlN0y5ye3Hl8/B1ujKnazjGF1V/w/OSJQMFc6edjDxM1gQu4a88BYjtnKuYFpYQ+FUYSI0CJjDFZPM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=a75rByFQ; arc=none smtp.client-ip=192.198.163.13 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="a75rByFQ" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936075; x=1740472075; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=gZq/xU5HPMr+7bL2m3UNHQGfAGkvgf+PBtN9vzhZEYs=; b=a75rByFQrOok+2Uguyju8s4RZ2M0psBa1EfZe3UwyyPQzYl39gyJxcTZ gLzaFJH/+1PPfy5tsAo0C9GPSeicze3GWDC0CQiE6pR0wtRGnnJuTKKAU IlemdrV9sFgSRya26WExpxtvC2+70YXqe5rB//qILLgsTedVhWfMMc3+8 x2JcnJb2biJaIslyqajdgb8I5ACrivgehrMyC7Fo66STy7wqtCvCTkK1I xIeXpMD7kVVebBCWTkhBJBtlc7PcUEtXc7FsDJQGHLBEae+P2vXa+J4HF kO6NbVAKs6EBuehAN7b/kVI1olUIj6butUH5GMoMOCljWaL/DG5LgYUMb A==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="6155250" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6155250" Received: from orviesa009.jf.intel.com ([10.64.159.149]) by fmvoesa107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:27:53 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6615480" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa009-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:27:53 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 026/130] [MARKER] The start of TDX KVM patch series: TDX architectural definitions Date: Mon, 26 Feb 2024 00:25:28 -0800 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata This empty commit is to mark the start of patch series of TDX architectural definitions. Signed-off-by: Isaku Yamahata --- Documentation/virt/kvm/index.rst | 2 ++ .../virt/kvm/intel-tdx-layer-status.rst | 29 +++++++++++++++++++ 2 files changed, 31 insertions(+) create mode 100644 Documentation/virt/kvm/intel-tdx-layer-status.rst diff --git a/Documentation/virt/kvm/index.rst b/Documentation/virt/kvm/index.rst index ad13ec55ddfe..ccff56dca2b1 100644 --- a/Documentation/virt/kvm/index.rst +++ b/Documentation/virt/kvm/index.rst @@ -19,3 +19,5 @@ KVM vcpu-requests halt-polling review-checklist + + intel-tdx-layer-status diff --git a/Documentation/virt/kvm/intel-tdx-layer-status.rst b/Documentation/virt/kvm/intel-tdx-layer-status.rst new file mode 100644 index 000000000000..f11ea701dc19 --- /dev/null +++ b/Documentation/virt/kvm/intel-tdx-layer-status.rst @@ -0,0 +1,29 @@ +.. SPDX-License-Identifier: GPL-2.0 + +=================================== +Intel Trust Dodmain Extensions(TDX) +=================================== + +Layer status +============ +What qemu can do +---------------- +- TDX VM TYPE is exposed to Qemu. +- Qemu can try to create VM of TDX VM type and then fails. + +Patch Layer status +------------------ + Patch layer Status + +* TDX, VMX coexistence: Applied +* TDX architectural definitions: Applying +* TD VM creation/destruction: Not yet +* TD vcpu creation/destruction: Not yet +* TDX EPT violation: Not yet +* TD finalization: Not yet +* TD vcpu enter/exit: Not yet +* TD vcpu interrupts/exit/hypercall: Not yet + +* KVM MMU GPA shared bits: Not yet +* KVM TDP refactoring for TDX: Not yet +* KVM TDP MMU hooks: Not yet From patchwork Mon Feb 26 08:25:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571455 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B91A855E68; Mon, 26 Feb 2024 08:27:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.13 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936077; cv=none; b=WP8ZQ+Qu4Q7PVzkYVoTEER5AEQXPGcmwV/JS6b3C9L4dnHmdKizETD4vXpdi5t7aOJ9+ZOcRXx/VwV/6nCYqS2QRjlq0xq6S2vabHNLZzRLVG/GYiTqChaTwMZ/+o7T86/Jg9pdKC9HuaUuBlF96OjFTXbyPhsfYwnV22sFYHYo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936077; c=relaxed/simple; bh=tNU7BzaaqXHHxYsiLqg4zyO+pCGDtgdoU5GDQGjv/KY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Isj8wsq3FzHM6r/pW5e/LY3IHHEFLDNKhQp3Ki+B+m110zZvo1mbc1S+EcfrwRApbkaEAIGx5vs608ZNIP22plRobmt5kBxnUsknLy1moo4+mHZjJmGPvD24O4eljCjEpe7mHtt8s0yUpcHOn56dHrhoulaOJH1KRzeE7Vt2J64= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=b4lz2/6L; arc=none smtp.client-ip=192.198.163.13 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="b4lz2/6L" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936075; x=1740472075; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=tNU7BzaaqXHHxYsiLqg4zyO+pCGDtgdoU5GDQGjv/KY=; b=b4lz2/6LMldaO/UukIgc/8YES6Ba5C0+hjSGG7j9p32xiiNcidu/F1BK yYiS8yX9cr+AeVxQzMICPq1HOFqgo4suRo77W3p2LD650wDgHMS4eNbDx +enee+e2K0UE+M1BRFbIqZU8RkzAkeiFBtZHPxuJxbfLIljiHB3Uk4b8P xvJP6Og6ny2CpUOB/renM61xQeHTfEHoY+O+nRKcpkzhPi6YgxZhilwLe m018K2RxdHlRt/4vAUn6wAOfZK5+8kt9W6jfbajxc+QxOob5bc8dZbZxu CeL7kKHZr/eP6hHwIN9/VpXu5mzK3Ntdy+AwggMSrnNFpcnwo5YCD38mr Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="6155254" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6155254" Received: from orviesa009.jf.intel.com ([10.64.159.149]) by fmvoesa107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:27:53 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6615488" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa009-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:27:53 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com, Sean Christopherson , Xiaoyao Li Subject: [PATCH v19 027/130] KVM: TDX: Define TDX architectural definitions Date: Mon, 26 Feb 2024 00:25:29 -0800 Message-Id: <522cbfe6e5a351f88480790fe3c3be36c82ca4b1.1708933498.git.isaku.yamahata@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata Define architectural definitions for KVM to issue the TDX SEAMCALLs. Structures and values that are architecturally defined in the TDX module specifications the chapter of ABI Reference. Co-developed-by: Sean Christopherson Signed-off-by: Sean Christopherson Signed-off-by: Isaku Yamahata Reviewed-by: Paolo Bonzini Reviewed-by: Xiaoyao Li --- v19: - drop tdvmcall constants by Xiaoyao v18: - Add metadata field id Signed-off-by: Isaku Yamahata --- arch/x86/kvm/vmx/tdx_arch.h | 265 ++++++++++++++++++++++++++++++++++++ 1 file changed, 265 insertions(+) create mode 100644 arch/x86/kvm/vmx/tdx_arch.h diff --git a/arch/x86/kvm/vmx/tdx_arch.h b/arch/x86/kvm/vmx/tdx_arch.h new file mode 100644 index 000000000000..e2c1a6f429d7 --- /dev/null +++ b/arch/x86/kvm/vmx/tdx_arch.h @@ -0,0 +1,265 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* architectural constants/data definitions for TDX SEAMCALLs */ + +#ifndef __KVM_X86_TDX_ARCH_H +#define __KVM_X86_TDX_ARCH_H + +#include + +/* + * TDX SEAMCALL API function leaves + */ +#define TDH_VP_ENTER 0 +#define TDH_MNG_ADDCX 1 +#define TDH_MEM_PAGE_ADD 2 +#define TDH_MEM_SEPT_ADD 3 +#define TDH_VP_ADDCX 4 +#define TDH_MEM_PAGE_RELOCATE 5 +#define TDH_MEM_PAGE_AUG 6 +#define TDH_MEM_RANGE_BLOCK 7 +#define TDH_MNG_KEY_CONFIG 8 +#define TDH_MNG_CREATE 9 +#define TDH_VP_CREATE 10 +#define TDH_MNG_RD 11 +#define TDH_MR_EXTEND 16 +#define TDH_MR_FINALIZE 17 +#define TDH_VP_FLUSH 18 +#define TDH_MNG_VPFLUSHDONE 19 +#define TDH_MNG_KEY_FREEID 20 +#define TDH_MNG_INIT 21 +#define TDH_VP_INIT 22 +#define TDH_MEM_SEPT_RD 25 +#define TDH_VP_RD 26 +#define TDH_MNG_KEY_RECLAIMID 27 +#define TDH_PHYMEM_PAGE_RECLAIM 28 +#define TDH_MEM_PAGE_REMOVE 29 +#define TDH_MEM_SEPT_REMOVE 30 +#define TDH_SYS_RD 34 +#define TDH_MEM_TRACK 38 +#define TDH_MEM_RANGE_UNBLOCK 39 +#define TDH_PHYMEM_CACHE_WB 40 +#define TDH_PHYMEM_PAGE_WBINVD 41 +#define TDH_VP_WR 43 +#define TDH_SYS_LP_SHUTDOWN 44 + +/* TDX control structure (TDR/TDCS/TDVPS) field access codes */ +#define TDX_NON_ARCH BIT_ULL(63) +#define TDX_CLASS_SHIFT 56 +#define TDX_FIELD_MASK GENMASK_ULL(31, 0) + +#define __BUILD_TDX_FIELD(non_arch, class, field) \ + (((non_arch) ? TDX_NON_ARCH : 0) | \ + ((u64)(class) << TDX_CLASS_SHIFT) | \ + ((u64)(field) & TDX_FIELD_MASK)) + +#define BUILD_TDX_FIELD(class, field) \ + __BUILD_TDX_FIELD(false, (class), (field)) + +#define BUILD_TDX_FIELD_NON_ARCH(class, field) \ + __BUILD_TDX_FIELD(true, (class), (field)) + + +/* Class code for TD */ +#define TD_CLASS_EXECUTION_CONTROLS 17ULL + +/* Class code for TDVPS */ +#define TDVPS_CLASS_VMCS 0ULL +#define TDVPS_CLASS_GUEST_GPR 16ULL +#define TDVPS_CLASS_OTHER_GUEST 17ULL +#define TDVPS_CLASS_MANAGEMENT 32ULL + +enum tdx_tdcs_execution_control { + TD_TDCS_EXEC_TSC_OFFSET = 10, +}; + +/* @field is any of enum tdx_tdcs_execution_control */ +#define TDCS_EXEC(field) BUILD_TDX_FIELD(TD_CLASS_EXECUTION_CONTROLS, (field)) + +/* @field is the VMCS field encoding */ +#define TDVPS_VMCS(field) BUILD_TDX_FIELD(TDVPS_CLASS_VMCS, (field)) + +enum tdx_vcpu_guest_other_state { + TD_VCPU_STATE_DETAILS_NON_ARCH = 0x100, +}; + +union tdx_vcpu_state_details { + struct { + u64 vmxip : 1; + u64 reserved : 63; + }; + u64 full; +}; + +/* @field is any of enum tdx_guest_other_state */ +#define TDVPS_STATE(field) BUILD_TDX_FIELD(TDVPS_CLASS_OTHER_GUEST, (field)) +#define TDVPS_STATE_NON_ARCH(field) BUILD_TDX_FIELD_NON_ARCH(TDVPS_CLASS_OTHER_GUEST, (field)) + +/* Management class fields */ +enum tdx_vcpu_guest_management { + TD_VCPU_PEND_NMI = 11, +}; + +/* @field is any of enum tdx_vcpu_guest_management */ +#define TDVPS_MANAGEMENT(field) BUILD_TDX_FIELD(TDVPS_CLASS_MANAGEMENT, (field)) + +#define TDX_EXTENDMR_CHUNKSIZE 256 + +struct tdx_cpuid_value { + u32 eax; + u32 ebx; + u32 ecx; + u32 edx; +} __packed; + +#define TDX_TD_ATTRIBUTE_DEBUG BIT_ULL(0) +#define TDX_TD_ATTR_SEPT_VE_DISABLE BIT_ULL(28) +#define TDX_TD_ATTRIBUTE_PKS BIT_ULL(30) +#define TDX_TD_ATTRIBUTE_KL BIT_ULL(31) +#define TDX_TD_ATTRIBUTE_PERFMON BIT_ULL(63) + +/* + * TD_PARAMS is provided as an input to TDH_MNG_INIT, the size of which is 1024B. + */ +#define TDX_MAX_VCPUS (~(u16)0) + +struct td_params { + u64 attributes; + u64 xfam; + u16 max_vcpus; + u8 reserved0[6]; + + u64 eptp_controls; + u64 exec_controls; + u16 tsc_frequency; + u8 reserved1[38]; + + u64 mrconfigid[6]; + u64 mrowner[6]; + u64 mrownerconfig[6]; + u64 reserved2[4]; + + union { + DECLARE_FLEX_ARRAY(struct tdx_cpuid_value, cpuid_values); + u8 reserved3[768]; + }; +} __packed __aligned(1024); + +/* + * Guest uses MAX_PA for GPAW when set. + * 0: GPA.SHARED bit is GPA[47] + * 1: GPA.SHARED bit is GPA[51] + */ +#define TDX_EXEC_CONTROL_MAX_GPAW BIT_ULL(0) + +/* + * TDH.VP.ENTER, TDG.VP.VMCALL preserves RBP + * 0: RBP can be used for TDG.VP.VMCALL input. RBP is clobbered. + * 1: RBP can't be used for TDG.VP.VMCALL input. RBP is preserved. + */ +#define TDX_CONTROL_FLAG_NO_RBP_MOD BIT_ULL(2) + + +/* + * TDX requires the frequency to be defined in units of 25MHz, which is the + * frequency of the core crystal clock on TDX-capable platforms, i.e. the TDX + * module can only program frequencies that are multiples of 25MHz. The + * frequency must be between 100mhz and 10ghz (inclusive). + */ +#define TDX_TSC_KHZ_TO_25MHZ(tsc_in_khz) ((tsc_in_khz) / (25 * 1000)) +#define TDX_TSC_25MHZ_TO_KHZ(tsc_in_25mhz) ((tsc_in_25mhz) * (25 * 1000)) +#define TDX_MIN_TSC_FREQUENCY_KHZ (100 * 1000) +#define TDX_MAX_TSC_FREQUENCY_KHZ (10 * 1000 * 1000) + +union tdx_sept_entry { + struct { + u64 r : 1; + u64 w : 1; + u64 x : 1; + u64 mt : 3; + u64 ipat : 1; + u64 leaf : 1; + u64 a : 1; + u64 d : 1; + u64 xu : 1; + u64 ignored0 : 1; + u64 pfn : 40; + u64 reserved : 5; + u64 vgp : 1; + u64 pwa : 1; + u64 ignored1 : 1; + u64 sss : 1; + u64 spp : 1; + u64 ignored2 : 1; + u64 sve : 1; + }; + u64 raw; +}; + +enum tdx_sept_entry_state { + TDX_SEPT_FREE = 0, + TDX_SEPT_BLOCKED = 1, + TDX_SEPT_PENDING = 2, + TDX_SEPT_PENDING_BLOCKED = 3, + TDX_SEPT_PRESENT = 4, +}; + +union tdx_sept_level_state { + struct { + u64 level : 3; + u64 reserved0 : 5; + u64 state : 8; + u64 reserved1 : 48; + }; + u64 raw; +}; + +/* + * Global scope metadata field ID. + * See Table "Global Scope Metadata", TDX module 1.5 ABI spec. + */ +#define MD_FIELD_ID_SYS_ATTRIBUTES 0x0A00000200000000ULL +#define MD_FIELD_ID_FEATURES0 0x0A00000300000008ULL +#define MD_FIELD_ID_ATTRS_FIXED0 0x1900000300000000ULL +#define MD_FIELD_ID_ATTRS_FIXED1 0x1900000300000001ULL +#define MD_FIELD_ID_XFAM_FIXED0 0x1900000300000002ULL +#define MD_FIELD_ID_XFAM_FIXED1 0x1900000300000003ULL + +#define MD_FIELD_ID_TDCS_BASE_SIZE 0x9800000100000100ULL +#define MD_FIELD_ID_TDVPS_BASE_SIZE 0x9800000100000200ULL + +#define MD_FIELD_ID_NUM_CPUID_CONFIG 0x9900000100000004ULL +#define MD_FIELD_ID_CPUID_CONFIG_LEAVES 0x9900000300000400ULL +#define MD_FIELD_ID_CPUID_CONFIG_VALUES 0x9900000300000500ULL + +#define MD_FIELD_ID_FEATURES0_NO_RBP_MOD BIT_ULL(18) + +#define TDX_MAX_NR_CPUID_CONFIGS 37 + +#define TDX_MD_ELEMENT_SIZE_8BITS 0 +#define TDX_MD_ELEMENT_SIZE_16BITS 1 +#define TDX_MD_ELEMENT_SIZE_32BITS 2 +#define TDX_MD_ELEMENT_SIZE_64BITS 3 + +union tdx_md_field_id { + struct { + u64 field : 24; + u64 reserved0 : 8; + u64 element_size_code : 2; + u64 last_element_in_field : 4; + u64 reserved1 : 3; + u64 inc_size : 1; + u64 write_mask_valid : 1; + u64 context : 3; + u64 reserved2 : 1; + u64 class : 6; + u64 reserved3 : 1; + u64 non_arch : 1; + }; + u64 raw; +}; + +#define TDX_MD_ELEMENT_SIZE_CODE(_field_id) \ + ({ union tdx_md_field_id _fid = { .raw = (_field_id)}; \ + _fid.element_size_code; }) + +#endif /* __KVM_X86_TDX_ARCH_H */ From patchwork Mon Feb 26 08:25:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571456 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9FD265646E; Mon, 26 Feb 2024 08:27:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.13 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936078; cv=none; b=ZDK+8rCkIf44HCxRSQosSTmb1aV8fYuMzv2d4BPcGAd7+QdJmAyK/zH0ISrdQpZY+KjRbWGsGSP6iCdilmUmWzA8AVeNQl4vRanyz58W6n/pPRUwrLNr/5e8RXcCFULiHDmdncw88QMeNesH4uCsSyaUmq0N8McLqrGbaZQiwCw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936078; c=relaxed/simple; bh=yABnwpyEKSvL9YUubu3fELC/Z5Mahr2JAsazFYhrkAc=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=JhXLcLDRNjtZov6/acnpuG4cJGdELFX40z/aKDp7YDWvOtAfNLTD4hbzdmLfILkJg5hMrQkibM7tXsvff0yjrWWM2tdtFCarZa1OiZgQAHmDl0O0Q8tfDJvgw3UXOHqvCPX8bwffK+eHXNLwlMNxcKVd7IlprY+VY0QroYUdY0I= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=JA1wXDvu; arc=none smtp.client-ip=192.198.163.13 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="JA1wXDvu" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936076; x=1740472076; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=yABnwpyEKSvL9YUubu3fELC/Z5Mahr2JAsazFYhrkAc=; b=JA1wXDvuqhSljIsuFmR6eRgRyMPxYCafCYb4FHyq4EbyBXejITDSS/8R QkB2OM6u+TTmvw+AuvgjdQcm41Reey4rYFiFgv9hpAY+VG9BIE9FIwDb7 WQutphSdYKWlhFL2VQ2zXAJLueJbXDH0hTjWcwuDlMLxLYAKlVKOKM2rV ODbfL53HlUgISovkQvC+1HEQJyWPEoVnf4u6V8/UlRAW0j2Y2XXjSG4OA w5rwAyxjgS273vGp5RWMdqs4FF71xp2TT2vCkhATcnvljeOjnTXUAVAvO RKzfQ63nMGOjAvLqswPPWSDEanUrpqxExz30V5/iZIsTzqtAZKERL9huR w==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="6155259" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6155259" Received: from orviesa009.jf.intel.com ([10.64.159.149]) by fmvoesa107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:27:54 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6615496" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa009-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:27:54 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com, Sean Christopherson , Yuan Yao , Xiaoyao Li Subject: [PATCH v19 028/130] KVM: TDX: Add TDX "architectural" error codes Date: Mon, 26 Feb 2024 00:25:30 -0800 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Sean Christopherson Add error codes for the TDX SEAMCALLs both for TDX VMM side for TDH SEAMCALL and TDX guest side for TDG.VP.VMCALL. KVM issues the TDX SEAMCALLs and checks its error code. KVM handles hypercall from the TDX guest and may return an error. So error code for the TDX guest is also needed. TDX SEAMCALL uses bits 31:0 to return more information, so these error codes will only exactly match RAX[63:32]. Error codes for TDG.VP.VMCALL is defined by TDX Guest-Host-Communication interface spec. Signed-off-by: Sean Christopherson Signed-off-by: Isaku Yamahata Reviewed-by: Paolo Bonzini Reviewed-by: Yuan Yao Reviewed-by: Xiaoyao Li --- v19: - Drop TDX_EPT_WALK_FAILED, TDX_EPT_ENTRY_NOT_FREE - Rename TDG_VP_VMCALL_ => TDVMCALL_ to match the existing code - Move TDVMCALL error codes to shared/tdx.h - Added TDX_OPERAND_ID_TDR --- arch/x86/include/asm/shared/tdx.h | 8 +++++++- arch/x86/kvm/vmx/tdx_errno.h | 34 +++++++++++++++++++++++++++++++ 2 files changed, 41 insertions(+), 1 deletion(-) create mode 100644 arch/x86/kvm/vmx/tdx_errno.h diff --git a/arch/x86/include/asm/shared/tdx.h b/arch/x86/include/asm/shared/tdx.h index fdfd41511b02..28c4a62b7dba 100644 --- a/arch/x86/include/asm/shared/tdx.h +++ b/arch/x86/include/asm/shared/tdx.h @@ -26,7 +26,13 @@ #define TDVMCALL_GET_QUOTE 0x10002 #define TDVMCALL_REPORT_FATAL_ERROR 0x10003 -#define TDVMCALL_STATUS_RETRY 1 +/* + * TDG.VP.VMCALL Status Codes (returned in R10) + */ +#define TDVMCALL_SUCCESS 0x0000000000000000ULL +#define TDVMCALL_RETRY 0x0000000000000001ULL +#define TDVMCALL_INVALID_OPERAND 0x8000000000000000ULL +#define TDVMCALL_TDREPORT_FAILED 0x8000000000000001ULL /* * Bitmasks of exposed registers (with VMM). diff --git a/arch/x86/kvm/vmx/tdx_errno.h b/arch/x86/kvm/vmx/tdx_errno.h new file mode 100644 index 000000000000..5366bf476d2c --- /dev/null +++ b/arch/x86/kvm/vmx/tdx_errno.h @@ -0,0 +1,34 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* architectural status code for SEAMCALL */ + +#ifndef __KVM_X86_TDX_ERRNO_H +#define __KVM_X86_TDX_ERRNO_H + +#define TDX_SEAMCALL_STATUS_MASK 0xFFFFFFFF00000000ULL + +/* + * TDX SEAMCALL Status Codes (returned in RAX) + */ +#define TDX_NON_RECOVERABLE_VCPU 0x4000000100000000ULL +#define TDX_INTERRUPTED_RESUMABLE 0x8000000300000000ULL +#define TDX_OPERAND_INVALID 0xC000010000000000ULL +#define TDX_OPERAND_BUSY 0x8000020000000000ULL +#define TDX_PREVIOUS_TLB_EPOCH_BUSY 0x8000020100000000ULL +#define TDX_VCPU_NOT_ASSOCIATED 0x8000070200000000ULL +#define TDX_KEY_GENERATION_FAILED 0x8000080000000000ULL +#define TDX_KEY_STATE_INCORRECT 0xC000081100000000ULL +#define TDX_KEY_CONFIGURED 0x0000081500000000ULL +#define TDX_NO_HKID_READY_TO_WBCACHE 0x0000082100000000ULL +#define TDX_FLUSHVP_NOT_DONE 0x8000082400000000ULL +#define TDX_EPT_ENTRY_STATE_INCORRECT 0xC0000B0D00000000ULL + +/* + * TDX module operand ID, appears in 31:0 part of error code as + * detail information + */ +#define TDX_OPERAND_ID_RCX 0x01 +#define TDX_OPERAND_ID_TDR 0x80 +#define TDX_OPERAND_ID_SEPT 0x92 +#define TDX_OPERAND_ID_TD_EPOCH 0xa9 + +#endif /* __KVM_X86_TDX_ERRNO_H */ From patchwork Mon Feb 26 08:25:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571457 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A274856B7F; Mon, 26 Feb 2024 08:27:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.13 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936080; cv=none; b=h54WaVaSFQEsDoRiFRTh5RbAuF5jttLOBDq/P/P2tTOCVOFcYiAjsNXc5JCqBoebIZsQdbUOgY94Gv4Ea12KjtfOMnigbt6mbHkoTiNtvgZyuGsMrWlJ/qGXPRduKTPAUt3HBkg2eQcoCQQfz8X+GSALky0H65UrJ22TXHUiLHE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936080; c=relaxed/simple; bh=alim40x+Y9lFNx4C2cK1mohNsqNIRwdAHp3QxJCHc1Q=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=P97zzO1r8b6VRmr9a5ydyVjPVG/9V/Jdc8wtaY9r6JzaxLeHhjq/jCCSUvNz0tr3sIZezUaU3UPtmuNKAWB4E4ALegrYd1HY7GRLH3SpkdjZqzW92lTsH9DPd4op0WGndUKoSSN58iUYQz6xgP4uAdmmgH+02xcfQJxfi0GqtR0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=bIopRedQ; arc=none smtp.client-ip=192.198.163.13 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="bIopRedQ" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936077; x=1740472077; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=alim40x+Y9lFNx4C2cK1mohNsqNIRwdAHp3QxJCHc1Q=; b=bIopRedQQjI8oo6Lx73J9h4Mw/3e/38Y3kGQgoDKLTurCnaBHJr/o1he UvexwcQVpK2auibcTPxM3dc/ypNpIgL6lHdwVEj3abmAYkKqecUKjw8Y0 zWtF+vCHcLLlexoFwfYSB3QMI39rZN6kpduqyKgMh8ycJg1q4Wma8cHJ7 KZTQzaFO3BBO0ps3O8ZUkOy93ooz5J0bLWr8bMuLO4TK5f6Zvc+zU4uji nYQJuB5zaGbb9+uuvuszZQitrqYt/iPmeSXYiIf2cmzjbs/6g9AGircp/ Njsin/n3ks1f+EdfYIC9CLGnPhK05xuJnmzT8w3048IHspfD4sx1ZQ08W Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="6155266" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6155266" Received: from orviesa009.jf.intel.com ([10.64.159.149]) by fmvoesa107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:27:55 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6615507" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa009-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:27:55 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com, Sean Christopherson , Binbin Wu , Yuan Yao Subject: [PATCH v19 029/130] KVM: TDX: Add C wrapper functions for SEAMCALLs to the TDX module Date: Mon, 26 Feb 2024 00:25:31 -0800 Message-Id: <7cfd33d896fce7b49bcf4b7179d0ded22c06b8c2.1708933498.git.isaku.yamahata@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata A VMM interacts with the TDX module using a new instruction (SEAMCALL). For instance, a TDX VMM does not have full access to the VM control structure corresponding to VMX VMCS. Instead, a VMM induces the TDX module to act on behalf via SEAMCALLs. Define C wrapper functions for SEAMCALLs for readability. Some SEAMCALL APIs donate host pages to TDX module or guest TD, and the donated pages are encrypted. Those require the VMM to flush the cache lines to avoid cache line alias. Signed-off-by: Sean Christopherson Signed-off-by: Isaku Yamahata Reviewed-by: Binbin Wu Reviewed-by: Yuan Yao --- Changes v19: - Update the commit message to match the patch by Yuan - Use seamcall() and seamcall_ret() by paolo v18: - removed stub functions for __seamcall{,_ret}() - Added Reviewed-by Binbin - Make tdx_seamcall() use struct tdx_module_args instead of taking each inputs. v15 -> v16: - use struct tdx_module_args instead of struct tdx_module_output - Add tdh_mem_sept_rd() for SEPT_VE_DISABLE=1. Signed-off-by: Isaku Yamahata --- arch/x86/kvm/vmx/tdx_ops.h | 360 +++++++++++++++++++++++++++++++++++++ 1 file changed, 360 insertions(+) create mode 100644 arch/x86/kvm/vmx/tdx_ops.h diff --git a/arch/x86/kvm/vmx/tdx_ops.h b/arch/x86/kvm/vmx/tdx_ops.h new file mode 100644 index 000000000000..c5bb165b260e --- /dev/null +++ b/arch/x86/kvm/vmx/tdx_ops.h @@ -0,0 +1,360 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* constants/data definitions for TDX SEAMCALLs */ + +#ifndef __KVM_X86_TDX_OPS_H +#define __KVM_X86_TDX_OPS_H + +#include + +#include +#include +#include + +#include "tdx_errno.h" +#include "tdx_arch.h" +#include "x86.h" + +static inline u64 tdx_seamcall(u64 op, struct tdx_module_args *in, + struct tdx_module_args *out) +{ + u64 ret; + + if (out) { + *out = *in; + ret = seamcall_ret(op, out); + } else + ret = seamcall(op, in); + + if (unlikely(ret == TDX_SEAMCALL_UD)) { + /* + * SEAMCALLs fail with TDX_SEAMCALL_UD returned when VMX is off. + * This can happen when the host gets rebooted or live + * updated. In this case, the instruction execution is ignored + * as KVM is shut down, so the error code is suppressed. Other + * than this, the error is unexpected and the execution can't + * continue as the TDX features reply on VMX to be on. + */ + kvm_spurious_fault(); + return 0; + } + return ret; +} + +static inline u64 tdh_mng_addcx(hpa_t tdr, hpa_t addr) +{ + struct tdx_module_args in = { + .rcx = addr, + .rdx = tdr, + }; + + clflush_cache_range(__va(addr), PAGE_SIZE); + return tdx_seamcall(TDH_MNG_ADDCX, &in, NULL); +} + +static inline u64 tdh_mem_page_add(hpa_t tdr, gpa_t gpa, hpa_t hpa, hpa_t source, + struct tdx_module_args *out) +{ + struct tdx_module_args in = { + .rcx = gpa, + .rdx = tdr, + .r8 = hpa, + .r9 = source, + }; + + clflush_cache_range(__va(hpa), PAGE_SIZE); + return tdx_seamcall(TDH_MEM_PAGE_ADD, &in, out); +} + +static inline u64 tdh_mem_sept_add(hpa_t tdr, gpa_t gpa, int level, hpa_t page, + struct tdx_module_args *out) +{ + struct tdx_module_args in = { + .rcx = gpa | level, + .rdx = tdr, + .r8 = page, + }; + + clflush_cache_range(__va(page), PAGE_SIZE); + return tdx_seamcall(TDH_MEM_SEPT_ADD, &in, out); +} + +static inline u64 tdh_mem_sept_rd(hpa_t tdr, gpa_t gpa, int level, + struct tdx_module_args *out) +{ + struct tdx_module_args in = { + .rcx = gpa | level, + .rdx = tdr, + }; + + return tdx_seamcall(TDH_MEM_SEPT_RD, &in, out); +} + +static inline u64 tdh_mem_sept_remove(hpa_t tdr, gpa_t gpa, int level, + struct tdx_module_args *out) +{ + struct tdx_module_args in = { + .rcx = gpa | level, + .rdx = tdr, + }; + + return tdx_seamcall(TDH_MEM_SEPT_REMOVE, &in, out); +} + +static inline u64 tdh_vp_addcx(hpa_t tdvpr, hpa_t addr) +{ + struct tdx_module_args in = { + .rcx = addr, + .rdx = tdvpr, + }; + + clflush_cache_range(__va(addr), PAGE_SIZE); + return tdx_seamcall(TDH_VP_ADDCX, &in, NULL); +} + +static inline u64 tdh_mem_page_relocate(hpa_t tdr, gpa_t gpa, hpa_t hpa, + struct tdx_module_args *out) +{ + struct tdx_module_args in = { + .rcx = gpa, + .rdx = tdr, + .r8 = hpa, + }; + + clflush_cache_range(__va(hpa), PAGE_SIZE); + return tdx_seamcall(TDH_MEM_PAGE_RELOCATE, &in, out); +} + +static inline u64 tdh_mem_page_aug(hpa_t tdr, gpa_t gpa, hpa_t hpa, + struct tdx_module_args *out) +{ + struct tdx_module_args in = { + .rcx = gpa, + .rdx = tdr, + .r8 = hpa, + }; + + clflush_cache_range(__va(hpa), PAGE_SIZE); + return tdx_seamcall(TDH_MEM_PAGE_AUG, &in, out); +} + +static inline u64 tdh_mem_range_block(hpa_t tdr, gpa_t gpa, int level, + struct tdx_module_args *out) +{ + struct tdx_module_args in = { + .rcx = gpa | level, + .rdx = tdr, + }; + + return tdx_seamcall(TDH_MEM_RANGE_BLOCK, &in, out); +} + +static inline u64 tdh_mng_key_config(hpa_t tdr) +{ + struct tdx_module_args in = { + .rcx = tdr, + }; + + return tdx_seamcall(TDH_MNG_KEY_CONFIG, &in, NULL); +} + +static inline u64 tdh_mng_create(hpa_t tdr, int hkid) +{ + struct tdx_module_args in = { + .rcx = tdr, + .rdx = hkid, + }; + + clflush_cache_range(__va(tdr), PAGE_SIZE); + return tdx_seamcall(TDH_MNG_CREATE, &in, NULL); +} + +static inline u64 tdh_vp_create(hpa_t tdr, hpa_t tdvpr) +{ + struct tdx_module_args in = { + .rcx = tdvpr, + .rdx = tdr, + }; + + clflush_cache_range(__va(tdvpr), PAGE_SIZE); + return tdx_seamcall(TDH_VP_CREATE, &in, NULL); +} + +static inline u64 tdh_mng_rd(hpa_t tdr, u64 field, struct tdx_module_args *out) +{ + struct tdx_module_args in = { + .rcx = tdr, + .rdx = field, + }; + + return tdx_seamcall(TDH_MNG_RD, &in, out); +} + +static inline u64 tdh_mr_extend(hpa_t tdr, gpa_t gpa, + struct tdx_module_args *out) +{ + struct tdx_module_args in = { + .rcx = gpa, + .rdx = tdr, + }; + + return tdx_seamcall(TDH_MR_EXTEND, &in, out); +} + +static inline u64 tdh_mr_finalize(hpa_t tdr) +{ + struct tdx_module_args in = { + .rcx = tdr, + }; + + return tdx_seamcall(TDH_MR_FINALIZE, &in, NULL); +} + +static inline u64 tdh_vp_flush(hpa_t tdvpr) +{ + struct tdx_module_args in = { + .rcx = tdvpr, + }; + + return tdx_seamcall(TDH_VP_FLUSH, &in, NULL); +} + +static inline u64 tdh_mng_vpflushdone(hpa_t tdr) +{ + struct tdx_module_args in = { + .rcx = tdr, + }; + + return tdx_seamcall(TDH_MNG_VPFLUSHDONE, &in, NULL); +} + +static inline u64 tdh_mng_key_freeid(hpa_t tdr) +{ + struct tdx_module_args in = { + .rcx = tdr, + }; + + return tdx_seamcall(TDH_MNG_KEY_FREEID, &in, NULL); +} + +static inline u64 tdh_mng_init(hpa_t tdr, hpa_t td_params, + struct tdx_module_args *out) +{ + struct tdx_module_args in = { + .rcx = tdr, + .rdx = td_params, + }; + + return tdx_seamcall(TDH_MNG_INIT, &in, out); +} + +static inline u64 tdh_vp_init(hpa_t tdvpr, u64 rcx) +{ + struct tdx_module_args in = { + .rcx = tdvpr, + .rdx = rcx, + }; + + return tdx_seamcall(TDH_VP_INIT, &in, NULL); +} + +static inline u64 tdh_vp_rd(hpa_t tdvpr, u64 field, + struct tdx_module_args *out) +{ + struct tdx_module_args in = { + .rcx = tdvpr, + .rdx = field, + }; + + return tdx_seamcall(TDH_VP_RD, &in, out); +} + +static inline u64 tdh_mng_key_reclaimid(hpa_t tdr) +{ + struct tdx_module_args in = { + .rcx = tdr, + }; + + return tdx_seamcall(TDH_MNG_KEY_RECLAIMID, &in, NULL); +} + +static inline u64 tdh_phymem_page_reclaim(hpa_t page, + struct tdx_module_args *out) +{ + struct tdx_module_args in = { + .rcx = page, + }; + + return tdx_seamcall(TDH_PHYMEM_PAGE_RECLAIM, &in, out); +} + +static inline u64 tdh_mem_page_remove(hpa_t tdr, gpa_t gpa, int level, + struct tdx_module_args *out) +{ + struct tdx_module_args in = { + .rcx = gpa | level, + .rdx = tdr, + }; + + return tdx_seamcall(TDH_MEM_PAGE_REMOVE, &in, out); +} + +static inline u64 tdh_sys_lp_shutdown(void) +{ + struct tdx_module_args in = { + }; + + return tdx_seamcall(TDH_SYS_LP_SHUTDOWN, &in, NULL); +} + +static inline u64 tdh_mem_track(hpa_t tdr) +{ + struct tdx_module_args in = { + .rcx = tdr, + }; + + return tdx_seamcall(TDH_MEM_TRACK, &in, NULL); +} + +static inline u64 tdh_mem_range_unblock(hpa_t tdr, gpa_t gpa, int level, + struct tdx_module_args *out) +{ + struct tdx_module_args in = { + .rcx = gpa | level, + .rdx = tdr, + }; + + return tdx_seamcall(TDH_MEM_RANGE_UNBLOCK, &in, out); +} + +static inline u64 tdh_phymem_cache_wb(bool resume) +{ + struct tdx_module_args in = { + .rcx = resume ? 1 : 0, + }; + + return tdx_seamcall(TDH_PHYMEM_CACHE_WB, &in, NULL); +} + +static inline u64 tdh_phymem_page_wbinvd(hpa_t page) +{ + struct tdx_module_args in = { + .rcx = page, + }; + + return tdx_seamcall(TDH_PHYMEM_PAGE_WBINVD, &in, NULL); +} + +static inline u64 tdh_vp_wr(hpa_t tdvpr, u64 field, u64 val, u64 mask, + struct tdx_module_args *out) +{ + struct tdx_module_args in = { + .rcx = tdvpr, + .rdx = field, + .r8 = val, + .r9 = mask, + }; + + return tdx_seamcall(TDH_VP_WR, &in, out); +} + +#endif /* __KVM_X86_TDX_OPS_H */ From patchwork Mon Feb 26 08:25:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571458 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0BB275732A; Mon, 26 Feb 2024 08:27:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.13 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936080; cv=none; b=mxLno+P7wfEnjpza4zjjSqdcvDygnB+5dfApbBlO2/CnO4Fpe5j//fPn3huImDIz6N5cLmQoTgzecrDRWh+LoY+jTOD8x/mA2Wvch0e8lgWRfhOF9VzslzrZJjU4s8y0sNG9pnca5ekOM6xCBO0BpUhqFH27GVGKiGi1sIAQeNw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936080; c=relaxed/simple; bh=YpJuz/yxCnxq5gnhwOF3JBdeNECXepwyhjKiI0Sg8g4=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=NY3nHis0qg1Kr1i+eqdGj9xKOHKszG8XLerotSAsYvE6pe+ojHABy8IFhvn41wXtS1GDAqIkD4dcST75Owwf+31KXydUzfdubvE1eyy2KwZO4Fl1Ee9WH1cXxfmFCGMPRfWrZlLbGmORatV2v+Qny1a6GV0L4Fj4Z4yDadAjMaA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=J2TPyPea; arc=none smtp.client-ip=192.198.163.13 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="J2TPyPea" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936078; x=1740472078; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=YpJuz/yxCnxq5gnhwOF3JBdeNECXepwyhjKiI0Sg8g4=; b=J2TPyPeaqwKZYXJ0Td5kCkKM1Xphy3gK2luA68a8EwupBvCPGh7UurG8 pBAJlAMiG5q1bjZlmoqxEUDwwRywtptGFC4FGlCkJaN4bUb/fh83xQMG3 baHsFbjoJ9kB3hDutqYPhg5X6kuzBfoGe1VTNanbio+2W3YotSRRJ8snC ULd4c594fG2vtupUBYCFQc2Cc6wijDHhes4nwATqzPHuxH+UMb9PVl9nN KIyj0yCqevSSg2u5/VqfVFgJJ4glvjJiRrGUNSPj7+WgaTqUvwz9cjOWY 7Cjaegdv64l1xflS/00DB1TUc7TOw05B0zonUrRncH+Lf1Hp9lDJ3/KX6 g==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="6155270" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6155270" Received: from orviesa009.jf.intel.com ([10.64.159.149]) by fmvoesa107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:27:56 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6615522" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa009-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:27:55 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com, Binbin Wu , Yuan Yao Subject: [PATCH v19 030/130] KVM: TDX: Add helper functions to print TDX SEAMCALL error Date: Mon, 26 Feb 2024 00:25:32 -0800 Message-Id: <1259755bde3a07ec4dc2c78626fa348cf7323b33.1708933498.git.isaku.yamahata@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata Add helper functions to print out errors from the TDX module in a uniform manner. Signed-off-by: Isaku Yamahata Reviewed-by: Binbin Wu Reviewed-by: Yuan Yao --- v19: - dropped unnecessary include v18: - Added Reviewed-by Binbin. Signed-off-by: Isaku Yamahata --- arch/x86/kvm/Makefile | 2 +- arch/x86/kvm/vmx/tdx_error.c | 21 +++++++++++++++++++++ arch/x86/kvm/vmx/tdx_ops.h | 4 ++++ 3 files changed, 26 insertions(+), 1 deletion(-) create mode 100644 arch/x86/kvm/vmx/tdx_error.c diff --git a/arch/x86/kvm/Makefile b/arch/x86/kvm/Makefile index 5b85ef84b2e9..44b0594da877 100644 --- a/arch/x86/kvm/Makefile +++ b/arch/x86/kvm/Makefile @@ -24,7 +24,7 @@ kvm-intel-y += vmx/vmx.o vmx/vmenter.o vmx/pmu_intel.o vmx/vmcs12.o \ kvm-intel-$(CONFIG_X86_SGX_KVM) += vmx/sgx.o kvm-intel-$(CONFIG_KVM_HYPERV) += vmx/hyperv.o vmx/hyperv_evmcs.o -kvm-intel-$(CONFIG_INTEL_TDX_HOST) += vmx/tdx.o +kvm-intel-$(CONFIG_INTEL_TDX_HOST) += vmx/tdx.o vmx/tdx_error.o kvm-amd-y += svm/svm.o svm/vmenter.o svm/pmu.o svm/nested.o svm/avic.o \ svm/sev.o diff --git a/arch/x86/kvm/vmx/tdx_error.c b/arch/x86/kvm/vmx/tdx_error.c new file mode 100644 index 000000000000..42fcabe1f6c7 --- /dev/null +++ b/arch/x86/kvm/vmx/tdx_error.c @@ -0,0 +1,21 @@ +// SPDX-License-Identifier: GPL-2.0 +/* functions to record TDX SEAMCALL error */ + +#include +#include + +#include "tdx_ops.h" + +void pr_tdx_error(u64 op, u64 error_code, const struct tdx_module_args *out) +{ + if (!out) { + pr_err_ratelimited("SEAMCALL (0x%016llx) failed: 0x%016llx\n", + op, error_code); + return; + } + +#define MSG \ + "SEAMCALL (0x%016llx) failed: 0x%016llx RCX 0x%016llx RDX 0x%016llx R8 0x%016llx R9 0x%016llx R10 0x%016llx R11 0x%016llx\n" + pr_err_ratelimited(MSG, op, error_code, out->rcx, out->rdx, out->r8, + out->r9, out->r10, out->r11); +} diff --git a/arch/x86/kvm/vmx/tdx_ops.h b/arch/x86/kvm/vmx/tdx_ops.h index c5bb165b260e..d80212b1daf3 100644 --- a/arch/x86/kvm/vmx/tdx_ops.h +++ b/arch/x86/kvm/vmx/tdx_ops.h @@ -40,6 +40,10 @@ static inline u64 tdx_seamcall(u64 op, struct tdx_module_args *in, return ret; } +#ifdef CONFIG_INTEL_TDX_HOST +void pr_tdx_error(u64 op, u64 error_code, const struct tdx_module_args *out); +#endif + static inline u64 tdh_mng_addcx(hpa_t tdr, hpa_t addr) { struct tdx_module_args in = { From patchwork Mon Feb 26 08:25:33 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571459 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0BBDF5810B; Mon, 26 Feb 2024 08:27:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.13 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936081; cv=none; b=WnVs/KAvVBb9OZn567GyUehYllCkbi3sWu/uqggKk9qlh0PBxcSeugX8eMr7nz15SL3OkmEQ7/wd+N4MbR12dGMsYlhq2mzZ2yC36y2dAQmDV8KQ96vH5rzID+6+Yu9x7i9zGXeZTqz9Uw5qCgaXd+XZpYJuYtEzKp5GPElAD9M= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936081; c=relaxed/simple; bh=DLPNi9jXjbr4HdbjAnuHBOpBxwIVUIeOA5wtFcSAGOI=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=qe3tAl+Ld4fXFQxPX2rb8mcCuWpHJ+FCVvkAR77Lhm+f1SOEETo7NTq5R3z67f8HInsuJ50Ny7R7OUtBeheD4EwSEIba6dAoMpStYn9ohlNOWnzV0mSFjVIISRHKittVo9+qzmnpislBrFVn+fMuYcv1sYAlSD7RPK+egLbVWGI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=REiU1YFq; arc=none smtp.client-ip=192.198.163.13 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="REiU1YFq" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936079; x=1740472079; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=DLPNi9jXjbr4HdbjAnuHBOpBxwIVUIeOA5wtFcSAGOI=; b=REiU1YFqWxMmNqcRf2OHTnzd2OT18/iYDKk3KA+SD8vtZqDjcjGqD4VY wWHZJE229o7TE7BIuA8Ih2o1V2gbOMco9urQ/2rnZquyP4ZZc+qprhOpy /azNWXjgjcQuaaDLKv2DBGTPJrmoMsuK+E+KlpH8j1xzpKq1/WLC6pjrf Vt4wGnCdjShGhaul+8RFaaQGuNi5z8mT8C2ZTsFRHIhRjD4SB05FoMS4u OW6YkV6q/fdd3d82Y4ud3pUbIJGAleyrYrKTRaxZ4OstQqUwW8wEgyrWs xb5+nHtnDGPe42z5jJaphALGQZ7gstLEAnAHvQ27glOK2gMR9OtGIqmqE w==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="6155276" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6155276" Received: from orviesa009.jf.intel.com ([10.64.159.149]) by fmvoesa107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:27:57 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6615540" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa009-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:27:56 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 031/130] [MARKER] The start of TDX KVM patch series: TD VM creation/destruction Date: Mon, 26 Feb 2024 00:25:33 -0800 Message-Id: <3dd6180a7058d680819a5990ddbe9ca7c8fb2324.1708933498.git.isaku.yamahata@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata This empty commit is to mark the start of patch series of TD VM creation/destruction. Signed-off-by: Isaku Yamahata --- Documentation/virt/kvm/intel-tdx-layer-status.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/Documentation/virt/kvm/intel-tdx-layer-status.rst b/Documentation/virt/kvm/intel-tdx-layer-status.rst index f11ea701dc19..098150da6ea2 100644 --- a/Documentation/virt/kvm/intel-tdx-layer-status.rst +++ b/Documentation/virt/kvm/intel-tdx-layer-status.rst @@ -16,8 +16,8 @@ Patch Layer status Patch layer Status * TDX, VMX coexistence: Applied -* TDX architectural definitions: Applying -* TD VM creation/destruction: Not yet +* TDX architectural definitions: Applied +* TD VM creation/destruction: Applying * TD vcpu creation/destruction: Not yet * TDX EPT violation: Not yet * TD finalization: Not yet From patchwork Mon Feb 26 08:25:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571460 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4B92358239; Mon, 26 Feb 2024 08:28:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.13 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936082; cv=none; b=tqG41N9tsw2jSUIWLHfRYZL2WmTDWgG/pAjd2O1+Drd/O9NuYly8YxFW4SI3eS/9HlWKReXFdnxMfH6tajjuEZSjQHOI2WZ/xjC9RIBIAkwO1anR94g1AoXYjrHTYv5cVCn1AU6tsquK65JO84gy59AYiWV2gWkRp51NO2NymC8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936082; c=relaxed/simple; bh=bjQSqSS3f2d0HbCCIxYTXagxQRR+xdW29IEToT0E4RY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=feuEKfpSMyJwkf/F7oyK7iuOUu/sX3oYUut1U1UuFPZNiIsP4nIuT6GnZmFt6sw/8zlJOj0aLkTrCIbvcW22d2uxTqdeOvngSxEdCMTpgRf56EAMpOt8M4vcX+MXWV35infvsJ9g/an3rgLH00uugwr5XvmoGnpXeUebHkKhKz0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=Fg19aK5I; arc=none smtp.client-ip=192.198.163.13 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="Fg19aK5I" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936080; x=1740472080; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=bjQSqSS3f2d0HbCCIxYTXagxQRR+xdW29IEToT0E4RY=; b=Fg19aK5IOsXnC4P+SqqkVq/wb937vNbcCD3NzyuDLNcfmmVgtm7QU/73 sza7BszFdYgwWUoGo069Ogr1NdlQZ3gpAQQyNHP9ysyVa9l5yzsSjVOvL 8r//3hhV6GeSjyWiYKQAAT0HcVwpF19uRn4kxDKJgzjjz/CzriGy7HWIg aJQuSrwZ0hDasGKt7Q+9x3yMzeXigbFj4RRaTvKMGz4E6xAFDSNH2T5ZA Uz/YikiBSw/wh065o6w/qvEdZIBm1nDmKeWuacB2eLenihcjihW+g1+bz WyQNw9qnfp5NY1sM2Aajm/3fu6HMtWRV40WmygmsTqmQXsxpjTQiwNp5Y g==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="6155281" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6155281" Received: from orviesa009.jf.intel.com ([10.64.159.149]) by fmvoesa107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:27:58 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6615557" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa009-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:27:58 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 032/130] KVM: TDX: Add helper functions to allocate/free TDX private host key id Date: Mon, 26 Feb 2024 00:25:34 -0800 Message-Id: <7348e22ba8d0eeab7ba093f3e83bfa7ee4da1928.1708933498.git.isaku.yamahata@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata Add helper functions to allocate/free TDX private host key id (HKID). The memory controller encrypts TDX memory with the assigned TDX HKIDs. The global TDX HKID is to encrypt the TDX module, its memory, and some dynamic data (TDR). The private TDX HKID is assigned to guest TD to encrypt guest memory and the related data. When VMM releases an encrypted page for reuse, the page needs a cache flush with the used HKID. VMM needs the global TDX HKID and the private TDX HKIDs to flush encrypted pages. Signed-off-by: Isaku Yamahata --- v19: - Removed stale comment in tdx_guest_keyid_alloc() by Binbin - Update sanity check in tdx_guest_keyid_free() by Binbin v18: - Moved the functions to kvm tdx from arch/x86/virt/vmx/tdx/ - Drop exporting symbols as the host tdx does. Signed-off-by: Isaku Yamahata --- arch/x86/kvm/vmx/tdx.c | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c index a7e096fd8361..cde971122c1e 100644 --- a/arch/x86/kvm/vmx/tdx.c +++ b/arch/x86/kvm/vmx/tdx.c @@ -11,6 +11,34 @@ #undef pr_fmt #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt +/* + * Key id globally used by TDX module: TDX module maps TDR with this TDX global + * key id. TDR includes key id assigned to the TD. Then TDX module maps other + * TD-related pages with the assigned key id. TDR requires this TDX global key + * id for cache flush unlike other TD-related pages. + */ +/* TDX KeyID pool */ +static DEFINE_IDA(tdx_guest_keyid_pool); + +static int __used tdx_guest_keyid_alloc(void) +{ + if (WARN_ON_ONCE(!tdx_guest_keyid_start || !tdx_nr_guest_keyids)) + return -EINVAL; + + return ida_alloc_range(&tdx_guest_keyid_pool, tdx_guest_keyid_start, + tdx_guest_keyid_start + tdx_nr_guest_keyids - 1, + GFP_KERNEL); +} + +static void __used tdx_guest_keyid_free(int keyid) +{ + if (WARN_ON_ONCE(keyid < tdx_guest_keyid_start || + keyid > tdx_guest_keyid_start + tdx_nr_guest_keyids - 1)) + return; + + ida_free(&tdx_guest_keyid_pool, keyid); +} + static int __init tdx_module_setup(void) { int ret; From patchwork Mon Feb 26 08:25:35 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571461 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 69B2E5823B; Mon, 26 Feb 2024 08:28:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.13 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936082; cv=none; b=Q3ECGF1qNiyBdqTZRZ8XL/eMMtmzfrR7gtUKJeUdaa+M5D1wWB9+mUEezw8mTxbycMD/xeJi6+S4+JpzCJJKyjMl4/qmxwb2p7vEDqFrUe2m4nnoV4T8p1CjIQMA6DUlrmB50JcaSsnNVICKiaTCul89/im0lQ6j4lvQL/q/t+U= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936082; c=relaxed/simple; bh=T4VOISizeXwQxFildPjoTKkuyQeaeUMNwtucQCo8zKs=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=dORc3HGsvNYUlieNYgotMfZrMjXhA9w66M+xcReGlPzbUrm/csDQlSkyx+u9stqCQI/dp8aMuQaPshlK2l3aMEwQnI2huKiMDDfAMavIyhk85jHGNfH0fO9EN+wZboIkAMpq1W//mVnNmiYDvHWyEXcmDy85gmZHF/bt/IJ2Uis= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=U2Ckyu5G; arc=none smtp.client-ip=192.198.163.13 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="U2Ckyu5G" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936080; x=1740472080; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=T4VOISizeXwQxFildPjoTKkuyQeaeUMNwtucQCo8zKs=; b=U2Ckyu5GY2k4Z826Sc2Di6nwSqajyLLpHUL8tkcoIO0sfMKlveZCRb4t zHuKT1/fHWxbwyytxdcJp54rUp6I4tApOJUhfnSMM3FqnJCmM4g6kw0fH plTq2Brq79QrFr/SBBs30/KrxYqqsZ7KwkYyNj0VR0Lr8mTi7FzAwqYZq pnngbTDe1i9FshY55oWX7z8q+4NpFWg17t9LhEnOH15DBnCmlnduFNBwY W0MlVY4VwY1hNUnLPAb6rrPDMWlsVbnml1Ek8EWts8xj8TENJrMY5WndC f/8kg09TkfT3izG7GCa8BVVS62MNe0ELPR7dQdLCPGrIO87+WsK/n9xfw Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="6155287" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6155287" Received: from orviesa009.jf.intel.com ([10.64.159.149]) by fmvoesa107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:27:59 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6615573" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa009-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:27:58 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 033/130] KVM: TDX: Add helper function to read TDX metadata in array Date: Mon, 26 Feb 2024 00:25:35 -0800 Message-Id: <72b528863d14b322df553efa285ef4637ee67581.1708933498.git.isaku.yamahata@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata To read meta data in series, use table. Instead of metadata_read(fid0, &data0); metadata_read(...); ... table = { {fid0, &data0}, ...}; metadata-read(tables). TODO: Once the TDX host code introduces its framework to read TDX metadata, drop this patch and convert the code that uses this. Signed-off-by: Isaku Yamahata --- v18: - newly added --- arch/x86/kvm/vmx/tdx.c | 45 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 45 insertions(+) diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c index cde971122c1e..dce21f675155 100644 --- a/arch/x86/kvm/vmx/tdx.c +++ b/arch/x86/kvm/vmx/tdx.c @@ -6,6 +6,7 @@ #include "capabilities.h" #include "x86_ops.h" #include "x86.h" +#include "tdx_arch.h" #include "tdx.h" #undef pr_fmt @@ -39,6 +40,50 @@ static void __used tdx_guest_keyid_free(int keyid) ida_free(&tdx_guest_keyid_pool, keyid); } +#define TDX_MD_MAP(_fid, _ptr) \ + { .fid = MD_FIELD_ID_##_fid, \ + .ptr = (_ptr), } + +struct tdx_md_map { + u64 fid; + void *ptr; +}; + +static size_t tdx_md_element_size(u64 fid) +{ + switch (TDX_MD_ELEMENT_SIZE_CODE(fid)) { + case TDX_MD_ELEMENT_SIZE_8BITS: + return 1; + case TDX_MD_ELEMENT_SIZE_16BITS: + return 2; + case TDX_MD_ELEMENT_SIZE_32BITS: + return 4; + case TDX_MD_ELEMENT_SIZE_64BITS: + return 8; + default: + WARN_ON_ONCE(1); + return 0; + } +} + +static int __used tdx_md_read(struct tdx_md_map *maps, int nr_maps) +{ + struct tdx_md_map *m; + int ret, i; + u64 tmp; + + for (i = 0; i < nr_maps; i++) { + m = &maps[i]; + ret = tdx_sys_metadata_field_read(m->fid, &tmp); + if (ret) + return ret; + + memcpy(m->ptr, &tmp, tdx_md_element_size(m->fid)); + } + + return 0; +} + static int __init tdx_module_setup(void) { int ret; From patchwork Mon Feb 26 08:25:36 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571462 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2648358AD6; Mon, 26 Feb 2024 08:28:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.13 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936083; cv=none; b=ADl7j/XI4HIG/rXcG87UHUPSwJfmhbu4Dv5Yh6Vb4/hs9hmJSFLvd59Skai3dUwXIkazZQc4PAcg71JxlQaU/VwbhiAbrG958XOVlLhYoAuducPoVOFM5nXkIgkJBp3SH0XmYowxFIEY5zl1hSSf571lUTv0kAiAubOzoaIlhmE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936083; c=relaxed/simple; bh=K69f0zNuV/UzQ9kbwGm3AyR8yiZyeyS3gj6Lw7kZQdg=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=qv+yZ/DhCi/unAL6B8ZLmX6h6SIZX57vIcXgLK2B+RqBAF/QuZX7gliJ+sa0vkXWuINGi2FdtFK/N8w2lt0+ahBNc101UsSyGIQiAfyV95oQjeTa2AeLJg/tZ1iNUV/DxblV+JPPe0cpOvN1BQOpT4wn5742FnZTmoAZZNtca8g= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=PhExB/FN; arc=none smtp.client-ip=192.198.163.13 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="PhExB/FN" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936081; x=1740472081; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=K69f0zNuV/UzQ9kbwGm3AyR8yiZyeyS3gj6Lw7kZQdg=; b=PhExB/FN9Mbs5mJnYZEQ351zxzrc/UjJSQKhCo2v//8jRsokNIwnT+jS UqkgPKqI1oV8yDFYf0J5G4+tDiNfA8ajlDEE1oUaZ3HGWAyByvoZjl3zo feWdtTZTRslyC7DgdFCxuU4BNrwHuxBB5/MRVbMx1H4letvInF/fuiIxF TJjQIExyPltW+vxNlOYQhjwQP6AnNDDcT/7NZTd8BCHb6nkjNFHzR2F74 J8pGjXSoEcn/uZWHWuSgrLweaNqjWdPjx3pDpGfeLb/HrCGu4F2FPmRDj lW8KGG5YofQSMzNFYunOQLfLtFtp9bno7clhiIbzzfT6gSnwy6gRYFjDE A==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="6155292" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6155292" Received: from orviesa009.jf.intel.com ([10.64.159.149]) by fmvoesa107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:27:59 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6615583" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa009-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:27:59 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 034/130] KVM: TDX: Get system-wide info about TDX module on initialization Date: Mon, 26 Feb 2024 00:25:36 -0800 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata TDX KVM needs system-wide information about the TDX module, store it in struct tdx_info. Signed-off-by: Isaku Yamahata Reviewed-by: Binbin Wu --- v19: - Added features0 - Use tdx_sys_metadata_read() - Fix error recovery path by Yuan Change v18: - Newly Added Signed-off-by: Isaku Yamahata --- arch/x86/include/uapi/asm/kvm.h | 11 +++++ arch/x86/kvm/vmx/main.c | 9 +++- arch/x86/kvm/vmx/tdx.c | 80 ++++++++++++++++++++++++++++++++- arch/x86/kvm/vmx/x86_ops.h | 2 + 4 files changed, 100 insertions(+), 2 deletions(-) diff --git a/arch/x86/include/uapi/asm/kvm.h b/arch/x86/include/uapi/asm/kvm.h index aa7a56a47564..45b2c2304491 100644 --- a/arch/x86/include/uapi/asm/kvm.h +++ b/arch/x86/include/uapi/asm/kvm.h @@ -567,4 +567,15 @@ struct kvm_pmu_event_filter { #define KVM_X86_TDX_VM 2 #define KVM_X86_SNP_VM 3 +#define KVM_TDX_CPUID_NO_SUBLEAF ((__u32)-1) + +struct kvm_tdx_cpuid_config { + __u32 leaf; + __u32 sub_leaf; + __u32 eax; + __u32 ebx; + __u32 ecx; + __u32 edx; +}; + #endif /* _ASM_X86_KVM_H */ diff --git a/arch/x86/kvm/vmx/main.c b/arch/x86/kvm/vmx/main.c index fa19682b366c..a948a6959ac7 100644 --- a/arch/x86/kvm/vmx/main.c +++ b/arch/x86/kvm/vmx/main.c @@ -32,6 +32,13 @@ static __init int vt_hardware_setup(void) return 0; } +static void vt_hardware_unsetup(void) +{ + if (enable_tdx) + tdx_hardware_unsetup(); + vmx_hardware_unsetup(); +} + static int vt_vm_init(struct kvm *kvm) { if (is_td(kvm)) @@ -54,7 +61,7 @@ struct kvm_x86_ops vt_x86_ops __initdata = { .check_processor_compatibility = vmx_check_processor_compat, - .hardware_unsetup = vmx_hardware_unsetup, + .hardware_unsetup = vt_hardware_unsetup, /* TDX cpu enablement is done by tdx_hardware_setup(). */ .hardware_enable = vmx_hardware_enable, diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c index dce21f675155..5edfb99abb89 100644 --- a/arch/x86/kvm/vmx/tdx.c +++ b/arch/x86/kvm/vmx/tdx.c @@ -40,6 +40,21 @@ static void __used tdx_guest_keyid_free(int keyid) ida_free(&tdx_guest_keyid_pool, keyid); } +struct tdx_info { + u64 features0; + u64 attributes_fixed0; + u64 attributes_fixed1; + u64 xfam_fixed0; + u64 xfam_fixed1; + + u16 num_cpuid_config; + /* This must the last member. */ + DECLARE_FLEX_ARRAY(struct kvm_tdx_cpuid_config, cpuid_configs); +}; + +/* Info about the TDX module. */ +static struct tdx_info *tdx_info; + #define TDX_MD_MAP(_fid, _ptr) \ { .fid = MD_FIELD_ID_##_fid, \ .ptr = (_ptr), } @@ -66,7 +81,7 @@ static size_t tdx_md_element_size(u64 fid) } } -static int __used tdx_md_read(struct tdx_md_map *maps, int nr_maps) +static int tdx_md_read(struct tdx_md_map *maps, int nr_maps) { struct tdx_md_map *m; int ret, i; @@ -84,9 +99,26 @@ static int __used tdx_md_read(struct tdx_md_map *maps, int nr_maps) return 0; } +#define TDX_INFO_MAP(_field_id, _member) \ + TD_SYSINFO_MAP(_field_id, struct tdx_info, _member) + static int __init tdx_module_setup(void) { + u16 num_cpuid_config; int ret; + u32 i; + + struct tdx_md_map mds[] = { + TDX_MD_MAP(NUM_CPUID_CONFIG, &num_cpuid_config), + }; + + struct tdx_metadata_field_mapping fields[] = { + TDX_INFO_MAP(FEATURES0, features0), + TDX_INFO_MAP(ATTRS_FIXED0, attributes_fixed0), + TDX_INFO_MAP(ATTRS_FIXED1, attributes_fixed1), + TDX_INFO_MAP(XFAM_FIXED0, xfam_fixed0), + TDX_INFO_MAP(XFAM_FIXED1, xfam_fixed1), + }; ret = tdx_enable(); if (ret) { @@ -94,7 +126,48 @@ static int __init tdx_module_setup(void) return ret; } + ret = tdx_md_read(mds, ARRAY_SIZE(mds)); + if (ret) + return ret; + + tdx_info = kzalloc(sizeof(*tdx_info) + + sizeof(*tdx_info->cpuid_configs) * num_cpuid_config, + GFP_KERNEL); + if (!tdx_info) + return -ENOMEM; + tdx_info->num_cpuid_config = num_cpuid_config; + + ret = tdx_sys_metadata_read(fields, ARRAY_SIZE(fields), tdx_info); + if (ret) + goto error_out; + + for (i = 0; i < num_cpuid_config; i++) { + struct kvm_tdx_cpuid_config *c = &tdx_info->cpuid_configs[i]; + u64 leaf, eax_ebx, ecx_edx; + struct tdx_md_map cpuids[] = { + TDX_MD_MAP(CPUID_CONFIG_LEAVES + i, &leaf), + TDX_MD_MAP(CPUID_CONFIG_VALUES + i * 2, &eax_ebx), + TDX_MD_MAP(CPUID_CONFIG_VALUES + i * 2 + 1, &ecx_edx), + }; + + ret = tdx_md_read(cpuids, ARRAY_SIZE(cpuids)); + if (ret) + goto error_out; + + c->leaf = (u32)leaf; + c->sub_leaf = leaf >> 32; + c->eax = (u32)eax_ebx; + c->ebx = eax_ebx >> 32; + c->ecx = (u32)ecx_edx; + c->edx = ecx_edx >> 32; + } + return 0; + +error_out: + /* kfree() accepts NULL. */ + kfree(tdx_info); + return ret; } bool tdx_is_vm_type_supported(unsigned long type) @@ -162,3 +235,8 @@ int __init tdx_hardware_setup(struct kvm_x86_ops *x86_ops) out: return r; } + +void tdx_hardware_unsetup(void) +{ + kfree(tdx_info); +} diff --git a/arch/x86/kvm/vmx/x86_ops.h b/arch/x86/kvm/vmx/x86_ops.h index f4da88a228d0..e8cb4ae81cf1 100644 --- a/arch/x86/kvm/vmx/x86_ops.h +++ b/arch/x86/kvm/vmx/x86_ops.h @@ -136,9 +136,11 @@ void vmx_setup_mce(struct kvm_vcpu *vcpu); #ifdef CONFIG_INTEL_TDX_HOST int __init tdx_hardware_setup(struct kvm_x86_ops *x86_ops); +void tdx_hardware_unsetup(void); bool tdx_is_vm_type_supported(unsigned long type); #else static inline int tdx_hardware_setup(struct kvm_x86_ops *x86_ops) { return -EOPNOTSUPP; } +static inline void tdx_hardware_unsetup(void) {} static inline bool tdx_is_vm_type_supported(unsigned long type) { return false; } #endif From patchwork Mon Feb 26 08:25:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571463 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8562659B62; Mon, 26 Feb 2024 08:28:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.13 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936084; cv=none; b=Y8+3Zm7lymClEg9HEBsyuSXYIIagcQME9VMmh20q5DNUMBDQFkI0muCK9m9RuVMOu3i7eyD58xLzPRdaGB/gTz6eAk2e63bsYufGBqMd420xRqgVYw+JptOn7qcfbu66E+sZ5rR/WM59gQBqlIwSuEo+9PXX5GZ1Gf75TYIPUW4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936084; c=relaxed/simple; bh=IbPdSA5+Gq2omZqSV56w9i5tpK2g28tpRmPxwftvXo8=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=QhGpcSMe4jKUKaMLJ+pDEjr9nE96LfmyKYL/oqPWwsKBxO73kggU7Di3R+JGKVNOQZtdJJmwosjRqDUyve5/nxn5pgSH6Yvdw3rzZ4DrBpdzkhnOLRwI6Ic8BYER/sqXDtRTVtQNmKiuPpWMrnGIfBdMyedaW2n0UcOwUHQqr6A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=YkJ6wtO/; arc=none smtp.client-ip=192.198.163.13 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="YkJ6wtO/" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936082; x=1740472082; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=IbPdSA5+Gq2omZqSV56w9i5tpK2g28tpRmPxwftvXo8=; b=YkJ6wtO/jWaMnq14wD8f/goh5nbMt4WBInp6R3fy20Jxv2Y93In9PzGf w21CwfsrHV1rgZ8Ns/lP2ooTZEG5oGWmb56wD+LHOwERhUebr+VT1bgar +B9WZpGoIqyLLu1T1kkWgY2QgfsQV0j1gJ4NgtDF0rL/foNnzx6GMD3mL vsyuYv4RecwM7HWpSIA0MeyTHYKkBRNKZ0XONzEpvWOrLYDXYudFFC5hQ QWWWjgOuqoMb1rbOD/6H188IHaO+8zmH988D5M7rgBa2gr8i3E0HJqS/J nK7zRTu0Du5mnPPXi2eGGcL3DMgw0HwXOiZyJS5cYb2FTcsqcCFPN9q3m A==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="6155300" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6155300" Received: from orviesa009.jf.intel.com ([10.64.159.149]) by fmvoesa107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:00 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6615606" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa009-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:00 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 035/130] KVM: TDX: Add place holder for TDX VM specific mem_enc_op ioctl Date: Mon, 26 Feb 2024 00:25:37 -0800 Message-Id: <079540d563ab0f5d8991ad4d3b1546c05dc2fb01.1708933498.git.isaku.yamahata@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata KVM_MEMORY_ENCRYPT_OP was introduced for VM-scoped operations specific for guest state-protected VM. It defined subcommands for technology-specific operations under KVM_MEMORY_ENCRYPT_OP. Despite its name, the subcommands are not limited to memory encryption, but various technology-specific operations are defined. It's natural to repurpose KVM_MEMORY_ENCRYPT_OP for TDX specific operations and define subcommands. TDX requires VM-scoped TDX-specific operations for device model, for example, qemu. Getting system-wide parameters, TDX-specific VM initialization. Add a place holder function for TDX specific VM-scoped ioctl as mem_enc_op. TDX specific sub-commands will be added to retrieve/pass TDX specific parameters. Make mem_enc_ioctl non-optional as it's always filled. Signed-off-by: Isaku Yamahata --- v15: - change struct kvm_tdx_cmd to drop unused member. --- arch/x86/include/asm/kvm-x86-ops.h | 2 +- arch/x86/include/uapi/asm/kvm.h | 26 ++++++++++++++++++++++++++ arch/x86/kvm/vmx/main.c | 10 ++++++++++ arch/x86/kvm/vmx/tdx.c | 26 ++++++++++++++++++++++++++ arch/x86/kvm/vmx/x86_ops.h | 4 ++++ arch/x86/kvm/x86.c | 4 ---- 6 files changed, 67 insertions(+), 5 deletions(-) diff --git a/arch/x86/include/asm/kvm-x86-ops.h b/arch/x86/include/asm/kvm-x86-ops.h index 8be71a5c5c87..00b371d9a1ca 100644 --- a/arch/x86/include/asm/kvm-x86-ops.h +++ b/arch/x86/include/asm/kvm-x86-ops.h @@ -123,7 +123,7 @@ KVM_X86_OP(enter_smm) KVM_X86_OP(leave_smm) KVM_X86_OP(enable_smi_window) #endif -KVM_X86_OP_OPTIONAL(mem_enc_ioctl) +KVM_X86_OP(mem_enc_ioctl) KVM_X86_OP_OPTIONAL(mem_enc_register_region) KVM_X86_OP_OPTIONAL(mem_enc_unregister_region) KVM_X86_OP_OPTIONAL(vm_copy_enc_context_from) diff --git a/arch/x86/include/uapi/asm/kvm.h b/arch/x86/include/uapi/asm/kvm.h index 45b2c2304491..9ea46d143bef 100644 --- a/arch/x86/include/uapi/asm/kvm.h +++ b/arch/x86/include/uapi/asm/kvm.h @@ -567,6 +567,32 @@ struct kvm_pmu_event_filter { #define KVM_X86_TDX_VM 2 #define KVM_X86_SNP_VM 3 +/* Trust Domain eXtension sub-ioctl() commands. */ +enum kvm_tdx_cmd_id { + KVM_TDX_CAPABILITIES = 0, + + KVM_TDX_CMD_NR_MAX, +}; + +struct kvm_tdx_cmd { + /* enum kvm_tdx_cmd_id */ + __u32 id; + /* flags for sub-commend. If sub-command doesn't use this, set zero. */ + __u32 flags; + /* + * data for each sub-command. An immediate or a pointer to the actual + * data in process virtual address. If sub-command doesn't use it, + * set zero. + */ + __u64 data; + /* + * Auxiliary error code. The sub-command may return TDX SEAMCALL + * status code in addition to -Exxx. + * Defined for consistency with struct kvm_sev_cmd. + */ + __u64 error; +}; + #define KVM_TDX_CPUID_NO_SUBLEAF ((__u32)-1) struct kvm_tdx_cpuid_config { diff --git a/arch/x86/kvm/vmx/main.c b/arch/x86/kvm/vmx/main.c index a948a6959ac7..082e82ce6580 100644 --- a/arch/x86/kvm/vmx/main.c +++ b/arch/x86/kvm/vmx/main.c @@ -47,6 +47,14 @@ static int vt_vm_init(struct kvm *kvm) return vmx_vm_init(kvm); } +static int vt_mem_enc_ioctl(struct kvm *kvm, void __user *argp) +{ + if (!is_td(kvm)) + return -ENOTTY; + + return tdx_vm_ioctl(kvm, argp); +} + #define VMX_REQUIRED_APICV_INHIBITS \ (BIT(APICV_INHIBIT_REASON_DISABLE)| \ BIT(APICV_INHIBIT_REASON_ABSENT) | \ @@ -200,6 +208,8 @@ struct kvm_x86_ops vt_x86_ops __initdata = { .vcpu_deliver_sipi_vector = kvm_vcpu_deliver_sipi_vector, .get_untagged_addr = vmx_get_untagged_addr, + + .mem_enc_ioctl = vt_mem_enc_ioctl, }; struct kvm_x86_init_ops vt_init_ops __initdata = { diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c index 5edfb99abb89..07a3f0f75f87 100644 --- a/arch/x86/kvm/vmx/tdx.c +++ b/arch/x86/kvm/vmx/tdx.c @@ -55,6 +55,32 @@ struct tdx_info { /* Info about the TDX module. */ static struct tdx_info *tdx_info; +int tdx_vm_ioctl(struct kvm *kvm, void __user *argp) +{ + struct kvm_tdx_cmd tdx_cmd; + int r; + + if (copy_from_user(&tdx_cmd, argp, sizeof(struct kvm_tdx_cmd))) + return -EFAULT; + if (tdx_cmd.error) + return -EINVAL; + + mutex_lock(&kvm->lock); + + switch (tdx_cmd.id) { + default: + r = -EINVAL; + goto out; + } + + if (copy_to_user(argp, &tdx_cmd, sizeof(struct kvm_tdx_cmd))) + r = -EFAULT; + +out: + mutex_unlock(&kvm->lock); + return r; +} + #define TDX_MD_MAP(_fid, _ptr) \ { .fid = MD_FIELD_ID_##_fid, \ .ptr = (_ptr), } diff --git a/arch/x86/kvm/vmx/x86_ops.h b/arch/x86/kvm/vmx/x86_ops.h index e8cb4ae81cf1..f6c57ad44f80 100644 --- a/arch/x86/kvm/vmx/x86_ops.h +++ b/arch/x86/kvm/vmx/x86_ops.h @@ -138,10 +138,14 @@ void vmx_setup_mce(struct kvm_vcpu *vcpu); int __init tdx_hardware_setup(struct kvm_x86_ops *x86_ops); void tdx_hardware_unsetup(void); bool tdx_is_vm_type_supported(unsigned long type); + +int tdx_vm_ioctl(struct kvm *kvm, void __user *argp); #else static inline int tdx_hardware_setup(struct kvm_x86_ops *x86_ops) { return -EOPNOTSUPP; } static inline void tdx_hardware_unsetup(void) {} static inline bool tdx_is_vm_type_supported(unsigned long type) { return false; } + +static inline int tdx_vm_ioctl(struct kvm *kvm, void __user *argp) { return -EOPNOTSUPP; } #endif #endif /* __KVM_X86_VMX_X86_OPS_H */ diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 442b356e4939..c459a5e9e520 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -7247,10 +7247,6 @@ int kvm_arch_vm_ioctl(struct file *filp, unsigned int ioctl, unsigned long arg) goto out; } case KVM_MEMORY_ENCRYPT_OP: { - r = -ENOTTY; - if (!kvm_x86_ops.mem_enc_ioctl) - goto out; - r = static_call(kvm_x86_mem_enc_ioctl)(kvm, argp); break; } From patchwork Mon Feb 26 08:25:38 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571464 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C825059B73; Mon, 26 Feb 2024 08:28:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.13 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936084; cv=none; b=UTa68KEDXNbMAHcoAf7bsLvhMDCx2N6Lmy7u3g0JOWicwndgrK1Jers3AdC2f7Sgk6UwNb13dqHQVsMkQdvLOKDSJIA0hWYeTQfZv3+1zgZxJKN50lLtbW3ORelf6M8HwqXyOtBLaSWk0edDzoQ1Swu6qgi8lCTMxvP4MAS0B4I= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936084; c=relaxed/simple; bh=l+3Kq91FdtoMuKPtJHkyFU2e9BGW3nCpaz0IbZQ7Utk=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=bWw0kzDEGHjA7TOGS1XkQKtEgAVnLFGFcuTSofaYR3kP2dm82VOO+ODsCIxEwF34Ccx2AzmEu7PCpNHz/ztsrmhAnG8VXigUlQ7DCeuNN7BFCp5Otc/zdDnzt3IwH2hASLFatjn1peJV5DOfGtHhFwyHQBSj69A9u9xX+m8kn5U= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=c3c+959t; arc=none smtp.client-ip=192.198.163.13 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="c3c+959t" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936083; x=1740472083; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=l+3Kq91FdtoMuKPtJHkyFU2e9BGW3nCpaz0IbZQ7Utk=; b=c3c+959txCD2e1cHNnEVTrJDA8qmnTnCRlTxEbnJ410UEfwy5RdSM4bF u26yQnBLGyqEYQxDNobaUs9XrvzhVunc8LpHgNdR3EtKb9PF0iwL0ytcZ HpBaSER0Td2tSV+/HI8BT4dDiXmYsdFzY9p71lCAbJKlL7RejLnqLp6iu +GCSJfjG7e3HzZ/rwEPKjWyQqp/ZANHx/x17h+ZrGErd2LhV7cTp9s3g1 KgC+6udG0nMtWS8MQqAAwzJsDNb31Jsp3/yE0KdIFaWAPVb7MC8xdutf9 4uboLPs/qgQZy11/icakmdFTcXdhiRkbz0LfT5M1UYS0aGnl6qqdnFn2Z w==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="6155306" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6155306" Received: from orviesa009.jf.intel.com ([10.64.159.149]) by fmvoesa107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:01 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6615624" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa009-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:01 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com, Sean Christopherson Subject: [PATCH v19 036/130] KVM: TDX: x86: Add ioctl to get TDX systemwide parameters Date: Mon, 26 Feb 2024 00:25:38 -0800 Message-Id: <167f8f7e9b19154d30c7fe8f733f947592eb244c.1708933498.git.isaku.yamahata@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Sean Christopherson Implement an ioctl to get system-wide parameters for TDX. Although the function is systemwide, vm scoped mem_enc ioctl works for userspace VMM like qemu and device scoped version is not define, re-use vm scoped mem_enc. Signed-off-by: Sean Christopherson Signed-off-by: Isaku Yamahata --- v18: - drop the use of tdhsysinfo_struct and TDH.SYS.INFO, use TDH.SYS.RD(). For that, dynamically allocate/free tdx_info. - drop the change of tools/arch/x86/include/uapi/asm/kvm.h. v14 -> v15: - ABI change: added supported_gpaw and reserved area. Signed-off-by: Isaku Yamahata --- arch/x86/include/uapi/asm/kvm.h | 17 ++++++++++ arch/x86/kvm/vmx/tdx.c | 56 +++++++++++++++++++++++++++++++++ arch/x86/kvm/vmx/tdx.h | 3 ++ 3 files changed, 76 insertions(+) diff --git a/arch/x86/include/uapi/asm/kvm.h b/arch/x86/include/uapi/asm/kvm.h index 9ea46d143bef..e28189c81691 100644 --- a/arch/x86/include/uapi/asm/kvm.h +++ b/arch/x86/include/uapi/asm/kvm.h @@ -604,4 +604,21 @@ struct kvm_tdx_cpuid_config { __u32 edx; }; +/* supported_gpaw */ +#define TDX_CAP_GPAW_48 (1 << 0) +#define TDX_CAP_GPAW_52 (1 << 1) + +struct kvm_tdx_capabilities { + __u64 attrs_fixed0; + __u64 attrs_fixed1; + __u64 xfam_fixed0; + __u64 xfam_fixed1; + __u32 supported_gpaw; + __u32 padding; + __u64 reserved[251]; + + __u32 nr_cpuid_configs; + struct kvm_tdx_cpuid_config cpuid_configs[]; +}; + #endif /* _ASM_X86_KVM_H */ diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c index 07a3f0f75f87..816ccdb4bc41 100644 --- a/arch/x86/kvm/vmx/tdx.c +++ b/arch/x86/kvm/vmx/tdx.c @@ -6,6 +6,7 @@ #include "capabilities.h" #include "x86_ops.h" #include "x86.h" +#include "mmu.h" #include "tdx_arch.h" #include "tdx.h" @@ -55,6 +56,58 @@ struct tdx_info { /* Info about the TDX module. */ static struct tdx_info *tdx_info; +static int tdx_get_capabilities(struct kvm_tdx_cmd *cmd) +{ + struct kvm_tdx_capabilities __user *user_caps; + struct kvm_tdx_capabilities *caps = NULL; + int ret = 0; + + if (cmd->flags) + return -EINVAL; + + caps = kmalloc(sizeof(*caps), GFP_KERNEL); + if (!caps) + return -ENOMEM; + + user_caps = (void __user *)cmd->data; + if (copy_from_user(caps, user_caps, sizeof(*caps))) { + ret = -EFAULT; + goto out; + } + + if (caps->nr_cpuid_configs < tdx_info->num_cpuid_config) { + ret = -E2BIG; + goto out; + } + + *caps = (struct kvm_tdx_capabilities) { + .attrs_fixed0 = tdx_info->attributes_fixed0, + .attrs_fixed1 = tdx_info->attributes_fixed1, + .xfam_fixed0 = tdx_info->xfam_fixed0, + .xfam_fixed1 = tdx_info->xfam_fixed1, + .supported_gpaw = TDX_CAP_GPAW_48 | + ((kvm_get_shadow_phys_bits() >= 52 && + cpu_has_vmx_ept_5levels()) ? TDX_CAP_GPAW_52 : 0), + .nr_cpuid_configs = tdx_info->num_cpuid_config, + .padding = 0, + }; + + if (copy_to_user(user_caps, caps, sizeof(*caps))) { + ret = -EFAULT; + goto out; + } + if (copy_to_user(user_caps->cpuid_configs, &tdx_info->cpuid_configs, + tdx_info->num_cpuid_config * + sizeof(tdx_info->cpuid_configs[0]))) { + ret = -EFAULT; + } + +out: + /* kfree() accepts NULL. */ + kfree(caps); + return ret; +} + int tdx_vm_ioctl(struct kvm *kvm, void __user *argp) { struct kvm_tdx_cmd tdx_cmd; @@ -68,6 +121,9 @@ int tdx_vm_ioctl(struct kvm *kvm, void __user *argp) mutex_lock(&kvm->lock); switch (tdx_cmd.id) { + case KVM_TDX_CAPABILITIES: + r = tdx_get_capabilities(&tdx_cmd); + break; default: r = -EINVAL; goto out; diff --git a/arch/x86/kvm/vmx/tdx.h b/arch/x86/kvm/vmx/tdx.h index 473013265bd8..22c0b57f69ca 100644 --- a/arch/x86/kvm/vmx/tdx.h +++ b/arch/x86/kvm/vmx/tdx.h @@ -3,6 +3,9 @@ #define __KVM_X86_TDX_H #ifdef CONFIG_INTEL_TDX_HOST + +#include "tdx_ops.h" + struct kvm_tdx { struct kvm kvm; /* TDX specific members follow. */ From patchwork Mon Feb 26 08:25:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571465 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9FF775A7B6; Mon, 26 Feb 2024 08:28:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.13 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936085; cv=none; b=c1DD+m7uSZV9CP4mcnpn1SXsnGRrqwplUR+lVmseajTyGq8ARySqU/gxkYDEjObzZqGOfKFbNQTQ6FRMwRFAUAk6YkAIMUUCBCY5M/PzVLzD/ayaknEu3hzyMhqW1S3dbTFKLKSZr57nv7a9q2xPWRdM2Oybd1dIM8H+aBH/e1s= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936085; c=relaxed/simple; bh=L+VqlxX10T0o55WrNVdRU8lzKiSJUabdBC/i3CdqN64=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=jl6YvDi9W2VjWwzor+FuQJmLD7T143x8wVFI0gMP8kKILoYpvtxLMvY0Cq3jetUSpCc4cwSOsuopCWgwBE7iIaxBiy0OlXIG4ihxwpA95IT4SVGzNyTzuMehEkgNFvWW+ckMhKH68Utn0lztAAhCP6Usgj+gQzOIE3zeHE2NqQY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=Vi/SzKHI; arc=none smtp.client-ip=192.198.163.13 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="Vi/SzKHI" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936083; x=1740472083; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=L+VqlxX10T0o55WrNVdRU8lzKiSJUabdBC/i3CdqN64=; b=Vi/SzKHIOtY09RfVFqJMxHgtvTp0fpARUV6TvHOWtFNH/j9Jmpv7IOcR DVuAc3ni/uRJSTVhy936TuoaB0KzI0uexDfMUBF3NdEtNZJ3Il3T8oJgo ntjLOzcfePndo4Dam/KGKNweus4z39HWvoTYn+0V1INJchEYkNRUwilmV WttPd5CvAA950kdxoowlhG2W3MU6KBDurS6doa93Q/uHFPbVlIQKMdJtW R5N8g4GLvTVigkGrjD83FthREv9zvwg/UIvtgEKJMJr7TUXyMrMJmRZeO v1AXVmaHv2UNYcQPAZot0plp4kcmK+6rEaFMesi+wGnktlTcpGepMJFwm w==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="6155312" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6155312" Received: from orviesa009.jf.intel.com ([10.64.159.149]) by fmvoesa107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:02 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6615645" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa009-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:01 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 037/130] KVM: TDX: Make KVM_CAP_MAX_VCPUS backend specific Date: Mon, 26 Feb 2024 00:25:39 -0800 Message-Id: <9bd868a287599eb2a854f6983f13b4500f47d2ae.1708933498.git.isaku.yamahata@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata TDX has its own limitation on the maximum number of vcpus that the guest can accommodate. Allow x86 kvm backend to implement its own KVM_ENABLE_CAP handler and implement TDX backend for KVM_CAP_MAX_VCPUS. user space VMM, e.g. qemu, can specify its value instead of KVM_MAX_VCPUS. When creating TD (TDH.MNG.INIT), the maximum number of vcpu needs to be specified as struct td_params_struct. and the value is a part of measurement. The user space has to specify the value somehow. There are two options for it. option 1. API (Set KVM_CAP_MAX_VCPU) to specify the value (this patch) option 2. Add max_vcpu as a parameter to initialize the guest. (TDG.MNG.INIT) The flow will be create VM (KVM_CREATE_VM), create VCPUs (KVM_CREATE_VCPU), initialize TDX part of VM and, initialize TDX part of vcpu. Because the creation of vcpu is independent from TDX VM initialization, Choose the option 1. The work flow will be, KVM_CREATE_VM, set KVM_CAP_MAX_VCPU and, KVM_CREATE_VCPU. Signed-off-by: Isaku Yamahata Signed-off-by: Isaku Yamahata --- v18: - use TDX instead of "x86, tdx" in subject - use min(max_vcpu, TDX_MAX_VCPU) instead of min3(max_vcpu, KVM_MAX_VCPU, TDX_MAX_VCPU) - make "if (KVM_MAX_VCPU) and if (TDX_MAX_VCPU)" into one if statement --- arch/x86/include/asm/kvm-x86-ops.h | 2 ++ arch/x86/include/asm/kvm_host.h | 2 ++ arch/x86/kvm/vmx/main.c | 22 ++++++++++++++++++++++ arch/x86/kvm/vmx/tdx.c | 29 +++++++++++++++++++++++++++++ arch/x86/kvm/vmx/x86_ops.h | 5 +++++ arch/x86/kvm/x86.c | 4 ++++ 6 files changed, 64 insertions(+) diff --git a/arch/x86/include/asm/kvm-x86-ops.h b/arch/x86/include/asm/kvm-x86-ops.h index 00b371d9a1ca..1b0dacc6b6c0 100644 --- a/arch/x86/include/asm/kvm-x86-ops.h +++ b/arch/x86/include/asm/kvm-x86-ops.h @@ -21,6 +21,8 @@ KVM_X86_OP(hardware_unsetup) KVM_X86_OP(has_emulated_msr) KVM_X86_OP(vcpu_after_set_cpuid) KVM_X86_OP(is_vm_type_supported) +KVM_X86_OP_OPTIONAL(max_vcpus); +KVM_X86_OP_OPTIONAL(vm_enable_cap) KVM_X86_OP(vm_init) KVM_X86_OP_OPTIONAL(vm_destroy) KVM_X86_OP_OPTIONAL_RET0(vcpu_precreate) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 37cda8aa07b6..cf8eb46b3a20 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1604,7 +1604,9 @@ struct kvm_x86_ops { void (*vcpu_after_set_cpuid)(struct kvm_vcpu *vcpu); bool (*is_vm_type_supported)(unsigned long vm_type); + int (*max_vcpus)(struct kvm *kvm); unsigned int vm_size; + int (*vm_enable_cap)(struct kvm *kvm, struct kvm_enable_cap *cap); int (*vm_init)(struct kvm *kvm); void (*vm_destroy)(struct kvm *kvm); diff --git a/arch/x86/kvm/vmx/main.c b/arch/x86/kvm/vmx/main.c index 082e82ce6580..e8a1a7533eea 100644 --- a/arch/x86/kvm/vmx/main.c +++ b/arch/x86/kvm/vmx/main.c @@ -6,6 +6,7 @@ #include "nested.h" #include "pmu.h" #include "tdx.h" +#include "tdx_arch.h" static bool enable_tdx __ro_after_init; module_param_named(tdx, enable_tdx, bool, 0444); @@ -16,6 +17,17 @@ static bool vt_is_vm_type_supported(unsigned long type) (enable_tdx && tdx_is_vm_type_supported(type)); } +static int vt_max_vcpus(struct kvm *kvm) +{ + if (!kvm) + return KVM_MAX_VCPUS; + + if (is_td(kvm)) + return min(kvm->max_vcpus, TDX_MAX_VCPUS); + + return kvm->max_vcpus; +} + static __init int vt_hardware_setup(void) { int ret; @@ -39,6 +51,14 @@ static void vt_hardware_unsetup(void) vmx_hardware_unsetup(); } +static int vt_vm_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap) +{ + if (is_td(kvm)) + return tdx_vm_enable_cap(kvm, cap); + + return -EINVAL; +} + static int vt_vm_init(struct kvm *kvm) { if (is_td(kvm)) @@ -77,7 +97,9 @@ struct kvm_x86_ops vt_x86_ops __initdata = { .has_emulated_msr = vmx_has_emulated_msr, .is_vm_type_supported = vt_is_vm_type_supported, + .max_vcpus = vt_max_vcpus, .vm_size = sizeof(struct kvm_vmx), + .vm_enable_cap = vt_vm_enable_cap, .vm_init = vt_vm_init, .vm_destroy = vmx_vm_destroy, diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c index 816ccdb4bc41..ee015f3ce2c9 100644 --- a/arch/x86/kvm/vmx/tdx.c +++ b/arch/x86/kvm/vmx/tdx.c @@ -56,6 +56,35 @@ struct tdx_info { /* Info about the TDX module. */ static struct tdx_info *tdx_info; +int tdx_vm_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap) +{ + int r; + + switch (cap->cap) { + case KVM_CAP_MAX_VCPUS: { + if (cap->flags || cap->args[0] == 0) + return -EINVAL; + if (cap->args[0] > KVM_MAX_VCPUS || + cap->args[0] > TDX_MAX_VCPUS) + return -E2BIG; + + mutex_lock(&kvm->lock); + if (kvm->created_vcpus) + r = -EBUSY; + else { + kvm->max_vcpus = cap->args[0]; + r = 0; + } + mutex_unlock(&kvm->lock); + break; + } + default: + r = -EINVAL; + break; + } + return r; +} + static int tdx_get_capabilities(struct kvm_tdx_cmd *cmd) { struct kvm_tdx_capabilities __user *user_caps; diff --git a/arch/x86/kvm/vmx/x86_ops.h b/arch/x86/kvm/vmx/x86_ops.h index f6c57ad44f80..0031a8d61589 100644 --- a/arch/x86/kvm/vmx/x86_ops.h +++ b/arch/x86/kvm/vmx/x86_ops.h @@ -139,12 +139,17 @@ int __init tdx_hardware_setup(struct kvm_x86_ops *x86_ops); void tdx_hardware_unsetup(void); bool tdx_is_vm_type_supported(unsigned long type); +int tdx_vm_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap); int tdx_vm_ioctl(struct kvm *kvm, void __user *argp); #else static inline int tdx_hardware_setup(struct kvm_x86_ops *x86_ops) { return -EOPNOTSUPP; } static inline void tdx_hardware_unsetup(void) {} static inline bool tdx_is_vm_type_supported(unsigned long type) { return false; } +static inline int tdx_vm_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap) +{ + return -EINVAL; +}; static inline int tdx_vm_ioctl(struct kvm *kvm, void __user *argp) { return -EOPNOTSUPP; } #endif diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index c459a5e9e520..3f16e9450d2f 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -4726,6 +4726,8 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) break; case KVM_CAP_MAX_VCPUS: r = KVM_MAX_VCPUS; + if (kvm_x86_ops.max_vcpus) + r = static_call(kvm_x86_max_vcpus)(kvm); break; case KVM_CAP_MAX_VCPU_ID: r = KVM_MAX_VCPU_IDS; @@ -6709,6 +6711,8 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm, break; default: r = -EINVAL; + if (kvm_x86_ops.vm_enable_cap) + r = static_call(kvm_x86_vm_enable_cap)(kvm, cap); break; } return r; From patchwork Mon Feb 26 08:25:40 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571466 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DC3425B66D; Mon, 26 Feb 2024 08:28:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.13 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936088; cv=none; b=SNxQ5imHEHJ8s6asLhQeOEl1kqLHNScvoPEEYmF4uY+j2li91LkHshvJ8jVTr6kt4pA1pvqK2vSG7PEV7ow0FUEKUWMhZFn9Bo33sluIBUs0UpA8waaLd+F0KsHQjWRjJ8j2VSwMw9e0vkDVC5ZSIQjuMweS0+URTPDEWSQ7nZw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936088; c=relaxed/simple; bh=ho7z1m9iFNpQz+LgRLwGYrNockeCsSGtj7yhSIt7oSk=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=pQR0J1kRZebrFEq1NlPNJdtQuiUaqPdC6THS5qWLi4Cx8SWxFq4MHxcseGwqM49eFzUezd5hdd7n3R4IXMiAluqsSXt2xkdH6Kt7bnsTN37zJQwdjd6PmBF77jOvRlk6tdopKu4c8tHp4L1GUlUB9Xse4HMthy+42m096HCgpII= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=SnszU8FW; arc=none smtp.client-ip=192.198.163.13 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="SnszU8FW" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936085; x=1740472085; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=ho7z1m9iFNpQz+LgRLwGYrNockeCsSGtj7yhSIt7oSk=; b=SnszU8FWf0pm6wxrWVUB2LTV5Hc7rjk+LVYPJP181Btz6nFCsONFQppn /roYnaKqeG6BlT5sJLRowut4S4R4Hy0MHRXkOrzvI5HopusYMTgGKardq y/orTEGmNDWSb9MqW5EeIAxMCVwEAqdJz49NqbcVvsrMTz/baXt0hTDJ1 RZPQ3+2Gg0/eh+zF8QNi/+tTzg3PKpqQGLTX3xmIpY6yGWe1t+6CKAbZw JpxX7TbnfacFS+UB8UYQXWB5O+0oLl8UQAKv+36TylyYzlEpOXrNEKp9P YzekRuOBPp1V4aU8kUbBtI32S45y+k8jWdgBuIlCXaPGtc9XBVxifANk0 Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="6155318" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6155318" Received: from orviesa009.jf.intel.com ([10.64.159.149]) by fmvoesa107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:02 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6615670" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa009-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:02 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com, Sean Christopherson Subject: [PATCH v19 038/130] KVM: TDX: create/destroy VM structure Date: Mon, 26 Feb 2024 00:25:40 -0800 Message-Id: <7a508f88e8c8b5199da85b7a9959882ddf390796.1708933498.git.isaku.yamahata@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata As the first step to create TDX guest, create/destroy VM struct. Assign TDX private Host Key ID (HKID) to the TDX guest for memory encryption and allocate extra pages for the TDX guest. On destruction, free allocated pages, and HKID. Before tearing down private page tables, TDX requires some resources of the guest TD to be destroyed (i.e. HKID must have been reclaimed, etc). Add mmu notifier release callback before tearing down private page tables for it. Add vm_free() of kvm_x86_ops hook at the end of kvm_arch_destroy_vm() because some per-VM TDX resources, e.g. TDR, need to be freed after other TDX resources, e.g. HKID, were freed. Co-developed-by: Kai Huang Signed-off-by: Kai Huang Signed-off-by: Sean Christopherson Signed-off-by: Isaku Yamahata --- v19: - fix check error code of TDH.PHYMEM.PAGE.RECLAIM. RCX and TDR. v18: - Use TDH.SYS.RD() instead of struct tdsysinfo_struct. - Rename tdx_reclaim_td_page() to tdx_reclaim_control_page() - return -EAGAIN on TDX_RND_NO_ENTROPY of TDH.MNG.CREATE(), TDH.MNG.ADDCX() - fix comment to remove extra the. - use true instead of 1 for boolean. - remove an extra white line. v16: - Simplified tdx_reclaim_page() - Reorganize the locking of tdx_release_hkid(), and use smp_call_mask() instead of smp_call_on_cpu() to hold spinlock to race with invalidation on releasing guest memfd Signed-off-by: Isaku Yamahata --- arch/x86/include/asm/kvm-x86-ops.h | 2 + arch/x86/include/asm/kvm_host.h | 2 + arch/x86/kvm/Kconfig | 3 +- arch/x86/kvm/mmu/mmu.c | 7 + arch/x86/kvm/vmx/main.c | 26 +- arch/x86/kvm/vmx/tdx.c | 475 ++++++++++++++++++++++++++++- arch/x86/kvm/vmx/tdx.h | 6 +- arch/x86/kvm/vmx/x86_ops.h | 6 + arch/x86/kvm/x86.c | 1 + 9 files changed, 520 insertions(+), 8 deletions(-) diff --git a/arch/x86/include/asm/kvm-x86-ops.h b/arch/x86/include/asm/kvm-x86-ops.h index 1b0dacc6b6c0..97f0a681e02c 100644 --- a/arch/x86/include/asm/kvm-x86-ops.h +++ b/arch/x86/include/asm/kvm-x86-ops.h @@ -24,7 +24,9 @@ KVM_X86_OP(is_vm_type_supported) KVM_X86_OP_OPTIONAL(max_vcpus); KVM_X86_OP_OPTIONAL(vm_enable_cap) KVM_X86_OP(vm_init) +KVM_X86_OP_OPTIONAL(flush_shadow_all_private) KVM_X86_OP_OPTIONAL(vm_destroy) +KVM_X86_OP_OPTIONAL(vm_free) KVM_X86_OP_OPTIONAL_RET0(vcpu_precreate) KVM_X86_OP(vcpu_create) KVM_X86_OP(vcpu_free) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index cf8eb46b3a20..6dedb5cb71ef 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1608,7 +1608,9 @@ struct kvm_x86_ops { unsigned int vm_size; int (*vm_enable_cap)(struct kvm *kvm, struct kvm_enable_cap *cap); int (*vm_init)(struct kvm *kvm); + void (*flush_shadow_all_private)(struct kvm *kvm); void (*vm_destroy)(struct kvm *kvm); + void (*vm_free)(struct kvm *kvm); /* Create, but do not attach this VCPU */ int (*vcpu_precreate)(struct kvm *kvm); diff --git a/arch/x86/kvm/Kconfig b/arch/x86/kvm/Kconfig index 87e3da7b0439..bc077d6f4b43 100644 --- a/arch/x86/kvm/Kconfig +++ b/arch/x86/kvm/Kconfig @@ -76,7 +76,6 @@ config KVM_WERROR config KVM_SW_PROTECTED_VM bool "Enable support for KVM software-protected VMs" - depends on EXPERT depends on KVM && X86_64 select KVM_GENERIC_PRIVATE_MEM help @@ -89,6 +88,8 @@ config KVM_SW_PROTECTED_VM config KVM_INTEL tristate "KVM for Intel (and compatible) processors support" depends on KVM && IA32_FEAT_CTL + select KVM_SW_PROTECTED_VM if INTEL_TDX_HOST + select KVM_GENERIC_MEMORY_ATTRIBUTES if INTEL_TDX_HOST help Provides support for KVM on processors equipped with Intel's VT extensions, a.k.a. Virtual Machine Extensions (VMX). diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index c45252ed2ffd..2becc86c71b2 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -6866,6 +6866,13 @@ static void kvm_mmu_zap_all(struct kvm *kvm) void kvm_arch_flush_shadow_all(struct kvm *kvm) { + /* + * kvm_mmu_zap_all() zaps both private and shared page tables. Before + * tearing down private page tables, TDX requires some TD resources to + * be destroyed (i.e. keyID must have been reclaimed, etc). Invoke + * kvm_x86_flush_shadow_all_private() for this. + */ + static_call_cond(kvm_x86_flush_shadow_all_private)(kvm); kvm_mmu_zap_all(kvm); } diff --git a/arch/x86/kvm/vmx/main.c b/arch/x86/kvm/vmx/main.c index e8a1a7533eea..437c6d5e802e 100644 --- a/arch/x86/kvm/vmx/main.c +++ b/arch/x86/kvm/vmx/main.c @@ -62,11 +62,31 @@ static int vt_vm_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap) static int vt_vm_init(struct kvm *kvm) { if (is_td(kvm)) - return -EOPNOTSUPP; /* Not ready to create guest TD yet. */ + return tdx_vm_init(kvm); return vmx_vm_init(kvm); } +static void vt_flush_shadow_all_private(struct kvm *kvm) +{ + if (is_td(kvm)) + tdx_mmu_release_hkid(kvm); +} + +static void vt_vm_destroy(struct kvm *kvm) +{ + if (is_td(kvm)) + return; + + vmx_vm_destroy(kvm); +} + +static void vt_vm_free(struct kvm *kvm) +{ + if (is_td(kvm)) + tdx_vm_free(kvm); +} + static int vt_mem_enc_ioctl(struct kvm *kvm, void __user *argp) { if (!is_td(kvm)) @@ -101,7 +121,9 @@ struct kvm_x86_ops vt_x86_ops __initdata = { .vm_size = sizeof(struct kvm_vmx), .vm_enable_cap = vt_vm_enable_cap, .vm_init = vt_vm_init, - .vm_destroy = vmx_vm_destroy, + .flush_shadow_all_private = vt_flush_shadow_all_private, + .vm_destroy = vt_vm_destroy, + .vm_free = vt_vm_free, .vcpu_precreate = vmx_vcpu_precreate, .vcpu_create = vmx_vcpu_create, diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c index ee015f3ce2c9..1cf2b15da257 100644 --- a/arch/x86/kvm/vmx/tdx.c +++ b/arch/x86/kvm/vmx/tdx.c @@ -5,10 +5,11 @@ #include "capabilities.h" #include "x86_ops.h" -#include "x86.h" #include "mmu.h" #include "tdx_arch.h" #include "tdx.h" +#include "tdx_ops.h" +#include "x86.h" #undef pr_fmt #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt @@ -22,7 +23,7 @@ /* TDX KeyID pool */ static DEFINE_IDA(tdx_guest_keyid_pool); -static int __used tdx_guest_keyid_alloc(void) +static int tdx_guest_keyid_alloc(void) { if (WARN_ON_ONCE(!tdx_guest_keyid_start || !tdx_nr_guest_keyids)) return -EINVAL; @@ -32,7 +33,7 @@ static int __used tdx_guest_keyid_alloc(void) GFP_KERNEL); } -static void __used tdx_guest_keyid_free(int keyid) +static void tdx_guest_keyid_free(int keyid) { if (WARN_ON_ONCE(keyid < tdx_guest_keyid_start || keyid > tdx_guest_keyid_start + tdx_nr_guest_keyids - 1)) @@ -48,6 +49,8 @@ struct tdx_info { u64 xfam_fixed0; u64 xfam_fixed1; + u8 nr_tdcs_pages; + u16 num_cpuid_config; /* This must the last member. */ DECLARE_FLEX_ARRAY(struct kvm_tdx_cpuid_config, cpuid_configs); @@ -85,6 +88,282 @@ int tdx_vm_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap) return r; } +/* + * Some TDX SEAMCALLs (TDH.MNG.CREATE, TDH.PHYMEM.CACHE.WB, + * TDH.MNG.KEY.RECLAIMID, TDH.MNG.KEY.FREEID etc) tries to acquire a global lock + * internally in TDX module. If failed, TDX_OPERAND_BUSY is returned without + * spinning or waiting due to a constraint on execution time. It's caller's + * responsibility to avoid race (or retry on TDX_OPERAND_BUSY). Use this mutex + * to avoid race in TDX module because the kernel knows better about scheduling. + */ +static DEFINE_MUTEX(tdx_lock); +static struct mutex *tdx_mng_key_config_lock; + +static __always_inline hpa_t set_hkid_to_hpa(hpa_t pa, u16 hkid) +{ + return pa | ((hpa_t)hkid << boot_cpu_data.x86_phys_bits); +} + +static inline bool is_td_created(struct kvm_tdx *kvm_tdx) +{ + return kvm_tdx->tdr_pa; +} + +static inline void tdx_hkid_free(struct kvm_tdx *kvm_tdx) +{ + tdx_guest_keyid_free(kvm_tdx->hkid); + kvm_tdx->hkid = -1; +} + +static inline bool is_hkid_assigned(struct kvm_tdx *kvm_tdx) +{ + return kvm_tdx->hkid > 0; +} + +static void tdx_clear_page(unsigned long page_pa) +{ + const void *zero_page = (const void *) __va(page_to_phys(ZERO_PAGE(0))); + void *page = __va(page_pa); + unsigned long i; + + /* + * When re-assign one page from old keyid to a new keyid, MOVDIR64B is + * required to clear/write the page with new keyid to prevent integrity + * error when read on the page with new keyid. + * + * clflush doesn't flush cache with HKID set. The cache line could be + * poisoned (even without MKTME-i), clear the poison bit. + */ + for (i = 0; i < PAGE_SIZE; i += 64) + movdir64b(page + i, zero_page); + /* + * MOVDIR64B store uses WC buffer. Prevent following memory reads + * from seeing potentially poisoned cache. + */ + __mb(); +} + +static int __tdx_reclaim_page(hpa_t pa) +{ + struct tdx_module_args out; + u64 err; + + do { + err = tdh_phymem_page_reclaim(pa, &out); + /* + * TDH.PHYMEM.PAGE.RECLAIM is allowed only when TD is shutdown. + * state. i.e. destructing TD. + * TDH.PHYMEM.PAGE.RECLAIM requires TDR and target page. + * Because we're destructing TD, it's rare to contend with TDR. + */ + } while (unlikely(err == (TDX_OPERAND_BUSY | TDX_OPERAND_ID_RCX) || + err == (TDX_OPERAND_BUSY | TDX_OPERAND_ID_TDR))); + if (WARN_ON_ONCE(err)) { + pr_tdx_error(TDH_PHYMEM_PAGE_RECLAIM, err, &out); + return -EIO; + } + + return 0; +} + +static int tdx_reclaim_page(hpa_t pa) +{ + int r; + + r = __tdx_reclaim_page(pa); + if (!r) + tdx_clear_page(pa); + return r; +} + +static void tdx_reclaim_control_page(unsigned long td_page_pa) +{ + WARN_ON_ONCE(!td_page_pa); + + /* + * TDCX are being reclaimed. TDX module maps TDCX with HKID + * assigned to the TD. Here the cache associated to the TD + * was already flushed by TDH.PHYMEM.CACHE.WB before here, So + * cache doesn't need to be flushed again. + */ + if (tdx_reclaim_page(td_page_pa)) + /* + * Leak the page on failure: + * tdx_reclaim_page() returns an error if and only if there's an + * unexpected, fatal error, e.g. a SEAMCALL with bad params, + * incorrect concurrency in KVM, a TDX Module bug, etc. + * Retrying at a later point is highly unlikely to be + * successful. + * No log here as tdx_reclaim_page() already did. + */ + return; + free_page((unsigned long)__va(td_page_pa)); +} + +static void tdx_do_tdh_phymem_cache_wb(void *unused) +{ + u64 err = 0; + + do { + err = tdh_phymem_cache_wb(!!err); + } while (err == TDX_INTERRUPTED_RESUMABLE); + + /* Other thread may have done for us. */ + if (err == TDX_NO_HKID_READY_TO_WBCACHE) + err = TDX_SUCCESS; + if (WARN_ON_ONCE(err)) + pr_tdx_error(TDH_PHYMEM_CACHE_WB, err, NULL); +} + +void tdx_mmu_release_hkid(struct kvm *kvm) +{ + bool packages_allocated, targets_allocated; + struct kvm_tdx *kvm_tdx = to_kvm_tdx(kvm); + cpumask_var_t packages, targets; + u64 err; + int i; + + if (!is_hkid_assigned(kvm_tdx)) + return; + + if (!is_td_created(kvm_tdx)) { + tdx_hkid_free(kvm_tdx); + return; + } + + packages_allocated = zalloc_cpumask_var(&packages, GFP_KERNEL); + targets_allocated = zalloc_cpumask_var(&targets, GFP_KERNEL); + cpus_read_lock(); + + /* + * We can destroy multiple guest TDs simultaneously. Prevent + * tdh_phymem_cache_wb from returning TDX_BUSY by serialization. + */ + mutex_lock(&tdx_lock); + + /* + * Go through multiple TDX HKID state transitions with three SEAMCALLs + * to make TDH.PHYMEM.PAGE.RECLAIM() usable. Make the transition atomic + * to other functions to operate private pages and Secure-EPT pages. + * + * Avoid race for kvm_gmem_release() to call kvm_mmu_unmap_gfn_range(). + * This function is called via mmu notifier, mmu_release(). + * kvm_gmem_release() is called via fput() on process exit. + */ + write_lock(&kvm->mmu_lock); + + for_each_online_cpu(i) { + if (packages_allocated && + cpumask_test_and_set_cpu(topology_physical_package_id(i), + packages)) + continue; + if (targets_allocated) + cpumask_set_cpu(i, targets); + } + if (targets_allocated) + on_each_cpu_mask(targets, tdx_do_tdh_phymem_cache_wb, NULL, true); + else + on_each_cpu(tdx_do_tdh_phymem_cache_wb, NULL, true); + /* + * In the case of error in tdx_do_tdh_phymem_cache_wb(), the following + * tdh_mng_key_freeid() will fail. + */ + err = tdh_mng_key_freeid(kvm_tdx->tdr_pa); + if (WARN_ON_ONCE(err)) { + pr_tdx_error(TDH_MNG_KEY_FREEID, err, NULL); + pr_err("tdh_mng_key_freeid() failed. HKID %d is leaked.\n", + kvm_tdx->hkid); + } else + tdx_hkid_free(kvm_tdx); + + write_unlock(&kvm->mmu_lock); + mutex_unlock(&tdx_lock); + cpus_read_unlock(); + free_cpumask_var(targets); + free_cpumask_var(packages); +} + +void tdx_vm_free(struct kvm *kvm) +{ + struct kvm_tdx *kvm_tdx = to_kvm_tdx(kvm); + u64 err; + int i; + + /* + * tdx_mmu_release_hkid() failed to reclaim HKID. Something went wrong + * heavily with TDX module. Give up freeing TD pages. As the function + * already warned, don't warn it again. + */ + if (is_hkid_assigned(kvm_tdx)) + return; + + if (kvm_tdx->tdcs_pa) { + for (i = 0; i < tdx_info->nr_tdcs_pages; i++) { + if (kvm_tdx->tdcs_pa[i]) + tdx_reclaim_control_page(kvm_tdx->tdcs_pa[i]); + } + kfree(kvm_tdx->tdcs_pa); + kvm_tdx->tdcs_pa = NULL; + } + + if (!kvm_tdx->tdr_pa) + return; + if (__tdx_reclaim_page(kvm_tdx->tdr_pa)) + return; + /* + * TDX module maps TDR with TDX global HKID. TDX module may access TDR + * while operating on TD (Especially reclaiming TDCS). Cache flush with + * TDX global HKID is needed. + */ + err = tdh_phymem_page_wbinvd(set_hkid_to_hpa(kvm_tdx->tdr_pa, + tdx_global_keyid)); + if (WARN_ON_ONCE(err)) { + pr_tdx_error(TDH_PHYMEM_PAGE_WBINVD, err, NULL); + return; + } + tdx_clear_page(kvm_tdx->tdr_pa); + + free_page((unsigned long)__va(kvm_tdx->tdr_pa)); + kvm_tdx->tdr_pa = 0; +} + +static int tdx_do_tdh_mng_key_config(void *param) +{ + hpa_t *tdr_p = param; + u64 err; + + do { + err = tdh_mng_key_config(*tdr_p); + + /* + * If it failed to generate a random key, retry it because this + * is typically caused by an entropy error of the CPU's random + * number generator. + */ + } while (err == TDX_KEY_GENERATION_FAILED); + + if (WARN_ON_ONCE(err)) { + pr_tdx_error(TDH_MNG_KEY_CONFIG, err, NULL); + return -EIO; + } + + return 0; +} + +static int __tdx_td_init(struct kvm *kvm); + +int tdx_vm_init(struct kvm *kvm) +{ + /* + * TDX has its own limit of the number of vcpus in addition to + * KVM_MAX_VCPUS. + */ + kvm->max_vcpus = min(kvm->max_vcpus, TDX_MAX_VCPUS); + + /* Place holder for TDX specific logic. */ + return __tdx_td_init(kvm); +} + static int tdx_get_capabilities(struct kvm_tdx_cmd *cmd) { struct kvm_tdx_capabilities __user *user_caps; @@ -137,6 +416,176 @@ static int tdx_get_capabilities(struct kvm_tdx_cmd *cmd) return ret; } +static int __tdx_td_init(struct kvm *kvm) +{ + struct kvm_tdx *kvm_tdx = to_kvm_tdx(kvm); + cpumask_var_t packages; + unsigned long *tdcs_pa = NULL; + unsigned long tdr_pa = 0; + unsigned long va; + int ret, i; + u64 err; + + ret = tdx_guest_keyid_alloc(); + if (ret < 0) + return ret; + kvm_tdx->hkid = ret; + + va = __get_free_page(GFP_KERNEL_ACCOUNT); + if (!va) + goto free_hkid; + tdr_pa = __pa(va); + + tdcs_pa = kcalloc(tdx_info->nr_tdcs_pages, sizeof(*kvm_tdx->tdcs_pa), + GFP_KERNEL_ACCOUNT | __GFP_ZERO); + if (!tdcs_pa) + goto free_tdr; + for (i = 0; i < tdx_info->nr_tdcs_pages; i++) { + va = __get_free_page(GFP_KERNEL_ACCOUNT); + if (!va) + goto free_tdcs; + tdcs_pa[i] = __pa(va); + } + + if (!zalloc_cpumask_var(&packages, GFP_KERNEL)) { + ret = -ENOMEM; + goto free_tdcs; + } + cpus_read_lock(); + /* + * Need at least one CPU of the package to be online in order to + * program all packages for host key id. Check it. + */ + for_each_present_cpu(i) + cpumask_set_cpu(topology_physical_package_id(i), packages); + for_each_online_cpu(i) + cpumask_clear_cpu(topology_physical_package_id(i), packages); + if (!cpumask_empty(packages)) { + ret = -EIO; + /* + * Because it's hard for human operator to figure out the + * reason, warn it. + */ +#define MSG_ALLPKG "All packages need to have online CPU to create TD. Online CPU and retry.\n" + pr_warn_ratelimited(MSG_ALLPKG); + goto free_packages; + } + + /* + * Acquire global lock to avoid TDX_OPERAND_BUSY: + * TDH.MNG.CREATE and other APIs try to lock the global Key Owner + * Table (KOT) to track the assigned TDX private HKID. It doesn't spin + * to acquire the lock, returns TDX_OPERAND_BUSY instead, and let the + * caller to handle the contention. This is because of time limitation + * usable inside the TDX module and OS/VMM knows better about process + * scheduling. + * + * APIs to acquire the lock of KOT: + * TDH.MNG.CREATE, TDH.MNG.KEY.FREEID, TDH.MNG.VPFLUSHDONE, and + * TDH.PHYMEM.CACHE.WB. + */ + mutex_lock(&tdx_lock); + err = tdh_mng_create(tdr_pa, kvm_tdx->hkid); + mutex_unlock(&tdx_lock); + if (err == TDX_RND_NO_ENTROPY) { + ret = -EAGAIN; + goto free_packages; + } + if (WARN_ON_ONCE(err)) { + pr_tdx_error(TDH_MNG_CREATE, err, NULL); + ret = -EIO; + goto free_packages; + } + kvm_tdx->tdr_pa = tdr_pa; + + for_each_online_cpu(i) { + int pkg = topology_physical_package_id(i); + + if (cpumask_test_and_set_cpu(pkg, packages)) + continue; + + /* + * Program the memory controller in the package with an + * encryption key associated to a TDX private host key id + * assigned to this TDR. Concurrent operations on same memory + * controller results in TDX_OPERAND_BUSY. Avoid this race by + * mutex. + */ + mutex_lock(&tdx_mng_key_config_lock[pkg]); + ret = smp_call_on_cpu(i, tdx_do_tdh_mng_key_config, + &kvm_tdx->tdr_pa, true); + mutex_unlock(&tdx_mng_key_config_lock[pkg]); + if (ret) + break; + } + cpus_read_unlock(); + free_cpumask_var(packages); + if (ret) { + i = 0; + goto teardown; + } + + kvm_tdx->tdcs_pa = tdcs_pa; + for (i = 0; i < tdx_info->nr_tdcs_pages; i++) { + err = tdh_mng_addcx(kvm_tdx->tdr_pa, tdcs_pa[i]); + if (err == TDX_RND_NO_ENTROPY) { + /* Here it's hard to allow userspace to retry. */ + ret = -EBUSY; + goto teardown; + } + if (WARN_ON_ONCE(err)) { + pr_tdx_error(TDH_MNG_ADDCX, err, NULL); + ret = -EIO; + goto teardown; + } + } + + /* + * Note, TDH_MNG_INIT cannot be invoked here. TDH_MNG_INIT requires a dedicated + * ioctl() to define the configure CPUID values for the TD. + */ + return 0; + + /* + * The sequence for freeing resources from a partially initialized TD + * varies based on where in the initialization flow failure occurred. + * Simply use the full teardown and destroy, which naturally play nice + * with partial initialization. + */ +teardown: + for (; i < tdx_info->nr_tdcs_pages; i++) { + if (tdcs_pa[i]) { + free_page((unsigned long)__va(tdcs_pa[i])); + tdcs_pa[i] = 0; + } + } + if (!kvm_tdx->tdcs_pa) + kfree(tdcs_pa); + tdx_mmu_release_hkid(kvm); + tdx_vm_free(kvm); + return ret; + +free_packages: + cpus_read_unlock(); + free_cpumask_var(packages); +free_tdcs: + for (i = 0; i < tdx_info->nr_tdcs_pages; i++) { + if (tdcs_pa[i]) + free_page((unsigned long)__va(tdcs_pa[i])); + } + kfree(tdcs_pa); + kvm_tdx->tdcs_pa = NULL; + +free_tdr: + if (tdr_pa) + free_page((unsigned long)__va(tdr_pa)); + kvm_tdx->tdr_pa = 0; +free_hkid: + if (is_hkid_assigned(kvm_tdx)) + tdx_hkid_free(kvm_tdx); + return ret; +} + int tdx_vm_ioctl(struct kvm *kvm, void __user *argp) { struct kvm_tdx_cmd tdx_cmd; @@ -215,12 +664,13 @@ static int tdx_md_read(struct tdx_md_map *maps, int nr_maps) static int __init tdx_module_setup(void) { - u16 num_cpuid_config; + u16 num_cpuid_config, tdcs_base_size; int ret; u32 i; struct tdx_md_map mds[] = { TDX_MD_MAP(NUM_CPUID_CONFIG, &num_cpuid_config), + TDX_MD_MAP(TDCS_BASE_SIZE, &tdcs_base_size), }; struct tdx_metadata_field_mapping fields[] = { @@ -273,6 +723,8 @@ static int __init tdx_module_setup(void) c->edx = ecx_edx >> 32; } + tdx_info->nr_tdcs_pages = tdcs_base_size / PAGE_SIZE; + return 0; error_out: @@ -319,13 +771,27 @@ int __init tdx_hardware_setup(struct kvm_x86_ops *x86_ops) struct tdx_enabled enable = { .err = ATOMIC_INIT(0), }; + int max_pkgs; int r = 0; + int i; + if (!cpu_feature_enabled(X86_FEATURE_MOVDIR64B)) { + pr_warn("MOVDIR64B is reqiured for TDX\n"); + return -EOPNOTSUPP; + } if (!enable_ept) { pr_warn("Cannot enable TDX with EPT disabled\n"); return -EINVAL; } + max_pkgs = topology_max_packages(); + tdx_mng_key_config_lock = kcalloc(max_pkgs, sizeof(*tdx_mng_key_config_lock), + GFP_KERNEL); + if (!tdx_mng_key_config_lock) + return -ENOMEM; + for (i = 0; i < max_pkgs; i++) + mutex_init(&tdx_mng_key_config_lock[i]); + if (!zalloc_cpumask_var(&enable.enabled, GFP_KERNEL)) { r = -ENOMEM; goto out; @@ -350,4 +816,5 @@ int __init tdx_hardware_setup(struct kvm_x86_ops *x86_ops) void tdx_hardware_unsetup(void) { kfree(tdx_info); + kfree(tdx_mng_key_config_lock); } diff --git a/arch/x86/kvm/vmx/tdx.h b/arch/x86/kvm/vmx/tdx.h index 22c0b57f69ca..ae117f864cfb 100644 --- a/arch/x86/kvm/vmx/tdx.h +++ b/arch/x86/kvm/vmx/tdx.h @@ -8,7 +8,11 @@ struct kvm_tdx { struct kvm kvm; - /* TDX specific members follow. */ + + unsigned long tdr_pa; + unsigned long *tdcs_pa; + + int hkid; }; struct vcpu_tdx { diff --git a/arch/x86/kvm/vmx/x86_ops.h b/arch/x86/kvm/vmx/x86_ops.h index 0031a8d61589..7f123ffe4d42 100644 --- a/arch/x86/kvm/vmx/x86_ops.h +++ b/arch/x86/kvm/vmx/x86_ops.h @@ -140,6 +140,9 @@ void tdx_hardware_unsetup(void); bool tdx_is_vm_type_supported(unsigned long type); int tdx_vm_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap); +int tdx_vm_init(struct kvm *kvm); +void tdx_mmu_release_hkid(struct kvm *kvm); +void tdx_vm_free(struct kvm *kvm); int tdx_vm_ioctl(struct kvm *kvm, void __user *argp); #else static inline int tdx_hardware_setup(struct kvm_x86_ops *x86_ops) { return -EOPNOTSUPP; } @@ -150,6 +153,9 @@ static inline int tdx_vm_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap) { return -EINVAL; }; +static inline int tdx_vm_init(struct kvm *kvm) { return -EOPNOTSUPP; } +static inline void tdx_mmu_release_hkid(struct kvm *kvm) {} +static inline void tdx_vm_free(struct kvm *kvm) {} static inline int tdx_vm_ioctl(struct kvm *kvm, void __user *argp) { return -EOPNOTSUPP; } #endif diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 3f16e9450d2f..1f01c7f91652 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -12743,6 +12743,7 @@ void kvm_arch_destroy_vm(struct kvm *kvm) kvm_page_track_cleanup(kvm); kvm_xen_destroy_vm(kvm); kvm_hv_destroy_vm(kvm); + static_call_cond(kvm_x86_vm_free)(kvm); } static void memslot_rmap_free(struct kvm_memory_slot *slot) From patchwork Mon Feb 26 08:25:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571467 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 25EB45B68B; Mon, 26 Feb 2024 08:28:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.13 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936087; cv=none; b=mTl2Rt9fnN8R95H++g1hQvlkpjCnIOSdQuWF40cBekol8GUFIFA+qCtNIzIoJDpm+I9v/qWs/HLocHmSwV2rkb3iimoWNd/t5RiRnvGFuEu8LVidVqYpSpZQgrIedN1B2kfv/ggFZt4h6gfIQnbL24kyjdyAuSfepGerDTvPzN4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936087; c=relaxed/simple; bh=OJUmd0bYn1D5xie7Hcmu/a5DYtUvD3qiMfpVf7+fzsg=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=MvJTeGzNhBZoV3DsgXs01QTOtF2g26obc2KABTgOoJlA7t5xIH9MsLkIiXPXeJyF8af+tZcUrgeY8q3l9VRJl4Vv1HmF0ckxNIZ9z4aZ84lDgEPjxWfNhZ/UyCSdlohGcbDQeW4XME25p9TeSwqtuQYKiLS+A7XedIkCZnXxw/Q= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=Fo3Q2BMr; arc=none smtp.client-ip=192.198.163.13 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="Fo3Q2BMr" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936085; x=1740472085; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=OJUmd0bYn1D5xie7Hcmu/a5DYtUvD3qiMfpVf7+fzsg=; b=Fo3Q2BMrOTHT78peuEfIQN5HYtx+gc2Zu1+Ky2b/NUmJlxbomAvLDv18 yDKek9hGBgLVcqpXp2rJEkaeyRTxsoBZfQ8o1CTgxI7S1oVYVolQFmDAj DKggBW+jo8E+Suio2fG9sSZrC8sDNoBoq6koRf6I2/Z9BLKx+T2dr1j2H 5nQDtNwE4nZXeqVf2ZaBJ3WUjy2Jrp3r2Xj3F3QOKAJtlj3fxXmNongoj fu6XUzAG9iSeVBM/rj93avX5+Qd53MMyRjUBJVs+43hziZPgHKcbDh3bI ZXBgy3J8NiSlhAomMQV3/e+ql5zXThz89GCjL8S7WhUGyAEGnO+ILNx+L g==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="6155322" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6155322" Received: from orviesa009.jf.intel.com ([10.64.159.149]) by fmvoesa107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:03 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6615685" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa009-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:03 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com, Xiaoyao Li Subject: [PATCH v19 039/130] KVM: TDX: initialize VM with TDX specific parameters Date: Mon, 26 Feb 2024 00:25:41 -0800 Message-Id: <5eca97e6a3978cf4dcf1cff21be6ec8b639a66b9.1708933498.git.isaku.yamahata@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata TDX requires additional parameters for TDX VM for confidential execution to protect the confidentiality of its memory contents and CPU state from any other software, including VMM. When creating a guest TD VM before creating vcpu, the number of vcpu, TSC frequency (the values are the same among vcpus, and it can't change.) CPUIDs which the TDX module emulates. Guest TDs can trust those CPUIDs and sha384 values for measurement. Add a new subcommand, KVM_TDX_INIT_VM, to pass parameters for the TDX guest. It assigns an encryption key to the TDX guest for memory encryption. TDX encrypts memory per guest basis. The device model, say qemu, passes per-VM parameters for the TDX guest. The maximum number of vcpus, TSC frequency (TDX guest has fixed VM-wide TSC frequency, not per vcpu. The TDX guest can not change it.), attributes (production or debug), available extended features (which configure guest XCR0, IA32_XSS MSR), CPUIDs, sha384 measurements, etc. Call this subcommand before creating vcpu and KVM_SET_CPUID2, i.e. CPUID configurations aren't available yet. So CPUIDs configuration values need to be passed in struct kvm_tdx_init_vm. The device model's responsibility to make this CPUID config for KVM_TDX_INIT_VM and KVM_SET_CPUID2. Signed-off-by: Xiaoyao Li Signed-off-by: Isaku Yamahata --- v19: - Check NO_RBP_MOD of feature0 and set it - Update the comment for PT and CET v18: - remove the change of tools/arch/x86/include/uapi/asm/kvm.h - typo in comment. sha348 => sha384 - updated comment in setup_tdparams_xfam() - fix setup_tdparams_xfam() to use init_vm instead of td_params v15 -> v16: - Removed AMX check as the KVM upstream supports AMX. - Added CET flag to guest supported xss v14 -> v15: - add check if the reserved area of init_vm is zero Signed-off-by: Isaku Yamahata --- arch/x86/include/uapi/asm/kvm.h | 27 ++++ arch/x86/kvm/cpuid.c | 7 + arch/x86/kvm/cpuid.h | 2 + arch/x86/kvm/vmx/tdx.c | 273 ++++++++++++++++++++++++++++++-- arch/x86/kvm/vmx/tdx.h | 18 +++ arch/x86/kvm/vmx/tdx_arch.h | 6 + 6 files changed, 323 insertions(+), 10 deletions(-) diff --git a/arch/x86/include/uapi/asm/kvm.h b/arch/x86/include/uapi/asm/kvm.h index e28189c81691..9ac0246bd974 100644 --- a/arch/x86/include/uapi/asm/kvm.h +++ b/arch/x86/include/uapi/asm/kvm.h @@ -570,6 +570,7 @@ struct kvm_pmu_event_filter { /* Trust Domain eXtension sub-ioctl() commands. */ enum kvm_tdx_cmd_id { KVM_TDX_CAPABILITIES = 0, + KVM_TDX_INIT_VM, KVM_TDX_CMD_NR_MAX, }; @@ -621,4 +622,30 @@ struct kvm_tdx_capabilities { struct kvm_tdx_cpuid_config cpuid_configs[]; }; +struct kvm_tdx_init_vm { + __u64 attributes; + __u64 mrconfigid[6]; /* sha384 digest */ + __u64 mrowner[6]; /* sha384 digest */ + __u64 mrownerconfig[6]; /* sha384 digest */ + /* + * For future extensibility to make sizeof(struct kvm_tdx_init_vm) = 8KB. + * This should be enough given sizeof(TD_PARAMS) = 1024. + * 8KB was chosen given because + * sizeof(struct kvm_cpuid_entry2) * KVM_MAX_CPUID_ENTRIES(=256) = 8KB. + */ + __u64 reserved[1004]; + + /* + * Call KVM_TDX_INIT_VM before vcpu creation, thus before + * KVM_SET_CPUID2. + * This configuration supersedes KVM_SET_CPUID2s for VCPUs because the + * TDX module directly virtualizes those CPUIDs without VMM. The user + * space VMM, e.g. qemu, should make KVM_SET_CPUID2 consistent with + * those values. If it doesn't, KVM may have wrong idea of vCPUIDs of + * the guest, and KVM may wrongly emulate CPUIDs or MSRs that the TDX + * module doesn't virtualize. + */ + struct kvm_cpuid2 cpuid; +}; + #endif /* _ASM_X86_KVM_H */ diff --git a/arch/x86/kvm/cpuid.c b/arch/x86/kvm/cpuid.c index adba49afb5fe..8cdcd6f406aa 100644 --- a/arch/x86/kvm/cpuid.c +++ b/arch/x86/kvm/cpuid.c @@ -1443,6 +1443,13 @@ int kvm_dev_ioctl_get_cpuid(struct kvm_cpuid2 *cpuid, return r; } +struct kvm_cpuid_entry2 *kvm_find_cpuid_entry2( + struct kvm_cpuid_entry2 *entries, int nent, u32 function, u64 index) +{ + return cpuid_entry2_find(entries, nent, function, index); +} +EXPORT_SYMBOL_GPL(kvm_find_cpuid_entry2); + struct kvm_cpuid_entry2 *kvm_find_cpuid_entry_index(struct kvm_vcpu *vcpu, u32 function, u32 index) { diff --git a/arch/x86/kvm/cpuid.h b/arch/x86/kvm/cpuid.h index 856e3037e74f..215d1c68c6d1 100644 --- a/arch/x86/kvm/cpuid.h +++ b/arch/x86/kvm/cpuid.h @@ -13,6 +13,8 @@ void kvm_set_cpu_caps(void); void kvm_update_cpuid_runtime(struct kvm_vcpu *vcpu); void kvm_update_pv_runtime(struct kvm_vcpu *vcpu); +struct kvm_cpuid_entry2 *kvm_find_cpuid_entry2(struct kvm_cpuid_entry2 *entries, + int nent, u32 function, u64 index); struct kvm_cpuid_entry2 *kvm_find_cpuid_entry_index(struct kvm_vcpu *vcpu, u32 function, u32 index); struct kvm_cpuid_entry2 *kvm_find_cpuid_entry(struct kvm_vcpu *vcpu, diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c index 1cf2b15da257..b11f105db3cd 100644 --- a/arch/x86/kvm/vmx/tdx.c +++ b/arch/x86/kvm/vmx/tdx.c @@ -8,7 +8,6 @@ #include "mmu.h" #include "tdx_arch.h" #include "tdx.h" -#include "tdx_ops.h" #include "x86.h" #undef pr_fmt @@ -350,18 +349,21 @@ static int tdx_do_tdh_mng_key_config(void *param) return 0; } -static int __tdx_td_init(struct kvm *kvm); - int tdx_vm_init(struct kvm *kvm) { + /* + * This function initializes only KVM software construct. It doesn't + * initialize TDX stuff, e.g. TDCS, TDR, TDCX, HKID etc. + * It is handled by KVM_TDX_INIT_VM, __tdx_td_init(). + */ + /* * TDX has its own limit of the number of vcpus in addition to * KVM_MAX_VCPUS. */ kvm->max_vcpus = min(kvm->max_vcpus, TDX_MAX_VCPUS); - /* Place holder for TDX specific logic. */ - return __tdx_td_init(kvm); + return 0; } static int tdx_get_capabilities(struct kvm_tdx_cmd *cmd) @@ -416,9 +418,162 @@ static int tdx_get_capabilities(struct kvm_tdx_cmd *cmd) return ret; } -static int __tdx_td_init(struct kvm *kvm) +static int setup_tdparams_eptp_controls(struct kvm_cpuid2 *cpuid, + struct td_params *td_params) +{ + const struct kvm_cpuid_entry2 *entry; + int max_pa = 36; + + entry = kvm_find_cpuid_entry2(cpuid->entries, cpuid->nent, 0x80000008, 0); + if (entry) + max_pa = entry->eax & 0xff; + + td_params->eptp_controls = VMX_EPTP_MT_WB; + /* + * No CPU supports 4-level && max_pa > 48. + * "5-level paging and 5-level EPT" section 4.1 4-level EPT + * "4-level EPT is limited to translating 48-bit guest-physical + * addresses." + * cpu_has_vmx_ept_5levels() check is just in case. + */ + if (!cpu_has_vmx_ept_5levels() && max_pa > 48) + return -EINVAL; + if (cpu_has_vmx_ept_5levels() && max_pa > 48) { + td_params->eptp_controls |= VMX_EPTP_PWL_5; + td_params->exec_controls |= TDX_EXEC_CONTROL_MAX_GPAW; + } else { + td_params->eptp_controls |= VMX_EPTP_PWL_4; + } + + return 0; +} + +static void setup_tdparams_cpuids(struct kvm_cpuid2 *cpuid, + struct td_params *td_params) +{ + int i; + + /* + * td_params.cpuid_values: The number and the order of cpuid_value must + * be same to the one of struct tdsysinfo.{num_cpuid_config, cpuid_configs} + * It's assumed that td_params was zeroed. + */ + for (i = 0; i < tdx_info->num_cpuid_config; i++) { + const struct kvm_tdx_cpuid_config *c = &tdx_info->cpuid_configs[i]; + /* KVM_TDX_CPUID_NO_SUBLEAF means index = 0. */ + u32 index = c->sub_leaf == KVM_TDX_CPUID_NO_SUBLEAF ? 0 : c->sub_leaf; + const struct kvm_cpuid_entry2 *entry = + kvm_find_cpuid_entry2(cpuid->entries, cpuid->nent, + c->leaf, index); + struct tdx_cpuid_value *value = &td_params->cpuid_values[i]; + + if (!entry) + continue; + + /* + * tdsysinfo.cpuid_configs[].{eax, ebx, ecx, edx} + * bit 1 means it can be configured to zero or one. + * bit 0 means it must be zero. + * Mask out non-configurable bits. + */ + value->eax = entry->eax & c->eax; + value->ebx = entry->ebx & c->ebx; + value->ecx = entry->ecx & c->ecx; + value->edx = entry->edx & c->edx; + } +} + +static int setup_tdparams_xfam(struct kvm_cpuid2 *cpuid, struct td_params *td_params) +{ + const struct kvm_cpuid_entry2 *entry; + u64 guest_supported_xcr0; + u64 guest_supported_xss; + + /* Setup td_params.xfam */ + entry = kvm_find_cpuid_entry2(cpuid->entries, cpuid->nent, 0xd, 0); + if (entry) + guest_supported_xcr0 = (entry->eax | ((u64)entry->edx << 32)); + else + guest_supported_xcr0 = 0; + guest_supported_xcr0 &= kvm_caps.supported_xcr0; + + entry = kvm_find_cpuid_entry2(cpuid->entries, cpuid->nent, 0xd, 1); + if (entry) + guest_supported_xss = (entry->ecx | ((u64)entry->edx << 32)); + else + guest_supported_xss = 0; + + /* + * PT and CET can be exposed to TD guest regardless of KVM's XSS, PT + * and, CET support. + */ + guest_supported_xss &= + (kvm_caps.supported_xss | XFEATURE_MASK_PT | TDX_TD_XFAM_CET); + + td_params->xfam = guest_supported_xcr0 | guest_supported_xss; + if (td_params->xfam & XFEATURE_MASK_LBR) { + /* + * TODO: once KVM supports LBR(save/restore LBR related + * registers around TDENTER), remove this guard. + */ +#define MSG_LBR "TD doesn't support LBR yet. KVM needs to save/restore IA32_LBR_DEPTH properly.\n" + pr_warn(MSG_LBR); + return -EOPNOTSUPP; + } + + return 0; +} + +static int setup_tdparams(struct kvm *kvm, struct td_params *td_params, + struct kvm_tdx_init_vm *init_vm) +{ + struct kvm_cpuid2 *cpuid = &init_vm->cpuid; + int ret; + + if (kvm->created_vcpus) + return -EBUSY; + + if (init_vm->attributes & TDX_TD_ATTRIBUTE_PERFMON) { + /* + * TODO: save/restore PMU related registers around TDENTER. + * Once it's done, remove this guard. + */ +#define MSG_PERFMON "TD doesn't support perfmon yet. KVM needs to save/restore host perf registers properly.\n" + pr_warn(MSG_PERFMON); + return -EOPNOTSUPP; + } + + td_params->max_vcpus = kvm->max_vcpus; + td_params->attributes = init_vm->attributes; + td_params->exec_controls = TDX_CONTROL_FLAG_NO_RBP_MOD; + td_params->tsc_frequency = TDX_TSC_KHZ_TO_25MHZ(kvm->arch.default_tsc_khz); + + ret = setup_tdparams_eptp_controls(cpuid, td_params); + if (ret) + return ret; + setup_tdparams_cpuids(cpuid, td_params); + ret = setup_tdparams_xfam(cpuid, td_params); + if (ret) + return ret; + +#define MEMCPY_SAME_SIZE(dst, src) \ + do { \ + BUILD_BUG_ON(sizeof(dst) != sizeof(src)); \ + memcpy((dst), (src), sizeof(dst)); \ + } while (0) + + MEMCPY_SAME_SIZE(td_params->mrconfigid, init_vm->mrconfigid); + MEMCPY_SAME_SIZE(td_params->mrowner, init_vm->mrowner); + MEMCPY_SAME_SIZE(td_params->mrownerconfig, init_vm->mrownerconfig); + + return 0; +} + +static int __tdx_td_init(struct kvm *kvm, struct td_params *td_params, + u64 *seamcall_err) { struct kvm_tdx *kvm_tdx = to_kvm_tdx(kvm); + struct tdx_module_args out; cpumask_var_t packages; unsigned long *tdcs_pa = NULL; unsigned long tdr_pa = 0; @@ -426,6 +581,7 @@ static int __tdx_td_init(struct kvm *kvm) int ret, i; u64 err; + *seamcall_err = 0; ret = tdx_guest_keyid_alloc(); if (ret < 0) return ret; @@ -540,10 +696,23 @@ static int __tdx_td_init(struct kvm *kvm) } } - /* - * Note, TDH_MNG_INIT cannot be invoked here. TDH_MNG_INIT requires a dedicated - * ioctl() to define the configure CPUID values for the TD. - */ + err = tdh_mng_init(kvm_tdx->tdr_pa, __pa(td_params), &out); + if ((err & TDX_SEAMCALL_STATUS_MASK) == TDX_OPERAND_INVALID) { + /* + * Because a user gives operands, don't warn. + * Return a hint to the user because it's sometimes hard for the + * user to figure out which operand is invalid. SEAMCALL status + * code includes which operand caused invalid operand error. + */ + *seamcall_err = err; + ret = -EINVAL; + goto teardown; + } else if (WARN_ON_ONCE(err)) { + pr_tdx_error(TDH_MNG_INIT, err, &out); + ret = -EIO; + goto teardown; + } + return 0; /* @@ -586,6 +755,76 @@ static int __tdx_td_init(struct kvm *kvm) return ret; } +static int tdx_td_init(struct kvm *kvm, struct kvm_tdx_cmd *cmd) +{ + struct kvm_tdx *kvm_tdx = to_kvm_tdx(kvm); + struct kvm_tdx_init_vm *init_vm = NULL; + struct td_params *td_params = NULL; + int ret; + + BUILD_BUG_ON(sizeof(*init_vm) != 8 * 1024); + BUILD_BUG_ON(sizeof(struct td_params) != 1024); + + if (is_hkid_assigned(kvm_tdx)) + return -EINVAL; + + if (cmd->flags) + return -EINVAL; + + init_vm = kzalloc(sizeof(*init_vm) + + sizeof(init_vm->cpuid.entries[0]) * KVM_MAX_CPUID_ENTRIES, + GFP_KERNEL); + if (!init_vm) + return -ENOMEM; + if (copy_from_user(init_vm, (void __user *)cmd->data, sizeof(*init_vm))) { + ret = -EFAULT; + goto out; + } + if (init_vm->cpuid.nent > KVM_MAX_CPUID_ENTRIES) { + ret = -E2BIG; + goto out; + } + if (copy_from_user(init_vm->cpuid.entries, + (void __user *)cmd->data + sizeof(*init_vm), + flex_array_size(init_vm, cpuid.entries, init_vm->cpuid.nent))) { + ret = -EFAULT; + goto out; + } + + if (memchr_inv(init_vm->reserved, 0, sizeof(init_vm->reserved))) { + ret = -EINVAL; + goto out; + } + if (init_vm->cpuid.padding) { + ret = -EINVAL; + goto out; + } + + td_params = kzalloc(sizeof(struct td_params), GFP_KERNEL); + if (!td_params) { + ret = -ENOMEM; + goto out; + } + + ret = setup_tdparams(kvm, td_params, init_vm); + if (ret) + goto out; + + ret = __tdx_td_init(kvm, td_params, &cmd->error); + if (ret) + goto out; + + kvm_tdx->tsc_offset = td_tdcs_exec_read64(kvm_tdx, TD_TDCS_EXEC_TSC_OFFSET); + kvm_tdx->attributes = td_params->attributes; + kvm_tdx->xfam = td_params->xfam; + +out: + /* kfree() accepts NULL. */ + kfree(init_vm); + kfree(td_params); + return ret; +} + int tdx_vm_ioctl(struct kvm *kvm, void __user *argp) { struct kvm_tdx_cmd tdx_cmd; @@ -602,6 +841,9 @@ int tdx_vm_ioctl(struct kvm *kvm, void __user *argp) case KVM_TDX_CAPABILITIES: r = tdx_get_capabilities(&tdx_cmd); break; + case KVM_TDX_INIT_VM: + r = tdx_td_init(kvm, &tdx_cmd); + break; default: r = -EINVAL; goto out; @@ -725,6 +967,17 @@ static int __init tdx_module_setup(void) tdx_info->nr_tdcs_pages = tdcs_base_size / PAGE_SIZE; + /* + * Make TDH.VP.ENTER preserve RBP so that the stack unwinder + * always work around it. Query the feature. + */ + if (!(tdx_info->features0 & MD_FIELD_ID_FEATURES0_NO_RBP_MOD) && + !IS_ENABLED(CONFIG_FRAME_POINTER)) { + pr_err("Too old version of TDX module. Consider upgrade.\n"); + ret = -EOPNOTSUPP; + goto error_out; + } + return 0; error_out: diff --git a/arch/x86/kvm/vmx/tdx.h b/arch/x86/kvm/vmx/tdx.h index ae117f864cfb..184fe394da86 100644 --- a/arch/x86/kvm/vmx/tdx.h +++ b/arch/x86/kvm/vmx/tdx.h @@ -12,7 +12,11 @@ struct kvm_tdx { unsigned long tdr_pa; unsigned long *tdcs_pa; + u64 attributes; + u64 xfam; int hkid; + + u64 tsc_offset; }; struct vcpu_tdx { @@ -39,6 +43,20 @@ static inline struct vcpu_tdx *to_tdx(struct kvm_vcpu *vcpu) { return container_of(vcpu, struct vcpu_tdx, vcpu); } + +static __always_inline u64 td_tdcs_exec_read64(struct kvm_tdx *kvm_tdx, u32 field) +{ + struct tdx_module_args out; + u64 err; + + err = tdh_mng_rd(kvm_tdx->tdr_pa, TDCS_EXEC(field), &out); + if (unlikely(err)) { + pr_err("TDH_MNG_RD[EXEC.0x%x] failed: 0x%llx\n", field, err); + return 0; + } + return out.r8; +} + #else struct kvm_tdx { struct kvm kvm; diff --git a/arch/x86/kvm/vmx/tdx_arch.h b/arch/x86/kvm/vmx/tdx_arch.h index e2c1a6f429d7..efc3c61c14ab 100644 --- a/arch/x86/kvm/vmx/tdx_arch.h +++ b/arch/x86/kvm/vmx/tdx_arch.h @@ -117,6 +117,12 @@ struct tdx_cpuid_value { #define TDX_TD_ATTRIBUTE_KL BIT_ULL(31) #define TDX_TD_ATTRIBUTE_PERFMON BIT_ULL(63) +/* + * TODO: Once XFEATURE_CET_{U, S} in arch/x86/include/asm/fpu/types.h is + * defined, Replace these with define ones. + */ +#define TDX_TD_XFAM_CET (BIT(11) | BIT(12)) + /* * TD_PARAMS is provided as an input to TDH_MNG_INIT, the size of which is 1024B. */ From patchwork Mon Feb 26 08:25:42 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571468 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id ECA025C602; Mon, 26 Feb 2024 08:28:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.13 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936088; cv=none; b=mfKBO3uWnWUN0L/BtHGg74/6fKt/VXLEoKsPGhC3i4t63zhKHR8anFaOXqDsMLpl5obx/7bKB+7mA1U0IjiCNgjIt5Ma7EHnGEj3uMwphH2DQarOSxzABujZwgfuel4kd5sTlGdtojO6LUvgDJTidFKetMQyEmZBg0T4UA7EpoQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936088; c=relaxed/simple; bh=01ck+n/mgG26TqykBveQJPhCe2LvphqDpOCGfMdUFLs=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=nmsDlKZXOWuzy6FmAD7A8QtdDfuLSlnGd49TxneykFOOuj3CXHCUujKjyoLZsc2LPQ8JBxKTWVVpaOqc+0Lcw9nXunf4HRIvnQZan8ZwR/BEORuRWwdDw3DHxOGzj8Mp/51rwfXOPkZKQSaKWVCTy/nA0AMQhGZFgwslNEQGxeY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=d2Kz+Gwm; arc=none smtp.client-ip=192.198.163.13 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="d2Kz+Gwm" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936086; x=1740472086; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=01ck+n/mgG26TqykBveQJPhCe2LvphqDpOCGfMdUFLs=; b=d2Kz+Gwm41TGVsTFwjox4dGk3nClz9Jw8n/2tx/8IwPs9yqppB4viLs5 1uzviCWXELKFv4TFrKU6EJNfQZuLpqm0mRsTqzMo+TlOQWFpqjBb5jKNf 9i1Vrn4upAM8fF25gx/wHYm9yp2GiwFbhG9ljgDTVt0i4Xv+0///TqS3Y R5zvqGRD518OcQerKbJ63YjLH+ma2X6TedsnxVzvWF7JrbMxBQrqVHWWz bxFOcpNOP9ra2sDNn+r+gwZ9iHLS7/d+Tnj03lyT49p8Bc/JnoldkeJck a6rqxW7x2V9x9Wa7NmZrLwfT9kMC9akXkiN/coqJuD9Xq0rbUvOh3g6Zk g==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="6155327" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6155327" Received: from orviesa009.jf.intel.com ([10.64.159.149]) by fmvoesa107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:04 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6615699" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa009-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:04 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 040/130] KVM: TDX: Make pmu_intel.c ignore guest TD case Date: Mon, 26 Feb 2024 00:25:42 -0800 Message-Id: <2eb2e7205d23ec858e6d800fee7da2306216e8f0.1708933498.git.isaku.yamahata@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata Because TDX KVM doesn't support PMU yet (it's future work of TDX KVM support as another patch series) and pmu_intel.c touches vmx specific structure in vcpu initialization, as workaround add dummy structure to struct vcpu_tdx and pmu_intel.c can ignore TDX case. Signed-off-by: Isaku Yamahata --- v18: - Removed unnecessary change to vmx.c which caused kernel warning. --- arch/x86/kvm/vmx/pmu_intel.c | 46 +++++++++++++++++++++++++++++++++++- arch/x86/kvm/vmx/pmu_intel.h | 28 ++++++++++++++++++++++ arch/x86/kvm/vmx/tdx.h | 8 ++++++- arch/x86/kvm/vmx/vmx.h | 32 +------------------------ 4 files changed, 81 insertions(+), 33 deletions(-) create mode 100644 arch/x86/kvm/vmx/pmu_intel.h diff --git a/arch/x86/kvm/vmx/pmu_intel.c b/arch/x86/kvm/vmx/pmu_intel.c index 315c7c2ba89b..7e6e51efd879 100644 --- a/arch/x86/kvm/vmx/pmu_intel.c +++ b/arch/x86/kvm/vmx/pmu_intel.c @@ -19,6 +19,7 @@ #include "lapic.h" #include "nested.h" #include "pmu.h" +#include "tdx.h" #define MSR_PMC_FULL_WIDTH_BIT (MSR_IA32_PMC0 - MSR_IA32_PERFCTR0) @@ -68,6 +69,26 @@ static int fixed_pmc_events[] = { [2] = PSEUDO_ARCH_REFERENCE_CYCLES, }; +struct lbr_desc *vcpu_to_lbr_desc(struct kvm_vcpu *vcpu) +{ +#ifdef CONFIG_INTEL_TDX_HOST + if (is_td_vcpu(vcpu)) + return &to_tdx(vcpu)->lbr_desc; +#endif + + return &to_vmx(vcpu)->lbr_desc; +} + +struct x86_pmu_lbr *vcpu_to_lbr_records(struct kvm_vcpu *vcpu) +{ +#ifdef CONFIG_INTEL_TDX_HOST + if (is_td_vcpu(vcpu)) + return &to_tdx(vcpu)->lbr_desc.records; +#endif + + return &to_vmx(vcpu)->lbr_desc.records; +} + static void reprogram_fixed_counters(struct kvm_pmu *pmu, u64 data) { struct kvm_pmc *pmc; @@ -179,6 +200,23 @@ static inline struct kvm_pmc *get_fw_gp_pmc(struct kvm_pmu *pmu, u32 msr) return get_gp_pmc(pmu, msr, MSR_IA32_PMC0); } +bool intel_pmu_lbr_is_compatible(struct kvm_vcpu *vcpu) +{ + if (is_td_vcpu(vcpu)) + return false; + return cpuid_model_is_consistent(vcpu); +} + +bool intel_pmu_lbr_is_enabled(struct kvm_vcpu *vcpu) +{ + struct x86_pmu_lbr *lbr = vcpu_to_lbr_records(vcpu); + + if (is_td_vcpu(vcpu)) + return false; + + return lbr->nr && (vcpu_get_perf_capabilities(vcpu) & PMU_CAP_LBR_FMT); +} + static bool intel_pmu_is_valid_lbr_msr(struct kvm_vcpu *vcpu, u32 index) { struct x86_pmu_lbr *records = vcpu_to_lbr_records(vcpu); @@ -244,6 +282,9 @@ static inline void intel_pmu_release_guest_lbr_event(struct kvm_vcpu *vcpu) { struct lbr_desc *lbr_desc = vcpu_to_lbr_desc(vcpu); + if (is_td_vcpu(vcpu)) + return; + if (lbr_desc->event) { perf_event_release_kernel(lbr_desc->event); lbr_desc->event = NULL; @@ -285,6 +326,9 @@ int intel_pmu_create_guest_lbr_event(struct kvm_vcpu *vcpu) PERF_SAMPLE_BRANCH_USER, }; + if (WARN_ON_ONCE(is_td_vcpu(vcpu))) + return 0; + if (unlikely(lbr_desc->event)) { __set_bit(INTEL_PMC_IDX_FIXED_VLBR, pmu->pmc_in_use); return 0; @@ -578,7 +622,7 @@ static void intel_pmu_refresh(struct kvm_vcpu *vcpu) INTEL_PMC_MAX_GENERIC, pmu->nr_arch_fixed_counters); perf_capabilities = vcpu_get_perf_capabilities(vcpu); - if (cpuid_model_is_consistent(vcpu) && + if (intel_pmu_lbr_is_compatible(vcpu) && (perf_capabilities & PMU_CAP_LBR_FMT)) x86_perf_get_lbr(&lbr_desc->records); else diff --git a/arch/x86/kvm/vmx/pmu_intel.h b/arch/x86/kvm/vmx/pmu_intel.h new file mode 100644 index 000000000000..66bba47c1269 --- /dev/null +++ b/arch/x86/kvm/vmx/pmu_intel.h @@ -0,0 +1,28 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef __KVM_X86_VMX_PMU_INTEL_H +#define __KVM_X86_VMX_PMU_INTEL_H + +struct lbr_desc *vcpu_to_lbr_desc(struct kvm_vcpu *vcpu); +struct x86_pmu_lbr *vcpu_to_lbr_records(struct kvm_vcpu *vcpu); + +bool intel_pmu_lbr_is_compatible(struct kvm_vcpu *vcpu); +bool intel_pmu_lbr_is_enabled(struct kvm_vcpu *vcpu); +int intel_pmu_create_guest_lbr_event(struct kvm_vcpu *vcpu); + +struct lbr_desc { + /* Basic info about guest LBR records. */ + struct x86_pmu_lbr records; + + /* + * Emulate LBR feature via passthrough LBR registers when the + * per-vcpu guest LBR event is scheduled on the current pcpu. + * + * The records may be inaccurate if the host reclaims the LBR. + */ + struct perf_event *event; + + /* True if LBRs are marked as not intercepted in the MSR bitmap */ + bool msr_passthrough; +}; + +#endif /* __KVM_X86_VMX_PMU_INTEL_H */ diff --git a/arch/x86/kvm/vmx/tdx.h b/arch/x86/kvm/vmx/tdx.h index 184fe394da86..173ed19207fb 100644 --- a/arch/x86/kvm/vmx/tdx.h +++ b/arch/x86/kvm/vmx/tdx.h @@ -4,6 +4,7 @@ #ifdef CONFIG_INTEL_TDX_HOST +#include "pmu_intel.h" #include "tdx_ops.h" struct kvm_tdx { @@ -21,7 +22,12 @@ struct kvm_tdx { struct vcpu_tdx { struct kvm_vcpu vcpu; - /* TDX specific members follow. */ + + /* + * Dummy to make pmu_intel not corrupt memory. + * TODO: Support PMU for TDX. Future work. + */ + struct lbr_desc lbr_desc; }; static inline bool is_td(struct kvm *kvm) diff --git a/arch/x86/kvm/vmx/vmx.h b/arch/x86/kvm/vmx/vmx.h index e3b0985bb74a..04ed2a9eada1 100644 --- a/arch/x86/kvm/vmx/vmx.h +++ b/arch/x86/kvm/vmx/vmx.h @@ -11,6 +11,7 @@ #include "capabilities.h" #include "../kvm_cache_regs.h" #include "posted_intr.h" +#include "pmu_intel.h" #include "vmcs.h" #include "vmx_ops.h" #include "../cpuid.h" @@ -93,22 +94,6 @@ union vmx_exit_reason { u32 full; }; -struct lbr_desc { - /* Basic info about guest LBR records. */ - struct x86_pmu_lbr records; - - /* - * Emulate LBR feature via passthrough LBR registers when the - * per-vcpu guest LBR event is scheduled on the current pcpu. - * - * The records may be inaccurate if the host reclaims the LBR. - */ - struct perf_event *event; - - /* True if LBRs are marked as not intercepted in the MSR bitmap */ - bool msr_passthrough; -}; - /* * The nested_vmx structure is part of vcpu_vmx, and holds information we need * for correct emulation of VMX (i.e., nested VMX) on this vcpu. @@ -659,21 +644,6 @@ static __always_inline struct vcpu_vmx *to_vmx(struct kvm_vcpu *vcpu) return container_of(vcpu, struct vcpu_vmx, vcpu); } -static inline struct lbr_desc *vcpu_to_lbr_desc(struct kvm_vcpu *vcpu) -{ - return &to_vmx(vcpu)->lbr_desc; -} - -static inline struct x86_pmu_lbr *vcpu_to_lbr_records(struct kvm_vcpu *vcpu) -{ - return &vcpu_to_lbr_desc(vcpu)->records; -} - -static inline bool intel_pmu_lbr_is_enabled(struct kvm_vcpu *vcpu) -{ - return !!vcpu_to_lbr_records(vcpu)->nr; -} - void intel_pmu_cross_mapped_check(struct kvm_pmu *pmu); int intel_pmu_create_guest_lbr_event(struct kvm_vcpu *vcpu); void vmx_passthrough_lbr_msrs(struct kvm_vcpu *vcpu); From patchwork Mon Feb 26 08:25:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571470 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1C1BE5D8EF; Mon, 26 Feb 2024 08:28:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.13 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936090; cv=none; b=Zt6jYr/yNu+WdlVPhAbP1UCiWA37UMjxe5F05EhGmaxFrKoVMceygYIv6dUaAnIrHvX8tai0uY9b9EHeCEiCBYFczBiG5AcuQBGttmS7cLUmScHzZH4ZN0fSI6XAPf1ISK0Ep0nlN6HrQShEOHr3SSCy6mYa9PzOv7J6jLEovxo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936090; c=relaxed/simple; bh=B6Szq744QTs7ccXNfQ/JakGeoI4rVafNlokaNEo2vm0=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=ZTasb/bcnA7XmPbP2e95DOk4Jq8os0kCNvi1etHVxA//UOrHgDxAjF+7oahVILfllnHVtsEg3OCdsdwVF6ifm/2AmpokpVDt/dnZl1ztuyARe+IY6pPSCaAEaFc+GrOG39Q7h4b8XuKKRa58rlDqNtxRBq3PDcYFoX8nNdl4aXk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=nTtIUIC7; arc=none smtp.client-ip=192.198.163.13 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="nTtIUIC7" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936088; x=1740472088; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=B6Szq744QTs7ccXNfQ/JakGeoI4rVafNlokaNEo2vm0=; b=nTtIUIC7WX/z9liZguyosSGGZNuMTgmNSkUFWcXuvnlQezUpdimbugf/ CZDbtdeTLdQD6AnWHd8Tw510G6lH0UPQgxhKmAQ/eEj83z+G0gq9e3i3K C4pWIuBFyT2pVSxQB/UoDUBjZjXutukg5Puge1y+7pX5geX0qnoHM+1bJ 62g7cL287xlED5HCahfwJ3s1G2JDFPpNSiflzbh5rJjGI5I+y1EsHbgOF XOJ/OpHFNLWtNLE9lYwwlgwqsiuc1peqXhKnz+jQS5+5zRAYMO/494Iab k9qlZsDGcD75edtfNYzzshe4xRl3R+Mi7mt5+v0m/s3SQ1VY0zeU8F0sc A==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="6155331" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6155331" Received: from orviesa009.jf.intel.com ([10.64.159.149]) by fmvoesa107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:04 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6615708" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa009-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:04 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com, Binbin Wu Subject: [PATCH v19 041/130] KVM: TDX: Refuse to unplug the last cpu on the package Date: Mon, 26 Feb 2024 00:25:43 -0800 Message-Id: <15d4d08cf1fe16777f8f8d84ee940e41afbad406.1708933498.git.isaku.yamahata@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata In order to reclaim TDX HKID, (i.e. when deleting guest TD), needs to call TDH.PHYMEM.PAGE.WBINVD on all packages. If we have active TDX HKID, refuse to offline the last online cpu to guarantee at least one CPU online per package. Add arch callback for cpu offline. Because TDX doesn't support suspend, this also refuses suspend if TDs are running. If no TD is running, suspend is allowed. Suggested-by: Sean Christopherson Signed-off-by: Isaku Yamahata Reviewed-by: Binbin Wu --- v18: - Added reviewed-by BinBin --- arch/x86/include/asm/kvm-x86-ops.h | 1 + arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/vmx/main.c | 1 + arch/x86/kvm/vmx/tdx.c | 41 ++++++++++++++++++++++++++++++ arch/x86/kvm/vmx/x86_ops.h | 2 ++ arch/x86/kvm/x86.c | 5 ++++ include/linux/kvm_host.h | 1 + virt/kvm/kvm_main.c | 12 +++++++-- 8 files changed, 62 insertions(+), 2 deletions(-) diff --git a/arch/x86/include/asm/kvm-x86-ops.h b/arch/x86/include/asm/kvm-x86-ops.h index 97f0a681e02c..f78200492a3d 100644 --- a/arch/x86/include/asm/kvm-x86-ops.h +++ b/arch/x86/include/asm/kvm-x86-ops.h @@ -18,6 +18,7 @@ KVM_X86_OP(check_processor_compatibility) KVM_X86_OP(hardware_enable) KVM_X86_OP(hardware_disable) KVM_X86_OP(hardware_unsetup) +KVM_X86_OP_OPTIONAL_RET0(offline_cpu) KVM_X86_OP(has_emulated_msr) KVM_X86_OP(vcpu_after_set_cpuid) KVM_X86_OP(is_vm_type_supported) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 6dedb5cb71ef..0e2408a4707e 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1600,6 +1600,7 @@ struct kvm_x86_ops { int (*hardware_enable)(void); void (*hardware_disable)(void); void (*hardware_unsetup)(void); + int (*offline_cpu)(void); bool (*has_emulated_msr)(struct kvm *kvm, u32 index); void (*vcpu_after_set_cpuid)(struct kvm_vcpu *vcpu); diff --git a/arch/x86/kvm/vmx/main.c b/arch/x86/kvm/vmx/main.c index 437c6d5e802e..d69dd474775b 100644 --- a/arch/x86/kvm/vmx/main.c +++ b/arch/x86/kvm/vmx/main.c @@ -110,6 +110,7 @@ struct kvm_x86_ops vt_x86_ops __initdata = { .check_processor_compatibility = vmx_check_processor_compat, .hardware_unsetup = vt_hardware_unsetup, + .offline_cpu = tdx_offline_cpu, /* TDX cpu enablement is done by tdx_hardware_setup(). */ .hardware_enable = vmx_hardware_enable, diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c index b11f105db3cd..f2ee5abac14e 100644 --- a/arch/x86/kvm/vmx/tdx.c +++ b/arch/x86/kvm/vmx/tdx.c @@ -97,6 +97,7 @@ int tdx_vm_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap) */ static DEFINE_MUTEX(tdx_lock); static struct mutex *tdx_mng_key_config_lock; +static atomic_t nr_configured_hkid; static __always_inline hpa_t set_hkid_to_hpa(hpa_t pa, u16 hkid) { @@ -112,6 +113,7 @@ static inline void tdx_hkid_free(struct kvm_tdx *kvm_tdx) { tdx_guest_keyid_free(kvm_tdx->hkid); kvm_tdx->hkid = -1; + atomic_dec(&nr_configured_hkid); } static inline bool is_hkid_assigned(struct kvm_tdx *kvm_tdx) @@ -586,6 +588,7 @@ static int __tdx_td_init(struct kvm *kvm, struct td_params *td_params, if (ret < 0) return ret; kvm_tdx->hkid = ret; + atomic_inc(&nr_configured_hkid); va = __get_free_page(GFP_KERNEL_ACCOUNT); if (!va) @@ -1071,3 +1074,41 @@ void tdx_hardware_unsetup(void) kfree(tdx_info); kfree(tdx_mng_key_config_lock); } + +int tdx_offline_cpu(void) +{ + int curr_cpu = smp_processor_id(); + cpumask_var_t packages; + int ret = 0; + int i; + + /* No TD is running. Allow any cpu to be offline. */ + if (!atomic_read(&nr_configured_hkid)) + return 0; + + /* + * In order to reclaim TDX HKID, (i.e. when deleting guest TD), need to + * call TDH.PHYMEM.PAGE.WBINVD on all packages to program all memory + * controller with pconfig. If we have active TDX HKID, refuse to + * offline the last online cpu. + */ + if (!zalloc_cpumask_var(&packages, GFP_KERNEL)) + return -ENOMEM; + for_each_online_cpu(i) { + if (i != curr_cpu) + cpumask_set_cpu(topology_physical_package_id(i), packages); + } + /* Check if this cpu is the last online cpu of this package. */ + if (!cpumask_test_cpu(topology_physical_package_id(curr_cpu), packages)) + ret = -EBUSY; + free_cpumask_var(packages); + if (ret) + /* + * Because it's hard for human operator to understand the + * reason, warn it. + */ +#define MSG_ALLPKG_ONLINE \ + "TDX requires all packages to have an online CPU. Delete all TDs in order to offline all CPUs of a package.\n" + pr_warn_ratelimited(MSG_ALLPKG_ONLINE); + return ret; +} diff --git a/arch/x86/kvm/vmx/x86_ops.h b/arch/x86/kvm/vmx/x86_ops.h index 7f123ffe4d42..33ab6800eab8 100644 --- a/arch/x86/kvm/vmx/x86_ops.h +++ b/arch/x86/kvm/vmx/x86_ops.h @@ -138,6 +138,7 @@ void vmx_setup_mce(struct kvm_vcpu *vcpu); int __init tdx_hardware_setup(struct kvm_x86_ops *x86_ops); void tdx_hardware_unsetup(void); bool tdx_is_vm_type_supported(unsigned long type); +int tdx_offline_cpu(void); int tdx_vm_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap); int tdx_vm_init(struct kvm *kvm); @@ -148,6 +149,7 @@ int tdx_vm_ioctl(struct kvm *kvm, void __user *argp); static inline int tdx_hardware_setup(struct kvm_x86_ops *x86_ops) { return -EOPNOTSUPP; } static inline void tdx_hardware_unsetup(void) {} static inline bool tdx_is_vm_type_supported(unsigned long type) { return false; } +static inline int tdx_offline_cpu(void) { return 0; } static inline int tdx_vm_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap) { diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 1f01c7f91652..e27ea5ed2968 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -12508,6 +12508,11 @@ void kvm_arch_hardware_disable(void) drop_user_return_notifiers(); } +int kvm_arch_offline_cpu(unsigned int cpu) +{ + return static_call(kvm_x86_offline_cpu)(); +} + bool kvm_vcpu_is_reset_bsp(struct kvm_vcpu *vcpu) { return vcpu->kvm->arch.bsp_vcpu_id == vcpu->vcpu_id; diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index eeaf4e73317c..813952eca1bf 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -1502,6 +1502,7 @@ static inline void kvm_create_vcpu_debugfs(struct kvm_vcpu *vcpu) {} int kvm_arch_hardware_enable(void); void kvm_arch_hardware_disable(void); #endif +int kvm_arch_offline_cpu(unsigned int cpu); int kvm_arch_vcpu_runnable(struct kvm_vcpu *vcpu); bool kvm_arch_vcpu_in_kernel(struct kvm_vcpu *vcpu); int kvm_arch_vcpu_should_kick(struct kvm_vcpu *vcpu); diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 2f0a8e28795e..de38f308738e 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -5653,13 +5653,21 @@ static void hardware_disable_nolock(void *junk) __this_cpu_write(hardware_enabled, false); } +__weak int kvm_arch_offline_cpu(unsigned int cpu) +{ + return 0; +} + static int kvm_offline_cpu(unsigned int cpu) { + int r = 0; + mutex_lock(&kvm_lock); - if (kvm_usage_count) + r = kvm_arch_offline_cpu(cpu); + if (!r && kvm_usage_count) hardware_disable_nolock(NULL); mutex_unlock(&kvm_lock); - return 0; + return r; } static void hardware_disable_all_nolock(void) From patchwork Mon Feb 26 08:25:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571469 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 27F1D5D8F3; Mon, 26 Feb 2024 08:28:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.13 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936090; cv=none; b=GordT0IfblK9Fyh33lId2AbHeV9X+ukFQqk7bAOzgpWTEqT0JQjOjk01QHeizogc29f72SnNkAzpMOxh3Puj8xDLEspYj4TuwpTVLvSV+G9JyfgT+GywA/mtDL+lIZibrHQxRVwpy8tQ3vcoOM8/oWNXx5LTyuDqYDlfBs5qg6Y= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936090; c=relaxed/simple; bh=i0N39TZmKLiC6CILEgWJyLqxHNkrsso5GLu5HE01vt8=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=E//l1YDN8SHeVngf7Mj4U17kTpZ2ii8k3o/6Bf4tfkmbEyb142Pt0rAvoP/tBoZM+aCYpw6bEOs4ehhsEB+jGjycfn4UjjKzWQa8Wg4xvS5euo/87wW2Y902UK+u49f/EhF+OX/Lj8h/wU7BBbYSmvD6C5I288OPnOBOhmHR67Q= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=ckxs4VcK; arc=none smtp.client-ip=192.198.163.13 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="ckxs4VcK" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936088; x=1740472088; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=i0N39TZmKLiC6CILEgWJyLqxHNkrsso5GLu5HE01vt8=; b=ckxs4VcKQN+iJgpO0aEBJu7qI+jwqx2Hqx2Yhk6e8OVW+6xEmNN0dqao Qa4KNBF8/bh11y+UlH6BvnqMPdlu43+HNqn5zNJN/0DTI9Sb7uxOZTs9N oAyE2ScDAbtLVt5mHuiwlyLQ/SspDY9lmaGWlRsSMXFm7HU92gQ6MaKFR BY9fBz+1FXVWQZ3JNNVPBYpsET3xeVg25aB1hC7iEItKXfABGUzAujYl8 ZX4g+kJfIYfCanoBblBey+eV09N+W70SqxwqANjkPuAxJFTon4PDeHMTQ ZZ2L4OHeu+g40CVTxIBHxUggxXglJ/cVZwXWA0nPrx2ycv82WYg776vEq g==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="6155335" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6155335" Received: from orviesa009.jf.intel.com ([10.64.159.149]) by fmvoesa107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:05 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6615719" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa009-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:05 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 042/130] [MARKER] The start of TDX KVM patch series: TD vcpu creation/destruction Date: Mon, 26 Feb 2024 00:25:44 -0800 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata This empty commit is to mark the start of patch series of TD vcpu creation/destruction. Signed-off-by: Isaku Yamahata --- Documentation/virt/kvm/intel-tdx-layer-status.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/Documentation/virt/kvm/intel-tdx-layer-status.rst b/Documentation/virt/kvm/intel-tdx-layer-status.rst index 098150da6ea2..25082e9c0b20 100644 --- a/Documentation/virt/kvm/intel-tdx-layer-status.rst +++ b/Documentation/virt/kvm/intel-tdx-layer-status.rst @@ -9,7 +9,7 @@ Layer status What qemu can do ---------------- - TDX VM TYPE is exposed to Qemu. -- Qemu can try to create VM of TDX VM type and then fails. +- Qemu can create/destroy guest of TDX vm type. Patch Layer status ------------------ @@ -17,8 +17,8 @@ Patch Layer status * TDX, VMX coexistence: Applied * TDX architectural definitions: Applied -* TD VM creation/destruction: Applying -* TD vcpu creation/destruction: Not yet +* TD VM creation/destruction: Applied +* TD vcpu creation/destruction: Applying * TDX EPT violation: Not yet * TD finalization: Not yet * TD vcpu enter/exit: Not yet From patchwork Mon Feb 26 08:25:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571471 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C608D5D918; Mon, 26 Feb 2024 08:28:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.13 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936091; cv=none; b=GQyrAJ86XjRuIaty089hSxDPC40TdyZnJDW9XiX18g8gTkcSNR5756LARCRjbZ62OMCB1MPV+isgHvInfhQoPLL2cSaER9eNhfrpqUGIapz6ZZEFGE9AbxyJEE4HVL71jY3ksyQwrvD5CuJn897POogffiFUr+g0dNjpyMvq2c8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936091; c=relaxed/simple; bh=PKxBb1/7MZ4y5rGfKNqIu7WXSIt1UDXpkovYfwoMgx8=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=eC59KhmVeNtfEs7pHz8YOi7SBogvYjHC2kkvNd6Q6Wj6+wLPecft5+u4FMiKMVs9vNXg/JAjCwX1beR+8ubamzxek10ejWCpIZzz1qMLxV9nYIIH+4App9SRTNpVpIOYmar/Evx1m8V0bse3hpMNI7Ep8q2PkOha6aXb5rTo5zs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=Uj0qCm7D; arc=none smtp.client-ip=192.198.163.13 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="Uj0qCm7D" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936089; x=1740472089; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=PKxBb1/7MZ4y5rGfKNqIu7WXSIt1UDXpkovYfwoMgx8=; b=Uj0qCm7Dga/PrAdx0xPtkfnskVBKQacmGBxLizm4JQFDSD0NfWm71H8Y DEmNR48zVQDnEV4bjtKkchOpCBReWstr30/qHbMssxviRiG1IJiITd6ZF Dhf7c7KUzQSUdaYfcr63Zbqk6Zerq6dXvRy+Q3Rkq7aqf1DvLrF0nuQvu +4e1X2JWr6fTw4I1nyhCzn7bEPxA5QHLuoGqHpgMxpKn5tKexLjTnLAyX kv8+HRKPAdNlWlaFG+h3/U/XisBlL9deHSBtqiusYh9uo7BwfAirfSRoz ZiouosoV304E8jEU9saCLZF1+qHqsxXnqoWapTDl18uQAzmTNIn1smXIE A==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="6155345" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6155345" Received: from orviesa009.jf.intel.com ([10.64.159.149]) by fmvoesa107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:05 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6615736" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa009-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:05 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 043/130] KVM: TDX: create/free TDX vcpu structure Date: Mon, 26 Feb 2024 00:25:45 -0800 Message-Id: <51c4203e844159451f5a78fb18cc5bebcc38a76e.1708933498.git.isaku.yamahata@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata The next step of TDX guest creation is to create vcpu. Create TDX vcpu structures, initialize it that doesn't require TDX SEAMCALL. TDX specific vcpu initialization will be implemented as independent KVM_TDX_INIT_VCPU so that when error occurs it's easy to determine which component has the issue, KVM or TDX. Signed-off-by: Isaku Yamahata --- v19: - removed stale comment in tdx_vcpu_create(). v18: - update commit log to use create instead of allocate because the patch doesn't newly allocate memory for TDX vcpu. v15 -> v16: - Add AMX support as the KVM upstream supports it. Signed-off-by: Isaku Yamahata --- arch/x86/kvm/vmx/main.c | 44 ++++++++++++++++++++++++++++++++++---- arch/x86/kvm/vmx/tdx.c | 44 ++++++++++++++++++++++++++++++++++++++ arch/x86/kvm/vmx/x86_ops.h | 10 +++++++++ arch/x86/kvm/x86.c | 2 ++ 4 files changed, 96 insertions(+), 4 deletions(-) diff --git a/arch/x86/kvm/vmx/main.c b/arch/x86/kvm/vmx/main.c index d69dd474775b..5796fb45433f 100644 --- a/arch/x86/kvm/vmx/main.c +++ b/arch/x86/kvm/vmx/main.c @@ -87,6 +87,42 @@ static void vt_vm_free(struct kvm *kvm) tdx_vm_free(kvm); } +static int vt_vcpu_precreate(struct kvm *kvm) +{ + if (is_td(kvm)) + return 0; + + return vmx_vcpu_precreate(kvm); +} + +static int vt_vcpu_create(struct kvm_vcpu *vcpu) +{ + if (is_td_vcpu(vcpu)) + return tdx_vcpu_create(vcpu); + + return vmx_vcpu_create(vcpu); +} + +static void vt_vcpu_free(struct kvm_vcpu *vcpu) +{ + if (is_td_vcpu(vcpu)) { + tdx_vcpu_free(vcpu); + return; + } + + vmx_vcpu_free(vcpu); +} + +static void vt_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event) +{ + if (is_td_vcpu(vcpu)) { + tdx_vcpu_reset(vcpu, init_event); + return; + } + + vmx_vcpu_reset(vcpu, init_event); +} + static int vt_mem_enc_ioctl(struct kvm *kvm, void __user *argp) { if (!is_td(kvm)) @@ -126,10 +162,10 @@ struct kvm_x86_ops vt_x86_ops __initdata = { .vm_destroy = vt_vm_destroy, .vm_free = vt_vm_free, - .vcpu_precreate = vmx_vcpu_precreate, - .vcpu_create = vmx_vcpu_create, - .vcpu_free = vmx_vcpu_free, - .vcpu_reset = vmx_vcpu_reset, + .vcpu_precreate = vt_vcpu_precreate, + .vcpu_create = vt_vcpu_create, + .vcpu_free = vt_vcpu_free, + .vcpu_reset = vt_vcpu_reset, .prepare_switch_to_guest = vmx_prepare_switch_to_guest, .vcpu_load = vmx_vcpu_load, diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c index f2ee5abac14e..51283d2cd011 100644 --- a/arch/x86/kvm/vmx/tdx.c +++ b/arch/x86/kvm/vmx/tdx.c @@ -368,6 +368,50 @@ int tdx_vm_init(struct kvm *kvm) return 0; } +int tdx_vcpu_create(struct kvm_vcpu *vcpu) +{ + struct kvm_tdx *kvm_tdx = to_kvm_tdx(vcpu->kvm); + + WARN_ON_ONCE(vcpu->arch.cpuid_entries); + WARN_ON_ONCE(vcpu->arch.cpuid_nent); + + /* TDX only supports x2APIC, which requires an in-kernel local APIC. */ + if (!vcpu->arch.apic) + return -EINVAL; + + fpstate_set_confidential(&vcpu->arch.guest_fpu); + + vcpu->arch.efer = EFER_SCE | EFER_LME | EFER_LMA | EFER_NX; + + vcpu->arch.cr0_guest_owned_bits = -1ul; + vcpu->arch.cr4_guest_owned_bits = -1ul; + + vcpu->arch.tsc_offset = to_kvm_tdx(vcpu->kvm)->tsc_offset; + vcpu->arch.l1_tsc_offset = vcpu->arch.tsc_offset; + vcpu->arch.guest_state_protected = + !(to_kvm_tdx(vcpu->kvm)->attributes & TDX_TD_ATTRIBUTE_DEBUG); + + if ((kvm_tdx->xfam & XFEATURE_MASK_XTILE) == XFEATURE_MASK_XTILE) + vcpu->arch.xfd_no_write_intercept = true; + + return 0; +} + +void tdx_vcpu_free(struct kvm_vcpu *vcpu) +{ + /* This is stub for now. More logic will come. */ +} + +void tdx_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event) +{ + + /* Ignore INIT silently because TDX doesn't support INIT event. */ + if (init_event) + return; + + /* This is stub for now. More logic will come here. */ +} + static int tdx_get_capabilities(struct kvm_tdx_cmd *cmd) { struct kvm_tdx_capabilities __user *user_caps; diff --git a/arch/x86/kvm/vmx/x86_ops.h b/arch/x86/kvm/vmx/x86_ops.h index 33ab6800eab8..bb73a9b5b354 100644 --- a/arch/x86/kvm/vmx/x86_ops.h +++ b/arch/x86/kvm/vmx/x86_ops.h @@ -144,7 +144,12 @@ int tdx_vm_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap); int tdx_vm_init(struct kvm *kvm); void tdx_mmu_release_hkid(struct kvm *kvm); void tdx_vm_free(struct kvm *kvm); + int tdx_vm_ioctl(struct kvm *kvm, void __user *argp); + +int tdx_vcpu_create(struct kvm_vcpu *vcpu); +void tdx_vcpu_free(struct kvm_vcpu *vcpu); +void tdx_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event); #else static inline int tdx_hardware_setup(struct kvm_x86_ops *x86_ops) { return -EOPNOTSUPP; } static inline void tdx_hardware_unsetup(void) {} @@ -158,7 +163,12 @@ static inline int tdx_vm_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap) static inline int tdx_vm_init(struct kvm *kvm) { return -EOPNOTSUPP; } static inline void tdx_mmu_release_hkid(struct kvm *kvm) {} static inline void tdx_vm_free(struct kvm *kvm) {} + static inline int tdx_vm_ioctl(struct kvm *kvm, void __user *argp) { return -EOPNOTSUPP; } + +static inline int tdx_vcpu_create(struct kvm_vcpu *vcpu) { return -EOPNOTSUPP; } +static inline void tdx_vcpu_free(struct kvm_vcpu *vcpu) {} +static inline void tdx_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event) {} #endif #endif /* __KVM_X86_VMX_X86_OPS_H */ diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index e27ea5ed2968..c002761bb662 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -502,6 +502,7 @@ int kvm_set_apic_base(struct kvm_vcpu *vcpu, struct msr_data *msr_info) kvm_recalculate_apic_map(vcpu->kvm); return 0; } +EXPORT_SYMBOL_GPL(kvm_set_apic_base); /* * Handle a fault on a hardware virtualization (VMX or SVM) instruction. @@ -12517,6 +12518,7 @@ bool kvm_vcpu_is_reset_bsp(struct kvm_vcpu *vcpu) { return vcpu->kvm->arch.bsp_vcpu_id == vcpu->vcpu_id; } +EXPORT_SYMBOL_GPL(kvm_vcpu_is_reset_bsp); bool kvm_vcpu_is_bsp(struct kvm_vcpu *vcpu) { From patchwork Mon Feb 26 08:25:46 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571473 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 785435EE68; Mon, 26 Feb 2024 08:28:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.13 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936093; cv=none; b=jo1E3bJK2Qhjjwy5dAoFXg+tWC2shPi0KgipjZAs0Tz2RuAw7+U5sXUT00goTv77nBjL1raU52oUzktvoQnZMMvSk9DsYjcnS1UcTHleXl2Hg6l+jvxf+J4ZzJe0q6CZT2hzMXzH4Td3DsM8s9yvWDeQdhcW18Z6jiYqeVWj6ys= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936093; c=relaxed/simple; bh=REv47zLeoxS3Byb4dXvmUYlegjywmaaj7+3hzE3rczs=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=mOx1pNQxiCEgG0NVL9x2sBtnYBK6pe//XpxV2sfvAuK9O7JDXukdQKp524rCDYK6UOMjERnrOBz5Qz1nkcq0FneMsXjOSzjzE9iKUJSR9Lp2TWUJJxTmqW72zt53UctDh55RrVhIZKMAs6FZ4LpBwSxJmJ46DYoTiSMLHXvFBiE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=H8/2wT0F; arc=none smtp.client-ip=192.198.163.13 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="H8/2wT0F" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936090; x=1740472090; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=REv47zLeoxS3Byb4dXvmUYlegjywmaaj7+3hzE3rczs=; b=H8/2wT0FbewJQB2ZPI2fgUnxS8qceU4dwBi8qFfdzn9biLYXRA0cDF1l BXDHvcLUkREOxDLklD9lC4vB5ZBRjEv71F2vQ0CWOVwAtK7lFneefIp5S o0aTStK79jQI4FBM2qkTiQZIKLoOnAtpgOog+mPoWBYb9DBNQegkjSTiM P5TGVgvh7o1RyqpufEUegMj74YLI8lhr11C1yqSCpGtpByYWKww4rB3V2 dMEtWbqKNBaDQBTHaNWahaZ/J4rqT3fkyYfzvmulz5Y08FkekcVkh+U4m R7NYUHNvKx+Jd8twiFL8LdlQAXgpeGk+LAA7HnuyB2lsW6scCaBtZ+uLJ w==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="6155349" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6155349" Received: from orviesa009.jf.intel.com ([10.64.159.149]) by fmvoesa107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:06 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6615745" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa009-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:06 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com, Sean Christopherson Subject: [PATCH v19 044/130] KVM: TDX: Do TDX specific vcpu initialization Date: Mon, 26 Feb 2024 00:25:46 -0800 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata TD guest vcpu needs TDX specific initialization before running. Repurpose KVM_MEMORY_ENCRYPT_OP to vcpu-scope, add a new sub-command KVM_TDX_INIT_VCPU, and implement the callback for it. Signed-off-by: Sean Christopherson Signed-off-by: Isaku Yamahata --- v18: - Use tdh_sys_rd() instead of struct tdsysinfo_struct. - Rename tdx_reclaim_td_page() => tdx_reclaim_control_page() - Remove the change of tools/arch/x86/include/uapi/asm/kvm.h. --- arch/x86/include/asm/kvm-x86-ops.h | 1 + arch/x86/include/asm/kvm_host.h | 1 + arch/x86/include/uapi/asm/kvm.h | 1 + arch/x86/kvm/vmx/main.c | 9 ++ arch/x86/kvm/vmx/tdx.c | 184 ++++++++++++++++++++++++++++- arch/x86/kvm/vmx/tdx.h | 8 ++ arch/x86/kvm/vmx/x86_ops.h | 4 + arch/x86/kvm/x86.c | 6 + 8 files changed, 211 insertions(+), 3 deletions(-) diff --git a/arch/x86/include/asm/kvm-x86-ops.h b/arch/x86/include/asm/kvm-x86-ops.h index f78200492a3d..a8e96804a252 100644 --- a/arch/x86/include/asm/kvm-x86-ops.h +++ b/arch/x86/include/asm/kvm-x86-ops.h @@ -129,6 +129,7 @@ KVM_X86_OP(leave_smm) KVM_X86_OP(enable_smi_window) #endif KVM_X86_OP(mem_enc_ioctl) +KVM_X86_OP_OPTIONAL(vcpu_mem_enc_ioctl) KVM_X86_OP_OPTIONAL(mem_enc_register_region) KVM_X86_OP_OPTIONAL(mem_enc_unregister_region) KVM_X86_OP_OPTIONAL(vm_copy_enc_context_from) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 0e2408a4707e..5da3c211955d 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1778,6 +1778,7 @@ struct kvm_x86_ops { #endif int (*mem_enc_ioctl)(struct kvm *kvm, void __user *argp); + int (*vcpu_mem_enc_ioctl)(struct kvm_vcpu *vcpu, void __user *argp); int (*mem_enc_register_region)(struct kvm *kvm, struct kvm_enc_region *argp); int (*mem_enc_unregister_region)(struct kvm *kvm, struct kvm_enc_region *argp); int (*vm_copy_enc_context_from)(struct kvm *kvm, unsigned int source_fd); diff --git a/arch/x86/include/uapi/asm/kvm.h b/arch/x86/include/uapi/asm/kvm.h index 9ac0246bd974..4000a2e087a8 100644 --- a/arch/x86/include/uapi/asm/kvm.h +++ b/arch/x86/include/uapi/asm/kvm.h @@ -571,6 +571,7 @@ struct kvm_pmu_event_filter { enum kvm_tdx_cmd_id { KVM_TDX_CAPABILITIES = 0, KVM_TDX_INIT_VM, + KVM_TDX_INIT_VCPU, KVM_TDX_CMD_NR_MAX, }; diff --git a/arch/x86/kvm/vmx/main.c b/arch/x86/kvm/vmx/main.c index 5796fb45433f..d0f75020579f 100644 --- a/arch/x86/kvm/vmx/main.c +++ b/arch/x86/kvm/vmx/main.c @@ -131,6 +131,14 @@ static int vt_mem_enc_ioctl(struct kvm *kvm, void __user *argp) return tdx_vm_ioctl(kvm, argp); } +static int vt_vcpu_mem_enc_ioctl(struct kvm_vcpu *vcpu, void __user *argp) +{ + if (!is_td_vcpu(vcpu)) + return -EINVAL; + + return tdx_vcpu_ioctl(vcpu, argp); +} + #define VMX_REQUIRED_APICV_INHIBITS \ (BIT(APICV_INHIBIT_REASON_DISABLE)| \ BIT(APICV_INHIBIT_REASON_ABSENT) | \ @@ -291,6 +299,7 @@ struct kvm_x86_ops vt_x86_ops __initdata = { .get_untagged_addr = vmx_get_untagged_addr, .mem_enc_ioctl = vt_mem_enc_ioctl, + .vcpu_mem_enc_ioctl = vt_vcpu_mem_enc_ioctl, }; struct kvm_x86_init_ops vt_init_ops __initdata = { diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c index 51283d2cd011..aa1da51b8af7 100644 --- a/arch/x86/kvm/vmx/tdx.c +++ b/arch/x86/kvm/vmx/tdx.c @@ -49,6 +49,7 @@ struct tdx_info { u64 xfam_fixed1; u8 nr_tdcs_pages; + u8 nr_tdvpx_pages; u16 num_cpuid_config; /* This must the last member. */ @@ -104,6 +105,11 @@ static __always_inline hpa_t set_hkid_to_hpa(hpa_t pa, u16 hkid) return pa | ((hpa_t)hkid << boot_cpu_data.x86_phys_bits); } +static inline bool is_td_vcpu_created(struct vcpu_tdx *tdx) +{ + return tdx->td_vcpu_created; +} + static inline bool is_td_created(struct kvm_tdx *kvm_tdx) { return kvm_tdx->tdr_pa; @@ -121,6 +127,11 @@ static inline bool is_hkid_assigned(struct kvm_tdx *kvm_tdx) return kvm_tdx->hkid > 0; } +static inline bool is_td_finalized(struct kvm_tdx *kvm_tdx) +{ + return kvm_tdx->finalized; +} + static void tdx_clear_page(unsigned long page_pa) { const void *zero_page = (const void *) __va(page_to_phys(ZERO_PAGE(0))); @@ -399,7 +410,32 @@ int tdx_vcpu_create(struct kvm_vcpu *vcpu) void tdx_vcpu_free(struct kvm_vcpu *vcpu) { - /* This is stub for now. More logic will come. */ + struct vcpu_tdx *tdx = to_tdx(vcpu); + int i; + + /* + * This methods can be called when vcpu allocation/initialization + * failed. So it's possible that hkid, tdvpx and tdvpr are not assigned + * yet. + */ + if (is_hkid_assigned(to_kvm_tdx(vcpu->kvm))) { + WARN_ON_ONCE(tdx->tdvpx_pa); + WARN_ON_ONCE(tdx->tdvpr_pa); + return; + } + + if (tdx->tdvpx_pa) { + for (i = 0; i < tdx_info->nr_tdvpx_pages; i++) { + if (tdx->tdvpx_pa[i]) + tdx_reclaim_control_page(tdx->tdvpx_pa[i]); + } + kfree(tdx->tdvpx_pa); + tdx->tdvpx_pa = NULL; + } + if (tdx->tdvpr_pa) { + tdx_reclaim_control_page(tdx->tdvpr_pa); + tdx->tdvpr_pa = 0; + } } void tdx_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event) @@ -408,8 +444,13 @@ void tdx_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event) /* Ignore INIT silently because TDX doesn't support INIT event. */ if (init_event) return; + if (KVM_BUG_ON(is_td_vcpu_created(to_tdx(vcpu)), vcpu->kvm)) + return; - /* This is stub for now. More logic will come here. */ + /* + * Don't update mp_state to runnable because more initialization + * is needed by TDX_VCPU_INIT. + */ } static int tdx_get_capabilities(struct kvm_tdx_cmd *cmd) @@ -904,6 +945,137 @@ int tdx_vm_ioctl(struct kvm *kvm, void __user *argp) return r; } +/* VMM can pass one 64bit auxiliary data to vcpu via RCX for guest BIOS. */ +static int tdx_td_vcpu_init(struct kvm_vcpu *vcpu, u64 vcpu_rcx) +{ + struct kvm_tdx *kvm_tdx = to_kvm_tdx(vcpu->kvm); + struct vcpu_tdx *tdx = to_tdx(vcpu); + unsigned long *tdvpx_pa = NULL; + unsigned long tdvpr_pa; + unsigned long va; + int ret, i; + u64 err; + + if (is_td_vcpu_created(tdx)) + return -EINVAL; + + /* + * vcpu_free method frees allocated pages. Avoid partial setup so + * that the method can't handle it. + */ + va = __get_free_page(GFP_KERNEL_ACCOUNT); + if (!va) + return -ENOMEM; + tdvpr_pa = __pa(va); + + tdvpx_pa = kcalloc(tdx_info->nr_tdvpx_pages, sizeof(*tdx->tdvpx_pa), + GFP_KERNEL_ACCOUNT); + if (!tdvpx_pa) { + ret = -ENOMEM; + goto free_tdvpr; + } + for (i = 0; i < tdx_info->nr_tdvpx_pages; i++) { + va = __get_free_page(GFP_KERNEL_ACCOUNT); + if (!va) { + ret = -ENOMEM; + goto free_tdvpx; + } + tdvpx_pa[i] = __pa(va); + } + + err = tdh_vp_create(kvm_tdx->tdr_pa, tdvpr_pa); + if (KVM_BUG_ON(err, vcpu->kvm)) { + ret = -EIO; + pr_tdx_error(TDH_VP_CREATE, err, NULL); + goto free_tdvpx; + } + tdx->tdvpr_pa = tdvpr_pa; + + tdx->tdvpx_pa = tdvpx_pa; + for (i = 0; i < tdx_info->nr_tdvpx_pages; i++) { + err = tdh_vp_addcx(tdx->tdvpr_pa, tdvpx_pa[i]); + if (KVM_BUG_ON(err, vcpu->kvm)) { + pr_tdx_error(TDH_VP_ADDCX, err, NULL); + for (; i < tdx_info->nr_tdvpx_pages; i++) { + free_page((unsigned long)__va(tdvpx_pa[i])); + tdvpx_pa[i] = 0; + } + /* vcpu_free method frees TDVPX and TDR donated to TDX */ + return -EIO; + } + } + + err = tdh_vp_init(tdx->tdvpr_pa, vcpu_rcx); + if (KVM_BUG_ON(err, vcpu->kvm)) { + pr_tdx_error(TDH_VP_INIT, err, NULL); + return -EIO; + } + + vcpu->arch.mp_state = KVM_MP_STATE_RUNNABLE; + tdx->td_vcpu_created = true; + return 0; + +free_tdvpx: + for (i = 0; i < tdx_info->nr_tdvpx_pages; i++) { + if (tdvpx_pa[i]) + free_page((unsigned long)__va(tdvpx_pa[i])); + tdvpx_pa[i] = 0; + } + kfree(tdvpx_pa); + tdx->tdvpx_pa = NULL; +free_tdvpr: + if (tdvpr_pa) + free_page((unsigned long)__va(tdvpr_pa)); + tdx->tdvpr_pa = 0; + + return ret; +} + +int tdx_vcpu_ioctl(struct kvm_vcpu *vcpu, void __user *argp) +{ + struct msr_data apic_base_msr; + struct kvm_tdx *kvm_tdx = to_kvm_tdx(vcpu->kvm); + struct vcpu_tdx *tdx = to_tdx(vcpu); + struct kvm_tdx_cmd cmd; + int ret; + + if (tdx->initialized) + return -EINVAL; + + if (!is_hkid_assigned(kvm_tdx) || is_td_finalized(kvm_tdx)) + return -EINVAL; + + if (copy_from_user(&cmd, argp, sizeof(cmd))) + return -EFAULT; + + if (cmd.error) + return -EINVAL; + + /* Currently only KVM_TDX_INTI_VCPU is defined for vcpu operation. */ + if (cmd.flags || cmd.id != KVM_TDX_INIT_VCPU) + return -EINVAL; + + /* + * As TDX requires X2APIC, set local apic mode to X2APIC. User space + * VMM, e.g. qemu, is required to set CPUID[0x1].ecx.X2APIC=1 by + * KVM_SET_CPUID2. Otherwise kvm_set_apic_base() will fail. + */ + apic_base_msr = (struct msr_data) { + .host_initiated = true, + .data = APIC_DEFAULT_PHYS_BASE | LAPIC_MODE_X2APIC | + (kvm_vcpu_is_reset_bsp(vcpu) ? MSR_IA32_APICBASE_BSP : 0), + }; + if (kvm_set_apic_base(vcpu, &apic_base_msr)) + return -EINVAL; + + ret = tdx_td_vcpu_init(vcpu, (u64)cmd.data); + if (ret) + return ret; + + tdx->initialized = true; + return 0; +} + #define TDX_MD_MAP(_fid, _ptr) \ { .fid = MD_FIELD_ID_##_fid, \ .ptr = (_ptr), } @@ -953,13 +1125,14 @@ static int tdx_md_read(struct tdx_md_map *maps, int nr_maps) static int __init tdx_module_setup(void) { - u16 num_cpuid_config, tdcs_base_size; + u16 num_cpuid_config, tdcs_base_size, tdvps_base_size; int ret; u32 i; struct tdx_md_map mds[] = { TDX_MD_MAP(NUM_CPUID_CONFIG, &num_cpuid_config), TDX_MD_MAP(TDCS_BASE_SIZE, &tdcs_base_size), + TDX_MD_MAP(TDVPS_BASE_SIZE, &tdvps_base_size), }; struct tdx_metadata_field_mapping fields[] = { @@ -1013,6 +1186,11 @@ static int __init tdx_module_setup(void) } tdx_info->nr_tdcs_pages = tdcs_base_size / PAGE_SIZE; + /* + * TDVPS = TDVPR(4K page) + TDVPX(multiple 4K pages). + * -1 for TDVPR. + */ + tdx_info->nr_tdvpx_pages = tdvps_base_size / PAGE_SIZE - 1; /* * Make TDH.VP.ENTER preserve RBP so that the stack unwinder diff --git a/arch/x86/kvm/vmx/tdx.h b/arch/x86/kvm/vmx/tdx.h index 173ed19207fb..d3077151252c 100644 --- a/arch/x86/kvm/vmx/tdx.h +++ b/arch/x86/kvm/vmx/tdx.h @@ -17,12 +17,20 @@ struct kvm_tdx { u64 xfam; int hkid; + bool finalized; + u64 tsc_offset; }; struct vcpu_tdx { struct kvm_vcpu vcpu; + unsigned long tdvpr_pa; + unsigned long *tdvpx_pa; + bool td_vcpu_created; + + bool initialized; + /* * Dummy to make pmu_intel not corrupt memory. * TODO: Support PMU for TDX. Future work. diff --git a/arch/x86/kvm/vmx/x86_ops.h b/arch/x86/kvm/vmx/x86_ops.h index bb73a9b5b354..f5820f617b2e 100644 --- a/arch/x86/kvm/vmx/x86_ops.h +++ b/arch/x86/kvm/vmx/x86_ops.h @@ -150,6 +150,8 @@ int tdx_vm_ioctl(struct kvm *kvm, void __user *argp); int tdx_vcpu_create(struct kvm_vcpu *vcpu); void tdx_vcpu_free(struct kvm_vcpu *vcpu); void tdx_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event); + +int tdx_vcpu_ioctl(struct kvm_vcpu *vcpu, void __user *argp); #else static inline int tdx_hardware_setup(struct kvm_x86_ops *x86_ops) { return -EOPNOTSUPP; } static inline void tdx_hardware_unsetup(void) {} @@ -169,6 +171,8 @@ static inline int tdx_vm_ioctl(struct kvm *kvm, void __user *argp) { return -EOP static inline int tdx_vcpu_create(struct kvm_vcpu *vcpu) { return -EOPNOTSUPP; } static inline void tdx_vcpu_free(struct kvm_vcpu *vcpu) {} static inline void tdx_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event) {} + +static inline int tdx_vcpu_ioctl(struct kvm_vcpu *vcpu, void __user *argp) { return -EOPNOTSUPP; } #endif #endif /* __KVM_X86_VMX_X86_OPS_H */ diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index c002761bb662..2bd4b7c8fa51 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -6274,6 +6274,12 @@ long kvm_arch_vcpu_ioctl(struct file *filp, case KVM_SET_DEVICE_ATTR: r = kvm_vcpu_ioctl_device_attr(vcpu, ioctl, argp); break; + case KVM_MEMORY_ENCRYPT_OP: + r = -ENOTTY; + if (!kvm_x86_ops.vcpu_mem_enc_ioctl) + goto out; + r = kvm_x86_ops.vcpu_mem_enc_ioctl(vcpu, argp); + break; default: r = -EINVAL; } From patchwork Mon Feb 26 08:25:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571472 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B041B5EE75; Mon, 26 Feb 2024 08:28:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.13 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936092; cv=none; b=MCqLFRwpD61tqhX5oFYzNvpUcZgXf7k4ifESCu4xL+g44RPRS8E6fHJSgF9Ocgt9TPX4sFSzxppdNxBJTk3+p+FTx4Ze94VIOdNg4UxH1Ve5FxEuG+z16wJrhs2a1jhXLSuNHTp5T5RwaP3ryZ7yzFxIrNcSaSi24uNAth2vsGs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936092; c=relaxed/simple; bh=ESlsFtK7TCNNodzGvNFsc1Y94K335DvZHywlXfgF6Fc=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=F5ZKgIpxdT4+NaZmIoiJy9LP5cQ/FHV3klWcLWoIs9hNHxaCznzLWUgkIrn8HRg1mzIzymo3gaw30OeddUbaFBQL6/vwx57bVw8mpp7ogIWfJ8gv2CKmw0dBanfCFQyltUVV2swKkzdi3U97961wgkY3ju4UadGJexB1MlV7sGE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=A7igyK3A; arc=none smtp.client-ip=192.198.163.13 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="A7igyK3A" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936090; x=1740472090; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=ESlsFtK7TCNNodzGvNFsc1Y94K335DvZHywlXfgF6Fc=; b=A7igyK3AJBVHmD9O1hdK2HbcpoQ5RFi24AZV1kCsTrWtgIVDkcVZe9Az peHOc/cOtPZsrDppgWqPYg15nq+TxCVYYyorV2zVCPWBdMq761X/qU5xP 6OQ6wrRk0jE/3zNzyCfp6g9nmvdyMTpDOi3mjz6G0dr3YLvp4mUG9DA7G L6KnNVMUkFD/EIjcCXW+NAImd1mJM9Mzs9XSom4L30DmRXBPSKingUPhS Egn1z6v+jvlV2cBeUzOta7swtoXBvr2Y6RtIMbA0INLAtchU9X1wgh2KI LcAgyybRevIYj6Rcmoiug4kCsTKKCGFCPc17JQJg4bO/8K/UEqs6QXZPF g==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="6155353" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6155353" Received: from orviesa009.jf.intel.com ([10.64.159.149]) by fmvoesa107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:07 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6615753" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa009-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:06 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 045/130] [MARKER] The start of TDX KVM patch series: KVM MMU GPA shared bits Date: Mon, 26 Feb 2024 00:25:47 -0800 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata This empty commit is to mark the start of patch series of KVM MMU GPA shared bits. Signed-off-by: Isaku Yamahata --- Documentation/virt/kvm/intel-tdx-layer-status.rst | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/Documentation/virt/kvm/intel-tdx-layer-status.rst b/Documentation/virt/kvm/intel-tdx-layer-status.rst index 25082e9c0b20..8b8186e7bfeb 100644 --- a/Documentation/virt/kvm/intel-tdx-layer-status.rst +++ b/Documentation/virt/kvm/intel-tdx-layer-status.rst @@ -10,6 +10,7 @@ What qemu can do ---------------- - TDX VM TYPE is exposed to Qemu. - Qemu can create/destroy guest of TDX vm type. +- Qemu can create/destroy vcpu of TDX vm type. Patch Layer status ------------------ @@ -18,12 +19,12 @@ Patch Layer status * TDX, VMX coexistence: Applied * TDX architectural definitions: Applied * TD VM creation/destruction: Applied -* TD vcpu creation/destruction: Applying +* TD vcpu creation/destruction: Applied * TDX EPT violation: Not yet * TD finalization: Not yet * TD vcpu enter/exit: Not yet * TD vcpu interrupts/exit/hypercall: Not yet -* KVM MMU GPA shared bits: Not yet +* KVM MMU GPA shared bits: Applying * KVM TDP refactoring for TDX: Not yet * KVM TDP MMU hooks: Not yet From patchwork Mon Feb 26 08:25:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571474 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 464A05EE9B; Mon, 26 Feb 2024 08:28:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.13 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936093; cv=none; b=imq5Q2hk34yP/2sZK6zbs4DIU86UxVILaokfD7EHOB+QfAkcPJ9MUxJ6XgDwwBd1BU7MQjx6E8IqsDK7iJfpEhamJWl9KcrrR57FhQ7UxWLw50PWE7G3danC2VlUv3Xx0FqkwjqMis+PNF3NMyakZy+1rIH1Mu/DCY0ViYKom2o= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936093; c=relaxed/simple; bh=D+jzGkz5XMS9nEYBnMXKeOkISdExJrH5pF6akuAEA/U=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=bhnAuHlEzae/UL1eKrBYkyjlb3cb9oEw+loF+UZjkttps6900meAsobiaQFiNGZK9DpSwBu1uWWKKoKQLIo9cvYZs0yqFNJNsYz4Zv4zJybS/+7HcHJmjMPVHix6vlFSDQdLzC3K6vTsfSi+pBZFnVj+lQnEz6SRqcG/8yxQ2Nw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=Q8kaec9r; arc=none smtp.client-ip=192.198.163.13 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="Q8kaec9r" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936091; x=1740472091; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=D+jzGkz5XMS9nEYBnMXKeOkISdExJrH5pF6akuAEA/U=; b=Q8kaec9rLQ2VmGOomF57fDYoi/0oTOS092v9sT1crly2kPLxmDh/09PP 7ZsopVvnuT0x9WUtprx/mS+ocvsTcRObSfyIqT41rA8AIHkEkSnBId40X JoA1v9kANu9gnQIvULmgJy3HZMSJrriU5SQNSRBiogi4carzkDzBK2V+7 2/0jrz42x6cWCnLJP5x2OWxM0z8+pnlrluVMZh5X/bz8ohgM1VSHhCly2 BO2RNc8bvHT7VslF+88hBSQ0O6WKN/jJ4VinmuW865FptqN1/joMZkr99 tUrGDxxaMSt0xpSdOdOOBeEhLGY2v8a+GUwTyZ1T1YYON70uHscCiZre0 w==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="6155357" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6155357" Received: from orviesa009.jf.intel.com ([10.64.159.149]) by fmvoesa107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:07 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6615766" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa009-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:07 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com, Rick Edgecombe , Binbin Wu Subject: [PATCH v19 046/130] KVM: x86/mmu: Add address conversion functions for TDX shared bit of GPA Date: Mon, 26 Feb 2024 00:25:48 -0800 Message-Id: <973a3e06111fe84f2b1e971636cbaa3facf7b120.1708933498.git.isaku.yamahata@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata TDX repurposes one GPA bit (51 bit or 47 bit based on configuration) to indicate the GPA is private(if cleared) or shared (if set) with VMM. If GPA.shared is set, GPA is covered by the existing conventional EPT pointed by EPTP. If GPA.shared bit is cleared, GPA is covered by TDX module. VMM has to issue SEAMCALLs to operate. Add a member to remember GPA shared bit for each guest TDs, add address conversion functions between private GPA and shared GPA and test if GPA is private. Because struct kvm_arch (or struct kvm which includes struct kvm_arch. See kvm_arch_alloc_vm() that passes __GPF_ZERO) is zero-cleared when allocated, the new member to remember GPA shared bit is guaranteed to be zero with this patch unless it's initialized explicitly. default or SEV-SNP TDX: S = (47 or 51) - 12 gfn_shared_mask 0 S bit kvm_is_private_gpa() always false true if GFN has S bit set kvm_gfn_to_shared() nop set S bit kvm_gfn_to_private() nop clear S bit fault.is_private means that host page should be gotten from guest_memfd is_private_gpa() means that KVM MMU should invoke private MMU hooks. Co-developed-by: Rick Edgecombe Signed-off-by: Rick Edgecombe Signed-off-by: Isaku Yamahata Reviewed-by: Binbin Wu --- v19: - Add comment on default vm case. - Added behavior table in the commit message - drop CONFIG_KVM_MMU_PRIVATE v18: - Added Reviewed-by Binbin Signed-off-by: Isaku Yamahata --- arch/x86/include/asm/kvm_host.h | 2 ++ arch/x86/kvm/mmu.h | 33 +++++++++++++++++++++++++++++++++ arch/x86/kvm/vmx/tdx.c | 5 +++++ 3 files changed, 40 insertions(+) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 5da3c211955d..de6dd42d226f 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1505,6 +1505,8 @@ struct kvm_arch { */ #define SPLIT_DESC_CACHE_MIN_NR_OBJECTS (SPTE_ENT_PER_PAGE + 1) struct kvm_mmu_memory_cache split_desc_cache; + + gfn_t gfn_shared_mask; }; struct kvm_vm_stat { diff --git a/arch/x86/kvm/mmu.h b/arch/x86/kvm/mmu.h index d96c93a25b3b..395b55684cb9 100644 --- a/arch/x86/kvm/mmu.h +++ b/arch/x86/kvm/mmu.h @@ -322,4 +322,37 @@ static inline gpa_t kvm_translate_gpa(struct kvm_vcpu *vcpu, return gpa; return translate_nested_gpa(vcpu, gpa, access, exception); } + +/* + * default or SEV-SNP TDX: where S = (47 or 51) - 12 + * gfn_shared_mask 0 S bit + * is_private_gpa() always false if GPA has S bit set + * gfn_to_shared() nop set S bit + * gfn_to_private() nop clear S bit + * + * fault.is_private means that host page should be gotten from guest_memfd + * is_private_gpa() means that KVM MMU should invoke private MMU hooks. + */ +static inline gfn_t kvm_gfn_shared_mask(const struct kvm *kvm) +{ + return kvm->arch.gfn_shared_mask; +} + +static inline gfn_t kvm_gfn_to_shared(const struct kvm *kvm, gfn_t gfn) +{ + return gfn | kvm_gfn_shared_mask(kvm); +} + +static inline gfn_t kvm_gfn_to_private(const struct kvm *kvm, gfn_t gfn) +{ + return gfn & ~kvm_gfn_shared_mask(kvm); +} + +static inline bool kvm_is_private_gpa(const struct kvm *kvm, gpa_t gpa) +{ + gfn_t mask = kvm_gfn_shared_mask(kvm); + + return mask && !(gpa_to_gfn(gpa) & mask); +} + #endif diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c index aa1da51b8af7..54e0d4efa2bd 100644 --- a/arch/x86/kvm/vmx/tdx.c +++ b/arch/x86/kvm/vmx/tdx.c @@ -906,6 +906,11 @@ static int tdx_td_init(struct kvm *kvm, struct kvm_tdx_cmd *cmd) kvm_tdx->attributes = td_params->attributes; kvm_tdx->xfam = td_params->xfam; + if (td_params->exec_controls & TDX_EXEC_CONTROL_MAX_GPAW) + kvm->arch.gfn_shared_mask = gpa_to_gfn(BIT_ULL(51)); + else + kvm->arch.gfn_shared_mask = gpa_to_gfn(BIT_ULL(47)); + out: /* kfree() accepts NULL. */ kfree(init_vm); From patchwork Mon Feb 26 08:25:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571475 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E0C2B1D53D; Mon, 26 Feb 2024 08:28:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.13 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936096; cv=none; b=IIYoGwXBaY9BfgN4yreWglM+6Fpqvd+McNA9UgY0G7nzDSR9iCy9Jld2HsOTUsbWl5J6ItTQQZHrWzoQyleCkV21hBSTYktnPDiMsp5Uech4DZd5L92qmVFkI/JudLpmZjnaoS2Bj03g2k5c9CwypotNAguqRdlecSDtDV9BF+s= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936096; c=relaxed/simple; bh=zdJ2GT4ARrSNbJp3F6eaxutpQ4DdrmPtIT6M7sepMRw=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=K0VPsCuiWdx6/pASSD9k7M0K7I10cjRrsA3sDVZLHIIbQx6cT03qNGRm6zlTlRFYdk+rvU7X7LH5i1GEJ8wwhokSSp/LSQs8WU3uDDRfZOPjWAd1vditshAQ3c6Qacxr1n7d9P87dw4K1EaryfQOEl90mu7ADFJF0s9vFkh2KbE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=lS6EKVrq; arc=none smtp.client-ip=192.198.163.13 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="lS6EKVrq" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936093; x=1740472093; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=zdJ2GT4ARrSNbJp3F6eaxutpQ4DdrmPtIT6M7sepMRw=; b=lS6EKVrqRND4ovKO0N4YszzzVAP4X5sTqJrq+AklmJKqrAcJT3US3mXy ImfPMwHIWPriS28wozlotMaHuY4l94wKvT3zXRCSApUuSPeGG3CU8N12o JMKoeTWKFfWokqVrIUTYD1WWV6U5Izepz7A5LCcUASDi88GR8xAv6+L9U OLrIGZE+opoJnzd3i6LFfJJm+3fpBThvw5PEtm4yvgaZSPFx26AyyGauk JHqsvdk138H/QnVDRi09Xp4IszzQPoKJQnllZeTGxsL/hHbGUsEkNVioX IXToMrvWdgXSnz3CSQL5+BGS9mumYGBTKYVOSKyBBwATMHPOb5S9SkpYX w==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="6155363" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6155363" Received: from orviesa009.jf.intel.com ([10.64.159.149]) by fmvoesa107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:08 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6615777" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa009-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:08 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 047/130] [MARKER] The start of TDX KVM patch series: KVM TDP refactoring for TDX Date: Mon, 26 Feb 2024 00:25:49 -0800 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata This empty commit is to mark the start of patch series of KVM TDP refactoring for TDX. Signed-off-by: Isaku Yamahata --- Documentation/virt/kvm/intel-tdx-layer-status.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/Documentation/virt/kvm/intel-tdx-layer-status.rst b/Documentation/virt/kvm/intel-tdx-layer-status.rst index 8b8186e7bfeb..e893a3d714c7 100644 --- a/Documentation/virt/kvm/intel-tdx-layer-status.rst +++ b/Documentation/virt/kvm/intel-tdx-layer-status.rst @@ -25,6 +25,6 @@ Patch Layer status * TD vcpu enter/exit: Not yet * TD vcpu interrupts/exit/hypercall: Not yet -* KVM MMU GPA shared bits: Applying -* KVM TDP refactoring for TDX: Not yet +* KVM MMU GPA shared bits: Applied +* KVM TDP refactoring for TDX: Applying * KVM TDP MMU hooks: Not yet From patchwork Mon Feb 26 08:25:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571477 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3B3A5208AD; Mon, 26 Feb 2024 08:28:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.13 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936096; cv=none; b=cOkcmMdvfKn7+diOmqEWPvhTZBvxAEAxVoO17o1KDn6IuktEo9mWkKFstQzB8DLadJJ5/K12p78tcOX/9GRwNVISXm2ffgd54wf9DidfVW4g5DI9IXWDqzAhdjcSssMVYl8Z7G1RCW+DgK4ONM7XXaxIQ1GUBKqoLELYDjjSH58= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936096; c=relaxed/simple; bh=X5tL+FTdFYIo/iu/ZRdr5tfelqekqbOSzf0LKCUZinM=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=R2/YdGBUkH50UA6m1BYvaF0JBHsLGEtQFNus/vYTrFBeoLDFe4GW51bEJEIVDpktxXgqAleIHM9z0KQ1FNWT42rBCc87VgBXYSsPrIbZx4RS2nwouevshCLAcwstLdziHCvBx6Ixr2g2FE3aMjOuJixoUYOkySB46AxLwEMvEe8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=HoRbKJ3G; arc=none smtp.client-ip=192.198.163.13 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="HoRbKJ3G" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936093; x=1740472093; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=X5tL+FTdFYIo/iu/ZRdr5tfelqekqbOSzf0LKCUZinM=; b=HoRbKJ3GiHQhNmG6aAF5deRzleDhO17EooWcUO7f5nZTrsa3E5IyWJBc a8xXDLRNI+Wxs/uMNrBBWZv+qDmJ9bbxPhh9fwRKzXybOvuHOucYESVhr RpJ4CRXqkjuh2fvqIod4s/pSCvkizcGJhMAUCts0C0rkdyrkhsQXYrNV6 sJA5wyvVLyNbp1fzJsnONkK/gdvyapbA3YGcaNWD2vdiU8V+2a1V65dnp LLj1tjOeSAVUctwCygyDLH/CfXvlz8ltuTZGj3nj8HHcNmpPERxPQebfa y+m7bxFbZhC/SEVmMAhTVo2rziMp9ucGdOD6tFT7nyt5jIk/TZkNWuyan g==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="6155368" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6155368" Received: from orviesa009.jf.intel.com ([10.64.159.149]) by fmvoesa107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:09 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6615787" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa009-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:09 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 048/130] KVM: Allow page-sized MMU caches to be initialized with custom 64-bit values Date: Mon, 26 Feb 2024 00:25:50 -0800 Message-Id: <9c392612eac4f3c489ad12dd4a4d505cf10d36dc.1708933498.git.isaku.yamahata@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Sean Christopherson Add support to MMU caches for initializing a page with a custom 64-bit value, e.g. to pre-fill an entire page table with non-zero PTE values. The functionality will be used by x86 to support Intel's TDX, which needs to set bit 63 in all non-present PTEs in order to prevent !PRESENT page faults from getting reflected into the guest (Intel's EPT Violation #VE architecture made the less than brilliant decision of having the per-PTE behavior be opt-out instead of opt-in). Signed-off-by: Sean Christopherson Signed-off-by: Isaku Yamahata --- include/linux/kvm_types.h | 1 + virt/kvm/kvm_main.c | 16 ++++++++++++++-- 2 files changed, 15 insertions(+), 2 deletions(-) diff --git a/include/linux/kvm_types.h b/include/linux/kvm_types.h index 9d1f7835d8c1..60c8d5c9eab9 100644 --- a/include/linux/kvm_types.h +++ b/include/linux/kvm_types.h @@ -94,6 +94,7 @@ struct gfn_to_pfn_cache { struct kvm_mmu_memory_cache { gfp_t gfp_zero; gfp_t gfp_custom; + u64 init_value; struct kmem_cache *kmem_cache; int capacity; int nobjs; diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index de38f308738e..d399009ef1d7 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -401,12 +401,17 @@ static void kvm_flush_shadow_all(struct kvm *kvm) static inline void *mmu_memory_cache_alloc_obj(struct kvm_mmu_memory_cache *mc, gfp_t gfp_flags) { + void *page; + gfp_flags |= mc->gfp_zero; if (mc->kmem_cache) return kmem_cache_alloc(mc->kmem_cache, gfp_flags); - else - return (void *)__get_free_page(gfp_flags); + + page = (void *)__get_free_page(gfp_flags); + if (page && mc->init_value) + memset64(page, mc->init_value, PAGE_SIZE / sizeof(mc->init_value)); + return page; } int __kvm_mmu_topup_memory_cache(struct kvm_mmu_memory_cache *mc, int capacity, int min) @@ -421,6 +426,13 @@ int __kvm_mmu_topup_memory_cache(struct kvm_mmu_memory_cache *mc, int capacity, if (WARN_ON_ONCE(!capacity)) return -EIO; + /* + * Custom init values can be used only for page allocations, + * and obviously conflict with __GFP_ZERO. + */ + if (WARN_ON_ONCE(mc->init_value && (mc->kmem_cache || mc->gfp_zero))) + return -EIO; + mc->objects = kvmalloc_array(sizeof(void *), capacity, gfp); if (!mc->objects) return -ENOMEM; From patchwork Mon Feb 26 08:25:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571476 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8006B3DBBF; Mon, 26 Feb 2024 08:28:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.13 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936096; cv=none; b=u2hvQu2aI6C4bxQt80aCs++PlsY1UTLXhXILz27ukot1pQKbQ6Zq4bgpu/MQaphTcPohlPzREOmQKw9sAbggSDSaasBq6thgyBIbzHiSeGtJeTok0o0fz8B0fZ3/EGOcJmo0xMhg4FG75asQq/ahc8L1R/r7ScbRpJA466m4/1Y= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936096; c=relaxed/simple; bh=wNBryeQQw95QhWpkNBLeGWBbrsE5OGf6Bj8TyqRgf1I=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=IveZqE4Ahm1CbmvOPRRrvFhcXynP5LVMUVfZRDxx+wBRH3K8KxOI+JudxC5DtmEQ/J3U4hNI27XeaURT2d9LbyHdGI9DopbXbkurnOXuBVm5KfAWlrwRWJA3Vae+DOBM5lUT225HEMgJlRp5lqcBFty4loeLLtSdHSS32Zl+U3E= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=Md8qd05y; arc=none smtp.client-ip=192.198.163.13 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="Md8qd05y" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936093; x=1740472093; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=wNBryeQQw95QhWpkNBLeGWBbrsE5OGf6Bj8TyqRgf1I=; b=Md8qd05yrHH3YnIQS64OX7aGFGxu7cVM3wPhUed0QIPZedzJcflWN4m+ f51buCWfspViByMjHNkgFiqQHplZ0TtJb9trFgbYnsIixf/4A7JHyCoxq WmKN+eeiJR6V3Zhp3bhf5FLlKjX6r3fesnTRYX96tYTAwysQ9gQCc/2r4 9sBy4cO/IqTO5tTD0afLI9u+EFmtjhekBobddlRMM9zRIzKOaY4DFlaaz lQ+mrRyqRQ8g68FEqdogGwBX3H2jBbLBm3N8Uj316gXk9RAoRXMy8rHPV ES6WO0Opr737CDLllymxDCejD0pbUkmYel5w3BVm3xToXkTV51iYoa94X A==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="6155373" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6155373" Received: from orviesa009.jf.intel.com ([10.64.159.149]) by fmvoesa107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:10 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6615795" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa009-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:09 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com, Sean Christopherson Subject: [PATCH v19 049/130] KVM: x86/mmu: Replace hardcoded value 0 for the initial value for SPTE Date: Mon, 26 Feb 2024 00:25:51 -0800 Message-Id: <69fc71c7fbb5f5abfb6d2948d56bb6fa85ebd7d5.1708933498.git.isaku.yamahata@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata The TDX support will need the "suppress #VE" bit (bit 63) set as the initial value for SPTE. To reduce code change size, introduce a new macro SHADOW_NONPRESENT_VALUE for the initial value for the shadow page table entry (SPTE) and replace hard-coded value 0 for it. Initialize shadow page tables with their value. The plan is to unconditionally set the "suppress #VE" bit for both AMD and Intel as: 1) AMD hardware uses the bit 63 as NX for present SPTE and ignored for non-present SPTE; 2) for conventional VMX guests, KVM never enables the "EPT-violation #VE" in VMCS control and "suppress #VE" bit is ignored by hardware. Signed-off-by: Sean Christopherson Signed-off-by: Isaku Yamahata --- arch/x86/kvm/mmu/mmu.c | 20 +++++++++++++++----- arch/x86/kvm/mmu/paging_tmpl.h | 2 +- arch/x86/kvm/mmu/spte.h | 2 ++ arch/x86/kvm/mmu/tdp_mmu.c | 14 +++++++------- 4 files changed, 25 insertions(+), 13 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 2becc86c71b2..211c0e72f45d 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -567,9 +567,9 @@ static u64 mmu_spte_clear_track_bits(struct kvm *kvm, u64 *sptep) if (!is_shadow_present_pte(old_spte) || !spte_has_volatile_bits(old_spte)) - __update_clear_spte_fast(sptep, 0ull); + __update_clear_spte_fast(sptep, SHADOW_NONPRESENT_VALUE); else - old_spte = __update_clear_spte_slow(sptep, 0ull); + old_spte = __update_clear_spte_slow(sptep, SHADOW_NONPRESENT_VALUE); if (!is_shadow_present_pte(old_spte)) return old_spte; @@ -603,7 +603,7 @@ static u64 mmu_spte_clear_track_bits(struct kvm *kvm, u64 *sptep) */ static void mmu_spte_clear_no_track(u64 *sptep) { - __update_clear_spte_fast(sptep, 0ull); + __update_clear_spte_fast(sptep, SHADOW_NONPRESENT_VALUE); } static u64 mmu_spte_get_lockless(u64 *sptep) @@ -1950,7 +1950,8 @@ static bool kvm_sync_page_check(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp) static int kvm_sync_spte(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp, int i) { - if (!sp->spt[i]) + /* sp->spt[i] has initial value of shadow page table allocation */ + if (sp->spt[i] == SHADOW_NONPRESENT_VALUE) return 0; return vcpu->arch.mmu->sync_spte(vcpu, sp, i); @@ -6204,7 +6205,16 @@ int kvm_mmu_create(struct kvm_vcpu *vcpu) vcpu->arch.mmu_page_header_cache.kmem_cache = mmu_page_header_cache; vcpu->arch.mmu_page_header_cache.gfp_zero = __GFP_ZERO; - vcpu->arch.mmu_shadow_page_cache.gfp_zero = __GFP_ZERO; + /* + * When X86_64, initial SEPT entries are initialized with + * SHADOW_NONPRESENT_VALUE. Otherwise zeroed. See + * mmu_memory_cache_alloc_obj(). + */ + if (IS_ENABLED(CONFIG_X86_64)) + vcpu->arch.mmu_shadow_page_cache.init_value = + SHADOW_NONPRESENT_VALUE; + if (!vcpu->arch.mmu_shadow_page_cache.init_value) + vcpu->arch.mmu_shadow_page_cache.gfp_zero = __GFP_ZERO; vcpu->arch.mmu = &vcpu->arch.root_mmu; vcpu->arch.walk_mmu = &vcpu->arch.root_mmu; diff --git a/arch/x86/kvm/mmu/paging_tmpl.h b/arch/x86/kvm/mmu/paging_tmpl.h index 4d4e98fe4f35..bebd73cd61bb 100644 --- a/arch/x86/kvm/mmu/paging_tmpl.h +++ b/arch/x86/kvm/mmu/paging_tmpl.h @@ -911,7 +911,7 @@ static int FNAME(sync_spte)(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp, int gpa_t pte_gpa; gfn_t gfn; - if (WARN_ON_ONCE(!sp->spt[i])) + if (WARN_ON_ONCE(sp->spt[i] == SHADOW_NONPRESENT_VALUE)) return 0; first_pte_gpa = FNAME(get_level1_sp_gpa)(sp); diff --git a/arch/x86/kvm/mmu/spte.h b/arch/x86/kvm/mmu/spte.h index a129951c9a88..4d1799ba2bf8 100644 --- a/arch/x86/kvm/mmu/spte.h +++ b/arch/x86/kvm/mmu/spte.h @@ -149,6 +149,8 @@ static_assert(MMIO_SPTE_GEN_LOW_BITS == 8 && MMIO_SPTE_GEN_HIGH_BITS == 11); #define MMIO_SPTE_GEN_MASK GENMASK_ULL(MMIO_SPTE_GEN_LOW_BITS + MMIO_SPTE_GEN_HIGH_BITS - 1, 0) +#define SHADOW_NONPRESENT_VALUE 0ULL + extern u64 __read_mostly shadow_host_writable_mask; extern u64 __read_mostly shadow_mmu_writable_mask; extern u64 __read_mostly shadow_nx_mask; diff --git a/arch/x86/kvm/mmu/tdp_mmu.c b/arch/x86/kvm/mmu/tdp_mmu.c index 6ae19b4ee5b1..bdeb23ff9e71 100644 --- a/arch/x86/kvm/mmu/tdp_mmu.c +++ b/arch/x86/kvm/mmu/tdp_mmu.c @@ -570,7 +570,7 @@ static inline int tdp_mmu_zap_spte_atomic(struct kvm *kvm, * here since the SPTE is going from non-present to non-present. Use * the raw write helper to avoid an unnecessary check on volatile bits. */ - __kvm_tdp_mmu_write_spte(iter->sptep, 0); + __kvm_tdp_mmu_write_spte(iter->sptep, SHADOW_NONPRESENT_VALUE); return 0; } @@ -707,8 +707,8 @@ static void __tdp_mmu_zap_root(struct kvm *kvm, struct kvm_mmu_page *root, continue; if (!shared) - tdp_mmu_iter_set_spte(kvm, &iter, 0); - else if (tdp_mmu_set_spte_atomic(kvm, &iter, 0)) + tdp_mmu_iter_set_spte(kvm, &iter, SHADOW_NONPRESENT_VALUE); + else if (tdp_mmu_set_spte_atomic(kvm, &iter, SHADOW_NONPRESENT_VALUE)) goto retry; } } @@ -764,8 +764,8 @@ bool kvm_tdp_mmu_zap_sp(struct kvm *kvm, struct kvm_mmu_page *sp) if (WARN_ON_ONCE(!is_shadow_present_pte(old_spte))) return false; - tdp_mmu_set_spte(kvm, kvm_mmu_page_as_id(sp), sp->ptep, old_spte, 0, - sp->gfn, sp->role.level + 1); + tdp_mmu_set_spte(kvm, kvm_mmu_page_as_id(sp), sp->ptep, old_spte, + SHADOW_NONPRESENT_VALUE, sp->gfn, sp->role.level + 1); return true; } @@ -799,7 +799,7 @@ static bool tdp_mmu_zap_leafs(struct kvm *kvm, struct kvm_mmu_page *root, !is_last_spte(iter.old_spte, iter.level)) continue; - tdp_mmu_iter_set_spte(kvm, &iter, 0); + tdp_mmu_iter_set_spte(kvm, &iter, SHADOW_NONPRESENT_VALUE); flush = true; } @@ -1226,7 +1226,7 @@ static bool set_spte_gfn(struct kvm *kvm, struct tdp_iter *iter, * invariant that the PFN of a present * leaf SPTE can never change. * See handle_changed_spte(). */ - tdp_mmu_iter_set_spte(kvm, iter, 0); + tdp_mmu_iter_set_spte(kvm, iter, SHADOW_NONPRESENT_VALUE); if (!pte_write(range->arg.pte)) { new_spte = kvm_mmu_changed_pte_notifier_make_spte(iter->old_spte, From patchwork Mon Feb 26 08:25:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571478 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 247BC5F848; Mon, 26 Feb 2024 08:28:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.13 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936098; cv=none; b=usiRSGURtuOW3xpxprppGNarNfIp7aH2WQKB2xtFdXCxRaP5mH3EW1sR0QnU9ewAaN0kq3DySczn0loQ65e9s5reWwxu8bbQYb/QiMYLCuazCtvo0dlTAsoIgkF9+fsxGn59xRucNtpGj3yqBPtDN2TidCSML8FOH03/sq7PicU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936098; c=relaxed/simple; bh=ZLlfjMU1TFlONfmUGruGN81Gisxxqe67EafNn/XTWPg=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=YuneVuffTYYLbw7rsr9xm1y0+sCwcUF5nP66leJ5mu/GQDPq7p5WFdq+19m2TG+DNs8Gn54p9pZlNGYvC4j/EBg2KKV3Sk5s6T7b3OBgxtw/WWcmaqwdO7ZH7rHarhmSA3YlNUkeVwOD2GMu/Ml7619p5hRO4nQZiJfL3o84XJ4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=XhTRymRZ; arc=none smtp.client-ip=192.198.163.13 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="XhTRymRZ" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936096; x=1740472096; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=ZLlfjMU1TFlONfmUGruGN81Gisxxqe67EafNn/XTWPg=; b=XhTRymRZwWxAuU5gF4S3s7EB13n59eeAt6SUFajNzrxvOEK/+ECkOfTu c6gpw45eaJev/YJMD2GoZ75MZ1mj0zHyT4Nq//DaKfM/2rp8lWxdv5mPL QkhEf3KIu2ZTzLIYRMm6FUi+k3N1qFYlWEXCaCbSOH56PsWqfkjA0m4X0 +5hbJ3sdI+/stx2IUHOjFGDEYOmC1eDupEB4f9JjNHJRy0TZn9HHCTKhe PHL2oPBkfnLT34wUwoSiAKvM1jSI365j4CTNf2WqFVIzTOBBpZBUSmuFi 7D9oqgzPo/JPaEByBupZIqouIQ9q7olQzpeY4b/c3aI/4U02P343LjAY9 Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="6155378" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6155378" Received: from orviesa009.jf.intel.com ([10.64.159.149]) by fmvoesa107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:11 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6615804" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa009-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:11 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com, Sean Christopherson , Binbin Wu Subject: [PATCH v19 050/130] KVM: x86/mmu: Allow non-zero value for non-present SPTE and removed SPTE Date: Mon, 26 Feb 2024 00:25:52 -0800 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Sean Christopherson For TD guest, the current way to emulate MMIO doesn't work any more, as KVM is not able to access the private memory of TD guest and do the emulation. Instead, TD guest expects to receive #VE when it accesses the MMIO and then it can explicitly make hypercall to KVM to get the expected information. To achieve this, the TDX module always enables "EPT-violation #VE" in the VMCS control. And accordingly, for the MMIO spte for the shared GPA, 1. KVM needs to set "suppress #VE" bit for the non-present SPTE so that EPT violation happens on TD accessing MMIO range. 2. On EPT violation, KVM sets the MMIO spte to clear "suppress #VE" bit so the TD guest can receive the #VE instead of EPT misconfiguration unlike VMX case. For the shared GPA that is not populated yet, EPT violation need to be triggered when TD guest accesses such shared GPA. The non-present SPTE value for shared GPA should set "suppress #VE" bit. Add "suppress #VE" bit (bit 63) to SHADOW_NONPRESENT_VALUE and REMOVED_SPTE. Unconditionally set the "suppress #VE" bit (which is bit 63) for both AMD and Intel as: 1) AMD hardware doesn't use this bit when present bit is off; 2) for normal VMX guest, KVM never enables the "EPT-violation #VE" in VMCS control and "suppress #VE" bit is ignored by hardware. Signed-off-by: Sean Christopherson Signed-off-by: Isaku Yamahata Reviewed-by: Binbin Wu --- v19: - fix typo in the commit message Signed-off-by: Isaku Yamahata --- arch/x86/kvm/mmu/spte.h | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/arch/x86/kvm/mmu/spte.h b/arch/x86/kvm/mmu/spte.h index 4d1799ba2bf8..26bc95bbc962 100644 --- a/arch/x86/kvm/mmu/spte.h +++ b/arch/x86/kvm/mmu/spte.h @@ -149,7 +149,20 @@ static_assert(MMIO_SPTE_GEN_LOW_BITS == 8 && MMIO_SPTE_GEN_HIGH_BITS == 11); #define MMIO_SPTE_GEN_MASK GENMASK_ULL(MMIO_SPTE_GEN_LOW_BITS + MMIO_SPTE_GEN_HIGH_BITS - 1, 0) +/* + * Non-present SPTE value for both VMX and SVM for TDP MMU. + * For SVM NPT, for non-present spte (bit 0 = 0), other bits are ignored. + * For VMX EPT, bit 63 is ignored if #VE is disabled. (EPT_VIOLATION_VE=0) + * bit 63 is #VE suppress if #VE is enabled. (EPT_VIOLATION_VE=1) + * For TDX: + * TDX module sets EPT_VIOLATION_VE for Secure-EPT and conventional EPT + */ +#ifdef CONFIG_X86_64 +#define SHADOW_NONPRESENT_VALUE BIT_ULL(63) +static_assert(!(SHADOW_NONPRESENT_VALUE & SPTE_MMU_PRESENT_MASK)); +#else #define SHADOW_NONPRESENT_VALUE 0ULL +#endif extern u64 __read_mostly shadow_host_writable_mask; extern u64 __read_mostly shadow_mmu_writable_mask; @@ -196,7 +209,7 @@ extern u64 __read_mostly shadow_nonpresent_or_rsvd_mask; * * Only used by the TDP MMU. */ -#define REMOVED_SPTE 0x5a0ULL +#define REMOVED_SPTE (SHADOW_NONPRESENT_VALUE | 0x5a0ULL) /* Removed SPTEs must not be misconstrued as shadow present PTEs. */ static_assert(!(REMOVED_SPTE & SPTE_MMU_PRESENT_MASK)); From patchwork Mon Feb 26 08:25:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571479 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 35B665F853; Mon, 26 Feb 2024 08:28:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.13 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936098; cv=none; b=qWXeDc0bX5k7xghLPNS/7Ve2TKI2TWspeanuKomPC3PXnQw6W7u01ZwkmTdbO3e97yMhFLeauECIkkAiRKgSF5kVtFTRaOC98mfsJAUmOo+ssY+YDvw+QHNs7VANamyLOP24KsX15zOEu1IlYNAB9ArJh17HOniNALPXhoTprMk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936098; c=relaxed/simple; bh=/QQnC6NuR+jveGFHm7lY9Jr4kJQvgq3Qc9nLEe3cDso=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=IdI3TIPqChpP/iokltpRsSNrb2NWaZis22YZlkr+bFaSw4jrStJsvtAqREh99vXI4m7/qJ8T4R9RRryh7Kh445bAE/qjm78BahwsOiRhJEqfrRkFcjremKFUMZWDhYtxbXwug62fOqiBVALyVXzTRJ3l8M8U0Bh0XeqMp9ooBTI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=Lb72AzBs; arc=none smtp.client-ip=192.198.163.13 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="Lb72AzBs" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936096; x=1740472096; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=/QQnC6NuR+jveGFHm7lY9Jr4kJQvgq3Qc9nLEe3cDso=; b=Lb72AzBs6KSkDvVgrO7QyqpivCvt1HSLoGV3o6ud0CDDhv7GDaK9EgRx 4OoAvEJkGs0zf/sJnXZm8HWYm1d1HjijIvITNN+FnM5f5GhHvvTlUkWRL r/eNTG4yPqPfj3zZZZHqbvZAGOHJW+B8G/4TfffA2MdMIvaCZhrcq2TpB XR6BBfgY9ti2cqYMXXRs5wHJU9OHmTUt9oCWH11IyzMUFFZt4Ozuct4Lb 5SSzxjHhmLYtvVbO1XKNzgQm8hmR7MMLgrJymcwkNi14ZJ6A1ynEC0vwk WS75WXtfcy5RQlYRpqe8y47JjayMDIbrX8iEaabVhZB2P5JBWMmNLHHe7 g==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="6155382" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6155382" Received: from orviesa009.jf.intel.com ([10.64.159.149]) by fmvoesa107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:11 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6615809" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa009-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:11 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 051/130] KVM: x86/mmu: Add Suppress VE bit to shadow_mmio_mask/shadow_present_mask Date: Mon, 26 Feb 2024 00:25:53 -0800 Message-Id: <29a80b4fef28dad24e6366f2f707fec5266875da.1708933498.git.isaku.yamahata@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata To make use of the same value of shadow_mmio_mask and shadow_present_mask for TDX and VMX, add Suppress-VE bit to shadow_mmio_mask and shadow_present_mask so that they can be common for both VMX and TDX. TDX will require shadow_mmio_mask and shadow_present_mask to include VMX_SUPPRESS_VE for shared GPA so that EPT violation is triggered for shared GPA. For VMX, VMX_SUPPRESS_VE doesn't matter for MMIO because the spte value is required to cause EPT misconfig. the additional bit doesn't affect VMX logic to add the bit to shadow_mmio_{value, mask}. Signed-off-by: Isaku Yamahata --- arch/x86/include/asm/vmx.h | 1 + arch/x86/kvm/mmu/spte.c | 6 ++++-- 2 files changed, 5 insertions(+), 2 deletions(-) diff --git a/arch/x86/include/asm/vmx.h b/arch/x86/include/asm/vmx.h index 0e73616b82f3..76ed39541a52 100644 --- a/arch/x86/include/asm/vmx.h +++ b/arch/x86/include/asm/vmx.h @@ -513,6 +513,7 @@ enum vmcs_field { #define VMX_EPT_IPAT_BIT (1ull << 6) #define VMX_EPT_ACCESS_BIT (1ull << 8) #define VMX_EPT_DIRTY_BIT (1ull << 9) +#define VMX_EPT_SUPPRESS_VE_BIT (1ull << 63) #define VMX_EPT_RWX_MASK (VMX_EPT_READABLE_MASK | \ VMX_EPT_WRITABLE_MASK | \ VMX_EPT_EXECUTABLE_MASK) diff --git a/arch/x86/kvm/mmu/spte.c b/arch/x86/kvm/mmu/spte.c index 4a599130e9c9..02a466de2991 100644 --- a/arch/x86/kvm/mmu/spte.c +++ b/arch/x86/kvm/mmu/spte.c @@ -429,7 +429,9 @@ void kvm_mmu_set_ept_masks(bool has_ad_bits, bool has_exec_only) shadow_dirty_mask = has_ad_bits ? VMX_EPT_DIRTY_BIT : 0ull; shadow_nx_mask = 0ull; shadow_x_mask = VMX_EPT_EXECUTABLE_MASK; - shadow_present_mask = has_exec_only ? 0ull : VMX_EPT_READABLE_MASK; + /* VMX_EPT_SUPPRESS_VE_BIT is needed for W or X violation. */ + shadow_present_mask = + (has_exec_only ? 0ull : VMX_EPT_READABLE_MASK) | VMX_EPT_SUPPRESS_VE_BIT; /* * EPT overrides the host MTRRs, and so KVM must program the desired * memtype directly into the SPTEs. Note, this mask is just the mask @@ -446,7 +448,7 @@ void kvm_mmu_set_ept_masks(bool has_ad_bits, bool has_exec_only) * of an EPT paging-structure entry is 110b (write/execute). */ kvm_mmu_set_mmio_spte_mask(VMX_EPT_MISCONFIG_WX_VALUE, - VMX_EPT_RWX_MASK, 0); + VMX_EPT_RWX_MASK | VMX_EPT_SUPPRESS_VE_BIT, 0); } EXPORT_SYMBOL_GPL(kvm_mmu_set_ept_masks); From patchwork Mon Feb 26 08:25:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571480 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DE7115F87D; Mon, 26 Feb 2024 08:28:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.13 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936099; cv=none; b=pKU/s1rt26Ek165HTYLqGqoYHKru397MA+Xey6Rp7vVEGqlrwiSjbClfs9/HVJWMEs6tl9mK0WfcAH8tZJMrhTE1QyRwXC+N7vETp3zLHzCNbkgBbs17IJVpcV0i4L4bzQ7ReEMuQOwLoGtdVuqWkcjDVToIXjoNmJaMdLkXWAw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936099; c=relaxed/simple; bh=3JKoRj7pW/Us//1MU97aOKv1aywjwzLMmgcEekrtcoM=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Ir5XgQUbhiqlmcCU0yIoKkqrl+ms2Yh8ibKOblm8DzIC48eKRAkuoJNyHTwjeVCQ2Ou8AreIq5rvC2DNT3nGhBR9recvKJD6cht427c5Uk3XZGEz2wlsMnIDRc1gd7YUShmJgdI/1qeX9NoB7BXrWwqTc9940rxk7DMyygZP4d4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=BP9gW6LP; arc=none smtp.client-ip=192.198.163.13 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="BP9gW6LP" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936097; x=1740472097; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=3JKoRj7pW/Us//1MU97aOKv1aywjwzLMmgcEekrtcoM=; b=BP9gW6LPt+5yl64t+vseQKD83FoRXkbMCUhigS4ZqB1F5Gc5WJMZ/C+Z X2kDG3f33vyCl3+jktdbjqJQNnG5l8bEQESC+kjuNIaxUERYB8q7uPy27 hYse/Zpc+2UPigw403LP05iJlyFDUp8P1L0t4tEmUYFhCxR0q+ABahwz7 yX9cX3FWAsROLZHa9MaBJSR+EAZc7sXOaFjHd3sGnkGsnfgsg/5OjGBjC JCpsmUcgjK5JWm1EPmT02FKBRE7HFTqffHk5U/ZZ3JVmLshFJx3+G80ue l0EWhKAGb6+x/zBeFi1N5U9WOZEZQJJ6854at24O4afVYxOMrT0Gksvdd w==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="6155391" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6155391" Received: from orviesa009.jf.intel.com ([10.64.159.149]) by fmvoesa107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:12 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6615818" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa009-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:12 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com, Sean Christopherson Subject: [PATCH v19 052/130] KVM: x86/mmu: Track shadow MMIO value on a per-VM basis Date: Mon, 26 Feb 2024 00:25:54 -0800 Message-Id: <34d7a0c8724f4fce4da50fe3028373c31213aa8a.1708933498.git.isaku.yamahata@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata TDX will use a different shadow PTE entry value for MMIO from VMX. Add members to kvm_arch and track value for MMIO per-VM instead of global variables. By using the per-VM EPT entry value for MMIO, the existing VMX logic is kept working. Introduce a separate setter function so that guest TD can override later. Also require mmio spte caching for TDX. Actually this is true case because TDX requires EPT and KVM EPT allows mmio spte caching. Signed-off-by: Sean Christopherson Signed-off-by: Isaku Yamahata --- v19: - fix typo in the commit message. --- arch/x86/include/asm/kvm_host.h | 2 ++ arch/x86/kvm/mmu.h | 1 + arch/x86/kvm/mmu/mmu.c | 8 +++++--- arch/x86/kvm/mmu/spte.c | 10 ++++++++-- arch/x86/kvm/mmu/spte.h | 4 ++-- arch/x86/kvm/mmu/tdp_mmu.c | 6 +++--- 6 files changed, 21 insertions(+), 10 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index de6dd42d226f..6c10d8d1017f 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1312,6 +1312,8 @@ struct kvm_arch { */ spinlock_t mmu_unsync_pages_lock; + u64 shadow_mmio_value; + struct iommu_domain *iommu_domain; bool iommu_noncoherent; #define __KVM_HAVE_ARCH_NONCOHERENT_DMA diff --git a/arch/x86/kvm/mmu.h b/arch/x86/kvm/mmu.h index 395b55684cb9..ab2854f337ab 100644 --- a/arch/x86/kvm/mmu.h +++ b/arch/x86/kvm/mmu.h @@ -101,6 +101,7 @@ static inline u8 kvm_get_shadow_phys_bits(void) } void kvm_mmu_set_mmio_spte_mask(u64 mmio_value, u64 mmio_mask, u64 access_mask); +void kvm_mmu_set_mmio_spte_value(struct kvm *kvm, u64 mmio_value); void kvm_mmu_set_me_spte_mask(u64 me_value, u64 me_mask); void kvm_mmu_set_ept_masks(bool has_ad_bits, bool has_exec_only); diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 211c0e72f45d..84e7a289ad07 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -2515,7 +2515,7 @@ static int mmu_page_zap_pte(struct kvm *kvm, struct kvm_mmu_page *sp, return kvm_mmu_prepare_zap_page(kvm, child, invalid_list); } - } else if (is_mmio_spte(pte)) { + } else if (is_mmio_spte(kvm, pte)) { mmu_spte_clear_no_track(spte); } return 0; @@ -4184,7 +4184,7 @@ static int handle_mmio_page_fault(struct kvm_vcpu *vcpu, u64 addr, bool direct) if (WARN_ON_ONCE(reserved)) return -EINVAL; - if (is_mmio_spte(spte)) { + if (is_mmio_spte(vcpu->kvm, spte)) { gfn_t gfn = get_mmio_spte_gfn(spte); unsigned int access = get_mmio_spte_access(spte); @@ -4837,7 +4837,7 @@ EXPORT_SYMBOL_GPL(kvm_mmu_new_pgd); static bool sync_mmio_spte(struct kvm_vcpu *vcpu, u64 *sptep, gfn_t gfn, unsigned int access) { - if (unlikely(is_mmio_spte(*sptep))) { + if (unlikely(is_mmio_spte(vcpu->kvm, *sptep))) { if (gfn != get_mmio_spte_gfn(*sptep)) { mmu_spte_clear_no_track(sptep); return true; @@ -6357,6 +6357,8 @@ static bool kvm_has_zapped_obsolete_pages(struct kvm *kvm) void kvm_mmu_init_vm(struct kvm *kvm) { + + kvm->arch.shadow_mmio_value = shadow_mmio_value; INIT_LIST_HEAD(&kvm->arch.active_mmu_pages); INIT_LIST_HEAD(&kvm->arch.zapped_obsolete_pages); INIT_LIST_HEAD(&kvm->arch.possible_nx_huge_pages); diff --git a/arch/x86/kvm/mmu/spte.c b/arch/x86/kvm/mmu/spte.c index 02a466de2991..318135daf685 100644 --- a/arch/x86/kvm/mmu/spte.c +++ b/arch/x86/kvm/mmu/spte.c @@ -74,10 +74,10 @@ u64 make_mmio_spte(struct kvm_vcpu *vcpu, u64 gfn, unsigned int access) u64 spte = generation_mmio_spte_mask(gen); u64 gpa = gfn << PAGE_SHIFT; - WARN_ON_ONCE(!shadow_mmio_value); + WARN_ON_ONCE(!vcpu->kvm->arch.shadow_mmio_value); access &= shadow_mmio_access_mask; - spte |= shadow_mmio_value | access; + spte |= vcpu->kvm->arch.shadow_mmio_value | access; spte |= gpa | shadow_nonpresent_or_rsvd_mask; spte |= (gpa & shadow_nonpresent_or_rsvd_mask) << SHADOW_NONPRESENT_OR_RSVD_MASK_LEN; @@ -411,6 +411,12 @@ void kvm_mmu_set_mmio_spte_mask(u64 mmio_value, u64 mmio_mask, u64 access_mask) } EXPORT_SYMBOL_GPL(kvm_mmu_set_mmio_spte_mask); +void kvm_mmu_set_mmio_spte_value(struct kvm *kvm, u64 mmio_value) +{ + kvm->arch.shadow_mmio_value = mmio_value; +} +EXPORT_SYMBOL_GPL(kvm_mmu_set_mmio_spte_value); + void kvm_mmu_set_me_spte_mask(u64 me_value, u64 me_mask) { /* shadow_me_value must be a subset of shadow_me_mask */ diff --git a/arch/x86/kvm/mmu/spte.h b/arch/x86/kvm/mmu/spte.h index 26bc95bbc962..1a163aee9ec6 100644 --- a/arch/x86/kvm/mmu/spte.h +++ b/arch/x86/kvm/mmu/spte.h @@ -264,9 +264,9 @@ static inline struct kvm_mmu_page *root_to_sp(hpa_t root) return spte_to_child_sp(root); } -static inline bool is_mmio_spte(u64 spte) +static inline bool is_mmio_spte(struct kvm *kvm, u64 spte) { - return (spte & shadow_mmio_mask) == shadow_mmio_value && + return (spte & shadow_mmio_mask) == kvm->arch.shadow_mmio_value && likely(enable_mmio_caching); } diff --git a/arch/x86/kvm/mmu/tdp_mmu.c b/arch/x86/kvm/mmu/tdp_mmu.c index bdeb23ff9e71..04c6af49c3e8 100644 --- a/arch/x86/kvm/mmu/tdp_mmu.c +++ b/arch/x86/kvm/mmu/tdp_mmu.c @@ -462,8 +462,8 @@ static void handle_changed_spte(struct kvm *kvm, int as_id, gfn_t gfn, * impact the guest since both the former and current SPTEs * are nonpresent. */ - if (WARN_ON_ONCE(!is_mmio_spte(old_spte) && - !is_mmio_spte(new_spte) && + if (WARN_ON_ONCE(!is_mmio_spte(kvm, old_spte) && + !is_mmio_spte(kvm, new_spte) && !is_removed_spte(new_spte))) pr_err("Unexpected SPTE change! Nonpresent SPTEs\n" "should not be replaced with another,\n" @@ -978,7 +978,7 @@ static int tdp_mmu_map_handle_target_level(struct kvm_vcpu *vcpu, } /* If a MMIO SPTE is installed, the MMIO will need to be emulated. */ - if (unlikely(is_mmio_spte(new_spte))) { + if (unlikely(is_mmio_spte(vcpu->kvm, new_spte))) { vcpu->stat.pf_mmio_spte_created++; trace_mark_mmio_spte(rcu_dereference(iter->sptep), iter->gfn, new_spte); From patchwork Mon Feb 26 08:25:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571481 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 737115FDAF; Mon, 26 Feb 2024 08:28:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.13 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936100; cv=none; b=MGDrCyWrqjud1rRcP3nDMjpJKnxQik0eCvRnMsjPEG7J6rXfKKWWgWcn7RVwgek6LiqQ2EFr0w2iR1M3KhNTRmg5lfmQKB+uLgmukTTqgLK2iPrl/FMatH+O/HEAKXnPlK1izs7iT8oW8V3oLrjCD+HA9REkM/5ttnLu+fsbENY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936100; c=relaxed/simple; bh=F2Aq5/aMdhbNlEsFZSbUWHxeFsIV+hsLMCdd356a0sA=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=QAdx+35Sfilj71eBECALqBfp8HpdLqbwr+YkDtg9dpWgInCi/80AYz6k/ByM673FEXm6u+ypptwRXlcTgivNaTtqAZTqSwenihd4EMIaddT7aRPFNfl7uCc2wN6VUL/DiA0pxUL8uX24vRdsj5s92BJHsJnYnOnXHMYlTeLxkZs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=ic9reTHp; arc=none smtp.client-ip=192.198.163.13 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="ic9reTHp" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936098; x=1740472098; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=F2Aq5/aMdhbNlEsFZSbUWHxeFsIV+hsLMCdd356a0sA=; b=ic9reTHpRInzMtCmwML4COq2IRqv2HnGLk5075regJrHirutjdcYEtpp MCe5FBJVRF4In/47/BX6i4+Xq/uK4RTe28FTjBOa4IQxfYRQp8jRsKHiS MKbnurSw0AfC/MAR46rtO3vqNXKtfw0KgVURkZWGqUK8zcGjpxB7Rw0Kt gkoqCBZwBf+E4/gBv31/ENKAz+JStgLfK21VjiT+bUnyKgk/2UgEsAYwK pCcL2ubYgqvkHbFXH0+77ty0yo5Dw2TWkA/B7rvE61leJ58k96XFtWA15 r5h84fsrq4siR97hHcEzZ4qhhS1St203OerndOUQGHRsQVlRol2UO4zCR w==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="6155396" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6155396" Received: from orviesa009.jf.intel.com ([10.64.159.149]) by fmvoesa107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:13 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6615830" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa009-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:13 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com, Binbin Wu Subject: [PATCH v19 053/130] KVM: x86/mmu: Disallow fast page fault on private GPA Date: Mon, 26 Feb 2024 00:25:55 -0800 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata TDX requires TDX SEAMCALL to operate Secure EPT instead of direct memory access and TDX SEAMCALL is heavy operation. Fast page fault on private GPA doesn't make sense. Disallow fast page fault on private GPA. Signed-off-by: Isaku Yamahata Reviewed-by: Paolo Bonzini Reviewed-by: Binbin Wu --- v19: - updated comment to mention VM type other than TDX. --- arch/x86/kvm/mmu/mmu.c | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 84e7a289ad07..eeebbc67e42b 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -3339,8 +3339,18 @@ static int kvm_handle_noslot_fault(struct kvm_vcpu *vcpu, return RET_PF_CONTINUE; } -static bool page_fault_can_be_fast(struct kvm_page_fault *fault) +static bool page_fault_can_be_fast(struct kvm *kvm, struct kvm_page_fault *fault) { + /* + * TDX private mapping doesn't support fast page fault because the EPT + * entry is read/written with TDX SEAMCALLs instead of direct memory + * access. + * For other VM type, kvm_is_private_gpa() is always false because + * gfn_shared_mask is zero. + */ + if (kvm_is_private_gpa(kvm, fault->addr)) + return false; + /* * Page faults with reserved bits set, i.e. faults on MMIO SPTEs, only * reach the common page fault handler if the SPTE has an invalid MMIO @@ -3450,7 +3460,7 @@ static int fast_page_fault(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault) u64 *sptep; uint retry_count = 0; - if (!page_fault_can_be_fast(fault)) + if (!page_fault_can_be_fast(vcpu->kvm, fault)) return ret; walk_shadow_page_lockless_begin(vcpu); From patchwork Mon Feb 26 08:25:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571482 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8E5795FDC0; Mon, 26 Feb 2024 08:28:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.13 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936100; cv=none; b=QoxhJDjrasOzNsDb2krqPytU7vF/G62c2lRAXDqMi2/MQbmQpSmmIX3spWfsE18tE8upxAsV3Z3+hl5uPP/lCzWOuRzyjydABXUby8Talic80nJf1T+zQ9NtVzZ6V45qq25RWZlWaYbxRyRq5V5Q0Dy1fyWFSjiv9ntrG0n/faA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936100; c=relaxed/simple; bh=buJPokmaPQyeldNGCqDdcI5fhmwGvphJPlHxpErBVWU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=D7tWll/sQV898fAWQg3+FBpofZD8gIfJFo2BZOoUKPrUXVykCp5HIECs/pFvXL/n1s4YFFdxt1bkwFMfAXgRM5U3LoV4YeRJbXFYrMCqJQ+TDDC8RqGnskIDHeMpylVtouh7P53BL/QkPe3s3Fj4TzLXcV3WIcuz/CCOAcFWONY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=dQAexgko; arc=none smtp.client-ip=192.198.163.13 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="dQAexgko" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936098; x=1740472098; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=buJPokmaPQyeldNGCqDdcI5fhmwGvphJPlHxpErBVWU=; b=dQAexgkoBhcWoXUZukHZ7S+CCU1RWbJjaWgvoCFbmxIqMEXV0Hwc1SMw yLo7Bh3EFx920d294wnjLV++y8ana3vW55JDotbP1Dg/KsG7aFaRuCXDH bz9hKri1MfGm0XkgXQInoFrGbbS5+jfFZNUhtfxWukyp4Ewer/0hMP/xw FkvcNdxgn+CPCYoI5caK267f70vnTi12AAjfkyKhAEnesAdvYqK5LGS6Y bzvSxgRuCoKIdKQywgPmf7rLr1Et+kQt8SBH8fgeEcawFrFn4t6ZBG68X 9zgV+WXktZrM4Gd3KKydW6HdzgLPdW+9RICwiAlTZhUsSvrD60ptDW2fZ w==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="6155404" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6155404" Received: from orviesa009.jf.intel.com ([10.64.159.149]) by fmvoesa107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:14 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6615844" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa009-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:14 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 054/130] KVM: VMX: Introduce test mode related to EPT violation VE Date: Mon, 26 Feb 2024 00:25:56 -0800 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata To support TDX, KVM is enhanced to operate with #VE. For TDX, KVM programs to inject #VE conditionally and set #VE suppress bit in EPT entry. For VMX case, #VE isn't used. If #VE happens for VMX, it's a bug. To be defensive (test that VMX case isn't broken), introduce option ept_violation_ve_test and when it's set, set error. Suggested-by: Paolo Bonzini Signed-off-by: Isaku Yamahata --- arch/x86/include/asm/vmx.h | 12 +++++++ arch/x86/kvm/vmx/vmcs.h | 5 +++ arch/x86/kvm/vmx/vmx.c | 69 +++++++++++++++++++++++++++++++++++++- arch/x86/kvm/vmx/vmx.h | 6 +++- 4 files changed, 90 insertions(+), 2 deletions(-) diff --git a/arch/x86/include/asm/vmx.h b/arch/x86/include/asm/vmx.h index 76ed39541a52..f703bae0c4ac 100644 --- a/arch/x86/include/asm/vmx.h +++ b/arch/x86/include/asm/vmx.h @@ -70,6 +70,7 @@ #define SECONDARY_EXEC_ENCLS_EXITING VMCS_CONTROL_BIT(ENCLS_EXITING) #define SECONDARY_EXEC_RDSEED_EXITING VMCS_CONTROL_BIT(RDSEED_EXITING) #define SECONDARY_EXEC_ENABLE_PML VMCS_CONTROL_BIT(PAGE_MOD_LOGGING) +#define SECONDARY_EXEC_EPT_VIOLATION_VE VMCS_CONTROL_BIT(EPT_VIOLATION_VE) #define SECONDARY_EXEC_PT_CONCEAL_VMX VMCS_CONTROL_BIT(PT_CONCEAL_VMX) #define SECONDARY_EXEC_ENABLE_XSAVES VMCS_CONTROL_BIT(XSAVES) #define SECONDARY_EXEC_MODE_BASED_EPT_EXEC VMCS_CONTROL_BIT(MODE_BASED_EPT_EXEC) @@ -225,6 +226,8 @@ enum vmcs_field { VMREAD_BITMAP_HIGH = 0x00002027, VMWRITE_BITMAP = 0x00002028, VMWRITE_BITMAP_HIGH = 0x00002029, + VE_INFORMATION_ADDRESS = 0x0000202A, + VE_INFORMATION_ADDRESS_HIGH = 0x0000202B, XSS_EXIT_BITMAP = 0x0000202C, XSS_EXIT_BITMAP_HIGH = 0x0000202D, ENCLS_EXITING_BITMAP = 0x0000202E, @@ -630,4 +633,13 @@ enum vmx_l1d_flush_state { extern enum vmx_l1d_flush_state l1tf_vmx_mitigation; +struct vmx_ve_information { + u32 exit_reason; + u32 delivery; + u64 exit_qualification; + u64 guest_linear_address; + u64 guest_physical_address; + u16 eptp_index; +}; + #endif diff --git a/arch/x86/kvm/vmx/vmcs.h b/arch/x86/kvm/vmx/vmcs.h index 7c1996b433e2..b25625314658 100644 --- a/arch/x86/kvm/vmx/vmcs.h +++ b/arch/x86/kvm/vmx/vmcs.h @@ -140,6 +140,11 @@ static inline bool is_nm_fault(u32 intr_info) return is_exception_n(intr_info, NM_VECTOR); } +static inline bool is_ve_fault(u32 intr_info) +{ + return is_exception_n(intr_info, VE_VECTOR); +} + /* Undocumented: icebp/int1 */ static inline bool is_icebp(u32 intr_info) { diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index d928acc15d0f..6fe895bd7807 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -127,6 +127,9 @@ module_param(error_on_inconsistent_vmcs_config, bool, 0444); static bool __read_mostly dump_invalid_vmcs = 0; module_param(dump_invalid_vmcs, bool, 0644); +static bool __read_mostly ept_violation_ve_test; +module_param(ept_violation_ve_test, bool, 0444); + #define MSR_BITMAP_MODE_X2APIC 1 #define MSR_BITMAP_MODE_X2APIC_APICV 2 @@ -862,6 +865,13 @@ void vmx_update_exception_bitmap(struct kvm_vcpu *vcpu) eb = (1u << PF_VECTOR) | (1u << UD_VECTOR) | (1u << MC_VECTOR) | (1u << DB_VECTOR) | (1u << AC_VECTOR); + /* + * #VE isn't used for VMX, but for TDX. To test against unexpected + * change related to #VE for VMX, intercept unexpected #VE and warn on + * it. + */ + if (ept_violation_ve_test) + eb |= 1u << VE_VECTOR; /* * Guest access to VMware backdoor ports could legitimately * trigger #GP because of TSS I/O permission bitmap. @@ -2597,6 +2607,9 @@ static int setup_vmcs_config(struct vmcs_config *vmcs_conf, &_cpu_based_2nd_exec_control)) return -EIO; } + if (!ept_violation_ve_test) + _cpu_based_2nd_exec_control &= ~SECONDARY_EXEC_EPT_VIOLATION_VE; + #ifndef CONFIG_X86_64 if (!(_cpu_based_2nd_exec_control & SECONDARY_EXEC_VIRTUALIZE_APIC_ACCESSES)) @@ -2621,6 +2634,7 @@ static int setup_vmcs_config(struct vmcs_config *vmcs_conf, return -EIO; vmx_cap->ept = 0; + _cpu_based_2nd_exec_control &= ~SECONDARY_EXEC_EPT_VIOLATION_VE; } if (!(_cpu_based_2nd_exec_control & SECONDARY_EXEC_ENABLE_VPID) && vmx_cap->vpid) { @@ -4584,6 +4598,7 @@ static u32 vmx_secondary_exec_control(struct vcpu_vmx *vmx) exec_control &= ~SECONDARY_EXEC_ENABLE_VPID; if (!enable_ept) { exec_control &= ~SECONDARY_EXEC_ENABLE_EPT; + exec_control &= ~SECONDARY_EXEC_EPT_VIOLATION_VE; enable_unrestricted_guest = 0; } if (!enable_unrestricted_guest) @@ -4707,8 +4722,40 @@ static void init_vmcs(struct vcpu_vmx *vmx) exec_controls_set(vmx, vmx_exec_control(vmx)); - if (cpu_has_secondary_exec_ctrls()) + if (cpu_has_secondary_exec_ctrls()) { secondary_exec_controls_set(vmx, vmx_secondary_exec_control(vmx)); + if (secondary_exec_controls_get(vmx) & + SECONDARY_EXEC_EPT_VIOLATION_VE) { + if (!vmx->ve_info) { + /* ve_info must be page aligned. */ + struct page *page; + + BUILD_BUG_ON(sizeof(*vmx->ve_info) > PAGE_SIZE); + page = alloc_page(GFP_KERNEL_ACCOUNT | __GFP_ZERO); + if (page) + vmx->ve_info = page_to_virt(page); + } + if (vmx->ve_info) { + /* + * Allow #VE delivery. CPU sets this field to + * 0xFFFFFFFF on #VE delivery. Another #VE can + * occur only if software clears the field. + */ + vmx->ve_info->delivery = 0; + vmcs_write64(VE_INFORMATION_ADDRESS, + __pa(vmx->ve_info)); + } else { + /* + * Because SECONDARY_EXEC_EPT_VIOLATION_VE is + * used only when ept_violation_ve_test is true, + * it's okay to go with the bit disabled. + */ + pr_err("Failed to allocate ve_info. disabling EPT_VIOLATION_VE.\n"); + secondary_exec_controls_clearbit(vmx, + SECONDARY_EXEC_EPT_VIOLATION_VE); + } + } + } if (cpu_has_tertiary_exec_ctrls()) tertiary_exec_controls_set(vmx, vmx_tertiary_exec_control(vmx)); @@ -5196,6 +5243,12 @@ static int handle_exception_nmi(struct kvm_vcpu *vcpu) if (is_invalid_opcode(intr_info)) return handle_ud(vcpu); + /* + * #VE isn't supposed to happen. Although vcpu can send + */ + if (KVM_BUG_ON(is_ve_fault(intr_info), vcpu->kvm)) + return -EIO; + error_code = 0; if (intr_info & INTR_INFO_DELIVER_CODE_MASK) error_code = vmcs_read32(VM_EXIT_INTR_ERROR_CODE); @@ -6383,6 +6436,18 @@ void dump_vmcs(struct kvm_vcpu *vcpu) if (secondary_exec_control & SECONDARY_EXEC_ENABLE_VPID) pr_err("Virtual processor ID = 0x%04x\n", vmcs_read16(VIRTUAL_PROCESSOR_ID)); + if (secondary_exec_control & SECONDARY_EXEC_EPT_VIOLATION_VE) { + struct vmx_ve_information *ve_info; + + pr_err("VE info address = 0x%016llx\n", + vmcs_read64(VE_INFORMATION_ADDRESS)); + ve_info = __va(vmcs_read64(VE_INFORMATION_ADDRESS)); + pr_err("ve_info: 0x%08x 0x%08x 0x%016llx 0x%016llx 0x%016llx 0x%04x\n", + ve_info->exit_reason, ve_info->delivery, + ve_info->exit_qualification, + ve_info->guest_linear_address, + ve_info->guest_physical_address, ve_info->eptp_index); + } } /* @@ -7423,6 +7488,8 @@ void vmx_vcpu_free(struct kvm_vcpu *vcpu) free_vpid(vmx->vpid); nested_vmx_free_vcpu(vcpu); free_loaded_vmcs(vmx->loaded_vmcs); + if (vmx->ve_info) + free_page((unsigned long)vmx->ve_info); } int vmx_vcpu_create(struct kvm_vcpu *vcpu) diff --git a/arch/x86/kvm/vmx/vmx.h b/arch/x86/kvm/vmx/vmx.h index 04ed2a9eada1..79ff54f08fee 100644 --- a/arch/x86/kvm/vmx/vmx.h +++ b/arch/x86/kvm/vmx/vmx.h @@ -349,6 +349,9 @@ struct vcpu_vmx { DECLARE_BITMAP(read, MAX_POSSIBLE_PASSTHROUGH_MSRS); DECLARE_BITMAP(write, MAX_POSSIBLE_PASSTHROUGH_MSRS); } shadow_msr_intercept; + + /* ve_info must be page aligned. */ + struct vmx_ve_information *ve_info; }; struct kvm_vmx { @@ -561,7 +564,8 @@ static inline u8 vmx_get_rvi(void) SECONDARY_EXEC_ENABLE_VMFUNC | \ SECONDARY_EXEC_BUS_LOCK_DETECTION | \ SECONDARY_EXEC_NOTIFY_VM_EXITING | \ - SECONDARY_EXEC_ENCLS_EXITING) + SECONDARY_EXEC_ENCLS_EXITING | \ + SECONDARY_EXEC_EPT_VIOLATION_VE) #define KVM_REQUIRED_VMX_TERTIARY_VM_EXEC_CONTROL 0 #define KVM_OPTIONAL_VMX_TERTIARY_VM_EXEC_CONTROL \ From patchwork Mon Feb 26 08:25:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571483 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4D4F25FEFD; Mon, 26 Feb 2024 08:28:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.13 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936101; cv=none; b=XIYkraWvsOum4snIRa/fjy7kAZ21/XiG2k3H06dS+sLgi9JD9DCcPn6ADl0aN87cAiBznmXKdkyDYpkqNFcG9K1kaBIajj8S0MI1wxcWrOSFzTJ7DEGFmp/tPq0973gtUkyMmxOTNJWKXvI+OTt426sZej0oqeM31+K2jx97Xkg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936101; c=relaxed/simple; bh=WkpnJmQUuLJFyL2brOxfTQdsOI1DqG1Lui1rEGPKdd4=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=tZNXtDZQ76fWVUZD9uExPkBHD96tZlTOBMQAVxi8s5qt5paS+DHRsK4lkDGWgYHpxD6PCcGPGpEqXQW65RlUpibIjTxYli7gV9JAsnpVZ9NR+9bTXzXu4zbW4OHSEfE1Pqi9yRk+aFRSicjm+s8erSnIpCXJWAdPFOl3C43lr/M= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=mTQ0wlRq; arc=none smtp.client-ip=192.198.163.13 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="mTQ0wlRq" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936099; x=1740472099; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=WkpnJmQUuLJFyL2brOxfTQdsOI1DqG1Lui1rEGPKdd4=; b=mTQ0wlRqccGzQf6PIa2T0glQv+Q0fTuTNNwSL7bNcZ4JHZMjB+y6lYLU yw4yIEaBO3ikTle7Ymwu5MtoCv95LQpxyPMJ4V/2+N5TrwDl+KKU1fQyH 9Eh7hyEaIZH2BViOMX0buxol+LUl0t1lyh/NLOR2EMSeYHv7W1KPruXtG QVyjdu3nHtHBAsapZi9MyHhosqHCQdxTojM5mOWsSfLVaBQNqrthl+zzP yyd6wFR7UVH0BKQo8FdP+C5mNSv3Syr2uz7BLmeh4C6CoxcprHB1bbPwG XUKtky30cGV2MGaJWItqt1YCIQj0Jz0f2nhEgwfRp3R63gNNX3Mp0h/GH w==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="6155409" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6155409" Received: from orviesa009.jf.intel.com ([10.64.159.149]) by fmvoesa107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:15 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6615858" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa009-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:15 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 055/130] [MARKER] The start of TDX KVM patch series: KVM TDP MMU hooks Date: Mon, 26 Feb 2024 00:25:57 -0800 Message-Id: <973a712b27d94dcead1ea6ef31404a751f1c2a0b.1708933498.git.isaku.yamahata@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata This empty commit is to mark the start of patch series of KVM TDP MMU hooks. Signed-off-by: Isaku Yamahata --- Documentation/virt/kvm/intel-tdx-layer-status.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/Documentation/virt/kvm/intel-tdx-layer-status.rst b/Documentation/virt/kvm/intel-tdx-layer-status.rst index e893a3d714c7..7903473abad1 100644 --- a/Documentation/virt/kvm/intel-tdx-layer-status.rst +++ b/Documentation/virt/kvm/intel-tdx-layer-status.rst @@ -26,5 +26,5 @@ Patch Layer status * TD vcpu interrupts/exit/hypercall: Not yet * KVM MMU GPA shared bits: Applied -* KVM TDP refactoring for TDX: Applying -* KVM TDP MMU hooks: Not yet +* KVM TDP refactoring for TDX: Applied +* KVM TDP MMU hooks: Applying From patchwork Mon Feb 26 08:25:58 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571484 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 64C4A60267; Mon, 26 Feb 2024 08:28:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.13 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936102; cv=none; b=V3It+9z3izXN7gDMC/Ss9qS4CmnCCaRc8sIWT4oqk+UxknAwSe0QSuTHoiX9LiTIcb1vsEteDdgbTYs7Isx2S04EpIVpkCrxP540Zu3AWf2gAHIiGK7h2Xae7otRiJWr1+JEtRJLoMljedVA5O5CAx8meW5XfMa+uoYI6ScJdvg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936102; c=relaxed/simple; bh=KyoDbEPluROCgq0d+Zl+o1tJ80HCvssyvcFn1Vqq17w=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=kC3DAEKULnwCIO4IQgrXV9+KC3GGGWBt3ISJ3NhhR+IW63JPo2hYKZUHSBY2PXB530jYymx3yHOltSUVIfM04IJP5h4yQyzJTQ5Nv6k0A8vGLLHyf9askKWrOlmPaYZ+uC0fcn/PvGEfd+5AEKpJILMNpQNtQlU/eTFWIOKqniA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=IjqsqOwq; arc=none smtp.client-ip=192.198.163.13 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="IjqsqOwq" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936100; x=1740472100; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=KyoDbEPluROCgq0d+Zl+o1tJ80HCvssyvcFn1Vqq17w=; b=IjqsqOwqUrhse9ODZvCxjadcGVIAs0uE40N2kDL4Z+LArnS7Sj6+1IEN WPGX/823xJuAugpzcSx9BW+wzHAG5+5ubD+5GXmzx2yPJdw5I1RK6iNkF nvNCTTOF7bMp4jLipaoA4ipoNpTZzZ/1Pb7orzhQ+fyoCiBMzgFJuYGRE 7uGhGMBg4f4PRqqd0alNYvLHWVTJSdITR0lkHYOfWc5wUSu+2Q+Oqq6Di brvI8TrxBr80PViCIh8/veA/nMAQJsgHPVlcAgycJ0GJx30eLOqNcx3zF yKfJTgcereYEluYe7tLeb1bkPOlwzL5M7ejfyFKBIo25s72oiDCVoFpQW Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="6155414" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6155414" Received: from orviesa009.jf.intel.com ([10.64.159.149]) by fmvoesa107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:15 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6615867" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa009-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:15 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 056/130] KVM: x86/tdp_mmu: Init role member of struct kvm_mmu_page at allocation Date: Mon, 26 Feb 2024 00:25:58 -0800 Message-Id: <5d2307efb227b927cc9fa3e18787fde8e1cb13e2.1708933498.git.isaku.yamahata@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata Refactor tdp_mmu_alloc_sp() and tdp_mmu_init_sp and eliminate tdp_mmu_init_child_sp(). Currently tdp_mmu_init_sp() (or tdp_mmu_init_child_sp()) sets kvm_mmu_page.role after tdp_mmu_alloc_sp() allocating struct kvm_mmu_page and its page table page. This patch makes tdp_mmu_alloc_sp() initialize kvm_mmu_page.role instead of tdp_mmu_init_sp(). To handle private page tables, argument of is_private needs to be passed down. Given that already page level is passed down, it would be cumbersome to add one more parameter about sp. Instead replace the level argument with union kvm_mmu_page_role. Thus the number of argument won't be increased and more info about sp can be passed down. For private sp, secure page table will be also allocated in addition to struct kvm_mmu_page and page table (spt member). The allocation functions (tdp_mmu_alloc_sp() and __tdp_mmu_alloc_sp_for_split()) need to know if the allocation is for the conventional page table or private page table. Pass union kvm_mmu_role to those functions and initialize role member of struct kvm_mmu_page. Signed-off-by: Isaku Yamahata --- arch/x86/kvm/mmu/tdp_iter.h | 12 ++++++++++ arch/x86/kvm/mmu/tdp_mmu.c | 44 ++++++++++++++++--------------------- 2 files changed, 31 insertions(+), 25 deletions(-) diff --git a/arch/x86/kvm/mmu/tdp_iter.h b/arch/x86/kvm/mmu/tdp_iter.h index fae559559a80..e1e40e3f5eb7 100644 --- a/arch/x86/kvm/mmu/tdp_iter.h +++ b/arch/x86/kvm/mmu/tdp_iter.h @@ -135,4 +135,16 @@ void tdp_iter_start(struct tdp_iter *iter, struct kvm_mmu_page *root, void tdp_iter_next(struct tdp_iter *iter); void tdp_iter_restart(struct tdp_iter *iter); +static inline union kvm_mmu_page_role tdp_iter_child_role(struct tdp_iter *iter) +{ + union kvm_mmu_page_role child_role; + struct kvm_mmu_page *parent_sp; + + parent_sp = sptep_to_sp(rcu_dereference(iter->sptep)); + + child_role = parent_sp->role; + child_role.level--; + return child_role; +} + #endif /* __KVM_X86_MMU_TDP_ITER_H */ diff --git a/arch/x86/kvm/mmu/tdp_mmu.c b/arch/x86/kvm/mmu/tdp_mmu.c index 04c6af49c3e8..87233b3ceaef 100644 --- a/arch/x86/kvm/mmu/tdp_mmu.c +++ b/arch/x86/kvm/mmu/tdp_mmu.c @@ -177,24 +177,30 @@ static struct kvm_mmu_page *tdp_mmu_next_root(struct kvm *kvm, kvm_mmu_page_as_id(_root) != _as_id) { \ } else -static struct kvm_mmu_page *tdp_mmu_alloc_sp(struct kvm_vcpu *vcpu) +static struct kvm_mmu_page *tdp_mmu_alloc_sp(struct kvm_vcpu *vcpu, + union kvm_mmu_page_role role) { struct kvm_mmu_page *sp; sp = kvm_mmu_memory_cache_alloc(&vcpu->arch.mmu_page_header_cache); sp->spt = kvm_mmu_memory_cache_alloc(&vcpu->arch.mmu_shadow_page_cache); + sp->role = role; return sp; } static void tdp_mmu_init_sp(struct kvm_mmu_page *sp, tdp_ptep_t sptep, - gfn_t gfn, union kvm_mmu_page_role role) + gfn_t gfn) { INIT_LIST_HEAD(&sp->possible_nx_huge_page_link); set_page_private(virt_to_page(sp->spt), (unsigned long)sp); - sp->role = role; + /* + * role must be set before calling this function. At least role.level + * is not 0 (PG_LEVEL_NONE). + */ + WARN_ON_ONCE(!sp->role.word); sp->gfn = gfn; sp->ptep = sptep; sp->tdp_mmu_page = true; @@ -202,20 +208,6 @@ static void tdp_mmu_init_sp(struct kvm_mmu_page *sp, tdp_ptep_t sptep, trace_kvm_mmu_get_page(sp, true); } -static void tdp_mmu_init_child_sp(struct kvm_mmu_page *child_sp, - struct tdp_iter *iter) -{ - struct kvm_mmu_page *parent_sp; - union kvm_mmu_page_role role; - - parent_sp = sptep_to_sp(rcu_dereference(iter->sptep)); - - role = parent_sp->role; - role.level--; - - tdp_mmu_init_sp(child_sp, iter->sptep, iter->gfn, role); -} - hpa_t kvm_tdp_mmu_get_vcpu_root_hpa(struct kvm_vcpu *vcpu) { union kvm_mmu_page_role role = vcpu->arch.mmu->root_role; @@ -234,8 +226,8 @@ hpa_t kvm_tdp_mmu_get_vcpu_root_hpa(struct kvm_vcpu *vcpu) goto out; } - root = tdp_mmu_alloc_sp(vcpu); - tdp_mmu_init_sp(root, NULL, 0, role); + root = tdp_mmu_alloc_sp(vcpu, role); + tdp_mmu_init_sp(root, NULL, 0); /* * TDP MMU roots are kept until they are explicitly invalidated, either @@ -1068,8 +1060,8 @@ int kvm_tdp_mmu_map(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault) * The SPTE is either non-present or points to a huge page that * needs to be split. */ - sp = tdp_mmu_alloc_sp(vcpu); - tdp_mmu_init_child_sp(sp, &iter); + sp = tdp_mmu_alloc_sp(vcpu, tdp_iter_child_role(&iter)); + tdp_mmu_init_sp(sp, iter.sptep, iter.gfn); sp->nx_huge_page_disallowed = fault->huge_page_disallowed; @@ -1312,7 +1304,7 @@ bool kvm_tdp_mmu_wrprot_slot(struct kvm *kvm, return spte_set; } -static struct kvm_mmu_page *__tdp_mmu_alloc_sp_for_split(gfp_t gfp) +static struct kvm_mmu_page *__tdp_mmu_alloc_sp_for_split(gfp_t gfp, union kvm_mmu_page_role role) { struct kvm_mmu_page *sp; @@ -1322,6 +1314,7 @@ static struct kvm_mmu_page *__tdp_mmu_alloc_sp_for_split(gfp_t gfp) if (!sp) return NULL; + sp->role = role; sp->spt = (void *)__get_free_page(gfp); if (!sp->spt) { kmem_cache_free(mmu_page_header_cache, sp); @@ -1335,6 +1328,7 @@ static struct kvm_mmu_page *tdp_mmu_alloc_sp_for_split(struct kvm *kvm, struct tdp_iter *iter, bool shared) { + union kvm_mmu_page_role role = tdp_iter_child_role(iter); struct kvm_mmu_page *sp; kvm_lockdep_assert_mmu_lock_held(kvm, shared); @@ -1348,7 +1342,7 @@ static struct kvm_mmu_page *tdp_mmu_alloc_sp_for_split(struct kvm *kvm, * If this allocation fails we drop the lock and retry with reclaim * allowed. */ - sp = __tdp_mmu_alloc_sp_for_split(GFP_NOWAIT | __GFP_ACCOUNT); + sp = __tdp_mmu_alloc_sp_for_split(GFP_NOWAIT | __GFP_ACCOUNT, role); if (sp) return sp; @@ -1360,7 +1354,7 @@ static struct kvm_mmu_page *tdp_mmu_alloc_sp_for_split(struct kvm *kvm, write_unlock(&kvm->mmu_lock); iter->yielded = true; - sp = __tdp_mmu_alloc_sp_for_split(GFP_KERNEL_ACCOUNT); + sp = __tdp_mmu_alloc_sp_for_split(GFP_KERNEL_ACCOUNT, role); if (shared) read_lock(&kvm->mmu_lock); @@ -1455,7 +1449,7 @@ static int tdp_mmu_split_huge_pages_root(struct kvm *kvm, continue; } - tdp_mmu_init_child_sp(sp, &iter); + tdp_mmu_init_sp(sp, iter.sptep, iter.gfn); if (tdp_mmu_split_huge_page(kvm, &iter, sp, shared)) goto retry; From patchwork Mon Feb 26 08:25:59 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571485 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BA7136027B; Mon, 26 Feb 2024 08:28:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.13 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936102; cv=none; b=ie6Y+tMHPXwj+b+lzl/G3BdCkqhemCh73vlSuaAVAIc/rtndwITu9HoCFnws/TG0yCYQs0lbo6Afpn9J8dUt9Gd+UH4XW+Bvl3Kb8gARHXWtifsttfeJw+CE4bm8N2Zg8tmz3xCL2nCmao4Q3aKfkqQQihYmTk3F4WDCcijHmOQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936102; c=relaxed/simple; bh=rcbdqsS4hYOqhGj7tlgdIuvBw0XJL9OLMgxeHGcJc1Y=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=tyYbbYcOXSwFrSjomAaUIsWeudUNbGJAAO341ecLmxgC5FhUtHeQLCgsS6JgaIkLWtYNLuVDtoe5h2vajnJhsUvcRZherY4p3le3k3tPgxEdmpU8fe9dam0U3B2nKmh0SSbbCEGSOemVUB+Ay8HKlASeiGN1Cfcbxht7jot4b0Q= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=hyGN813+; arc=none smtp.client-ip=192.198.163.13 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="hyGN813+" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936101; x=1740472101; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=rcbdqsS4hYOqhGj7tlgdIuvBw0XJL9OLMgxeHGcJc1Y=; b=hyGN813+aARFCVEVzlmtpmYG0zvxeYcxG4N6tEadnCO9qoi9okMDXeEI 6OeqgoAjmjqnY/9amSZWF+v5GrzwIpfixQ+xgb94kyusF4IAidGjfcXCB AFOZeaB6LbQw9sDj1GIqEUxwmKyVRF7JjjhcM12bE72u8yQOkskWiTyuO OM1Eef+6f4QWA7hxPSOsApB4f/Dn7ivKGxoY+55+tfmLPnbL8o4cO9bIE fj081J5NtRKUFHHZV7ai9rXi9fIq5+V2zQ675oNTOB49RUWH+Erbsa3Tq 06RwUYvZbA/EqX7DH4yGhzXHKKmGY6haZB99BiD9NyJfE2i0n7xo9xqI8 A==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="6155421" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6155421" Received: from orviesa009.jf.intel.com ([10.64.159.149]) by fmvoesa107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:16 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6615878" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa009-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:16 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 057/130] KVM: x86/mmu: Add a new is_private member for union kvm_mmu_page_role Date: Mon, 26 Feb 2024 00:25:59 -0800 Message-Id: <3692118f525c21cbe3670e1c20b1bbbf3be2b4ba.1708933498.git.isaku.yamahata@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata Because TDX support introduces private mapping, add a new member in union kvm_mmu_page_role with access functions to check the member. Signed-off-by: Isaku Yamahata --- v19: - Fix is_private_sptep() when NULL case. - drop CONFIG_KVM_MMU_PRIVATE --- arch/x86/include/asm/kvm_host.h | 13 ++++++++++++- arch/x86/kvm/mmu/mmu_internal.h | 5 +++++ arch/x86/kvm/mmu/spte.h | 7 +++++++ 3 files changed, 24 insertions(+), 1 deletion(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 6c10d8d1017f..dcc6f7c38a83 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -349,7 +349,8 @@ union kvm_mmu_page_role { unsigned ad_disabled:1; unsigned guest_mode:1; unsigned passthrough:1; - unsigned :5; + unsigned is_private:1; + unsigned :4; /* * This is left at the top of the word so that @@ -361,6 +362,16 @@ union kvm_mmu_page_role { }; }; +static inline bool kvm_mmu_page_role_is_private(union kvm_mmu_page_role role) +{ + return !!role.is_private; +} + +static inline void kvm_mmu_page_role_set_private(union kvm_mmu_page_role *role) +{ + role->is_private = 1; +} + /* * kvm_mmu_extended_role complements kvm_mmu_page_role, tracking properties * relevant to the current MMU configuration. When loading CR0, CR4, or EFER, diff --git a/arch/x86/kvm/mmu/mmu_internal.h b/arch/x86/kvm/mmu/mmu_internal.h index 0443bfcf5d9c..e3f54701f98d 100644 --- a/arch/x86/kvm/mmu/mmu_internal.h +++ b/arch/x86/kvm/mmu/mmu_internal.h @@ -145,6 +145,11 @@ static inline int kvm_mmu_page_as_id(struct kvm_mmu_page *sp) return kvm_mmu_role_as_id(sp->role); } +static inline bool is_private_sp(const struct kvm_mmu_page *sp) +{ + return kvm_mmu_page_role_is_private(sp->role); +} + static inline bool kvm_mmu_page_ad_need_write_protect(struct kvm_mmu_page *sp) { /* diff --git a/arch/x86/kvm/mmu/spte.h b/arch/x86/kvm/mmu/spte.h index 1a163aee9ec6..3ef8ea18321b 100644 --- a/arch/x86/kvm/mmu/spte.h +++ b/arch/x86/kvm/mmu/spte.h @@ -264,6 +264,13 @@ static inline struct kvm_mmu_page *root_to_sp(hpa_t root) return spte_to_child_sp(root); } +static inline bool is_private_sptep(u64 *sptep) +{ + if (WARN_ON_ONCE(!sptep)) + return false; + return is_private_sp(sptep_to_sp(sptep)); +} + static inline bool is_mmio_spte(struct kvm *kvm, u64 spte) { return (spte & shadow_mmio_mask) == kvm->arch.shadow_mmio_value && From patchwork Mon Feb 26 08:26:00 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571486 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5BEA2604C6; Mon, 26 Feb 2024 08:28:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.13 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936103; cv=none; b=NWd40F6+jC6pwOw0zu6NYfQPP/QFiemphaSzZTavFpwqrhUpV0fyu/cePGkB+fbjd4CDx8Yay703+cXV6PUfTv6z2zZQS+h70M3/vgJgoETPZze3dCCuvM0jBzMPJct3egGaozpV75I5xyBjdv3Pcmx7GW+EzlOq+BKOM83ypR4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936103; c=relaxed/simple; bh=eQXT76E/Iv2Z7iSGfh0gwWuL1GwbfdwraxK817hkAaI=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=iMV4uhUJ2UG5ErFl+9NgcqzsTCEeisDPQ5VIFs0ZZX53wAieZkAVjRBCluY5PluFd6eB9d0qrNAKF1sYNGG117v26Zy6+j6/Atw52zlP9gzZLDUZvx8cXfx+03wKiA8YJLRyp/NGy2t9tG1vXwUnYwmseNkyh/6x3IiP6Wlwek8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=ggo4w80H; arc=none smtp.client-ip=192.198.163.13 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="ggo4w80H" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936101; x=1740472101; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=eQXT76E/Iv2Z7iSGfh0gwWuL1GwbfdwraxK817hkAaI=; b=ggo4w80HNsoA2lb7IsfIRTEScdruT4XfscosPXg6SDz2An4iJtnqedSr qvOeuvlb/9dauNoulXbo8L8U2xziEMCHTTd9FokpN7+gZMUB+NocKQeH2 5QbiCK9ciJCq2H4zrqmU7YjzzOIIowd2WTl/YoIlfG42R6L3mtwSxryF7 3OInRgZvas9dUBphpO8kXqWOjjwZXcyz2ldRCt/CYoZjjbHHTG8COJkmN AfysKU9sFO51xSsbpz+dG5Wi7HZI0sheiBGaiG6eWdfFKsZbxXRmWTbrg piierkOZU4G08U83u+gOOJx0hgO+IK9qMBslAcLBs+OdCY+LZTeUw2W73 A==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="6155425" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6155425" Received: from orviesa009.jf.intel.com ([10.64.159.149]) by fmvoesa107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:17 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6615886" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa009-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:17 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com, Binbin Wu Subject: [PATCH v19 058/130] KVM: x86/mmu: Add a private pointer to struct kvm_mmu_page Date: Mon, 26 Feb 2024 00:26:00 -0800 Message-Id: <9d86b5a2787d20ffb5a58f86e43601a660521f16.1708933498.git.isaku.yamahata@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata For private GPA, CPU refers a private page table whose contents are encrypted. The dedicated APIs to operate on it (e.g. updating/reading its PTE entry) are used and their cost is expensive. When KVM resolves KVM page fault, it walks the page tables. To reuse the existing KVM MMU code and mitigate the heavy cost to directly walk private page table, allocate one more page to copy the dummy page table for KVM MMU code to directly walk. Resolve KVM page fault with the existing code, and do additional operations necessary for the private page table. To distinguish such cases, the existing KVM page table is called a shared page table (i.e. not associated with private page table), and the page table with private page table is called a private page table. The relationship is depicted below. Add a private pointer to struct kvm_mmu_page for private page table and add helper functions to allocate/initialize/free a private page table page. KVM page fault | | | V | -------------+---------- | | | | V V | shared GPA private GPA | | | | V V | shared PT root dummy PT root | private PT root | | | | V V | V shared PT dummy PT ----propagate----> private PT | | | | | \-----------------+------\ | | | | | V | V V shared guest page | private guest page | non-encrypted memory | encrypted memory | PT: page table - Shared PT is visible to KVM and it is used by CPU. - Private PT is used by CPU but it is invisible to KVM. - Dummy PT is visible to KVM but not used by CPU. It is used to propagate PT change to the actual private PT which is used by CPU. Signed-off-by: Isaku Yamahata Reviewed-by: Binbin Wu --- v19: - typo in the comment in kvm_mmu_alloc_private_spt() - drop CONFIG_KVM_MMU_PRIVATE --- arch/x86/include/asm/kvm_host.h | 5 +++ arch/x86/kvm/mmu/mmu.c | 7 ++++ arch/x86/kvm/mmu/mmu_internal.h | 63 ++++++++++++++++++++++++++++++--- arch/x86/kvm/mmu/tdp_mmu.c | 1 + 4 files changed, 72 insertions(+), 4 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index dcc6f7c38a83..efd3fda1c177 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -825,6 +825,11 @@ struct kvm_vcpu_arch { struct kvm_mmu_memory_cache mmu_shadow_page_cache; struct kvm_mmu_memory_cache mmu_shadowed_info_cache; struct kvm_mmu_memory_cache mmu_page_header_cache; + /* + * This cache is to allocate private page table. E.g. Secure-EPT used + * by the TDX module. + */ + struct kvm_mmu_memory_cache mmu_private_spt_cache; /* * QEMU userspace and the guest each have their own FPU state. diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index eeebbc67e42b..0d6d4506ec97 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -685,6 +685,12 @@ static int mmu_topup_memory_caches(struct kvm_vcpu *vcpu, bool maybe_indirect) 1 + PT64_ROOT_MAX_LEVEL + PTE_PREFETCH_NUM); if (r) return r; + if (kvm_gfn_shared_mask(vcpu->kvm)) { + r = kvm_mmu_topup_memory_cache(&vcpu->arch.mmu_private_spt_cache, + PT64_ROOT_MAX_LEVEL); + if (r) + return r; + } r = kvm_mmu_topup_memory_cache(&vcpu->arch.mmu_shadow_page_cache, PT64_ROOT_MAX_LEVEL); if (r) @@ -704,6 +710,7 @@ static void mmu_free_memory_caches(struct kvm_vcpu *vcpu) kvm_mmu_free_memory_cache(&vcpu->arch.mmu_pte_list_desc_cache); kvm_mmu_free_memory_cache(&vcpu->arch.mmu_shadow_page_cache); kvm_mmu_free_memory_cache(&vcpu->arch.mmu_shadowed_info_cache); + kvm_mmu_free_memory_cache(&vcpu->arch.mmu_private_spt_cache); kvm_mmu_free_memory_cache(&vcpu->arch.mmu_page_header_cache); } diff --git a/arch/x86/kvm/mmu/mmu_internal.h b/arch/x86/kvm/mmu/mmu_internal.h index e3f54701f98d..002f3f80bf3b 100644 --- a/arch/x86/kvm/mmu/mmu_internal.h +++ b/arch/x86/kvm/mmu/mmu_internal.h @@ -101,7 +101,21 @@ struct kvm_mmu_page { int root_count; refcount_t tdp_mmu_root_count; }; - unsigned int unsync_children; + union { + struct { + unsigned int unsync_children; + /* + * Number of writes since the last time traversal + * visited this page. + */ + atomic_t write_flooding_count; + }; + /* + * Associated private shadow page table, e.g. Secure-EPT page + * passed to the TDX module. + */ + void *private_spt; + }; union { struct kvm_rmap_head parent_ptes; /* rmap pointers to parent sptes */ tdp_ptep_t ptep; @@ -124,9 +138,6 @@ struct kvm_mmu_page { int clear_spte_count; #endif - /* Number of writes since the last time traversal visited this page. */ - atomic_t write_flooding_count; - #ifdef CONFIG_X86_64 /* Used for freeing the page asynchronously if it is a TDP MMU page. */ struct rcu_head rcu_head; @@ -150,6 +161,50 @@ static inline bool is_private_sp(const struct kvm_mmu_page *sp) return kvm_mmu_page_role_is_private(sp->role); } +static inline void *kvm_mmu_private_spt(struct kvm_mmu_page *sp) +{ + return sp->private_spt; +} + +static inline void kvm_mmu_init_private_spt(struct kvm_mmu_page *sp, void *private_spt) +{ + sp->private_spt = private_spt; +} + +static inline void kvm_mmu_alloc_private_spt(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp) +{ + bool is_root = vcpu->arch.root_mmu.root_role.level == sp->role.level; + + KVM_BUG_ON(!kvm_mmu_page_role_is_private(sp->role), vcpu->kvm); + if (is_root) + /* + * Because TDX module assigns root Secure-EPT page and set it to + * Secure-EPTP when TD vcpu is created, secure page table for + * root isn't needed. + */ + sp->private_spt = NULL; + else { + /* + * Because the TDX module doesn't trust VMM and initializes + * the pages itself, KVM doesn't initialize them. Allocate + * pages with garbage and give them to the TDX module. + */ + sp->private_spt = kvm_mmu_memory_cache_alloc(&vcpu->arch.mmu_private_spt_cache); + /* + * Because mmu_private_spt_cache is topped up before starting + * kvm page fault resolving, the allocation above shouldn't + * fail. + */ + WARN_ON_ONCE(!sp->private_spt); + } +} + +static inline void kvm_mmu_free_private_spt(struct kvm_mmu_page *sp) +{ + if (sp->private_spt) + free_page((unsigned long)sp->private_spt); +} + static inline bool kvm_mmu_page_ad_need_write_protect(struct kvm_mmu_page *sp) { /* diff --git a/arch/x86/kvm/mmu/tdp_mmu.c b/arch/x86/kvm/mmu/tdp_mmu.c index 87233b3ceaef..d47f0daf1b03 100644 --- a/arch/x86/kvm/mmu/tdp_mmu.c +++ b/arch/x86/kvm/mmu/tdp_mmu.c @@ -53,6 +53,7 @@ void kvm_mmu_uninit_tdp_mmu(struct kvm *kvm) static void tdp_mmu_free_sp(struct kvm_mmu_page *sp) { + kvm_mmu_free_private_spt(sp); free_page((unsigned long)sp->spt); kmem_cache_free(mmu_page_header_cache, sp); } From patchwork Mon Feb 26 08:26:01 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571488 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 80F8E605D9; Mon, 26 Feb 2024 08:28:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.13 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936104; cv=none; b=jFoQzNkMmsJyBQ619Of0CjzXzvcSR4oQsAdy9j1ZImmD2jpoGyX5Zh92gzHHZrW/GMR65zIJp/x9HFiKRZmRSl5/qFTsbeKUS0gCG4OimAj7jQDpd8kwB055I4W8InGLjDFP0MaBUCXHU8whX7iZWhHhSL5K+P7y6Y03tXIqE+0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936104; c=relaxed/simple; bh=Ec2gGLjcpP1TqXDhmYgjFxqNVDJsro62cXTKaDeY8HM=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=LpMI5T2u9FrbS6Eq1Kk8no4TfOIlObd1kZFWwi9ke4PbhbzgQEFgnamCBspI3Mp7Sx3Uxf13AfHuuOptiGAbq3H/ob79/ftYowBuwGhsWUaqYJ6UQhpCOcHetu4Ipr6J0L7gba5927DeAA6KB9qjF0C08KqVbMOx/KuD2gUt04k= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=jeSHSvdN; arc=none smtp.client-ip=192.198.163.13 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="jeSHSvdN" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936102; x=1740472102; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Ec2gGLjcpP1TqXDhmYgjFxqNVDJsro62cXTKaDeY8HM=; b=jeSHSvdNYX7lwImRmAgBbA8hkvO/Ep5insnSpY4LUC9DbdRu+oo2RxvM 5vkKjrOU5ESTBID71P601bFkLH6amlM4W0KMYU1jCZCKp6FpcxN2ht3oe UGQDAgj/Mp2HUbb/Z3WH4lDc3br7s6C2+PA1xPCrRqqp+as7agr0sLBsk jlDaBvQNIfRnHjDXURP6L2gFnkXBr44RFn6QpzeQJNrjDbTLOGBGmaB0z ysTOSjS6Gj3TWYEi1hlsxUqltO7ZhrDP1l86yORvYAKNc4oVgxEcFd/gm /ZvnFPOs0OKhryF1HJtgLr0xe3CkRNOtoZNNt1+REDsMldT2TwliAtS63 w==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="6155430" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6155430" Received: from orviesa009.jf.intel.com ([10.64.159.149]) by fmvoesa107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:17 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6615893" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa009-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:17 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com, Sean Christopherson Subject: [PATCH v19 059/130] KVM: x86/tdp_mmu: Don't zap private pages for unsupported cases Date: Mon, 26 Feb 2024 00:26:01 -0800 Message-Id: <1ed955a44cd81738b498fe52823766622d8ad57f.1708933498.git.isaku.yamahata@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Sean Christopherson TDX supports only write-back(WB) memory type for private memory architecturally so that (virtualized) memory type change doesn't make sense for private memory. Also currently, page migration isn't supported for TDX yet. (TDX architecturally supports page migration. it's KVM and kernel implementation issue.) Regarding memory type change (mtrr virtualization and lapic page mapping change), pages are zapped by kvm_zap_gfn_range(). On the next KVM page fault, the SPTE entry with a new memory type for the page is populated. Regarding page migration, pages are zapped by the mmu notifier. On the next KVM page fault, the new migrated page is populated. Don't zap private pages on unmapping for those two cases. When deleting/moving a KVM memory slot, zap private pages. Typically tearing down VM. Don't invalidate private page tables. i.e. zap only leaf SPTEs for KVM mmu that has a shared bit mask. The existing kvm_tdp_mmu_invalidate_all_roots() depends on role.invalid with read-lock of mmu_lock so that other vcpu can operate on KVM mmu concurrently. It marks the root page table invalid and zaps SPTEs of the root page tables. The TDX module doesn't allow to unlink a protected root page table from the hardware and then allocate a new one for it. i.e. replacing a protected root page table. Instead, zap only leaf SPTEs for KVM mmu with a shared bit mask set. Signed-off-by: Sean Christopherson Signed-off-by: Isaku Yamahata --- arch/x86/kvm/mmu/mmu.c | 61 ++++++++++++++++++++++++++++++++++++-- arch/x86/kvm/mmu/tdp_mmu.c | 37 +++++++++++++++++++---- arch/x86/kvm/mmu/tdp_mmu.h | 5 ++-- 3 files changed, 92 insertions(+), 11 deletions(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 0d6d4506ec97..30c86e858ae4 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -6339,7 +6339,7 @@ static void kvm_mmu_zap_all_fast(struct kvm *kvm) * e.g. before kvm_zap_obsolete_pages() could drop mmu_lock and yield. */ if (tdp_mmu_enabled) - kvm_tdp_mmu_invalidate_all_roots(kvm); + kvm_tdp_mmu_invalidate_all_roots(kvm, true); /* * Notify all vcpus to reload its shadow page table and flush TLB. @@ -6459,7 +6459,16 @@ void kvm_zap_gfn_range(struct kvm *kvm, gfn_t gfn_start, gfn_t gfn_end) flush = kvm_rmap_zap_gfn_range(kvm, gfn_start, gfn_end); if (tdp_mmu_enabled) - flush = kvm_tdp_mmu_zap_leafs(kvm, gfn_start, gfn_end, flush); + /* + * zap_private = false. Zap only shared pages. + * + * kvm_zap_gfn_range() is used when MTRR or PAT memory + * type was changed. Later on the next kvm page fault, + * populate it with updated spte entry. + * Because only WB is supported for private pages, don't + * care of private pages. + */ + flush = kvm_tdp_mmu_zap_leafs(kvm, gfn_start, gfn_end, flush, false); if (flush) kvm_flush_remote_tlbs_range(kvm, gfn_start, gfn_end - gfn_start); @@ -6905,10 +6914,56 @@ void kvm_arch_flush_shadow_all(struct kvm *kvm) kvm_mmu_zap_all(kvm); } +static void kvm_mmu_zap_memslot(struct kvm *kvm, struct kvm_memory_slot *slot) +{ + bool flush = false; + + write_lock(&kvm->mmu_lock); + + /* + * Zapping non-leaf SPTEs, a.k.a. not-last SPTEs, isn't required, worst + * case scenario we'll have unused shadow pages lying around until they + * are recycled due to age or when the VM is destroyed. + */ + if (tdp_mmu_enabled) { + struct kvm_gfn_range range = { + .slot = slot, + .start = slot->base_gfn, + .end = slot->base_gfn + slot->npages, + .may_block = true, + + /* + * This handles both private gfn and shared gfn. + * All private page should be zapped on memslot deletion. + */ + .only_private = true, + .only_shared = true, + }; + + flush = kvm_tdp_mmu_unmap_gfn_range(kvm, &range, flush); + } else { + /* TDX supports only TDP-MMU case. */ + WARN_ON_ONCE(1); + flush = true; + } + if (flush) + kvm_flush_remote_tlbs(kvm); + + write_unlock(&kvm->mmu_lock); +} + void kvm_arch_flush_shadow_memslot(struct kvm *kvm, struct kvm_memory_slot *slot) { - kvm_mmu_zap_all_fast(kvm); + if (kvm_gfn_shared_mask(kvm)) + /* + * Secure-EPT requires to release PTs from the leaf. The + * optimization to zap root PT first with child PT doesn't + * work. + */ + kvm_mmu_zap_memslot(kvm, slot); + else + kvm_mmu_zap_all_fast(kvm); } void kvm_mmu_invalidate_mmio_sptes(struct kvm *kvm, u64 gen) diff --git a/arch/x86/kvm/mmu/tdp_mmu.c b/arch/x86/kvm/mmu/tdp_mmu.c index d47f0daf1b03..e7514a807134 100644 --- a/arch/x86/kvm/mmu/tdp_mmu.c +++ b/arch/x86/kvm/mmu/tdp_mmu.c @@ -37,7 +37,7 @@ void kvm_mmu_uninit_tdp_mmu(struct kvm *kvm) * for zapping and thus puts the TDP MMU's reference to each root, i.e. * ultimately frees all roots. */ - kvm_tdp_mmu_invalidate_all_roots(kvm); + kvm_tdp_mmu_invalidate_all_roots(kvm, false); kvm_tdp_mmu_zap_invalidated_roots(kvm); WARN_ON(atomic64_read(&kvm->arch.tdp_mmu_pages)); @@ -771,7 +771,8 @@ bool kvm_tdp_mmu_zap_sp(struct kvm *kvm, struct kvm_mmu_page *sp) * operation can cause a soft lockup. */ static bool tdp_mmu_zap_leafs(struct kvm *kvm, struct kvm_mmu_page *root, - gfn_t start, gfn_t end, bool can_yield, bool flush) + gfn_t start, gfn_t end, bool can_yield, bool flush, + bool zap_private) { struct tdp_iter iter; @@ -779,6 +780,10 @@ static bool tdp_mmu_zap_leafs(struct kvm *kvm, struct kvm_mmu_page *root, lockdep_assert_held_write(&kvm->mmu_lock); + WARN_ON_ONCE(zap_private && !is_private_sp(root)); + if (!zap_private && is_private_sp(root)) + return false; + rcu_read_lock(); for_each_tdp_pte_min_level(iter, root, PG_LEVEL_4K, start, end) { @@ -810,13 +815,15 @@ static bool tdp_mmu_zap_leafs(struct kvm *kvm, struct kvm_mmu_page *root, * true if a TLB flush is needed before releasing the MMU lock, i.e. if one or * more SPTEs were zapped since the MMU lock was last acquired. */ -bool kvm_tdp_mmu_zap_leafs(struct kvm *kvm, gfn_t start, gfn_t end, bool flush) +bool kvm_tdp_mmu_zap_leafs(struct kvm *kvm, gfn_t start, gfn_t end, bool flush, + bool zap_private) { struct kvm_mmu_page *root; lockdep_assert_held_write(&kvm->mmu_lock); for_each_tdp_mmu_root_yield_safe(kvm, root) - flush = tdp_mmu_zap_leafs(kvm, root, start, end, true, flush); + flush = tdp_mmu_zap_leafs(kvm, root, start, end, true, flush, + zap_private && is_private_sp(root)); return flush; } @@ -891,7 +898,7 @@ void kvm_tdp_mmu_zap_invalidated_roots(struct kvm *kvm) * Note, kvm_tdp_mmu_zap_invalidated_roots() is gifted the TDP MMU's reference. * See kvm_tdp_mmu_get_vcpu_root_hpa(). */ -void kvm_tdp_mmu_invalidate_all_roots(struct kvm *kvm) +void kvm_tdp_mmu_invalidate_all_roots(struct kvm *kvm, bool skip_private) { struct kvm_mmu_page *root; @@ -916,6 +923,12 @@ void kvm_tdp_mmu_invalidate_all_roots(struct kvm *kvm) * or get/put references to roots. */ list_for_each_entry(root, &kvm->arch.tdp_mmu_roots, link) { + /* + * Skip private root since private page table + * is only torn down when VM is destroyed. + */ + if (skip_private && is_private_sp(root)) + continue; /* * Note, invalid roots can outlive a memslot update! Invalid * roots must be *zapped* before the memslot update completes, @@ -1104,14 +1117,26 @@ int kvm_tdp_mmu_map(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault) return ret; } +/* Used by mmu notifier via kvm_unmap_gfn_range() */ bool kvm_tdp_mmu_unmap_gfn_range(struct kvm *kvm, struct kvm_gfn_range *range, bool flush) { struct kvm_mmu_page *root; + bool zap_private = false; + + if (kvm_gfn_shared_mask(kvm)) { + if (!range->only_private && !range->only_shared) + /* attributes change */ + zap_private = !(range->arg.attributes & + KVM_MEMORY_ATTRIBUTE_PRIVATE); + else + zap_private = range->only_private; + } __for_each_tdp_mmu_root_yield_safe(kvm, root, range->slot->as_id, false) flush = tdp_mmu_zap_leafs(kvm, root, range->start, range->end, - range->may_block, flush); + range->may_block, flush, + zap_private && is_private_sp(root)); return flush; } diff --git a/arch/x86/kvm/mmu/tdp_mmu.h b/arch/x86/kvm/mmu/tdp_mmu.h index 20d97aa46c49..b3cf58a50357 100644 --- a/arch/x86/kvm/mmu/tdp_mmu.h +++ b/arch/x86/kvm/mmu/tdp_mmu.h @@ -19,10 +19,11 @@ __must_check static inline bool kvm_tdp_mmu_get_root(struct kvm_mmu_page *root) void kvm_tdp_mmu_put_root(struct kvm *kvm, struct kvm_mmu_page *root); -bool kvm_tdp_mmu_zap_leafs(struct kvm *kvm, gfn_t start, gfn_t end, bool flush); +bool kvm_tdp_mmu_zap_leafs(struct kvm *kvm, gfn_t start, gfn_t end, bool flush, + bool zap_private); bool kvm_tdp_mmu_zap_sp(struct kvm *kvm, struct kvm_mmu_page *sp); void kvm_tdp_mmu_zap_all(struct kvm *kvm); -void kvm_tdp_mmu_invalidate_all_roots(struct kvm *kvm); +void kvm_tdp_mmu_invalidate_all_roots(struct kvm *kvm, bool skip_private); void kvm_tdp_mmu_zap_invalidated_roots(struct kvm *kvm); int kvm_tdp_mmu_map(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault); From patchwork Mon Feb 26 08:26:02 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571489 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 01BEA60876; Mon, 26 Feb 2024 08:28:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.13 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936105; cv=none; b=sGhgyqEdFUf8lWb7lOV77o+02SbrUKwPdJtp/cW9WLAoMPYBQs/QSmrVrc8v9XruHABGFOvqP02GRH8qb15y7LoG3/H5OzC1e+cPMp3S/Waah4Yh0W4PuvqwbS1NJhH2AS946qZqKgVB67x4JR9lsG1FP01p09CG/ijyH/K+OVw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936105; c=relaxed/simple; bh=JTMod/uEmH5xtpRfr6dh/AqM/hDApc2nFe1BtZqLSW4=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=qmkJLLocgZ0XIpEJcN+G4fUDMZJTZGWnM2St24jtGUGzzEOkX/QBIm3bskK/hllzxkFrx36De0wuyB6GocyxfAxA5uxRkIF51Z1++NEE/9KeEpsWoL8CnCJtUEFWeZn2fM6shm5ZZy9DPcXYc3Rq4h8TDoXXgatefOsf0RJT6fQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=l2Z44U/U; arc=none smtp.client-ip=192.198.163.13 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="l2Z44U/U" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936103; x=1740472103; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=JTMod/uEmH5xtpRfr6dh/AqM/hDApc2nFe1BtZqLSW4=; b=l2Z44U/URH/LRmIQ0ckFvOKQ6Lej7k8IXtDGoTHnQ01PVXyma/zelY/U daFBvg8qSpMGN4QlU/5APDZRApfa9/a25p+uLwH8TNsoqVWQ2KHYZWt39 0SZpnRgiI+5Q8MCbjw3p9GvweEVq6inR1bHn0K2LEHv2QS1Ao5BVb3Msq NpHVEzQ2+YCJPxnuAQ7XwdDiZkJarOLh4g9yJ62Fkl6XsoIQDyBR7QeUN XGMosTlpZud4fOIM9n23tHbJ0Cz9ezhVG7nhuZjjlJBRti34eSMHt1RYA imdZOZd1hMqvDZ4mxN/rSbu5iJjWUuVTc1YwLULC/CAMHorFh+IA358ex A==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="6155436" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6155436" Received: from orviesa009.jf.intel.com ([10.64.159.149]) by fmvoesa107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:18 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6615905" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa009-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:18 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com, Binbin Wu Subject: [PATCH v19 060/130] KVM: x86/tdp_mmu: Apply mmu notifier callback to only shared GPA Date: Mon, 26 Feb 2024 00:26:02 -0800 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata The private GPAs that typically guest memfd backs aren't subject to MMU notifier because it isn't mapped into virtual address of user process. kvm_tdp_mmu_handle_gfn() handles the callback of the MMU notifier, clear_flush_young(), clear_young(), test_young()() and change_pte(). Make kvm_tdp_mmu_handle_gfn() aware of private mapping and skip private mapping. Even with AS_UNMOVABLE set, those mmu notifier are called. For example, ksmd triggers change_pte(). Signed-off-by: Isaku Yamahata Reviewed-by: Binbin Wu --- v19: - type: test_gfn() => test_young() v18: - newly added Signed-off-by: Isaku Yamahata --- arch/x86/kvm/mmu/tdp_mmu.c | 22 +++++++++++++++++++++- 1 file changed, 21 insertions(+), 1 deletion(-) diff --git a/arch/x86/kvm/mmu/tdp_mmu.c b/arch/x86/kvm/mmu/tdp_mmu.c index e7514a807134..10507920f36b 100644 --- a/arch/x86/kvm/mmu/tdp_mmu.c +++ b/arch/x86/kvm/mmu/tdp_mmu.c @@ -1157,9 +1157,29 @@ static __always_inline bool kvm_tdp_mmu_handle_gfn(struct kvm *kvm, * into this helper allow blocking; it'd be dead, wasteful code. */ for_each_tdp_mmu_root(kvm, root, range->slot->as_id) { + gfn_t start, end; + + /* + * This function is called on behalf of mmu_notifier of + * clear_flush_young(), clear_young(), test_young()(), and + * change_pte(). They apply to only shared GPAs. + */ + WARN_ON_ONCE(range->only_private); + WARN_ON_ONCE(!range->only_shared); + if (is_private_sp(root)) + continue; + + /* + * For TDX shared mapping, set GFN shared bit to the range, + * so the handler() doesn't need to set it, to avoid duplicated + * code in multiple handler()s. + */ + start = kvm_gfn_to_shared(kvm, range->start); + end = kvm_gfn_to_shared(kvm, range->end); + rcu_read_lock(); - tdp_root_for_each_leaf_pte(iter, root, range->start, range->end) + tdp_root_for_each_leaf_pte(iter, root, start, end) ret |= handler(kvm, &iter, range); rcu_read_unlock(); From patchwork Mon Feb 26 08:26:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571490 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E234B60BA8; Mon, 26 Feb 2024 08:28:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.13 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936105; cv=none; b=IkydHhxru7tIZWsZuhWJUefonMLI+liBkZKfT07A12/qdWfaa7lpbDdnfLHV72VYROwCnwtEJfcrEKN0ATYdTUMBJuHs66ko++F0nM72yMFLN9D9ZwQgwW4cXxvMcRjzFJG6ChqGLR8uktGHOZJYKiXaeDMXEpzjSxI1Xq1nNRE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936105; c=relaxed/simple; bh=WUIyQRqsBA8fWp2BC5BUSOmDMUGqWMDNMhmPzCSRUbg=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=eeCNVLuXTMFnHMnPioKHkvI7sE3on+rV+NbAc0a0q0vAzmPRQF7JDkinIHycPrQ2xgsadqUFWmjAwqzZPPNz8aZgZ22WT6IJpRIh4nkzx3YJVHXfbAcGdkM2DapbP3jxO++JyLWuwZQF/3frECMLFNeJjxTWHghCvokQq+6j+6E= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=KrnSeZr8; arc=none smtp.client-ip=192.198.163.13 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="KrnSeZr8" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936104; x=1740472104; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=WUIyQRqsBA8fWp2BC5BUSOmDMUGqWMDNMhmPzCSRUbg=; b=KrnSeZr8iGAXQOUuWkSiDSSnLKeXAg3QWYR8v2GOKnm+ZhWVBcZw3w/y 2azjPk+/luaTKbJJ+mbOfqzUH39tb4P4f8DOPhiYpmpR0yM5o/OiaaVwy a0rGN466rSN9K8Ze98XrQhIJf5OcKsErN7BrZrN5Pc7qNryUPT/Zozg0T q0D+kQfnNEXO668VTQcxEdMKbFfOeduqGcEqyBYXjo7m6Pj7+WRqrLTX8 1o9KMHPjNR1iVuBbzaM1wcBZqQ0NNvihBdiZ+id4ycQ4wwZo+QLidUUsd KWdUJZZaKDs6p/fmwjRZuMcbZfBAHtmUK94SMLnEAgVqEINH2P7P7kcKB g==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="6155447" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6155447" Received: from orviesa009.jf.intel.com ([10.64.159.149]) by fmvoesa107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:19 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6615914" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa009-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:19 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com, Binbin Wu Subject: [PATCH v19 061/130] KVM: x86/tdp_mmu: Sprinkle __must_check Date: Mon, 26 Feb 2024 00:26:03 -0800 Message-Id: <6dd201694cf2ce7f866e0634c3113db7a9024f8b.1708933498.git.isaku.yamahata@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata TDP MMU allows tdp_mmu_set_spte_atomic() and tdp_mmu_zap_spte_atomic() to return -EBUSY or -EAGAIN error. The caller must check the return value and retry. Sprinkle __must_check to guarantee it. Signed-off-by: Isaku Yamahata Reviewed-by: Binbin Wu --- v19: - Added Reviewed-by: Binbin --- arch/x86/kvm/mmu/tdp_mmu.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/arch/x86/kvm/mmu/tdp_mmu.c b/arch/x86/kvm/mmu/tdp_mmu.c index 10507920f36b..a90907b31c54 100644 --- a/arch/x86/kvm/mmu/tdp_mmu.c +++ b/arch/x86/kvm/mmu/tdp_mmu.c @@ -507,9 +507,9 @@ static void handle_changed_spte(struct kvm *kvm, int as_id, gfn_t gfn, * no side-effects other than setting iter->old_spte to the last * known value of the spte. */ -static inline int tdp_mmu_set_spte_atomic(struct kvm *kvm, - struct tdp_iter *iter, - u64 new_spte) +static inline int __must_check tdp_mmu_set_spte_atomic(struct kvm *kvm, + struct tdp_iter *iter, + u64 new_spte) { u64 *sptep = rcu_dereference(iter->sptep); @@ -539,8 +539,8 @@ static inline int tdp_mmu_set_spte_atomic(struct kvm *kvm, return 0; } -static inline int tdp_mmu_zap_spte_atomic(struct kvm *kvm, - struct tdp_iter *iter) +static inline int __must_check tdp_mmu_zap_spte_atomic(struct kvm *kvm, + struct tdp_iter *iter) { int ret; From patchwork Mon Feb 26 08:26:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571495 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 17B35612CC; Mon, 26 Feb 2024 08:28:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.13 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936108; cv=none; b=BDuoIDci2g2zwIPdb/kaa4SRNay5dhhiPq7W7Jr/iDAwicuJdQykFdvsDxxD72TbxsiJd8R8LF3p5jziDMOGjjjFy7l465JvYks15WN5Z3c8z2evh3He2RUHepZbW8pXdpugNFD/SAJfHseefLAellm08b6SFziZQuNjHpkTaZc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936108; c=relaxed/simple; bh=aLZoC63A3uu0TZNttub3PZIIePoiXtGMeP+dT2lN9PE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Xc9fiwhRMVTyfcBKscGA4tgN8wQJ9J3ySAdq0U4sHICTooknKLEDmReGshIN/gx4ZV4Ieu+RCjHJJHDmZUhsB+6OSJ4j04S0OFZUB5cphOq2CnRQ3nuMfhh0bWJOoBfLCzFlw0+GONrQHwOZeh4YmrzaLn5I1KqZZG+Tw7YOJ2s= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=RbNM9isi; arc=none smtp.client-ip=192.198.163.13 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="RbNM9isi" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936105; x=1740472105; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=aLZoC63A3uu0TZNttub3PZIIePoiXtGMeP+dT2lN9PE=; b=RbNM9isiXMcluXj+h8EpIKzSpkXgKnEExoJP6KCpiMiEheYGplTO3p0I Q4xircZP2rgYBUmJGILfaRg2Z8nf9MbSN6vs4DUwWcoihE9g+QJP7ViAL TPcD2gJcxitXFDnNMATzImAfHI9eqwjwS4uRF6Q5xg+AqGtIhCK+JJMsU zbDY7NOMybPOe3+LETwZNDVZLFgdR7egwjhop9oh9Yv0rv6SEnsBY5v1A RKsl+uGB5Kl6shjrNib3heRCNQ3q22irnwSs2EOF/GKiQFUydzMGDAfjQ xxHNmI8m+Yqul6SikR5c0D00mX0MFUtSb1ClY6puY5n/rCP4cJvAEoU6n w==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="6155455" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6155455" Received: from orviesa009.jf.intel.com ([10.64.159.149]) by fmvoesa107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:19 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6615922" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa009-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:19 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 062/130] KVM: x86/tdp_mmu: Support TDX private mapping for TDP MMU Date: Mon, 26 Feb 2024 00:26:04 -0800 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata Allocate protected page table for private page table, and add hooks to operate on protected page table. This patch adds allocation/free of protected page tables and hooks. When calling hooks to update SPTE entry, freeze the entry, call hooks and unfreeze the entry to allow concurrent updates on page tables. Which is the advantage of TDP MMU. As kvm_gfn_shared_mask() returns false always, those hooks aren't called yet with this patch. When the faulting GPA is private, the KVM fault is called private. When resolving private KVM fault, allocate protected page table and call hooks to operate on protected page table. On the change of the private PTE entry, invoke kvm_x86_ops hook in __handle_changed_spte() to propagate the change to protected page table. The following depicts the relationship. private KVM page fault | | | V | private GPA | CPU protected EPTP | | | V | V private PT root | protected PT root | | | V | V private PT --hook to propagate-->protected PT | | | \--------------------+------\ | | | | | V V | private guest page | | non-encrypted memory | encrypted memory | PT: page table The existing KVM TDP MMU code uses atomic update of SPTE. On populating the EPT entry, atomically set the entry. However, it requires TLB shootdown to zap SPTE. To address it, the entry is frozen with the special SPTE value that clears the present bit. After the TLB shootdown, the entry is set to the eventual value (unfreeze). For protected page table, hooks are called to update protected page table in addition to direct access to the private SPTE. For the zapping case, it works to freeze the SPTE. It can call hooks in addition to TLB shootdown. For populating the private SPTE entry, there can be a race condition without further protection vcpu 1: populating 2M private SPTE vcpu 2: populating 4K private SPTE vcpu 2: TDX SEAMCALL to update 4K protected SPTE => error vcpu 1: TDX SEAMCALL to update 2M protected SPTE To avoid the race, the frozen SPTE is utilized. Instead of atomic update of the private entry, freeze the entry, call the hook that update protected SPTE, set the entry to the final value. Support 4K page only at this stage. 2M page support can be done in future patches. Co-developed-by: Kai Huang Signed-off-by: Kai Huang Signed-off-by: Isaku Yamahata --- v19: - drop CONFIG_KVM_MMU_PRIVATE v18: - Rename freezed => frozen v14 -> v15: - Refined is_private condition check in kvm_tdp_mmu_map(). Add kvm_gfn_shared_mask() check. - catch up for struct kvm_range change Signed-off-by: Isaku Yamahata --- arch/x86/include/asm/kvm-x86-ops.h | 5 + arch/x86/include/asm/kvm_host.h | 11 ++ arch/x86/kvm/mmu/mmu.c | 17 +- arch/x86/kvm/mmu/mmu_internal.h | 13 +- arch/x86/kvm/mmu/tdp_iter.h | 2 +- arch/x86/kvm/mmu/tdp_mmu.c | 308 +++++++++++++++++++++++++---- arch/x86/kvm/mmu/tdp_mmu.h | 2 +- virt/kvm/kvm_main.c | 1 + 8 files changed, 320 insertions(+), 39 deletions(-) diff --git a/arch/x86/include/asm/kvm-x86-ops.h b/arch/x86/include/asm/kvm-x86-ops.h index a8e96804a252..e1c75f8c1b25 100644 --- a/arch/x86/include/asm/kvm-x86-ops.h +++ b/arch/x86/include/asm/kvm-x86-ops.h @@ -101,6 +101,11 @@ KVM_X86_OP_OPTIONAL_RET0(set_tss_addr) KVM_X86_OP_OPTIONAL_RET0(set_identity_map_addr) KVM_X86_OP_OPTIONAL_RET0(get_mt_mask) KVM_X86_OP(load_mmu_pgd) +KVM_X86_OP_OPTIONAL(link_private_spt) +KVM_X86_OP_OPTIONAL(free_private_spt) +KVM_X86_OP_OPTIONAL(set_private_spte) +KVM_X86_OP_OPTIONAL(remove_private_spte) +KVM_X86_OP_OPTIONAL(zap_private_spte) KVM_X86_OP(has_wbinvd_exit) KVM_X86_OP(get_l2_tsc_offset) KVM_X86_OP(get_l2_tsc_multiplier) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index efd3fda1c177..bc0767c884f7 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -468,6 +468,7 @@ struct kvm_mmu { int (*sync_spte)(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp, int i); struct kvm_mmu_root_info root; + hpa_t private_root_hpa; union kvm_cpu_role cpu_role; union kvm_mmu_page_role root_role; @@ -1740,6 +1741,16 @@ struct kvm_x86_ops { void (*load_mmu_pgd)(struct kvm_vcpu *vcpu, hpa_t root_hpa, int root_level); + int (*link_private_spt)(struct kvm *kvm, gfn_t gfn, enum pg_level level, + void *private_spt); + int (*free_private_spt)(struct kvm *kvm, gfn_t gfn, enum pg_level level, + void *private_spt); + int (*set_private_spte)(struct kvm *kvm, gfn_t gfn, enum pg_level level, + kvm_pfn_t pfn); + int (*remove_private_spte)(struct kvm *kvm, gfn_t gfn, enum pg_level level, + kvm_pfn_t pfn); + int (*zap_private_spte)(struct kvm *kvm, gfn_t gfn, enum pg_level level); + bool (*has_wbinvd_exit)(void); u64 (*get_l2_tsc_offset)(struct kvm_vcpu *vcpu); diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 30c86e858ae4..0e0321ad9ca2 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -3717,7 +3717,12 @@ static int mmu_alloc_direct_roots(struct kvm_vcpu *vcpu) goto out_unlock; if (tdp_mmu_enabled) { - root = kvm_tdp_mmu_get_vcpu_root_hpa(vcpu); + if (kvm_gfn_shared_mask(vcpu->kvm) && + !VALID_PAGE(mmu->private_root_hpa)) { + root = kvm_tdp_mmu_get_vcpu_root_hpa(vcpu, true); + mmu->private_root_hpa = root; + } + root = kvm_tdp_mmu_get_vcpu_root_hpa(vcpu, false); mmu->root.hpa = root; } else if (shadow_root_level >= PT64_ROOT_4LEVEL) { root = mmu_alloc_root(vcpu, 0, 0, shadow_root_level); @@ -4627,7 +4632,7 @@ int kvm_tdp_page_fault(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault) if (kvm_mmu_honors_guest_mtrrs(vcpu->kvm)) { for ( ; fault->max_level > PG_LEVEL_4K; --fault->max_level) { int page_num = KVM_PAGES_PER_HPAGE(fault->max_level); - gfn_t base = gfn_round_for_level(fault->gfn, + gfn_t base = gfn_round_for_level(gpa_to_gfn(fault->addr), fault->max_level); if (kvm_mtrr_check_gfn_range_consistency(vcpu, base, page_num)) @@ -4662,6 +4667,7 @@ int kvm_mmu_map_tdp_page(struct kvm_vcpu *vcpu, gpa_t gpa, u64 error_code, }; WARN_ON_ONCE(!vcpu->arch.mmu->root_role.direct); + fault.gfn = gpa_to_gfn(fault.addr) & ~kvm_gfn_shared_mask(vcpu->kvm); fault.slot = kvm_vcpu_gfn_to_memslot(vcpu, fault.gfn); r = mmu_topup_memory_caches(vcpu, false); @@ -6166,6 +6172,7 @@ static int __kvm_mmu_create(struct kvm_vcpu *vcpu, struct kvm_mmu *mmu) mmu->root.hpa = INVALID_PAGE; mmu->root.pgd = 0; + mmu->private_root_hpa = INVALID_PAGE; for (i = 0; i < KVM_MMU_NUM_PREV_ROOTS; i++) mmu->prev_roots[i] = KVM_MMU_ROOT_INFO_INVALID; @@ -7211,6 +7218,12 @@ int kvm_mmu_vendor_module_init(void) void kvm_mmu_destroy(struct kvm_vcpu *vcpu) { kvm_mmu_unload(vcpu); + if (tdp_mmu_enabled) { + write_lock(&vcpu->kvm->mmu_lock); + mmu_free_root_page(vcpu->kvm, &vcpu->arch.mmu->private_root_hpa, + NULL); + write_unlock(&vcpu->kvm->mmu_lock); + } free_mmu_pages(&vcpu->arch.root_mmu); free_mmu_pages(&vcpu->arch.guest_mmu); mmu_free_memory_caches(vcpu); diff --git a/arch/x86/kvm/mmu/mmu_internal.h b/arch/x86/kvm/mmu/mmu_internal.h index 002f3f80bf3b..9e2c7c6d85bf 100644 --- a/arch/x86/kvm/mmu/mmu_internal.h +++ b/arch/x86/kvm/mmu/mmu_internal.h @@ -6,6 +6,8 @@ #include #include +#include "mmu.h" + #ifdef CONFIG_KVM_PROVE_MMU #define KVM_MMU_WARN_ON(x) WARN_ON_ONCE(x) #else @@ -205,6 +207,15 @@ static inline void kvm_mmu_free_private_spt(struct kvm_mmu_page *sp) free_page((unsigned long)sp->private_spt); } +static inline gfn_t kvm_gfn_for_root(struct kvm *kvm, struct kvm_mmu_page *root, + gfn_t gfn) +{ + if (is_private_sp(root)) + return kvm_gfn_to_private(kvm, gfn); + else + return kvm_gfn_to_shared(kvm, gfn); +} + static inline bool kvm_mmu_page_ad_need_write_protect(struct kvm_mmu_page *sp) { /* @@ -363,7 +374,7 @@ static inline int kvm_mmu_do_page_fault(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa, int r; if (vcpu->arch.mmu->root_role.direct) { - fault.gfn = fault.addr >> PAGE_SHIFT; + fault.gfn = gpa_to_gfn(fault.addr) & ~kvm_gfn_shared_mask(vcpu->kvm); fault.slot = kvm_vcpu_gfn_to_memslot(vcpu, fault.gfn); } diff --git a/arch/x86/kvm/mmu/tdp_iter.h b/arch/x86/kvm/mmu/tdp_iter.h index e1e40e3f5eb7..a9c9cd0db20a 100644 --- a/arch/x86/kvm/mmu/tdp_iter.h +++ b/arch/x86/kvm/mmu/tdp_iter.h @@ -91,7 +91,7 @@ struct tdp_iter { tdp_ptep_t pt_path[PT64_ROOT_MAX_LEVEL]; /* A pointer to the current SPTE */ tdp_ptep_t sptep; - /* The lowest GFN mapped by the current SPTE */ + /* The lowest GFN (shared bits included) mapped by the current SPTE */ gfn_t gfn; /* The level of the root page given to the iterator */ int root_level; diff --git a/arch/x86/kvm/mmu/tdp_mmu.c b/arch/x86/kvm/mmu/tdp_mmu.c index a90907b31c54..1a0e4baa8311 100644 --- a/arch/x86/kvm/mmu/tdp_mmu.c +++ b/arch/x86/kvm/mmu/tdp_mmu.c @@ -187,6 +187,9 @@ static struct kvm_mmu_page *tdp_mmu_alloc_sp(struct kvm_vcpu *vcpu, sp->spt = kvm_mmu_memory_cache_alloc(&vcpu->arch.mmu_shadow_page_cache); sp->role = role; + if (kvm_mmu_page_role_is_private(role)) + kvm_mmu_alloc_private_spt(vcpu, sp); + return sp; } @@ -209,7 +212,8 @@ static void tdp_mmu_init_sp(struct kvm_mmu_page *sp, tdp_ptep_t sptep, trace_kvm_mmu_get_page(sp, true); } -hpa_t kvm_tdp_mmu_get_vcpu_root_hpa(struct kvm_vcpu *vcpu) +static struct kvm_mmu_page *kvm_tdp_mmu_get_vcpu_root(struct kvm_vcpu *vcpu, + bool private) { union kvm_mmu_page_role role = vcpu->arch.mmu->root_role; struct kvm *kvm = vcpu->kvm; @@ -221,6 +225,8 @@ hpa_t kvm_tdp_mmu_get_vcpu_root_hpa(struct kvm_vcpu *vcpu) * Check for an existing root before allocating a new one. Note, the * role check prevents consuming an invalid root. */ + if (private) + kvm_mmu_page_role_set_private(&role); for_each_tdp_mmu_root(kvm, root, kvm_mmu_role_as_id(role)) { if (root->role.word == role.word && kvm_tdp_mmu_get_root(root)) @@ -244,12 +250,17 @@ hpa_t kvm_tdp_mmu_get_vcpu_root_hpa(struct kvm_vcpu *vcpu) spin_unlock(&kvm->arch.tdp_mmu_pages_lock); out: - return __pa(root->spt); + return root; +} + +hpa_t kvm_tdp_mmu_get_vcpu_root_hpa(struct kvm_vcpu *vcpu, bool private) +{ + return __pa(kvm_tdp_mmu_get_vcpu_root(vcpu, private)->spt); } static void handle_changed_spte(struct kvm *kvm, int as_id, gfn_t gfn, - u64 old_spte, u64 new_spte, int level, - bool shared); + u64 old_spte, u64 new_spte, + union kvm_mmu_page_role role, bool shared); static void tdp_account_mmu_page(struct kvm *kvm, struct kvm_mmu_page *sp) { @@ -376,12 +387,78 @@ static void handle_removed_pt(struct kvm *kvm, tdp_ptep_t pt, bool shared) REMOVED_SPTE, level); } handle_changed_spte(kvm, kvm_mmu_page_as_id(sp), gfn, - old_spte, REMOVED_SPTE, level, shared); + old_spte, REMOVED_SPTE, sp->role, + shared); + } + + if (is_private_sp(sp) && + WARN_ON(static_call(kvm_x86_free_private_spt)(kvm, sp->gfn, sp->role.level, + kvm_mmu_private_spt(sp)))) { + /* + * Failed to unlink Secure EPT page and there is nothing to do + * further. Intentionally leak the page to prevent the kernel + * from accessing the encrypted page. + */ + kvm_mmu_init_private_spt(sp, NULL); } call_rcu(&sp->rcu_head, tdp_mmu_free_sp_rcu_callback); } +static void *get_private_spt(gfn_t gfn, u64 new_spte, int level) +{ + if (is_shadow_present_pte(new_spte) && !is_last_spte(new_spte, level)) { + struct kvm_mmu_page *sp = to_shadow_page(pfn_to_hpa(spte_to_pfn(new_spte))); + void *private_spt = kvm_mmu_private_spt(sp); + + WARN_ON_ONCE(!private_spt); + WARN_ON_ONCE(sp->role.level + 1 != level); + WARN_ON_ONCE(sp->gfn != gfn); + return private_spt; + } + + return NULL; +} + +static void handle_removed_private_spte(struct kvm *kvm, gfn_t gfn, + u64 old_spte, u64 new_spte, + int level) +{ + bool was_present = is_shadow_present_pte(old_spte); + bool is_present = is_shadow_present_pte(new_spte); + bool was_leaf = was_present && is_last_spte(old_spte, level); + bool is_leaf = is_present && is_last_spte(new_spte, level); + kvm_pfn_t old_pfn = spte_to_pfn(old_spte); + kvm_pfn_t new_pfn = spte_to_pfn(new_spte); + int ret; + + /* Ignore change of software only bits. e.g. host_writable */ + if (was_leaf == is_leaf && was_present == is_present) + return; + + /* + * Allow only leaf page to be zapped. Reclaim Non-leaf page tables at + * destroying VM. + */ + WARN_ON_ONCE(is_present); + if (!was_leaf) + return; + + /* non-present -> non-present doesn't make sense. */ + KVM_BUG_ON(!was_present, kvm); + KVM_BUG_ON(new_pfn, kvm); + + /* Zapping leaf spte is allowed only when write lock is held. */ + lockdep_assert_held_write(&kvm->mmu_lock); + ret = static_call(kvm_x86_zap_private_spte)(kvm, gfn, level); + /* Because write lock is held, operation should success. */ + if (KVM_BUG_ON(ret, kvm)) + return; + + ret = static_call(kvm_x86_remove_private_spte)(kvm, gfn, level, old_pfn); + KVM_BUG_ON(ret, kvm); +} + /** * handle_changed_spte - handle bookkeeping associated with an SPTE change * @kvm: kvm instance @@ -389,7 +466,7 @@ static void handle_removed_pt(struct kvm *kvm, tdp_ptep_t pt, bool shared) * @gfn: the base GFN that was mapped by the SPTE * @old_spte: The value of the SPTE before the change * @new_spte: The value of the SPTE after the change - * @level: the level of the PT the SPTE is part of in the paging structure + * @role: the role of the PT the SPTE is part of in the paging structure * @shared: This operation may not be running under the exclusive use of * the MMU lock and the operation must synchronize with other * threads that might be modifying SPTEs. @@ -399,14 +476,18 @@ static void handle_removed_pt(struct kvm *kvm, tdp_ptep_t pt, bool shared) * and fast_pf_fix_direct_spte()). */ static void handle_changed_spte(struct kvm *kvm, int as_id, gfn_t gfn, - u64 old_spte, u64 new_spte, int level, - bool shared) + u64 old_spte, u64 new_spte, + union kvm_mmu_page_role role, bool shared) { + bool is_private = kvm_mmu_page_role_is_private(role); + int level = role.level; bool was_present = is_shadow_present_pte(old_spte); bool is_present = is_shadow_present_pte(new_spte); bool was_leaf = was_present && is_last_spte(old_spte, level); bool is_leaf = is_present && is_last_spte(new_spte, level); - bool pfn_changed = spte_to_pfn(old_spte) != spte_to_pfn(new_spte); + kvm_pfn_t old_pfn = spte_to_pfn(old_spte); + kvm_pfn_t new_pfn = spte_to_pfn(new_spte); + bool pfn_changed = old_pfn != new_pfn; WARN_ON_ONCE(level > PT64_ROOT_MAX_LEVEL); WARN_ON_ONCE(level < PG_LEVEL_4K); @@ -473,7 +554,7 @@ static void handle_changed_spte(struct kvm *kvm, int as_id, gfn_t gfn, if (was_leaf && is_dirty_spte(old_spte) && (!is_present || !is_dirty_spte(new_spte) || pfn_changed)) - kvm_set_pfn_dirty(spte_to_pfn(old_spte)); + kvm_set_pfn_dirty(old_pfn); /* * Recursively handle child PTs if the change removed a subtree from @@ -482,14 +563,82 @@ static void handle_changed_spte(struct kvm *kvm, int as_id, gfn_t gfn, * pages are kernel allocations and should never be migrated. */ if (was_present && !was_leaf && - (is_leaf || !is_present || WARN_ON_ONCE(pfn_changed))) + (is_leaf || !is_present || WARN_ON_ONCE(pfn_changed))) { + KVM_BUG_ON(is_private != is_private_sptep(spte_to_child_pt(old_spte, level)), + kvm); handle_removed_pt(kvm, spte_to_child_pt(old_spte, level), shared); + } + + /* + * Secure-EPT requires to remove Secure-EPT tables after removing + * children. hooks after handling lower page table by above + * handle_remove_pt(). + */ + if (is_private && !is_present) + handle_removed_private_spte(kvm, gfn, old_spte, new_spte, role.level); if (was_leaf && is_accessed_spte(old_spte) && (!is_present || !is_accessed_spte(new_spte) || pfn_changed)) kvm_set_pfn_accessed(spte_to_pfn(old_spte)); } +static int __must_check __set_private_spte_present(struct kvm *kvm, tdp_ptep_t sptep, + gfn_t gfn, u64 old_spte, + u64 new_spte, int level) +{ + bool was_present = is_shadow_present_pte(old_spte); + bool is_present = is_shadow_present_pte(new_spte); + bool is_leaf = is_present && is_last_spte(new_spte, level); + kvm_pfn_t new_pfn = spte_to_pfn(new_spte); + int ret = 0; + + lockdep_assert_held(&kvm->mmu_lock); + /* TDP MMU doesn't change present -> present */ + KVM_BUG_ON(was_present, kvm); + + /* + * Use different call to either set up middle level + * private page table, or leaf. + */ + if (is_leaf) + ret = static_call(kvm_x86_set_private_spte)(kvm, gfn, level, new_pfn); + else { + void *private_spt = get_private_spt(gfn, new_spte, level); + + KVM_BUG_ON(!private_spt, kvm); + ret = static_call(kvm_x86_link_private_spt)(kvm, gfn, level, private_spt); + } + + return ret; +} + +static int __must_check set_private_spte_present(struct kvm *kvm, tdp_ptep_t sptep, + gfn_t gfn, u64 old_spte, + u64 new_spte, int level) +{ + int ret; + + /* + * For private page table, callbacks are needed to propagate SPTE + * change into the protected page table. In order to atomically update + * both the SPTE and the protected page tables with callbacks, utilize + * freezing SPTE. + * - Freeze the SPTE. Set entry to REMOVED_SPTE. + * - Trigger callbacks for protected page tables. + * - Unfreeze the SPTE. Set the entry to new_spte. + */ + lockdep_assert_held(&kvm->mmu_lock); + if (!try_cmpxchg64(sptep, &old_spte, REMOVED_SPTE)) + return -EBUSY; + + ret = __set_private_spte_present(kvm, sptep, gfn, old_spte, new_spte, level); + if (ret) + __kvm_tdp_mmu_write_spte(sptep, old_spte); + else + __kvm_tdp_mmu_write_spte(sptep, new_spte); + return ret; +} + /* * tdp_mmu_set_spte_atomic - Set a TDP MMU SPTE atomically * and handle the associated bookkeeping. Do not mark the page dirty @@ -512,6 +661,7 @@ static inline int __must_check tdp_mmu_set_spte_atomic(struct kvm *kvm, u64 new_spte) { u64 *sptep = rcu_dereference(iter->sptep); + bool frozen = false; /* * The caller is responsible for ensuring the old SPTE is not a REMOVED @@ -523,19 +673,45 @@ static inline int __must_check tdp_mmu_set_spte_atomic(struct kvm *kvm, lockdep_assert_held_read(&kvm->mmu_lock); - /* - * Note, fast_pf_fix_direct_spte() can also modify TDP MMU SPTEs and - * does not hold the mmu_lock. On failure, i.e. if a different logical - * CPU modified the SPTE, try_cmpxchg64() updates iter->old_spte with - * the current value, so the caller operates on fresh data, e.g. if it - * retries tdp_mmu_set_spte_atomic() - */ - if (!try_cmpxchg64(sptep, &iter->old_spte, new_spte)) - return -EBUSY; + if (is_private_sptep(iter->sptep) && !is_removed_spte(new_spte)) { + int ret; - handle_changed_spte(kvm, iter->as_id, iter->gfn, iter->old_spte, - new_spte, iter->level, true); + if (is_shadow_present_pte(new_spte)) { + /* + * Populating case. handle_changed_spte() can + * process without freezing because it only updates + * stats. + */ + ret = set_private_spte_present(kvm, iter->sptep, iter->gfn, + iter->old_spte, new_spte, iter->level); + if (ret) + return ret; + } else { + /* + * Zapping case. handle_changed_spte() calls Secure-EPT + * blocking or removal. Freeze the entry. + */ + if (!try_cmpxchg64(sptep, &iter->old_spte, REMOVED_SPTE)) + return -EBUSY; + frozen = true; + } + } else { + /* + * Note, fast_pf_fix_direct_spte() can also modify TDP MMU SPTEs + * and does not hold the mmu_lock. On failure, i.e. if a + * different logical CPU modified the SPTE, try_cmpxchg64() + * updates iter->old_spte with the current value, so the caller + * operates on fresh data, e.g. if it retries + * tdp_mmu_set_spte_atomic() + */ + if (!try_cmpxchg64(sptep, &iter->old_spte, new_spte)) + return -EBUSY; + } + handle_changed_spte(kvm, iter->as_id, iter->gfn, iter->old_spte, + new_spte, sptep_to_sp(sptep)->role, true); + if (frozen) + __kvm_tdp_mmu_write_spte(sptep, new_spte); return 0; } @@ -585,6 +761,8 @@ static inline int __must_check tdp_mmu_zap_spte_atomic(struct kvm *kvm, static u64 tdp_mmu_set_spte(struct kvm *kvm, int as_id, tdp_ptep_t sptep, u64 old_spte, u64 new_spte, gfn_t gfn, int level) { + union kvm_mmu_page_role role; + lockdep_assert_held_write(&kvm->mmu_lock); /* @@ -597,8 +775,17 @@ static u64 tdp_mmu_set_spte(struct kvm *kvm, int as_id, tdp_ptep_t sptep, WARN_ON_ONCE(is_removed_spte(old_spte) || is_removed_spte(new_spte)); old_spte = kvm_tdp_mmu_write_spte(sptep, old_spte, new_spte, level); + if (is_private_sptep(sptep) && !is_removed_spte(new_spte) && + is_shadow_present_pte(new_spte)) { + lockdep_assert_held_write(&kvm->mmu_lock); + /* Because write spin lock is held, no race. It should success. */ + KVM_BUG_ON(__set_private_spte_present(kvm, sptep, gfn, old_spte, + new_spte, level), kvm); + } - handle_changed_spte(kvm, as_id, gfn, old_spte, new_spte, level, false); + role = sptep_to_sp(sptep)->role; + role.level = level; + handle_changed_spte(kvm, as_id, gfn, old_spte, new_spte, role, false); return old_spte; } @@ -621,8 +808,11 @@ static inline void tdp_mmu_iter_set_spte(struct kvm *kvm, struct tdp_iter *iter, continue; \ else -#define tdp_mmu_for_each_pte(_iter, _mmu, _start, _end) \ - for_each_tdp_pte(_iter, root_to_sp(_mmu->root.hpa), _start, _end) +#define tdp_mmu_for_each_pte(_iter, _mmu, _private, _start, _end) \ + for_each_tdp_pte(_iter, \ + root_to_sp((_private) ? _mmu->private_root_hpa : \ + _mmu->root.hpa), \ + _start, _end) /* * Yield if the MMU lock is contended or this thread needs to return control @@ -784,6 +974,14 @@ static bool tdp_mmu_zap_leafs(struct kvm *kvm, struct kvm_mmu_page *root, if (!zap_private && is_private_sp(root)) return false; + /* + * start and end doesn't have GFN shared bit. This function zaps + * a region including alias. Adjust shared bit of [start, end) if the + * root is shared. + */ + start = kvm_gfn_for_root(kvm, root, start); + end = kvm_gfn_for_root(kvm, root, end); + rcu_read_lock(); for_each_tdp_pte_min_level(iter, root, PG_LEVEL_4K, start, end) { @@ -960,10 +1158,26 @@ static int tdp_mmu_map_handle_target_level(struct kvm_vcpu *vcpu, if (unlikely(!fault->slot)) new_spte = make_mmio_spte(vcpu, iter->gfn, ACC_ALL); - else - wrprot = make_spte(vcpu, sp, fault->slot, ACC_ALL, iter->gfn, - fault->pfn, iter->old_spte, fault->prefetch, true, - fault->map_writable, &new_spte); + else { + unsigned long pte_access = ACC_ALL; + gfn_t gfn = iter->gfn; + + if (kvm_gfn_shared_mask(vcpu->kvm)) { + if (fault->is_private) + gfn |= kvm_gfn_shared_mask(vcpu->kvm); + else + /* + * TDX shared GPAs are no executable, enforce + * this for the SDV. + */ + pte_access &= ~ACC_EXEC_MASK; + } + + wrprot = make_spte(vcpu, sp, fault->slot, pte_access, gfn, + fault->pfn, iter->old_spte, + fault->prefetch, true, fault->map_writable, + &new_spte); + } if (new_spte == iter->old_spte) ret = RET_PF_SPURIOUS; @@ -1041,6 +1255,8 @@ int kvm_tdp_mmu_map(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault) struct kvm *kvm = vcpu->kvm; struct tdp_iter iter; struct kvm_mmu_page *sp; + gfn_t raw_gfn; + bool is_private = fault->is_private && kvm_gfn_shared_mask(kvm); int ret = RET_PF_RETRY; kvm_mmu_hugepage_adjust(vcpu, fault); @@ -1049,7 +1265,17 @@ int kvm_tdp_mmu_map(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault) rcu_read_lock(); - tdp_mmu_for_each_pte(iter, mmu, fault->gfn, fault->gfn + 1) { + raw_gfn = gpa_to_gfn(fault->addr); + + if (is_error_noslot_pfn(fault->pfn) || + !kvm_pfn_to_refcounted_page(fault->pfn)) { + if (is_private) { + rcu_read_unlock(); + return -EFAULT; + } + } + + tdp_mmu_for_each_pte(iter, mmu, is_private, raw_gfn, raw_gfn + 1) { int r; if (fault->nx_huge_page_workaround_enabled) @@ -1079,9 +1305,14 @@ int kvm_tdp_mmu_map(struct kvm_vcpu *vcpu, struct kvm_page_fault *fault) sp->nx_huge_page_disallowed = fault->huge_page_disallowed; - if (is_shadow_present_pte(iter.old_spte)) + if (is_shadow_present_pte(iter.old_spte)) { + /* + * TODO: large page support. + * Doesn't support large page for TDX now + */ + KVM_BUG_ON(is_private_sptep(iter.sptep), vcpu->kvm); r = tdp_mmu_split_huge_page(kvm, &iter, sp, true); - else + } else r = tdp_mmu_link_sp(kvm, &iter, sp, true); /* @@ -1362,6 +1593,8 @@ static struct kvm_mmu_page *__tdp_mmu_alloc_sp_for_split(gfp_t gfp, union kvm_mm sp->role = role; sp->spt = (void *)__get_free_page(gfp); + /* TODO: large page support for private GPA. */ + WARN_ON_ONCE(kvm_mmu_page_role_is_private(role)); if (!sp->spt) { kmem_cache_free(mmu_page_header_cache, sp); return NULL; @@ -1378,6 +1611,10 @@ static struct kvm_mmu_page *tdp_mmu_alloc_sp_for_split(struct kvm *kvm, struct kvm_mmu_page *sp; kvm_lockdep_assert_mmu_lock_held(kvm, shared); + KVM_BUG_ON(kvm_mmu_page_role_is_private(role) != + is_private_sptep(iter->sptep), kvm); + /* TODO: Large page isn't supported for private SPTE yet. */ + KVM_BUG_ON(kvm_mmu_page_role_is_private(role), kvm); /* * Since we are allocating while under the MMU lock we have to be @@ -1802,7 +2039,7 @@ int kvm_tdp_mmu_get_walk(struct kvm_vcpu *vcpu, u64 addr, u64 *sptes, *root_level = vcpu->arch.mmu->root_role.level; - tdp_mmu_for_each_pte(iter, mmu, gfn, gfn + 1) { + tdp_mmu_for_each_pte(iter, mmu, false, gfn, gfn + 1) { leaf = iter.level; sptes[leaf] = iter.old_spte; } @@ -1829,7 +2066,10 @@ u64 *kvm_tdp_mmu_fast_pf_get_last_sptep(struct kvm_vcpu *vcpu, u64 addr, gfn_t gfn = addr >> PAGE_SHIFT; tdp_ptep_t sptep = NULL; - tdp_mmu_for_each_pte(iter, mmu, gfn, gfn + 1) { + /* fast page fault for private GPA isn't supported. */ + WARN_ON_ONCE(kvm_is_private_gpa(vcpu->kvm, addr)); + + tdp_mmu_for_each_pte(iter, mmu, false, gfn, gfn + 1) { *spte = iter.old_spte; sptep = iter.sptep; } diff --git a/arch/x86/kvm/mmu/tdp_mmu.h b/arch/x86/kvm/mmu/tdp_mmu.h index b3cf58a50357..bc9124737142 100644 --- a/arch/x86/kvm/mmu/tdp_mmu.h +++ b/arch/x86/kvm/mmu/tdp_mmu.h @@ -10,7 +10,7 @@ void kvm_mmu_init_tdp_mmu(struct kvm *kvm); void kvm_mmu_uninit_tdp_mmu(struct kvm *kvm); -hpa_t kvm_tdp_mmu_get_vcpu_root_hpa(struct kvm_vcpu *vcpu); +hpa_t kvm_tdp_mmu_get_vcpu_root_hpa(struct kvm_vcpu *vcpu, bool private); __must_check static inline bool kvm_tdp_mmu_get_root(struct kvm_mmu_page *root) { diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index d399009ef1d7..e27c22449d85 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -201,6 +201,7 @@ struct page *kvm_pfn_to_refcounted_page(kvm_pfn_t pfn) return NULL; } +EXPORT_SYMBOL_GPL(kvm_pfn_to_refcounted_page); /* * Switches to specified vcpu, until a matching vcpu_put() From patchwork Mon Feb 26 08:26:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571492 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 28C6A612D8; Mon, 26 Feb 2024 08:28:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.13 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936107; cv=none; b=tDjUwdQZoUNe0KbmJcRwbKCU3lLmVn/nl7Gn6yMF2PyZ1r+R+bwPup94xn6mEK7mC1RsQMNqYSU4IYcHuA1oRM2mRV1AWPaX/ulTbvQzcg8LmqyRiUKXQ3QpkEI9gJjqz0+xj9qL7LK+QHgVCxq0KiaKt89rYnYDsUm9dh1UPfQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936107; c=relaxed/simple; bh=VWeErb8XXD2SBqMy8L7JRBcxtOCdpKU+Rt6/TTjVA8Q=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=bNCh4IZAiQQ2ohquMBuctk9x5chJolpnKOTvGuSctCUUTpon/AHy4jSACtiF7UCq4YsrqYX1L1zwnYj8Js2R0y0rX0SgAWn+rCjY0IqgZN24AL9qHeRP9j+xuJYdO+igf/CRzc9qTV2ar/S+g4e8ceWzLiCiJ+f3++eFjCq91UI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=LP9kL6Hf; arc=none smtp.client-ip=192.198.163.13 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="LP9kL6Hf" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936105; x=1740472105; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=VWeErb8XXD2SBqMy8L7JRBcxtOCdpKU+Rt6/TTjVA8Q=; b=LP9kL6HfpQkCOFu72Fo/6V1G8P2/RMyqlwS6xn1NqFI+Z0GE+gD4O5FW ddwzW0OQ5zFY1ku6uZ2P0Scqn/ucdZ6bAQvXoVQs7JbOH9WmRfjEeiGBA 8FgUNmBsY2F/XT9+E017Fb3aVy2Q5QtpN0yXFaFwvc8em6emngqqDik2f vs1zJh7rF5lnpsQKVKyXq6hx5hBepkZjLaC9X3V34d0sKaVbjpCS3/dJR 1kOuxSlDF1i9WDhq7jeOQnVkcLTFn5jfUp4OkdDGUbfcMNa1KosYfkvG4 plRbcLaoRdSJCnmPjysYZLMAsufGvy86vpO7FO3NNTyazpbaJaudTFkZj g==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="6155470" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6155470" Received: from orviesa009.jf.intel.com ([10.64.159.149]) by fmvoesa107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:20 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6615935" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa009-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:20 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 063/130] [MARKER] The start of TDX KVM patch series: TDX EPT violation Date: Mon, 26 Feb 2024 00:26:05 -0800 Message-Id: <605cd09c62227230c84e79016626331294f438f3.1708933498.git.isaku.yamahata@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata This empty commit is to mark the start of patch series of TDX EPT violation. Signed-off-by: Isaku Yamahata --- Documentation/virt/kvm/intel-tdx-layer-status.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/Documentation/virt/kvm/intel-tdx-layer-status.rst b/Documentation/virt/kvm/intel-tdx-layer-status.rst index 7903473abad1..c4d67dd9ddf8 100644 --- a/Documentation/virt/kvm/intel-tdx-layer-status.rst +++ b/Documentation/virt/kvm/intel-tdx-layer-status.rst @@ -20,11 +20,11 @@ Patch Layer status * TDX architectural definitions: Applied * TD VM creation/destruction: Applied * TD vcpu creation/destruction: Applied -* TDX EPT violation: Not yet +* TDX EPT violation: Applying * TD finalization: Not yet * TD vcpu enter/exit: Not yet * TD vcpu interrupts/exit/hypercall: Not yet * KVM MMU GPA shared bits: Applied * KVM TDP refactoring for TDX: Applied -* KVM TDP MMU hooks: Applying +* KVM TDP MMU hooks: Applied From patchwork Mon Feb 26 08:26:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571494 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E89B161677; Mon, 26 Feb 2024 08:28:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.13 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936108; cv=none; b=CCQFa7pr+PdW5NyjdNtOAKzwbi0F8/ojCwrPsCFrbxliPoDtJfnr6he6WxNrbvTSPvL+xRwyUaDap4C6fBRMLB1oigJwArXm8Fj1ioKM3xqOBJ686l/b0QmA7cfOncn5gyPZ6sHR7cI5cGq44xPw23z4hv3jMTpg4+4dMNJeUrw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936108; c=relaxed/simple; bh=BKyLEUsx8ByDpiKr07UAt4uH6XYvHHe1y8EuoJK+8NM=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=GW0R9FfeSbe2BodRw0gl9SI+bBJV5H052zXy+x1I5W7gE1sgBKZ4kS6ydYnr3mr5dw8SvRnYV2lpWy/rxLAyEhNUuF6drcdknRi71uenkDSTx8G5pf5JbfQTiIS8zBAQW1QqAKMj98wtuOaa6Bxxgb48rhPdUPHt033mmXfZYyc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=HiBKXFRO; arc=none smtp.client-ip=192.198.163.13 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="HiBKXFRO" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936106; x=1740472106; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=BKyLEUsx8ByDpiKr07UAt4uH6XYvHHe1y8EuoJK+8NM=; b=HiBKXFRO/hqTtSXK3xYmu8jqaGlmz3Mah5Yx28xC7TOekyLjLGwQPCCX +qMkvDijsYCZjLGNjwjSDU1HPkOUDttwb2u3yHR0WtGkrLLG7INWnFO0m +hcyRlVBOd7phYs/9PqqzVEVGo8htb53JAay/Yf8lnB8hFC4sG2VB1xmS 1ZuoSVz5ZQy6mPop/ttM5gFSQ9uxXJyaf6Jsbo+yX1uVIQSGNiTDTay72 lHkIehQLn9J3ic5iqGC9jLa4ReGsBSvK1JMzXFeouarX2HmgXDecFERvD YSVI27pbLLJSGNHMaKpwtV5E0C6SVCRIsAsldToqSlTw9Qng3NXBUFDkh w==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="6155476" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6155476" Received: from orviesa009.jf.intel.com ([10.64.159.149]) by fmvoesa107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:21 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6615946" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa009-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:21 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com, Yan Zhao , Yuan Yao , Binbin Wu Subject: [PATCH v19 064/130] KVM: x86/mmu: Do not enable page track for TD guest Date: Mon, 26 Feb 2024 00:26:06 -0800 Message-Id: <066ce30f61d65f3a5e701f0cf1c2f71fa3df2a0d.1708933498.git.isaku.yamahata@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Yan Zhao TDX does not support write protection and hence page track. Though !tdp_enabled and kvm_shadow_root_allocated(kvm) are always false for TD guest, should also return false when external write tracking is enabled. Cc: Yuan Yao Signed-off-by: Yan Zhao Reviewed-by: Binbin Wu --- v19: - drop TDX: from the short log - Added reviewed-by: BinBin Signed-off-by: Isaku Yamahata --- arch/x86/kvm/mmu/page_track.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/arch/x86/kvm/mmu/page_track.c b/arch/x86/kvm/mmu/page_track.c index c87da11f3a04..ce698ab213c1 100644 --- a/arch/x86/kvm/mmu/page_track.c +++ b/arch/x86/kvm/mmu/page_track.c @@ -22,6 +22,9 @@ bool kvm_page_track_write_tracking_enabled(struct kvm *kvm) { + if (kvm->arch.vm_type == KVM_X86_TDX_VM) + return false; + return IS_ENABLED(CONFIG_KVM_EXTERNAL_WRITE_TRACKING) || !tdp_enabled || kvm_shadow_root_allocated(kvm); } From patchwork Mon Feb 26 08:26:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571487 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D236160871; Mon, 26 Feb 2024 08:28:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.19 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936104; cv=none; b=PgXssba69nVAdtcPB+bGbZPWIW4FHIa+u3+VfPs44C5SauM8IJ5nP6sVhpDL4uiLvRxJuZirvHaSxT2lp+tGdGev6V9VrQK0caV0kTbexi72UItj1tqThR6/ImxsA9gqkRfTJO2iK0o7i/VQ/1oOnEMbI8PoJ5q4M5nsM8kV7Zc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936104; c=relaxed/simple; bh=vYP5GsQKWqAKjPBvrpOeKL1gl92Mny66GvJCS5Chluo=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Zfre4WhHYivrqOy/USY3hDWB8oIr6TPA/NqZB4LDCLn+1dae35SFV166KLbM2IzeNQP0y4/jXvsGQmnmSpaqu16sBfGeZMuuorm7U3yt6IMZTuExFFNA70tJx2kwBvNYahE0KUvkiFTxeDjIm+bRgqNXngHXFfPPMqpaFB/YvX8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=PiSERU1/; arc=none smtp.client-ip=198.175.65.19 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="PiSERU1/" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936103; x=1740472103; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=vYP5GsQKWqAKjPBvrpOeKL1gl92Mny66GvJCS5Chluo=; b=PiSERU1/MQ2oIszD7CPYfDdVbAVu1iI0YNHMfmvKFFACKI7knf1AeCRg Ak0LfXH2NZvr44m0lR4tyyIe9budvuqZuLjvKVts/315TD58iGN+f2R60 2fSYQUIOI8LQ4UqdLGIuCvxC98PSDLVE+wZfAruVy54G8alLkLxsWO9aS z2SEmN7LCmQto3MY6SfsVA/HPHEt4/3xYhRTg5c6v5GN4bk3SIBOVhvCD SI4c7DfRBhaVfp8XG/U7F1OmUsQppgC9ovkRul6Cp5S5ESoicEaDYD3he hga55+Vzbb4pBBf4005SJk1kw41a9SajUz/Q7bc0tEaC7mMBeVqPDioQM w==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="3069443" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="3069443" Received: from orviesa004.jf.intel.com ([10.64.159.144]) by orvoesa111.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:22 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="11272333" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:22 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com, Sean Christopherson , Binbin Wu Subject: [PATCH v19 065/130] KVM: VMX: Split out guts of EPT violation to common/exposed function Date: Mon, 26 Feb 2024 00:26:07 -0800 Message-Id: <527220db2f197df57e67fa291272347e24a04386.1708933498.git.isaku.yamahata@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Sean Christopherson The difference of TDX EPT violation is how to retrieve information, GPA, and exit qualification. To share the code to handle EPT violation, split out the guts of EPT violation handler so that VMX/TDX exit handler can call it after retrieving GPA and exit qualification. Signed-off-by: Sean Christopherson Signed-off-by: Isaku Yamahata Reviewed-by: Paolo Bonzini Reviewed-by: Kai Huang Reviewed-by: Binbin Wu --- arch/x86/kvm/vmx/common.h | 33 +++++++++++++++++++++++++++++++++ arch/x86/kvm/vmx/vmx.c | 25 +++---------------------- 2 files changed, 36 insertions(+), 22 deletions(-) create mode 100644 arch/x86/kvm/vmx/common.h diff --git a/arch/x86/kvm/vmx/common.h b/arch/x86/kvm/vmx/common.h new file mode 100644 index 000000000000..235908f3e044 --- /dev/null +++ b/arch/x86/kvm/vmx/common.h @@ -0,0 +1,33 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +#ifndef __KVM_X86_VMX_COMMON_H +#define __KVM_X86_VMX_COMMON_H + +#include + +#include "mmu.h" + +static inline int __vmx_handle_ept_violation(struct kvm_vcpu *vcpu, gpa_t gpa, + unsigned long exit_qualification) +{ + u64 error_code; + + /* Is it a read fault? */ + error_code = (exit_qualification & EPT_VIOLATION_ACC_READ) + ? PFERR_USER_MASK : 0; + /* Is it a write fault? */ + error_code |= (exit_qualification & EPT_VIOLATION_ACC_WRITE) + ? PFERR_WRITE_MASK : 0; + /* Is it a fetch fault? */ + error_code |= (exit_qualification & EPT_VIOLATION_ACC_INSTR) + ? PFERR_FETCH_MASK : 0; + /* ept page table entry is present? */ + error_code |= (exit_qualification & EPT_VIOLATION_RWX_MASK) + ? PFERR_PRESENT_MASK : 0; + + error_code |= (exit_qualification & EPT_VIOLATION_GVA_TRANSLATED) != 0 ? + PFERR_GUEST_FINAL_MASK : PFERR_GUEST_PAGE_MASK; + + return kvm_mmu_page_fault(vcpu, gpa, error_code, NULL, 0); +} + +#endif /* __KVM_X86_VMX_COMMON_H */ diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 6fe895bd7807..162bb134aae6 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -50,6 +50,7 @@ #include #include "capabilities.h" +#include "common.h" #include "cpuid.h" #include "hyperv.h" #include "kvm_onhyperv.h" @@ -5779,11 +5780,8 @@ static int handle_task_switch(struct kvm_vcpu *vcpu) static int handle_ept_violation(struct kvm_vcpu *vcpu) { - unsigned long exit_qualification; + unsigned long exit_qualification = vmx_get_exit_qual(vcpu); gpa_t gpa; - u64 error_code; - - exit_qualification = vmx_get_exit_qual(vcpu); /* * EPT violation happened while executing iret from NMI, @@ -5798,23 +5796,6 @@ static int handle_ept_violation(struct kvm_vcpu *vcpu) gpa = vmcs_read64(GUEST_PHYSICAL_ADDRESS); trace_kvm_page_fault(vcpu, gpa, exit_qualification); - - /* Is it a read fault? */ - error_code = (exit_qualification & EPT_VIOLATION_ACC_READ) - ? PFERR_USER_MASK : 0; - /* Is it a write fault? */ - error_code |= (exit_qualification & EPT_VIOLATION_ACC_WRITE) - ? PFERR_WRITE_MASK : 0; - /* Is it a fetch fault? */ - error_code |= (exit_qualification & EPT_VIOLATION_ACC_INSTR) - ? PFERR_FETCH_MASK : 0; - /* ept page table entry is present? */ - error_code |= (exit_qualification & EPT_VIOLATION_RWX_MASK) - ? PFERR_PRESENT_MASK : 0; - - error_code |= (exit_qualification & EPT_VIOLATION_GVA_TRANSLATED) != 0 ? - PFERR_GUEST_FINAL_MASK : PFERR_GUEST_PAGE_MASK; - vcpu->arch.exit_qualification = exit_qualification; /* @@ -5828,7 +5809,7 @@ static int handle_ept_violation(struct kvm_vcpu *vcpu) if (unlikely(allow_smaller_maxphyaddr && !kvm_vcpu_is_legal_gpa(vcpu, gpa))) return kvm_emulate_instruction(vcpu, 0); - return kvm_mmu_page_fault(vcpu, gpa, error_code, NULL, 0); + return __vmx_handle_ept_violation(vcpu, gpa, exit_qualification); } static int handle_ept_misconfig(struct kvm_vcpu *vcpu) From patchwork Mon Feb 26 08:26:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571491 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3731660DCA; Mon, 26 Feb 2024 08:28:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.19 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936106; cv=none; b=jKMTEMqQUhHhZLTA8F7UdamRYEAy8QSoSaNRpYVbGQ4Fa1mfiq3XsZUyGw2xyaK4Zuxv0kQgygZRW+0UYuArxiIvU9lQXIcaQ/Ff6J9H61LwvhmgKwd7clt7vj/NqTE5JlsoQM7tBhiNdSENmWtDDrQlvCFBRuql7o0RrGYLTdI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936106; c=relaxed/simple; bh=f2K3xma6cqKdxbAmy7RDBYX7WxriqJwDIAHVJ97Ro9w=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=eHPzV/ga2jRQye/orTE6mvIQzrqWtalHTgfOZgwXE21VMdBA3Dh1vxuXorc86Xwtamit5SLeTUDz33vmrz7qnsY8kIEUZ671A5hWJYm17hrlPufqiokpr+XH3GAeVBq/rtf3G7FpENqFXTieONdCrcodCmcRlOAZyQkfLDPpPS4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=EF+SgFz7; arc=none smtp.client-ip=198.175.65.19 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="EF+SgFz7" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936104; x=1740472104; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=f2K3xma6cqKdxbAmy7RDBYX7WxriqJwDIAHVJ97Ro9w=; b=EF+SgFz7nBfY2vGHVofJ/si57Cr4qaeSvjIed5qW9+zZ80HiTysqLhwl zXz3P+BZzX7+UwGYm1EQTHy64ymJDHEW/2cyJdSjDsKEEyT2rQpJfxuzb dhqShygbcRqjGGFqc5pVe/DrgMMYi/0ha8nBtXnaUtpUfMTEuAzLq9oaN e6i+DjPARau2o8AFyqWUvUKDsjJQNiFbT/eLxOydZyRpOsKhRR4ZU8d9T xJn1QKJKwKMAxMITHGsbsdXrMFRrkorQsSxfq8TCRWLAfaTtzr7HBgblt A72/dO1d5fHqoUpiuMnGFFIyucZwMr9dCt7yGuggMuD/OWvg0hTV1qyIA Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="3069450" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="3069450" Received: from orviesa004.jf.intel.com ([10.64.159.144]) by orvoesa111.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:24 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="11272338" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:23 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 066/130] KVM: TDX: Add accessors VMX VMCS helpers Date: Mon, 26 Feb 2024 00:26:08 -0800 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata TDX defines SEAMCALL APIs to access TDX control structures corresponding to the VMX VMCS. Introduce helper accessors to hide its SEAMCALL ABI details. Signed-off-by: Isaku Yamahata --- v19: - deleted unnecessary stub functions, tdvps_state_non_arch_check() and tdvps_management_check(). --- arch/x86/kvm/vmx/tdx.h | 92 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 92 insertions(+) diff --git a/arch/x86/kvm/vmx/tdx.h b/arch/x86/kvm/vmx/tdx.h index d3077151252c..8a0d1bfe34a0 100644 --- a/arch/x86/kvm/vmx/tdx.h +++ b/arch/x86/kvm/vmx/tdx.h @@ -58,6 +58,98 @@ static inline struct vcpu_tdx *to_tdx(struct kvm_vcpu *vcpu) return container_of(vcpu, struct vcpu_tdx, vcpu); } +static __always_inline void tdvps_vmcs_check(u32 field, u8 bits) +{ +#define VMCS_ENC_ACCESS_TYPE_MASK 0x1UL +#define VMCS_ENC_ACCESS_TYPE_FULL 0x0UL +#define VMCS_ENC_ACCESS_TYPE_HIGH 0x1UL +#define VMCS_ENC_ACCESS_TYPE(field) ((field) & VMCS_ENC_ACCESS_TYPE_MASK) + + /* TDX is 64bit only. HIGH field isn't supported. */ + BUILD_BUG_ON_MSG(__builtin_constant_p(field) && + VMCS_ENC_ACCESS_TYPE(field) == VMCS_ENC_ACCESS_TYPE_HIGH, + "Read/Write to TD VMCS *_HIGH fields not supported"); + + BUILD_BUG_ON(bits != 16 && bits != 32 && bits != 64); + +#define VMCS_ENC_WIDTH_MASK GENMASK(14, 13) +#define VMCS_ENC_WIDTH_16BIT (0UL << 13) +#define VMCS_ENC_WIDTH_64BIT (1UL << 13) +#define VMCS_ENC_WIDTH_32BIT (2UL << 13) +#define VMCS_ENC_WIDTH_NATURAL (3UL << 13) +#define VMCS_ENC_WIDTH(field) ((field) & VMCS_ENC_WIDTH_MASK) + + /* TDX is 64bit only. i.e. natural width = 64bit. */ + BUILD_BUG_ON_MSG(bits != 64 && __builtin_constant_p(field) && + (VMCS_ENC_WIDTH(field) == VMCS_ENC_WIDTH_64BIT || + VMCS_ENC_WIDTH(field) == VMCS_ENC_WIDTH_NATURAL), + "Invalid TD VMCS access for 64-bit field"); + BUILD_BUG_ON_MSG(bits != 32 && __builtin_constant_p(field) && + VMCS_ENC_WIDTH(field) == VMCS_ENC_WIDTH_32BIT, + "Invalid TD VMCS access for 32-bit field"); + BUILD_BUG_ON_MSG(bits != 16 && __builtin_constant_p(field) && + VMCS_ENC_WIDTH(field) == VMCS_ENC_WIDTH_16BIT, + "Invalid TD VMCS access for 16-bit field"); +} + +#define TDX_BUILD_TDVPS_ACCESSORS(bits, uclass, lclass) \ +static __always_inline u##bits td_##lclass##_read##bits(struct vcpu_tdx *tdx, \ + u32 field) \ +{ \ + struct tdx_module_args out; \ + u64 err; \ + \ + tdvps_##lclass##_check(field, bits); \ + err = tdh_vp_rd(tdx->tdvpr_pa, TDVPS_##uclass(field), &out); \ + if (KVM_BUG_ON(err, tdx->vcpu.kvm)) { \ + pr_err("TDH_VP_RD["#uclass".0x%x] failed: 0x%llx\n", \ + field, err); \ + return 0; \ + } \ + return (u##bits)out.r8; \ +} \ +static __always_inline void td_##lclass##_write##bits(struct vcpu_tdx *tdx, \ + u32 field, u##bits val) \ +{ \ + struct tdx_module_args out; \ + u64 err; \ + \ + tdvps_##lclass##_check(field, bits); \ + err = tdh_vp_wr(tdx->tdvpr_pa, TDVPS_##uclass(field), val, \ + GENMASK_ULL(bits - 1, 0), &out); \ + if (KVM_BUG_ON(err, tdx->vcpu.kvm)) \ + pr_err("TDH_VP_WR["#uclass".0x%x] = 0x%llx failed: 0x%llx\n", \ + field, (u64)val, err); \ +} \ +static __always_inline void td_##lclass##_setbit##bits(struct vcpu_tdx *tdx, \ + u32 field, u64 bit) \ +{ \ + struct tdx_module_args out; \ + u64 err; \ + \ + tdvps_##lclass##_check(field, bits); \ + err = tdh_vp_wr(tdx->tdvpr_pa, TDVPS_##uclass(field), bit, bit, &out); \ + if (KVM_BUG_ON(err, tdx->vcpu.kvm)) \ + pr_err("TDH_VP_WR["#uclass".0x%x] |= 0x%llx failed: 0x%llx\n", \ + field, bit, err); \ +} \ +static __always_inline void td_##lclass##_clearbit##bits(struct vcpu_tdx *tdx, \ + u32 field, u64 bit) \ +{ \ + struct tdx_module_args out; \ + u64 err; \ + \ + tdvps_##lclass##_check(field, bits); \ + err = tdh_vp_wr(tdx->tdvpr_pa, TDVPS_##uclass(field), 0, bit, &out); \ + if (KVM_BUG_ON(err, tdx->vcpu.kvm)) \ + pr_err("TDH_VP_WR["#uclass".0x%x] &= ~0x%llx failed: 0x%llx\n", \ + field, bit, err); \ +} + +TDX_BUILD_TDVPS_ACCESSORS(16, VMCS, vmcs); +TDX_BUILD_TDVPS_ACCESSORS(32, VMCS, vmcs); +TDX_BUILD_TDVPS_ACCESSORS(64, VMCS, vmcs); + static __always_inline u64 td_tdcs_exec_read64(struct kvm_tdx *kvm_tdx, u32 field) { struct tdx_module_args out; From patchwork Mon Feb 26 08:26:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571493 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4843D612E9; Mon, 26 Feb 2024 08:28:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.19 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936107; cv=none; b=dmVpCdUd9iZg9MoLK6YvHsgVEj0HAnzs4lwXEK+a+uU+gBzonhwM4xhkEY07tGyNFr3O+u+lVGhXT1OHze9KjEsZdVt0MmqBOAiC5Cb8YP9UHLw3G2RdAoIYf7Pvl3+oeXyFGm5mU60U9EYgFLCoWcpZvWa89QQTiUQRLd9dXLA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936107; c=relaxed/simple; bh=wWbhRZCirvnTpoR7lG5s4eITsW63C8P/4FabG1p/CFY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=vA9DtJmiF3Btqgxb5A0x0HAHaMG1V6EtFrsObdPKeLabL6r7PFgs+hIAG2tKdYbmouChRD8XM7OZN4upQQ/Lfq0S2kCno+NNctbrNX+8i2y8CfaoHGsrI+XuwD5yGlZFOWP+/HTTFdEKiaTMBI2bjhajCnICynjYmxjBZpyFJCg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=aIcQMgsv; arc=none smtp.client-ip=198.175.65.19 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="aIcQMgsv" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936106; x=1740472106; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=wWbhRZCirvnTpoR7lG5s4eITsW63C8P/4FabG1p/CFY=; b=aIcQMgsvBi0xaOZk6Q8j+bmyuRUUHtNOiGzlDxQC05zUoik7puaubR/G aZOjJSJOcmwfJrXBNhnBHPeIKLTB1qcBgfslfKTIoexJiTCJJMzOuyeU9 2OwZblamdyzE5BC35qErdsHabqWbvI2WoNniaABZuUFKdejtSd/s2X59t QaxXEn6H4S1WNt/2jChBHIO5Ep5cliM12529Ap45RWup9AAgfJh482r+m hjd08H6MCRo/FTtPwZVheCQD3ha7LoFIe4gttONSXvKSt7XeB/HeKiPqA aRZFe1ZuQxqLaDAoijaq8S0fnH2kUEDxy9rIvBh/viUk/ugQwuffrrQIK A==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="3069454" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="3069454" Received: from orviesa004.jf.intel.com ([10.64.159.144]) by orvoesa111.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:25 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="11272356" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:25 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com, Sean Christopherson Subject: [PATCH v19 067/130] KVM: TDX: Add load_mmu_pgd method for TDX Date: Mon, 26 Feb 2024 00:26:09 -0800 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Sean Christopherson For virtual IO, the guest TD shares guest pages with VMM without encryption. Shared EPT is used to map guest pages in unprotected way. Add the VMCS field encoding for the shared EPTP, which will be used by TDX to have separate EPT walks for private GPAs (existing EPTP) versus shared GPAs (new shared EPTP). Set shared EPT pointer value for the TDX guest to initialize TDX MMU. Signed-off-by: Sean Christopherson Signed-off-by: Isaku Yamahata Reviewed-by: Paolo Bonzini --- v19: - Add WARN_ON_ONCE() to tdx_load_mmu_pgd() and drop unconditional mask --- arch/x86/include/asm/vmx.h | 1 + arch/x86/kvm/vmx/main.c | 13 ++++++++++++- arch/x86/kvm/vmx/tdx.c | 6 ++++++ arch/x86/kvm/vmx/x86_ops.h | 4 ++++ 4 files changed, 23 insertions(+), 1 deletion(-) diff --git a/arch/x86/include/asm/vmx.h b/arch/x86/include/asm/vmx.h index f703bae0c4ac..9deb663a42e3 100644 --- a/arch/x86/include/asm/vmx.h +++ b/arch/x86/include/asm/vmx.h @@ -236,6 +236,7 @@ enum vmcs_field { TSC_MULTIPLIER_HIGH = 0x00002033, TERTIARY_VM_EXEC_CONTROL = 0x00002034, TERTIARY_VM_EXEC_CONTROL_HIGH = 0x00002035, + SHARED_EPT_POINTER = 0x0000203C, PID_POINTER_TABLE = 0x00002042, PID_POINTER_TABLE_HIGH = 0x00002043, GUEST_PHYSICAL_ADDRESS = 0x00002400, diff --git a/arch/x86/kvm/vmx/main.c b/arch/x86/kvm/vmx/main.c index d0f75020579f..076a471d9aea 100644 --- a/arch/x86/kvm/vmx/main.c +++ b/arch/x86/kvm/vmx/main.c @@ -123,6 +123,17 @@ static void vt_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event) vmx_vcpu_reset(vcpu, init_event); } +static void vt_load_mmu_pgd(struct kvm_vcpu *vcpu, hpa_t root_hpa, + int pgd_level) +{ + if (is_td_vcpu(vcpu)) { + tdx_load_mmu_pgd(vcpu, root_hpa, pgd_level); + return; + } + + vmx_load_mmu_pgd(vcpu, root_hpa, pgd_level); +} + static int vt_mem_enc_ioctl(struct kvm *kvm, void __user *argp) { if (!is_td(kvm)) @@ -256,7 +267,7 @@ struct kvm_x86_ops vt_x86_ops __initdata = { .write_tsc_offset = vmx_write_tsc_offset, .write_tsc_multiplier = vmx_write_tsc_multiplier, - .load_mmu_pgd = vmx_load_mmu_pgd, + .load_mmu_pgd = vt_load_mmu_pgd, .check_intercept = vmx_check_intercept, .handle_exit_irqoff = vmx_handle_exit_irqoff, diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c index 54e0d4efa2bd..143a3c2a16bc 100644 --- a/arch/x86/kvm/vmx/tdx.c +++ b/arch/x86/kvm/vmx/tdx.c @@ -453,6 +453,12 @@ void tdx_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event) */ } +void tdx_load_mmu_pgd(struct kvm_vcpu *vcpu, hpa_t root_hpa, int pgd_level) +{ + WARN_ON_ONCE(root_hpa & ~PAGE_MASK); + td_vmcs_write64(to_tdx(vcpu), SHARED_EPT_POINTER, root_hpa); +} + static int tdx_get_capabilities(struct kvm_tdx_cmd *cmd) { struct kvm_tdx_capabilities __user *user_caps; diff --git a/arch/x86/kvm/vmx/x86_ops.h b/arch/x86/kvm/vmx/x86_ops.h index f5820f617b2e..24161fa404aa 100644 --- a/arch/x86/kvm/vmx/x86_ops.h +++ b/arch/x86/kvm/vmx/x86_ops.h @@ -152,6 +152,8 @@ void tdx_vcpu_free(struct kvm_vcpu *vcpu); void tdx_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event); int tdx_vcpu_ioctl(struct kvm_vcpu *vcpu, void __user *argp); + +void tdx_load_mmu_pgd(struct kvm_vcpu *vcpu, hpa_t root_hpa, int root_level); #else static inline int tdx_hardware_setup(struct kvm_x86_ops *x86_ops) { return -EOPNOTSUPP; } static inline void tdx_hardware_unsetup(void) {} @@ -173,6 +175,8 @@ static inline void tdx_vcpu_free(struct kvm_vcpu *vcpu) {} static inline void tdx_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event) {} static inline int tdx_vcpu_ioctl(struct kvm_vcpu *vcpu, void __user *argp) { return -EOPNOTSUPP; } + +static inline void tdx_load_mmu_pgd(struct kvm_vcpu *vcpu, hpa_t root_hpa, int root_level) {} #endif #endif /* __KVM_X86_VMX_X86_OPS_H */ From patchwork Mon Feb 26 08:26:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571497 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3BE5861691; Mon, 26 Feb 2024 08:28:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.19 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936109; cv=none; b=affIA8d/ZYFA2eF8wWTb3kHFNl3QwL8F2jBf71xDg8P1gjMQBEMUSJwHxzRsuRljpvS4Zeol4rnKvEIkLxWnWtSGJseqVTAdKJbdVC10gxsdhAH4Q1bqu5mTuQQvWIp8IDiXJLH1hJ83QEVauKcqvNIo2AhQ3wrXvfYyaY0FTlM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936109; c=relaxed/simple; bh=mBPPHS+6lnLS94iBXvUmiGc7FJFyMVT0Ja3kQbVsMDM=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=ZrM2FezaNOj9vgmlrQxznU+cIbE4B/HUJ82wZ6LFyMAy8dPGiNltk/P76gAqguQtlwLhWiV5+mj+3m5cwNlTSZjVhQSEKezjhkdhisOcblyC7klbhjSrdV2K+pWYaCUI0XzaXs7h7Vqq6lP1a4IOJa8/KAvjrBletvcjynZAelg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=OWIjBoO6; arc=none smtp.client-ip=198.175.65.19 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="OWIjBoO6" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936106; x=1740472106; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=mBPPHS+6lnLS94iBXvUmiGc7FJFyMVT0Ja3kQbVsMDM=; b=OWIjBoO6vIdzlfkl7eKa0Kv4NslV+wxRp7//Lv40urC+sBUsthqLIi0r 8lB9C7dNSoJlan7qyJ2zeK27wXPaZYj1pAsF0x+LiU1tffIPmilJwjXwh rLkZaGb3e59p8FA70F+WL9zOg9jIPbY96AoC7Y2gOHCInUnobis4QIxvi hl5pHEKYyZY5XsuVDBdfXuv0qVm9ffPRLGUH/CMlUx9ZX+/vCAjIbuYKx U5sG/VfzeSKkuRIMHqzVz16gU5kGZ3EJRu2lw3oqdXqzIUgRRWEljv0aZ y6DUs+GWSyZytwRdJAGaQq69P76v5tRdaKTCD0UllZxlyFWJ5/Y2eTkhT Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="3069455" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="3069455" Received: from orviesa004.jf.intel.com ([10.64.159.144]) by orvoesa111.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:26 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="11272376" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:26 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com, Yuan Yao Subject: [PATCH v19 068/130] KVM: TDX: Retry seamcall when TDX_OPERAND_BUSY with operand SEPT Date: Mon, 26 Feb 2024 00:26:10 -0800 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Yuan Yao TDX module internally uses locks to protect internal resources. It tries to acquire the locks. If it fails to obtain the lock, it returns TDX_OPERAND_BUSY error without spin because its execution time limitation. TDX SEAMCALL API reference describes what resources are used. It's known which TDX SEAMCALL can cause contention with which resources. VMM can avoid contention inside the TDX module by avoiding contentious TDX SEAMCALL with, for example, spinlock. Because OS knows better its process scheduling and its scalability, a lock at OS/VMM layer would work better than simply retrying TDX SEAMCALLs. TDH.MEM.* API except for TDH.MEM.TRACK operates on a secure EPT tree and the TDX module internally tries to acquire the lock of the secure EPT tree. They return TDX_OPERAND_BUSY | TDX_OPERAND_ID_SEPT in case of failure to get the lock. TDX KVM allows sept callbacks to return error so that TDP MMU layer can retry. Retry TDX TDH.MEM.* API on the error because the error is a rare event caused by zero-step attack mitigation. Signed-off-by: Yuan Yao Signed-off-by: Isaku Yamahata --- v19: - fix typo TDG.VP.ENTER => TDH.VP.ENTER, TDX_OPRRAN_BUSY => TDX_OPERAND_BUSY - drop the description on TDH.VP.ENTER as this patch doesn't touch TDH.VP.ENTER --- arch/x86/kvm/vmx/tdx_ops.h | 48 +++++++++++++++++++++++++++++++------- 1 file changed, 39 insertions(+), 9 deletions(-) diff --git a/arch/x86/kvm/vmx/tdx_ops.h b/arch/x86/kvm/vmx/tdx_ops.h index d80212b1daf3..e5c069b96126 100644 --- a/arch/x86/kvm/vmx/tdx_ops.h +++ b/arch/x86/kvm/vmx/tdx_ops.h @@ -44,6 +44,36 @@ static inline u64 tdx_seamcall(u64 op, struct tdx_module_args *in, void pr_tdx_error(u64 op, u64 error_code, const struct tdx_module_args *out); #endif +/* + * TDX module acquires its internal lock for resources. It doesn't spin to get + * locks because of its restrictions of allowed execution time. Instead, it + * returns TDX_OPERAND_BUSY with an operand id. + * + * Multiple VCPUs can operate on SEPT. Also with zero-step attack mitigation, + * TDH.VP.ENTER may rarely acquire SEPT lock and release it when zero-step + * attack is suspected. It results in TDX_OPERAND_BUSY | TDX_OPERAND_ID_SEPT + * with TDH.MEM.* operation. Note: TDH.MEM.TRACK is an exception. + * + * Because TDP MMU uses read lock for scalability, spin lock around SEAMCALL + * spoils TDP MMU effort. Retry several times with the assumption that SEPT + * lock contention is rare. But don't loop forever to avoid lockup. Let TDP + * MMU retry. + */ +#define TDX_ERROR_SEPT_BUSY (TDX_OPERAND_BUSY | TDX_OPERAND_ID_SEPT) + +static inline u64 tdx_seamcall_sept(u64 op, struct tdx_module_args *in, + struct tdx_module_args *out) +{ +#define SEAMCALL_RETRY_MAX 16 + int retry = SEAMCALL_RETRY_MAX; + u64 ret; + + do { + ret = tdx_seamcall(op, in, out); + } while (ret == TDX_ERROR_SEPT_BUSY && retry-- > 0); + return ret; +} + static inline u64 tdh_mng_addcx(hpa_t tdr, hpa_t addr) { struct tdx_module_args in = { @@ -66,7 +96,7 @@ static inline u64 tdh_mem_page_add(hpa_t tdr, gpa_t gpa, hpa_t hpa, hpa_t source }; clflush_cache_range(__va(hpa), PAGE_SIZE); - return tdx_seamcall(TDH_MEM_PAGE_ADD, &in, out); + return tdx_seamcall_sept(TDH_MEM_PAGE_ADD, &in, out); } static inline u64 tdh_mem_sept_add(hpa_t tdr, gpa_t gpa, int level, hpa_t page, @@ -79,7 +109,7 @@ static inline u64 tdh_mem_sept_add(hpa_t tdr, gpa_t gpa, int level, hpa_t page, }; clflush_cache_range(__va(page), PAGE_SIZE); - return tdx_seamcall(TDH_MEM_SEPT_ADD, &in, out); + return tdx_seamcall_sept(TDH_MEM_SEPT_ADD, &in, out); } static inline u64 tdh_mem_sept_rd(hpa_t tdr, gpa_t gpa, int level, @@ -90,7 +120,7 @@ static inline u64 tdh_mem_sept_rd(hpa_t tdr, gpa_t gpa, int level, .rdx = tdr, }; - return tdx_seamcall(TDH_MEM_SEPT_RD, &in, out); + return tdx_seamcall_sept(TDH_MEM_SEPT_RD, &in, out); } static inline u64 tdh_mem_sept_remove(hpa_t tdr, gpa_t gpa, int level, @@ -101,7 +131,7 @@ static inline u64 tdh_mem_sept_remove(hpa_t tdr, gpa_t gpa, int level, .rdx = tdr, }; - return tdx_seamcall(TDH_MEM_SEPT_REMOVE, &in, out); + return tdx_seamcall_sept(TDH_MEM_SEPT_REMOVE, &in, out); } static inline u64 tdh_vp_addcx(hpa_t tdvpr, hpa_t addr) @@ -125,7 +155,7 @@ static inline u64 tdh_mem_page_relocate(hpa_t tdr, gpa_t gpa, hpa_t hpa, }; clflush_cache_range(__va(hpa), PAGE_SIZE); - return tdx_seamcall(TDH_MEM_PAGE_RELOCATE, &in, out); + return tdx_seamcall_sept(TDH_MEM_PAGE_RELOCATE, &in, out); } static inline u64 tdh_mem_page_aug(hpa_t tdr, gpa_t gpa, hpa_t hpa, @@ -138,7 +168,7 @@ static inline u64 tdh_mem_page_aug(hpa_t tdr, gpa_t gpa, hpa_t hpa, }; clflush_cache_range(__va(hpa), PAGE_SIZE); - return tdx_seamcall(TDH_MEM_PAGE_AUG, &in, out); + return tdx_seamcall_sept(TDH_MEM_PAGE_AUG, &in, out); } static inline u64 tdh_mem_range_block(hpa_t tdr, gpa_t gpa, int level, @@ -149,7 +179,7 @@ static inline u64 tdh_mem_range_block(hpa_t tdr, gpa_t gpa, int level, .rdx = tdr, }; - return tdx_seamcall(TDH_MEM_RANGE_BLOCK, &in, out); + return tdx_seamcall_sept(TDH_MEM_RANGE_BLOCK, &in, out); } static inline u64 tdh_mng_key_config(hpa_t tdr) @@ -299,7 +329,7 @@ static inline u64 tdh_mem_page_remove(hpa_t tdr, gpa_t gpa, int level, .rdx = tdr, }; - return tdx_seamcall(TDH_MEM_PAGE_REMOVE, &in, out); + return tdx_seamcall_sept(TDH_MEM_PAGE_REMOVE, &in, out); } static inline u64 tdh_sys_lp_shutdown(void) @@ -327,7 +357,7 @@ static inline u64 tdh_mem_range_unblock(hpa_t tdr, gpa_t gpa, int level, .rdx = tdr, }; - return tdx_seamcall(TDH_MEM_RANGE_UNBLOCK, &in, out); + return tdx_seamcall_sept(TDH_MEM_RANGE_UNBLOCK, &in, out); } static inline u64 tdh_phymem_cache_wb(bool resume) From patchwork Mon Feb 26 08:26:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571496 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 33518626A7; Mon, 26 Feb 2024 08:28:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.19 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936109; cv=none; b=XS5FBJ7wVGAbht0Ne6DC8ypofqUasDdVXH8kpBELWxGHxCtur2wTs/Pzy4DvAO7PJfn7BEpSdyBrx0Z0zYqv4ObwRjxmIFacm8/aOwmr3Np4UnJ11yPuDQjjjJntcYeu6L1ju5W2aAsn6WyfoCnLparP6OuMewx9Wty6u80mM88= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936109; c=relaxed/simple; bh=ItvOXDST0y8Qvf46BwFZbHkP1zwOR0dJGkEPf5xC34E=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=QhczXVsQtosaGzGQzxGidfvxvsG3SQ4SjklY4yHQFwmra7COrU8KGc26YKjGkYLmoY9BQir1ShcrL90M+uzWMFmS0LgkbNjNcPC7vl7fmMllPklyY6Iz+fzNN3mKbKPH7TVXd41igvte3I6YJIrakXS4AhCDUTSdLFoa1EbBFAs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=JdCitgEa; arc=none smtp.client-ip=198.175.65.19 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="JdCitgEa" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936108; x=1740472108; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=ItvOXDST0y8Qvf46BwFZbHkP1zwOR0dJGkEPf5xC34E=; b=JdCitgEaKrL00iRIetnhGe2THkHFOxpTszG4272V2pCojn9Qj8h3NbRX DO9xxr54c6YMLB96uy9oe75XYLZLpHNg0pu/ibyqxXmhl7+H/ufrYoyPo raim+waACoKB19wBbdHP8N6r8OmBTioEVxuy6j03eLJTDgdJk4OlB8hsD 2j+DyLIpcQ48VS/Q4rbWW+FTczQjT9enaY/JACy1V9uh0WyLuZphoiLQF xroQghiB+XV8kskJnpVDUbY9MouqNZYSkymZULITrejF8twawoq/P4vyR KR2x5cmp/lS9pqYZVYAVf1CJ1tyF1mEfRXAaqqumoQp3h3OrLwgDiugYS g==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="3069460" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="3069460" Received: from orviesa004.jf.intel.com ([10.64.159.144]) by orvoesa111.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:27 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="11272394" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:26 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 069/130] KVM: TDX: Require TDP MMU and mmio caching for TDX Date: Mon, 26 Feb 2024 00:26:11 -0800 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata As TDP MMU is becoming main stream than the legacy MMU, the legacy MMU support for TDX isn't implemented. TDX requires KVM mmio caching. Disable TDX support when TDP MMU or mmio caching aren't supported. Signed-off-by: Isaku Yamahata --- arch/x86/kvm/mmu/mmu.c | 1 + arch/x86/kvm/vmx/main.c | 13 +++++++++++++ 2 files changed, 14 insertions(+) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 0e0321ad9ca2..b8d6ce02e66d 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -104,6 +104,7 @@ module_param_named(flush_on_reuse, force_flush_and_sync_on_reuse, bool, 0644); * If the hardware supports that we don't need to do shadow paging. */ bool tdp_enabled = false; +EXPORT_SYMBOL_GPL(tdp_enabled); static bool __ro_after_init tdp_mmu_allowed; diff --git a/arch/x86/kvm/vmx/main.c b/arch/x86/kvm/vmx/main.c index 076a471d9aea..54df6653193e 100644 --- a/arch/x86/kvm/vmx/main.c +++ b/arch/x86/kvm/vmx/main.c @@ -3,6 +3,7 @@ #include "x86_ops.h" #include "vmx.h" +#include "mmu.h" #include "nested.h" #include "pmu.h" #include "tdx.h" @@ -36,6 +37,18 @@ static __init int vt_hardware_setup(void) if (ret) return ret; + /* TDX requires KVM TDP MMU. */ + if (enable_tdx && !tdp_enabled) { + enable_tdx = false; + pr_warn_ratelimited("TDX requires TDP MMU. Please enable TDP MMU for TDX.\n"); + } + + /* TDX requires MMIO caching. */ + if (enable_tdx && !enable_mmio_caching) { + enable_tdx = false; + pr_warn_ratelimited("TDX requires mmio caching. Please enable mmio caching for TDX.\n"); + } + enable_tdx = enable_tdx && !tdx_hardware_setup(&vt_x86_ops); if (enable_tdx) vt_x86_ops.vm_size = max_t(unsigned int, vt_x86_ops.vm_size, From patchwork Mon Feb 26 08:26:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571498 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C976862A09; Mon, 26 Feb 2024 08:28:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.19 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936112; cv=none; b=HUzqyskCbVV1FzZRX7/COu6H/9fhfCelFWuxwk+4zaaJSKvFBXP0dmG4T6OpKmVMGhR7zVUbfemTJAtn0Ls8j5s2FaqGol7jkC4cOEuM30hwtjWcZQDyqhuE2jxs1ibJ5mXVRmXoovW1b3BPqevouAFNdYkNj47CpF5Ws1e6xNE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936112; c=relaxed/simple; bh=cOIa/c1mL8lEIWnCrohNsbjcGldly9dZiHJor/YrCwI=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=HU9G95YMqpmuo6kVjTk/sfPkifCrYPaBeZFZtnvudJXiI+JUScA4i/RjuxSSptCAzp4liLPxXNTAK0nsu+nRUBaGKABDwHt2mz0k/Z8iiOnmKAOCVz4AF2C7CVdZvCgDBZUYEPcqhxrTOGWAFqwhyNyhPCV1O/+g7Pg+pgCtxsU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=ZSLl1hQt; arc=none smtp.client-ip=198.175.65.19 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="ZSLl1hQt" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936109; x=1740472109; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=cOIa/c1mL8lEIWnCrohNsbjcGldly9dZiHJor/YrCwI=; b=ZSLl1hQtcDA8vxIMqPQZQPp0ithGY9yxwHvW22sRc9eR/sDDRrYAW8f9 JLsNcoTL8Wq+677w9Ey/te+0/Di0v6jFrcZs2Hcjmcok4Q4tavuUPcBHr A77GCb5eV1uISO/eQUhcRcC5IXHK/F/e0nu8OniR9EptxzM9XxzU94RdW ChgDSm1cf4X1vOIYGH+2qEL0FTyIT4UxuZunvsRU05YeI4jVziHTiZXYU dFbPs1bVivaMaNbnd167dIA6cgVPIxqAd4LSQ1XTjQh5w/lvhGQPeURiz O1gpalC5K23w16ZaAzc75xilvgCeHdIGOxQ2Zo3ZOwUt6z2N6hFT0obz6 g==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="3069465" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="3069465" Received: from orviesa004.jf.intel.com ([10.64.159.144]) by orvoesa111.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:29 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="11272420" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:28 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 070/130] KVM: TDX: TDP MMU TDX support Date: Mon, 26 Feb 2024 00:26:12 -0800 Message-Id: <56cdb0da8bbf17dc293a2a6b4ff74f6e3e034bbd.1708933498.git.isaku.yamahata@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata Implement hooks of TDP MMU for TDX backend. TLB flush, TLB shootdown, propagating the change private EPT entry to Secure EPT and freeing Secure EPT page. TLB flush handles both shared EPT and private EPT. It flushes shared EPT same as VMX. It also waits for the TDX TLB shootdown. For the hook to free Secure EPT page, unlinks the Secure EPT page from the Secure EPT so that the page can be freed to OS. Propagate the entry change to Secure EPT. The possible entry changes are present -> non-present(zapping) and non-present -> present(population). On population just link the Secure EPT page or the private guest page to the Secure EPT by TDX SEAMCALL. Because TDP MMU allows concurrent zapping/population, zapping requires synchronous TLB shoot down with the frozen EPT entry. It zaps the secure entry, increments TLB counter, sends IPI to remote vcpus to trigger TLB flush, and then unlinks the private guest page from the Secure EPT. For simplicity, batched zapping with exclude lock is handled as concurrent zapping. Although it's inefficient, it can be optimized in the future. For MMIO SPTE, the spte value changes as follows. initial value (suppress VE bit is set) -> Guest issues MMIO and triggers EPT violation -> KVM updates SPTE value to MMIO value (suppress VE bit is cleared) -> Guest MMIO resumes. It triggers VE exception in guest TD -> Guest VE handler issues TDG.VP.VMCALL -> KVM handles MMIO -> Guest VE handler resumes its execution after MMIO instruction Signed-off-by: Isaku Yamahata --- v19: - Compile fix when CONFIG_HYPERV != y. It's due to the following patch. Catch it up. https://lore.kernel.org/all/20231018192325.1893896-1-seanjc@google.com/ - Add comments on tlb shootdown to explan the sequence. - Use gmem_max_level callback, delete tdp_max_page_level. v18: - rename tdx_sept_page_aug() -> tdx_mem_page_aug() - checkpatch: space => tab v15 -> v16: - Add the handling of TD_ATTR_SEPT_VE_DISABLE case. v14 -> v15: - Implemented tdx_flush_tlb_current() - Removed unnecessary invept in tdx_flush_tlb(). It was carry over from the very old code base. Signed-off-by: Isaku Yamahata --- arch/x86/kvm/mmu/spte.c | 3 +- arch/x86/kvm/vmx/main.c | 91 ++++++++- arch/x86/kvm/vmx/tdx.c | 372 +++++++++++++++++++++++++++++++++++++ arch/x86/kvm/vmx/tdx.h | 2 +- arch/x86/kvm/vmx/tdx_ops.h | 6 + arch/x86/kvm/vmx/x86_ops.h | 13 ++ 6 files changed, 481 insertions(+), 6 deletions(-) diff --git a/arch/x86/kvm/mmu/spte.c b/arch/x86/kvm/mmu/spte.c index 318135daf685..83926a35ea47 100644 --- a/arch/x86/kvm/mmu/spte.c +++ b/arch/x86/kvm/mmu/spte.c @@ -74,7 +74,8 @@ u64 make_mmio_spte(struct kvm_vcpu *vcpu, u64 gfn, unsigned int access) u64 spte = generation_mmio_spte_mask(gen); u64 gpa = gfn << PAGE_SHIFT; - WARN_ON_ONCE(!vcpu->kvm->arch.shadow_mmio_value); + WARN_ON_ONCE(!vcpu->kvm->arch.shadow_mmio_value && + !kvm_gfn_shared_mask(vcpu->kvm)); access &= shadow_mmio_access_mask; spte |= vcpu->kvm->arch.shadow_mmio_value | access; diff --git a/arch/x86/kvm/vmx/main.c b/arch/x86/kvm/vmx/main.c index 54df6653193e..8c5bac3defdf 100644 --- a/arch/x86/kvm/vmx/main.c +++ b/arch/x86/kvm/vmx/main.c @@ -29,6 +29,10 @@ static int vt_max_vcpus(struct kvm *kvm) return kvm->max_vcpus; } +#if IS_ENABLED(CONFIG_HYPERV) +static int vt_flush_remote_tlbs(struct kvm *kvm); +#endif + static __init int vt_hardware_setup(void) { int ret; @@ -49,11 +53,29 @@ static __init int vt_hardware_setup(void) pr_warn_ratelimited("TDX requires mmio caching. Please enable mmio caching for TDX.\n"); } +#if IS_ENABLED(CONFIG_HYPERV) + /* + * TDX KVM overrides flush_remote_tlbs method and assumes + * flush_remote_tlbs_range = NULL that falls back to + * flush_remote_tlbs. Disable TDX if there are conflicts. + */ + if (vt_x86_ops.flush_remote_tlbs || + vt_x86_ops.flush_remote_tlbs_range) { + enable_tdx = false; + pr_warn_ratelimited("TDX requires baremetal. Not Supported on VMM guest.\n"); + } +#endif + enable_tdx = enable_tdx && !tdx_hardware_setup(&vt_x86_ops); if (enable_tdx) vt_x86_ops.vm_size = max_t(unsigned int, vt_x86_ops.vm_size, sizeof(struct kvm_tdx)); +#if IS_ENABLED(CONFIG_HYPERV) + if (enable_tdx) + vt_x86_ops.flush_remote_tlbs = vt_flush_remote_tlbs; +#endif + return 0; } @@ -136,6 +158,56 @@ static void vt_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event) vmx_vcpu_reset(vcpu, init_event); } +static void vt_flush_tlb_all(struct kvm_vcpu *vcpu) +{ + if (is_td_vcpu(vcpu)) { + tdx_flush_tlb(vcpu); + return; + } + + vmx_flush_tlb_all(vcpu); +} + +static void vt_flush_tlb_current(struct kvm_vcpu *vcpu) +{ + if (is_td_vcpu(vcpu)) { + tdx_flush_tlb_current(vcpu); + return; + } + + vmx_flush_tlb_current(vcpu); +} + +#if IS_ENABLED(CONFIG_HYPERV) +static int vt_flush_remote_tlbs(struct kvm *kvm) +{ + if (is_td(kvm)) + return tdx_sept_flush_remote_tlbs(kvm); + + /* + * fallback to KVM_REQ_TLB_FLUSH. + * See kvm_arch_flush_remote_tlb() and kvm_flush_remote_tlbs(). + */ + return -EOPNOTSUPP; +} +#endif + +static void vt_flush_tlb_gva(struct kvm_vcpu *vcpu, gva_t addr) +{ + if (KVM_BUG_ON(is_td_vcpu(vcpu), vcpu->kvm)) + return; + + vmx_flush_tlb_gva(vcpu, addr); +} + +static void vt_flush_tlb_guest(struct kvm_vcpu *vcpu) +{ + if (is_td_vcpu(vcpu)) + return; + + vmx_flush_tlb_guest(vcpu); +} + static void vt_load_mmu_pgd(struct kvm_vcpu *vcpu, hpa_t root_hpa, int pgd_level) { @@ -163,6 +235,15 @@ static int vt_vcpu_mem_enc_ioctl(struct kvm_vcpu *vcpu, void __user *argp) return tdx_vcpu_ioctl(vcpu, argp); } +static int vt_gmem_max_level(struct kvm *kvm, kvm_pfn_t pfn, gfn_t gfn, + bool is_private, u8 *max_level) +{ + if (is_td(kvm)) + return tdx_gmem_max_level(kvm, pfn, gfn, is_private, max_level); + + return 0; +} + #define VMX_REQUIRED_APICV_INHIBITS \ (BIT(APICV_INHIBIT_REASON_DISABLE)| \ BIT(APICV_INHIBIT_REASON_ABSENT) | \ @@ -228,10 +309,10 @@ struct kvm_x86_ops vt_x86_ops __initdata = { .set_rflags = vmx_set_rflags, .get_if_flag = vmx_get_if_flag, - .flush_tlb_all = vmx_flush_tlb_all, - .flush_tlb_current = vmx_flush_tlb_current, - .flush_tlb_gva = vmx_flush_tlb_gva, - .flush_tlb_guest = vmx_flush_tlb_guest, + .flush_tlb_all = vt_flush_tlb_all, + .flush_tlb_current = vt_flush_tlb_current, + .flush_tlb_gva = vt_flush_tlb_gva, + .flush_tlb_guest = vt_flush_tlb_guest, .vcpu_pre_run = vmx_vcpu_pre_run, .vcpu_run = vmx_vcpu_run, @@ -324,6 +405,8 @@ struct kvm_x86_ops vt_x86_ops __initdata = { .mem_enc_ioctl = vt_mem_enc_ioctl, .vcpu_mem_enc_ioctl = vt_vcpu_mem_enc_ioctl, + + .gmem_max_level = vt_gmem_max_level, }; struct kvm_x86_init_ops vt_init_ops __initdata = { diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c index 143a3c2a16bc..39ef80857b6a 100644 --- a/arch/x86/kvm/vmx/tdx.c +++ b/arch/x86/kvm/vmx/tdx.c @@ -8,6 +8,7 @@ #include "mmu.h" #include "tdx_arch.h" #include "tdx.h" +#include "vmx.h" #include "x86.h" #undef pr_fmt @@ -364,6 +365,19 @@ static int tdx_do_tdh_mng_key_config(void *param) int tdx_vm_init(struct kvm *kvm) { + /* + * Because guest TD is protected, VMM can't parse the instruction in TD. + * Instead, guest uses MMIO hypercall. For unmodified device driver, + * #VE needs to be injected for MMIO and #VE handler in TD converts MMIO + * instruction into MMIO hypercall. + * + * SPTE value for MMIO needs to be setup so that #VE is injected into + * TD instead of triggering EPT MISCONFIG. + * - RWX=0 so that EPT violation is triggered. + * - suppress #VE bit is cleared to inject #VE. + */ + kvm_mmu_set_mmio_spte_value(kvm, 0); + /* * This function initializes only KVM software construct. It doesn't * initialize TDX stuff, e.g. TDCS, TDR, TDCX, HKID etc. @@ -459,6 +473,307 @@ void tdx_load_mmu_pgd(struct kvm_vcpu *vcpu, hpa_t root_hpa, int pgd_level) td_vmcs_write64(to_tdx(vcpu), SHARED_EPT_POINTER, root_hpa); } +static void tdx_unpin(struct kvm *kvm, kvm_pfn_t pfn) +{ + struct page *page = pfn_to_page(pfn); + + put_page(page); +} + +static int tdx_mem_page_aug(struct kvm *kvm, gfn_t gfn, + enum pg_level level, kvm_pfn_t pfn) +{ + int tdx_level = pg_level_to_tdx_sept_level(level); + struct kvm_tdx *kvm_tdx = to_kvm_tdx(kvm); + union tdx_sept_level_state level_state; + hpa_t hpa = pfn_to_hpa(pfn); + gpa_t gpa = gfn_to_gpa(gfn); + struct tdx_module_args out; + union tdx_sept_entry entry; + u64 err; + + err = tdh_mem_page_aug(kvm_tdx->tdr_pa, gpa, hpa, &out); + if (unlikely(err == TDX_ERROR_SEPT_BUSY)) { + tdx_unpin(kvm, pfn); + return -EAGAIN; + } + if (unlikely(err == (TDX_EPT_ENTRY_STATE_INCORRECT | TDX_OPERAND_ID_RCX))) { + entry.raw = out.rcx; + level_state.raw = out.rdx; + if (level_state.level == tdx_level && + level_state.state == TDX_SEPT_PENDING && + entry.leaf && entry.pfn == pfn && entry.sve) { + tdx_unpin(kvm, pfn); + WARN_ON_ONCE(!(to_kvm_tdx(kvm)->attributes & + TDX_TD_ATTR_SEPT_VE_DISABLE)); + return -EAGAIN; + } + } + if (KVM_BUG_ON(err, kvm)) { + pr_tdx_error(TDH_MEM_PAGE_AUG, err, &out); + tdx_unpin(kvm, pfn); + return -EIO; + } + + return 0; +} + +static int tdx_sept_set_private_spte(struct kvm *kvm, gfn_t gfn, + enum pg_level level, kvm_pfn_t pfn) +{ + struct kvm_tdx *kvm_tdx = to_kvm_tdx(kvm); + + /* TODO: handle large pages. */ + if (KVM_BUG_ON(level != PG_LEVEL_4K, kvm)) + return -EINVAL; + + /* + * Because restricted mem doesn't support page migration with + * a_ops->migrate_page (yet), no callback isn't triggered for KVM on + * page migration. Until restricted mem supports page migration, + * prevent page migration. + * TODO: Once restricted mem introduces callback on page migration, + * implement it and remove get_page/put_page(). + */ + get_page(pfn_to_page(pfn)); + + if (likely(is_td_finalized(kvm_tdx))) + return tdx_mem_page_aug(kvm, gfn, level, pfn); + + /* TODO: tdh_mem_page_add() comes here for the initial memory. */ + + return 0; +} + +static int tdx_sept_drop_private_spte(struct kvm *kvm, gfn_t gfn, + enum pg_level level, kvm_pfn_t pfn) +{ + int tdx_level = pg_level_to_tdx_sept_level(level); + struct kvm_tdx *kvm_tdx = to_kvm_tdx(kvm); + struct tdx_module_args out; + gpa_t gpa = gfn_to_gpa(gfn); + hpa_t hpa = pfn_to_hpa(pfn); + hpa_t hpa_with_hkid; + u64 err; + + /* TODO: handle large pages. */ + if (KVM_BUG_ON(level != PG_LEVEL_4K, kvm)) + return -EINVAL; + + if (unlikely(!is_hkid_assigned(kvm_tdx))) { + /* + * The HKID assigned to this TD was already freed and cache + * was already flushed. We don't have to flush again. + */ + err = tdx_reclaim_page(hpa); + if (KVM_BUG_ON(err, kvm)) + return -EIO; + tdx_unpin(kvm, pfn); + return 0; + } + + do { + /* + * When zapping private page, write lock is held. So no race + * condition with other vcpu sept operation. Race only with + * TDH.VP.ENTER. + */ + err = tdh_mem_page_remove(kvm_tdx->tdr_pa, gpa, tdx_level, &out); + } while (unlikely(err == TDX_ERROR_SEPT_BUSY)); + if (KVM_BUG_ON(err, kvm)) { + pr_tdx_error(TDH_MEM_PAGE_REMOVE, err, &out); + return -EIO; + } + + hpa_with_hkid = set_hkid_to_hpa(hpa, (u16)kvm_tdx->hkid); + do { + /* + * TDX_OPERAND_BUSY can happen on locking PAMT entry. Because + * this page was removed above, other thread shouldn't be + * repeatedly operating on this page. Just retry loop. + */ + err = tdh_phymem_page_wbinvd(hpa_with_hkid); + } while (unlikely(err == (TDX_OPERAND_BUSY | TDX_OPERAND_ID_RCX))); + if (KVM_BUG_ON(err, kvm)) { + pr_tdx_error(TDH_PHYMEM_PAGE_WBINVD, err, NULL); + return -EIO; + } + tdx_clear_page(hpa); + tdx_unpin(kvm, pfn); + return 0; +} + +static int tdx_sept_link_private_spt(struct kvm *kvm, gfn_t gfn, + enum pg_level level, void *private_spt) +{ + int tdx_level = pg_level_to_tdx_sept_level(level); + struct kvm_tdx *kvm_tdx = to_kvm_tdx(kvm); + gpa_t gpa = gfn_to_gpa(gfn); + hpa_t hpa = __pa(private_spt); + struct tdx_module_args out; + u64 err; + + err = tdh_mem_sept_add(kvm_tdx->tdr_pa, gpa, tdx_level, hpa, &out); + if (unlikely(err == TDX_ERROR_SEPT_BUSY)) + return -EAGAIN; + if (KVM_BUG_ON(err, kvm)) { + pr_tdx_error(TDH_MEM_SEPT_ADD, err, &out); + return -EIO; + } + + return 0; +} + +static int tdx_sept_zap_private_spte(struct kvm *kvm, gfn_t gfn, + enum pg_level level) +{ + int tdx_level = pg_level_to_tdx_sept_level(level); + struct kvm_tdx *kvm_tdx = to_kvm_tdx(kvm); + gpa_t gpa = gfn_to_gpa(gfn) & KVM_HPAGE_MASK(level); + struct tdx_module_args out; + u64 err; + + /* This can be called when destructing guest TD after freeing HKID. */ + if (unlikely(!is_hkid_assigned(kvm_tdx))) + return 0; + + /* For now large page isn't supported yet. */ + WARN_ON_ONCE(level != PG_LEVEL_4K); + err = tdh_mem_range_block(kvm_tdx->tdr_pa, gpa, tdx_level, &out); + if (unlikely(err == TDX_ERROR_SEPT_BUSY)) + return -EAGAIN; + if (KVM_BUG_ON(err, kvm)) { + pr_tdx_error(TDH_MEM_RANGE_BLOCK, err, &out); + return -EIO; + } + return 0; +} + +/* + * TLB shoot down procedure: + * There is a global epoch counter and each vcpu has local epoch counter. + * - TDH.MEM.RANGE.BLOCK(TDR. level, range) on one vcpu + * This blocks the subsequenct creation of TLB translation on that range. + * This corresponds to clear the present bit(all RXW) in EPT entry + * - TDH.MEM.TRACK(TDR): advances the epoch counter which is global. + * - IPI to remote vcpus + * - TDExit and re-entry with TDH.VP.ENTER on remote vcpus + * - On re-entry, TDX module compares the local epoch counter with the global + * epoch counter. If the local epoch counter is older than the global epoch + * counter, update the local epoch counter and flushes TLB. + */ +static void tdx_track(struct kvm *kvm) +{ + struct kvm_tdx *kvm_tdx = to_kvm_tdx(kvm); + u64 err; + + KVM_BUG_ON(!is_hkid_assigned(kvm_tdx), kvm); + /* If TD isn't finalized, it's before any vcpu running. */ + if (unlikely(!is_td_finalized(kvm_tdx))) + return; + + /* + * tdx_flush_tlb() waits for this function to issue TDH.MEM.TRACK() by + * the counter. The counter is used instead of bool because multiple + * TDH_MEM_TRACK() can be issued concurrently by multiple vcpus. + * + * optimization: The TLB shoot down procedure described in The TDX + * specification is, TDH.MEM.TRACK(), send IPI to remote vcpus, confirm + * all remote vcpus exit to VMM, and execute vcpu, both local and + * remote. Twist the sequence to reduce IPI overhead as follows. + * + * local remote + * ----- ------ + * increment tdh_mem_track + * + * request KVM_REQ_TLB_FLUSH + * send IPI + * + * TDEXIT to KVM due to IPI + * + * IPI handler calls tdx_flush_tlb() + * to process KVM_REQ_TLB_FLUSH. + * spin wait for tdh_mem_track == 0 + * + * TDH.MEM.TRACK() + * + * decrement tdh_mem_track + * + * complete KVM_REQ_TLB_FLUSH + * + * TDH.VP.ENTER to flush tlbs TDH.VP.ENTER to flush tlbs + */ + atomic_inc(&kvm_tdx->tdh_mem_track); + /* + * KVM_REQ_TLB_FLUSH waits for the empty IPI handler, ack_flush(), with + * KVM_REQUEST_WAIT. + */ + kvm_make_all_cpus_request(kvm, KVM_REQ_TLB_FLUSH); + + do { + err = tdh_mem_track(kvm_tdx->tdr_pa); + } while (unlikely((err & TDX_SEAMCALL_STATUS_MASK) == TDX_OPERAND_BUSY)); + + /* Release remote vcpu waiting for TDH.MEM.TRACK in tdx_flush_tlb(). */ + atomic_dec(&kvm_tdx->tdh_mem_track); + + if (KVM_BUG_ON(err, kvm)) + pr_tdx_error(TDH_MEM_TRACK, err, NULL); + +} + +static int tdx_sept_free_private_spt(struct kvm *kvm, gfn_t gfn, + enum pg_level level, void *private_spt) +{ + struct kvm_tdx *kvm_tdx = to_kvm_tdx(kvm); + + /* + * The HKID assigned to this TD was already freed and cache was + * already flushed. We don't have to flush again. + */ + if (!is_hkid_assigned(kvm_tdx)) + return tdx_reclaim_page(__pa(private_spt)); + + /* + * free_private_spt() is (obviously) called when a shadow page is being + * zapped. KVM doesn't (yet) zap private SPs while the TD is active. + * Note: This function is for private shadow page. Not for private + * guest page. private guest page can be zapped during TD is active. + * shared <-> private conversion and slot move/deletion. + */ + KVM_BUG_ON(is_hkid_assigned(kvm_tdx), kvm); + return -EINVAL; +} + +int tdx_sept_flush_remote_tlbs(struct kvm *kvm) +{ + if (unlikely(!is_td(kvm))) + return -EOPNOTSUPP; + + if (is_hkid_assigned(to_kvm_tdx(kvm))) + tdx_track(kvm); + + return 0; +} + +static int tdx_sept_remove_private_spte(struct kvm *kvm, gfn_t gfn, + enum pg_level level, kvm_pfn_t pfn) +{ + /* + * TDX requires TLB tracking before dropping private page. Do + * it here, although it is also done later. + * If hkid isn't assigned, the guest is destroying and no vcpu + * runs further. TLB shootdown isn't needed. + * + * TODO: Call TDH.MEM.TRACK() only when we have called + * TDH.MEM.RANGE.BLOCK(), but not call TDH.MEM.TRACK() yet. + */ + if (is_hkid_assigned(to_kvm_tdx(kvm))) + tdx_track(kvm); + + return tdx_sept_drop_private_spte(kvm, gfn, level, pfn); +} + static int tdx_get_capabilities(struct kvm_tdx_cmd *cmd) { struct kvm_tdx_capabilities __user *user_caps; @@ -924,6 +1239,39 @@ static int tdx_td_init(struct kvm *kvm, struct kvm_tdx_cmd *cmd) return ret; } +void tdx_flush_tlb(struct kvm_vcpu *vcpu) +{ + /* + * Don't need to flush shared EPTP: + * "TD VCPU TLB Address Spaced Identifier" in the TDX module spec: + * The TLB entries for TD are tagged with: + * SEAM (1 bit) + * VPID + * Secure EPT root (51:12 bits) with HKID = 0 + * PCID + * for *both* Secure-EPT and Shared-EPT. + * TLB flush with Secure-EPT root by tdx_track() results in flushing + * the conversion of both Secure-EPT and Shared-EPT. + */ + + /* + * See tdx_track(). Wait for tlb shootdown initiater to finish + * TDH_MEM_TRACK() so that shared-EPT/secure-EPT TLB is flushed + * on the next TDENTER. + */ + while (atomic_read(&to_kvm_tdx(vcpu->kvm)->tdh_mem_track)) + cpu_relax(); +} + +void tdx_flush_tlb_current(struct kvm_vcpu *vcpu) +{ + /* + * flush_tlb_current() is used only the first time for the vcpu to run. + * As it isn't performance critical, keep this function simple. + */ + tdx_track(vcpu->kvm); +} + int tdx_vm_ioctl(struct kvm *kvm, void __user *argp) { struct kvm_tdx_cmd tdx_cmd; @@ -1087,6 +1435,17 @@ int tdx_vcpu_ioctl(struct kvm_vcpu *vcpu, void __user *argp) return 0; } +int tdx_gmem_max_level(struct kvm *kvm, kvm_pfn_t pfn, gfn_t gfn, + bool is_private, u8 *max_level) +{ + if (!is_private) + return 0; + + /* TODO: Enable 2mb and 1gb large page support. */ + *max_level = min(*max_level, PG_LEVEL_4K); + return 0; +} + #define TDX_MD_MAP(_fid, _ptr) \ { .fid = MD_FIELD_ID_##_fid, \ .ptr = (_ptr), } @@ -1297,8 +1656,21 @@ int __init tdx_hardware_setup(struct kvm_x86_ops *x86_ops) on_each_cpu(vmx_off, &enable.enabled, true); cpus_read_unlock(); free_cpumask_var(enable.enabled); + if (r) + goto out; + + x86_ops->link_private_spt = tdx_sept_link_private_spt; + x86_ops->free_private_spt = tdx_sept_free_private_spt; + x86_ops->set_private_spte = tdx_sept_set_private_spte; + x86_ops->remove_private_spte = tdx_sept_remove_private_spte; + x86_ops->zap_private_spte = tdx_sept_zap_private_spte; + + return 0; out: + /* kfree() accepts NULL. */ + kfree(tdx_mng_key_config_lock); + tdx_mng_key_config_lock = NULL; return r; } diff --git a/arch/x86/kvm/vmx/tdx.h b/arch/x86/kvm/vmx/tdx.h index 8a0d1bfe34a0..75596b9dcf3f 100644 --- a/arch/x86/kvm/vmx/tdx.h +++ b/arch/x86/kvm/vmx/tdx.h @@ -18,6 +18,7 @@ struct kvm_tdx { int hkid; bool finalized; + atomic_t tdh_mem_track; u64 tsc_offset; }; @@ -162,7 +163,6 @@ static __always_inline u64 td_tdcs_exec_read64(struct kvm_tdx *kvm_tdx, u32 fiel } return out.r8; } - #else struct kvm_tdx { struct kvm kvm; diff --git a/arch/x86/kvm/vmx/tdx_ops.h b/arch/x86/kvm/vmx/tdx_ops.h index e5c069b96126..d27f281152cb 100644 --- a/arch/x86/kvm/vmx/tdx_ops.h +++ b/arch/x86/kvm/vmx/tdx_ops.h @@ -44,6 +44,12 @@ static inline u64 tdx_seamcall(u64 op, struct tdx_module_args *in, void pr_tdx_error(u64 op, u64 error_code, const struct tdx_module_args *out); #endif +static inline int pg_level_to_tdx_sept_level(enum pg_level level) +{ + WARN_ON_ONCE(level == PG_LEVEL_NONE); + return level - 1; +} + /* * TDX module acquires its internal lock for resources. It doesn't spin to get * locks because of its restrictions of allowed execution time. Instead, it diff --git a/arch/x86/kvm/vmx/x86_ops.h b/arch/x86/kvm/vmx/x86_ops.h index 24161fa404aa..d5f75efd87e6 100644 --- a/arch/x86/kvm/vmx/x86_ops.h +++ b/arch/x86/kvm/vmx/x86_ops.h @@ -153,7 +153,12 @@ void tdx_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event); int tdx_vcpu_ioctl(struct kvm_vcpu *vcpu, void __user *argp); +void tdx_flush_tlb(struct kvm_vcpu *vcpu); +void tdx_flush_tlb_current(struct kvm_vcpu *vcpu); +int tdx_sept_flush_remote_tlbs(struct kvm *kvm); void tdx_load_mmu_pgd(struct kvm_vcpu *vcpu, hpa_t root_hpa, int root_level); +int tdx_gmem_max_level(struct kvm *kvm, kvm_pfn_t pfn, gfn_t gfn, + bool is_private, u8 *max_level); #else static inline int tdx_hardware_setup(struct kvm_x86_ops *x86_ops) { return -EOPNOTSUPP; } static inline void tdx_hardware_unsetup(void) {} @@ -176,7 +181,15 @@ static inline void tdx_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event) {} static inline int tdx_vcpu_ioctl(struct kvm_vcpu *vcpu, void __user *argp) { return -EOPNOTSUPP; } +static inline void tdx_flush_tlb(struct kvm_vcpu *vcpu) {} +static inline void tdx_flush_tlb_current(struct kvm_vcpu *vcpu) {} +static inline int tdx_sept_flush_remote_tlbs(struct kvm *kvm) { return 0; } static inline void tdx_load_mmu_pgd(struct kvm_vcpu *vcpu, hpa_t root_hpa, int root_level) {} +static inline int tdx_gmem_max_level(struct kvm *kvm, kvm_pfn_t pfn, gfn_t gfn, + bool is_private, u8 *max_level) +{ + return -EOPNOTSUPP; +} #endif #endif /* __KVM_X86_VMX_X86_OPS_H */ From patchwork Mon Feb 26 08:26:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571499 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5E785634FA; Mon, 26 Feb 2024 08:28:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.19 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936112; cv=none; b=f5rTPIZRpCTfbZGmWOtDRsYuvzICNNOiG9jlH8DHm9o69lR3enp+9HQGYwW+uAUuZZiQQKaRrJfdqTz8SSxugZz1HuMzOOTYHPshX6tIDw1B0HjzOvcehx6BAeLIH3A8ZpILVpsoPUqCxDB75s0+XwYguTxUT8EkZyMsQv4QN18= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936112; c=relaxed/simple; bh=XCouPvykKRbiuUVkgxuTWRQsNvpWFhGpyLsG7vsW4vY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=rdYWeHrXR85bPYeVfHHxV8lb3FXv202rb5OwtUe6EyKsTTaFcjbVfz5e1596QTvNc86bbkytNWDIFPfSs+JqqdvBLY7GTm6mraMZF+LIlD+s/OtfmQyc2iOYEFQwDlicLMEoBEzZk/9n8+0Fm3oBSKLZweIzoGHkk7A4MiPcoYA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=WHGHP071; arc=none smtp.client-ip=198.175.65.19 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="WHGHP071" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936111; x=1740472111; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=XCouPvykKRbiuUVkgxuTWRQsNvpWFhGpyLsG7vsW4vY=; b=WHGHP071EefHT2T9wCscumgBTW2PK8o/Oijrcj7hNLrmJLYGJVoSMhwy tBgc6pyPcbgW3IuAWyYyh6JEl96okGAkoXzu5/meJPlx2TUCavXmIbcZ+ pVjrw/CvdCzxMWpF9fZKM6RY1Lnvx7uhXv3HTph/wExytsgcW0EhPzWae ghTaF90IG/jBA2en0Y2n8/7FFuG6Q7frtUmeXeTjG4JgxjAzkOOXR8IP9 juG7jpTucfm3tjepEWUCLTdrrHpvv5Cxk9a6Owfz+8eyIYUwioSW5KNbe 8dJkq9r1uqrGbLo3oxjS5uLbaRGlYOVBBQQyALk/LKd55+FUKkoUCWLD3 w==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="3069469" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="3069469" Received: from orviesa004.jf.intel.com ([10.64.159.144]) by orvoesa111.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:30 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="11272440" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:30 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 071/130] KVM: TDX: MTRR: implement get_mt_mask() for TDX Date: Mon, 26 Feb 2024 00:26:13 -0800 Message-Id: <1502930eaf250202d01a1707949a2eac9682fec4.1708933498.git.isaku.yamahata@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata Because TDX virtualize cpuid[0x1].EDX[MTRR: bit 12] to fixed 1, guest TD thinks MTRR is supported. Although TDX supports only WB for private GPA, it's desirable to support MTRR for shared GPA. As guest access to MTRR MSRs causes #VE and KVM/x86 tracks the values of MTRR MSRs, the remaining part is to implement get_mt_mask method for TDX for shared GPA. Suggested-by: Kai Huang Signed-off-by: Isaku Yamahata --- v19: - typo in the commit message - Deleted stale paragraph in the commit message --- arch/x86/kvm/vmx/main.c | 10 +++++++++- arch/x86/kvm/vmx/tdx.c | 23 +++++++++++++++++++++++ arch/x86/kvm/vmx/x86_ops.h | 2 ++ 3 files changed, 34 insertions(+), 1 deletion(-) diff --git a/arch/x86/kvm/vmx/main.c b/arch/x86/kvm/vmx/main.c index 8c5bac3defdf..c5672909fdae 100644 --- a/arch/x86/kvm/vmx/main.c +++ b/arch/x86/kvm/vmx/main.c @@ -219,6 +219,14 @@ static void vt_load_mmu_pgd(struct kvm_vcpu *vcpu, hpa_t root_hpa, vmx_load_mmu_pgd(vcpu, root_hpa, pgd_level); } +static u8 vt_get_mt_mask(struct kvm_vcpu *vcpu, gfn_t gfn, bool is_mmio) +{ + if (is_td_vcpu(vcpu)) + return tdx_get_mt_mask(vcpu, gfn, is_mmio); + + return vmx_get_mt_mask(vcpu, gfn, is_mmio); +} + static int vt_mem_enc_ioctl(struct kvm *kvm, void __user *argp) { if (!is_td(kvm)) @@ -348,7 +356,7 @@ struct kvm_x86_ops vt_x86_ops __initdata = { .set_tss_addr = vmx_set_tss_addr, .set_identity_map_addr = vmx_set_identity_map_addr, - .get_mt_mask = vmx_get_mt_mask, + .get_mt_mask = vt_get_mt_mask, .get_exit_info = vmx_get_exit_info, diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c index 39ef80857b6a..e65fff43cb1b 100644 --- a/arch/x86/kvm/vmx/tdx.c +++ b/arch/x86/kvm/vmx/tdx.c @@ -393,6 +393,29 @@ int tdx_vm_init(struct kvm *kvm) return 0; } +u8 tdx_get_mt_mask(struct kvm_vcpu *vcpu, gfn_t gfn, bool is_mmio) +{ + if (is_mmio) + return MTRR_TYPE_UNCACHABLE << VMX_EPT_MT_EPTE_SHIFT; + + if (!kvm_arch_has_noncoherent_dma(vcpu->kvm)) + return (MTRR_TYPE_WRBACK << VMX_EPT_MT_EPTE_SHIFT) | VMX_EPT_IPAT_BIT; + + /* + * TDX enforces CR0.CD = 0 and KVM MTRR emulation enforces writeback. + * TODO: implement MTRR MSR emulation so that + * MTRRCap: SMRR=0: SMRR interface unsupported + * WC=0: write combining unsupported + * FIX=0: Fixed range registers unsupported + * VCNT=0: number of variable range regitsers = 0 + * MTRRDefType: E=1, FE=0, type=writeback only. Don't allow other value. + * E=1: enable MTRR + * FE=0: disable fixed range MTRRs + * type: default memory type=writeback + */ + return MTRR_TYPE_WRBACK << VMX_EPT_MT_EPTE_SHIFT; +} + int tdx_vcpu_create(struct kvm_vcpu *vcpu) { struct kvm_tdx *kvm_tdx = to_kvm_tdx(vcpu->kvm); diff --git a/arch/x86/kvm/vmx/x86_ops.h b/arch/x86/kvm/vmx/x86_ops.h index d5f75efd87e6..5335d35bc655 100644 --- a/arch/x86/kvm/vmx/x86_ops.h +++ b/arch/x86/kvm/vmx/x86_ops.h @@ -150,6 +150,7 @@ int tdx_vm_ioctl(struct kvm *kvm, void __user *argp); int tdx_vcpu_create(struct kvm_vcpu *vcpu); void tdx_vcpu_free(struct kvm_vcpu *vcpu); void tdx_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event); +u8 tdx_get_mt_mask(struct kvm_vcpu *vcpu, gfn_t gfn, bool is_mmio); int tdx_vcpu_ioctl(struct kvm_vcpu *vcpu, void __user *argp); @@ -178,6 +179,7 @@ static inline int tdx_vm_ioctl(struct kvm *kvm, void __user *argp) { return -EOP static inline int tdx_vcpu_create(struct kvm_vcpu *vcpu) { return -EOPNOTSUPP; } static inline void tdx_vcpu_free(struct kvm_vcpu *vcpu) {} static inline void tdx_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event) {} +static inline u8 tdx_get_mt_mask(struct kvm_vcpu *vcpu, gfn_t gfn, bool is_mmio) { return 0; } static inline int tdx_vcpu_ioctl(struct kvm_vcpu *vcpu, void __user *argp) { return -EOPNOTSUPP; } From patchwork Mon Feb 26 08:26:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571500 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D3A7E657C3; Mon, 26 Feb 2024 08:28:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.19 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936113; cv=none; b=CiiuA4G/7VfzlpJC5ZAZxpYlepRcxHmkxuie5o6ZkwxxgNBKsnLZr603lj84o3W0OYvA05NxPN9YC80FwSZHO1vscyA8tKYUbv2kFZd6huNIqi7C8E2s9Pp+VHvhT+ECT2Nrg0YDGE6iLy014qi6zhvsaRScI8YCHaRby0cNySM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936113; c=relaxed/simple; bh=TZ+z91imsScQD0p1/VsSa1GqbET/iM4ZKqAoChx+Rzg=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=tOSPfaeSh0OCfXTNb8YS7PtZHc6Dw2QEd3er6594Dti3WL4cT+dK2Bh6vUqjQoAluLVgcZAaxs0HPXdsPoXNxHa2oEwvfIXtRCqs8J7br4QQuIqrsexwvSknHN12tiHWKhzcBoVXJ6jdlCnEIuZlmTnAnM9M1mVWQPFLD6diPz4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=dCuGSrxg; arc=none smtp.client-ip=198.175.65.19 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="dCuGSrxg" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936112; x=1740472112; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=TZ+z91imsScQD0p1/VsSa1GqbET/iM4ZKqAoChx+Rzg=; b=dCuGSrxg/9emwAZa6+REA8n1nXl4SE9b5HLrK89qtX09/ca2h9bDfQAR 97cuTUqYzPlwTGO4CVi+LjagPiJQzY+f6KY2cD9To9mXGmwRmRIABtR4H F7Jfmj47ChxY2qh22LqM8cY2YjHpeeRh17A3Uql8BnaNsK97+2vh/Y4DF r0A52LxOz/6u43tfjB1nVwqBD+Jka685pnAG8KgwJYAhFTEJ499tTb/5u VbM/PUGL2JpCX8rJLO/QTfGFQrB7d6olKk+15OnUFy/RVz3Pmse1YKBDE Sl6jZef/YMxBdJ5/CeErLg2HnPj+iuUTnMYyYDLZ2Y0ai5mZr8j9vQmBg w==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="3069475" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="3069475" Received: from orviesa004.jf.intel.com ([10.64.159.144]) by orvoesa111.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:31 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="11272446" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:30 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 072/130] [MARKER] The start of TDX KVM patch series: TD finalization Date: Mon, 26 Feb 2024 00:26:14 -0800 Message-Id: <43cd628fcbde2fd2e9e299382dc11c125b9f266c.1708933498.git.isaku.yamahata@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata This empty commit is to mark the start of patch series of TD finalization. Signed-off-by: Isaku Yamahata --- Documentation/virt/kvm/intel-tdx-layer-status.rst | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/Documentation/virt/kvm/intel-tdx-layer-status.rst b/Documentation/virt/kvm/intel-tdx-layer-status.rst index c4d67dd9ddf8..46ae049b6b85 100644 --- a/Documentation/virt/kvm/intel-tdx-layer-status.rst +++ b/Documentation/virt/kvm/intel-tdx-layer-status.rst @@ -11,6 +11,7 @@ What qemu can do - TDX VM TYPE is exposed to Qemu. - Qemu can create/destroy guest of TDX vm type. - Qemu can create/destroy vcpu of TDX vm type. +- Qemu can populate initial guest memory image. Patch Layer status ------------------ @@ -20,8 +21,8 @@ Patch Layer status * TDX architectural definitions: Applied * TD VM creation/destruction: Applied * TD vcpu creation/destruction: Applied -* TDX EPT violation: Applying -* TD finalization: Not yet +* TDX EPT violation: Applied +* TD finalization: Applying * TD vcpu enter/exit: Not yet * TD vcpu interrupts/exit/hypercall: Not yet From patchwork Mon Feb 26 08:26:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571501 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1595E65BA7; Mon, 26 Feb 2024 08:28:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.19 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936114; cv=none; b=Ur/wNuGW6ByTz1uMGIhKT0DW/kxkAChvkyKHc8e/NRlErik0x9LkYk44NtqCemhUOE6hW9S1jDmpbXd5ufWO6HKkT2Y6lQJ0K6lLiIX4As17PH5sjHLkG9iy0Nj8rb17kfNYF1nczWIFuZtUNU8aoPBl/mfPz9TYUOhwCnKzDOE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936114; c=relaxed/simple; bh=dkFZVaecF0CWHpUjngt3GvwMS5f8pMevrJfNVmUMsaA=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=IRuN5mAdTQE99CwEe51VMy230F4I25j1L2vXWf8MJDYNyo8iz+TIEPtPA5G8NYERqrAJ/yXoPFxMF6qmnmm/xmpR5Yf06YwHY3r4UWNxdotQPrJPIbN58DPLJO6aoYcDSGXIRYIvd1QDoWPISkQAjqYW78q00JE2DKk/Vaxac2M= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=DqJ8POca; arc=none smtp.client-ip=198.175.65.19 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="DqJ8POca" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936112; x=1740472112; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=dkFZVaecF0CWHpUjngt3GvwMS5f8pMevrJfNVmUMsaA=; b=DqJ8POcaoyNxYgSFI55YNRbqK/6UwME70qJAURWM3IQozfiXrAmJL0If dxqVjOqzRssR/iLCDLAdOGT/EJ/LqlRZXWnyvIsQCB323EFxjaPpqCy2y TVejVqWOl7QePvDLiZ3ElqBCE3wcf07c4mO8y8i1Y0E1CIYlLouqkr4JT xfs8A8eEYb1Gp+zISbdKT9kkCJMn7QDT03ZHcdyuO9xmtJ4CwmVQ+2EeE kjf/l0Q14seUGyc6+Kamj5qKuXIm3GcWSAgKSVkJYjcgz9tB5w3JCyXgv xZdPo1OGTP1XZec6L413sr31R3amW+a/6lOQBs6APxOZiX7t8CQjSa98J A==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="3069480" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="3069480" Received: from orviesa004.jf.intel.com ([10.64.159.144]) by orvoesa111.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:32 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="11272457" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:31 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 073/130] KVM: x86: Add hooks in kvm_arch_vcpu_memory_mapping() Date: Mon, 26 Feb 2024 00:26:15 -0800 Message-Id: <349a549e44fb03f2ec00fd9c45caed308dabbf25.1708933498.git.isaku.yamahata@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata In the case of TDX, the memory contents needs to be provided to be encrypted when populating guest memory before running the guest. Add hooks in kvm_mmu_map_tdp_page() for KVM_MEMORY_MAPPING before/after calling kvm_mmu_tdp_page(). TDX KVM will implement the hooks. Signed-off-by: Isaku Yamahata --- v19: - newly added --- arch/x86/include/asm/kvm-x86-ops.h | 2 ++ arch/x86/include/asm/kvm_host.h | 5 +++++ arch/x86/kvm/x86.c | 13 ++++++++++++- 3 files changed, 19 insertions(+), 1 deletion(-) diff --git a/arch/x86/include/asm/kvm-x86-ops.h b/arch/x86/include/asm/kvm-x86-ops.h index e1c75f8c1b25..fb3ae97c724e 100644 --- a/arch/x86/include/asm/kvm-x86-ops.h +++ b/arch/x86/include/asm/kvm-x86-ops.h @@ -151,6 +151,8 @@ KVM_X86_OP(vcpu_deliver_sipi_vector) KVM_X86_OP_OPTIONAL_RET0(vcpu_get_apicv_inhibit_reasons); KVM_X86_OP_OPTIONAL(get_untagged_addr) KVM_X86_OP_OPTIONAL_RET0(gmem_max_level) +KVM_X86_OP_OPTIONAL(pre_memory_mapping); +KVM_X86_OP_OPTIONAL(post_memory_mapping); #undef KVM_X86_OP #undef KVM_X86_OP_OPTIONAL diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index bc0767c884f7..36694e784c27 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1839,6 +1839,11 @@ struct kvm_x86_ops { int (*gmem_max_level)(struct kvm *kvm, kvm_pfn_t pfn, gfn_t gfn, bool is_private, u8 *max_level); + int (*pre_memory_mapping)(struct kvm_vcpu *vcpu, + struct kvm_memory_mapping *mapping, + u64 *error_code, u8 *max_level); + void (*post_memory_mapping)(struct kvm_vcpu *vcpu, + struct kvm_memory_mapping *mapping); }; struct kvm_x86_nested_ops { diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 2bd4b7c8fa51..23ece956c816 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -5826,10 +5826,21 @@ int kvm_arch_vcpu_memory_mapping(struct kvm_vcpu *vcpu, u8 max_level = KVM_MAX_HUGEPAGE_LEVEL; u64 error_code = PFERR_WRITE_MASK; u8 goal_level = PG_LEVEL_4K; - int r; + int r = 0; + + if (kvm_x86_ops.pre_memory_mapping) + r = static_call(kvm_x86_pre_memory_mapping)(vcpu, mapping, &error_code, &max_level); + else { + if (mapping->source) + r = -EINVAL; + } + if (r) + return r; r = kvm_mmu_map_tdp_page(vcpu, gfn_to_gpa(mapping->base_gfn), error_code, max_level, &goal_level); + if (kvm_x86_ops.post_memory_mapping) + static_call(kvm_x86_post_memory_mapping)(vcpu, mapping); if (r) return r; From patchwork Mon Feb 26 08:26:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571502 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 622CC65BD8; Mon, 26 Feb 2024 08:28:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.19 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936114; cv=none; b=fi+AvL+LZDB63FCwiO+nfbcFP+wINKU3cLrKlgxIMYCjqbJOr68qlAJ/SxStKpSW9yiV8veMk9qsW9DSe+cHFWtLot89s34kIKZX8CR7HReZvzjINmOFQav7PleglOH/iNwl+Qq3rkMLKf7uiGJhH0RconR5UkMQJ15RHxduwno= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936114; c=relaxed/simple; bh=25C879RC4XCDozj4CupsIuo0NDO0r8Kx5QkcO4HDod8=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=NlaWEmBy7fRpg0oPZo7cvJOhBSvkewtueowc5/GzhvZqh3ya6+UrqihNKJyMnwzkxgxC5sC75WGUbBcyrtmxyqp2/N7z/ekKiomVnuvcpTASgaaRNw1wfMNYdSwacYsiaOZZ+wMnsp0kXZcaGkQ3qDoIXc6+BLu05d1zZg0bixw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=Z8UzlLYl; arc=none smtp.client-ip=198.175.65.19 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="Z8UzlLYl" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936113; x=1740472113; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=25C879RC4XCDozj4CupsIuo0NDO0r8Kx5QkcO4HDod8=; b=Z8UzlLYlxoCAbVxVkWufi7J+AJqDBmsxdqHiKphHcbZPKdBApgQ5JUlT iqONuLrbzvVL0BtTSqb8JLHPFbvhZ+qBrbQshiiGt8unPIaR+i4wD6zbF bht7QaaZRqAujTaXeaUWshtTQRHV4cCZdW/xScC0ZdZEQwl0mh1XWXZKN 5bqBLjGRrJCyg7a9gMRWHlIlJWNYEGnOxC13+LysGFnt8c/rz/mo4S7R2 T+3+MVmUx+UErx+KaNEPga4Ujyk82t32j4zKlVj4P8hAJJAzuNsbT2jMh T6YmohGkQbZBHPLdMUjVX9F2iiOCZc77ix3yWe+hY2u8kpMWVa7i40jzm g==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="3069484" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="3069484" Received: from orviesa004.jf.intel.com ([10.64.159.144]) by orvoesa111.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:32 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="11272466" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:32 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 074/130] KVM: TDX: Create initial guest memory Date: Mon, 26 Feb 2024 00:26:16 -0800 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata Because the guest memory is protected in TDX, the creation of the initial guest memory requires a dedicated TDX module API, tdh_mem_page_add, instead of directly copying the memory contents into the guest memory in the case of the default VM type. KVM MMU page fault handler callback, set_private_spte, handles it. Implement the hooks for KVM_MEMORY_MAPPING, pre_memory_mapping() and post_memory_mapping() to handle it. Signed-off-by: Isaku Yamahata --- v19: - Switched to use KVM_MEMORY_MAPPING - Dropped measurement extension - updated commit message. private_page_add() => set_private_spte() --- arch/x86/kvm/vmx/main.c | 23 ++++++++++ arch/x86/kvm/vmx/tdx.c | 93 ++++++++++++++++++++++++++++++++++++-- arch/x86/kvm/vmx/tdx.h | 4 ++ arch/x86/kvm/vmx/x86_ops.h | 12 +++++ 4 files changed, 129 insertions(+), 3 deletions(-) diff --git a/arch/x86/kvm/vmx/main.c b/arch/x86/kvm/vmx/main.c index c5672909fdae..7258a6304b4b 100644 --- a/arch/x86/kvm/vmx/main.c +++ b/arch/x86/kvm/vmx/main.c @@ -252,6 +252,27 @@ static int vt_gmem_max_level(struct kvm *kvm, kvm_pfn_t pfn, gfn_t gfn, return 0; } +static int vt_pre_memory_mapping(struct kvm_vcpu *vcpu, + struct kvm_memory_mapping *mapping, + u64 *error_code, u8 *max_level) +{ + if (is_td_vcpu(vcpu)) + return tdx_pre_memory_mapping(vcpu, mapping, error_code, max_level); + + if (mapping->source) + return -EINVAL; + return 0; +} + +static void vt_post_memory_mapping(struct kvm_vcpu *vcpu, + struct kvm_memory_mapping *mapping) +{ + if (!is_td_vcpu(vcpu)) + return; + + tdx_post_memory_mapping(vcpu, mapping); +} + #define VMX_REQUIRED_APICV_INHIBITS \ (BIT(APICV_INHIBIT_REASON_DISABLE)| \ BIT(APICV_INHIBIT_REASON_ABSENT) | \ @@ -415,6 +436,8 @@ struct kvm_x86_ops vt_x86_ops __initdata = { .vcpu_mem_enc_ioctl = vt_vcpu_mem_enc_ioctl, .gmem_max_level = vt_gmem_max_level, + .pre_memory_mapping = vt_pre_memory_mapping, + .post_memory_mapping = vt_post_memory_mapping, }; struct kvm_x86_init_ops vt_init_ops __initdata = { diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c index e65fff43cb1b..8cf6e5dab3e9 100644 --- a/arch/x86/kvm/vmx/tdx.c +++ b/arch/x86/kvm/vmx/tdx.c @@ -390,6 +390,7 @@ int tdx_vm_init(struct kvm *kvm) */ kvm->max_vcpus = min(kvm->max_vcpus, TDX_MAX_VCPUS); + mutex_init(&to_kvm_tdx(kvm)->source_lock); return 0; } @@ -541,6 +542,51 @@ static int tdx_mem_page_aug(struct kvm *kvm, gfn_t gfn, return 0; } +static int tdx_mem_page_add(struct kvm *kvm, gfn_t gfn, + enum pg_level level, kvm_pfn_t pfn) +{ + struct kvm_tdx *kvm_tdx = to_kvm_tdx(kvm); + hpa_t hpa = pfn_to_hpa(pfn); + gpa_t gpa = gfn_to_gpa(gfn); + struct tdx_module_args out; + hpa_t source_pa; + u64 err; + + lockdep_assert_held(&kvm_tdx->source_lock); + + /* + * KVM_MEMORY_MAPPING for TD supports only 4K page because + * tdh_mem_page_add() supports only 4K page. + */ + if (KVM_BUG_ON(level != PG_LEVEL_4K, kvm)) + return -EINVAL; + + if (KVM_BUG_ON(!kvm_tdx->source_page, kvm)) { + tdx_unpin(kvm, pfn); + return -EINVAL; + } + + source_pa = pfn_to_hpa(page_to_pfn(kvm_tdx->source_page)); + do { + err = tdh_mem_page_add(kvm_tdx->tdr_pa, gpa, hpa, source_pa, + &out); + /* + * This path is executed during populating initial guest memory + * image. i.e. before running any vcpu. Race is rare. + */ + } while (unlikely(err == TDX_ERROR_SEPT_BUSY)); + /* + * Don't warn: This is for KVM_MEMORY_MAPPING. So tdh_mem_page_add() can + * fail with parameters user provided. + */ + if (err) { + tdx_unpin(kvm, pfn); + return -EIO; + } + + return 0; +} + static int tdx_sept_set_private_spte(struct kvm *kvm, gfn_t gfn, enum pg_level level, kvm_pfn_t pfn) { @@ -563,9 +609,7 @@ static int tdx_sept_set_private_spte(struct kvm *kvm, gfn_t gfn, if (likely(is_td_finalized(kvm_tdx))) return tdx_mem_page_aug(kvm, gfn, level, pfn); - /* TODO: tdh_mem_page_add() comes here for the initial memory. */ - - return 0; + return tdx_mem_page_add(kvm, gfn, level, pfn); } static int tdx_sept_drop_private_spte(struct kvm *kvm, gfn_t gfn, @@ -1469,6 +1513,49 @@ int tdx_gmem_max_level(struct kvm *kvm, kvm_pfn_t pfn, gfn_t gfn, return 0; } +#define TDX_SEPT_PFERR (PFERR_WRITE_MASK | PFERR_GUEST_ENC_MASK) + +int tdx_pre_memory_mapping(struct kvm_vcpu *vcpu, + struct kvm_memory_mapping *mapping, + u64 *error_code, u8 *max_level) +{ + struct kvm_tdx *kvm_tdx = to_kvm_tdx(vcpu->kvm); + struct page *page; + int r = 0; + + /* memory contents is needed for encryption. */ + if (!mapping->source) + return -EINVAL; + + /* Once TD is finalized, the initial guest memory is fixed. */ + if (is_td_finalized(to_kvm_tdx(vcpu->kvm))) + return -EINVAL; + + /* TDX supports only 4K to pre-populate. */ + *max_level = PG_LEVEL_4K; + *error_code = TDX_SEPT_PFERR; + + r = get_user_pages_fast(mapping->source, 1, 0, &page); + if (r < 0) + return r; + if (r != 1) + return -ENOMEM; + + mutex_lock(&kvm_tdx->source_lock); + kvm_tdx->source_page = page; + return 0; +} + +void tdx_post_memory_mapping(struct kvm_vcpu *vcpu, + struct kvm_memory_mapping *mapping) +{ + struct kvm_tdx *kvm_tdx = to_kvm_tdx(vcpu->kvm); + + put_page(kvm_tdx->source_page); + kvm_tdx->source_page = NULL; + mutex_unlock(&kvm_tdx->source_lock); +} + #define TDX_MD_MAP(_fid, _ptr) \ { .fid = MD_FIELD_ID_##_fid, \ .ptr = (_ptr), } diff --git a/arch/x86/kvm/vmx/tdx.h b/arch/x86/kvm/vmx/tdx.h index 75596b9dcf3f..d822e790e3e5 100644 --- a/arch/x86/kvm/vmx/tdx.h +++ b/arch/x86/kvm/vmx/tdx.h @@ -21,6 +21,10 @@ struct kvm_tdx { atomic_t tdh_mem_track; u64 tsc_offset; + + /* For KVM_MEMORY_MAPPING */ + struct mutex source_lock; + struct page *source_page; }; struct vcpu_tdx { diff --git a/arch/x86/kvm/vmx/x86_ops.h b/arch/x86/kvm/vmx/x86_ops.h index 5335d35bc655..191f2964ec8e 100644 --- a/arch/x86/kvm/vmx/x86_ops.h +++ b/arch/x86/kvm/vmx/x86_ops.h @@ -160,6 +160,11 @@ int tdx_sept_flush_remote_tlbs(struct kvm *kvm); void tdx_load_mmu_pgd(struct kvm_vcpu *vcpu, hpa_t root_hpa, int root_level); int tdx_gmem_max_level(struct kvm *kvm, kvm_pfn_t pfn, gfn_t gfn, bool is_private, u8 *max_level); +int tdx_pre_memory_mapping(struct kvm_vcpu *vcpu, + struct kvm_memory_mapping *mapping, + u64 *error_code, u8 *max_level); +void tdx_post_memory_mapping(struct kvm_vcpu *vcpu, + struct kvm_memory_mapping *mapping); #else static inline int tdx_hardware_setup(struct kvm_x86_ops *x86_ops) { return -EOPNOTSUPP; } static inline void tdx_hardware_unsetup(void) {} @@ -192,6 +197,13 @@ static inline int tdx_gmem_max_level(struct kvm *kvm, kvm_pfn_t pfn, gfn_t gfn, { return -EOPNOTSUPP; } +int tdx_pre_memory_mapping(struct kvm_vcpu *vcpu, + struct kvm_memory_mapping *mapping, + u64 *error_code, u8 *max_level) +{ + return -EOPNOTSUPP; +} +void tdx_post_memory_mapping(struct kvm_vcpu *vcpu, struct kvm_memory_mapping *mapping) {} #endif #endif /* __KVM_X86_VMX_X86_OPS_H */ From patchwork Mon Feb 26 08:26:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571503 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C4E29692E8; Mon, 26 Feb 2024 08:28:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.19 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936115; cv=none; b=d9L+eKTP792pbFzCaiJvLGa7VDjEY7rBotYwv4VJ5HHwoMYPVRI4OYqXoVVYE9G0q0Urk50ypNSe+QSsfW3NTVPJ3EgIfA8O/kQV5cWaQkPnxd682v1ro3UK92iTMNKuiwKWucf9rOxxdZyVepnnyrROmQGMg2HeDYEWaNkJq6Y= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936115; c=relaxed/simple; bh=WRkO1YUitiN/S3sFKUXbIqDdVI858msvJkdcbzWDebk=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=hR4goN0BYE5c81ecOatLcbffezkfgxYA2MOqWLk1NR07hVxVZSZH77MAmQhAfGPGuTiySbI9e5XHbl1ao6lcq7pgIbbahIlI+nPISvCWXqgTKS4nvm+nzt6OB90v0UfQpKCWFenM076UlvF3uFNIP8xc/3atQykrH/M23vqqIlw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=Mn9Rlq5/; arc=none smtp.client-ip=198.175.65.19 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="Mn9Rlq5/" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936114; x=1740472114; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=WRkO1YUitiN/S3sFKUXbIqDdVI858msvJkdcbzWDebk=; b=Mn9Rlq5/8AJ2Nv56ZkMlKpMEvr3k1OefYg5jM0bXaxR7oTyFYHe/G7Si vzbngtkR3zODUo6tTHLDtCsn2EIvR7IqKJbuLiidSgZ+8TLstoa3SlYNa zJGNJTtaQuBZj+IlOwF0JhpQmJopF8aaxt9W1HZxEoBRF4MvSbanefEuM 2IM4XYlTvrUBzC8qgsKygjhlflLVr1Tifl7yHxNvhLixE4YREKLw0KbeT gsAzHpjYxq5hsGjtgs1bqQyaT0vtxH2Chz1nIxIlisdoi+pqU7dCDiJyT zzXfkRy0Oc9VRM3AGOhnGCa45RLmHjCKGIzPWq+9+Kur+gO5f/qC10lN5 A==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="3069485" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="3069485" Received: from orviesa004.jf.intel.com ([10.64.159.144]) by orvoesa111.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:33 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="11272476" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:33 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 075/130] KVM: TDX: Extend memory measurement with initial guest memory Date: Mon, 26 Feb 2024 00:26:17 -0800 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata TDX allows to extned memory measurement with the initial memory. Define new subcommand, KVM_TDX_EXTEND_MEMORY, of VM-scoped KVM_MEMORY_ENCRYPT_OP. it extends memory measurement of the TDX guest. The memory region must be populated with KVM_MEMORY_MAPPING command. Signed-off-by: Isaku Yamahata --- v19: - newly added - Split KVM_TDX_INIT_MEM_REGION into only extension function --- arch/x86/include/uapi/asm/kvm.h | 1 + arch/x86/kvm/vmx/tdx.c | 64 +++++++++++++++++++++++++++++++++ 2 files changed, 65 insertions(+) diff --git a/arch/x86/include/uapi/asm/kvm.h b/arch/x86/include/uapi/asm/kvm.h index 4000a2e087a8..34167404020c 100644 --- a/arch/x86/include/uapi/asm/kvm.h +++ b/arch/x86/include/uapi/asm/kvm.h @@ -572,6 +572,7 @@ enum kvm_tdx_cmd_id { KVM_TDX_CAPABILITIES = 0, KVM_TDX_INIT_VM, KVM_TDX_INIT_VCPU, + KVM_TDX_EXTEND_MEMORY, KVM_TDX_CMD_NR_MAX, }; diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c index 8cf6e5dab3e9..3cfba63a7762 100644 --- a/arch/x86/kvm/vmx/tdx.c +++ b/arch/x86/kvm/vmx/tdx.c @@ -1339,6 +1339,67 @@ void tdx_flush_tlb_current(struct kvm_vcpu *vcpu) tdx_track(vcpu->kvm); } +static int tdx_extend_memory(struct kvm *kvm, struct kvm_tdx_cmd *cmd) +{ + struct kvm_tdx *kvm_tdx = to_kvm_tdx(kvm); + struct kvm_memory_mapping mapping; + struct tdx_module_args out; + bool extended = false; + int idx, ret = 0; + gpa_t gpa; + u64 err; + int i; + + /* Once TD is finalized, the initial guest memory is fixed. */ + if (is_td_finalized(kvm_tdx)) + return -EINVAL; + + if (cmd->flags) + return -EINVAL; + + if (copy_from_user(&mapping, (void __user *)cmd->data, sizeof(mapping))) + return -EFAULT; + + /* Sanity check */ + if (mapping.source || !mapping.nr_pages || + mapping.nr_pages & GENMASK_ULL(63, 63 - PAGE_SHIFT) || + mapping.base_gfn + (mapping.nr_pages << PAGE_SHIFT) <= mapping.base_gfn || + !kvm_is_private_gpa(kvm, mapping.base_gfn) || + !kvm_is_private_gpa(kvm, mapping.base_gfn + (mapping.nr_pages << PAGE_SHIFT))) + return -EINVAL; + + idx = srcu_read_lock(&kvm->srcu); + while (mapping.nr_pages) { + if (signal_pending(current)) { + ret = -ERESTARTSYS; + break; + } + + if (need_resched()) + cond_resched(); + + gpa = gfn_to_gpa(mapping.base_gfn); + for (i = 0; i < PAGE_SIZE; i += TDX_EXTENDMR_CHUNKSIZE) { + err = tdh_mr_extend(kvm_tdx->tdr_pa, gpa + i, &out); + if (err) { + ret = -EIO; + break; + } + } + mapping.base_gfn++; + mapping.nr_pages--; + extended = true; + } + srcu_read_unlock(&kvm->srcu, idx); + + if (extended && mapping.nr_pages > 0) + ret = -EAGAIN; + if (copy_to_user((void __user *)cmd->data, &mapping, sizeof(mapping))) + ret = -EFAULT; + + return ret; +} + int tdx_vm_ioctl(struct kvm *kvm, void __user *argp) { struct kvm_tdx_cmd tdx_cmd; @@ -1358,6 +1419,9 @@ int tdx_vm_ioctl(struct kvm *kvm, void __user *argp) case KVM_TDX_INIT_VM: r = tdx_td_init(kvm, &tdx_cmd); break; + case KVM_TDX_EXTEND_MEMORY: + r = tdx_extend_memory(kvm, &tdx_cmd); + break; default: r = -EINVAL; goto out; From patchwork Mon Feb 26 08:26:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571504 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2F177692FB; Mon, 26 Feb 2024 08:28:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.19 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936115; cv=none; b=sJBloOYQR6+Kmxz2Bq79Vn/e6UvGX8zQRVdCs7xQ1PUBESHtCoVW3L7g2MD5KT7RSWDH9mcIWw037V0BwmdEEf4uoHgbDQlIkQOSR372uQhLq9HMA9KaGlNpfbqe3HF/iJxtxqNCNe8dAEGQout/ImUGZXFI5g30Ja7KLni1Hhk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936115; c=relaxed/simple; bh=WLwpU+ZFzvbhXg3wiYCTpPwFcaOG9hkZbPu7n4G5fQY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=uBKvNJfrpx1jE/7O7MCuSzmMA5iBlDh4dPxrEz8+kdugwRuyK0HaALZkwRvNThIVmO3d+FQcMJbFei2UHsyLPQZr7x6pnKZLP3aTPcq/oGCjg38thsCczofCQgZeNveTRBMf4Ha1yCRWoWjS5B/fTHExcu5ZL76+8Dg6Tl4R3mY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=QOpse7s+; arc=none smtp.client-ip=198.175.65.19 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="QOpse7s+" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936114; x=1740472114; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=WLwpU+ZFzvbhXg3wiYCTpPwFcaOG9hkZbPu7n4G5fQY=; b=QOpse7s+vdx0m3tbYa2IF/qB8lW/GHA0QqOwflfbpmBs3kyVXE2UbDsU 2i7a4lnRNodp5+utyRAIjO4GPV31jstPp+riqKl3DxRu5wXa0eMP8Uilk Yd+qdZsOjbU+oam7ly1TTrum5G14m7HsPeUDjfYCJltZPuKRljysveAVr QnhWc3ZReED8jQfRfplEdAQ7VjTICue+8UfnDB7h/THMSQg/PGqfcUD0r wkMuDEcMBhbiU6tL2DwBvVREtp/XYQ3HV3QHkxxFfYxDbuHZXJrNZ+7ex O74sdVUOlBer4Kfpf0fACtSgbeEq/urYCZw6yfQ1Zni1iXVeBfVZrfCrz w==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="3069489" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="3069489" Received: from orviesa004.jf.intel.com ([10.64.159.144]) by orvoesa111.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:34 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="11272485" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:33 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 076/130] KVM: TDX: Finalize VM initialization Date: Mon, 26 Feb 2024 00:26:18 -0800 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata To protect the initial contents of the guest TD, the TDX module measures the guest TD during the build process as SHA-384 measurement. The measurement of the guest TD contents needs to be completed to make the guest TD ready to run. Add a new subcommand, KVM_TDX_FINALIZE_VM, for VM-scoped KVM_MEMORY_ENCRYPT_OP to finalize the measurement and mark the TDX VM ready to run. Signed-off-by: Isaku Yamahata --- v18: - Remove the change of tools/arch/x86/include/uapi/asm/kvm.h. v14 -> v15: - removed unconditional tdx_track() by tdx_flush_tlb_current() that does tdx_track(). Signed-off-by: Isaku Yamahata --- arch/x86/include/uapi/asm/kvm.h | 1 + arch/x86/kvm/vmx/tdx.c | 21 +++++++++++++++++++++ 2 files changed, 22 insertions(+) diff --git a/arch/x86/include/uapi/asm/kvm.h b/arch/x86/include/uapi/asm/kvm.h index 34167404020c..c160f60189d1 100644 --- a/arch/x86/include/uapi/asm/kvm.h +++ b/arch/x86/include/uapi/asm/kvm.h @@ -573,6 +573,7 @@ enum kvm_tdx_cmd_id { KVM_TDX_INIT_VM, KVM_TDX_INIT_VCPU, KVM_TDX_EXTEND_MEMORY, + KVM_TDX_FINALIZE_VM, KVM_TDX_CMD_NR_MAX, }; diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c index 3cfba63a7762..6aff3f7e2488 100644 --- a/arch/x86/kvm/vmx/tdx.c +++ b/arch/x86/kvm/vmx/tdx.c @@ -1400,6 +1400,24 @@ static int tdx_extend_memory(struct kvm *kvm, struct kvm_tdx_cmd *cmd) return ret; } +static int tdx_td_finalizemr(struct kvm *kvm) +{ + struct kvm_tdx *kvm_tdx = to_kvm_tdx(kvm); + u64 err; + + if (!is_hkid_assigned(kvm_tdx) || is_td_finalized(kvm_tdx)) + return -EINVAL; + + err = tdh_mr_finalize(kvm_tdx->tdr_pa); + if (WARN_ON_ONCE(err)) { + pr_tdx_error(TDH_MR_FINALIZE, err, NULL); + return -EIO; + } + + kvm_tdx->finalized = true; + return 0; +} + int tdx_vm_ioctl(struct kvm *kvm, void __user *argp) { struct kvm_tdx_cmd tdx_cmd; @@ -1422,6 +1440,9 @@ int tdx_vm_ioctl(struct kvm *kvm, void __user *argp) case KVM_TDX_EXTEND_MEMORY: r = tdx_extend_memory(kvm, &tdx_cmd); break; + case KVM_TDX_FINALIZE_VM: + r = tdx_td_finalizemr(kvm); + break; default: r = -EINVAL; goto out; From patchwork Mon Feb 26 08:26:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571505 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EC98269D19; Mon, 26 Feb 2024 08:28:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.19 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936116; cv=none; b=hiN0APu9MG0tfPE/q9OAwN2cKnZKFOWLxPmjR1ZfNo7IqkSsBodLDBuGEHqAV7bpDLX7En5YOg8HoixQBlyfB5bedwVhNdF9sQecL23CKk4z20G7TG/jUFWYd/5j2WZ5xG3pmqYqV54ddz5wnVdt+oDqq+LGge1d+2/+StvEBf4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936116; c=relaxed/simple; bh=R5wOuoM4Onpjff9OlffMGMEw35e7AU+j3x/NDTZE/oY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=swzmfGverUUzAX7kiOw/ii6OwKNrY2gfHsUQcC12SQeGSPi3HE2Ze21sjTuHFFenFBrYe8e/UZH8WfSPw/k7zJ7ImLPH7iqX1Au7nNoV33GMJxkFxBdwgtU9IkUCiEHO/LipyE6CAX9QPp9KBGnLrsYEY4pXcFcu5l0UiZ8GdlU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=eeM+zcRy; arc=none smtp.client-ip=198.175.65.19 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="eeM+zcRy" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936115; x=1740472115; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=R5wOuoM4Onpjff9OlffMGMEw35e7AU+j3x/NDTZE/oY=; b=eeM+zcRyOmknzq8W90XX8fcpNu8R3FvtucKZqTyn2yqltOwmYhi2IoRC JqCjybDESXdeDKp/a/ZqCxCfPB/DqpFcPJjBgF7oSQRZUsktdnhZXGuUd pgEQ7Y17dEUInwZnzxBT5ZVWQalynW+IOHUSaY89ptud2UBSmFxgwa0Lt i6BdPLIOI3zxWLML94wVYLowA0QdiBURpcNyjxXpzekSd9FZ4DcKP5fzH JzB8NcreJaWn8g5GZsVkKbWizD9fBlH/DTxNWUpsD8MiGcjw5KbroLvdJ rMqGxUKpq44z2+JpBZqxxzUauEUhgpVWCitd+qWhJCEe7hy6i8QCNfG1b Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="3069496" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="3069496" Received: from orviesa004.jf.intel.com ([10.64.159.144]) by orvoesa111.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:35 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="11272497" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:34 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 077/130] [MARKER] The start of TDX KVM patch series: TD vcpu enter/exit Date: Mon, 26 Feb 2024 00:26:19 -0800 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata This empty commit is to mark the start of patch series of TD vcpu enter/exit. Signed-off-by: Isaku Yamahata --- Documentation/virt/kvm/intel-tdx-layer-status.rst | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/Documentation/virt/kvm/intel-tdx-layer-status.rst b/Documentation/virt/kvm/intel-tdx-layer-status.rst index 46ae049b6b85..33e107bcb5cf 100644 --- a/Documentation/virt/kvm/intel-tdx-layer-status.rst +++ b/Documentation/virt/kvm/intel-tdx-layer-status.rst @@ -12,6 +12,7 @@ What qemu can do - Qemu can create/destroy guest of TDX vm type. - Qemu can create/destroy vcpu of TDX vm type. - Qemu can populate initial guest memory image. +- Qemu can finalize guest TD. Patch Layer status ------------------ @@ -22,8 +23,8 @@ Patch Layer status * TD VM creation/destruction: Applied * TD vcpu creation/destruction: Applied * TDX EPT violation: Applied -* TD finalization: Applying -* TD vcpu enter/exit: Not yet +* TD finalization: Applied +* TD vcpu enter/exit: Applying * TD vcpu interrupts/exit/hypercall: Not yet * KVM MMU GPA shared bits: Applied From patchwork Mon Feb 26 08:26:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571506 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0D9141EEE9; Mon, 26 Feb 2024 08:28:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.19 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936119; cv=none; b=VFmuxvZozetJr6BOC+MfUesQsIJQvNZjpsKlNP69D/BXFuA53bpdnfTQ3ghlHmmnNaz4NB275HVeAOPgYI082GpgA+jtagMi8iX4OVyBaFIfoAkByOS2MI5lR8JBa3gwKbHeiPTue6/bBzYbpwUdMlhpKf1AJGn5+1ZOdtFq3uc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936119; c=relaxed/simple; bh=83+PksMV8WT23lQfWUC1NU2JQlcLGHpTbnnZQT1EEfs=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=YhBtBYDLBbfbmPSA2c3gXKXAQ39zye7pX4LpOwffI7OdMNro3kyl5wsSkneGF9jZlyjYIba+TNGotTRgOX3lZOe2v6H0lE3jkq3DFqRHqBahvwNNt7/6KgXU+rx3oBCPYHCttayMce2wiPIfvoh+5CbOer5h6Ig599lDTV9va0w= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=Hm4TInZb; arc=none smtp.client-ip=198.175.65.19 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="Hm4TInZb" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936117; x=1740472117; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=83+PksMV8WT23lQfWUC1NU2JQlcLGHpTbnnZQT1EEfs=; b=Hm4TInZbCiAHyYbKOYl4jIO8c0kzN6hSSVvcrRw4WTNcSt3bmn4WU/fF ZYkvm89ZrW58CzH9PcoN/xQLVFakD2HspGvlcmbm7VJ2i2XM9/redU3CK KdnwtphGqSrW4dmCcRTfrdz+ocI9kLq9FgyH8rtpeb0Ah82N1zEO/g+MO EBuJWi26zQuB2DIs503JzCON3TJDBgKp0TJkSQ/coDTdnizt8maIs1WiH XcQWkjTDxoOXMHk0Y6WKstQCOc+roUEPDwrgkrN2nSPxoa+6glhrQ1Xv6 3GUIeoNIge3/sjF3prgeFji0uVDuoPw77AxmIqygxUdT+XUzzJxwCoAfn A==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="3069499" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="3069499" Received: from orviesa004.jf.intel.com ([10.64.159.144]) by orvoesa111.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:37 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="11272522" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:36 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 078/130] KVM: TDX: Implement TDX vcpu enter/exit path Date: Mon, 26 Feb 2024 00:26:20 -0800 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata This patch implements running TDX vcpu. Once vcpu runs on the logical processor (LP), the TDX vcpu is associated with it. When the TDX vcpu moves to another LP, the TDX vcpu needs to flush its status on the LP. When destroying TDX vcpu, it needs to complete flush and flush cpu memory cache. Track which LP the TDX vcpu run and flush it as necessary. Do nothing on sched_in event as TDX doesn't support pause loop. TDX vcpu execution requires restoring PMU debug store after returning back to KVM because the TDX module unconditionally resets the value. To reuse the existing code, export perf_restore_debug_store. Signed-off-by: Isaku Yamahata --- v19: - Removed export_symbol_gpl(host_xcr0) to the patch that uses it Changes v15 -> v16: - use __seamcall_saved_ret() - As struct tdx_module_args doesn't match with vcpu.arch.regs, copy regs before/after calling __seamcall_saved_ret(). Signed-off-by: Isaku Yamahata --- arch/x86/kvm/vmx/main.c | 21 +++++++++- arch/x86/kvm/vmx/tdx.c | 84 ++++++++++++++++++++++++++++++++++++++ arch/x86/kvm/vmx/tdx.h | 33 +++++++++++++++ arch/x86/kvm/vmx/x86_ops.h | 2 + 4 files changed, 138 insertions(+), 2 deletions(-) diff --git a/arch/x86/kvm/vmx/main.c b/arch/x86/kvm/vmx/main.c index 7258a6304b4b..d72651ce99ac 100644 --- a/arch/x86/kvm/vmx/main.c +++ b/arch/x86/kvm/vmx/main.c @@ -158,6 +158,23 @@ static void vt_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event) vmx_vcpu_reset(vcpu, init_event); } +static int vt_vcpu_pre_run(struct kvm_vcpu *vcpu) +{ + if (is_td_vcpu(vcpu)) + /* Unconditionally continue to vcpu_run(). */ + return 1; + + return vmx_vcpu_pre_run(vcpu); +} + +static fastpath_t vt_vcpu_run(struct kvm_vcpu *vcpu) +{ + if (is_td_vcpu(vcpu)) + return tdx_vcpu_run(vcpu); + + return vmx_vcpu_run(vcpu); +} + static void vt_flush_tlb_all(struct kvm_vcpu *vcpu) { if (is_td_vcpu(vcpu)) { @@ -343,8 +360,8 @@ struct kvm_x86_ops vt_x86_ops __initdata = { .flush_tlb_gva = vt_flush_tlb_gva, .flush_tlb_guest = vt_flush_tlb_guest, - .vcpu_pre_run = vmx_vcpu_pre_run, - .vcpu_run = vmx_vcpu_run, + .vcpu_pre_run = vt_vcpu_pre_run, + .vcpu_run = vt_vcpu_run, .handle_exit = vmx_handle_exit, .skip_emulated_instruction = vmx_skip_emulated_instruction, .update_emulated_instruction = vmx_update_emulated_instruction, diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c index 6aff3f7e2488..fdf9196cb592 100644 --- a/arch/x86/kvm/vmx/tdx.c +++ b/arch/x86/kvm/vmx/tdx.c @@ -11,6 +11,9 @@ #include "vmx.h" #include "x86.h" +#include +#include "trace.h" + #undef pr_fmt #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt @@ -491,6 +494,87 @@ void tdx_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event) */ } +static noinstr void tdx_vcpu_enter_exit(struct vcpu_tdx *tdx) +{ + struct tdx_module_args args; + + /* + * Avoid section mismatch with to_tdx() with KVM_VM_BUG(). The caller + * should call to_tdx(). + */ + struct kvm_vcpu *vcpu = &tdx->vcpu; + + guest_state_enter_irqoff(); + + /* + * TODO: optimization: + * - Eliminate copy between args and vcpu->arch.regs. + * - copyin/copyout registers only if (tdx->tdvmvall.regs_mask != 0) + * which means TDG.VP.VMCALL. + */ + args = (struct tdx_module_args) { + .rcx = tdx->tdvpr_pa, +#define REG(reg, REG) .reg = vcpu->arch.regs[VCPU_REGS_ ## REG] + REG(rdx, RDX), + REG(r8, R8), + REG(r9, R9), + REG(r10, R10), + REG(r11, R11), + REG(r12, R12), + REG(r13, R13), + REG(r14, R14), + REG(r15, R15), + REG(rbx, RBX), + REG(rdi, RDI), + REG(rsi, RSI), +#undef REG + }; + + tdx->exit_reason.full = __seamcall_saved_ret(TDH_VP_ENTER, &args); + +#define REG(reg, REG) vcpu->arch.regs[VCPU_REGS_ ## REG] = args.reg + REG(rcx, RCX); + REG(rdx, RDX); + REG(r8, R8); + REG(r9, R9); + REG(r10, R10); + REG(r11, R11); + REG(r12, R12); + REG(r13, R13); + REG(r14, R14); + REG(r15, R15); + REG(rbx, RBX); + REG(rdi, RDI); + REG(rsi, RSI); +#undef REG + + WARN_ON_ONCE(!kvm_rebooting && + (tdx->exit_reason.full & TDX_SW_ERROR) == TDX_SW_ERROR); + + guest_state_exit_irqoff(); +} + +fastpath_t tdx_vcpu_run(struct kvm_vcpu *vcpu) +{ + struct vcpu_tdx *tdx = to_tdx(vcpu); + + if (unlikely(!tdx->initialized)) + return -EINVAL; + if (unlikely(vcpu->kvm->vm_bugged)) { + tdx->exit_reason.full = TDX_NON_RECOVERABLE_VCPU; + return EXIT_FASTPATH_NONE; + } + + trace_kvm_entry(vcpu); + + tdx_vcpu_enter_exit(tdx); + + vcpu->arch.regs_avail &= ~VMX_REGS_LAZY_LOAD_SET; + trace_kvm_exit(vcpu, KVM_ISA_VMX); + + return EXIT_FASTPATH_NONE; +} + void tdx_load_mmu_pgd(struct kvm_vcpu *vcpu, hpa_t root_hpa, int pgd_level) { WARN_ON_ONCE(root_hpa & ~PAGE_MASK); diff --git a/arch/x86/kvm/vmx/tdx.h b/arch/x86/kvm/vmx/tdx.h index d822e790e3e5..81d301fbe638 100644 --- a/arch/x86/kvm/vmx/tdx.h +++ b/arch/x86/kvm/vmx/tdx.h @@ -27,6 +27,37 @@ struct kvm_tdx { struct page *source_page; }; +union tdx_exit_reason { + struct { + /* 31:0 mirror the VMX Exit Reason format */ + u64 basic : 16; + u64 reserved16 : 1; + u64 reserved17 : 1; + u64 reserved18 : 1; + u64 reserved19 : 1; + u64 reserved20 : 1; + u64 reserved21 : 1; + u64 reserved22 : 1; + u64 reserved23 : 1; + u64 reserved24 : 1; + u64 reserved25 : 1; + u64 bus_lock_detected : 1; + u64 enclave_mode : 1; + u64 smi_pending_mtf : 1; + u64 smi_from_vmx_root : 1; + u64 reserved30 : 1; + u64 failed_vmentry : 1; + + /* 63:32 are TDX specific */ + u64 details_l1 : 8; + u64 class : 8; + u64 reserved61_48 : 14; + u64 non_recoverable : 1; + u64 error : 1; + }; + u64 full; +}; + struct vcpu_tdx { struct kvm_vcpu vcpu; @@ -34,6 +65,8 @@ struct vcpu_tdx { unsigned long *tdvpx_pa; bool td_vcpu_created; + union tdx_exit_reason exit_reason; + bool initialized; /* diff --git a/arch/x86/kvm/vmx/x86_ops.h b/arch/x86/kvm/vmx/x86_ops.h index 191f2964ec8e..3e29a6fe28ef 100644 --- a/arch/x86/kvm/vmx/x86_ops.h +++ b/arch/x86/kvm/vmx/x86_ops.h @@ -150,6 +150,7 @@ int tdx_vm_ioctl(struct kvm *kvm, void __user *argp); int tdx_vcpu_create(struct kvm_vcpu *vcpu); void tdx_vcpu_free(struct kvm_vcpu *vcpu); void tdx_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event); +fastpath_t tdx_vcpu_run(struct kvm_vcpu *vcpu); u8 tdx_get_mt_mask(struct kvm_vcpu *vcpu, gfn_t gfn, bool is_mmio); int tdx_vcpu_ioctl(struct kvm_vcpu *vcpu, void __user *argp); @@ -184,6 +185,7 @@ static inline int tdx_vm_ioctl(struct kvm *kvm, void __user *argp) { return -EOP static inline int tdx_vcpu_create(struct kvm_vcpu *vcpu) { return -EOPNOTSUPP; } static inline void tdx_vcpu_free(struct kvm_vcpu *vcpu) {} static inline void tdx_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event) {} +static inline fastpath_t tdx_vcpu_run(struct kvm_vcpu *vcpu) { return EXIT_FASTPATH_NONE; } static inline u8 tdx_get_mt_mask(struct kvm_vcpu *vcpu, gfn_t gfn, bool is_mmio) { return 0; } static inline int tdx_vcpu_ioctl(struct kvm_vcpu *vcpu, void __user *argp) { return -EOPNOTSUPP; } From patchwork Mon Feb 26 08:26:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571507 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CAC5F63CB1; Mon, 26 Feb 2024 08:28:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.19 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936119; cv=none; b=GtcMfK6Q4ajvNKzyjvOj6cxR1y3H59iwZCBwPbKNW5kyAmPU+NaXpi3zIqXPmt83Lp2X65eURhbFs+BvSAjf5/hY3+4thldCrRAD2Vswn1lUhVVc9iSraiVEVuwqabpyWo5typ0c4aoC9cXDbDumtkcbib3mEHm2Iwn+4IrrwQQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936119; c=relaxed/simple; bh=rnzww5cGdN7mG73mfyRxtwlDq6YIEmaeSGmlC0VnQT4=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=IDrF16JQGWGPzMAHZ2pq3Qj9tMoYFVCzVB8Qq8kJ8jLYHbPnXZkHFlZa5BOQC8oRU1Fz+i0ddstIUbR1mEKr+aBUZHA5dlW/6Q9mRDC0tHOp69PAa4/8xeWMKjL/99F/SlqMqCwOu3s3f/DoMtjGVxPOPTvJEPR6Ik1NeJGXzIM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=MQYaFh59; arc=none smtp.client-ip=198.175.65.19 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="MQYaFh59" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936118; x=1740472118; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=rnzww5cGdN7mG73mfyRxtwlDq6YIEmaeSGmlC0VnQT4=; b=MQYaFh59grH0iWwdzqFdfFgg0oLVHjSYwjNxq3YPz8APzM1C9NDJAU6u D0Q0K4qN7didWX6Cn+OVpQUtsxBFqGjLbP9B3ZG5Yp4AeEf9JiiAecEje ccggBzNGMoPMD6ijQReL2iS1O23hksFmHhLkKSDdOSLKjnHICmYERs/Ur GwU+Ct+4ltG7ZtShRp4HR+YyYSfjz9+fny0/gNKOrhbocyNyrouBHcliU k820vAxWGUMy4bvdjRVtyjLbOrmQl7NIXk7CoR/Qgi57tEwmOGiYa0L/t UHey6da+3E0am/3fl6vN3vtAJ+XMYmOJxfgZc9kpfdD+c4Rc8n+SYA47C A==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="3069503" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="3069503" Received: from orviesa004.jf.intel.com ([10.64.159.144]) by orvoesa111.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:38 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="11272527" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:37 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 079/130] KVM: TDX: vcpu_run: save/restore host state(host kernel gs) Date: Mon, 26 Feb 2024 00:26:21 -0800 Message-Id: <4a766983346b2c01e943348af3c5ca6691e272f9.1708933498.git.isaku.yamahata@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata On entering/exiting TDX vcpu, Preserved or clobbered CPU state is different from VMX case. Add TDX hooks to save/restore host/guest CPU state. Save/restore kernel GS base MSR. Signed-off-by: Isaku Yamahata Reviewed-by: Paolo Bonzini --- arch/x86/kvm/vmx/main.c | 30 +++++++++++++++++++++++++-- arch/x86/kvm/vmx/tdx.c | 42 ++++++++++++++++++++++++++++++++++++++ arch/x86/kvm/vmx/tdx.h | 4 ++++ arch/x86/kvm/vmx/x86_ops.h | 4 ++++ 4 files changed, 78 insertions(+), 2 deletions(-) diff --git a/arch/x86/kvm/vmx/main.c b/arch/x86/kvm/vmx/main.c index d72651ce99ac..8275a242ce07 100644 --- a/arch/x86/kvm/vmx/main.c +++ b/arch/x86/kvm/vmx/main.c @@ -158,6 +158,32 @@ static void vt_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event) vmx_vcpu_reset(vcpu, init_event); } +static void vt_prepare_switch_to_guest(struct kvm_vcpu *vcpu) +{ + /* + * All host state is saved/restored across SEAMCALL/SEAMRET, and the + * guest state of a TD is obviously off limits. Deferring MSRs and DRs + * is pointless because the TDX module needs to load *something* so as + * not to expose guest state. + */ + if (is_td_vcpu(vcpu)) { + tdx_prepare_switch_to_guest(vcpu); + return; + } + + vmx_prepare_switch_to_guest(vcpu); +} + +static void vt_vcpu_put(struct kvm_vcpu *vcpu) +{ + if (is_td_vcpu(vcpu)) { + tdx_vcpu_put(vcpu); + return; + } + + vmx_vcpu_put(vcpu); +} + static int vt_vcpu_pre_run(struct kvm_vcpu *vcpu) { if (is_td_vcpu(vcpu)) @@ -326,9 +352,9 @@ struct kvm_x86_ops vt_x86_ops __initdata = { .vcpu_free = vt_vcpu_free, .vcpu_reset = vt_vcpu_reset, - .prepare_switch_to_guest = vmx_prepare_switch_to_guest, + .prepare_switch_to_guest = vt_prepare_switch_to_guest, .vcpu_load = vmx_vcpu_load, - .vcpu_put = vmx_vcpu_put, + .vcpu_put = vt_vcpu_put, .update_exception_bitmap = vmx_update_exception_bitmap, .get_msr_feature = vmx_get_msr_feature, diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c index fdf9196cb592..9616b1aab6ce 100644 --- a/arch/x86/kvm/vmx/tdx.c +++ b/arch/x86/kvm/vmx/tdx.c @@ -1,5 +1,6 @@ // SPDX-License-Identifier: GPL-2.0 #include +#include #include @@ -423,6 +424,7 @@ u8 tdx_get_mt_mask(struct kvm_vcpu *vcpu, gfn_t gfn, bool is_mmio) int tdx_vcpu_create(struct kvm_vcpu *vcpu) { struct kvm_tdx *kvm_tdx = to_kvm_tdx(vcpu->kvm); + struct vcpu_tdx *tdx = to_tdx(vcpu); WARN_ON_ONCE(vcpu->arch.cpuid_entries); WARN_ON_ONCE(vcpu->arch.cpuid_nent); @@ -446,9 +448,47 @@ int tdx_vcpu_create(struct kvm_vcpu *vcpu) if ((kvm_tdx->xfam & XFEATURE_MASK_XTILE) == XFEATURE_MASK_XTILE) vcpu->arch.xfd_no_write_intercept = true; + tdx->host_state_need_save = true; + tdx->host_state_need_restore = false; + return 0; } +void tdx_prepare_switch_to_guest(struct kvm_vcpu *vcpu) +{ + struct vcpu_tdx *tdx = to_tdx(vcpu); + + if (!tdx->host_state_need_save) + return; + + if (likely(is_64bit_mm(current->mm))) + tdx->msr_host_kernel_gs_base = current->thread.gsbase; + else + tdx->msr_host_kernel_gs_base = read_msr(MSR_KERNEL_GS_BASE); + + tdx->host_state_need_save = false; +} + +static void tdx_prepare_switch_to_host(struct kvm_vcpu *vcpu) +{ + struct vcpu_tdx *tdx = to_tdx(vcpu); + + tdx->host_state_need_save = true; + if (!tdx->host_state_need_restore) + return; + + ++vcpu->stat.host_state_reload; + + wrmsrl(MSR_KERNEL_GS_BASE, tdx->msr_host_kernel_gs_base); + tdx->host_state_need_restore = false; +} + +void tdx_vcpu_put(struct kvm_vcpu *vcpu) +{ + vmx_vcpu_pi_put(vcpu); + tdx_prepare_switch_to_host(vcpu); +} + void tdx_vcpu_free(struct kvm_vcpu *vcpu) { struct vcpu_tdx *tdx = to_tdx(vcpu); @@ -569,6 +609,8 @@ fastpath_t tdx_vcpu_run(struct kvm_vcpu *vcpu) tdx_vcpu_enter_exit(tdx); + tdx->host_state_need_restore = true; + vcpu->arch.regs_avail &= ~VMX_REGS_LAZY_LOAD_SET; trace_kvm_exit(vcpu, KVM_ISA_VMX); diff --git a/arch/x86/kvm/vmx/tdx.h b/arch/x86/kvm/vmx/tdx.h index 81d301fbe638..e96c416e73bf 100644 --- a/arch/x86/kvm/vmx/tdx.h +++ b/arch/x86/kvm/vmx/tdx.h @@ -69,6 +69,10 @@ struct vcpu_tdx { bool initialized; + bool host_state_need_save; + bool host_state_need_restore; + u64 msr_host_kernel_gs_base; + /* * Dummy to make pmu_intel not corrupt memory. * TODO: Support PMU for TDX. Future work. diff --git a/arch/x86/kvm/vmx/x86_ops.h b/arch/x86/kvm/vmx/x86_ops.h index 3e29a6fe28ef..9fd997c79c33 100644 --- a/arch/x86/kvm/vmx/x86_ops.h +++ b/arch/x86/kvm/vmx/x86_ops.h @@ -151,6 +151,8 @@ int tdx_vcpu_create(struct kvm_vcpu *vcpu); void tdx_vcpu_free(struct kvm_vcpu *vcpu); void tdx_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event); fastpath_t tdx_vcpu_run(struct kvm_vcpu *vcpu); +void tdx_prepare_switch_to_guest(struct kvm_vcpu *vcpu); +void tdx_vcpu_put(struct kvm_vcpu *vcpu); u8 tdx_get_mt_mask(struct kvm_vcpu *vcpu, gfn_t gfn, bool is_mmio); int tdx_vcpu_ioctl(struct kvm_vcpu *vcpu, void __user *argp); @@ -186,6 +188,8 @@ static inline int tdx_vcpu_create(struct kvm_vcpu *vcpu) { return -EOPNOTSUPP; } static inline void tdx_vcpu_free(struct kvm_vcpu *vcpu) {} static inline void tdx_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event) {} static inline fastpath_t tdx_vcpu_run(struct kvm_vcpu *vcpu) { return EXIT_FASTPATH_NONE; } +static inline void tdx_prepare_switch_to_guest(struct kvm_vcpu *vcpu) {} +static inline void tdx_vcpu_put(struct kvm_vcpu *vcpu) {} static inline u8 tdx_get_mt_mask(struct kvm_vcpu *vcpu, gfn_t gfn, bool is_mmio) { return 0; } static inline int tdx_vcpu_ioctl(struct kvm_vcpu *vcpu, void __user *argp) { return -EOPNOTSUPP; } From patchwork Mon Feb 26 08:26:22 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571508 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 232416E5E4; Mon, 26 Feb 2024 08:28:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.19 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936121; cv=none; b=MV9rvXCTES6Z4/U8fzT313F7G6mp61Na6i0YUJtrESG3hFG2SjuUj/iW++tkgnaJDngHnOecrRLGlAsa8P1LqjGIfLn4+VQLF8gvinHDcPm3A9g/hDuam6dPbsNBdJ0+EVDLxP4L8vSrinSvcp3HO75G5H2pHWG7VHXjXjeUXOQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936121; c=relaxed/simple; bh=UdHZZK+7HP69uy+KRU1ryfGK1QyA5zHgajtOtOag5f4=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=aNBd5NIOSnDSIhQCrOZVopaU4C/IIa6DBj4y/NLr/O5jhzzErxYPYCuX2TtLjAFx44pJWUuRS3/SqpDZMgLdtBpo8nHzuneJJaHszCTfm89okaSJBuwmOiEe05NHGAcHgy5LU4dRYTqhupjyB8cz9xCs+xpRU2FqmArI1+KfY2Q= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=KMGCGt8o; arc=none smtp.client-ip=198.175.65.19 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="KMGCGt8o" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936119; x=1740472119; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=UdHZZK+7HP69uy+KRU1ryfGK1QyA5zHgajtOtOag5f4=; b=KMGCGt8op5ApiszUjlILznFk/fwt0U1eFg2Uf1TXr5+Pq796q27/afre +1iRU6ZTLComkNe3tb2KgMkWIhvS2mAw3zP47MTiMR6a0raaSLMSVZxsG j/VYMm4kri0/8qUaMvNst2z2esHTTIwUmlCLChg2nbwPbhcN80DP7dUKy 0mQ2hlaqaxNVX2xnsP1iii497hIHrcYFqKbK8iSNdzhZ0cDXr1XUWPatj Mb5+IZp3uI5qEzq890B+EBWn6zD2Mlt9q0X9F+f49R41AmoqleiQf1AYk zjfxpTieiBA9sRVzPc7sjiQOaVCz3PrYOZauFAw28/rwd5uKTiOJVZQnG Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="3069507" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="3069507" Received: from orviesa004.jf.intel.com ([10.64.159.144]) by orvoesa111.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:38 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="11272534" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:37 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 080/130] KVM: TDX: restore host xsave state when exit from the guest TD Date: Mon, 26 Feb 2024 00:26:22 -0800 Message-Id: <2894ed10014279f4b8caab582e3b7e7061b5dad3.1708933498.git.isaku.yamahata@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata On exiting from the guest TD, xsave state is clobbered. Restore xsave state on TD exit. Signed-off-by: Isaku Yamahata --- v19: - Add EXPORT_SYMBOL_GPL(host_xcr0) v15 -> v16: - Added CET flag mask Signed-off-by: Isaku Yamahata --- arch/x86/kvm/vmx/tdx.c | 19 +++++++++++++++++++ arch/x86/kvm/x86.c | 1 + 2 files changed, 20 insertions(+) diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c index 9616b1aab6ce..199226c6cf55 100644 --- a/arch/x86/kvm/vmx/tdx.c +++ b/arch/x86/kvm/vmx/tdx.c @@ -2,6 +2,7 @@ #include #include +#include #include #include "capabilities.h" @@ -534,6 +535,23 @@ void tdx_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event) */ } +static void tdx_restore_host_xsave_state(struct kvm_vcpu *vcpu) +{ + struct kvm_tdx *kvm_tdx = to_kvm_tdx(vcpu->kvm); + + if (static_cpu_has(X86_FEATURE_XSAVE) && + host_xcr0 != (kvm_tdx->xfam & kvm_caps.supported_xcr0)) + xsetbv(XCR_XFEATURE_ENABLED_MASK, host_xcr0); + if (static_cpu_has(X86_FEATURE_XSAVES) && + /* PT can be exposed to TD guest regardless of KVM's XSS support */ + host_xss != (kvm_tdx->xfam & + (kvm_caps.supported_xss | XFEATURE_MASK_PT | TDX_TD_XFAM_CET))) + wrmsrl(MSR_IA32_XSS, host_xss); + if (static_cpu_has(X86_FEATURE_PKU) && + (kvm_tdx->xfam & XFEATURE_MASK_PKRU)) + write_pkru(vcpu->arch.host_pkru); +} + static noinstr void tdx_vcpu_enter_exit(struct vcpu_tdx *tdx) { struct tdx_module_args args; @@ -609,6 +627,7 @@ fastpath_t tdx_vcpu_run(struct kvm_vcpu *vcpu) tdx_vcpu_enter_exit(tdx); + tdx_restore_host_xsave_state(vcpu); tdx->host_state_need_restore = true; vcpu->arch.regs_avail &= ~VMX_REGS_LAZY_LOAD_SET; diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 23ece956c816..b361d948140f 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -315,6 +315,7 @@ const struct kvm_stats_header kvm_vcpu_stats_header = { }; u64 __read_mostly host_xcr0; +EXPORT_SYMBOL_GPL(host_xcr0); static struct kmem_cache *x86_emulator_cache; From patchwork Mon Feb 26 08:26:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571509 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D25296F071; Mon, 26 Feb 2024 08:28:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.19 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936121; cv=none; b=u0VTbfOy/e0mmTT+hCdTv/H2X18+zXQSBYZI8NMpSRSkSFKlHQP0C6X4IdD19DlH1mQgBchOZDdbdv1/nWDe9sG/wkEv8yj2r6jSKLA/Vtl4HRCDCVDIYcgcAiyw0OSQ2gd1Fvtxe6AhVIFfnsn/Tzy6R4QnAQowZ1KxPjWVdQI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936121; c=relaxed/simple; bh=aeEZd97eChWMmsHRdCg4ghs1LM1YPE3cIvcpKx5Pzcc=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=BJTBzNNTW36bZqDMor7IhAM96b94KRSv0szgo4E8XWhjPZeRvIi7rMtWyT6gPkrKPx1ruPuruamxLfl+VqCHEw553SUkLYyu1ANVpL5t0zACGNaajOM4Jf/O2M7SrL96D8Ngvs0RTo6wUIGzIZM6aNkA2536puB/VwXcvmoFtV8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=highW9cf; arc=none smtp.client-ip=198.175.65.19 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="highW9cf" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936120; x=1740472120; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=aeEZd97eChWMmsHRdCg4ghs1LM1YPE3cIvcpKx5Pzcc=; b=highW9cfKl1Y64namPsCx6lYBITBSvEGMeuvayvO49Sovvpahr57/JZg nuaQBVfV04Ego92BR1z5uPGv5D9j43Zob44qfeQAxodrEU0IUll0pyBGr pSlFdt0SMgikhgc1oULtm1067Oh78sjiIv6mctcIBymHWQwpNWSNUiIPg ePbLdAE/7/YCuXUfyuaLtzf2L+hUBv9WzSNCLN7ynlRETjCtxbkDLFDNl Bfwg56dmwxINDxyi7sEdkQyMxX1ERDPN+nd7zcxxrFp6T+VM/VWkYBIOk 1srTmGDAefOFQq4A/XhfTJuB0YZsvUJarFB8ztifhnk1fDwxw6FtZM+8Y Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="3069511" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="3069511" Received: from orviesa004.jf.intel.com ([10.64.159.144]) by orvoesa111.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:39 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="11272548" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:38 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com, Chao Gao Subject: [PATCH v19 081/130] KVM: x86: Allow to update cached values in kvm_user_return_msrs w/o wrmsr Date: Mon, 26 Feb 2024 00:26:23 -0800 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Chao Gao Several MSRs are constant and only used in userspace(ring 3). But VMs may have different values. KVM uses kvm_set_user_return_msr() to switch to guest's values and leverages user return notifier to restore them when the kernel is to return to userspace. To eliminate unnecessary wrmsr, KVM also caches the value it wrote to an MSR last time. TDX module unconditionally resets some of these MSRs to architectural INIT state on TD exit. It makes the cached values in kvm_user_return_msrs are inconsistent with values in hardware. This inconsistency needs to be fixed. Otherwise, it may mislead kvm_on_user_return() to skip restoring some MSRs to the host's values. kvm_set_user_return_msr() can help correct this case, but it is not optimal as it always does a wrmsr. So, introduce a variation of kvm_set_user_return_msr() to update cached values and skip that wrmsr. Signed-off-by: Chao Gao Signed-off-by: Isaku Yamahata Reviewed-by: Paolo Bonzini --- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/x86.c | 25 ++++++++++++++++++++----- 2 files changed, 21 insertions(+), 5 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 36694e784c27..3ab85c3d86ee 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -2259,6 +2259,7 @@ int kvm_pv_send_ipi(struct kvm *kvm, unsigned long ipi_bitmap_low, int kvm_add_user_return_msr(u32 msr); int kvm_find_user_return_msr(u32 msr); int kvm_set_user_return_msr(unsigned index, u64 val, u64 mask); +void kvm_user_return_update_cache(unsigned int index, u64 val); static inline bool kvm_is_supported_user_return_msr(u32 msr) { diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index b361d948140f..1b189e86a1f1 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -440,6 +440,15 @@ static void kvm_user_return_msr_cpu_online(void) } } +static void kvm_user_return_register_notifier(struct kvm_user_return_msrs *msrs) +{ + if (!msrs->registered) { + msrs->urn.on_user_return = kvm_on_user_return; + user_return_notifier_register(&msrs->urn); + msrs->registered = true; + } +} + int kvm_set_user_return_msr(unsigned slot, u64 value, u64 mask) { unsigned int cpu = smp_processor_id(); @@ -454,15 +463,21 @@ int kvm_set_user_return_msr(unsigned slot, u64 value, u64 mask) return 1; msrs->values[slot].curr = value; - if (!msrs->registered) { - msrs->urn.on_user_return = kvm_on_user_return; - user_return_notifier_register(&msrs->urn); - msrs->registered = true; - } + kvm_user_return_register_notifier(msrs); return 0; } EXPORT_SYMBOL_GPL(kvm_set_user_return_msr); +/* Update the cache, "curr", and register the notifier */ +void kvm_user_return_update_cache(unsigned int slot, u64 value) +{ + struct kvm_user_return_msrs *msrs = this_cpu_ptr(user_return_msrs); + + msrs->values[slot].curr = value; + kvm_user_return_register_notifier(msrs); +} +EXPORT_SYMBOL_GPL(kvm_user_return_update_cache); + static void drop_user_return_notifiers(void) { unsigned int cpu = smp_processor_id(); From patchwork Mon Feb 26 08:26:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571510 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F030671B3D; Mon, 26 Feb 2024 08:28:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.19 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936122; cv=none; b=VRAXNSrOb/zHSyJKZkcqj3unPqAJDpJU+Mlg5bsJ/1lfA5MJBNZVFganAgN4HWDiYGjgPnpPSDAKYhbjczJ1OawcvrKKiFMnxlV6X+uzMxkUPV405AfrZl3oshkAl84XJl8LpsXEO16C/g6/mRrzkPHGTdZMmw3KXz2QrpdvcG0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936122; c=relaxed/simple; bh=AYp+IzNByZIZ5yaCRPVPJqWoxGQPDva5mwiKOWh6qJY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=J0qGi+96ZUxh4xeolL50qgHwLmzw9BXuuA7rmhrF/D2Yz9zujt/thWsqBcMYowoFi8zfAPihP5fvBqzN2WDVHugLdHvF7RrqKNAtcZLOfzyd8QMEmSBBdaCNmW6REnfGDl26oCRTcS9eCLIDUu6W1JQy7eGboeUCdbEl70nb5MM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=ZwdUl8DL; arc=none smtp.client-ip=198.175.65.19 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="ZwdUl8DL" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936121; x=1740472121; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=AYp+IzNByZIZ5yaCRPVPJqWoxGQPDva5mwiKOWh6qJY=; b=ZwdUl8DLWqXC+5Ivn/kaGFe0bVgBV0MWFnZJzKAHahMj22HzrwtPD7sI N45c8gTncLaqNEwchhkReZMDa9yNJCLUhubBHZr13tgVwozqKh9jIJsRR whd27qlGqZWMXq9n0ILYY78y73W2XueH0S2qmLuo4PhhyH8yLq2ARnbyC ZQRwByi9h/tjhwamGq4tHMjYrqVCak6hcHtZ1p8rO/wU2aP1o8RucCw3G nIoZRzo0Q92kWrimJSCzmvYDFO4jr65nWDbYaGn/hT4ooI/NlG0YUVa2w KgT22B9OetDHphYpbv/lk0YHG6anDAkznpXHxUQmQ+mgUCcAZQU6oiuGv A==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="3069517" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="3069517" Received: from orviesa004.jf.intel.com ([10.64.159.144]) by orvoesa111.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:40 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="11272555" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:39 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 082/130] KVM: TDX: restore user ret MSRs Date: Mon, 26 Feb 2024 00:26:24 -0800 Message-Id: <8ba41a08c98034fd4f3886791d1d068b0d390f86.1708933498.git.isaku.yamahata@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata Several user ret MSRs are clobbered on TD exit. Restore those values on TD exit and before returning to ring 3. Because TSX_CTRL requires special treat, this patch doesn't address it. Signed-off-by: Isaku Yamahata Reviewed-by: Paolo Bonzini --- arch/x86/kvm/vmx/tdx.c | 43 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 43 insertions(+) diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c index 199226c6cf55..7e2b1e554246 100644 --- a/arch/x86/kvm/vmx/tdx.c +++ b/arch/x86/kvm/vmx/tdx.c @@ -535,6 +535,28 @@ void tdx_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event) */ } +struct tdx_uret_msr { + u32 msr; + unsigned int slot; + u64 defval; +}; + +static struct tdx_uret_msr tdx_uret_msrs[] = { + {.msr = MSR_SYSCALL_MASK, .defval = 0x20200 }, + {.msr = MSR_STAR,}, + {.msr = MSR_LSTAR,}, + {.msr = MSR_TSC_AUX,}, +}; + +static void tdx_user_return_update_cache(void) +{ + int i; + + for (i = 0; i < ARRAY_SIZE(tdx_uret_msrs); i++) + kvm_user_return_update_cache(tdx_uret_msrs[i].slot, + tdx_uret_msrs[i].defval); +} + static void tdx_restore_host_xsave_state(struct kvm_vcpu *vcpu) { struct kvm_tdx *kvm_tdx = to_kvm_tdx(vcpu->kvm); @@ -627,6 +649,7 @@ fastpath_t tdx_vcpu_run(struct kvm_vcpu *vcpu) tdx_vcpu_enter_exit(tdx); + tdx_user_return_update_cache(); tdx_restore_host_xsave_state(vcpu); tdx->host_state_need_restore = true; @@ -1972,6 +1995,26 @@ int __init tdx_hardware_setup(struct kvm_x86_ops *x86_ops) return -EINVAL; } + for (i = 0; i < ARRAY_SIZE(tdx_uret_msrs); i++) { + /* + * Here it checks if MSRs (tdx_uret_msrs) can be saved/restored + * before returning to user space. + * + * this_cpu_ptr(user_return_msrs)->registered isn't checked + * because the registration is done at vcpu runtime by + * kvm_set_user_return_msr(). + * Here is setting up cpu feature before running vcpu, + * registered is already false. + */ + tdx_uret_msrs[i].slot = kvm_find_user_return_msr(tdx_uret_msrs[i].msr); + if (tdx_uret_msrs[i].slot == -1) { + /* If any MSR isn't supported, it is a KVM bug */ + pr_err("MSR %x isn't included by kvm_find_user_return_msr\n", + tdx_uret_msrs[i].msr); + return -EIO; + } + } + max_pkgs = topology_max_packages(); tdx_mng_key_config_lock = kcalloc(max_pkgs, sizeof(*tdx_mng_key_config_lock), GFP_KERNEL); From patchwork Mon Feb 26 08:26:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571512 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 27A2E73F1F; Mon, 26 Feb 2024 08:28:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.19 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936123; cv=none; b=Id1x6DYdYajyXn2F/FI/BmCfZpeRKQQrFeH8BwdXQp+kkJjyRf25GSrtl+/5CsOUkDTPHzxcKs1YqETgim5LiHcogtWVV8/P5acgkglxy1XNQY49FJCtUfxDbyAkUQmL64zXsmotgI5TgE5YusX9SrTPoCB4htWyKbDxZgyi5+o= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936123; c=relaxed/simple; bh=QAWpp4LjklCnK07uK2feZ4n6OzER6GSYaRkObJrBCqk=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=q9PdOcmIH5YL5VofnwfxHT7j/AE8bUyh7Xe9Taujj7ON8TgkDmVw2lu0KvKNOWFr50H4suW8hPhGSWgKQVwzvuNO/tM/D1+v5IawPoeXkI4Zjxa5uNPi6HpQkKLq+IwPukYnA3wRoEAeZtlcbs6L2XR7GBDEPQfn6zDgVX549B4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=nu6Bu72i; arc=none smtp.client-ip=198.175.65.19 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="nu6Bu72i" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936121; x=1740472121; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=QAWpp4LjklCnK07uK2feZ4n6OzER6GSYaRkObJrBCqk=; b=nu6Bu72iExAw+Ha37v+ax1J+digzGAae9PZfXnRdBGF4Tcu9dyKUZReS CywflQCdhHZWzva/P4rBGQ+dwDAUqr60LfVsfbR5rbDAdvnwFAgII0U87 v2b8IRSJuKN0lhk46ZZ9BoeUAumb9tPc4JtQvrHIvQsdr8AXffkwwV3WC IbGBkD/bnf+2uUDlKRg0oJ0jYy8ydEC4nCCPotW8GC+qBZx3evBO0140X Kxfhtmg99LWud5+zlTVgZFcmbmERS9/aTJ/T/cuNEAULXNr5j9Y2gPALZ pW/q//GklbTDS4Cx3cefWnPIKOgTMZDhQrrUGAxya07dzq7ODV/5v2/pP Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="3069521" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="3069521" Received: from orviesa004.jf.intel.com ([10.64.159.144]) by orvoesa111.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:41 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="11272563" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:40 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com, Yang Weijiang Subject: [PATCH v19 083/130] KVM: TDX: Add TSX_CTRL msr into uret_msrs list Date: Mon, 26 Feb 2024 00:26:25 -0800 Message-Id: <06135e0897ae90c3dc7fd608948f8bdcd30a17ae.1708933498.git.isaku.yamahata@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Yang Weijiang TDX module resets the TSX_CTRL MSR to 0 at TD exit if TSX is enabled for TD. Or it preserves the TSX_CTRL MSR if TSX is disabled for TD. VMM can rely on uret_msrs mechanism to defer the reload of host value until exiting to user space. Signed-off-by: Yang Weijiang Signed-off-by: Isaku Yamahata --- v19: - fix the type of tdx_uret_tsx_ctrl_slot. unguent int => int. --- arch/x86/kvm/vmx/tdx.c | 33 +++++++++++++++++++++++++++++++-- arch/x86/kvm/vmx/tdx.h | 8 ++++++++ 2 files changed, 39 insertions(+), 2 deletions(-) diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c index 7e2b1e554246..83dcaf5b6fbd 100644 --- a/arch/x86/kvm/vmx/tdx.c +++ b/arch/x86/kvm/vmx/tdx.c @@ -547,14 +547,21 @@ static struct tdx_uret_msr tdx_uret_msrs[] = { {.msr = MSR_LSTAR,}, {.msr = MSR_TSC_AUX,}, }; +static int tdx_uret_tsx_ctrl_slot; -static void tdx_user_return_update_cache(void) +static void tdx_user_return_update_cache(struct kvm_vcpu *vcpu) { int i; for (i = 0; i < ARRAY_SIZE(tdx_uret_msrs); i++) kvm_user_return_update_cache(tdx_uret_msrs[i].slot, tdx_uret_msrs[i].defval); + /* + * TSX_CTRL is reset to 0 if guest TSX is supported. Otherwise + * preserved. + */ + if (to_kvm_tdx(vcpu->kvm)->tsx_supported && tdx_uret_tsx_ctrl_slot != -1) + kvm_user_return_update_cache(tdx_uret_tsx_ctrl_slot, 0); } static void tdx_restore_host_xsave_state(struct kvm_vcpu *vcpu) @@ -649,7 +656,7 @@ fastpath_t tdx_vcpu_run(struct kvm_vcpu *vcpu) tdx_vcpu_enter_exit(tdx); - tdx_user_return_update_cache(); + tdx_user_return_update_cache(vcpu); tdx_restore_host_xsave_state(vcpu); tdx->host_state_need_restore = true; @@ -1167,6 +1174,22 @@ static int setup_tdparams_xfam(struct kvm_cpuid2 *cpuid, struct td_params *td_pa return 0; } +static bool tdparams_tsx_supported(struct kvm_cpuid2 *cpuid) +{ + const struct kvm_cpuid_entry2 *entry; + u64 mask; + u32 ebx; + + entry = kvm_find_cpuid_entry2(cpuid->entries, cpuid->nent, 0x7, 0); + if (entry) + ebx = entry->ebx; + else + ebx = 0; + + mask = __feature_bit(X86_FEATURE_HLE) | __feature_bit(X86_FEATURE_RTM); + return ebx & mask; +} + static int setup_tdparams(struct kvm *kvm, struct td_params *td_params, struct kvm_tdx_init_vm *init_vm) { @@ -1209,6 +1232,7 @@ static int setup_tdparams(struct kvm *kvm, struct td_params *td_params, MEMCPY_SAME_SIZE(td_params->mrowner, init_vm->mrowner); MEMCPY_SAME_SIZE(td_params->mrownerconfig, init_vm->mrownerconfig); + to_kvm_tdx(kvm)->tsx_supported = tdparams_tsx_supported(cpuid); return 0; } @@ -2014,6 +2038,11 @@ int __init tdx_hardware_setup(struct kvm_x86_ops *x86_ops) return -EIO; } } + tdx_uret_tsx_ctrl_slot = kvm_find_user_return_msr(MSR_IA32_TSX_CTRL); + if (tdx_uret_tsx_ctrl_slot == -1 && boot_cpu_has(X86_FEATURE_MSR_TSX_CTRL)) { + pr_err("MSR_IA32_TSX_CTRL isn't included by kvm_find_user_return_msr\n"); + return -EIO; + } max_pkgs = topology_max_packages(); tdx_mng_key_config_lock = kcalloc(max_pkgs, sizeof(*tdx_mng_key_config_lock), diff --git a/arch/x86/kvm/vmx/tdx.h b/arch/x86/kvm/vmx/tdx.h index e96c416e73bf..44eab734e702 100644 --- a/arch/x86/kvm/vmx/tdx.h +++ b/arch/x86/kvm/vmx/tdx.h @@ -17,6 +17,14 @@ struct kvm_tdx { u64 xfam; int hkid; + /* + * Used on each TD-exit, see tdx_user_return_update_cache(). + * TSX_CTRL value on TD exit + * - set 0 if guest TSX enabled + * - preserved if guest TSX disabled + */ + bool tsx_supported; + bool finalized; atomic_t tdh_mem_track; From patchwork Mon Feb 26 08:26:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571511 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0463962A09; Mon, 26 Feb 2024 08:28:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.19 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936123; cv=none; b=X7lohWvRehkKhGURYUcxs72QemZ9GneVFNTy6+THHFzml5OQ7Ab/azZK2qcyj1RAnjd8Amy7IkmkVxXCQ/KB2tVf8dJEu2nOXLbQe6n71ZHCmDYJUWshNXau6wW+jn/ysIXr+nxUEZP5l84nO+2teiNjRYmxUYvQHpzTSUytX9w= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936123; c=relaxed/simple; bh=J0qMXx4Z1bmL0Bo7ShUuSxkVx6XY1Y4HAlOmBvk4LIo=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Vi4+s8zXfw+u79epR4L+vmqIICqu8pHrvFZu2jrrHi/uJCU8Jl6rKxuPOq3Fn4+KvJuHSBu6CV3RQQ++8BehXD2ei4JwCOJStqJ5A+CWGa+uwtsE4a/yGCXZLvJpioBhFkmHlAaD+GnEBjsLhQdQRwnLf7VJ79FAvK5DEQKvMp4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=HuTfOZXj; arc=none smtp.client-ip=198.175.65.19 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="HuTfOZXj" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936122; x=1740472122; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=J0qMXx4Z1bmL0Bo7ShUuSxkVx6XY1Y4HAlOmBvk4LIo=; b=HuTfOZXjCpF/+oHRJaIz5kHiAMFTc8/1v8Lfxi0Hn/gYk0yQuBtAewq/ 9Z97pH/cEN/cIIVWPJTCzXmJ2PFyd3lnQv5MDkJjv5GhDTq6r8hnRMtkx yt/0Dn3JwfdVCwc6mYdYXQIFoFScYV4biNncuI3DO3jlUwVxnLdfddDqm YulJ6F7dYefHsmHxmKI6BlIvGRDcksC9Fp5QaBhDPD/gGk6qLinsWF+DT 2DbgRwOAzxbSgaHQ3Jb2Abd8/NsSWvsBxuKiPfHcyrMhEwI/BetwX2TO1 pqh0Fn+e4fSPiOeZzDqs/naua7XZoBhR4n/yGfP7bsdpulJYHzRpVeRJP g==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="3069525" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="3069525" Received: from orviesa004.jf.intel.com ([10.64.159.144]) by orvoesa111.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:41 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="11272570" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:41 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 084/130] [MARKER] The start of TDX KVM patch series: TD vcpu exits/interrupts/hypercalls Date: Mon, 26 Feb 2024 00:26:26 -0800 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata This empty commit is to mark the start of patch series of TD vcpu exits, interrupts, and hypercalls. Signed-off-by: Isaku Yamahata --- Documentation/virt/kvm/intel-tdx-layer-status.rst | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/Documentation/virt/kvm/intel-tdx-layer-status.rst b/Documentation/virt/kvm/intel-tdx-layer-status.rst index 33e107bcb5cf..7a16fa284b6f 100644 --- a/Documentation/virt/kvm/intel-tdx-layer-status.rst +++ b/Documentation/virt/kvm/intel-tdx-layer-status.rst @@ -13,6 +13,7 @@ What qemu can do - Qemu can create/destroy vcpu of TDX vm type. - Qemu can populate initial guest memory image. - Qemu can finalize guest TD. +- Qemu can start to run vcpu. But vcpu can not make progress yet. Patch Layer status ------------------ @@ -24,7 +25,7 @@ Patch Layer status * TD vcpu creation/destruction: Applied * TDX EPT violation: Applied * TD finalization: Applied -* TD vcpu enter/exit: Applying +* TD vcpu enter/exit: Applied * TD vcpu interrupts/exit/hypercall: Not yet * KVM MMU GPA shared bits: Applied From patchwork Mon Feb 26 08:26:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571513 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E831F77A0E; Mon, 26 Feb 2024 08:28:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.19 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936124; cv=none; b=naC9StShK8ALOm/UTQQEpS0Ue/wN7bssa3BCunW3Rt7ANPhBg/5QUrTEJiDNNv+uxp4NeAF249uegQ5AnG9PYkfiA0ZaeBXf3Upv0NOVqiOERCUhcHMyXkdqZx8uUiXPoQbbtqEYMz2qau1Uhj5dq6J4i6sYjjSTpJZ0OK+WiiY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936124; c=relaxed/simple; bh=d16HpBuwInS8Tw2Ei/8iq4CZW3uueOnKvGugn8F5q+Q=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=tX6c46R/69xmEjBZjUGLz7uA83ZUwh15d4TfQ6+EzKpMc9s2tOP9Zjt1PMD8xvx774gETj2YCCefAdRH73XM/l2QwZOxTtlWlcJeCCXOJsF5+pEQi76Eys1kXpnKsGTcYjDdwxwm3wq4JALAOUz/QqUcY1/pZ5cfR1uGmaV0Vmg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=FCvGvELX; arc=none smtp.client-ip=198.175.65.19 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="FCvGvELX" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936123; x=1740472123; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=d16HpBuwInS8Tw2Ei/8iq4CZW3uueOnKvGugn8F5q+Q=; b=FCvGvELXmKt3QrxVDZt7OmY9LRVIVwB6avar2149XGqho1QiTDqi+LvN XXFUcyQyGBVBcu/P3Hezj+QVbIjJpSgSdUT0u63/OPq/r7nFAxES9hLZb 07JKcDgt6ChoP/zf9d+t8ef7O/hBuzwyCUXcXaih3l5gsxw8WbYX4BDtn x1AAdl1okVA1rRjIffdDNSNs01LMMqWawAqdoIqLMI/CmMWLCEYjImDdG q2JjEp4otB6SP4JdxdsqlEa3f+VonJqhZg4qfwYYdYbyALaKSFceoxg1U jtGEzbEqAE7y+RBq3jVyf2UvRLlMcf5v3VtrZ3FBHxFZLYD9G2Yuqs/Nk A==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="3069529" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="3069529" Received: from orviesa004.jf.intel.com ([10.64.159.144]) by orvoesa111.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:43 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="11272581" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:42 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com, Binbin Wu Subject: [PATCH v19 085/130] KVM: TDX: Complete interrupts after tdexit Date: Mon, 26 Feb 2024 00:26:27 -0800 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata This corresponds to VMX __vmx_complete_interrupts(). Because TDX virtualize vAPIC, KVM only needs to care NMI injection. Signed-off-by: Isaku Yamahata Reviewed-by: Paolo Bonzini Reviewed-by: Binbin Wu --- v19: - move tdvps_management_check() to this patch - typo: complete -> Complete in short log --- arch/x86/kvm/vmx/tdx.c | 10 ++++++++++ arch/x86/kvm/vmx/tdx.h | 4 ++++ 2 files changed, 14 insertions(+) diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c index 83dcaf5b6fbd..b8b168f74dfe 100644 --- a/arch/x86/kvm/vmx/tdx.c +++ b/arch/x86/kvm/vmx/tdx.c @@ -535,6 +535,14 @@ void tdx_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event) */ } +static void tdx_complete_interrupts(struct kvm_vcpu *vcpu) +{ + /* Avoid costly SEAMCALL if no nmi was injected */ + if (vcpu->arch.nmi_injected) + vcpu->arch.nmi_injected = td_management_read8(to_tdx(vcpu), + TD_VCPU_PEND_NMI); +} + struct tdx_uret_msr { u32 msr; unsigned int slot; @@ -663,6 +671,8 @@ fastpath_t tdx_vcpu_run(struct kvm_vcpu *vcpu) vcpu->arch.regs_avail &= ~VMX_REGS_LAZY_LOAD_SET; trace_kvm_exit(vcpu, KVM_ISA_VMX); + tdx_complete_interrupts(vcpu); + return EXIT_FASTPATH_NONE; } diff --git a/arch/x86/kvm/vmx/tdx.h b/arch/x86/kvm/vmx/tdx.h index 44eab734e702..0d8a98feb58e 100644 --- a/arch/x86/kvm/vmx/tdx.h +++ b/arch/x86/kvm/vmx/tdx.h @@ -142,6 +142,8 @@ static __always_inline void tdvps_vmcs_check(u32 field, u8 bits) "Invalid TD VMCS access for 16-bit field"); } +static __always_inline void tdvps_management_check(u64 field, u8 bits) {} + #define TDX_BUILD_TDVPS_ACCESSORS(bits, uclass, lclass) \ static __always_inline u##bits td_##lclass##_read##bits(struct vcpu_tdx *tdx, \ u32 field) \ @@ -200,6 +202,8 @@ TDX_BUILD_TDVPS_ACCESSORS(16, VMCS, vmcs); TDX_BUILD_TDVPS_ACCESSORS(32, VMCS, vmcs); TDX_BUILD_TDVPS_ACCESSORS(64, VMCS, vmcs); +TDX_BUILD_TDVPS_ACCESSORS(8, MANAGEMENT, management); + static __always_inline u64 td_tdcs_exec_read64(struct kvm_tdx *kvm_tdx, u32 field) { struct tdx_module_args out; From patchwork Mon Feb 26 08:26:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571514 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B41C877F1E; Mon, 26 Feb 2024 08:28:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.19 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936125; cv=none; b=o8Tfle4jvtAjxltikrT+OgvbYciQtvAkjvjW4JaGeCbtbAOqTRGyO4mPmfbd6TV1c2IiNtnenHfSVfJwN2oPsPeCR2dwGcNU1HnP1458FTO0jQ0SD2aTICfN0x7N4oT6KGP/ETpRlJbhK4xSbM/6hR74A4uVar4CJVaZopUbQPc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936125; c=relaxed/simple; bh=myKJ2+UbusG4kdK0f0eyTWXSAekqS+lLcnY6mljYNTQ=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=AYuirMiUVGXd+84OsTRSub90edb96Gwtdjod4GOsYkcyQZeVl6Pd6H/UCJ+ZKWD8G5yg4ZZLw9pWo4L1KnuIYWXdMCmHBe5P5bR1RmBO0aYXqVZnODSO5H2RSg0EZSAKUq4apYRNOrnNXWnyRr+/0K0DNza2gUPz1phOYl8kJBQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=fjBvjtBD; arc=none smtp.client-ip=198.175.65.19 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="fjBvjtBD" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936124; x=1740472124; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=myKJ2+UbusG4kdK0f0eyTWXSAekqS+lLcnY6mljYNTQ=; b=fjBvjtBD7SV3uB3T6tDxVmrDweBu1mkNEccDin1Ru2NIzBqVZK9pN8tl UnKYKjZ/fDVFkd7WXeqZ4R5H8kRON7A/4bnbvuNHBVbYyGCUQ6skjtvbO bKksCrc7MgM9ijLFOgPVLGwEzoVchZc/Okmy5WzqY5vVGgiq6OuBN+bye kvija0MnXYurYGKrBN4KwlgSiQTRMsGs8FW2qQrfkxawCKOd/FN6+L08d tU7K8DZdKv/2djfYXDC/pOt9hCYTMLTGAEnk4F2lzWY749cgd9sdJAvwZ nVKHEhoR1opUPMnlA7tgXxhbVpHEkPjXuWwXNvPbVb7Zh5DZYVuSQoH4l w==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="3069534" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="3069534" Received: from orviesa004.jf.intel.com ([10.64.159.144]) by orvoesa111.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:44 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="11272593" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:43 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 086/130] KVM: TDX: restore debug store when TD exit Date: Mon, 26 Feb 2024 00:26:28 -0800 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata Because debug store is clobbered, restore it on TD exit. Signed-off-by: Isaku Yamahata Reviewed-by: Paolo Bonzini --- arch/x86/events/intel/ds.c | 1 + arch/x86/kvm/vmx/tdx.c | 1 + 2 files changed, 2 insertions(+) diff --git a/arch/x86/events/intel/ds.c b/arch/x86/events/intel/ds.c index d49d661ec0a7..25670d8a485b 100644 --- a/arch/x86/events/intel/ds.c +++ b/arch/x86/events/intel/ds.c @@ -2428,3 +2428,4 @@ void perf_restore_debug_store(void) wrmsrl(MSR_IA32_DS_AREA, (unsigned long)ds); } +EXPORT_SYMBOL_GPL(perf_restore_debug_store); diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c index b8b168f74dfe..ad4d3d4eaf6c 100644 --- a/arch/x86/kvm/vmx/tdx.c +++ b/arch/x86/kvm/vmx/tdx.c @@ -665,6 +665,7 @@ fastpath_t tdx_vcpu_run(struct kvm_vcpu *vcpu) tdx_vcpu_enter_exit(tdx); tdx_user_return_update_cache(vcpu); + perf_restore_debug_store(); tdx_restore_host_xsave_state(vcpu); tdx->host_state_need_restore = true; From patchwork Mon Feb 26 08:26:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571515 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C2CF41F947; Mon, 26 Feb 2024 08:28:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.19 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936128; cv=none; b=lnoUTD9SV1/mBjUFEHsNjJNWXpaMzY2oiy49MD8IyITF34MPbS/2PftRtu6QCEPYD2MyspzvpF51TG9j+to3AkVnDgeDNFRfZZH/f06+0euCNv77AXq1MrlRlruqN1pRayJQUVA22eBw5PzIAt0yoxfyZkgNmL5tYBCtNtuXqjo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936128; c=relaxed/simple; bh=0jXFsyFxqksmMvFjQSUBOYDFyGyaHIa0Wn3nGU2QdQs=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=MrqbHD7QH6BpOBcoVNGMArMMlYn/EEEHG3onR5tx/+HyMXVHkECm/mRL7SckJufKVqd5Z0PzarU/Hk5h3RS+y0TqqhpJCPNwgDQLgsBeOlL/WpWdyHSHT1jM6d4ZLrnt+Fmh2MMtdK+Ky49DgZfI1ATMabQ8nA37XRzMs4lzz8E= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=PeFYcRJb; arc=none smtp.client-ip=198.175.65.19 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="PeFYcRJb" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936126; x=1740472126; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=0jXFsyFxqksmMvFjQSUBOYDFyGyaHIa0Wn3nGU2QdQs=; b=PeFYcRJbknxadnqYs3t/tCXMYDVSigCINxTwtGc9bl9r3Wfea26bSfrf R0XtlF1DHo8evpbN5YQnbXF8fKD90eOhVBy0POZLgEAYkMI0nov/3QQFv +pHk/h5B1+agUxfjp1cf4Fky7BeoQI1tSEFQak184PNRF6q3yVDFI4FGw A4s9fQwDSDp5BbuXsKp8U/NDYYiHDoPgYTDOSatjxDXb/4Urhxy0uT985 dyJqq8OAdgfQbDff2SMSeaA4s3cL+u4Glira6f0L84NgY07tTk7jcJYAF H0ldsklKc6FvaQ0RNHD3fsgNIOBlXAe12shEfaG0jFSXnubUfdesd69f/ A==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="3069541" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="3069541" Received: from orviesa004.jf.intel.com ([10.64.159.144]) by orvoesa111.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:46 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="11272616" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:45 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 087/130] KVM: TDX: handle vcpu migration over logical processor Date: Mon, 26 Feb 2024 00:26:29 -0800 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata For vcpu migration, in the case of VMX, VMCS is flushed on the source pcpu, and load it on the target pcpu. There are corresponding TDX SEAMCALL APIs, call them on vcpu migration. The logic is mostly same as VMX except the TDX SEAMCALLs are used. When shutting down the machine, (VMX or TDX) vcpus needs to be shutdown on each pcpu. Do the similar for TDX with TDX SEAMCALL APIs. Signed-off-by: Isaku Yamahata --- arch/x86/kvm/vmx/main.c | 32 ++++++- arch/x86/kvm/vmx/tdx.c | 190 ++++++++++++++++++++++++++++++++++++- arch/x86/kvm/vmx/tdx.h | 2 + arch/x86/kvm/vmx/x86_ops.h | 4 + 4 files changed, 221 insertions(+), 7 deletions(-) diff --git a/arch/x86/kvm/vmx/main.c b/arch/x86/kvm/vmx/main.c index 8275a242ce07..9b336c1a6508 100644 --- a/arch/x86/kvm/vmx/main.c +++ b/arch/x86/kvm/vmx/main.c @@ -33,6 +33,14 @@ static int vt_max_vcpus(struct kvm *kvm) static int vt_flush_remote_tlbs(struct kvm *kvm); #endif +static void vt_hardware_disable(void) +{ + /* Note, TDX *and* VMX need to be disabled if TDX is enabled. */ + if (enable_tdx) + tdx_hardware_disable(); + vmx_hardware_disable(); +} + static __init int vt_hardware_setup(void) { int ret; @@ -201,6 +209,16 @@ static fastpath_t vt_vcpu_run(struct kvm_vcpu *vcpu) return vmx_vcpu_run(vcpu); } +static void vt_vcpu_load(struct kvm_vcpu *vcpu, int cpu) +{ + if (is_td_vcpu(vcpu)) { + tdx_vcpu_load(vcpu, cpu); + return; + } + + vmx_vcpu_load(vcpu, cpu); +} + static void vt_flush_tlb_all(struct kvm_vcpu *vcpu) { if (is_td_vcpu(vcpu)) { @@ -262,6 +280,14 @@ static void vt_load_mmu_pgd(struct kvm_vcpu *vcpu, hpa_t root_hpa, vmx_load_mmu_pgd(vcpu, root_hpa, pgd_level); } +static void vt_sched_in(struct kvm_vcpu *vcpu, int cpu) +{ + if (is_td_vcpu(vcpu)) + return; + + vmx_sched_in(vcpu, cpu); +} + static u8 vt_get_mt_mask(struct kvm_vcpu *vcpu, gfn_t gfn, bool is_mmio) { if (is_td_vcpu(vcpu)) @@ -335,7 +361,7 @@ struct kvm_x86_ops vt_x86_ops __initdata = { /* TDX cpu enablement is done by tdx_hardware_setup(). */ .hardware_enable = vmx_hardware_enable, - .hardware_disable = vmx_hardware_disable, + .hardware_disable = vt_hardware_disable, .has_emulated_msr = vmx_has_emulated_msr, .is_vm_type_supported = vt_is_vm_type_supported, @@ -353,7 +379,7 @@ struct kvm_x86_ops vt_x86_ops __initdata = { .vcpu_reset = vt_vcpu_reset, .prepare_switch_to_guest = vt_prepare_switch_to_guest, - .vcpu_load = vmx_vcpu_load, + .vcpu_load = vt_vcpu_load, .vcpu_put = vt_vcpu_put, .update_exception_bitmap = vmx_update_exception_bitmap, @@ -440,7 +466,7 @@ struct kvm_x86_ops vt_x86_ops __initdata = { .request_immediate_exit = vmx_request_immediate_exit, - .sched_in = vmx_sched_in, + .sched_in = vt_sched_in, .cpu_dirty_log_size = PML_ENTITY_NUM, .update_cpu_dirty_logging = vmx_update_cpu_dirty_logging, diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c index ad4d3d4eaf6c..7aa9188f384d 100644 --- a/arch/x86/kvm/vmx/tdx.c +++ b/arch/x86/kvm/vmx/tdx.c @@ -106,6 +106,14 @@ static DEFINE_MUTEX(tdx_lock); static struct mutex *tdx_mng_key_config_lock; static atomic_t nr_configured_hkid; +/* + * A per-CPU list of TD vCPUs associated with a given CPU. Used when a CPU + * is brought down to invoke TDH_VP_FLUSH on the approapriate TD vCPUS. + * Protected by interrupt mask. This list is manipulated in process context + * of vcpu and IPI callback. See tdx_flush_vp_on_cpu(). + */ +static DEFINE_PER_CPU(struct list_head, associated_tdvcpus); + static __always_inline hpa_t set_hkid_to_hpa(hpa_t pa, u16 hkid) { return pa | ((hpa_t)hkid << boot_cpu_data.x86_phys_bits); @@ -138,6 +146,37 @@ static inline bool is_td_finalized(struct kvm_tdx *kvm_tdx) return kvm_tdx->finalized; } +static inline void tdx_disassociate_vp(struct kvm_vcpu *vcpu) +{ + lockdep_assert_irqs_disabled(); + + list_del(&to_tdx(vcpu)->cpu_list); + + /* + * Ensure tdx->cpu_list is updated is before setting vcpu->cpu to -1, + * otherwise, a different CPU can see vcpu->cpu = -1 and add the vCPU + * to its list before its deleted from this CPUs list. + */ + smp_wmb(); + + vcpu->cpu = -1; +} + +static void tdx_disassociate_vp_arg(void *vcpu) +{ + tdx_disassociate_vp(vcpu); +} + +static void tdx_disassociate_vp_on_cpu(struct kvm_vcpu *vcpu) +{ + int cpu = vcpu->cpu; + + if (unlikely(cpu == -1)) + return; + + smp_call_function_single(cpu, tdx_disassociate_vp_arg, vcpu, 1); +} + static void tdx_clear_page(unsigned long page_pa) { const void *zero_page = (const void *) __va(page_to_phys(ZERO_PAGE(0))); @@ -218,6 +257,87 @@ static void tdx_reclaim_control_page(unsigned long td_page_pa) free_page((unsigned long)__va(td_page_pa)); } +struct tdx_flush_vp_arg { + struct kvm_vcpu *vcpu; + u64 err; +}; + +static void tdx_flush_vp(void *arg_) +{ + struct tdx_flush_vp_arg *arg = arg_; + struct kvm_vcpu *vcpu = arg->vcpu; + u64 err; + + arg->err = 0; + lockdep_assert_irqs_disabled(); + + /* Task migration can race with CPU offlining. */ + if (unlikely(vcpu->cpu != raw_smp_processor_id())) + return; + + /* + * No need to do TDH_VP_FLUSH if the vCPU hasn't been initialized. The + * list tracking still needs to be updated so that it's correct if/when + * the vCPU does get initialized. + */ + if (is_td_vcpu_created(to_tdx(vcpu))) { + /* + * No need to retry. TDX Resources needed for TDH.VP.FLUSH are, + * TDVPR as exclusive, TDR as shared, and TDCS as shared. This + * vp flush function is called when destructing vcpu/TD or vcpu + * migration. No other thread uses TDVPR in those cases. + */ + err = tdh_vp_flush(to_tdx(vcpu)->tdvpr_pa); + if (unlikely(err && err != TDX_VCPU_NOT_ASSOCIATED)) { + /* + * This function is called in IPI context. Do not use + * printk to avoid console semaphore. + * The caller prints out the error message, instead. + */ + if (err) + arg->err = err; + } + } + + tdx_disassociate_vp(vcpu); +} + +static void tdx_flush_vp_on_cpu(struct kvm_vcpu *vcpu) +{ + struct tdx_flush_vp_arg arg = { + .vcpu = vcpu, + }; + int cpu = vcpu->cpu; + + if (unlikely(cpu == -1)) + return; + + smp_call_function_single(cpu, tdx_flush_vp, &arg, 1); + if (WARN_ON_ONCE(arg.err)) { + pr_err("cpu: %d ", cpu); + pr_tdx_error(TDH_VP_FLUSH, arg.err, NULL); + } +} + +void tdx_hardware_disable(void) +{ + int cpu = raw_smp_processor_id(); + struct list_head *tdvcpus = &per_cpu(associated_tdvcpus, cpu); + struct tdx_flush_vp_arg arg; + struct vcpu_tdx *tdx, *tmp; + unsigned long flags; + + lockdep_assert_preemption_disabled(); + + local_irq_save(flags); + /* Safe variant needed as tdx_disassociate_vp() deletes the entry. */ + list_for_each_entry_safe(tdx, tmp, tdvcpus, cpu_list) { + arg.vcpu = &tdx->vcpu; + tdx_flush_vp(&arg); + } + local_irq_restore(flags); +} + static void tdx_do_tdh_phymem_cache_wb(void *unused) { u64 err = 0; @@ -233,26 +353,31 @@ static void tdx_do_tdh_phymem_cache_wb(void *unused) pr_tdx_error(TDH_PHYMEM_CACHE_WB, err, NULL); } -void tdx_mmu_release_hkid(struct kvm *kvm) +static int __tdx_mmu_release_hkid(struct kvm *kvm) { bool packages_allocated, targets_allocated; struct kvm_tdx *kvm_tdx = to_kvm_tdx(kvm); cpumask_var_t packages, targets; + struct kvm_vcpu *vcpu; + unsigned long j; + int i, ret = 0; u64 err; - int i; if (!is_hkid_assigned(kvm_tdx)) - return; + return 0; if (!is_td_created(kvm_tdx)) { tdx_hkid_free(kvm_tdx); - return; + return 0; } packages_allocated = zalloc_cpumask_var(&packages, GFP_KERNEL); targets_allocated = zalloc_cpumask_var(&targets, GFP_KERNEL); cpus_read_lock(); + kvm_for_each_vcpu(j, vcpu, kvm) + tdx_flush_vp_on_cpu(vcpu); + /* * We can destroy multiple guest TDs simultaneously. Prevent * tdh_phymem_cache_wb from returning TDX_BUSY by serialization. @@ -270,6 +395,19 @@ void tdx_mmu_release_hkid(struct kvm *kvm) */ write_lock(&kvm->mmu_lock); + err = tdh_mng_vpflushdone(kvm_tdx->tdr_pa); + if (err == TDX_FLUSHVP_NOT_DONE) { + ret = -EBUSY; + goto out; + } + if (WARN_ON_ONCE(err)) { + pr_tdx_error(TDH_MNG_VPFLUSHDONE, err, NULL); + pr_err("tdh_mng_vpflushdone() failed. HKID %d is leaked.\n", + kvm_tdx->hkid); + ret = -EIO; + goto out; + } + for_each_online_cpu(i) { if (packages_allocated && cpumask_test_and_set_cpu(topology_physical_package_id(i), @@ -291,14 +429,24 @@ void tdx_mmu_release_hkid(struct kvm *kvm) pr_tdx_error(TDH_MNG_KEY_FREEID, err, NULL); pr_err("tdh_mng_key_freeid() failed. HKID %d is leaked.\n", kvm_tdx->hkid); + ret = -EIO; } else tdx_hkid_free(kvm_tdx); +out: write_unlock(&kvm->mmu_lock); mutex_unlock(&tdx_lock); cpus_read_unlock(); free_cpumask_var(targets); free_cpumask_var(packages); + + return ret; +} + +void tdx_mmu_release_hkid(struct kvm *kvm) +{ + while (__tdx_mmu_release_hkid(kvm) == -EBUSY) + ; } void tdx_vm_free(struct kvm *kvm) @@ -455,6 +603,26 @@ int tdx_vcpu_create(struct kvm_vcpu *vcpu) return 0; } +void tdx_vcpu_load(struct kvm_vcpu *vcpu, int cpu) +{ + struct vcpu_tdx *tdx = to_tdx(vcpu); + + if (vcpu->cpu == cpu) + return; + + tdx_flush_vp_on_cpu(vcpu); + + local_irq_disable(); + /* + * Pairs with the smp_wmb() in tdx_disassociate_vp() to ensure + * vcpu->cpu is read before tdx->cpu_list. + */ + smp_rmb(); + + list_add(&tdx->cpu_list, &per_cpu(associated_tdvcpus, cpu)); + local_irq_enable(); +} + void tdx_prepare_switch_to_guest(struct kvm_vcpu *vcpu) { struct vcpu_tdx *tdx = to_tdx(vcpu); @@ -495,6 +663,16 @@ void tdx_vcpu_free(struct kvm_vcpu *vcpu) struct vcpu_tdx *tdx = to_tdx(vcpu); int i; + /* + * When destroying VM, kvm_unload_vcpu_mmu() calls vcpu_load() for every + * vcpu after they already disassociated from the per cpu list by + * tdx_mmu_release_hkid(). So we need to disassociate them again, + * otherwise the freed vcpu data will be accessed when do + * list_{del,add}() on associated_tdvcpus list later. + */ + tdx_disassociate_vp_on_cpu(vcpu); + WARN_ON_ONCE(vcpu->cpu != -1); + /* * This methods can be called when vcpu allocation/initialization * failed. So it's possible that hkid, tdvpx and tdvpr are not assigned @@ -2030,6 +2208,10 @@ int __init tdx_hardware_setup(struct kvm_x86_ops *x86_ops) return -EINVAL; } + /* tdx_hardware_disable() uses associated_tdvcpus. */ + for_each_possible_cpu(i) + INIT_LIST_HEAD(&per_cpu(associated_tdvcpus, i)); + for (i = 0; i < ARRAY_SIZE(tdx_uret_msrs); i++) { /* * Here it checks if MSRs (tdx_uret_msrs) can be saved/restored diff --git a/arch/x86/kvm/vmx/tdx.h b/arch/x86/kvm/vmx/tdx.h index 0d8a98feb58e..7f8c78f06508 100644 --- a/arch/x86/kvm/vmx/tdx.h +++ b/arch/x86/kvm/vmx/tdx.h @@ -73,6 +73,8 @@ struct vcpu_tdx { unsigned long *tdvpx_pa; bool td_vcpu_created; + struct list_head cpu_list; + union tdx_exit_reason exit_reason; bool initialized; diff --git a/arch/x86/kvm/vmx/x86_ops.h b/arch/x86/kvm/vmx/x86_ops.h index 9fd997c79c33..5853f29f0af3 100644 --- a/arch/x86/kvm/vmx/x86_ops.h +++ b/arch/x86/kvm/vmx/x86_ops.h @@ -137,6 +137,7 @@ void vmx_setup_mce(struct kvm_vcpu *vcpu); #ifdef CONFIG_INTEL_TDX_HOST int __init tdx_hardware_setup(struct kvm_x86_ops *x86_ops); void tdx_hardware_unsetup(void); +void tdx_hardware_disable(void); bool tdx_is_vm_type_supported(unsigned long type); int tdx_offline_cpu(void); @@ -153,6 +154,7 @@ void tdx_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event); fastpath_t tdx_vcpu_run(struct kvm_vcpu *vcpu); void tdx_prepare_switch_to_guest(struct kvm_vcpu *vcpu); void tdx_vcpu_put(struct kvm_vcpu *vcpu); +void tdx_vcpu_load(struct kvm_vcpu *vcpu, int cpu); u8 tdx_get_mt_mask(struct kvm_vcpu *vcpu, gfn_t gfn, bool is_mmio); int tdx_vcpu_ioctl(struct kvm_vcpu *vcpu, void __user *argp); @@ -171,6 +173,7 @@ void tdx_post_memory_mapping(struct kvm_vcpu *vcpu, #else static inline int tdx_hardware_setup(struct kvm_x86_ops *x86_ops) { return -EOPNOTSUPP; } static inline void tdx_hardware_unsetup(void) {} +static inline void tdx_hardware_disable(void) {} static inline bool tdx_is_vm_type_supported(unsigned long type) { return false; } static inline int tdx_offline_cpu(void) { return 0; } @@ -190,6 +193,7 @@ static inline void tdx_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event) {} static inline fastpath_t tdx_vcpu_run(struct kvm_vcpu *vcpu) { return EXIT_FASTPATH_NONE; } static inline void tdx_prepare_switch_to_guest(struct kvm_vcpu *vcpu) {} static inline void tdx_vcpu_put(struct kvm_vcpu *vcpu) {} +static inline void tdx_vcpu_load(struct kvm_vcpu *vcpu, int cpu) {} static inline u8 tdx_get_mt_mask(struct kvm_vcpu *vcpu, gfn_t gfn, bool is_mmio) { return 0; } static inline int tdx_vcpu_ioctl(struct kvm_vcpu *vcpu, void __user *argp) { return -EOPNOTSUPP; } From patchwork Mon Feb 26 08:26:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571517 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5BE2F79DC5; Mon, 26 Feb 2024 08:28:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.19 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936131; cv=none; b=Z8xDLjFDHPoEw1J5kRN9bVeTEwCVXVb56nnhyD7T6MmNs+kksG9morNuppZMOV8iE6/UIWUJCUeGQFWF42C0DGq51Jr33pJDnXJq7vAZVTo0W0O3sh+A0J1wQqHEmhIto934eJY8aHmYRBJIjcSEqu38/iCPdDzCYFWYbRZvlyg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936131; c=relaxed/simple; bh=J0itU+rEBhLn1btxTLuPmm/pBpAavEDlMWDiVUOlSCE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=JibRA242OrgtHxf7sT3HalNi3tS40mo75KTKI8ck2LJi35/2I6VREDnbTPAMt2AZwMJwZuaqdYSEgMySUjhPKuW8cr1wX+ajqIQNbJjjYrvO/pKQqERDOSWLUXpDbf+U4HS8r3YPooGl7oeZrFbXvx+/giFRAW6RvzPP3JKIjK4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=NXSAlv6D; arc=none smtp.client-ip=198.175.65.19 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="NXSAlv6D" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936128; x=1740472128; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=J0itU+rEBhLn1btxTLuPmm/pBpAavEDlMWDiVUOlSCE=; b=NXSAlv6DARUWxW2f0ft9fb1YF+e+xn27cXeE3NTNSqC1xuLTQ4pUTvd5 nZkOfQeo23Ltk2uaKVq30MXxhPASkDYJjbZNoce/zArI4CJycnjJgPlDF p9mz1LYK4rqRNk68j4gDz9gQ/opsNMTc1xxvlO76EMsgSOAKnYOHXSy1Y 3AW0Ivx/4YtpGiwRbJ/3Wp1Q+pQAE8hQ+8Pt8EJa9cbLcYSC081Yay8Gt GR1aptc0TwMkdgGh89U868d45XuWdMA8nZjD8GuPiErwPZTo7ktaUaBXM eO+Rq+copS4v8/ux8a8BcZIgAd4+8hHlGuL9wOn+rht/6Rk65CHhCOde2 g==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="3069545" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="3069545" Received: from orviesa004.jf.intel.com ([10.64.159.144]) by orvoesa111.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:47 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="11272626" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:46 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com, Xiaoyao Li , Sean Christopherson , Chao Gao Subject: [PATCH v19 088/130] KVM: x86: Add a switch_db_regs flag to handle TDX's auto-switched behavior Date: Mon, 26 Feb 2024 00:26:30 -0800 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata Add a flag, KVM_DEBUGREG_AUTO_SWITCHED_GUEST, to skip saving/restoring DRs irrespective of any other flags. TDX-SEAM unconditionally saves and restores guest DRs and reset to architectural INIT state on TD exit. So, KVM needs to save host DRs before TD enter without restoring guest DRs and restore host DRs after TD exit. Opportunistically convert the KVM_DEBUGREG_* definitions to use BIT(). Reported-by: Xiaoyao Li Signed-off-by: Sean Christopherson Co-developed-by: Chao Gao Signed-off-by: Chao Gao Signed-off-by: Isaku Yamahata --- arch/x86/include/asm/kvm_host.h | 10 ++++++++-- arch/x86/kvm/vmx/tdx.c | 1 + arch/x86/kvm/x86.c | 11 ++++++++--- 3 files changed, 17 insertions(+), 5 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 3ab85c3d86ee..a9df898c6fbd 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -610,8 +610,14 @@ struct kvm_pmu { struct kvm_pmu_ops; enum { - KVM_DEBUGREG_BP_ENABLED = 1, - KVM_DEBUGREG_WONT_EXIT = 2, + KVM_DEBUGREG_BP_ENABLED = BIT(0), + KVM_DEBUGREG_WONT_EXIT = BIT(1), + /* + * Guest debug registers (DR0-3 and DR6) are saved/restored by hardware + * on exit from or enter to guest. KVM needn't switch them. Because DR7 + * is cleared on exit from guest, DR7 need to be saved/restored. + */ + KVM_DEBUGREG_AUTO_SWITCH = BIT(2), }; struct kvm_mtrr_range { diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c index 7aa9188f384d..ab7403a19c5d 100644 --- a/arch/x86/kvm/vmx/tdx.c +++ b/arch/x86/kvm/vmx/tdx.c @@ -586,6 +586,7 @@ int tdx_vcpu_create(struct kvm_vcpu *vcpu) vcpu->arch.efer = EFER_SCE | EFER_LME | EFER_LMA | EFER_NX; + vcpu->arch.switch_db_regs = KVM_DEBUGREG_AUTO_SWITCH; vcpu->arch.cr0_guest_owned_bits = -1ul; vcpu->arch.cr4_guest_owned_bits = -1ul; diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 1b189e86a1f1..fb7597c22f31 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -11013,7 +11013,7 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu) if (vcpu->arch.guest_fpu.xfd_err) wrmsrl(MSR_IA32_XFD_ERR, vcpu->arch.guest_fpu.xfd_err); - if (unlikely(vcpu->arch.switch_db_regs)) { + if (unlikely(vcpu->arch.switch_db_regs & ~KVM_DEBUGREG_AUTO_SWITCH)) { set_debugreg(0, 7); set_debugreg(vcpu->arch.eff_db[0], 0); set_debugreg(vcpu->arch.eff_db[1], 1); @@ -11059,6 +11059,7 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu) */ if (unlikely(vcpu->arch.switch_db_regs & KVM_DEBUGREG_WONT_EXIT)) { WARN_ON(vcpu->guest_debug & KVM_GUESTDBG_USE_HW_BP); + WARN_ON(vcpu->arch.switch_db_regs & KVM_DEBUGREG_AUTO_SWITCH); static_call(kvm_x86_sync_dirty_debug_regs)(vcpu); kvm_update_dr0123(vcpu); kvm_update_dr7(vcpu); @@ -11071,8 +11072,12 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu) * care about the messed up debug address registers. But if * we have some of them active, restore the old state. */ - if (hw_breakpoint_active()) - hw_breakpoint_restore(); + if (hw_breakpoint_active()) { + if (!(vcpu->arch.switch_db_regs & KVM_DEBUGREG_AUTO_SWITCH)) + hw_breakpoint_restore(); + else + set_debugreg(__this_cpu_read(cpu_dr7), 7); + } vcpu->arch.last_vmentry_cpu = vcpu->cpu; vcpu->arch.last_guest_tsc = kvm_read_l1_tsc(vcpu, rdtsc()); From patchwork Mon Feb 26 08:26:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571516 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 98C5479DDD; Mon, 26 Feb 2024 08:28:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.19 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936130; cv=none; b=H61zGj4csE9p6awGnU6cSoZZKEn70TD5ifx/jO2Ti92LU9+K0ozSEXQjI5kB0TgjMmqNPmROfEvQrkqwOVyv/CtHT/ubdFIIQyYex5xP+d3kQm3gf4hMaLANsG8TeSwZrKs55Puy0tToJdikB+ARL8sShn0ENsDDdV4b8qMuwxI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936130; c=relaxed/simple; bh=BHP76H37q8PEr+1VCgScyEXlairXZGF6vGYRWXEvFiQ=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=qhp0FLft5igcFBQ7MfgDy2HsH4M2QkMW5chFxVb/D9yIjPl4IjSh77G2bdwqVa5NvFaxcQgIe3cvCtSYZXIljp+kPqWMAdD6YeFAdy4hjoKe0HkUyugnam5tFC9yYhJo7KBMX/cdP4ptq2kPpOSutnAQJHgBrU/61OrTC10iWrM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=mH/eZuAD; arc=none smtp.client-ip=198.175.65.19 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="mH/eZuAD" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936129; x=1740472129; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=BHP76H37q8PEr+1VCgScyEXlairXZGF6vGYRWXEvFiQ=; b=mH/eZuADpYbw3lo/AM5yvbGcE1943OqwrdHGMaJo1Z9bTDxL5w+er8qI SFmCgWlWhJZO9OqBEs3ke7/ZYFn6p6v5sir1Og/GM1jp5iwHFs/7Kro8q V4J0FHIPD+uC8YsfmwUEZManF/GQV359YT9xTfMf13kO3OY73pFuIjTbl XbssH54uktowGLZ/2oZm9bAShfcKCdEuod6I2DH+rzAWbRACH0mjbX0XE kwyVydDD6xWKhY5XDpCeSHsQG5BHWWT7f1sq5TRElztMm4mFrkLjQGk8p R+em5OxmBPxPP/0FZ5gZC0rM95UIJBsTyryT8qQKimgqt9jujO8ESNO5v g==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="3069549" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="3069549" Received: from orviesa004.jf.intel.com ([10.64.159.144]) by orvoesa111.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:48 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="11272633" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:47 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 089/130] KVM: TDX: Add support for find pending IRQ in a protected local APIC Date: Mon, 26 Feb 2024 00:26:31 -0800 Message-Id: <8ea6ffce742842da725a7e3f891a38583bdb099b.1708933498.git.isaku.yamahata@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Sean Christopherson Add flag and hook to KVM's local APIC management to support determining whether or not a TDX guest as a pending IRQ. For TDX vCPUs, the virtual APIC page is owned by the TDX module and cannot be accessed by KVM. As a result, registers that are virtualized by the CPU, e.g. PPR, cannot be read or written by KVM. To deliver interrupts for TDX guests, KVM must send an IRQ to the CPU on the posted interrupt notification vector. And to determine if TDX vCPU has a pending interrupt, KVM must check if there is an outstanding notification. Return "no interrupt" in kvm_apic_has_interrupt() if the guest APIC is protected to short-circuit the various other flows that try to pull an IRQ out of the vAPIC, the only valid operation is querying _if_ an IRQ is pending, KVM can't do anything based on _which_ IRQ is pending. Intentionally omit sanity checks from other flows, e.g. PPR update, so as not to degrade non-TDX guests with unnecessary checks. A well-behaved KVM and userspace will never reach those flows for TDX guests, but reaching them is not fatal if something does go awry. Note, this doesn't handle interrupts that have been delivered to the vCPU but not yet recognized by the core, i.e. interrupts that are sitting in vmcs.GUEST_INTR_STATUS. Querying that state requires a SEAMCALL and will be supported in a future patch. Signed-off-by: Sean Christopherson Signed-off-by: Isaku Yamahata --- arch/x86/include/asm/kvm-x86-ops.h | 1 + arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/irq.c | 3 +++ arch/x86/kvm/lapic.c | 3 +++ arch/x86/kvm/lapic.h | 2 ++ arch/x86/kvm/vmx/main.c | 10 ++++++++++ arch/x86/kvm/vmx/tdx.c | 6 ++++++ arch/x86/kvm/vmx/x86_ops.h | 2 ++ 8 files changed, 28 insertions(+) diff --git a/arch/x86/include/asm/kvm-x86-ops.h b/arch/x86/include/asm/kvm-x86-ops.h index fb3ae97c724e..22d93d4124c8 100644 --- a/arch/x86/include/asm/kvm-x86-ops.h +++ b/arch/x86/include/asm/kvm-x86-ops.h @@ -124,6 +124,7 @@ KVM_X86_OP_OPTIONAL(pi_start_assignment) KVM_X86_OP_OPTIONAL(apicv_pre_state_restore) KVM_X86_OP_OPTIONAL(apicv_post_state_restore) KVM_X86_OP_OPTIONAL_RET0(dy_apicv_has_pending_interrupt) +KVM_X86_OP_OPTIONAL(protected_apic_has_interrupt) KVM_X86_OP_OPTIONAL(set_hv_timer) KVM_X86_OP_OPTIONAL(cancel_hv_timer) KVM_X86_OP(setup_mce) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index a9df898c6fbd..e0ffef1d377d 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1800,6 +1800,7 @@ struct kvm_x86_ops { void (*apicv_pre_state_restore)(struct kvm_vcpu *vcpu); void (*apicv_post_state_restore)(struct kvm_vcpu *vcpu); bool (*dy_apicv_has_pending_interrupt)(struct kvm_vcpu *vcpu); + bool (*protected_apic_has_interrupt)(struct kvm_vcpu *vcpu); int (*set_hv_timer)(struct kvm_vcpu *vcpu, u64 guest_deadline_tsc, bool *expired); diff --git a/arch/x86/kvm/irq.c b/arch/x86/kvm/irq.c index ad9ca8a60144..f253f4c6bf04 100644 --- a/arch/x86/kvm/irq.c +++ b/arch/x86/kvm/irq.c @@ -100,6 +100,9 @@ int kvm_cpu_has_interrupt(struct kvm_vcpu *v) if (kvm_cpu_has_extint(v)) return 1; + if (lapic_in_kernel(v) && v->arch.apic->guest_apic_protected) + return static_call(kvm_x86_protected_apic_has_interrupt)(v); + return kvm_apic_has_interrupt(v) != -1; /* LAPIC */ } EXPORT_SYMBOL_GPL(kvm_cpu_has_interrupt); diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c index 3242f3da2457..e8034f2f2dd1 100644 --- a/arch/x86/kvm/lapic.c +++ b/arch/x86/kvm/lapic.c @@ -2860,6 +2860,9 @@ int kvm_apic_has_interrupt(struct kvm_vcpu *vcpu) if (!kvm_apic_present(vcpu)) return -1; + if (apic->guest_apic_protected) + return -1; + __apic_update_ppr(apic, &ppr); return apic_has_interrupt_for_ppr(apic, ppr); } diff --git a/arch/x86/kvm/lapic.h b/arch/x86/kvm/lapic.h index 0a0ea4b5dd8c..749b7b629c47 100644 --- a/arch/x86/kvm/lapic.h +++ b/arch/x86/kvm/lapic.h @@ -66,6 +66,8 @@ struct kvm_lapic { bool sw_enabled; bool irr_pending; bool lvt0_in_nmi_mode; + /* Select registers in the vAPIC cannot be read/written. */ + bool guest_apic_protected; /* Number of bits set in ISR. */ s16 isr_count; /* The highest vector set in ISR; if -1 - invalid, must scan ISR. */ diff --git a/arch/x86/kvm/vmx/main.c b/arch/x86/kvm/vmx/main.c index 9b336c1a6508..5fd99e844b86 100644 --- a/arch/x86/kvm/vmx/main.c +++ b/arch/x86/kvm/vmx/main.c @@ -78,6 +78,8 @@ static __init int vt_hardware_setup(void) if (enable_tdx) vt_x86_ops.vm_size = max_t(unsigned int, vt_x86_ops.vm_size, sizeof(struct kvm_tdx)); + else + vt_x86_ops.protected_apic_has_interrupt = NULL; #if IS_ENABLED(CONFIG_HYPERV) if (enable_tdx) @@ -219,6 +221,13 @@ static void vt_vcpu_load(struct kvm_vcpu *vcpu, int cpu) vmx_vcpu_load(vcpu, cpu); } +static bool vt_protected_apic_has_interrupt(struct kvm_vcpu *vcpu) +{ + KVM_BUG_ON(!is_td_vcpu(vcpu), vcpu->kvm); + + return tdx_protected_apic_has_interrupt(vcpu); +} + static void vt_flush_tlb_all(struct kvm_vcpu *vcpu) { if (is_td_vcpu(vcpu)) { @@ -443,6 +452,7 @@ struct kvm_x86_ops vt_x86_ops __initdata = { .sync_pir_to_irr = vmx_sync_pir_to_irr, .deliver_interrupt = vmx_deliver_interrupt, .dy_apicv_has_pending_interrupt = pi_has_pending_interrupt, + .protected_apic_has_interrupt = vt_protected_apic_has_interrupt, .set_tss_addr = vmx_set_tss_addr, .set_identity_map_addr = vmx_set_identity_map_addr, diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c index ab7403a19c5d..a5b52aa6d153 100644 --- a/arch/x86/kvm/vmx/tdx.c +++ b/arch/x86/kvm/vmx/tdx.c @@ -583,6 +583,7 @@ int tdx_vcpu_create(struct kvm_vcpu *vcpu) return -EINVAL; fpstate_set_confidential(&vcpu->arch.guest_fpu); + vcpu->arch.apic->guest_apic_protected = true; vcpu->arch.efer = EFER_SCE | EFER_LME | EFER_LMA | EFER_NX; @@ -624,6 +625,11 @@ void tdx_vcpu_load(struct kvm_vcpu *vcpu, int cpu) local_irq_enable(); } +bool tdx_protected_apic_has_interrupt(struct kvm_vcpu *vcpu) +{ + return pi_has_pending_interrupt(vcpu); +} + void tdx_prepare_switch_to_guest(struct kvm_vcpu *vcpu) { struct vcpu_tdx *tdx = to_tdx(vcpu); diff --git a/arch/x86/kvm/vmx/x86_ops.h b/arch/x86/kvm/vmx/x86_ops.h index 5853f29f0af3..f3f90186926c 100644 --- a/arch/x86/kvm/vmx/x86_ops.h +++ b/arch/x86/kvm/vmx/x86_ops.h @@ -155,6 +155,7 @@ fastpath_t tdx_vcpu_run(struct kvm_vcpu *vcpu); void tdx_prepare_switch_to_guest(struct kvm_vcpu *vcpu); void tdx_vcpu_put(struct kvm_vcpu *vcpu); void tdx_vcpu_load(struct kvm_vcpu *vcpu, int cpu); +bool tdx_protected_apic_has_interrupt(struct kvm_vcpu *vcpu); u8 tdx_get_mt_mask(struct kvm_vcpu *vcpu, gfn_t gfn, bool is_mmio); int tdx_vcpu_ioctl(struct kvm_vcpu *vcpu, void __user *argp); @@ -194,6 +195,7 @@ static inline fastpath_t tdx_vcpu_run(struct kvm_vcpu *vcpu) { return EXIT_FASTP static inline void tdx_prepare_switch_to_guest(struct kvm_vcpu *vcpu) {} static inline void tdx_vcpu_put(struct kvm_vcpu *vcpu) {} static inline void tdx_vcpu_load(struct kvm_vcpu *vcpu, int cpu) {} +static inline bool tdx_protected_apic_has_interrupt(struct kvm_vcpu *vcpu) { return false; } static inline u8 tdx_get_mt_mask(struct kvm_vcpu *vcpu, gfn_t gfn, bool is_mmio) { return 0; } static inline int tdx_vcpu_ioctl(struct kvm_vcpu *vcpu, void __user *argp) { return -EOPNOTSUPP; } From patchwork Mon Feb 26 08:26:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571518 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 019CE7A72C; Mon, 26 Feb 2024 08:28:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.19 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936131; cv=none; b=jZs5YVjrLPzsyqL3mJZNp9BrSuhsoI4bjEbjPsPqFpvE+QxhYZoOTGAEMAe7LIDI09p85l/mFB+2NCWGuHAbeKhFUgKoTWhOlXNZHFn4wuOvjJG+a+MMwE5/7rEL4CycOLjIuX7jTYc3pAYVY/lfuRbl1xRmKOlDFRGXtgvImUY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936131; c=relaxed/simple; bh=/sAXojEGYW4Sd48ndXBIWs56QmQk33LvAtJbkrrQS6g=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=bwe57A30LHvEt2mIK/A0ynflSQi8mX/a2sN9GCUD4l/eClN25tcWP7v0vezIBlILhvMDJxWfb9hK4NU0yzUiSqDkggOE7Do5G2Bd68/Ovun6DpwN2cGKyzaIScplhWYneius+/tvTWk57L+6lLQ/EPPEGD9gDhn/k2EkdjXbtCw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=OvFjUD+U; arc=none smtp.client-ip=198.175.65.19 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="OvFjUD+U" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936130; x=1740472130; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=/sAXojEGYW4Sd48ndXBIWs56QmQk33LvAtJbkrrQS6g=; b=OvFjUD+Ugm0OSh9HGuQA9xtoDIMeO1/BOx2s9SRj520IZaAzojl7+Syh AD2shgY0seNVkyCNMQqjwtlNFFYmM+ZdrA6tTYWsGiJkO3SVoDbxX40W3 lxJXoc8fgPeft5LsgUBd/Bfw2ooKx9t2qqdzb+Wcb+rS7D5tM0X/4xE3F iM8RVfVeEeJjD8AvVUF8kferLvVBZYf781qQtbvNGAZ9sEqC4BEhyPp4W OBUnlgeaTegLhAuhSKu5MRPxRUe17pb6DyNyS4wuJokqzfwAuFArk67nw 5oqxZXzmTuDdK45PKRr1Vm6K4ilG6iYWY1Y8SWa9GHVJpfLIAttr9Pyys Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="3069551" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="3069551" Received: from orviesa004.jf.intel.com ([10.64.159.144]) by orvoesa111.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:49 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="11272637" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:48 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 090/130] KVM: x86: Assume timer IRQ was injected if APIC state is proteced Date: Mon, 26 Feb 2024 00:26:32 -0800 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Sean Christopherson If APIC state is protected, i.e. the vCPU is a TDX guest, assume a timer IRQ was injected when deciding whether or not to busy wait in the "timer advanced" path. The "real" vIRR is not readable/writable, so trying to query for a pending timer IRQ will return garbage. Note, TDX can scour the PIR if it wants to be more precise and skip the "wait" call entirely. Signed-off-by: Sean Christopherson Signed-off-by: Isaku Yamahata --- arch/x86/kvm/lapic.c | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c index e8034f2f2dd1..8025c7f614e0 100644 --- a/arch/x86/kvm/lapic.c +++ b/arch/x86/kvm/lapic.c @@ -1774,8 +1774,17 @@ static void apic_update_lvtt(struct kvm_lapic *apic) static bool lapic_timer_int_injected(struct kvm_vcpu *vcpu) { struct kvm_lapic *apic = vcpu->arch.apic; - u32 reg = kvm_lapic_get_reg(apic, APIC_LVTT); + u32 reg; + /* + * Assume a timer IRQ was "injected" if the APIC is protected. KVM's + * copy of the vIRR is bogus, it's the responsibility of the caller to + * precisely check whether or not a timer IRQ is pending. + */ + if (apic->guest_apic_protected) + return true; + + reg = kvm_lapic_get_reg(apic, APIC_LVTT); if (kvm_apic_hw_enabled(apic)) { int vec = reg & APIC_VECTOR_MASK; void *bitmap = apic->regs + APIC_ISR; From patchwork Mon Feb 26 08:26:33 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571519 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AAAA17A738; Mon, 26 Feb 2024 08:28:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.19 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936132; cv=none; b=TXyxODZQzkSEGHuR43FlcsM5emtEJ3UlrqMdRCI67QYfc7THldnKIdDPhEzfq28IKfSMf2Es2JBYqigj1qAGhQu0MMebF+9+McgahKFUibo26aNl3RQvziGgvrPFx2gGhAp9QZZd7JppO00Ebj5pAhpDGk93XtDqA+xQwKTTS9Y= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936132; c=relaxed/simple; bh=fE0TWc2+hDBVZkABNfd5lMn7VSSV0PbBPyLb8gy3rxs=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=AegHskvMlZe3eQaINrK45us7goM8IptJVvtZiC8/71ogkjvYOTU0ML5eop/T8d2Cv2WBS1H++J2gjRdKVxPEp4eNFo2o6zAW1dyR0Eq+9m+Kyt3IXPG4+3OjcMEo2dzQTa0gB/xVj4rGZGdvkqWYhu7i7hXaUcsAAxBOtfTKfxI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=M3itnoEj; arc=none smtp.client-ip=198.175.65.19 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="M3itnoEj" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936131; x=1740472131; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=fE0TWc2+hDBVZkABNfd5lMn7VSSV0PbBPyLb8gy3rxs=; b=M3itnoEjYQVIhFOezWTgUle7SENYJgeilh/q5sF4mSwcQLy0dh7rirwl tmmf5AJa3BXIvgJrC6ZYhDpcoukWMe3T+QwaxX0MBFgFsDPwXVlJ8QW0p ZmgMTQc/d5dXthBEUgHrt3jIjasN5vT/jQws8O4jYp1GM47ZcYsfpltSc mE110L6353esgRpTFpyZY2YrD9lyuhvLtomYKupi5jV9Y+203fVFdDkGj w7zguPtIjBoIh4D0mXh4JJspNeMwqCjl8nQnhcFQVEtqEiFwsmdOwd09W steyWtPdxJFCgNZzznYd7YUsz1Fb1f271sKvsZRhVIwRLYzSkW1RTXtF0 A==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="3069558" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="3069558" Received: from orviesa004.jf.intel.com ([10.64.159.144]) by orvoesa111.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:50 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="11272642" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:49 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 091/130] KVM: TDX: remove use of struct vcpu_vmx from posted_interrupt.c Date: Mon, 26 Feb 2024 00:26:33 -0800 Message-Id: <6c7774a44515d6787c9512cb05c3b305e9b5855c.1708933498.git.isaku.yamahata@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata As TDX will use posted_interrupt.c, the use of struct vcpu_vmx is a blocker. Because the members of struct pi_desc pi_desc and struct list_head pi_wakeup_list are only used in posted_interrupt.c, introduce common structure, struct vcpu_pi, make vcpu_vmx and vcpu_tdx has same layout in the top of structure. To minimize the diff size, avoid code conversion like, vmx->pi_desc => vmx->common->pi_desc. Instead add compile time check if the layout is expected. Signed-off-by: Isaku Yamahata --- arch/x86/kvm/vmx/posted_intr.c | 41 ++++++++++++++++++++++++++-------- arch/x86/kvm/vmx/posted_intr.h | 11 +++++++++ arch/x86/kvm/vmx/tdx.c | 1 + arch/x86/kvm/vmx/tdx.h | 8 +++++++ arch/x86/kvm/vmx/vmx.h | 14 +++++++----- 5 files changed, 60 insertions(+), 15 deletions(-) diff --git a/arch/x86/kvm/vmx/posted_intr.c b/arch/x86/kvm/vmx/posted_intr.c index af662312fd07..b66add9da0f3 100644 --- a/arch/x86/kvm/vmx/posted_intr.c +++ b/arch/x86/kvm/vmx/posted_intr.c @@ -11,6 +11,7 @@ #include "posted_intr.h" #include "trace.h" #include "vmx.h" +#include "tdx.h" /* * Maintain a per-CPU list of vCPUs that need to be awakened by wakeup_handler() @@ -31,9 +32,29 @@ static DEFINE_PER_CPU(struct list_head, wakeup_vcpus_on_cpu); */ static DEFINE_PER_CPU(raw_spinlock_t, wakeup_vcpus_on_cpu_lock); +/* + * The layout of the head of struct vcpu_vmx and struct vcpu_tdx must match with + * struct vcpu_pi. + */ +static_assert(offsetof(struct vcpu_pi, pi_desc) == + offsetof(struct vcpu_vmx, pi_desc)); +static_assert(offsetof(struct vcpu_pi, pi_wakeup_list) == + offsetof(struct vcpu_vmx, pi_wakeup_list)); +#ifdef CONFIG_INTEL_TDX_HOST +static_assert(offsetof(struct vcpu_pi, pi_desc) == + offsetof(struct vcpu_tdx, pi_desc)); +static_assert(offsetof(struct vcpu_pi, pi_wakeup_list) == + offsetof(struct vcpu_tdx, pi_wakeup_list)); +#endif + +static inline struct vcpu_pi *vcpu_to_pi(struct kvm_vcpu *vcpu) +{ + return (struct vcpu_pi *)vcpu; +} + static inline struct pi_desc *vcpu_to_pi_desc(struct kvm_vcpu *vcpu) { - return &(to_vmx(vcpu)->pi_desc); + return &vcpu_to_pi(vcpu)->pi_desc; } static int pi_try_set_control(struct pi_desc *pi_desc, u64 *pold, u64 new) @@ -52,8 +73,8 @@ static int pi_try_set_control(struct pi_desc *pi_desc, u64 *pold, u64 new) void vmx_vcpu_pi_load(struct kvm_vcpu *vcpu, int cpu) { - struct pi_desc *pi_desc = vcpu_to_pi_desc(vcpu); - struct vcpu_vmx *vmx = to_vmx(vcpu); + struct vcpu_pi *vcpu_pi = vcpu_to_pi(vcpu); + struct pi_desc *pi_desc = &vcpu_pi->pi_desc; struct pi_desc old, new; unsigned long flags; unsigned int dest; @@ -90,7 +111,7 @@ void vmx_vcpu_pi_load(struct kvm_vcpu *vcpu, int cpu) */ if (pi_desc->nv == POSTED_INTR_WAKEUP_VECTOR) { raw_spin_lock(&per_cpu(wakeup_vcpus_on_cpu_lock, vcpu->cpu)); - list_del(&vmx->pi_wakeup_list); + list_del(&vcpu_pi->pi_wakeup_list); raw_spin_unlock(&per_cpu(wakeup_vcpus_on_cpu_lock, vcpu->cpu)); } @@ -145,15 +166,15 @@ static bool vmx_can_use_vtd_pi(struct kvm *kvm) */ static void pi_enable_wakeup_handler(struct kvm_vcpu *vcpu) { - struct pi_desc *pi_desc = vcpu_to_pi_desc(vcpu); - struct vcpu_vmx *vmx = to_vmx(vcpu); + struct vcpu_pi *vcpu_pi = vcpu_to_pi(vcpu); + struct pi_desc *pi_desc = &vcpu_pi->pi_desc; struct pi_desc old, new; unsigned long flags; local_irq_save(flags); raw_spin_lock(&per_cpu(wakeup_vcpus_on_cpu_lock, vcpu->cpu)); - list_add_tail(&vmx->pi_wakeup_list, + list_add_tail(&vcpu_pi->pi_wakeup_list, &per_cpu(wakeup_vcpus_on_cpu, vcpu->cpu)); raw_spin_unlock(&per_cpu(wakeup_vcpus_on_cpu_lock, vcpu->cpu)); @@ -190,7 +211,8 @@ static bool vmx_needs_pi_wakeup(struct kvm_vcpu *vcpu) * notification vector is switched to the one that calls * back to the pi_wakeup_handler() function. */ - return vmx_can_use_ipiv(vcpu) || vmx_can_use_vtd_pi(vcpu->kvm); + return (vmx_can_use_ipiv(vcpu) && !is_td_vcpu(vcpu)) || + vmx_can_use_vtd_pi(vcpu->kvm); } void vmx_vcpu_pi_put(struct kvm_vcpu *vcpu) @@ -200,7 +222,8 @@ void vmx_vcpu_pi_put(struct kvm_vcpu *vcpu) if (!vmx_needs_pi_wakeup(vcpu)) return; - if (kvm_vcpu_is_blocking(vcpu) && !vmx_interrupt_blocked(vcpu)) + if (kvm_vcpu_is_blocking(vcpu) && + (is_td_vcpu(vcpu) || !vmx_interrupt_blocked(vcpu))) pi_enable_wakeup_handler(vcpu); /* diff --git a/arch/x86/kvm/vmx/posted_intr.h b/arch/x86/kvm/vmx/posted_intr.h index 26992076552e..2fe8222308b2 100644 --- a/arch/x86/kvm/vmx/posted_intr.h +++ b/arch/x86/kvm/vmx/posted_intr.h @@ -94,6 +94,17 @@ static inline bool pi_test_sn(struct pi_desc *pi_desc) (unsigned long *)&pi_desc->control); } +struct vcpu_pi { + struct kvm_vcpu vcpu; + + /* Posted interrupt descriptor */ + struct pi_desc pi_desc; + + /* Used if this vCPU is waiting for PI notification wakeup. */ + struct list_head pi_wakeup_list; + /* Until here common layout betwwn vcpu_vmx and vcpu_tdx. */ +}; + void vmx_vcpu_pi_load(struct kvm_vcpu *vcpu, int cpu); void vmx_vcpu_pi_put(struct kvm_vcpu *vcpu); void pi_wakeup_handler(void); diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c index a5b52aa6d153..1da58c36217c 100644 --- a/arch/x86/kvm/vmx/tdx.c +++ b/arch/x86/kvm/vmx/tdx.c @@ -584,6 +584,7 @@ int tdx_vcpu_create(struct kvm_vcpu *vcpu) fpstate_set_confidential(&vcpu->arch.guest_fpu); vcpu->arch.apic->guest_apic_protected = true; + INIT_LIST_HEAD(&tdx->pi_wakeup_list); vcpu->arch.efer = EFER_SCE | EFER_LME | EFER_LMA | EFER_NX; diff --git a/arch/x86/kvm/vmx/tdx.h b/arch/x86/kvm/vmx/tdx.h index 7f8c78f06508..eaffa7384725 100644 --- a/arch/x86/kvm/vmx/tdx.h +++ b/arch/x86/kvm/vmx/tdx.h @@ -4,6 +4,7 @@ #ifdef CONFIG_INTEL_TDX_HOST +#include "posted_intr.h" #include "pmu_intel.h" #include "tdx_ops.h" @@ -69,6 +70,13 @@ union tdx_exit_reason { struct vcpu_tdx { struct kvm_vcpu vcpu; + /* Posted interrupt descriptor */ + struct pi_desc pi_desc; + + /* Used if this vCPU is waiting for PI notification wakeup. */ + struct list_head pi_wakeup_list; + /* Until here same layout to struct vcpu_pi. */ + unsigned long tdvpr_pa; unsigned long *tdvpx_pa; bool td_vcpu_created; diff --git a/arch/x86/kvm/vmx/vmx.h b/arch/x86/kvm/vmx/vmx.h index 79ff54f08fee..634a9a250b95 100644 --- a/arch/x86/kvm/vmx/vmx.h +++ b/arch/x86/kvm/vmx/vmx.h @@ -235,6 +235,14 @@ struct nested_vmx { struct vcpu_vmx { struct kvm_vcpu vcpu; + + /* Posted interrupt descriptor */ + struct pi_desc pi_desc; + + /* Used if this vCPU is waiting for PI notification wakeup. */ + struct list_head pi_wakeup_list; + /* Until here same layout to struct vcpu_pi. */ + u8 fail; u8 x2apic_msr_bitmap_mode; @@ -304,12 +312,6 @@ struct vcpu_vmx { union vmx_exit_reason exit_reason; - /* Posted interrupt descriptor */ - struct pi_desc pi_desc; - - /* Used if this vCPU is waiting for PI notification wakeup. */ - struct list_head pi_wakeup_list; - /* Support for a guest hypervisor (nested VMX) */ struct nested_vmx nested; From patchwork Mon Feb 26 08:26:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571521 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 78AD57AE4B; Mon, 26 Feb 2024 08:28:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.19 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936133; cv=none; b=dvzLU4Q/Nwt5IAduafUMd6BZ808gchZBN/57W49a/himxPw8EXWULqJV8Ov69LgTUfByh5qocBuwgwuuT+cd5kZKoh9/5S93cwcjji7mOIxJH77caf/M1iMR6gfIS0QUhEqD17CilkmkxLdnvvQssXs7/dpN2i13eyH551abILw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936133; c=relaxed/simple; bh=PyikNyG7fKJSTvY484nYih3zF/u6N+ZdYnoP25Ocepo=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=OERPMMD6Oj8xikjs9vPyM+mMUm4m7xYZ8w0uIPN8B+SphxbIFjGCllIRC6P3Vy3wNzKaOZe3aaymmmV+EzsC/IhSBXC/jrM4wVCLPBxTu/YEonqCV47629olxs5/0zVvh3CmbcFmDlV+DFWQ88Vsz6m+dZLWz8d1fBqssstB5/g= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=fail smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=afHs07nx; arc=none smtp.client-ip=198.175.65.19 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="afHs07nx" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936132; x=1740472132; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=PyikNyG7fKJSTvY484nYih3zF/u6N+ZdYnoP25Ocepo=; b=afHs07nxATRYSzSJD41ixUbnBTr5Q4PGgIxjZUnLqgMmQzVP2Bta426o 6aI4ivjgBVJ2sm/WvmX8YGda6qJFfKlNL0TqhGUgX2xwTOMlsmrEzvmKC QTDhTzgCQ9AalPxP6MClGJ+TIzmKSktNtGUpTO34oeLT9M2LYzngRmIdB pUQx7qO9YlN39z0+lxSiGvo4ZycJVRF9ye4Y2pJzhJO2S/vgjuDHMyTIW svFxxfyzrEcYdzU/cBcQ/VZZ5ucHDjHuHuCyg0j2amJEi16+DdCYB8fHu wgRa73i7D5rKiFWFTneG/0pEqVzOprDycLe1xl7j0/Z1z1h++2PkDAmHN A==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="3069564" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="3069564" Received: from orviesa004.jf.intel.com ([10.64.159.144]) by orvoesa111.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:51 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="11272647" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:50 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 092/130] KVM: TDX: Implement interrupt injection Date: Mon, 26 Feb 2024 00:26:34 -0800 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata TDX supports interrupt inject into vcpu with posted interrupt. Wire up the corresponding kvm x86 operations to posted interrupt. Move kvm_vcpu_trigger_posted_interrupt() from vmx.c to common.h to share the code. VMX can inject interrupt by setting interrupt information field, VM_ENTRY_INTR_INFO_FIELD, of VMCS. TDX supports interrupt injection only by posted interrupt. Ignore the execution path to access VM_ENTRY_INTR_INFO_FIELD. As cpu state is protected and apicv is enabled for the TDX guest, VMM can inject interrupt by updating posted interrupt descriptor. Treat interrupt can be injected always. Signed-off-by: Isaku Yamahata Reviewed-by: Paolo Bonzini --- arch/x86/kvm/vmx/common.h | 71 ++++++++++++++++++++++++++ arch/x86/kvm/vmx/main.c | 93 ++++++++++++++++++++++++++++++---- arch/x86/kvm/vmx/posted_intr.c | 2 +- arch/x86/kvm/vmx/posted_intr.h | 2 + arch/x86/kvm/vmx/tdx.c | 25 +++++++++ arch/x86/kvm/vmx/vmx.c | 67 +----------------------- arch/x86/kvm/vmx/x86_ops.h | 7 ++- 7 files changed, 190 insertions(+), 77 deletions(-) diff --git a/arch/x86/kvm/vmx/common.h b/arch/x86/kvm/vmx/common.h index 235908f3e044..6f21d0d48809 100644 --- a/arch/x86/kvm/vmx/common.h +++ b/arch/x86/kvm/vmx/common.h @@ -4,6 +4,7 @@ #include +#include "posted_intr.h" #include "mmu.h" static inline int __vmx_handle_ept_violation(struct kvm_vcpu *vcpu, gpa_t gpa, @@ -30,4 +31,74 @@ static inline int __vmx_handle_ept_violation(struct kvm_vcpu *vcpu, gpa_t gpa, return kvm_mmu_page_fault(vcpu, gpa, error_code, NULL, 0); } +static inline void kvm_vcpu_trigger_posted_interrupt(struct kvm_vcpu *vcpu, + int pi_vec) +{ +#ifdef CONFIG_SMP + if (vcpu->mode == IN_GUEST_MODE) { + /* + * The vector of the virtual has already been set in the PIR. + * Send a notification event to deliver the virtual interrupt + * unless the vCPU is the currently running vCPU, i.e. the + * event is being sent from a fastpath VM-Exit handler, in + * which case the PIR will be synced to the vIRR before + * re-entering the guest. + * + * When the target is not the running vCPU, the following + * possibilities emerge: + * + * Case 1: vCPU stays in non-root mode. Sending a notification + * event posts the interrupt to the vCPU. + * + * Case 2: vCPU exits to root mode and is still runnable. The + * PIR will be synced to the vIRR before re-entering the guest. + * Sending a notification event is ok as the host IRQ handler + * will ignore the spurious event. + * + * Case 3: vCPU exits to root mode and is blocked. vcpu_block() + * has already synced PIR to vIRR and never blocks the vCPU if + * the vIRR is not empty. Therefore, a blocked vCPU here does + * not wait for any requested interrupts in PIR, and sending a + * notification event also results in a benign, spurious event. + */ + + if (vcpu != kvm_get_running_vcpu()) + __apic_send_IPI_mask(get_cpu_mask(vcpu->cpu), pi_vec); + return; + } +#endif + /* + * The vCPU isn't in the guest; wake the vCPU in case it is blocking, + * otherwise do nothing as KVM will grab the highest priority pending + * IRQ via ->sync_pir_to_irr() in vcpu_enter_guest(). + */ + kvm_vcpu_wake_up(vcpu); +} + +/* + * Send interrupt to vcpu via posted interrupt way. + * 1. If target vcpu is running(non-root mode), send posted interrupt + * notification to vcpu and hardware will sync PIR to vIRR atomically. + * 2. If target vcpu isn't running(root mode), kick it to pick up the + * interrupt from PIR in next vmentry. + */ +static inline void __vmx_deliver_posted_interrupt(struct kvm_vcpu *vcpu, + struct pi_desc *pi_desc, int vector) +{ + if (pi_test_and_set_pir(vector, pi_desc)) + return; + + /* If a previous notification has sent the IPI, nothing to do. */ + if (pi_test_and_set_on(pi_desc)) + return; + + /* + * The implied barrier in pi_test_and_set_on() pairs with the smp_mb_*() + * after setting vcpu->mode in vcpu_enter_guest(), thus the vCPU is + * guaranteed to see PID.ON=1 and sync the PIR to IRR if triggering a + * posted interrupt "fails" because vcpu->mode != IN_GUEST_MODE. + */ + kvm_vcpu_trigger_posted_interrupt(vcpu, POSTED_INTR_VECTOR); +} + #endif /* __KVM_X86_VMX_COMMON_H */ diff --git a/arch/x86/kvm/vmx/main.c b/arch/x86/kvm/vmx/main.c index 5fd99e844b86..f2c9d6358f9e 100644 --- a/arch/x86/kvm/vmx/main.c +++ b/arch/x86/kvm/vmx/main.c @@ -228,6 +228,34 @@ static bool vt_protected_apic_has_interrupt(struct kvm_vcpu *vcpu) return tdx_protected_apic_has_interrupt(vcpu); } +static void vt_apicv_pre_state_restore(struct kvm_vcpu *vcpu) +{ + struct pi_desc *pi = vcpu_to_pi_desc(vcpu); + + pi_clear_on(pi); + memset(pi->pir, 0, sizeof(pi->pir)); +} + +static int vt_sync_pir_to_irr(struct kvm_vcpu *vcpu) +{ + if (is_td_vcpu(vcpu)) + return -1; + + return vmx_sync_pir_to_irr(vcpu); +} + +static void vt_deliver_interrupt(struct kvm_lapic *apic, int delivery_mode, + int trig_mode, int vector) +{ + if (is_td_vcpu(apic->vcpu)) { + tdx_deliver_interrupt(apic, delivery_mode, trig_mode, + vector); + return; + } + + vmx_deliver_interrupt(apic, delivery_mode, trig_mode, vector); +} + static void vt_flush_tlb_all(struct kvm_vcpu *vcpu) { if (is_td_vcpu(vcpu)) { @@ -297,6 +325,53 @@ static void vt_sched_in(struct kvm_vcpu *vcpu, int cpu) vmx_sched_in(vcpu, cpu); } +static void vt_set_interrupt_shadow(struct kvm_vcpu *vcpu, int mask) +{ + if (is_td_vcpu(vcpu)) + return; + vmx_set_interrupt_shadow(vcpu, mask); +} + +static u32 vt_get_interrupt_shadow(struct kvm_vcpu *vcpu) +{ + if (is_td_vcpu(vcpu)) + return 0; + + return vmx_get_interrupt_shadow(vcpu); +} + +static void vt_inject_irq(struct kvm_vcpu *vcpu, bool reinjected) +{ + if (is_td_vcpu(vcpu)) + return; + + vmx_inject_irq(vcpu, reinjected); +} + +static void vt_cancel_injection(struct kvm_vcpu *vcpu) +{ + if (is_td_vcpu(vcpu)) + return; + + vmx_cancel_injection(vcpu); +} + +static int vt_interrupt_allowed(struct kvm_vcpu *vcpu, bool for_injection) +{ + if (is_td_vcpu(vcpu)) + return true; + + return vmx_interrupt_allowed(vcpu, for_injection); +} + +static void vt_enable_irq_window(struct kvm_vcpu *vcpu) +{ + if (is_td_vcpu(vcpu)) + return; + + vmx_enable_irq_window(vcpu); +} + static u8 vt_get_mt_mask(struct kvm_vcpu *vcpu, gfn_t gfn, bool is_mmio) { if (is_td_vcpu(vcpu)) @@ -426,31 +501,31 @@ struct kvm_x86_ops vt_x86_ops __initdata = { .handle_exit = vmx_handle_exit, .skip_emulated_instruction = vmx_skip_emulated_instruction, .update_emulated_instruction = vmx_update_emulated_instruction, - .set_interrupt_shadow = vmx_set_interrupt_shadow, - .get_interrupt_shadow = vmx_get_interrupt_shadow, + .set_interrupt_shadow = vt_set_interrupt_shadow, + .get_interrupt_shadow = vt_get_interrupt_shadow, .patch_hypercall = vmx_patch_hypercall, - .inject_irq = vmx_inject_irq, + .inject_irq = vt_inject_irq, .inject_nmi = vmx_inject_nmi, .inject_exception = vmx_inject_exception, - .cancel_injection = vmx_cancel_injection, - .interrupt_allowed = vmx_interrupt_allowed, + .cancel_injection = vt_cancel_injection, + .interrupt_allowed = vt_interrupt_allowed, .nmi_allowed = vmx_nmi_allowed, .get_nmi_mask = vmx_get_nmi_mask, .set_nmi_mask = vmx_set_nmi_mask, .enable_nmi_window = vmx_enable_nmi_window, - .enable_irq_window = vmx_enable_irq_window, + .enable_irq_window = vt_enable_irq_window, .update_cr8_intercept = vmx_update_cr8_intercept, .set_virtual_apic_mode = vmx_set_virtual_apic_mode, .set_apic_access_page_addr = vmx_set_apic_access_page_addr, .refresh_apicv_exec_ctrl = vmx_refresh_apicv_exec_ctrl, .load_eoi_exitmap = vmx_load_eoi_exitmap, - .apicv_pre_state_restore = vmx_apicv_pre_state_restore, + .apicv_pre_state_restore = vt_apicv_pre_state_restore, .required_apicv_inhibits = VMX_REQUIRED_APICV_INHIBITS, .hwapic_irr_update = vmx_hwapic_irr_update, .hwapic_isr_update = vmx_hwapic_isr_update, .guest_apic_has_interrupt = vmx_guest_apic_has_interrupt, - .sync_pir_to_irr = vmx_sync_pir_to_irr, - .deliver_interrupt = vmx_deliver_interrupt, + .sync_pir_to_irr = vt_sync_pir_to_irr, + .deliver_interrupt = vt_deliver_interrupt, .dy_apicv_has_pending_interrupt = pi_has_pending_interrupt, .protected_apic_has_interrupt = vt_protected_apic_has_interrupt, diff --git a/arch/x86/kvm/vmx/posted_intr.c b/arch/x86/kvm/vmx/posted_intr.c index b66add9da0f3..c86768b83f0b 100644 --- a/arch/x86/kvm/vmx/posted_intr.c +++ b/arch/x86/kvm/vmx/posted_intr.c @@ -52,7 +52,7 @@ static inline struct vcpu_pi *vcpu_to_pi(struct kvm_vcpu *vcpu) return (struct vcpu_pi *)vcpu; } -static inline struct pi_desc *vcpu_to_pi_desc(struct kvm_vcpu *vcpu) +struct pi_desc *vcpu_to_pi_desc(struct kvm_vcpu *vcpu) { return &vcpu_to_pi(vcpu)->pi_desc; } diff --git a/arch/x86/kvm/vmx/posted_intr.h b/arch/x86/kvm/vmx/posted_intr.h index 2fe8222308b2..0f9983b6910b 100644 --- a/arch/x86/kvm/vmx/posted_intr.h +++ b/arch/x86/kvm/vmx/posted_intr.h @@ -105,6 +105,8 @@ struct vcpu_pi { /* Until here common layout betwwn vcpu_vmx and vcpu_tdx. */ }; +struct pi_desc *vcpu_to_pi_desc(struct kvm_vcpu *vcpu); + void vmx_vcpu_pi_load(struct kvm_vcpu *vcpu, int cpu); void vmx_vcpu_pi_put(struct kvm_vcpu *vcpu); void pi_wakeup_handler(void); diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c index 1da58c36217c..1dfa9b503e0d 100644 --- a/arch/x86/kvm/vmx/tdx.c +++ b/arch/x86/kvm/vmx/tdx.c @@ -7,6 +7,7 @@ #include "capabilities.h" #include "x86_ops.h" +#include "common.h" #include "mmu.h" #include "tdx_arch.h" #include "tdx.h" @@ -603,6 +604,9 @@ int tdx_vcpu_create(struct kvm_vcpu *vcpu) tdx->host_state_need_save = true; tdx->host_state_need_restore = false; + tdx->pi_desc.nv = POSTED_INTR_VECTOR; + tdx->pi_desc.sn = 1; + return 0; } @@ -610,6 +614,7 @@ void tdx_vcpu_load(struct kvm_vcpu *vcpu, int cpu) { struct vcpu_tdx *tdx = to_tdx(vcpu); + vmx_vcpu_pi_load(vcpu, cpu); if (vcpu->cpu == cpu) return; @@ -848,6 +853,12 @@ fastpath_t tdx_vcpu_run(struct kvm_vcpu *vcpu) trace_kvm_entry(vcpu); + if (pi_test_on(&tdx->pi_desc)) { + apic->send_IPI_self(POSTED_INTR_VECTOR); + + kvm_wait_lapic_expire(vcpu); + } + tdx_vcpu_enter_exit(tdx); tdx_user_return_update_cache(vcpu); @@ -1213,6 +1224,16 @@ static int tdx_sept_remove_private_spte(struct kvm *kvm, gfn_t gfn, return tdx_sept_drop_private_spte(kvm, gfn, level, pfn); } +void tdx_deliver_interrupt(struct kvm_lapic *apic, int delivery_mode, + int trig_mode, int vector) +{ + struct kvm_vcpu *vcpu = apic->vcpu; + struct vcpu_tdx *tdx = to_tdx(vcpu); + + /* TDX supports only posted interrupt. No lapic emulation. */ + __vmx_deliver_posted_interrupt(vcpu, &tdx->pi_desc, vector); +} + static int tdx_get_capabilities(struct kvm_tdx_cmd *cmd) { struct kvm_tdx_capabilities __user *user_caps; @@ -1972,6 +1993,10 @@ int tdx_vcpu_ioctl(struct kvm_vcpu *vcpu, void __user *argp) if (ret) return ret; + td_vmcs_write16(tdx, POSTED_INTR_NV, POSTED_INTR_VECTOR); + td_vmcs_write64(tdx, POSTED_INTR_DESC_ADDR, __pa(&tdx->pi_desc)); + td_vmcs_setbit32(tdx, PIN_BASED_VM_EXEC_CONTROL, PIN_BASED_POSTED_INTR); + tdx->initialized = true; return 0; } diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 162bb134aae6..1349ec438837 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -4157,50 +4157,6 @@ void vmx_msr_filter_changed(struct kvm_vcpu *vcpu) pt_update_intercept_for_msr(vcpu); } -static inline void kvm_vcpu_trigger_posted_interrupt(struct kvm_vcpu *vcpu, - int pi_vec) -{ -#ifdef CONFIG_SMP - if (vcpu->mode == IN_GUEST_MODE) { - /* - * The vector of the virtual has already been set in the PIR. - * Send a notification event to deliver the virtual interrupt - * unless the vCPU is the currently running vCPU, i.e. the - * event is being sent from a fastpath VM-Exit handler, in - * which case the PIR will be synced to the vIRR before - * re-entering the guest. - * - * When the target is not the running vCPU, the following - * possibilities emerge: - * - * Case 1: vCPU stays in non-root mode. Sending a notification - * event posts the interrupt to the vCPU. - * - * Case 2: vCPU exits to root mode and is still runnable. The - * PIR will be synced to the vIRR before re-entering the guest. - * Sending a notification event is ok as the host IRQ handler - * will ignore the spurious event. - * - * Case 3: vCPU exits to root mode and is blocked. vcpu_block() - * has already synced PIR to vIRR and never blocks the vCPU if - * the vIRR is not empty. Therefore, a blocked vCPU here does - * not wait for any requested interrupts in PIR, and sending a - * notification event also results in a benign, spurious event. - */ - - if (vcpu != kvm_get_running_vcpu()) - __apic_send_IPI_mask(get_cpu_mask(vcpu->cpu), pi_vec); - return; - } -#endif - /* - * The vCPU isn't in the guest; wake the vCPU in case it is blocking, - * otherwise do nothing as KVM will grab the highest priority pending - * IRQ via ->sync_pir_to_irr() in vcpu_enter_guest(). - */ - kvm_vcpu_wake_up(vcpu); -} - static int vmx_deliver_nested_posted_interrupt(struct kvm_vcpu *vcpu, int vector) { @@ -4253,20 +4209,7 @@ static int vmx_deliver_posted_interrupt(struct kvm_vcpu *vcpu, int vector) if (!vcpu->arch.apic->apicv_active) return -1; - if (pi_test_and_set_pir(vector, &vmx->pi_desc)) - return 0; - - /* If a previous notification has sent the IPI, nothing to do. */ - if (pi_test_and_set_on(&vmx->pi_desc)) - return 0; - - /* - * The implied barrier in pi_test_and_set_on() pairs with the smp_mb_*() - * after setting vcpu->mode in vcpu_enter_guest(), thus the vCPU is - * guaranteed to see PID.ON=1 and sync the PIR to IRR if triggering a - * posted interrupt "fails" because vcpu->mode != IN_GUEST_MODE. - */ - kvm_vcpu_trigger_posted_interrupt(vcpu, POSTED_INTR_VECTOR); + __vmx_deliver_posted_interrupt(vcpu, &vmx->pi_desc, vector); return 0; } @@ -6946,14 +6889,6 @@ void vmx_load_eoi_exitmap(struct kvm_vcpu *vcpu, u64 *eoi_exit_bitmap) vmcs_write64(EOI_EXIT_BITMAP3, eoi_exit_bitmap[3]); } -void vmx_apicv_pre_state_restore(struct kvm_vcpu *vcpu) -{ - struct vcpu_vmx *vmx = to_vmx(vcpu); - - pi_clear_on(&vmx->pi_desc); - memset(vmx->pi_desc.pir, 0, sizeof(vmx->pi_desc.pir)); -} - void vmx_do_interrupt_irqoff(unsigned long entry); void vmx_do_nmi_irqoff(void); diff --git a/arch/x86/kvm/vmx/x86_ops.h b/arch/x86/kvm/vmx/x86_ops.h index f3f90186926c..12a212e71827 100644 --- a/arch/x86/kvm/vmx/x86_ops.h +++ b/arch/x86/kvm/vmx/x86_ops.h @@ -58,7 +58,6 @@ int vmx_check_intercept(struct kvm_vcpu *vcpu, bool vmx_apic_init_signal_blocked(struct kvm_vcpu *vcpu); void vmx_migrate_timers(struct kvm_vcpu *vcpu); void vmx_set_virtual_apic_mode(struct kvm_vcpu *vcpu); -void vmx_apicv_pre_state_restore(struct kvm_vcpu *vcpu); bool vmx_check_apicv_inhibit_reasons(enum kvm_apicv_inhibit reason); void vmx_hwapic_irr_update(struct kvm_vcpu *vcpu, int max_irr); void vmx_hwapic_isr_update(int max_isr); @@ -158,6 +157,9 @@ void tdx_vcpu_load(struct kvm_vcpu *vcpu, int cpu); bool tdx_protected_apic_has_interrupt(struct kvm_vcpu *vcpu); u8 tdx_get_mt_mask(struct kvm_vcpu *vcpu, gfn_t gfn, bool is_mmio); +void tdx_deliver_interrupt(struct kvm_lapic *apic, int delivery_mode, + int trig_mode, int vector); + int tdx_vcpu_ioctl(struct kvm_vcpu *vcpu, void __user *argp); void tdx_flush_tlb(struct kvm_vcpu *vcpu); @@ -198,6 +200,9 @@ static inline void tdx_vcpu_load(struct kvm_vcpu *vcpu, int cpu) {} static inline bool tdx_protected_apic_has_interrupt(struct kvm_vcpu *vcpu) { return false; } static inline u8 tdx_get_mt_mask(struct kvm_vcpu *vcpu, gfn_t gfn, bool is_mmio) { return 0; } +static inline void tdx_deliver_interrupt(struct kvm_lapic *apic, int delivery_mode, + int trig_mode, int vector) {} + static inline int tdx_vcpu_ioctl(struct kvm_vcpu *vcpu, void __user *argp) { return -EOPNOTSUPP; } static inline void tdx_flush_tlb(struct kvm_vcpu *vcpu) {} From patchwork Mon Feb 26 08:26:35 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571520 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A287E7AE52; Mon, 26 Feb 2024 08:28:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.19 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936133; cv=none; b=goPw+waxVSiY2AgP5WvkL/t1rFcXTUjbSMKIxK2On6Tujx4XGbjNjz0aOgwaDC8+Cn1cjtHWH8lwCq8SnFZL9hGVLq2xPQpGgxR8fAxEXqyR7tTAhAOOiKGUw0hgbYw55gJV9yDHTpGp9M6QdDW3XdRLUtHyr8cY2UZAc767z0Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936133; c=relaxed/simple; bh=8srNeTZplG2ftxUe7RfzcGruSkR0LpbTdRnxJyIOPTQ=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=n+5VFxFW+REiQ/tN9nESidefxIfI9rs6WSIv+tfQ4Mx/gnnIUk71qt5jJqDzOlUGTY3F5YVrBmD8NBpKaoqgRW4vUzXTFyCwz2DNwINa3BNrZcrAY7c40YL/LhJ35qQAAg3MUhEwqRD9gRqnfcu1kirLZtR4shbuXBYOkHG09x4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=U3pgJNqI; arc=none smtp.client-ip=198.175.65.19 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="U3pgJNqI" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936132; x=1740472132; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=8srNeTZplG2ftxUe7RfzcGruSkR0LpbTdRnxJyIOPTQ=; b=U3pgJNqI7q9mouSD+psZBjkzyPrmAktiNCS83+XBGchTJbZ3rlO4qfi0 wYcA32wNj2O5nyDBUPk6JmjxbhYqM6thgzKdHN3xWsxgj8hBk/ek077sA PR7GYXZMhHAFMNjiQtd2DEH0sgeRAfBcWPs9dy+IwTGXvw4maG8wHOFe0 HKoFHStkRL9pmNQeOLG/kktUbdhZj9cZK0fa46u5Sz2c8VTTNaSzMbg5z Ge7Q5hZ2fYKWCsSDHbkBbpsTQJTBQ1WWt1V4kThtwc8ao57daK+glbhwB UvRJy1gEf+4jYCQqLv75dRr80C5XYJQh0W0lx+ToFki6JLOkTr2plbx99 Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="3069567" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="3069567" Received: from orviesa004.jf.intel.com ([10.64.159.144]) by orvoesa111.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:51 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="11272653" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:51 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 093/130] KVM: TDX: Implements vcpu request_immediate_exit Date: Mon, 26 Feb 2024 00:26:35 -0800 Message-Id: <3fd2824a8f77412476b58155776e88dfe84a8c73.1708933498.git.isaku.yamahata@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata Now we are able to inject interrupts into TDX vcpu, it's ready to block TDX vcpu. Wire up kvm x86 methods for blocking/unblocking vcpu for TDX. To unblock on pending events, request immediate exit methods is also needed. Signed-off-by: Isaku Yamahata Reviewed-by: Paolo Bonzini --- arch/x86/kvm/vmx/main.c | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/arch/x86/kvm/vmx/main.c b/arch/x86/kvm/vmx/main.c index f2c9d6358f9e..ee6c04959d4c 100644 --- a/arch/x86/kvm/vmx/main.c +++ b/arch/x86/kvm/vmx/main.c @@ -372,6 +372,16 @@ static void vt_enable_irq_window(struct kvm_vcpu *vcpu) vmx_enable_irq_window(vcpu); } +static void vt_request_immediate_exit(struct kvm_vcpu *vcpu) +{ + if (is_td_vcpu(vcpu)) { + __kvm_request_immediate_exit(vcpu); + return; + } + + vmx_request_immediate_exit(vcpu); +} + static u8 vt_get_mt_mask(struct kvm_vcpu *vcpu, gfn_t gfn, bool is_mmio) { if (is_td_vcpu(vcpu)) @@ -549,7 +559,7 @@ struct kvm_x86_ops vt_x86_ops __initdata = { .check_intercept = vmx_check_intercept, .handle_exit_irqoff = vmx_handle_exit_irqoff, - .request_immediate_exit = vmx_request_immediate_exit, + .request_immediate_exit = vt_request_immediate_exit, .sched_in = vt_sched_in, From patchwork Mon Feb 26 08:26:36 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571523 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 93D077B3C1; Mon, 26 Feb 2024 08:28:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.19 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936135; cv=none; b=GsyUde9YgMOPKgJktt9P/u7PVb5Xi5cIjE6/KsP1BZbQUg3NThn7A4cF8cubvSTyjAGxB9fi5FdBe+N4fQ6ZFdotAIEofF4+v/dALQkjwB6RDBNtR+QNT1g3kxTjhY9TLrG/3QCNwt0a/yLB0DKj1VYaY92lAgnKaz14XOuynqU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936135; c=relaxed/simple; bh=FqA+upqL9gNfeYquYyLlBFbOWxO4m0XoA2LYxuCR4eo=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=IXfRbh+H9TxCq0XoGjfhvPepRkMZFnFnFwQ6Mor7t1e8KA5egp0Ms6A3wXQ7nCUzEabPZLK/T8QSrZQL3T6hFK0XeESwrlrZS7ZwJZfmRhArE1QtDem79wQ+NeB87SfOcEnTRZA77g7/BVljG4B2XIgpQG5wDoKBYk1ejpatgk4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=HUKOBSah; arc=none smtp.client-ip=198.175.65.19 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="HUKOBSah" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936133; x=1740472133; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=FqA+upqL9gNfeYquYyLlBFbOWxO4m0XoA2LYxuCR4eo=; b=HUKOBSah5zVQnYT0mHsayrpjfq0+7qsL9OGKwICQ8ZqvBBJUmPiEb2We n7KF7ygZST883g0W5Nq0mSamLAujMxpr4xjQAPq4EUWdwFLJz3dcG4cmk MW3dxJynsSF5EgHU+l8UW8tpwL5y8JghSV0tnf/6yq8EwRjtEKhXygnE5 V1ybjZpCWLyTK8g0asOJo0dxNIu316TuDQpT6iHPqjFDIXM1piymMjKlO 6ZirS/hB77d3jOcPOMwgzrvPVTNku/X/CXrwOdiG730i3HnHyysulWS1K 6b6R55waq3as+PUPOpR6P7DwG9v/iyTiNCZyy7p2mrKuMWysSbYn6jfQA A==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="3069571" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="3069571" Received: from orviesa004.jf.intel.com ([10.64.159.144]) by orvoesa111.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:52 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="11272658" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:52 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 094/130] KVM: TDX: Implement methods to inject NMI Date: Mon, 26 Feb 2024 00:26:36 -0800 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata TDX vcpu control structure defines one bit for pending NMI for VMM to inject NMI by setting the bit without knowing TDX vcpu NMI states. Because the vcpu state is protected, VMM can't know about NMI states of TDX vcpu. The TDX module handles actual injection and NMI states transition. Add methods for NMI and treat NMI can be injected always. Signed-off-by: Isaku Yamahata Reviewed-by: Paolo Bonzini --- arch/x86/kvm/vmx/main.c | 64 +++++++++++++++++++++++++++++++++++--- arch/x86/kvm/vmx/tdx.c | 6 ++++ arch/x86/kvm/vmx/x86_ops.h | 2 ++ 3 files changed, 67 insertions(+), 5 deletions(-) diff --git a/arch/x86/kvm/vmx/main.c b/arch/x86/kvm/vmx/main.c index ee6c04959d4c..6d6d443a2bbd 100644 --- a/arch/x86/kvm/vmx/main.c +++ b/arch/x86/kvm/vmx/main.c @@ -306,6 +306,60 @@ static void vt_flush_tlb_guest(struct kvm_vcpu *vcpu) vmx_flush_tlb_guest(vcpu); } +static void vt_inject_nmi(struct kvm_vcpu *vcpu) +{ + if (is_td_vcpu(vcpu)) { + tdx_inject_nmi(vcpu); + return; + } + + vmx_inject_nmi(vcpu); +} + +static int vt_nmi_allowed(struct kvm_vcpu *vcpu, bool for_injection) +{ + /* + * The TDX module manages NMI windows and NMI reinjection, and hides NMI + * blocking, all KVM can do is throw an NMI over the wall. + */ + if (is_td_vcpu(vcpu)) + return true; + + return vmx_nmi_allowed(vcpu, for_injection); +} + +static bool vt_get_nmi_mask(struct kvm_vcpu *vcpu) +{ + /* + * Assume NMIs are always unmasked. KVM could query PEND_NMI and treat + * NMIs as masked if a previous NMI is still pending, but SEAMCALLs are + * expensive and the end result is unchanged as the only relevant usage + * of get_nmi_mask() is to limit the number of pending NMIs, i.e. it + * only changes whether KVM or the TDX module drops an NMI. + */ + if (is_td_vcpu(vcpu)) + return false; + + return vmx_get_nmi_mask(vcpu); +} + +static void vt_set_nmi_mask(struct kvm_vcpu *vcpu, bool masked) +{ + if (is_td_vcpu(vcpu)) + return; + + vmx_set_nmi_mask(vcpu, masked); +} + +static void vt_enable_nmi_window(struct kvm_vcpu *vcpu) +{ + /* Refer the comment in vt_get_nmi_mask(). */ + if (is_td_vcpu(vcpu)) + return; + + vmx_enable_nmi_window(vcpu); +} + static void vt_load_mmu_pgd(struct kvm_vcpu *vcpu, hpa_t root_hpa, int pgd_level) { @@ -515,14 +569,14 @@ struct kvm_x86_ops vt_x86_ops __initdata = { .get_interrupt_shadow = vt_get_interrupt_shadow, .patch_hypercall = vmx_patch_hypercall, .inject_irq = vt_inject_irq, - .inject_nmi = vmx_inject_nmi, + .inject_nmi = vt_inject_nmi, .inject_exception = vmx_inject_exception, .cancel_injection = vt_cancel_injection, .interrupt_allowed = vt_interrupt_allowed, - .nmi_allowed = vmx_nmi_allowed, - .get_nmi_mask = vmx_get_nmi_mask, - .set_nmi_mask = vmx_set_nmi_mask, - .enable_nmi_window = vmx_enable_nmi_window, + .nmi_allowed = vt_nmi_allowed, + .get_nmi_mask = vt_get_nmi_mask, + .set_nmi_mask = vt_set_nmi_mask, + .enable_nmi_window = vt_enable_nmi_window, .enable_irq_window = vt_enable_irq_window, .update_cr8_intercept = vmx_update_cr8_intercept, .set_virtual_apic_mode = vmx_set_virtual_apic_mode, diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c index 1dfa9b503e0d..be21dca47992 100644 --- a/arch/x86/kvm/vmx/tdx.c +++ b/arch/x86/kvm/vmx/tdx.c @@ -874,6 +874,12 @@ fastpath_t tdx_vcpu_run(struct kvm_vcpu *vcpu) return EXIT_FASTPATH_NONE; } +void tdx_inject_nmi(struct kvm_vcpu *vcpu) +{ + ++vcpu->stat.nmi_injections; + td_management_write8(to_tdx(vcpu), TD_VCPU_PEND_NMI, 1); +} + void tdx_load_mmu_pgd(struct kvm_vcpu *vcpu, hpa_t root_hpa, int pgd_level) { WARN_ON_ONCE(root_hpa & ~PAGE_MASK); diff --git a/arch/x86/kvm/vmx/x86_ops.h b/arch/x86/kvm/vmx/x86_ops.h index 12a212e71827..539f3f9686fe 100644 --- a/arch/x86/kvm/vmx/x86_ops.h +++ b/arch/x86/kvm/vmx/x86_ops.h @@ -159,6 +159,7 @@ u8 tdx_get_mt_mask(struct kvm_vcpu *vcpu, gfn_t gfn, bool is_mmio); void tdx_deliver_interrupt(struct kvm_lapic *apic, int delivery_mode, int trig_mode, int vector); +void tdx_inject_nmi(struct kvm_vcpu *vcpu); int tdx_vcpu_ioctl(struct kvm_vcpu *vcpu, void __user *argp); @@ -202,6 +203,7 @@ static inline u8 tdx_get_mt_mask(struct kvm_vcpu *vcpu, gfn_t gfn, bool is_mmio) static inline void tdx_deliver_interrupt(struct kvm_lapic *apic, int delivery_mode, int trig_mode, int vector) {} +static inline void tdx_inject_nmi(struct kvm_vcpu *vcpu) {} static inline int tdx_vcpu_ioctl(struct kvm_vcpu *vcpu, void __user *argp) { return -EOPNOTSUPP; } From patchwork Mon Feb 26 08:26:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571522 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6687B7BAF1; Mon, 26 Feb 2024 08:28:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.19 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936135; cv=none; b=qLP2DJ4Rp4CGJr1d4fHPWqeElBWuzuq8a5gHPv/WE4un+cXdCIg4ODQocNDwH8BvuwywgK+aeiZ+ezQuzA5TCfbiBGl6Jmd4s++NXumbZ1FbyGnCJnVh9sFSi3BMYChZrwmUr/vm3hp+BW6XOU02ZvUsWf2o88Z7wxNXeMG9/JU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936135; c=relaxed/simple; bh=2YsbYPL8eW8m04NLRwFz+0vexIk0QRL8LnEfWGvgIaY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=swdb7nH3RrE8mM2aO4FwgNrwBk8zATLjopBeH82pZg20V0NTosqs17mkG+5zPyGKabM7zLzw2DmdSm0jWhso3l689jCBhJ88sw/vnhGoK37JsTlDWUcY7WczNsWvgy1Dnb00eeO7TlqIkpzmj7x8UVLVbuuG3zkB6k4TTcgu13Q= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=YHiUh41i; arc=none smtp.client-ip=198.175.65.19 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="YHiUh41i" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936134; x=1740472134; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=2YsbYPL8eW8m04NLRwFz+0vexIk0QRL8LnEfWGvgIaY=; b=YHiUh41iVHt3A45l0yrwWaJQ8i+6Rg2zayw1UdLHCfbKPNzRp6PdlYIn XGJDkNWifihtGHg/wz+A0ykIDRQWBA4YUTYfmv3nt8O5548F9bSpWPIxy 1NUEEwJ/tkE0Cye4ShGn+5HgjhZM0jH5IfzXqcDwe1cecOzFzKAQMyyuf XKYILB6ctY3EbCHH7zuJPytbrSTYSWR6YpxcDpIw3F7GB60mtivWELx7G 7abM3Kbj9jDmbD3KCi8vslmpByS0khULR7nQPbIaGt5cfivxLfSBg96ll OAdiu7Nm+vNbcA3U7C5PrXkK3SIrzX14G1TYlFTHVHNPht4s03AqI8JtL Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="3069576" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="3069576" Received: from orviesa004.jf.intel.com ([10.64.159.144]) by orvoesa111.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:53 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="11272664" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:52 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com, Sean Christopherson Subject: [PATCH v19 095/130] KVM: VMX: Modify NMI and INTR handlers to take intr_info as function argument Date: Mon, 26 Feb 2024 00:26:37 -0800 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Sean Christopherson TDX uses different ABI to get information about VM exit. Pass intr_info to the NMI and INTR handlers instead of pulling it from vcpu_vmx in preparation for sharing the bulk of the handlers with TDX. When the guest TD exits to VMM, RAX holds status and exit reason, RCX holds exit qualification etc rather than the VMCS fields because VMM doesn't have access to the VMCS. The eventual code will be VMX: - get exit reason, intr_info, exit_qualification, and etc from VMCS - call NMI/INTR handlers (common code) TDX: - get exit reason, intr_info, exit_qualification, and etc from guest registers - call NMI/INTR handlers (common code) Signed-off-by: Sean Christopherson Signed-off-by: Isaku Yamahata Reviewed-by: Paolo Bonzini Reviewed-by: Chao Gao --- arch/x86/kvm/vmx/vmx.c | 16 +++++++--------- 1 file changed, 7 insertions(+), 9 deletions(-) diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 1349ec438837..29d891e0795e 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -6912,24 +6912,22 @@ static void handle_nm_fault_irqoff(struct kvm_vcpu *vcpu) rdmsrl(MSR_IA32_XFD_ERR, vcpu->arch.guest_fpu.xfd_err); } -static void handle_exception_irqoff(struct vcpu_vmx *vmx) +static void handle_exception_irqoff(struct kvm_vcpu *vcpu, u32 intr_info) { - u32 intr_info = vmx_get_intr_info(&vmx->vcpu); - /* if exit due to PF check for async PF */ if (is_page_fault(intr_info)) - vmx->vcpu.arch.apf.host_apf_flags = kvm_read_and_reset_apf_flags(); + vcpu->arch.apf.host_apf_flags = kvm_read_and_reset_apf_flags(); /* if exit due to NM, handle before interrupts are enabled */ else if (is_nm_fault(intr_info)) - handle_nm_fault_irqoff(&vmx->vcpu); + handle_nm_fault_irqoff(vcpu); /* Handle machine checks before interrupts are enabled */ else if (is_machine_check(intr_info)) kvm_machine_check(); } -static void handle_external_interrupt_irqoff(struct kvm_vcpu *vcpu) +static void handle_external_interrupt_irqoff(struct kvm_vcpu *vcpu, + u32 intr_info) { - u32 intr_info = vmx_get_intr_info(vcpu); unsigned int vector = intr_info & INTR_INFO_VECTOR_MASK; gate_desc *desc = (gate_desc *)host_idt_base + vector; @@ -6952,9 +6950,9 @@ void vmx_handle_exit_irqoff(struct kvm_vcpu *vcpu) return; if (vmx->exit_reason.basic == EXIT_REASON_EXTERNAL_INTERRUPT) - handle_external_interrupt_irqoff(vcpu); + handle_external_interrupt_irqoff(vcpu, vmx_get_intr_info(vcpu)); else if (vmx->exit_reason.basic == EXIT_REASON_EXCEPTION_NMI) - handle_exception_irqoff(vmx); + handle_exception_irqoff(vcpu, vmx_get_intr_info(vcpu)); } /* From patchwork Mon Feb 26 08:26:38 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571524 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0D65C7C0B2; Mon, 26 Feb 2024 08:28:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.19 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936136; cv=none; b=RuOUYNWwGuFIcQqDv/0TtVuaO7nFyVIHK4403j5ogr9qdpaVX4+h9tLhduBKOkxo+ink1CtsQAb8FzSfZlg23F+bbiw1KrjnuXQGuGihTMvX7i2rW9RCZdHornL11hrGUKi/CvuCtzKDMbsA23oGwOysq65rDrt1o0gHUCPVXfc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936136; c=relaxed/simple; bh=IWPbF7kZorjnaCcjYxxog90DbjeIj9s2BnbE9skJqgc=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=nSqDUYS7EF1kMD7t1+Tjtcp4smWJVrXh3D+5NrJDG9jl3LKhrjxmngOD8lr8YqV8IoiAySgP6aARtTxQpg3FhKv6JEM+uVFnvdoBscGwH7Y4kkDX+jG81acCxUH/SXhVSKmMGZOjyZQBCDku+yiSTwix3Wt4Ie2nTasFA7QYR8o= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=n7gOygWI; arc=none smtp.client-ip=198.175.65.19 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="n7gOygWI" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936134; x=1740472134; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=IWPbF7kZorjnaCcjYxxog90DbjeIj9s2BnbE9skJqgc=; b=n7gOygWI4do3yG7L2pmJfWAc3i5aLceBpHChDsTRUel8KvTOyqw/gq5h YUZ1vePOyCBcz8bU4KdBUQvUFmym6hdXl6VVAnFCvTZ3eHuInntqsJ+eq om9p8JvxTHuXzHNWbBzM850H6nmuNo3fUE7SlAFEiNCXuNmiot3pJccui RFjvDhKQd73GyFgJH49Fo+Lb6I5+Z+VvdXk5uHzcmWYuMt6m95/MQB50w /uxE2semFsB9jLEsUSFTnfl6A0/g+fPFdDswxPugOk96ZDSiEFSWzxfMC O+ShRPcjI5lsS9nkTjgKnmeVDKveR0hI6oZgEmXIbc07yxRhEA1dfA9w+ A==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="3069583" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="3069583" Received: from orviesa004.jf.intel.com ([10.64.159.144]) by orvoesa111.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:54 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="11272669" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:53 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com, Sean Christopherson Subject: [PATCH v19 096/130] KVM: VMX: Move NMI/exception handler to common helper Date: Mon, 26 Feb 2024 00:26:38 -0800 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Sean Christopherson TDX mostly handles NMI/exception exit mostly the same to VMX case. The difference is how to retrieve exit qualification. To share the code with TDX, move NMI/exception to a common header, common.h. Signed-off-by: Sean Christopherson Signed-off-by: Isaku Yamahata --- arch/x86/kvm/vmx/common.h | 59 +++++++++++++++++++++++++++++++++ arch/x86/kvm/vmx/vmx.c | 68 +++++---------------------------------- 2 files changed, 67 insertions(+), 60 deletions(-) diff --git a/arch/x86/kvm/vmx/common.h b/arch/x86/kvm/vmx/common.h index 6f21d0d48809..632af7a76d0a 100644 --- a/arch/x86/kvm/vmx/common.h +++ b/arch/x86/kvm/vmx/common.h @@ -4,8 +4,67 @@ #include +#include + #include "posted_intr.h" #include "mmu.h" +#include "vmcs.h" +#include "x86.h" + +extern unsigned long vmx_host_idt_base; +void vmx_do_interrupt_irqoff(unsigned long entry); +void vmx_do_nmi_irqoff(void); + +static inline void vmx_handle_nm_fault_irqoff(struct kvm_vcpu *vcpu) +{ + /* + * Save xfd_err to guest_fpu before interrupt is enabled, so the + * MSR value is not clobbered by the host activity before the guest + * has chance to consume it. + * + * Do not blindly read xfd_err here, since this exception might + * be caused by L1 interception on a platform which doesn't + * support xfd at all. + * + * Do it conditionally upon guest_fpu::xfd. xfd_err matters + * only when xfd contains a non-zero value. + * + * Queuing exception is done in vmx_handle_exit. See comment there. + */ + if (vcpu->arch.guest_fpu.fpstate->xfd) + rdmsrl(MSR_IA32_XFD_ERR, vcpu->arch.guest_fpu.xfd_err); +} + +static inline void vmx_handle_exception_irqoff(struct kvm_vcpu *vcpu, + u32 intr_info) +{ + /* if exit due to PF check for async PF */ + if (is_page_fault(intr_info)) + vcpu->arch.apf.host_apf_flags = kvm_read_and_reset_apf_flags(); + /* if exit due to NM, handle before interrupts are enabled */ + else if (is_nm_fault(intr_info)) + vmx_handle_nm_fault_irqoff(vcpu); + /* Handle machine checks before interrupts are enabled */ + else if (is_machine_check(intr_info)) + kvm_machine_check(); +} + +static inline void vmx_handle_external_interrupt_irqoff(struct kvm_vcpu *vcpu, + u32 intr_info) +{ + unsigned int vector = intr_info & INTR_INFO_VECTOR_MASK; + gate_desc *desc = (gate_desc *)vmx_host_idt_base + vector; + + if (KVM_BUG(!is_external_intr(intr_info), vcpu->kvm, + "unexpected VM-Exit interrupt info: 0x%x", intr_info)) + return; + + kvm_before_interrupt(vcpu, KVM_HANDLING_IRQ); + vmx_do_interrupt_irqoff(gate_offset(desc)); + kvm_after_interrupt(vcpu); + + vcpu->arch.at_instruction_boundary = true; +} static inline int __vmx_handle_ept_violation(struct kvm_vcpu *vcpu, gpa_t gpa, unsigned long exit_qualification) diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 29d891e0795e..f8a00a766c40 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -518,7 +518,7 @@ static inline void vmx_segment_cache_clear(struct vcpu_vmx *vmx) vmx->segment_cache.bitmask = 0; } -static unsigned long host_idt_base; +unsigned long vmx_host_idt_base; #if IS_ENABLED(CONFIG_HYPERV) static bool __read_mostly enlightened_vmcs = true; @@ -4273,7 +4273,7 @@ void vmx_set_constant_host_state(struct vcpu_vmx *vmx) vmcs_write16(HOST_SS_SELECTOR, __KERNEL_DS); /* 22.2.4 */ vmcs_write16(HOST_TR_SELECTOR, GDT_ENTRY_TSS*8); /* 22.2.4 */ - vmcs_writel(HOST_IDTR_BASE, host_idt_base); /* 22.2.4 */ + vmcs_writel(HOST_IDTR_BASE, vmx_host_idt_base); /* 22.2.4 */ vmcs_writel(HOST_RIP, (unsigned long)vmx_vmexit); /* 22.2.5 */ @@ -5166,7 +5166,7 @@ static int handle_exception_nmi(struct kvm_vcpu *vcpu) intr_info = vmx_get_intr_info(vcpu); /* - * Machine checks are handled by handle_exception_irqoff(), or by + * Machine checks are handled by vmx_handle_exception_irqoff(), or by * vmx_vcpu_run() if a #MC occurs on VM-Entry. NMIs are handled by * vmx_vcpu_enter_exit(). */ @@ -5174,7 +5174,7 @@ static int handle_exception_nmi(struct kvm_vcpu *vcpu) return 1; /* - * Queue the exception here instead of in handle_nm_fault_irqoff(). + * Queue the exception here instead of in vmx_handle_nm_fault_irqoff(). * This ensures the nested_vmx check is not skipped so vmexit can * be reflected to L1 (when it intercepts #NM) before reaching this * point. @@ -6889,59 +6889,6 @@ void vmx_load_eoi_exitmap(struct kvm_vcpu *vcpu, u64 *eoi_exit_bitmap) vmcs_write64(EOI_EXIT_BITMAP3, eoi_exit_bitmap[3]); } -void vmx_do_interrupt_irqoff(unsigned long entry); -void vmx_do_nmi_irqoff(void); - -static void handle_nm_fault_irqoff(struct kvm_vcpu *vcpu) -{ - /* - * Save xfd_err to guest_fpu before interrupt is enabled, so the - * MSR value is not clobbered by the host activity before the guest - * has chance to consume it. - * - * Do not blindly read xfd_err here, since this exception might - * be caused by L1 interception on a platform which doesn't - * support xfd at all. - * - * Do it conditionally upon guest_fpu::xfd. xfd_err matters - * only when xfd contains a non-zero value. - * - * Queuing exception is done in vmx_handle_exit. See comment there. - */ - if (vcpu->arch.guest_fpu.fpstate->xfd) - rdmsrl(MSR_IA32_XFD_ERR, vcpu->arch.guest_fpu.xfd_err); -} - -static void handle_exception_irqoff(struct kvm_vcpu *vcpu, u32 intr_info) -{ - /* if exit due to PF check for async PF */ - if (is_page_fault(intr_info)) - vcpu->arch.apf.host_apf_flags = kvm_read_and_reset_apf_flags(); - /* if exit due to NM, handle before interrupts are enabled */ - else if (is_nm_fault(intr_info)) - handle_nm_fault_irqoff(vcpu); - /* Handle machine checks before interrupts are enabled */ - else if (is_machine_check(intr_info)) - kvm_machine_check(); -} - -static void handle_external_interrupt_irqoff(struct kvm_vcpu *vcpu, - u32 intr_info) -{ - unsigned int vector = intr_info & INTR_INFO_VECTOR_MASK; - gate_desc *desc = (gate_desc *)host_idt_base + vector; - - if (KVM_BUG(!is_external_intr(intr_info), vcpu->kvm, - "unexpected VM-Exit interrupt info: 0x%x", intr_info)) - return; - - kvm_before_interrupt(vcpu, KVM_HANDLING_IRQ); - vmx_do_interrupt_irqoff(gate_offset(desc)); - kvm_after_interrupt(vcpu); - - vcpu->arch.at_instruction_boundary = true; -} - void vmx_handle_exit_irqoff(struct kvm_vcpu *vcpu) { struct vcpu_vmx *vmx = to_vmx(vcpu); @@ -6950,9 +6897,10 @@ void vmx_handle_exit_irqoff(struct kvm_vcpu *vcpu) return; if (vmx->exit_reason.basic == EXIT_REASON_EXTERNAL_INTERRUPT) - handle_external_interrupt_irqoff(vcpu, vmx_get_intr_info(vcpu)); + vmx_handle_external_interrupt_irqoff(vcpu, + vmx_get_intr_info(vcpu)); else if (vmx->exit_reason.basic == EXIT_REASON_EXCEPTION_NMI) - handle_exception_irqoff(vcpu, vmx_get_intr_info(vcpu)); + vmx_handle_exception_irqoff(vcpu, vmx_get_intr_info(vcpu)); } /* @@ -8284,7 +8232,7 @@ __init int vmx_hardware_setup(void) int r; store_idt(&dt); - host_idt_base = dt.address; + vmx_host_idt_base = dt.address; vmx_setup_user_return_msrs(); From patchwork Mon Feb 26 08:26:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571525 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 021F97D3F6; Mon, 26 Feb 2024 08:28:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.19 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936137; cv=none; b=gNXLaorXzCRaWXBH6/SK6Jr8Z5znR1BOVE/4zoUdK9oXPQpwlhMGp+QZ8ZcvEaLFUJ2HO5iFGSSL5DT8i39Z8nUdF6tUt5Azs5xXK9rwL78jmzNxbKXLR9GzE+ngcJRmOtAl4QZpZPk6hyOlNxjQ//zOidlfnQU1YRzW/J4V2Tk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936137; c=relaxed/simple; bh=b4MOR2zOZ9SOVUfbFZBno3QMxpOn32TQq9uMQ+RFtDc=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=sbhVtEU0ipFOPBJL4OK8H5fv5BuOwpb+z8955emYspZwHjFsgKi3A9+nb+5+y+eX1UeVup5HqmS6kZ/z/G6TtEteSRc+mG3NBR/TFURJtdTIHgDvfAS4ve0XxfE6hINKTo/7WCAvCzS5LujrnMW2TJM8Nu9Dv8dac+y1K2VHFNY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=iZnzHOCJ; arc=none smtp.client-ip=198.175.65.19 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="iZnzHOCJ" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936136; x=1740472136; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=b4MOR2zOZ9SOVUfbFZBno3QMxpOn32TQq9uMQ+RFtDc=; b=iZnzHOCJAptIv+69i4+O4FlGt27KNka/KyTHnizhoSA/0oVCnH4JjXsu xZXO1Buc4ngB8E63rMt96tw3O4j+PgquzIZvm35SpJABaZ3wTNk1LrexY 0ok7ffgAVJjQqakGrTrewUMD5Dm3Nmp7chqf1S06yuMmJvayMHt/dBJ6T 2DaBDP3shZnSoXnT41Ypx1rF/JRkPCORSSTst+MVPIDIFYbA0aX6UkhOG CMJCVBm1GBv0khSMWhKLkMLgGjktfNT+do4nSMWViQpRYydAGsHowWEec dKtfVNGYFEPR/Qe22RANlhqeBC+ALPYRWcubUWqzGbKCuo/Zz4KUJEMVA w==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="3069586" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="3069586" Received: from orviesa004.jf.intel.com ([10.64.159.144]) by orvoesa111.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:56 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="11272673" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:55 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com, Sean Christopherson Subject: [PATCH v19 097/130] KVM: x86: Split core of hypercall emulation to helper function Date: Mon, 26 Feb 2024 00:26:39 -0800 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Sean Christopherson By necessity, TDX will use a different register ABI for hypercalls. Break out the core functionality so that it may be reused for TDX. Signed-off-by: Sean Christopherson Signed-off-by: Isaku Yamahata --- arch/x86/include/asm/kvm_host.h | 4 +++ arch/x86/kvm/x86.c | 56 ++++++++++++++++++++++----------- 2 files changed, 42 insertions(+), 18 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index e0ffef1d377d..bb8be091f996 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -2177,6 +2177,10 @@ static inline void kvm_clear_apicv_inhibit(struct kvm *kvm, kvm_set_or_clear_apicv_inhibit(kvm, reason, false); } +unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr, + unsigned long a0, unsigned long a1, + unsigned long a2, unsigned long a3, + int op_64_bit, int cpl); int kvm_emulate_hypercall(struct kvm_vcpu *vcpu); int kvm_mmu_page_fault(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa, u64 error_code, diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index fb7597c22f31..03950368d8db 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -10073,26 +10073,15 @@ static int complete_hypercall_exit(struct kvm_vcpu *vcpu) return kvm_skip_emulated_instruction(vcpu); } -int kvm_emulate_hypercall(struct kvm_vcpu *vcpu) +unsigned long __kvm_emulate_hypercall(struct kvm_vcpu *vcpu, unsigned long nr, + unsigned long a0, unsigned long a1, + unsigned long a2, unsigned long a3, + int op_64_bit, int cpl) { - unsigned long nr, a0, a1, a2, a3, ret; - int op_64_bit; - - if (kvm_xen_hypercall_enabled(vcpu->kvm)) - return kvm_xen_hypercall(vcpu); - - if (kvm_hv_hypercall_enabled(vcpu)) - return kvm_hv_hypercall(vcpu); - - nr = kvm_rax_read(vcpu); - a0 = kvm_rbx_read(vcpu); - a1 = kvm_rcx_read(vcpu); - a2 = kvm_rdx_read(vcpu); - a3 = kvm_rsi_read(vcpu); + unsigned long ret; trace_kvm_hypercall(nr, a0, a1, a2, a3); - op_64_bit = is_64_bit_hypercall(vcpu); if (!op_64_bit) { nr &= 0xFFFFFFFF; a0 &= 0xFFFFFFFF; @@ -10101,7 +10090,7 @@ int kvm_emulate_hypercall(struct kvm_vcpu *vcpu) a3 &= 0xFFFFFFFF; } - if (static_call(kvm_x86_get_cpl)(vcpu) != 0) { + if (cpl) { ret = -KVM_EPERM; goto out; } @@ -10162,18 +10151,49 @@ int kvm_emulate_hypercall(struct kvm_vcpu *vcpu) WARN_ON_ONCE(vcpu->run->hypercall.flags & KVM_EXIT_HYPERCALL_MBZ); vcpu->arch.complete_userspace_io = complete_hypercall_exit; + /* stat is incremented on completion. */ return 0; } default: ret = -KVM_ENOSYS; break; } + out: + ++vcpu->stat.hypercalls; + return ret; +} +EXPORT_SYMBOL_GPL(__kvm_emulate_hypercall); + +int kvm_emulate_hypercall(struct kvm_vcpu *vcpu) +{ + unsigned long nr, a0, a1, a2, a3, ret; + int op_64_bit; + int cpl; + + if (kvm_xen_hypercall_enabled(vcpu->kvm)) + return kvm_xen_hypercall(vcpu); + + if (kvm_hv_hypercall_enabled(vcpu)) + return kvm_hv_hypercall(vcpu); + + nr = kvm_rax_read(vcpu); + a0 = kvm_rbx_read(vcpu); + a1 = kvm_rcx_read(vcpu); + a2 = kvm_rdx_read(vcpu); + a3 = kvm_rsi_read(vcpu); + op_64_bit = is_64_bit_hypercall(vcpu); + cpl = static_call(kvm_x86_get_cpl)(vcpu); + + ret = __kvm_emulate_hypercall(vcpu, nr, a0, a1, a2, a3, op_64_bit, cpl); + if (nr == KVM_HC_MAP_GPA_RANGE && !ret) + /* MAP_GPA tosses the request to the user space. */ + return 0; + if (!op_64_bit) ret = (u32)ret; kvm_rax_write(vcpu, ret); - ++vcpu->stat.hypercalls; return kvm_skip_emulated_instruction(vcpu); } EXPORT_SYMBOL_GPL(kvm_emulate_hypercall); From patchwork Mon Feb 26 08:26:40 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571526 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7E62F7D416; Mon, 26 Feb 2024 08:28:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.19 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936138; cv=none; b=Zz/ZMj5glKFe79NdgfoycIlmc02B5s9ohcp6/FJGSl/wi8KOlTgKbtk/zkShOmd/Gwlrwo6kQT+kkHcgJH3N23sNKT2cpy3t8YiIBA65QLPvnrs2gRqZjienp8N5FmtobpjUOgdY/dwVDcZAFE8rd4e3txmffa2Lt4TElnSAD3o= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936138; c=relaxed/simple; bh=jlMt0YZGqlLhw75ET618T3F67gddxPyZ/5Jwlzv59nw=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=iadIr7V5RbLgPbQGYCwch82zgoExb7Q1P/QFx8jQ++BkjrSaG7MuXwp9UyY0niGjSvLfL5fVgecw1j6/GKGIbx6LdNaudckn8FbF/qp3BV9uvDGtxGHwZwtpgytm2ESbvKndCS63JpcZoswbBdmoww9m5IyIePXRqxFCdtItfZw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=cGkAQ9ms; arc=none smtp.client-ip=198.175.65.19 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="cGkAQ9ms" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936137; x=1740472137; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=jlMt0YZGqlLhw75ET618T3F67gddxPyZ/5Jwlzv59nw=; b=cGkAQ9ms0LIaYgN+LrulLEO+ifwpcfeTIJ4A2BJ7DjO4tTN9YW0cBZXd b7C7CJVhmUY8frdKvfcfPoZnTfomyPbnhAJ9Eo8uats3WON0W/hXoTFqM e8NsFVyL3gGqOTpJj37KwLBc29sUKXSFbiHVaxGpJgdRsnV7GwP+oR3Q6 pqe2GlEfnanTx0kDpUNY7TS/WJySAm36asxZGNdbI+QsJxNLixOG91dKe c+Z2o94GNWToAkFQdGwQJK+qeNTveCiqnJjloW0AseuGr66NfR16WMUup qTJbFXCONsmBdtbBZPEe8D7ampnUixLdLMmu9mX8BCcZiANV0N9ex/Kzi Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="3069588" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="3069588" Received: from orviesa004.jf.intel.com ([10.64.159.144]) by orvoesa111.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:57 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="11272676" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:56 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 098/130] KVM: TDX: Add a place holder to handle TDX VM exit Date: Mon, 26 Feb 2024 00:26:40 -0800 Message-Id: <88920c598dcb55c15219642f27d0781af6d0c044.1708933498.git.isaku.yamahata@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata Wire up handle_exit and handle_exit_irqoff methods and add a place holder to handle VM exit. Add helper functions to get exit info, exit qualification, etc. Signed-off-by: Isaku Yamahata Reviewed-by: Paolo Bonzini --- arch/x86/kvm/vmx/main.c | 37 ++++++++++++- arch/x86/kvm/vmx/tdx.c | 110 +++++++++++++++++++++++++++++++++++++ arch/x86/kvm/vmx/x86_ops.h | 10 ++++ 3 files changed, 154 insertions(+), 3 deletions(-) diff --git a/arch/x86/kvm/vmx/main.c b/arch/x86/kvm/vmx/main.c index 6d6d443a2bbd..c9a40456d965 100644 --- a/arch/x86/kvm/vmx/main.c +++ b/arch/x86/kvm/vmx/main.c @@ -228,6 +228,25 @@ static bool vt_protected_apic_has_interrupt(struct kvm_vcpu *vcpu) return tdx_protected_apic_has_interrupt(vcpu); } +static int vt_handle_exit(struct kvm_vcpu *vcpu, + enum exit_fastpath_completion fastpath) +{ + if (is_td_vcpu(vcpu)) + return tdx_handle_exit(vcpu, fastpath); + + return vmx_handle_exit(vcpu, fastpath); +} + +static void vt_handle_exit_irqoff(struct kvm_vcpu *vcpu) +{ + if (is_td_vcpu(vcpu)) { + tdx_handle_exit_irqoff(vcpu); + return; + } + + vmx_handle_exit_irqoff(vcpu); +} + static void vt_apicv_pre_state_restore(struct kvm_vcpu *vcpu) { struct pi_desc *pi = vcpu_to_pi_desc(vcpu); @@ -436,6 +455,18 @@ static void vt_request_immediate_exit(struct kvm_vcpu *vcpu) vmx_request_immediate_exit(vcpu); } +static void vt_get_exit_info(struct kvm_vcpu *vcpu, u32 *reason, + u64 *info1, u64 *info2, u32 *intr_info, u32 *error_code) +{ + if (is_td_vcpu(vcpu)) { + tdx_get_exit_info(vcpu, reason, info1, info2, intr_info, + error_code); + return; + } + + vmx_get_exit_info(vcpu, reason, info1, info2, intr_info, error_code); +} + static u8 vt_get_mt_mask(struct kvm_vcpu *vcpu, gfn_t gfn, bool is_mmio) { if (is_td_vcpu(vcpu)) @@ -562,7 +593,7 @@ struct kvm_x86_ops vt_x86_ops __initdata = { .vcpu_pre_run = vt_vcpu_pre_run, .vcpu_run = vt_vcpu_run, - .handle_exit = vmx_handle_exit, + .handle_exit = vt_handle_exit, .skip_emulated_instruction = vmx_skip_emulated_instruction, .update_emulated_instruction = vmx_update_emulated_instruction, .set_interrupt_shadow = vt_set_interrupt_shadow, @@ -597,7 +628,7 @@ struct kvm_x86_ops vt_x86_ops __initdata = { .set_identity_map_addr = vmx_set_identity_map_addr, .get_mt_mask = vt_get_mt_mask, - .get_exit_info = vmx_get_exit_info, + .get_exit_info = vt_get_exit_info, .vcpu_after_set_cpuid = vmx_vcpu_after_set_cpuid, @@ -611,7 +642,7 @@ struct kvm_x86_ops vt_x86_ops __initdata = { .load_mmu_pgd = vt_load_mmu_pgd, .check_intercept = vmx_check_intercept, - .handle_exit_irqoff = vmx_handle_exit_irqoff, + .handle_exit_irqoff = vt_handle_exit_irqoff, .request_immediate_exit = vt_request_immediate_exit, diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c index be21dca47992..71ab48cf72ba 100644 --- a/arch/x86/kvm/vmx/tdx.c +++ b/arch/x86/kvm/vmx/tdx.c @@ -120,6 +120,26 @@ static __always_inline hpa_t set_hkid_to_hpa(hpa_t pa, u16 hkid) return pa | ((hpa_t)hkid << boot_cpu_data.x86_phys_bits); } +static __always_inline unsigned long tdexit_exit_qual(struct kvm_vcpu *vcpu) +{ + return kvm_rcx_read(vcpu); +} + +static __always_inline unsigned long tdexit_ext_exit_qual(struct kvm_vcpu *vcpu) +{ + return kvm_rdx_read(vcpu); +} + +static __always_inline unsigned long tdexit_gpa(struct kvm_vcpu *vcpu) +{ + return kvm_r8_read(vcpu); +} + +static __always_inline unsigned long tdexit_intr_info(struct kvm_vcpu *vcpu) +{ + return kvm_r9_read(vcpu); +} + static inline bool is_td_vcpu_created(struct vcpu_tdx *tdx) { return tdx->td_vcpu_created; @@ -837,6 +857,12 @@ static noinstr void tdx_vcpu_enter_exit(struct vcpu_tdx *tdx) WARN_ON_ONCE(!kvm_rebooting && (tdx->exit_reason.full & TDX_SW_ERROR) == TDX_SW_ERROR); + if ((u16)tdx->exit_reason.basic == EXIT_REASON_EXCEPTION_NMI && + is_nmi(tdexit_intr_info(vcpu))) { + kvm_before_interrupt(vcpu, KVM_HANDLING_NMI); + vmx_do_nmi_irqoff(); + kvm_after_interrupt(vcpu); + } guest_state_exit_irqoff(); } @@ -880,6 +906,25 @@ void tdx_inject_nmi(struct kvm_vcpu *vcpu) td_management_write8(to_tdx(vcpu), TD_VCPU_PEND_NMI, 1); } +void tdx_handle_exit_irqoff(struct kvm_vcpu *vcpu) +{ + struct vcpu_tdx *tdx = to_tdx(vcpu); + u16 exit_reason = tdx->exit_reason.basic; + + if (exit_reason == EXIT_REASON_EXTERNAL_INTERRUPT) + vmx_handle_external_interrupt_irqoff(vcpu, + tdexit_intr_info(vcpu)); + else if (exit_reason == EXIT_REASON_EXCEPTION_NMI) + vmx_handle_exception_irqoff(vcpu, tdexit_intr_info(vcpu)); +} + +static int tdx_handle_triple_fault(struct kvm_vcpu *vcpu) +{ + vcpu->run->exit_reason = KVM_EXIT_SHUTDOWN; + vcpu->mmio_needed = 0; + return 0; +} + void tdx_load_mmu_pgd(struct kvm_vcpu *vcpu, hpa_t root_hpa, int pgd_level) { WARN_ON_ONCE(root_hpa & ~PAGE_MASK); @@ -1240,6 +1285,71 @@ void tdx_deliver_interrupt(struct kvm_lapic *apic, int delivery_mode, __vmx_deliver_posted_interrupt(vcpu, &tdx->pi_desc, vector); } +int tdx_handle_exit(struct kvm_vcpu *vcpu, fastpath_t fastpath) +{ + union tdx_exit_reason exit_reason = to_tdx(vcpu)->exit_reason; + + /* See the comment of tdh_sept_seamcall(). */ + if (unlikely(exit_reason.full == (TDX_OPERAND_BUSY | TDX_OPERAND_ID_SEPT))) + return 1; + + /* + * TDH.VP.ENTRY checks TD EPOCH which contend with TDH.MEM.TRACK and + * vcpu TDH.VP.ENTER. + */ + if (unlikely(exit_reason.full == (TDX_OPERAND_BUSY | TDX_OPERAND_ID_TD_EPOCH))) + return 1; + + if (unlikely(exit_reason.full == TDX_SEAMCALL_UD)) { + kvm_spurious_fault(); + /* + * In the case of reboot or kexec, loop with TDH.VP.ENTER and + * TDX_SEAMCALL_UD to avoid unnecessarily activity. + */ + return 1; + } + + if (unlikely(exit_reason.non_recoverable || exit_reason.error)) { + if (unlikely(exit_reason.basic == EXIT_REASON_TRIPLE_FAULT)) + return tdx_handle_triple_fault(vcpu); + + kvm_pr_unimpl("TD exit 0x%llx, %d hkid 0x%x hkid pa 0x%llx\n", + exit_reason.full, exit_reason.basic, + to_kvm_tdx(vcpu->kvm)->hkid, + set_hkid_to_hpa(0, to_kvm_tdx(vcpu->kvm)->hkid)); + goto unhandled_exit; + } + + WARN_ON_ONCE(fastpath != EXIT_FASTPATH_NONE); + + switch (exit_reason.basic) { + default: + break; + } + +unhandled_exit: + vcpu->run->exit_reason = KVM_EXIT_INTERNAL_ERROR; + vcpu->run->internal.suberror = KVM_INTERNAL_ERROR_UNEXPECTED_EXIT_REASON; + vcpu->run->internal.ndata = 2; + vcpu->run->internal.data[0] = exit_reason.full; + vcpu->run->internal.data[1] = vcpu->arch.last_vmentry_cpu; + return 0; +} + +void tdx_get_exit_info(struct kvm_vcpu *vcpu, u32 *reason, + u64 *info1, u64 *info2, u32 *intr_info, u32 *error_code) +{ + struct vcpu_tdx *tdx = to_tdx(vcpu); + + *reason = tdx->exit_reason.full; + + *info1 = tdexit_exit_qual(vcpu); + *info2 = tdexit_ext_exit_qual(vcpu); + + *intr_info = tdexit_intr_info(vcpu); + *error_code = 0; +} + static int tdx_get_capabilities(struct kvm_tdx_cmd *cmd) { struct kvm_tdx_capabilities __user *user_caps; diff --git a/arch/x86/kvm/vmx/x86_ops.h b/arch/x86/kvm/vmx/x86_ops.h index 539f3f9686fe..a12e3bfc96dd 100644 --- a/arch/x86/kvm/vmx/x86_ops.h +++ b/arch/x86/kvm/vmx/x86_ops.h @@ -155,11 +155,16 @@ void tdx_prepare_switch_to_guest(struct kvm_vcpu *vcpu); void tdx_vcpu_put(struct kvm_vcpu *vcpu); void tdx_vcpu_load(struct kvm_vcpu *vcpu, int cpu); bool tdx_protected_apic_has_interrupt(struct kvm_vcpu *vcpu); +void tdx_handle_exit_irqoff(struct kvm_vcpu *vcpu); +int tdx_handle_exit(struct kvm_vcpu *vcpu, + enum exit_fastpath_completion fastpath); u8 tdx_get_mt_mask(struct kvm_vcpu *vcpu, gfn_t gfn, bool is_mmio); void tdx_deliver_interrupt(struct kvm_lapic *apic, int delivery_mode, int trig_mode, int vector); void tdx_inject_nmi(struct kvm_vcpu *vcpu); +void tdx_get_exit_info(struct kvm_vcpu *vcpu, u32 *reason, + u64 *info1, u64 *info2, u32 *intr_info, u32 *error_code); int tdx_vcpu_ioctl(struct kvm_vcpu *vcpu, void __user *argp); @@ -199,11 +204,16 @@ static inline void tdx_prepare_switch_to_guest(struct kvm_vcpu *vcpu) {} static inline void tdx_vcpu_put(struct kvm_vcpu *vcpu) {} static inline void tdx_vcpu_load(struct kvm_vcpu *vcpu, int cpu) {} static inline bool tdx_protected_apic_has_interrupt(struct kvm_vcpu *vcpu) { return false; } +static inline void tdx_handle_exit_irqoff(struct kvm_vcpu *vcpu) {} +static inline int tdx_handle_exit(struct kvm_vcpu *vcpu, + enum exit_fastpath_completion fastpath) { return 0; } static inline u8 tdx_get_mt_mask(struct kvm_vcpu *vcpu, gfn_t gfn, bool is_mmio) { return 0; } static inline void tdx_deliver_interrupt(struct kvm_lapic *apic, int delivery_mode, int trig_mode, int vector) {} static inline void tdx_inject_nmi(struct kvm_vcpu *vcpu) {} +static inline void tdx_get_exit_info(struct kvm_vcpu *vcpu, u32 *reason, u64 *info1, + u64 *info2, u32 *intr_info, u32 *error_code) {} static inline int tdx_vcpu_ioctl(struct kvm_vcpu *vcpu, void __user *argp) { return -EOPNOTSUPP; } From patchwork Mon Feb 26 08:26:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571527 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 812EF7E56A; Mon, 26 Feb 2024 08:28:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.19 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936139; cv=none; b=TQZB2cztTHg2BhDTnvoKtOVW8SWB9CIg248vcgINPBna3xb2E0tmYXXV0oc50Vyya9xEdRe1ErRlCaLTYezMOSOhUGZXUiAe822IHb4X+zwKMy7w7o7dISSRmLaPbfkDI/ZEWrNL71uCkG7HVGUzIBIcxvVhjWHI4A8cAOko2x0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936139; c=relaxed/simple; bh=PfEUloIaN0+m6wTf244BqhPSXX4Whs3+DiuXu81jwnY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Clw63XQhANY6jLE0cSWiSwTxbAG63lote0dCpEWr2ROnvCj5eLjdwLtSQ3ZBQEQSCZlDBYRLVc6BkTXD+NFSweMOAK/nbe1M2uOqxQq0dGD4Y26OYtwwOT5pHwffPJ5hasKqfT2Yh1W980KmVPUCyA+Pu2MEub7G1oFG3lgWzZ8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=MxDdlcdF; arc=none smtp.client-ip=198.175.65.19 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="MxDdlcdF" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936138; x=1740472138; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=PfEUloIaN0+m6wTf244BqhPSXX4Whs3+DiuXu81jwnY=; b=MxDdlcdF+bIgSdhb00qqJOR6KPF+ODMX6dEGSgrzZ8tBSLqCqClTp1CV p41JWNiW4BerETvy6iPVJLjrK0r7fAkU/i2mxMc+QRYqzGlXUGdzrJ9uK gOc1O8UbBPowm2A5/vvurj/4e6ciN+MGY4RZecVV9AizduLZ9XfIsAoLd eeJjAjsVdRiHfjbyzgAbQ2D6gv+zQP/Sy8Nxx05RRSvVfascv7wQGFXMY 8bjx7Z6VcH/VU9XG8fzkLI8xcRsh5djnes575qoV03cmpSqNMhKiAs71o PU1/3PZ4r4awSbx5PraR6ef4z1Tbex1xYDatuvnS8muGSUSNidZ1rIENB g==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="3069592" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="3069592" Received: from orviesa004.jf.intel.com ([10.64.159.144]) by orvoesa111.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:57 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="11272680" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:57 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com, Yao Yuan Subject: [PATCH v19 099/130] KVM: TDX: Handle vmentry failure for INTEL TD guest Date: Mon, 26 Feb 2024 00:26:41 -0800 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Yao Yuan TDX module passes control back to VMM if it failed to vmentry for a TD, use same exit reason to notify user space, align with VMX. If VMM corrupted TD VMCS, machine check during entry can happens. vm exit reason will be EXIT_REASON_MCE_DURING_VMENTRY. If VMM corrupted TD VMCS with debug TD by TDH.VP.WR, the exit reason would be EXIT_REASON_INVALID_STATE or EXIT_REASON_MSR_LOAD_FAIL. Signed-off-by: Yao Yuan Signed-off-by: Isaku Yamahata --- arch/x86/kvm/vmx/tdx.c | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c index 71ab48cf72ba..cba0fd5029be 100644 --- a/arch/x86/kvm/vmx/tdx.c +++ b/arch/x86/kvm/vmx/tdx.c @@ -1320,6 +1320,28 @@ int tdx_handle_exit(struct kvm_vcpu *vcpu, fastpath_t fastpath) goto unhandled_exit; } + /* + * When TDX module saw VMEXIT_REASON_FAILED_VMENTER_MC etc, TDH.VP.ENTER + * returns with TDX_SUCCESS | exit_reason with failed_vmentry = 1. + * Because TDX module maintains TD VMCS correctness, usually vmentry + * failure shouldn't happen. In some corner cases it can happen. For + * example + * - machine check during entry: EXIT_REASON_MCE_DURING_VMENTRY + * - TDH.VP.WR with debug TD. VMM can corrupt TD VMCS + * - EXIT_REASON_INVALID_STATE + * - EXIT_REASON_MSR_LOAD_FAIL + */ + if (unlikely(exit_reason.failed_vmentry)) { + pr_err("TDExit: exit_reason 0x%016llx qualification=%016lx ext_qualification=%016lx\n", + exit_reason.full, tdexit_exit_qual(vcpu), tdexit_ext_exit_qual(vcpu)); + vcpu->run->exit_reason = KVM_EXIT_FAIL_ENTRY; + vcpu->run->fail_entry.hardware_entry_failure_reason + = exit_reason.full; + vcpu->run->fail_entry.cpu = vcpu->arch.last_vmentry_cpu; + + return 0; + } + WARN_ON_ONCE(fastpath != EXIT_FASTPATH_NONE); switch (exit_reason.basic) { From patchwork Mon Feb 26 08:26:42 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571528 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 13C937E583; Mon, 26 Feb 2024 08:28:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.19 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936139; cv=none; b=YVFj8Ad+Y3oEoqKhopnGdpFL8jyKaXbBOlnNoqo6U1YOb19DqWTi37mQu2LbiVmjAD8MiREq1Iyvc2De0p3K5EgphvWqnEpqy6zTpw0zP9P/T/3USdeF2SimVRF4xjlcHtGwyUEqb1uySqOo1P1G9cM9VZj5tMQMps6AAwiHydM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936139; c=relaxed/simple; bh=3tymZz66rzCmQ3uC8SJb3tbWD7d20lTx/+E3/aBaoK4=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=KzPJ3z9FLfv67OI/VHX/t65gymhFhDXsaW+R3c8u6LUT8dO6+mkaUh3dqeQAkS1ql35sptVbioQAh6z6kV+bgIXc1ouaIdrlwC0bjHR1YjS7eHBqsfn7GcXNxf472eJq5Yy9ks4qFdbjH5k1OVAt3oNTBK34O5L0wYQI0qKuPQI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=dwxbCuCl; arc=none smtp.client-ip=198.175.65.19 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="dwxbCuCl" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936138; x=1740472138; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=3tymZz66rzCmQ3uC8SJb3tbWD7d20lTx/+E3/aBaoK4=; b=dwxbCuClA5R87LNB8WiCzcFz+RnsuzEV1PsaJbrU2he+Xupj2U+hs6zr c8YQjjFiMNVrKw4b8vlSlQSw1dBY1qqzvVmDHzGIghNFB+sYyc8dUuKGF ikyQCIKJLjIRRloF2ai5a775ADflYbS9tdBiCy6yQiGqpO+3eZY7tL1Rb DGdfiVsW5SL0dBSGYB52joi8DNHULW1kH0n7Y9cx73f2QK5fK9hrBRgNt losF7c+CdoIWyuB4LZLmqj9O/R/nA71kuL5uW3W3a1PqY43bQ/2FSnuh0 JLskXJIlrzw0Ic1j8JF3RB1ePEHb6JqUWA33vn8R0vsJpZQWMiukthop0 A==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="3069596" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="3069596" Received: from orviesa004.jf.intel.com ([10.64.159.144]) by orvoesa111.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:58 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="11272684" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:57 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 100/130] KVM: TDX: handle EXIT_REASON_OTHER_SMI Date: Mon, 26 Feb 2024 00:26:42 -0800 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata If the control reaches EXIT_REASON_OTHER_SMI, #SMI is delivered and handled right after returning from the TDX module to KVM nothing needs to be done in KVM. Continue TDX vcpu execution. Signed-off-by: Isaku Yamahata Reviewed-by: Paolo Bonzini --- arch/x86/include/uapi/asm/vmx.h | 1 + arch/x86/kvm/vmx/tdx.c | 7 +++++++ 2 files changed, 8 insertions(+) diff --git a/arch/x86/include/uapi/asm/vmx.h b/arch/x86/include/uapi/asm/vmx.h index a5faf6d88f1b..b3a30ef3efdd 100644 --- a/arch/x86/include/uapi/asm/vmx.h +++ b/arch/x86/include/uapi/asm/vmx.h @@ -34,6 +34,7 @@ #define EXIT_REASON_TRIPLE_FAULT 2 #define EXIT_REASON_INIT_SIGNAL 3 #define EXIT_REASON_SIPI_SIGNAL 4 +#define EXIT_REASON_OTHER_SMI 6 #define EXIT_REASON_INTERRUPT_WINDOW 7 #define EXIT_REASON_NMI_WINDOW 8 diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c index cba0fd5029be..2f68e6f2b53a 100644 --- a/arch/x86/kvm/vmx/tdx.c +++ b/arch/x86/kvm/vmx/tdx.c @@ -1345,6 +1345,13 @@ int tdx_handle_exit(struct kvm_vcpu *vcpu, fastpath_t fastpath) WARN_ON_ONCE(fastpath != EXIT_FASTPATH_NONE); switch (exit_reason.basic) { + case EXIT_REASON_OTHER_SMI: + /* + * If reach here, it's not a Machine Check System Management + * Interrupt(MSMI). #SMI is delivered and handled right after + * SEAMRET, nothing needs to be done in KVM. + */ + return 1; default: break; } From patchwork Mon Feb 26 08:26:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571529 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B87F37E784; Mon, 26 Feb 2024 08:28:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.19 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936140; cv=none; b=Q6AkvedfZyF3gSLVFrlqrz6fnmy5LpfMM/GKn0e9sRgxHvehAqMJGuhOGiB8KNoSzw2eMz1S1xscINzs39/51ok08iiA7pZT3ZIf9oAOFDY8VgfyxG/6Ol4K/Ui+Y42DYPUlDIvp4Em9pK0xGxhhiN96LiCLGB71hSBcKiA+tSw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936140; c=relaxed/simple; bh=HofWRmC21b/CNK3j4sNmbUeHHzfuSu3ISYfW2FhCHEE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=J/nIxPfpMUYqS7bY1/BVYMDck9t8tas+7y6sYULxSIlmk6izkc4BCq9NdMaRYKd4XEpcMUkKvmZ69mG9NELXKLQ1Hy6I8ZERaEDfRdoIXkwyXO3R92PQSF0NYRAayXBz9koVtHx2Zo/3/aDFAQZ9Lpif91EnmmdFi4jXGUtJH9I= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=Rx0g048S; arc=none smtp.client-ip=198.175.65.19 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="Rx0g048S" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936139; x=1740472139; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=HofWRmC21b/CNK3j4sNmbUeHHzfuSu3ISYfW2FhCHEE=; b=Rx0g048SdzV9CUVkLkGJ7K6CP+pi2FOeQo8EnHw5E7709er7PFmbeK0J 15FVtEo+nAEW2pVc3N23rBOJiLHE6H3kTk0WFAmz7y0rujo01Q8iIs562 zXo/D8z7YfGwUWjOgS/ch3q3FMtE8gGAqZ3WcZ/4P2RAguYGc3IVa89Rf iUqlK23j9qm44+31IZeAWfnyrhLGWr6FrtPZXUQC5b5cgqPgE9QxZnyr5 6utOYZsgpfzJssw9oVPVLkoSIDk5O29mKE204JjJ3CF2ZSbgNmQfhWtIG dcO3OZCnBV447xvm8peZskyplJDi7dUZh2Dwxx2e2ZYj+O5jippu3r6tC A==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="3069603" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="3069603" Received: from orviesa004.jf.intel.com ([10.64.159.144]) by orvoesa111.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:59 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="11272687" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:58 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 101/130] KVM: TDX: handle ept violation/misconfig exit Date: Mon, 26 Feb 2024 00:26:43 -0800 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata On EPT violation, call a common function, __vmx_handle_ept_violation() to trigger x86 MMU code. On EPT misconfiguration, exit to ring 3 with KVM_EXIT_UNKNOWN. because EPT misconfiguration can't happen as MMIO is trigged by TDG.VP.VMCALL. No point to set a misconfiguration value for the fast path. Signed-off-by: Isaku Yamahata --- v14 -> v15: - use PFERR_GUEST_ENC_MASK to tell the fault is private Signed-off-by: Isaku Yamahata --- arch/x86/kvm/vmx/common.h | 3 +++ arch/x86/kvm/vmx/tdx.c | 49 +++++++++++++++++++++++++++++++++++++++ 2 files changed, 52 insertions(+) diff --git a/arch/x86/kvm/vmx/common.h b/arch/x86/kvm/vmx/common.h index 632af7a76d0a..027aa4175d2c 100644 --- a/arch/x86/kvm/vmx/common.h +++ b/arch/x86/kvm/vmx/common.h @@ -87,6 +87,9 @@ static inline int __vmx_handle_ept_violation(struct kvm_vcpu *vcpu, gpa_t gpa, error_code |= (exit_qualification & EPT_VIOLATION_GVA_TRANSLATED) != 0 ? PFERR_GUEST_FINAL_MASK : PFERR_GUEST_PAGE_MASK; + if (kvm_is_private_gpa(vcpu->kvm, gpa)) + error_code |= PFERR_GUEST_ENC_MASK; + return kvm_mmu_page_fault(vcpu, gpa, error_code, NULL, 0); } diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c index 2f68e6f2b53a..0db80fa020d2 100644 --- a/arch/x86/kvm/vmx/tdx.c +++ b/arch/x86/kvm/vmx/tdx.c @@ -1285,6 +1285,51 @@ void tdx_deliver_interrupt(struct kvm_lapic *apic, int delivery_mode, __vmx_deliver_posted_interrupt(vcpu, &tdx->pi_desc, vector); } +static int tdx_handle_ept_violation(struct kvm_vcpu *vcpu) +{ + unsigned long exit_qual; + + if (kvm_is_private_gpa(vcpu->kvm, tdexit_gpa(vcpu))) { + /* + * Always treat SEPT violations as write faults. Ignore the + * EXIT_QUALIFICATION reported by TDX-SEAM for SEPT violations. + * TD private pages are always RWX in the SEPT tables, + * i.e. they're always mapped writable. Just as importantly, + * treating SEPT violations as write faults is necessary to + * avoid COW allocations, which will cause TDAUGPAGE failures + * due to aliasing a single HPA to multiple GPAs. + */ +#define TDX_SEPT_VIOLATION_EXIT_QUAL EPT_VIOLATION_ACC_WRITE + exit_qual = TDX_SEPT_VIOLATION_EXIT_QUAL; + } else { + exit_qual = tdexit_exit_qual(vcpu); + if (exit_qual & EPT_VIOLATION_ACC_INSTR) { + pr_warn("kvm: TDX instr fetch to shared GPA = 0x%lx @ RIP = 0x%lx\n", + tdexit_gpa(vcpu), kvm_rip_read(vcpu)); + vcpu->run->exit_reason = KVM_EXIT_EXCEPTION; + vcpu->run->ex.exception = PF_VECTOR; + vcpu->run->ex.error_code = exit_qual; + return 0; + } + } + + trace_kvm_page_fault(vcpu, tdexit_gpa(vcpu), exit_qual); + return __vmx_handle_ept_violation(vcpu, tdexit_gpa(vcpu), exit_qual); +} + +static int tdx_handle_ept_misconfig(struct kvm_vcpu *vcpu) +{ + WARN_ON_ONCE(1); + + vcpu->run->exit_reason = KVM_EXIT_INTERNAL_ERROR; + vcpu->run->internal.suberror = KVM_INTERNAL_ERROR_UNEXPECTED_EXIT_REASON; + vcpu->run->internal.ndata = 2; + vcpu->run->internal.data[0] = EXIT_REASON_EPT_MISCONFIG; + vcpu->run->internal.data[1] = vcpu->arch.last_vmentry_cpu; + + return 0; +} + int tdx_handle_exit(struct kvm_vcpu *vcpu, fastpath_t fastpath) { union tdx_exit_reason exit_reason = to_tdx(vcpu)->exit_reason; @@ -1345,6 +1390,10 @@ int tdx_handle_exit(struct kvm_vcpu *vcpu, fastpath_t fastpath) WARN_ON_ONCE(fastpath != EXIT_FASTPATH_NONE); switch (exit_reason.basic) { + case EXIT_REASON_EPT_VIOLATION: + return tdx_handle_ept_violation(vcpu); + case EXIT_REASON_EPT_MISCONFIG: + return tdx_handle_ept_misconfig(vcpu); case EXIT_REASON_OTHER_SMI: /* * If reach here, it's not a Machine Check System Management From patchwork Mon Feb 26 08:26:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571530 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 11B507FBCB; Mon, 26 Feb 2024 08:29:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.19 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936141; cv=none; b=f2pD2Qza4MEuXg6AusBiQN2iH5j1aWFRxOULOyPRudX9nIE96r/F22KIB1fJzOZZYsYfgsZz285w1F0WLCfm/b5PIZYRWhQKeaCRo8by/O7DMpQIvsRVli47TqIlUZA77fEPa6eAeGEAgWlqwUtnaqlqnatQ2sS3QU954W0xkG4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936141; c=relaxed/simple; bh=L6TbWFTePbpuiyuGb3ssMzEB+jgZmBeUNscgSrYwAPg=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=MzmYhhYX43KK9LpdYziPhZ3uWTFMNHYDTFIsYBWYO0LSafQ4qcfcHDntMx8sBY8HKN6izhiUl7Ivd+KmZoTTbHEKdBTsNc2S0sTlEk4FGOn5rnwTouxNlKY2252bnFNFfeB3kzEVGbFtNU0++UQeE8VRzEkJpIJuURZ992jV9Hw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=AGD5uq6s; arc=none smtp.client-ip=198.175.65.19 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="AGD5uq6s" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936140; x=1740472140; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=L6TbWFTePbpuiyuGb3ssMzEB+jgZmBeUNscgSrYwAPg=; b=AGD5uq6s3jkYua/fiz56ZauCCmJ36NuXpz8pMHiCfER9+i2IEVJ9rv3o XMPAvMTjQxl7H5vIEKZrArFKIpb2Xk45RulSOnHX8j9I2My+Nxh3F8TGv FNvm0p8MCkhk1Di5LdYA3ZTo0F/ORSD47KE6VRGN4jsg9uInV5HdbUx0D rVb6iuKVojvV6JZkzFYo1WcsU9aPMNCCD7im3BYWSAHhHGpgERWtx/31A jjRDj5+2y1dOzHu7PFzz/KUXzO0Hbve+oTfpYt+Q/RMEcTkOIsD8ttER5 syrpvHtrs+4jx1uk04fF+uuGinur2DJomzNOvXSoarb6NirtRkiF5x0R1 w==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="3069604" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="3069604" Received: from orviesa004.jf.intel.com ([10.64.159.144]) by orvoesa111.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:29:00 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="11272692" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:28:59 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 102/130] KVM: TDX: handle EXCEPTION_NMI and EXTERNAL_INTERRUPT Date: Mon, 26 Feb 2024 00:26:44 -0800 Message-Id: <3ac413f1d4adbac7db88a2cade97ded3b076c540.1708933498.git.isaku.yamahata@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata Because guest TD state is protected, exceptions in guest TDs can't be intercepted. TDX VMM doesn't need to handle exceptions. tdx_handle_exit_irqoff() handles NMI and machine check. Ignore NMI and machine check and continue guest TD execution. For external interrupt, increment stats same to the VMX case. Signed-off-by: Isaku Yamahata Reviewed-by: Paolo Bonzini --- arch/x86/kvm/vmx/tdx.c | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c index 0db80fa020d2..bdd74682b474 100644 --- a/arch/x86/kvm/vmx/tdx.c +++ b/arch/x86/kvm/vmx/tdx.c @@ -918,6 +918,25 @@ void tdx_handle_exit_irqoff(struct kvm_vcpu *vcpu) vmx_handle_exception_irqoff(vcpu, tdexit_intr_info(vcpu)); } +static int tdx_handle_exception(struct kvm_vcpu *vcpu) +{ + u32 intr_info = tdexit_intr_info(vcpu); + + if (is_nmi(intr_info) || is_machine_check(intr_info)) + return 1; + + kvm_pr_unimpl("unexpected exception 0x%x(exit_reason 0x%llx qual 0x%lx)\n", + intr_info, + to_tdx(vcpu)->exit_reason.full, tdexit_exit_qual(vcpu)); + return -EFAULT; +} + +static int tdx_handle_external_interrupt(struct kvm_vcpu *vcpu) +{ + ++vcpu->stat.irq_exits; + return 1; +} + static int tdx_handle_triple_fault(struct kvm_vcpu *vcpu) { vcpu->run->exit_reason = KVM_EXIT_SHUTDOWN; @@ -1390,6 +1409,10 @@ int tdx_handle_exit(struct kvm_vcpu *vcpu, fastpath_t fastpath) WARN_ON_ONCE(fastpath != EXIT_FASTPATH_NONE); switch (exit_reason.basic) { + case EXIT_REASON_EXCEPTION_NMI: + return tdx_handle_exception(vcpu); + case EXIT_REASON_EXTERNAL_INTERRUPT: + return tdx_handle_external_interrupt(vcpu); case EXIT_REASON_EPT_VIOLATION: return tdx_handle_ept_violation(vcpu); case EXIT_REASON_EPT_MISCONFIG: From patchwork Mon Feb 26 08:26:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571537 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 79D827F7C4; Mon, 26 Feb 2024 08:29:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.19 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936143; cv=none; b=tWpyEOVSNLPLqTvqgPA89U9+8SlzmqqXCZMmkRc8sAaoTUDaDOBBLp4tvTq29PfhcU0kp7g3RiHBhjhR3wYIC4Q8K7IA+GwyOj8S06UTc+JToHqoXqZodc9XahDqvhjS7i0Hgzr5ZVkRLtHWfHH+owJEsAFRXfwhLtFUxg3vxZQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936143; c=relaxed/simple; bh=fS118DZC+NULTK6MvdkrJc4IgmIpKMeLepk4rQeITHQ=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=EqaWcEGgDa2VBGgyd3nOO8osbGXSxnY3s5zzsKvQHWBoQvAnZPqgIOAxD35bgloOVMpRNrk1ega0ZIg+TGG2mdBGmhbqQjm91aJ50RIxVpQCIQDr2lV3Kl1Rw0aBYV6MUVJNvo/fOnbUp8LtvMI5R9jWU7H7JnplogOhawLQ538= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=fail smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=aOfQTXDF; arc=none smtp.client-ip=198.175.65.19 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="aOfQTXDF" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936142; x=1740472142; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=fS118DZC+NULTK6MvdkrJc4IgmIpKMeLepk4rQeITHQ=; b=aOfQTXDFLwcxAcV6ikHjzHTTErTMnOlBJRm2o+6W44RlmXCc9ba2lcLc vHz2khciwe9TvAPv/u9A36odUWtUu3OyMsoqJnaiybO/D5cxxo00QeosY tVW0ByhUzpdjCBA4zn8V6ue5P8ycdZ4q+EEyLrbZNPo1pGzXo4gRY25Oo NUF+OQER7wVnMfYhNrECYul7arc2ntxiT4hDQGaUBCpBBvVHwjSBv9PQk jqjsefDZg07SNj9bBybxUUMcUtQjMzU5mMvx/s8NEDHvTS656IDFEakvD LF6y9ZijIJPZoRcIKZdJ5vIFl95hF1go505cAIs2Z0gQE7vBZpoOK5Unf g==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="3069613" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="3069613" Received: from orviesa004.jf.intel.com ([10.64.159.144]) by orvoesa111.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:29:02 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="11272700" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:29:01 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 103/130] KVM: TDX: Handle EXIT_REASON_OTHER_SMI with MSMI Date: Mon, 26 Feb 2024 00:26:45 -0800 Message-Id: <4a96a33c01b547f6e89ecf40224c80afa59c6aa4.1708933498.git.isaku.yamahata@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata When BIOS eMCA MCE-SMI morphing is enabled, the #MC is morphed to MSMI (Machine Check System Management Interrupt). Then the SMI causes TD exit with the read reason of EXIT_REASON_OTHER_SMI with MSMI bit set in the exit qualification to KVM instead of EXIT_REASON_EXCEPTION_NMI with MC exception. Handle EXIT_REASON_OTHER_SMI with MSMI bit set in the exit qualification as MCE(Machine Check Exception) happened during TD guest running. Signed-off-by: Isaku Yamahata --- arch/x86/kvm/vmx/tdx.c | 40 ++++++++++++++++++++++++++++++++++--- arch/x86/kvm/vmx/tdx_arch.h | 2 ++ 2 files changed, 39 insertions(+), 3 deletions(-) diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c index bdd74682b474..117c2315f087 100644 --- a/arch/x86/kvm/vmx/tdx.c +++ b/arch/x86/kvm/vmx/tdx.c @@ -916,6 +916,30 @@ void tdx_handle_exit_irqoff(struct kvm_vcpu *vcpu) tdexit_intr_info(vcpu)); else if (exit_reason == EXIT_REASON_EXCEPTION_NMI) vmx_handle_exception_irqoff(vcpu, tdexit_intr_info(vcpu)); + else if (unlikely(tdx->exit_reason.non_recoverable || + tdx->exit_reason.error)) { + /* + * The only reason it gets EXIT_REASON_OTHER_SMI is there is an + * #MSMI(Machine Check System Management Interrupt) with + * exit_qualification bit 0 set in TD guest. + * The #MSMI is delivered right after SEAMCALL returns, + * and an #MC is delivered to host kernel after SMI handler + * returns. + * + * The #MC right after SEAMCALL is fixed up and skipped in #MC + * handler because it's an #MC happens in TD guest we cannot + * handle it with host's context. + * + * Call KVM's machine check handler explicitly here. + */ + if (tdx->exit_reason.basic == EXIT_REASON_OTHER_SMI) { + unsigned long exit_qual; + + exit_qual = tdexit_exit_qual(vcpu); + if (exit_qual & TD_EXIT_OTHER_SMI_IS_MSMI) + kvm_machine_check(); + } + } } static int tdx_handle_exception(struct kvm_vcpu *vcpu) @@ -1381,6 +1405,11 @@ int tdx_handle_exit(struct kvm_vcpu *vcpu, fastpath_t fastpath) exit_reason.full, exit_reason.basic, to_kvm_tdx(vcpu->kvm)->hkid, set_hkid_to_hpa(0, to_kvm_tdx(vcpu->kvm)->hkid)); + + /* + * tdx_handle_exit_irqoff() handled EXIT_REASON_OTHER_SMI. It + * must be handled before enabling preemption because it's #MC. + */ goto unhandled_exit; } @@ -1419,9 +1448,14 @@ int tdx_handle_exit(struct kvm_vcpu *vcpu, fastpath_t fastpath) return tdx_handle_ept_misconfig(vcpu); case EXIT_REASON_OTHER_SMI: /* - * If reach here, it's not a Machine Check System Management - * Interrupt(MSMI). #SMI is delivered and handled right after - * SEAMRET, nothing needs to be done in KVM. + * Unlike VMX, all the SMI in SEAM non-root mode (i.e. when + * TD guest vcpu is running) will cause TD exit to TDX module, + * then SEAMRET to KVM. Once it exits to KVM, SMI is delivered + * and handled right away. + * + * - If it's an Machine Check System Management Interrupt + * (MSMI), it's handled above due to non_recoverable bit set. + * - If it's not an MSMI, don't need to do anything here. */ return 1; default: diff --git a/arch/x86/kvm/vmx/tdx_arch.h b/arch/x86/kvm/vmx/tdx_arch.h index efc3c61c14ab..87ef22e9cd49 100644 --- a/arch/x86/kvm/vmx/tdx_arch.h +++ b/arch/x86/kvm/vmx/tdx_arch.h @@ -42,6 +42,8 @@ #define TDH_VP_WR 43 #define TDH_SYS_LP_SHUTDOWN 44 +#define TD_EXIT_OTHER_SMI_IS_MSMI BIT(1) + /* TDX control structure (TDR/TDCS/TDVPS) field access codes */ #define TDX_NON_ARCH BIT_ULL(63) #define TDX_CLASS_SHIFT 56 From patchwork Mon Feb 26 08:26:46 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571538 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C101B86621; Mon, 26 Feb 2024 08:29:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.19 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936144; cv=none; b=hMiGqg/kPUs5F/mJxY44pol+5o1zD1R9TW/eG6uwufyboMb03yBcSstia9qmIYVqOmhMTNC4tWUryuGNDYWOSj+fX+ZA9aCx0BONdczyOPGFDzoVtx/yQxahpZISFvibyCuM92hbxCuZQDNTBCNvH2iB2iDo3sj63RRO8v9qBOQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936144; c=relaxed/simple; bh=1PDQelSePPzX2B7lYc0kxBcavk4BVnF4BPwDcuzpxVE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=b9BQrlnnnUugA3cvt5DMt0DZjZW1UkmBipnrRXqM3HH6Ak7Jq7WINX8c+TKyQzNz5FwXVELW3UgFzFCiKx0oUTMHFaR02aeWhBvNMzG9CXjjIkqkR2hlfVlQuBsvnuBlRFdBy4fTRikmafu1l8mWDY7nM/csnjdLi7BFaS/9SQw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=lhYBnZf7; arc=none smtp.client-ip=198.175.65.19 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="lhYBnZf7" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936143; x=1740472143; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=1PDQelSePPzX2B7lYc0kxBcavk4BVnF4BPwDcuzpxVE=; b=lhYBnZf7XmEy3uB7la/hwWMOh+JMNfwgqwdu1v+NvjTeTiC8sjh8koE9 3lDVsa172aCyqdvWQ2LfsKpUsOtQIZKb+LBdwNSbuKTZ462DoIFXv3OWc GAJf+KX+Hr/cDr8UqjxxywFjQHYnleNpEHJnwTtY3m7ZM9EDh66lSfCrW q7m26SmPxjEaqrpoAcoVpgUXtRTjMjP1+y1L2mG06kEuHIcYfM3w/DttX OHW7qBNbj7kPr7CmodX3bySzJu8OqZH4+tOT2QLy6T7qbgaB6zYKDuzYl JF+1XY+5ZuqYY8SqRKme9012bAeMfiGoPoannjcKGbGpm65WUvvEKy2VB w==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="3069615" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="3069615" Received: from orviesa004.jf.intel.com ([10.64.159.144]) by orvoesa111.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:29:03 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="11272704" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by orviesa004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:29:02 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com, Xiaoyao Li , Sean Christopherson Subject: [PATCH v19 104/130] KVM: TDX: Add a place holder for handler of TDX hypercalls (TDG.VP.VMCALL) Date: Mon, 26 Feb 2024 00:26:46 -0800 Message-Id: <1c66bfde36f08eacbe2f5c50f88adf80e3d87ea7.1708933498.git.isaku.yamahata@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata The TDX module specification defines TDG.VP.VMCALL API (TDVMCALL for short) for the guest TD to call hypercall to VMM. When the guest TD issues TDG.VP.VMCALL, the guest TD exits to VMM with a new exit reason of TDVMCALL. The arguments from the guest TD and returned values from the VMM are passed in the guest registers. The guest RCX registers indicates which registers are used. Define helper functions to access those registers as ABI. Define the TDVMCALL exit reason, which is carved out from the VMX exit reason namespace as the TDVMCALL exit from TDX guest to TDX-SEAM is really just a VM-Exit. Add a place holder to handle TDVMCALL exit. Co-developed-by: Xiaoyao Li Signed-off-by: Xiaoyao Li Signed-off-by: Sean Christopherson Signed-off-by: Isaku Yamahata --- arch/x86/include/uapi/asm/vmx.h | 4 ++- arch/x86/kvm/vmx/tdx.c | 53 +++++++++++++++++++++++++++++++++ arch/x86/kvm/vmx/tdx.h | 13 ++++++++ 3 files changed, 69 insertions(+), 1 deletion(-) diff --git a/arch/x86/include/uapi/asm/vmx.h b/arch/x86/include/uapi/asm/vmx.h index b3a30ef3efdd..f0f4a4cf84a7 100644 --- a/arch/x86/include/uapi/asm/vmx.h +++ b/arch/x86/include/uapi/asm/vmx.h @@ -93,6 +93,7 @@ #define EXIT_REASON_TPAUSE 68 #define EXIT_REASON_BUS_LOCK 74 #define EXIT_REASON_NOTIFY 75 +#define EXIT_REASON_TDCALL 77 #define VMX_EXIT_REASONS \ { EXIT_REASON_EXCEPTION_NMI, "EXCEPTION_NMI" }, \ @@ -156,7 +157,8 @@ { EXIT_REASON_UMWAIT, "UMWAIT" }, \ { EXIT_REASON_TPAUSE, "TPAUSE" }, \ { EXIT_REASON_BUS_LOCK, "BUS_LOCK" }, \ - { EXIT_REASON_NOTIFY, "NOTIFY" } + { EXIT_REASON_NOTIFY, "NOTIFY" }, \ + { EXIT_REASON_TDCALL, "TDCALL" } #define VMX_EXIT_REASON_FLAGS \ { VMX_EXIT_REASONS_FAILED_VMENTRY, "FAILED_VMENTRY" } diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c index 117c2315f087..0be58cd428b3 100644 --- a/arch/x86/kvm/vmx/tdx.c +++ b/arch/x86/kvm/vmx/tdx.c @@ -140,6 +140,41 @@ static __always_inline unsigned long tdexit_intr_info(struct kvm_vcpu *vcpu) return kvm_r9_read(vcpu); } +#define BUILD_TDVMCALL_ACCESSORS(param, gpr) \ +static __always_inline \ +unsigned long tdvmcall_##param##_read(struct kvm_vcpu *vcpu) \ +{ \ + return kvm_##gpr##_read(vcpu); \ +} \ +static __always_inline void tdvmcall_##param##_write(struct kvm_vcpu *vcpu, \ + unsigned long val) \ +{ \ + kvm_##gpr##_write(vcpu, val); \ +} +BUILD_TDVMCALL_ACCESSORS(a0, r12); +BUILD_TDVMCALL_ACCESSORS(a1, r13); +BUILD_TDVMCALL_ACCESSORS(a2, r14); +BUILD_TDVMCALL_ACCESSORS(a3, r15); + +static __always_inline unsigned long tdvmcall_exit_type(struct kvm_vcpu *vcpu) +{ + return kvm_r10_read(vcpu); +} +static __always_inline unsigned long tdvmcall_leaf(struct kvm_vcpu *vcpu) +{ + return kvm_r11_read(vcpu); +} +static __always_inline void tdvmcall_set_return_code(struct kvm_vcpu *vcpu, + long val) +{ + kvm_r10_write(vcpu, val); +} +static __always_inline void tdvmcall_set_return_val(struct kvm_vcpu *vcpu, + unsigned long val) +{ + kvm_r11_write(vcpu, val); +} + static inline bool is_td_vcpu_created(struct vcpu_tdx *tdx) { return tdx->td_vcpu_created; @@ -897,6 +932,11 @@ fastpath_t tdx_vcpu_run(struct kvm_vcpu *vcpu) tdx_complete_interrupts(vcpu); + if (tdx->exit_reason.basic == EXIT_REASON_TDCALL) + tdx->tdvmcall.rcx = vcpu->arch.regs[VCPU_REGS_RCX]; + else + tdx->tdvmcall.rcx = 0; + return EXIT_FASTPATH_NONE; } @@ -968,6 +1008,17 @@ static int tdx_handle_triple_fault(struct kvm_vcpu *vcpu) return 0; } +static int handle_tdvmcall(struct kvm_vcpu *vcpu) +{ + switch (tdvmcall_leaf(vcpu)) { + default: + break; + } + + tdvmcall_set_return_code(vcpu, TDVMCALL_INVALID_OPERAND); + return 1; +} + void tdx_load_mmu_pgd(struct kvm_vcpu *vcpu, hpa_t root_hpa, int pgd_level) { WARN_ON_ONCE(root_hpa & ~PAGE_MASK); @@ -1442,6 +1493,8 @@ int tdx_handle_exit(struct kvm_vcpu *vcpu, fastpath_t fastpath) return tdx_handle_exception(vcpu); case EXIT_REASON_EXTERNAL_INTERRUPT: return tdx_handle_external_interrupt(vcpu); + case EXIT_REASON_TDCALL: + return handle_tdvmcall(vcpu); case EXIT_REASON_EPT_VIOLATION: return tdx_handle_ept_violation(vcpu); case EXIT_REASON_EPT_MISCONFIG: diff --git a/arch/x86/kvm/vmx/tdx.h b/arch/x86/kvm/vmx/tdx.h index eaffa7384725..4399d474764f 100644 --- a/arch/x86/kvm/vmx/tdx.h +++ b/arch/x86/kvm/vmx/tdx.h @@ -83,6 +83,19 @@ struct vcpu_tdx { struct list_head cpu_list; + union { + struct { + union { + struct { + u16 gpr_mask; + u16 xmm_mask; + }; + u32 regs_mask; + }; + u32 reserved; + }; + u64 rcx; + } tdvmcall; union tdx_exit_reason exit_reason; bool initialized; From patchwork Mon Feb 26 08:26:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571539 Received: from mgamail.intel.com (unknown [192.198.163.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D69681272C0; Mon, 26 Feb 2024 08:29:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.8 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936145; cv=none; b=XYDRQTr7w1g0qxXrykQS0H6tNkxtjsVKq6Czr79XNMkEbNo0pbKTICzpym/olO2AiKtbzn4tpv6QMQedd3UDaj+bL6lZaAbfhcclLe2JAP3mEkR9s/WBjWwN/QWmTBH2axJ2/AoFi1c1OTO8IAhc9kg15NA/tNJ6dPYZop27Um0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936145; c=relaxed/simple; bh=b3kkpZaI6SioFJ/UllRObAK3OnMZFeZHEri4NH/LxV8=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=BJx/7ZdTfbByRpLXULA7I9dePyXNnXufahE7sPE6VZDMWnIFKJpB9w0QMJeKE5mu3WS/cLpr9+wEAb8PsBpXLFUeJWYNZO71MFcPgk3DBMRwNwwRfSFOPaNl37cSBWaFqtTW9fqtrPUJiKS/XvofzMqUUiSMG3d68eQRrUUM1po= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=CGFGMhNd; arc=none smtp.client-ip=192.198.163.8 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="CGFGMhNd" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936144; x=1740472144; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=b3kkpZaI6SioFJ/UllRObAK3OnMZFeZHEri4NH/LxV8=; b=CGFGMhNdhkR9ceqe2onv0Kq4lcMhvUZUOWwucl0+vR1Cyxlr1lUm4nTq 2FpGfP0wwcL/EeqPX3ayi5zO5NsuLzeH5+fAjuN9eJ8MkunCOG1DqmNKM /1rJ1lpZw/pA7vcbPyuFKM7BcjJr0w4TNYvqAdDyb5BzeotP2QGzRhev3 Sf+3fevNrIaEJD97xpKDHIfKFSyMOeyYDMrcfiuoOiyi8zlLZeyxHU0/Q 536J3b//aVRyfsmwUljS/mY55vt6tDCd3v5whjXWJNyPK+hHztVx4hywH e4yAJiRVyh9tw5q2zVvNwzaH7PkoIFECnSDwioMpILOT60f9Ew8x/zkVq A==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="20751314" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="20751314" Received: from fmviesa008.fm.intel.com ([10.60.135.148]) by fmvoesa102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:29:03 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6735055" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by fmviesa008-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:29:03 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 105/130] KVM: TDX: handle KVM hypercall with TDG.VP.VMCALL Date: Mon, 26 Feb 2024 00:26:47 -0800 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata The TDX Guest-Host communication interface (GHCI) specification defines the ABI for the guest TD to issue hypercall. It reserves vendor specific arguments for VMM specific use. Use it as KVM hypercall and handle it. Signed-off-by: Isaku Yamahata --- arch/x86/kvm/vmx/tdx.c | 33 +++++++++++++++++++++++++++++++++ 1 file changed, 33 insertions(+) diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c index 0be58cd428b3..c8eb47591105 100644 --- a/arch/x86/kvm/vmx/tdx.c +++ b/arch/x86/kvm/vmx/tdx.c @@ -1008,8 +1008,41 @@ static int tdx_handle_triple_fault(struct kvm_vcpu *vcpu) return 0; } +static int tdx_emulate_vmcall(struct kvm_vcpu *vcpu) +{ + unsigned long nr, a0, a1, a2, a3, ret; + + /* + * ABI for KVM tdvmcall argument: + * In Guest-Hypervisor Communication Interface(GHCI) specification, + * Non-zero leaf number (R10 != 0) is defined to indicate + * vendor-specific. KVM uses this for KVM hypercall. NOTE: KVM + * hypercall number starts from one. Zero isn't used for KVM hypercall + * number. + * + * R10: KVM hypercall number + * arguments: R11, R12, R13, R14. + */ + nr = kvm_r10_read(vcpu); + a0 = kvm_r11_read(vcpu); + a1 = kvm_r12_read(vcpu); + a2 = kvm_r13_read(vcpu); + a3 = kvm_r14_read(vcpu); + + ret = __kvm_emulate_hypercall(vcpu, nr, a0, a1, a2, a3, true, 0); + + tdvmcall_set_return_code(vcpu, ret); + + if (nr == KVM_HC_MAP_GPA_RANGE && !ret) + return 0; + return 1; +} + static int handle_tdvmcall(struct kvm_vcpu *vcpu) { + if (tdvmcall_exit_type(vcpu)) + return tdx_emulate_vmcall(vcpu); + switch (tdvmcall_leaf(vcpu)) { default: break; From patchwork Mon Feb 26 08:26:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571540 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0A7E8128379; Mon, 26 Feb 2024 08:29:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.8 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936146; cv=none; b=qKaAqZGzCD3KAJYSJ+v6n02nMmnHYkCMaTKD7yzEQET6PW/KOeko+wJctoJ6fuOHdvOhBsUbNfZLjiseex7hr8KyL0hzDhiaLkKKzqs7IBSaizs1JMiB5jt130qlwDIlPlxhIN+X+MYkKQ4PADs5LsMZ+Q99iCJuPOqAqhoK6yI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936146; c=relaxed/simple; bh=gMBjeQic0/mGScZZUcKmn0/6xULcS4yuQddjIJvT7Rg=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=fR9rryjhrsR/e3JjBm0los0mKTmgID8CH8eagw5WODsQr/5pOj1RHwkMb5zxze+VyanJZFbrRD7YHJo2XYeudcJskojuyfXcG4ATVdO9tWfVOakxGnNxbG5uExrUSd/4k7gdjm1Q01wMp8cjKjpEVl6YlOyb6OikbIWRJYkMrMc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=KwChySHP; arc=none smtp.client-ip=192.198.163.8 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="KwChySHP" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936145; x=1740472145; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=gMBjeQic0/mGScZZUcKmn0/6xULcS4yuQddjIJvT7Rg=; b=KwChySHPgguaOQDp9/L5zv/yIz+LM1IA9IyGa1QlXXPqHmPKc8UDHXFO jRgJhezDafNDdn/KQpioJ9BUpwu1JvIcWC9GO/HXXLQ778thdd3s59PD8 PFKi3ZW/zKpNYmjr2EWJ6t1FEU3ef0Tl4/CPUVIG6hZVVNOGIhYvMgjLr /Fy+S1euZXHXNgtlQnN0TgrcSinG7cVvWvb5avNjnBjkR1UsnF7M4acz9 HkTLJrig7LvHCbHq4IKKUDovzeVcuVuEIelO/RzeMG1YxXHaS2P4II3o/ xPDt45I/nksGbJM2wHCm5h54mV8t3ZGJnqwyNKBtvXjvUv3kKoqeZQc7q w==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="20751320" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="20751320" Received: from fmviesa008.fm.intel.com ([10.60.135.148]) by fmvoesa102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:29:03 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6735058" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by fmviesa008-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:29:03 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 106/130] KVM: TDX: Add KVM Exit for TDX TDG.VP.VMCALL Date: Mon, 26 Feb 2024 00:26:48 -0800 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata Some of TDG.VP.VMCALL require device model, for example, qemu, to handle them on behalf of kvm kernel module. TDVMCALL_REPORT_FATAL_ERROR, TDVMCALL_MAP_GPA, TDVMCALL_SETUP_EVENT_NOTIFY_INTERRUPT, and TDVMCALL_GET_QUOTE requires user space VMM handling. Introduce new kvm exit, KVM_EXIT_TDX, and functions to setup it. Device model should update R10 if necessary as return value. Signed-off-by: Isaku Yamahata --- v14 -> v15: - updated struct kvm_tdx_exit with union - export constants for reg bitmask Signed-off-by: Isaku Yamahata --- arch/x86/kvm/vmx/tdx.c | 83 ++++++++++++++++++++++++++++++++++++- include/uapi/linux/kvm.h | 89 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 170 insertions(+), 2 deletions(-) diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c index c8eb47591105..72dbe2ff9062 100644 --- a/arch/x86/kvm/vmx/tdx.c +++ b/arch/x86/kvm/vmx/tdx.c @@ -1038,6 +1038,78 @@ static int tdx_emulate_vmcall(struct kvm_vcpu *vcpu) return 1; } +static int tdx_complete_vp_vmcall(struct kvm_vcpu *vcpu) +{ + struct kvm_tdx_vmcall *tdx_vmcall = &vcpu->run->tdx.u.vmcall; + __u64 reg_mask = kvm_rcx_read(vcpu); + +#define COPY_REG(MASK, REG) \ + do { \ + if (reg_mask & TDX_VMCALL_REG_MASK_ ## MASK) \ + kvm_## REG ## _write(vcpu, tdx_vmcall->out_ ## REG); \ + } while (0) + + + COPY_REG(R10, r10); + COPY_REG(R11, r11); + COPY_REG(R12, r12); + COPY_REG(R13, r13); + COPY_REG(R14, r14); + COPY_REG(R15, r15); + COPY_REG(RBX, rbx); + COPY_REG(RDI, rdi); + COPY_REG(RSI, rsi); + COPY_REG(R8, r8); + COPY_REG(R9, r9); + COPY_REG(RDX, rdx); + +#undef COPY_REG + + return 1; +} + +static int tdx_vp_vmcall_to_user(struct kvm_vcpu *vcpu) +{ + struct kvm_tdx_vmcall *tdx_vmcall = &vcpu->run->tdx.u.vmcall; + __u64 reg_mask; + + vcpu->arch.complete_userspace_io = tdx_complete_vp_vmcall; + memset(tdx_vmcall, 0, sizeof(*tdx_vmcall)); + + vcpu->run->exit_reason = KVM_EXIT_TDX; + vcpu->run->tdx.type = KVM_EXIT_TDX_VMCALL; + + reg_mask = kvm_rcx_read(vcpu); + tdx_vmcall->reg_mask = reg_mask; + +#define COPY_REG(MASK, REG) \ + do { \ + if (reg_mask & TDX_VMCALL_REG_MASK_ ## MASK) { \ + tdx_vmcall->in_ ## REG = kvm_ ## REG ## _read(vcpu); \ + tdx_vmcall->out_ ## REG = tdx_vmcall->in_ ## REG; \ + } \ + } while (0) + + + COPY_REG(R10, r10); + COPY_REG(R11, r11); + COPY_REG(R12, r12); + COPY_REG(R13, r13); + COPY_REG(R14, r14); + COPY_REG(R15, r15); + COPY_REG(RBX, rbx); + COPY_REG(RDI, rdi); + COPY_REG(RSI, rsi); + COPY_REG(R8, r8); + COPY_REG(R9, r9); + COPY_REG(RDX, rdx); + +#undef COPY_REG + + /* notify userspace to handle the request */ + return 0; +} + static int handle_tdvmcall(struct kvm_vcpu *vcpu) { if (tdvmcall_exit_type(vcpu)) @@ -1048,8 +1120,15 @@ static int handle_tdvmcall(struct kvm_vcpu *vcpu) break; } - tdvmcall_set_return_code(vcpu, TDVMCALL_INVALID_OPERAND); - return 1; + /* + * Unknown VMCALL. Toss the request to the user space VMM, e.g. qemu, + * as it may know how to handle. + * + * Those VMCALLs require user space VMM: + * TDVMCALL_REPORT_FATAL_ERROR, TDVMCALL_MAP_GPA, + * TDVMCALL_SETUP_EVENT_NOTIFY_INTERRUPT, and TDVMCALL_GET_QUOTE. + */ + return tdx_vp_vmcall_to_user(vcpu); } void tdx_load_mmu_pgd(struct kvm_vcpu *vcpu, hpa_t root_hpa, int pgd_level) diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h index 5e2b28934aa9..a7aa804ef021 100644 --- a/include/uapi/linux/kvm.h +++ b/include/uapi/linux/kvm.h @@ -167,6 +167,92 @@ struct kvm_xen_exit { } u; }; +/* masks for reg_mask to indicate which registers are passed. */ +#define TDX_VMCALL_REG_MASK_RBX BIT_ULL(2) +#define TDX_VMCALL_REG_MASK_RDX BIT_ULL(3) +#define TDX_VMCALL_REG_MASK_RSI BIT_ULL(6) +#define TDX_VMCALL_REG_MASK_RDI BIT_ULL(7) +#define TDX_VMCALL_REG_MASK_R8 BIT_ULL(8) +#define TDX_VMCALL_REG_MASK_R9 BIT_ULL(9) +#define TDX_VMCALL_REG_MASK_R10 BIT_ULL(10) +#define TDX_VMCALL_REG_MASK_R11 BIT_ULL(11) +#define TDX_VMCALL_REG_MASK_R12 BIT_ULL(12) +#define TDX_VMCALL_REG_MASK_R13 BIT_ULL(13) +#define TDX_VMCALL_REG_MASK_R14 BIT_ULL(14) +#define TDX_VMCALL_REG_MASK_R15 BIT_ULL(15) + +struct kvm_tdx_exit { +#define KVM_EXIT_TDX_VMCALL 1 + __u32 type; + __u32 pad; + + union { + struct kvm_tdx_vmcall { + /* + * RAX(bit 0), RCX(bit 1) and RSP(bit 4) are reserved. + * RAX(bit 0): TDG.VP.VMCALL status code. + * RCX(bit 1): bitmap for used registers. + * RSP(bit 4): the caller stack. + */ + union { + __u64 in_rcx; + __u64 reg_mask; + }; + + /* + * Guest-Host-Communication Interface for TDX spec + * defines the ABI for TDG.VP.VMCALL. + */ + /* Input parameters: guest -> VMM */ + union { + __u64 in_r10; + __u64 type; + }; + union { + __u64 in_r11; + __u64 subfunction; + }; + /* + * Subfunction specific. + * Registers are used in this order to pass input + * arguments. r12=arg0, r13=arg1, etc. + */ + __u64 in_r12; + __u64 in_r13; + __u64 in_r14; + __u64 in_r15; + __u64 in_rbx; + __u64 in_rdi; + __u64 in_rsi; + __u64 in_r8; + __u64 in_r9; + __u64 in_rdx; + + /* Output parameters: VMM -> guest */ + union { + __u64 out_r10; + __u64 status_code; + }; + /* + * Subfunction specific. + * Registers are used in this order to output return + * values. r11=ret0, r12=ret1, etc. + */ + __u64 out_r11; + __u64 out_r12; + __u64 out_r13; + __u64 out_r14; + __u64 out_r15; + __u64 out_rbx; + __u64 out_rdi; + __u64 out_rsi; + __u64 out_r8; + __u64 out_r9; + __u64 out_rdx; + } vmcall; + } u; +}; + #define KVM_S390_GET_SKEYS_NONE 1 #define KVM_S390_SKEYS_MAX 1048576 @@ -210,6 +296,7 @@ struct kvm_xen_exit { #define KVM_EXIT_NOTIFY 37 #define KVM_EXIT_LOONGARCH_IOCSR 38 #define KVM_EXIT_MEMORY_FAULT 39 +#define KVM_EXIT_TDX 40 /* For KVM_EXIT_INTERNAL_ERROR */ /* Emulate instruction failed. */ @@ -470,6 +557,8 @@ struct kvm_run { __u64 gpa; __u64 size; } memory_fault; + /* KVM_EXIT_TDX_VMCALL */ + struct kvm_tdx_exit tdx; /* Fix the size of the union. */ char padding[256]; }; From patchwork Mon Feb 26 08:26:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571541 Received: from mgamail.intel.com (unknown [192.198.163.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8BBAF12881C; Mon, 26 Feb 2024 08:29:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.8 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936147; cv=none; b=YVq+1NArk+W2NPXvUXRrZUZqJnJQiX0Ye4JH+jGIUELXTAFEIrTR12B4DuYlsNohJF4gKIwguTSK5txOYoN/KpyzzEBTjo1khYgb3pX3leD4dRsLN/szPz1x+uRzlFRY/Oa4PCZnpAk8pHjklSombgelG1p8Ms+TpufSdE3vilk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936147; c=relaxed/simple; bh=nUfUpWtI/c8UEGyRqPpk6qguNHB+5fFp6Y3yMKH1okk=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=F6EoAFXmH5fy4jbTagrc0bxzFzDAcQqwaQA7QOeg9RNl3q2iPYmvD1sBEurXzxRkwKfSUNQDo0JYtGhl4zC48J/531e9taQn6XewDH2YvO5QrjEiEbJBC7iACsAVxSw7310Lz2Evd/0ES6C3nIj02i3RxcqCaH8Xbuc+iwwtYNQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=AI8GxbSH; arc=none smtp.client-ip=192.198.163.8 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="AI8GxbSH" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936145; x=1740472145; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=nUfUpWtI/c8UEGyRqPpk6qguNHB+5fFp6Y3yMKH1okk=; b=AI8GxbSHU0Ob6+AyGr6nlBldBTB6/PVF6J7fwga9K48YMyZ0vjbHk+Po gufAUH6aYQoF2q4yeqg0mvnBzOAyRHIDmdJ9gwWnLdYwMgsTXh4VMMx8t 9AjlTijLTtk1qeRfPr2VF7Lg1wLegyBMNgwpV10y3Dbk3+tG49v/FuCf7 iU50/uTUSYLo3TsNrFTRpFZxpt96vlgN4YtYucdX4ol/eQc3B8xcGU9Mc lsc68Zi5Z1oANkfB/FG+kTuAkHAxSWEjHee9PQuZmbLmGuDOJKcUvpYZ/ WlyV9flYytD6/hQ44aEKwq6IqhogVfodnv5B1ZHp74aJlCZHch0CcD7OZ A==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="20751324" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="20751324" Received: from fmviesa008.fm.intel.com ([10.60.135.148]) by fmvoesa102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:29:04 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6735062" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by fmviesa008-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:29:03 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 107/130] KVM: TDX: Handle TDX PV CPUID hypercall Date: Mon, 26 Feb 2024 00:26:49 -0800 Message-Id: <1999b2b095612e59746710e0a37f4caa466ee37f.1708933498.git.isaku.yamahata@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata Wire up TDX PV CPUID hypercall to the KVM backend function. Signed-off-by: Isaku Yamahata --- arch/x86/kvm/vmx/tdx.c | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c index 72dbe2ff9062..eb68d6c148b6 100644 --- a/arch/x86/kvm/vmx/tdx.c +++ b/arch/x86/kvm/vmx/tdx.c @@ -1110,12 +1110,34 @@ static int tdx_vp_vmcall_to_user(struct kvm_vcpu *vcpu) return 0; } +static int tdx_emulate_cpuid(struct kvm_vcpu *vcpu) +{ + u32 eax, ebx, ecx, edx; + + /* EAX and ECX for cpuid is stored in R12 and R13. */ + eax = tdvmcall_a0_read(vcpu); + ecx = tdvmcall_a1_read(vcpu); + + kvm_cpuid(vcpu, &eax, &ebx, &ecx, &edx, false); + + tdvmcall_a0_write(vcpu, eax); + tdvmcall_a1_write(vcpu, ebx); + tdvmcall_a2_write(vcpu, ecx); + tdvmcall_a3_write(vcpu, edx); + + tdvmcall_set_return_code(vcpu, TDVMCALL_SUCCESS); + + return 1; +} + static int handle_tdvmcall(struct kvm_vcpu *vcpu) { if (tdvmcall_exit_type(vcpu)) return tdx_emulate_vmcall(vcpu); switch (tdvmcall_leaf(vcpu)) { + case EXIT_REASON_CPUID: + return tdx_emulate_cpuid(vcpu); default: break; } From patchwork Mon Feb 26 08:26:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571542 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5602D1292DA; Mon, 26 Feb 2024 08:29:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.8 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936148; cv=none; b=a/YsmOn8nEzs+eSNflavLkI+eSSY9Tr8VUXGntD72ejrJc3eDI2cl1v+mg0QuiFY/Pdeh7OFPUvMxDmFuAAmpOOlaX1snbLYiXBr1mv9MaWizXjQwEGo5o/W6Vyx9WFQYxtMVrMDSzRtdDnW6PBIX8rLC/WTXAfhE4ATwUwO6Tc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936148; c=relaxed/simple; bh=iLKBzkaZ2x4CHI/IAJCoPiQoXu/6vbCP7L7Yic/bYOw=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=n82SATvrcv0RvVPDScS/m0ZRxVswKKmNMATSiBHiajF7ryMdKM7n1AR2L7hR7AGX6izPK6p6aQdv4dbjvAFcfumpIHbgjHtJ6if+97ngaR9POCfCQahpMBKq5amMlZAE2W/HN4eISi0S5NFz0Ryj/1HV6dv6P//rrroB3U52T7Q= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=U1Cz3jTU; arc=none smtp.client-ip=192.198.163.8 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="U1Cz3jTU" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936146; x=1740472146; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=iLKBzkaZ2x4CHI/IAJCoPiQoXu/6vbCP7L7Yic/bYOw=; b=U1Cz3jTUjgnNuhg1yJYijnV4hUVmUyhhU3r88mvz10roSxnMQ3TD9lMk Q1vR96ZJ79Yg6U7OUnOxzisCxp3cyToggRDufOlZcD8PSVgabXcr6kU9p E6Y1XUEpuVmu/ExXwbLEyU/UlfNbsXEO63WVYe1brBYZ122ZwfbtVkYkI zrZh2KeI7DuF+goB59WEoGIFh1X8JqDN/g4KrfhJ/wTno2Gyr9EHBYB4V TvQ6Jk8ys3ulFKosAUMyCyphQzjNiWbbrj0jJ4Hg9J37HZMdIn/2/kcJ4 rsFZMVbaqU7KT11bjImXCFIvuL+FH0RWsARL3NOcnpik7UtW79D1fjI7i A==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="20751330" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="20751330" Received: from fmviesa008.fm.intel.com ([10.60.135.148]) by fmvoesa102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:29:04 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6735065" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by fmviesa008-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:29:04 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 108/130] KVM: TDX: Handle TDX PV HLT hypercall Date: Mon, 26 Feb 2024 00:26:50 -0800 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata Wire up TDX PV HLT hypercall to the KVM backend function. Signed-off-by: Isaku Yamahata --- v19: - move tdvps_state_non_arch_check() to this patch v18: - drop buggy_hlt_workaround and use TDH.VP.RD(TD_VCPU_STATE_DETAILS) Signed-off-by: Isaku Yamahata --- arch/x86/kvm/vmx/tdx.c | 26 +++++++++++++++++++++++++- arch/x86/kvm/vmx/tdx.h | 4 ++++ 2 files changed, 29 insertions(+), 1 deletion(-) diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c index eb68d6c148b6..a2caf2ae838c 100644 --- a/arch/x86/kvm/vmx/tdx.c +++ b/arch/x86/kvm/vmx/tdx.c @@ -688,7 +688,18 @@ void tdx_vcpu_load(struct kvm_vcpu *vcpu, int cpu) bool tdx_protected_apic_has_interrupt(struct kvm_vcpu *vcpu) { - return pi_has_pending_interrupt(vcpu); + bool ret = pi_has_pending_interrupt(vcpu); + union tdx_vcpu_state_details details; + struct vcpu_tdx *tdx = to_tdx(vcpu); + + if (ret || vcpu->arch.mp_state != KVM_MP_STATE_HALTED) + return true; + + if (tdx->interrupt_disabled_hlt) + return false; + + details.full = td_state_non_arch_read64(tdx, TD_VCPU_STATE_DETAILS_NON_ARCH); + return !!details.vmxip; } void tdx_prepare_switch_to_guest(struct kvm_vcpu *vcpu) @@ -1130,6 +1141,17 @@ static int tdx_emulate_cpuid(struct kvm_vcpu *vcpu) return 1; } +static int tdx_emulate_hlt(struct kvm_vcpu *vcpu) +{ + struct vcpu_tdx *tdx = to_tdx(vcpu); + + /* See tdx_protected_apic_has_interrupt() to avoid heavy seamcall */ + tdx->interrupt_disabled_hlt = tdvmcall_a0_read(vcpu); + + tdvmcall_set_return_code(vcpu, TDVMCALL_SUCCESS); + return kvm_emulate_halt_noskip(vcpu); +} + static int handle_tdvmcall(struct kvm_vcpu *vcpu) { if (tdvmcall_exit_type(vcpu)) @@ -1138,6 +1160,8 @@ static int handle_tdvmcall(struct kvm_vcpu *vcpu) switch (tdvmcall_leaf(vcpu)) { case EXIT_REASON_CPUID: return tdx_emulate_cpuid(vcpu); + case EXIT_REASON_HLT: + return tdx_emulate_hlt(vcpu); default: break; } diff --git a/arch/x86/kvm/vmx/tdx.h b/arch/x86/kvm/vmx/tdx.h index 4399d474764f..11c74c34555f 100644 --- a/arch/x86/kvm/vmx/tdx.h +++ b/arch/x86/kvm/vmx/tdx.h @@ -104,6 +104,8 @@ struct vcpu_tdx { bool host_state_need_restore; u64 msr_host_kernel_gs_base; + bool interrupt_disabled_hlt; + /* * Dummy to make pmu_intel not corrupt memory. * TODO: Support PMU for TDX. Future work. @@ -166,6 +168,7 @@ static __always_inline void tdvps_vmcs_check(u32 field, u8 bits) } static __always_inline void tdvps_management_check(u64 field, u8 bits) {} +static __always_inline void tdvps_state_non_arch_check(u64 field, u8 bits) {} #define TDX_BUILD_TDVPS_ACCESSORS(bits, uclass, lclass) \ static __always_inline u##bits td_##lclass##_read##bits(struct vcpu_tdx *tdx, \ @@ -226,6 +229,7 @@ TDX_BUILD_TDVPS_ACCESSORS(32, VMCS, vmcs); TDX_BUILD_TDVPS_ACCESSORS(64, VMCS, vmcs); TDX_BUILD_TDVPS_ACCESSORS(8, MANAGEMENT, management); +TDX_BUILD_TDVPS_ACCESSORS(64, STATE_NON_ARCH, state_non_arch); static __always_inline u64 td_tdcs_exec_read64(struct kvm_tdx *kvm_tdx, u32 field) { From patchwork Mon Feb 26 08:26:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571543 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EDB27129A61; Mon, 26 Feb 2024 08:29:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.8 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936148; cv=none; b=YEx3xDhYEkSWPEq1jPA1TU0W4lF3Uu0FFC3u+l9z7B1qEYMtfvZUmK/lfb13J8Sks9xwmgaFBGWUTTsJQ2cCR5hHz1cPvq4uZ93k93TYMGbmhsldauV5G4UdpA0gdh+Q00J1h9th1C9eLmuhD/BwbB5fmfOElQ0fKNt9D9hDgKM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936148; c=relaxed/simple; bh=eVhOiOfXw4o2qt4bdllBY+jHKu0pt33vUmwn4aS9VO8=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Ll8jJKTcNoadgX+s/UzIbyYDxXpzmX1l5JwTn7omRrknyoDeytyFyxjngJBiUXG2m/IYi0gLfsv+HErXdO7cyKXq90mkWxM7lGXm7tXJN4n1axMqPBt54oBANy+EIM23exc2npnCaIyrYDQfsZGtKQMbHuiO5OqyWS7T3zxVPa0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=amdvzmG/; arc=none smtp.client-ip=192.198.163.8 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="amdvzmG/" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936147; x=1740472147; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=eVhOiOfXw4o2qt4bdllBY+jHKu0pt33vUmwn4aS9VO8=; b=amdvzmG/JT+1hjNqSrMOKgsBxZvJiTn+0tGCpt3c//0ZDNFoFoNCpyl1 6xJfpxYoknbMOa+gMYnv+vtCIaQ9DVWvU74CJuw561zJ1N8JGlN33sqVw ytVx50LbTX6emSSWcWCDvv3qE5Id3m+zQYArkx+o3+3cFCUczC7fyyhUB E+2JqMcMjBaK7QLy3yoyoSWREMrStQuO/0Phxhf1HaQ54QJVl1lzitr8D 1KqyhoL5VA/G+tJriBtxPCR5BCjAcy3WORe/XGQqvauxbj1l/Hkc66O8o Jq/DaVOwGnyqXAUNCuvaafI9UPXVTca3z0nJQEqKC1oynJBKkUL5M0ZmD w==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="20751334" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="20751334" Received: from fmviesa008.fm.intel.com ([10.60.135.148]) by fmvoesa102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:29:04 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6735068" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by fmviesa008-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:29:04 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 109/130] KVM: TDX: Handle TDX PV port io hypercall Date: Mon, 26 Feb 2024 00:26:51 -0800 Message-Id: <4f4aaf292008608a8717e9553c3315ee02f66b20.1708933498.git.isaku.yamahata@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata Wire up TDX PV port IO hypercall to the KVM backend function. Signed-off-by: Isaku Yamahata Reviewed-by: Paolo Bonzini --- v18: - Fix out case to set R10 and R11 correctly when user space handled port out. --- arch/x86/kvm/vmx/tdx.c | 67 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 67 insertions(+) diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c index a2caf2ae838c..55fc6cc6c816 100644 --- a/arch/x86/kvm/vmx/tdx.c +++ b/arch/x86/kvm/vmx/tdx.c @@ -1152,6 +1152,71 @@ static int tdx_emulate_hlt(struct kvm_vcpu *vcpu) return kvm_emulate_halt_noskip(vcpu); } +static int tdx_complete_pio_out(struct kvm_vcpu *vcpu) +{ + tdvmcall_set_return_code(vcpu, TDVMCALL_SUCCESS); + tdvmcall_set_return_val(vcpu, 0); + return 1; +} + +static int tdx_complete_pio_in(struct kvm_vcpu *vcpu) +{ + struct x86_emulate_ctxt *ctxt = vcpu->arch.emulate_ctxt; + unsigned long val = 0; + int ret; + + WARN_ON_ONCE(vcpu->arch.pio.count != 1); + + ret = ctxt->ops->pio_in_emulated(ctxt, vcpu->arch.pio.size, + vcpu->arch.pio.port, &val, 1); + WARN_ON_ONCE(!ret); + + tdvmcall_set_return_code(vcpu, TDVMCALL_SUCCESS); + tdvmcall_set_return_val(vcpu, val); + + return 1; +} + +static int tdx_emulate_io(struct kvm_vcpu *vcpu) +{ + struct x86_emulate_ctxt *ctxt = vcpu->arch.emulate_ctxt; + unsigned long val = 0; + unsigned int port; + int size, ret; + bool write; + + ++vcpu->stat.io_exits; + + size = tdvmcall_a0_read(vcpu); + write = tdvmcall_a1_read(vcpu); + port = tdvmcall_a2_read(vcpu); + + if (size != 1 && size != 2 && size != 4) { + tdvmcall_set_return_code(vcpu, TDVMCALL_INVALID_OPERAND); + return 1; + } + + if (write) { + val = tdvmcall_a3_read(vcpu); + ret = ctxt->ops->pio_out_emulated(ctxt, size, port, &val, 1); + + /* No need for a complete_userspace_io callback. */ + vcpu->arch.pio.count = 0; + } else + ret = ctxt->ops->pio_in_emulated(ctxt, size, port, &val, 1); + + if (ret) + tdvmcall_set_return_val(vcpu, val); + else { + if (write) + vcpu->arch.complete_userspace_io = tdx_complete_pio_out; + else + vcpu->arch.complete_userspace_io = tdx_complete_pio_in; + } + + return ret; +} + static int handle_tdvmcall(struct kvm_vcpu *vcpu) { if (tdvmcall_exit_type(vcpu)) @@ -1162,6 +1227,8 @@ static int handle_tdvmcall(struct kvm_vcpu *vcpu) return tdx_emulate_cpuid(vcpu); case EXIT_REASON_HLT: return tdx_emulate_hlt(vcpu); + case EXIT_REASON_IO_INSTRUCTION: + return tdx_emulate_io(vcpu); default: break; } From patchwork Mon Feb 26 08:26:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571544 Received: from mgamail.intel.com (unknown [192.198.163.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 44C70219EB; Mon, 26 Feb 2024 08:29:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.8 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936149; cv=none; b=X35YErc6RIv6K0r4tArGVQMbexyaXF05bh6wiV7uNDLU1nUJ3qWe+LznZBg/difeqwg/UrByb0ydnJOyYcv3SOAl/MXi7UXQ2Rrl8rbbP02DJbewGk3BnvmBWx1yE3FRC5prU+FM9MBn2+MD9s8xOShSj0mqWzjlJ+WPuRIkf08= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936149; c=relaxed/simple; bh=QnWGRm/Bs2Ha4+9BL7xtnfj7c9Dq5ASvmvWattRSzvY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=GMoqSWuONNDPOiLIfoV/UnO3s2o8ikSjI9C4dxdZa0Fi501W1Pmsko+BnFa7YzSdajD6twFGKZJObBFvZLiSOZ7LC00kU06iQcOhW7++qwN2A1LjI5/bR5LfrvHVCqtZcLr7wbUNZTN80v8ScIJSkqVqjjNTdW4WutmJ/lrSyCM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=ho0fjsl4; arc=none smtp.client-ip=192.198.163.8 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="ho0fjsl4" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936147; x=1740472147; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=QnWGRm/Bs2Ha4+9BL7xtnfj7c9Dq5ASvmvWattRSzvY=; b=ho0fjsl4g874J2qgHwTIgutlJBQQ5i6C5PKEzoeLVFobT9wrpVNjLhw3 hf0isHcd7RQTrTX6U+U9yeqsbfPvUy3pbfbkeNQsIrCQnRTzBBbGwPLY6 W2DMCk1bk6PUmrP5g/DJRd5dJchaagaEo/5Tjx5NF6pYUhJCKQphM0GjQ NXM/tOLOxmnU70Berd5wzpj9TZWXYyClhGNwfqlRcPL9OOdpl+M3EJUFR si3tkPhONHuyf/a9VQ8dEuJusiak3TU+3fFgwsCnyZLpE3MRkLLejgLP3 MiSsq5MEWSTrI5tK4uJyuAeSgUpV9zlbtjqWf6lSUGzk/urNuioZt11/L A==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="20751341" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="20751341" Received: from fmviesa008.fm.intel.com ([10.60.135.148]) by fmvoesa102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:29:05 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6735074" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by fmviesa008-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:29:05 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com, Sean Christopherson Subject: [PATCH v19 110/130] KVM: TDX: Handle TDX PV MMIO hypercall Date: Mon, 26 Feb 2024 00:26:52 -0800 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Sean Christopherson Export kvm_io_bus_read and kvm_mmio tracepoint and wire up TDX PV MMIO hypercall to the KVM backend functions. kvm_io_bus_read/write() searches KVM device emulated in kernel of the given MMIO address and emulates the MMIO. As TDX PV MMIO also needs it, export kvm_io_bus_read(). kvm_io_bus_write() is already exported. TDX PV MMIO emulates some of MMIO itself. To add trace point consistently with x86 kvm, export kvm_mmio tracepoint. Signed-off-by: Sean Christopherson Signed-off-by: Isaku Yamahata Reviewed-by: Paolo Bonzini --- arch/x86/kvm/vmx/tdx.c | 114 +++++++++++++++++++++++++++++++++++++++++ arch/x86/kvm/x86.c | 1 + virt/kvm/kvm_main.c | 2 + 3 files changed, 117 insertions(+) diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c index 55fc6cc6c816..389bb95d2af0 100644 --- a/arch/x86/kvm/vmx/tdx.c +++ b/arch/x86/kvm/vmx/tdx.c @@ -1217,6 +1217,118 @@ static int tdx_emulate_io(struct kvm_vcpu *vcpu) return ret; } +static int tdx_complete_mmio(struct kvm_vcpu *vcpu) +{ + unsigned long val = 0; + gpa_t gpa; + int size; + + KVM_BUG_ON(vcpu->mmio_needed != 1, vcpu->kvm); + vcpu->mmio_needed = 0; + + if (!vcpu->mmio_is_write) { + gpa = vcpu->mmio_fragments[0].gpa; + size = vcpu->mmio_fragments[0].len; + + memcpy(&val, vcpu->run->mmio.data, size); + tdvmcall_set_return_val(vcpu, val); + trace_kvm_mmio(KVM_TRACE_MMIO_READ, size, gpa, &val); + } + return 1; +} + +static inline int tdx_mmio_write(struct kvm_vcpu *vcpu, gpa_t gpa, int size, + unsigned long val) +{ + if (kvm_iodevice_write(vcpu, &vcpu->arch.apic->dev, gpa, size, &val) && + kvm_io_bus_write(vcpu, KVM_MMIO_BUS, gpa, size, &val)) + return -EOPNOTSUPP; + + trace_kvm_mmio(KVM_TRACE_MMIO_WRITE, size, gpa, &val); + return 0; +} + +static inline int tdx_mmio_read(struct kvm_vcpu *vcpu, gpa_t gpa, int size) +{ + unsigned long val; + + if (kvm_iodevice_read(vcpu, &vcpu->arch.apic->dev, gpa, size, &val) && + kvm_io_bus_read(vcpu, KVM_MMIO_BUS, gpa, size, &val)) + return -EOPNOTSUPP; + + tdvmcall_set_return_val(vcpu, val); + trace_kvm_mmio(KVM_TRACE_MMIO_READ, size, gpa, &val); + return 0; +} + +static int tdx_emulate_mmio(struct kvm_vcpu *vcpu) +{ + struct kvm_memory_slot *slot; + int size, write, r; + unsigned long val; + gpa_t gpa; + + KVM_BUG_ON(vcpu->mmio_needed, vcpu->kvm); + + size = tdvmcall_a0_read(vcpu); + write = tdvmcall_a1_read(vcpu); + gpa = tdvmcall_a2_read(vcpu); + val = write ? tdvmcall_a3_read(vcpu) : 0; + + if (size != 1 && size != 2 && size != 4 && size != 8) + goto error; + if (write != 0 && write != 1) + goto error; + + /* Strip the shared bit, allow MMIO with and without it set. */ + gpa = gpa & ~gfn_to_gpa(kvm_gfn_shared_mask(vcpu->kvm)); + + if (size > 8u || ((gpa + size - 1) ^ gpa) & PAGE_MASK) + goto error; + + slot = kvm_vcpu_gfn_to_memslot(vcpu, gpa_to_gfn(gpa)); + if (slot && !(slot->flags & KVM_MEMSLOT_INVALID)) + goto error; + + if (!kvm_io_bus_write(vcpu, KVM_FAST_MMIO_BUS, gpa, 0, NULL)) { + trace_kvm_fast_mmio(gpa); + return 1; + } + + if (write) + r = tdx_mmio_write(vcpu, gpa, size, val); + else + r = tdx_mmio_read(vcpu, gpa, size); + if (!r) { + /* Kernel completed device emulation. */ + tdvmcall_set_return_code(vcpu, TDVMCALL_SUCCESS); + return 1; + } + + /* Request the device emulation to userspace device model. */ + vcpu->mmio_needed = 1; + vcpu->mmio_is_write = write; + vcpu->arch.complete_userspace_io = tdx_complete_mmio; + + vcpu->run->mmio.phys_addr = gpa; + vcpu->run->mmio.len = size; + vcpu->run->mmio.is_write = write; + vcpu->run->exit_reason = KVM_EXIT_MMIO; + + if (write) { + memcpy(vcpu->run->mmio.data, &val, size); + } else { + vcpu->mmio_fragments[0].gpa = gpa; + vcpu->mmio_fragments[0].len = size; + trace_kvm_mmio(KVM_TRACE_MMIO_READ_UNSATISFIED, size, gpa, NULL); + } + return 0; + +error: + tdvmcall_set_return_code(vcpu, TDVMCALL_INVALID_OPERAND); + return 1; +} + static int handle_tdvmcall(struct kvm_vcpu *vcpu) { if (tdvmcall_exit_type(vcpu)) @@ -1229,6 +1341,8 @@ static int handle_tdvmcall(struct kvm_vcpu *vcpu) return tdx_emulate_hlt(vcpu); case EXIT_REASON_IO_INSTRUCTION: return tdx_emulate_io(vcpu); + case EXIT_REASON_EPT_VIOLATION: + return tdx_emulate_mmio(vcpu); default: break; } diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 03950368d8db..d5b18cad9dcd 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -13975,6 +13975,7 @@ EXPORT_SYMBOL_GPL(kvm_sev_es_string_io); EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_entry); EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_exit); +EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_mmio); EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_fast_mmio); EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_inj_virq); EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_page_fault); diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index e27c22449d85..bc14e1f2610c 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -2689,6 +2689,7 @@ struct kvm_memory_slot *kvm_vcpu_gfn_to_memslot(struct kvm_vcpu *vcpu, gfn_t gfn return NULL; } +EXPORT_SYMBOL_GPL(kvm_vcpu_gfn_to_memslot); bool kvm_is_visible_gfn(struct kvm *kvm, gfn_t gfn) { @@ -5992,6 +5993,7 @@ int kvm_io_bus_read(struct kvm_vcpu *vcpu, enum kvm_bus bus_idx, gpa_t addr, r = __kvm_io_bus_read(vcpu, bus, &range, val); return r < 0 ? r : 0; } +EXPORT_SYMBOL_GPL(kvm_io_bus_read); int kvm_io_bus_register_dev(struct kvm *kvm, enum kvm_bus bus_idx, gpa_t addr, int len, struct kvm_io_device *dev) From patchwork Mon Feb 26 08:26:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571546 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3033F12A15E; Mon, 26 Feb 2024 08:29:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.8 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936150; cv=none; b=CPd9c6/fPrDYNH5QWoAkFMhTe54T0zJRi1QUnZrfjqPMzZbHjALfhj3OoMyxw6tF1XpXGlzzP/NjOH3A0FR9MXnW+XAJx6eapPZFXs2oSscCE6EczT36sJUgoI/9AiAmvYzwhAg3bkHot6fDWBGGs/DpoM1moi5ZfZD2Se4TWbM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936150; c=relaxed/simple; bh=gelDu1xjsgqIRauLE8EobxA3kCvyXLgLpgUUyecgcS4=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Z7WvDfrjY+we4I14ggExV6RePvGv6NVkrc435B48V2YwyIYaVKffO1DqfvNjW3p1Sxw7KV1w50zV1UICapEiPL50UKgC0z0e8/4g7kNrunOwaZKtfk6H4LjXl+5QQYY4Qc65fW7JfxjJY44eJp400PYA0WlSsb1OldRfXsVl/kg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=mQ1AM6Sd; arc=none smtp.client-ip=192.198.163.8 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="mQ1AM6Sd" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936148; x=1740472148; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=gelDu1xjsgqIRauLE8EobxA3kCvyXLgLpgUUyecgcS4=; b=mQ1AM6SdHxSjdKPWai3O8kfX6PQ1cqWEKN5hlL6CpCJtK8WTZ/Lvqs8q qp6B46xwhiPrUa2r1eQqwf9uWeeiGKD9Tng/94ajgvG6aHq/Ssgpagda9 Ql3Gxj3FgOBvJpmb/ymNWCV8twR8uCEQJciSNBuHu1lQM9BJNVr89axhh yGGkIQmyMe0+dq+gMkDin6DAhezAlD+4QOtIoQJ36O4cXWz7C5j8PJZqc IOwiSzUrlC5ewkHHJb6XcEbihH9npJzEjWRU5TqlF/XOwciHS0u39hVbD Tde13/B9gXrG54tFnNmA620bs9COnWk9mySF0UOIrH6/UTAAJz+cqWt7K g==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="20751346" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="20751346" Received: from fmviesa008.fm.intel.com ([10.60.135.148]) by fmvoesa102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:29:05 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6735077" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by fmviesa008-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:29:05 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 111/130] KVM: TDX: Implement callbacks for MSR operations for TDX Date: Mon, 26 Feb 2024 00:26:53 -0800 Message-Id: <62f8890cb90e49a3e0b0d5946318c0267b80c540.1708933498.git.isaku.yamahata@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata Implements set_msr/get_msr/has_emulated_msr methods for TDX to handle hypercall from guest TD for paravirtualized rdmsr and wrmsr. The TDX module virtualizes MSRs. For some MSRs, it injects #VE to the guest TD upon RDMSR or WRMSR. The exact list of such MSRs are defined in the spec. Upon #VE, the guest TD may execute hypercalls, TDG.VP.VMCALL and TDG.VP.VMCALL, which are defined in GHCI (Guest-Host Communication Interface) so that the host VMM (e.g. KVM) can virtualize the MSRs. There are three classes of MSRs virtualization. - non-configurable: TDX module directly virtualizes it. VMM can't configure. the value set by KVM_SET_MSR_INDEX_LIST is ignored. - configurable: TDX module directly virtualizes it. VMM can configure at the VM creation time. The value set by KVM_SET_MSR_INDEX_LIST is used. - #VE case Guest TD would issue TDG.VP.VMCALL and VMM handles the MSR hypercall. The value set by KVM_SET_MSR_INDEX_LIST is used. Signed-off-by: Isaku Yamahata Reviewed-by: Paolo Bonzini --- arch/x86/kvm/vmx/main.c | 44 +++++++++++++++++++++--- arch/x86/kvm/vmx/tdx.c | 70 ++++++++++++++++++++++++++++++++++++++ arch/x86/kvm/vmx/x86_ops.h | 6 ++++ arch/x86/kvm/x86.c | 1 - arch/x86/kvm/x86.h | 2 ++ 5 files changed, 118 insertions(+), 5 deletions(-) diff --git a/arch/x86/kvm/vmx/main.c b/arch/x86/kvm/vmx/main.c index c9a40456d965..ed46e7e57c18 100644 --- a/arch/x86/kvm/vmx/main.c +++ b/arch/x86/kvm/vmx/main.c @@ -247,6 +247,42 @@ static void vt_handle_exit_irqoff(struct kvm_vcpu *vcpu) vmx_handle_exit_irqoff(vcpu); } +static int vt_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info) +{ + if (unlikely(is_td_vcpu(vcpu))) + return tdx_set_msr(vcpu, msr_info); + + return vmx_set_msr(vcpu, msr_info); +} + +/* + * The kvm parameter can be NULL (module initialization, or invocation before + * VM creation). Be sure to check the kvm parameter before using it. + */ +static bool vt_has_emulated_msr(struct kvm *kvm, u32 index) +{ + if (kvm && is_td(kvm)) + return tdx_has_emulated_msr(index, true); + + return vmx_has_emulated_msr(kvm, index); +} + +static int vt_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info) +{ + if (unlikely(is_td_vcpu(vcpu))) + return tdx_get_msr(vcpu, msr_info); + + return vmx_get_msr(vcpu, msr_info); +} + +static void vt_msr_filter_changed(struct kvm_vcpu *vcpu) +{ + if (is_td_vcpu(vcpu)) + return; + + vmx_msr_filter_changed(vcpu); +} + static void vt_apicv_pre_state_restore(struct kvm_vcpu *vcpu) { struct pi_desc *pi = vcpu_to_pi_desc(vcpu); @@ -541,7 +577,7 @@ struct kvm_x86_ops vt_x86_ops __initdata = { /* TDX cpu enablement is done by tdx_hardware_setup(). */ .hardware_enable = vmx_hardware_enable, .hardware_disable = vt_hardware_disable, - .has_emulated_msr = vmx_has_emulated_msr, + .has_emulated_msr = vt_has_emulated_msr, .is_vm_type_supported = vt_is_vm_type_supported, .max_vcpus = vt_max_vcpus, @@ -563,8 +599,8 @@ struct kvm_x86_ops vt_x86_ops __initdata = { .update_exception_bitmap = vmx_update_exception_bitmap, .get_msr_feature = vmx_get_msr_feature, - .get_msr = vmx_get_msr, - .set_msr = vmx_set_msr, + .get_msr = vt_get_msr, + .set_msr = vt_set_msr, .get_segment_base = vmx_get_segment_base, .get_segment = vmx_get_segment, .set_segment = vmx_set_segment, @@ -674,7 +710,7 @@ struct kvm_x86_ops vt_x86_ops __initdata = { .apic_init_signal_blocked = vmx_apic_init_signal_blocked, .migrate_timers = vmx_migrate_timers, - .msr_filter_changed = vmx_msr_filter_changed, + .msr_filter_changed = vt_msr_filter_changed, .complete_emulated_msr = kvm_complete_insn_gp, .vcpu_deliver_sipi_vector = kvm_vcpu_deliver_sipi_vector, diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c index 389bb95d2af0..c8f991b69720 100644 --- a/arch/x86/kvm/vmx/tdx.c +++ b/arch/x86/kvm/vmx/tdx.c @@ -1877,6 +1877,76 @@ void tdx_get_exit_info(struct kvm_vcpu *vcpu, u32 *reason, *error_code = 0; } +static bool tdx_is_emulated_kvm_msr(u32 index, bool write) +{ + switch (index) { + case MSR_KVM_POLL_CONTROL: + return true; + default: + return false; + } +} + +bool tdx_has_emulated_msr(u32 index, bool write) +{ + switch (index) { + case MSR_IA32_UCODE_REV: + case MSR_IA32_ARCH_CAPABILITIES: + case MSR_IA32_POWER_CTL: + case MSR_IA32_CR_PAT: + case MSR_IA32_TSC_DEADLINE: + case MSR_IA32_MISC_ENABLE: + case MSR_PLATFORM_INFO: + case MSR_MISC_FEATURES_ENABLES: + case MSR_IA32_MCG_CAP: + case MSR_IA32_MCG_STATUS: + case MSR_IA32_MCG_CTL: + case MSR_IA32_MCG_EXT_CTL: + case MSR_IA32_MC0_CTL ... MSR_IA32_MCx_CTL(KVM_MAX_MCE_BANKS) - 1: + case MSR_IA32_MC0_CTL2 ... MSR_IA32_MCx_CTL2(KVM_MAX_MCE_BANKS) - 1: + /* MSR_IA32_MCx_{CTL, STATUS, ADDR, MISC, CTL2} */ + return true; + case APIC_BASE_MSR ... APIC_BASE_MSR + 0xff: + /* + * x2APIC registers that are virtualized by the CPU can't be + * emulated, KVM doesn't have access to the virtual APIC page. + */ + switch (index) { + case X2APIC_MSR(APIC_TASKPRI): + case X2APIC_MSR(APIC_PROCPRI): + case X2APIC_MSR(APIC_EOI): + case X2APIC_MSR(APIC_ISR) ... X2APIC_MSR(APIC_ISR + APIC_ISR_NR): + case X2APIC_MSR(APIC_TMR) ... X2APIC_MSR(APIC_TMR + APIC_ISR_NR): + case X2APIC_MSR(APIC_IRR) ... X2APIC_MSR(APIC_IRR + APIC_ISR_NR): + return false; + default: + return true; + } + case MSR_IA32_APICBASE: + case MSR_EFER: + return !write; + case 0x4b564d00 ... 0x4b564dff: + /* KVM custom MSRs */ + return tdx_is_emulated_kvm_msr(index, write); + default: + return false; + } +} + +int tdx_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr) +{ + if (tdx_has_emulated_msr(msr->index, false)) + return kvm_get_msr_common(vcpu, msr); + return 1; +} + +int tdx_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr) +{ + if (tdx_has_emulated_msr(msr->index, true)) + return kvm_set_msr_common(vcpu, msr); + return 1; +} + static int tdx_get_capabilities(struct kvm_tdx_cmd *cmd) { struct kvm_tdx_capabilities __user *user_caps; diff --git a/arch/x86/kvm/vmx/x86_ops.h b/arch/x86/kvm/vmx/x86_ops.h index a12e3bfc96dd..017a73ab34bb 100644 --- a/arch/x86/kvm/vmx/x86_ops.h +++ b/arch/x86/kvm/vmx/x86_ops.h @@ -165,6 +165,9 @@ void tdx_deliver_interrupt(struct kvm_lapic *apic, int delivery_mode, void tdx_inject_nmi(struct kvm_vcpu *vcpu); void tdx_get_exit_info(struct kvm_vcpu *vcpu, u32 *reason, u64 *info1, u64 *info2, u32 *intr_info, u32 *error_code); +bool tdx_has_emulated_msr(u32 index, bool write); +int tdx_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr); +int tdx_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr); int tdx_vcpu_ioctl(struct kvm_vcpu *vcpu, void __user *argp); @@ -214,6 +217,9 @@ static inline void tdx_deliver_interrupt(struct kvm_lapic *apic, int delivery_mo static inline void tdx_inject_nmi(struct kvm_vcpu *vcpu) {} static inline void tdx_get_exit_info(struct kvm_vcpu *vcpu, u32 *reason, u64 *info1, u64 *info2, u32 *intr_info, u32 *error_code) {} +static inline bool tdx_has_emulated_msr(u32 index, bool write) { return false; } +static inline int tdx_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr) { return 1; } +static inline int tdx_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr) { return 1; } static inline int tdx_vcpu_ioctl(struct kvm_vcpu *vcpu, void __user *argp) { return -EOPNOTSUPP; } diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index d5b18cad9dcd..0e1d3853eeb4 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -90,7 +90,6 @@ #include "trace.h" #define MAX_IO_MSRS 256 -#define KVM_MAX_MCE_BANKS 32 struct kvm_caps kvm_caps __read_mostly = { .supported_mce_cap = MCG_CTL_P | MCG_SER_P, diff --git a/arch/x86/kvm/x86.h b/arch/x86/kvm/x86.h index 4e40c23d66ed..c87b7a777b67 100644 --- a/arch/x86/kvm/x86.h +++ b/arch/x86/kvm/x86.h @@ -9,6 +9,8 @@ #include "kvm_cache_regs.h" #include "kvm_emulate.h" +#define KVM_MAX_MCE_BANKS 32 + bool __kvm_is_vm_type_supported(unsigned long type); struct kvm_caps { From patchwork Mon Feb 26 08:26:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571545 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AD37112AAE9; Mon, 26 Feb 2024 08:29:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.8 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936150; cv=none; b=Y9rg2g9TLrbMt63Z+dxOVJTzLTICsBozySyEPAnQOCM4tfoxkto6MCkFjuqc7oaCBEVu3IXbtDCRKfdvaReuKGAtFpuIu3dluCMUMeb62SIV9xSawEetVSqDHyqhor+oyZFEUcYYGBJRImfKemEYVr/fjId6OicTpJ6fLclO8z4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936150; c=relaxed/simple; bh=+2fCF+WzvIZcAkBJTQ/JIVXOSBWrPoxPrF9MU/Z9Lmg=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=tgXY6lkVxyQ5fHJqjpOd8CLjACBdPHTb2K4R1SoHBR1emEjCZL3fLp4ucSbRpww+8+bohzkazxCgf6OSqwRdLoOpXX3MySxU7bjSt7M2hTndDgHcz0EaPdeLm5uD5eIJ63IJukm+Cn9A6Gd8fR2XM3fjEsP1qIJh/z/dpEohFIE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=SbtD+5cs; arc=none smtp.client-ip=192.198.163.8 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="SbtD+5cs" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936148; x=1740472148; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=+2fCF+WzvIZcAkBJTQ/JIVXOSBWrPoxPrF9MU/Z9Lmg=; b=SbtD+5csX/fsJlIOrA9PWvkGGQePK5AMfqofVa0abRLzzYV8wK7LArPI O2OcVyo2yE5Z7e9rPVnb/6mDLd8jC3sq+FSEgVu6EOhNFKbfxlx1eh/r0 3xYl8yonJNpLdLVSXioz9+gPs7ddJQqQr+WS6Ns7iIwjvQ/BIJsXDVaI4 1ENuXym9VctCZ1C1oSy7oN+xBbWwe1Qbga6mfzg7nM9xvNyEV5hq4HASx fJr0ZMJL1IZ62sXsppkD7coYbqCwUbwT0hd7Zb8Vef5qUsui0ocdHEDA0 CNfFo39Ft1bFNG+XWcR9fsqlw3Rgg/BfR948ifLe1QmsmVU1eVRsM09YN A==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="20751351" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="20751351" Received: from fmviesa008.fm.intel.com ([10.60.135.148]) by fmvoesa102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:29:06 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6735080" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by fmviesa008-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:29:05 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 112/130] KVM: TDX: Handle TDX PV rdmsr/wrmsr hypercall Date: Mon, 26 Feb 2024 00:26:54 -0800 Message-Id: <1fbd5bb56fc607d279deff729272b19b109fb524.1708933498.git.isaku.yamahata@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata Wire up TDX PV rdmsr/wrmsr hypercall to the KVM backend function. Signed-off-by: Isaku Yamahata Reviewed-by: Paolo Bonzini --- arch/x86/kvm/vmx/tdx.c | 39 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 39 insertions(+) diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c index c8f991b69720..4c635bfcaf7a 100644 --- a/arch/x86/kvm/vmx/tdx.c +++ b/arch/x86/kvm/vmx/tdx.c @@ -1329,6 +1329,41 @@ static int tdx_emulate_mmio(struct kvm_vcpu *vcpu) return 1; } +static int tdx_emulate_rdmsr(struct kvm_vcpu *vcpu) +{ + u32 index = tdvmcall_a0_read(vcpu); + u64 data; + + if (!kvm_msr_allowed(vcpu, index, KVM_MSR_FILTER_READ) || + kvm_get_msr(vcpu, index, &data)) { + trace_kvm_msr_read_ex(index); + tdvmcall_set_return_code(vcpu, TDVMCALL_INVALID_OPERAND); + return 1; + } + trace_kvm_msr_read(index, data); + + tdvmcall_set_return_code(vcpu, TDVMCALL_SUCCESS); + tdvmcall_set_return_val(vcpu, data); + return 1; +} + +static int tdx_emulate_wrmsr(struct kvm_vcpu *vcpu) +{ + u32 index = tdvmcall_a0_read(vcpu); + u64 data = tdvmcall_a1_read(vcpu); + + if (!kvm_msr_allowed(vcpu, index, KVM_MSR_FILTER_WRITE) || + kvm_set_msr(vcpu, index, data)) { + trace_kvm_msr_write_ex(index, data); + tdvmcall_set_return_code(vcpu, TDVMCALL_INVALID_OPERAND); + return 1; + } + + trace_kvm_msr_write(index, data); + tdvmcall_set_return_code(vcpu, TDVMCALL_SUCCESS); + return 1; +} + static int handle_tdvmcall(struct kvm_vcpu *vcpu) { if (tdvmcall_exit_type(vcpu)) @@ -1343,6 +1378,10 @@ static int handle_tdvmcall(struct kvm_vcpu *vcpu) return tdx_emulate_io(vcpu); case EXIT_REASON_EPT_VIOLATION: return tdx_emulate_mmio(vcpu); + case EXIT_REASON_MSR_READ: + return tdx_emulate_rdmsr(vcpu); + case EXIT_REASON_MSR_WRITE: + return tdx_emulate_wrmsr(vcpu); default: break; } From patchwork Mon Feb 26 08:26:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571547 Received: from mgamail.intel.com (unknown [192.198.163.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3CF3312B15D; Mon, 26 Feb 2024 08:29:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.8 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936151; cv=none; b=Siga6QaZoDpEK8cp+OBGz+6R1W+xQ56Sfg9viNz/RoO/sTV+iiTKJGHYRfoSr0G4YEnozZ/5Wo7RCVFw5uSMX1GGxyXaKuf7Xi8icJ3SqFVZG67VditYJRsDeY4mH17fRrMbVoVpMdwuToAgQxOMSzjRQVMazXrB+aPIN+ThWEA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936151; c=relaxed/simple; bh=XAPqg9P4ovUdN9yysJhqj4R/8c0qj5T2TQ9C6uUUWv0=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=GpBmXTl1sn+dZ0Pm0TSA+hBcsXZVBAHHkdzzsCBddNO4btFZ0tzp1+NJTYi0sN/ZuoC+bmlyhpOrIzzSXgacxbFLd7KpHaCWSfxBKKRCLNPfLl5xYN9/tG6BbXJ3+DMs6LeS7qfolfaSHVBZZpFtF4oJqCwfK+rFOFA8ifNjPJE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=fail (p=none dis=none) header.from=intel.com; spf=fail smtp.mailfrom=intel.com; dkim=fail (0-bit key) header.d=intel.com header.i=@intel.com header.b=Uf3z1wew reason="key not found in DNS"; arc=none smtp.client-ip=192.198.163.8 Authentication-Results: smtp.subspace.kernel.org; dmarc=fail (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=fail reason="key not found in DNS" (0-bit key) header.d=intel.com header.i=@intel.com header.b="Uf3z1wew" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936149; x=1740472149; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=XAPqg9P4ovUdN9yysJhqj4R/8c0qj5T2TQ9C6uUUWv0=; b=Uf3z1wewYIk9yE0vPZhK5DDDgrNNwmHAXPwQcWmZ+nRYslBBeW+QOOp8 WH8z2UGFzD36bNMYoHcg5N2XGHDZSs+X/rAn0YKlLsxv3z9nVbMK9BdjS IRVpUYMza7FHckszSKLOug9VKQN1JDeLnP/NHlz8+gemlXRucOIVLxIco msoB4dhK1N/2nzSlEW27LtHDItCjNRVdvD3enAP2TQT1CElvbL4G9L+kj kU9ZJUWKX/Dlv1As0EHBMTQ9v9cXHEPoDw+NASUSlHYVzl2ajnJrOz13H uw3Y39Vku1ULmyr3RSbxhy70+KMnnujgBm7zPuxxa/cjfkGdCeOkq9DCh w==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="20751356" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="20751356" Received: from fmviesa008.fm.intel.com ([10.60.135.148]) by fmvoesa102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:29:06 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6735084" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by fmviesa008-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:29:06 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 113/130] KVM: TDX: Handle MSR MTRRCap and MTRRDefType access Date: Mon, 26 Feb 2024 00:26:55 -0800 Message-Id: <81119d66392bc9446340a16f8a532c7e1b2665a2.1708933498.git.isaku.yamahata@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata Handle MTRRCap RO MSR to return all features are unsupported and handle MTRRDefType MSR to accept only E=1,FE=0,type=writeback. enable MTRR, disable Fixed range MTRRs, default memory type=writeback TDX virtualizes that cpuid to report MTRR to guest TD and TDX enforces guest CR0.CD=0. If guest tries to set CR0.CD=1, it results in #GP. While updating MTRR requires to set CR0.CD=1 (and other cache flushing operations). It means guest TD can't update MTRR. Virtualize MTRR as all features disabled and default memory type as writeback. Signed-off-by: Isaku Yamahata --- arch/x86/kvm/vmx/tdx.c | 99 ++++++++++++++++++++++++++++++++++-------- 1 file changed, 82 insertions(+), 17 deletions(-) diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c index 4c635bfcaf7a..2bddaef495d1 100644 --- a/arch/x86/kvm/vmx/tdx.c +++ b/arch/x86/kvm/vmx/tdx.c @@ -611,18 +611,7 @@ u8 tdx_get_mt_mask(struct kvm_vcpu *vcpu, gfn_t gfn, bool is_mmio) if (!kvm_arch_has_noncoherent_dma(vcpu->kvm)) return (MTRR_TYPE_WRBACK << VMX_EPT_MT_EPTE_SHIFT) | VMX_EPT_IPAT_BIT; - /* - * TDX enforces CR0.CD = 0 and KVM MTRR emulation enforces writeback. - * TODO: implement MTRR MSR emulation so that - * MTRRCap: SMRR=0: SMRR interface unsupported - * WC=0: write combining unsupported - * FIX=0: Fixed range registers unsupported - * VCNT=0: number of variable range regitsers = 0 - * MTRRDefType: E=1, FE=0, type=writeback only. Don't allow other value. - * E=1: enable MTRR - * FE=0: disable fixed range MTRRs - * type: default memory type=writeback - */ + /* TDX enforces CR0.CD = 0 and KVM MTRR emulation enforces writeback. */ return MTRR_TYPE_WRBACK << VMX_EPT_MT_EPTE_SHIFT; } @@ -1932,7 +1921,9 @@ bool tdx_has_emulated_msr(u32 index, bool write) case MSR_IA32_UCODE_REV: case MSR_IA32_ARCH_CAPABILITIES: case MSR_IA32_POWER_CTL: + case MSR_MTRRcap: case MSR_IA32_CR_PAT: + case MSR_MTRRdefType: case MSR_IA32_TSC_DEADLINE: case MSR_IA32_MISC_ENABLE: case MSR_PLATFORM_INFO: @@ -1974,16 +1965,47 @@ bool tdx_has_emulated_msr(u32 index, bool write) int tdx_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr) { - if (tdx_has_emulated_msr(msr->index, false)) - return kvm_get_msr_common(vcpu, msr); - return 1; + switch (msr->index) { + case MSR_MTRRcap: + /* + * Override kvm_mtrr_get_msr() which hardcodes the value. + * Report SMRR = 0, WC = 0, FIX = 0 VCNT = 0 to disable MTRR + * effectively. + */ + msr->data = 0; + return 0; + default: + if (tdx_has_emulated_msr(msr->index, false)) + return kvm_get_msr_common(vcpu, msr); + return 1; + } } int tdx_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr) { - if (tdx_has_emulated_msr(msr->index, true)) + switch (msr->index) { + case MSR_MTRRdefType: + /* + * Allow writeback only for all memory. + * Because it's reported that fixed range MTRR isn't supported + * and VCNT=0, enforce MTRRDefType.FE = 0 and don't care + * variable range MTRRs. Only default memory type matters. + * + * bit 11 E: MTRR enable/disable + * bit 12 FE: Fixed-range MTRRs enable/disable + * (E, FE) = (1, 1): enable MTRR and Fixed range MTRR + * (E, FE) = (1, 0): enable MTRR, disable Fixed range MTRR + * (E, FE) = (0, *): disable all MTRRs. all physical memory + * is UC + */ + if (msr->data != ((1 << 11) | MTRR_TYPE_WRBACK)) + return 1; return kvm_set_msr_common(vcpu, msr); - return 1; + default: + if (tdx_has_emulated_msr(msr->index, true)) + return kvm_set_msr_common(vcpu, msr); + return 1; + } } static int tdx_get_capabilities(struct kvm_tdx_cmd *cmd) @@ -2704,6 +2726,45 @@ static int tdx_td_vcpu_init(struct kvm_vcpu *vcpu, u64 vcpu_rcx) return ret; } +static int tdx_vcpu_init_mtrr(struct kvm_vcpu *vcpu) +{ + struct msr_data msr; + int ret; + int i; + + /* + * To avoid confusion with reporting VNCT = 0, explicitly disable + * vaiale-range reisters. + */ + for (i = 0; i < KVM_NR_VAR_MTRR; i++) { + /* phymask */ + msr = (struct msr_data) { + .host_initiated = true, + .index = 0x200 + 2 * i + 1, + .data = 0, /* valid = 0 to disable. */ + }; + ret = kvm_set_msr_common(vcpu, &msr); + if (ret) + return -EINVAL; + } + + /* Set MTRR to use writeback on reset. */ + msr = (struct msr_data) { + .host_initiated = true, + .index = MSR_MTRRdefType, + /* + * Set E(enable MTRR)=1, FE(enable fixed range MTRR)=0, default + * type=writeback on reset to avoid UC. Note E=0 means all + * memory is UC. + */ + .data = (1 << 11) | MTRR_TYPE_WRBACK, + }; + ret = kvm_set_msr_common(vcpu, &msr); + if (ret) + return -EINVAL; + return 0; +} + int tdx_vcpu_ioctl(struct kvm_vcpu *vcpu, void __user *argp) { struct msr_data apic_base_msr; @@ -2741,6 +2802,10 @@ int tdx_vcpu_ioctl(struct kvm_vcpu *vcpu, void __user *argp) if (kvm_set_apic_base(vcpu, &apic_base_msr)) return -EINVAL; + ret = tdx_vcpu_init_mtrr(vcpu); + if (ret) + return ret; + ret = tdx_td_vcpu_init(vcpu, (u64)cmd.data); if (ret) return ret; From patchwork Mon Feb 26 08:26:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571548 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3FF0F12BEAB; Mon, 26 Feb 2024 08:29:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.8 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936152; cv=none; b=qS4LzccXx67XqJds192r6MYkhU04eLE7FANV6CbSClIo22vrg5Ank5QOiB4XGOmaQ2HGXv935AZNsleIlTgehaxqVzmJPlqB8fNbi1F8yKfXtqIXavV6Wo90rDuoN5NogG6V4QPkz/9+SS1abgLuENIvjgclBy37HBBmT3jfWZw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936152; c=relaxed/simple; bh=MzeWQckTmtJUMXlbAxLBuko2QU4/xgbu5cjp+acvzuc=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=JeN0InB/riy1DMbVoVqmvDcW0Jm0yGjZhSNwtmL1MbPMKF1uhMNv9Lp31rdpAdbfi7Kd0KXEyjMMA421oTBLsxlnDZCPOfGwEWkYsaUsRqeNSfFnjlNsWU4CPGM39SLNni8F2ERbDfpmB6wwQ0pOZ+UtHeWkm2GJpZCYWJUDNvw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=ECTrG1hH; arc=none smtp.client-ip=192.198.163.8 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="ECTrG1hH" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936150; x=1740472150; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=MzeWQckTmtJUMXlbAxLBuko2QU4/xgbu5cjp+acvzuc=; b=ECTrG1hH65XwTxYmuPJ0R6vzP74EA4asV6C/ZX7YBXHYVaPug4CtCib+ R71oUMOuxrc6XT+4XBmRViLTeH2d51v8S8djiMg8X0aGmz4G2LVU2MoAP jUYL5v2U42axq/L7NfjxbZ+IRImUvpNgyXA1J5CsiW2U1EFUjiNqQ0NVR PCrhYMpsQBFgSkl6LfdwYxvr9V74sr+7yKm24AMC0kWoQYp66Fjj+9Zgp /kNRx7EALaq4PDLSJIOR8SYU2yHDsYNbdQLUwYLwy0nAGNQGNvN6vIyy0 YSV38uxPoafrb93arsT+eMf6byezpv/um3DPrSzSWiwAXwYfLya9tWom8 w==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="20751361" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="20751361" Received: from fmviesa008.fm.intel.com ([10.60.135.148]) by fmvoesa102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:29:07 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6735087" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by fmviesa008-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:29:06 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 114/130] KVM: TDX: Handle MSR IA32_FEAT_CTL MSR and IA32_MCG_EXT_CTL Date: Mon, 26 Feb 2024 00:26:56 -0800 Message-Id: <747b9360695a54e096cdb929139dc930da81b8c8.1708933498.git.isaku.yamahata@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata MCE and MCA is advertised via cpuid based on the TDX module spec. Guest kernel can access IA32_FEAT_CTL for checking if LMCE is enabled by platform and IA32_MCG_EXT_CTL to enable LMCE. Make TDX KVM handle them. Otherwise guest MSR access to them with TDG.VP.VMCALL on VE results in GP in guest. Because LMCE is disabled with qemu by default, "-cpu lmce=on" to qemu command line is needed to reproduce it. Signed-off-by: Isaku Yamahata --- arch/x86/kvm/vmx/tdx.c | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c index 2bddaef495d1..3481c0b6ef2c 100644 --- a/arch/x86/kvm/vmx/tdx.c +++ b/arch/x86/kvm/vmx/tdx.c @@ -1952,6 +1952,7 @@ bool tdx_has_emulated_msr(u32 index, bool write) default: return true; } + case MSR_IA32_FEAT_CTL: case MSR_IA32_APICBASE: case MSR_EFER: return !write; @@ -1966,6 +1967,20 @@ bool tdx_has_emulated_msr(u32 index, bool write) int tdx_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr) { switch (msr->index) { + case MSR_IA32_FEAT_CTL: + /* + * MCE and MCA are advertised via cpuid. guest kernel could + * check if LMCE is enabled or not. + */ + msr->data = FEAT_CTL_LOCKED; + if (vcpu->arch.mcg_cap & MCG_LMCE_P) + msr->data |= FEAT_CTL_LMCE_ENABLED; + return 0; + case MSR_IA32_MCG_EXT_CTL: + if (!msr->host_initiated && !(vcpu->arch.mcg_cap & MCG_LMCE_P)) + return 1; + msr->data = vcpu->arch.mcg_ext_ctl; + return 0; case MSR_MTRRcap: /* * Override kvm_mtrr_get_msr() which hardcodes the value. @@ -1984,6 +1999,11 @@ int tdx_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr) int tdx_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr) { switch (msr->index) { + case MSR_IA32_MCG_EXT_CTL: + if (!msr->host_initiated && !(vcpu->arch.mcg_cap & MCG_LMCE_P)) + return 1; + vcpu->arch.mcg_ext_ctl = msr->data; + return 0; case MSR_MTRRdefType: /* * Allow writeback only for all memory. From patchwork Mon Feb 26 08:26:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571549 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id ABE6612BF02; Mon, 26 Feb 2024 08:29:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.8 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936152; cv=none; b=XZ25khyz7FJPH8cWpZ7sdI7s/H1u8yY5rBMcmS0TfUGH8rh1WiQmvr2fnLYavHuMfa3QgwM7+CybCIks4a6AFTegBedqaa6RZYithfrc4Dhs4RgnNn/u67+x7aOLOuEZjRX/epJj6vCu+kRbFKwF0suarvACrPLKXnQtKNyVsYU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936152; c=relaxed/simple; bh=d5xutcnpm9SfZzS14NRBtPoOgUgb4pPOvm9954l6hF0=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=ErQM5G/+99og1iRH0WAzfIlCMg06VXcSLppdrPnlXWuf9R4qQwiK/4QoqR09MnARZNKHaeg4lYzJGBhHTon4LilS/Gzm6eLlRxWHqrt3ck9OKQYpiEukUvZFzrV8TMdtBzKBvSRLnIKlhyAvgOGMIQvPipJ0Tz1qBMRJKA/EjLc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=PsTnVErr; arc=none smtp.client-ip=192.198.163.8 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="PsTnVErr" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936150; x=1740472150; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=d5xutcnpm9SfZzS14NRBtPoOgUgb4pPOvm9954l6hF0=; b=PsTnVErrMLLtLWHvBolg8GgH3D0CxgetBkbaVR4cdwCTQCas6FdrkEyF xVE8Xz0xRBiNKUeyYLvUp9I//NiYPcy6BDr3XQktS+GwlU0PQMOCiBXKx JApG/kimsgvTIxtYBODo9pQgWXa5lPlD2tsi9uNthehvmYuTjUkoj9teR JT5tK4VZWr5Ey8lKS70MEOi2WaTEhtWKYSQ2aCUUzPACjRlyNnmlNkDLN lyVOwlPWmT3VeBx+ZCqnRLKkkY7k62I/MgW0tSmMhQcQBKtWY05LnPc83 YYgi5LKB0v/J5mGrgagmhG43IyNPf+gikytKml30UHgaYHPHV7eLnZDHf g==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="20751366" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="20751366" Received: from fmviesa008.fm.intel.com ([10.60.135.148]) by fmvoesa102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:29:07 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6735091" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by fmviesa008-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:29:07 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 115/130] KVM: TDX: Handle TDG.VP.VMCALL hypercall Date: Mon, 26 Feb 2024 00:26:57 -0800 Message-Id: <86f5018e3c7a1ab6649f93c8103611b195c1b5d5.1708933498.git.isaku.yamahata@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata Implement TDG.VP.VMCALL hypercall. If the input value is zero, return success code and zero in output registers. TDG.VP.VMCALL hypercall is a subleaf of TDG.VP.VMCALL to enumerate which TDG.VP.VMCALL sub leaves are supported. This hypercall is for future enhancement of the Guest-Host-Communication Interface (GHCI) specification. The GHCI version of 344426-001US defines it to require input R12 to be zero and to return zero in output registers, R11, R12, R13, and R14 so that guest TD enumerates no enhancement. Signed-off-by: Isaku Yamahata --- v19: - rename TDG_VP_VMCALL_GET_TD_VM_CALL_INFO => TDVMCALL_GET_TD_VM_CALL_INFO --- arch/x86/include/asm/shared/tdx.h | 1 + arch/x86/kvm/vmx/tdx.c | 16 ++++++++++++++++ 2 files changed, 17 insertions(+) diff --git a/arch/x86/include/asm/shared/tdx.h b/arch/x86/include/asm/shared/tdx.h index 28c4a62b7dba..3e8ce567fde0 100644 --- a/arch/x86/include/asm/shared/tdx.h +++ b/arch/x86/include/asm/shared/tdx.h @@ -22,6 +22,7 @@ #define TDCS_NOTIFY_ENABLES 0x9100000000000010 /* TDX hypercall Leaf IDs */ +#define TDVMCALL_GET_TD_VM_CALL_INFO 0x10000 #define TDVMCALL_MAP_GPA 0x10001 #define TDVMCALL_GET_QUOTE 0x10002 #define TDVMCALL_REPORT_FATAL_ERROR 0x10003 diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c index 3481c0b6ef2c..725cb40d0814 100644 --- a/arch/x86/kvm/vmx/tdx.c +++ b/arch/x86/kvm/vmx/tdx.c @@ -1353,6 +1353,20 @@ static int tdx_emulate_wrmsr(struct kvm_vcpu *vcpu) return 1; } +static int tdx_get_td_vm_call_info(struct kvm_vcpu *vcpu) +{ + if (tdvmcall_a0_read(vcpu)) + tdvmcall_set_return_code(vcpu, TDVMCALL_INVALID_OPERAND); + else { + tdvmcall_set_return_code(vcpu, TDVMCALL_SUCCESS); + kvm_r11_write(vcpu, 0); + tdvmcall_a0_write(vcpu, 0); + tdvmcall_a1_write(vcpu, 0); + tdvmcall_a2_write(vcpu, 0); + } + return 1; +} + static int handle_tdvmcall(struct kvm_vcpu *vcpu) { if (tdvmcall_exit_type(vcpu)) @@ -1371,6 +1385,8 @@ static int handle_tdvmcall(struct kvm_vcpu *vcpu) return tdx_emulate_rdmsr(vcpu); case EXIT_REASON_MSR_WRITE: return tdx_emulate_wrmsr(vcpu); + case TDVMCALL_GET_TD_VM_CALL_INFO: + return tdx_get_td_vm_call_info(vcpu); default: break; } From patchwork Mon Feb 26 08:26:58 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571550 Received: from mgamail.intel.com (unknown [192.198.163.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5D9D512BF30; Mon, 26 Feb 2024 08:29:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.8 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936154; cv=none; b=OLnDtbMoUXZOwwkUFApGLFyAopHkpf8puPNm2K4bfEqB9If8T3LQ/BUWFR2bRkmM5el69cgnoH7XknfNm7R4wSioBQhVt+L+1/fqm5VFZeS5WQkHq4ENDRqOKX3ZcACFf1JRamUKKxaTqromkJOKIaNSCfe5Fdlp82O28sBr0PU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936154; c=relaxed/simple; bh=07lgdSSf9Dxn5POomTuy/86nTXxosamxMuUiyjc3woM=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=dlqPy+mT7bLkJNG5ITLM1ZC6DZBASNUNgQnhS+b4N8FHlDMVMUEWeOSuxliI317dxa1WpoLrvqZ5FovzaS5Rfqy1M/0+gbeRP/HhqPev56TNLHs1Ze+v6GC6E9cNTmmmD7lwBXO0i766URBYbX1SoySfjuoqk+5U2QDriNHnUlQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=llh6S4KM; arc=none smtp.client-ip=192.198.163.8 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="llh6S4KM" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936151; x=1740472151; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=07lgdSSf9Dxn5POomTuy/86nTXxosamxMuUiyjc3woM=; b=llh6S4KM7g7DRFE0zuMNhMdAO0UdavDMtxv529EiRX8cLUXk/pZLn00E U+A5SCnXvx5c+ixJPrOP4CS8e4uZovk49P5jWLNVBb2sGi/n8/GXiA9V0 70xLD5Ahu7TD1tCQ/x0s4Gyky9YYKMoxS2mzqBgTC0FqTrYHvZ1MsnhtW 79bTenhE3AmoDhDGQLLt3rs8QibyM5T0cYStTXNLgrXqDoydtiHnGqM60 /WGKJMtf+IfYWjbcNQ5fAg1aPtV+pcJGd3U7S61eFzowVh4pSHZoor/8X ck2cqXuOXSCdWwLaJ2/v0D5APtyevQaMdTfoE2ZMNmdaajyJHWnVMLyNz g==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="20751371" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="20751371" Received: from fmviesa008.fm.intel.com ([10.60.135.148]) by fmvoesa102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:29:07 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6735096" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by fmviesa008-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:29:07 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 116/130] KVM: TDX: Silently discard SMI request Date: Mon, 26 Feb 2024 00:26:58 -0800 Message-Id: <9c4547ea234a2ba09ebe05219f180f08ac6fc2e3.1708933498.git.isaku.yamahata@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata TDX doesn't support system-management mode (SMM) and system-management interrupt (SMI) in guest TDs. Because guest state (vcpu state, memory state) is protected, it must go through the TDX module APIs to change guest state, injecting SMI and changing vcpu mode into SMM. The TDX module doesn't provide a way for VMM to inject SMI into guest TD and a way for VMM to switch guest vcpu mode into SMM. We have two options in KVM when handling SMM or SMI in the guest TD or the device model (e.g. QEMU): 1) silently ignore the request or 2) return a meaningful error. For simplicity, we implemented the option 1). Signed-off-by: Isaku Yamahata --- arch/x86/kvm/smm.h | 7 +++++- arch/x86/kvm/vmx/main.c | 45 ++++++++++++++++++++++++++++++++++---- arch/x86/kvm/vmx/tdx.c | 29 ++++++++++++++++++++++++ arch/x86/kvm/vmx/x86_ops.h | 12 ++++++++++ 4 files changed, 88 insertions(+), 5 deletions(-) diff --git a/arch/x86/kvm/smm.h b/arch/x86/kvm/smm.h index a1cf2ac5bd78..bc77902f5c18 100644 --- a/arch/x86/kvm/smm.h +++ b/arch/x86/kvm/smm.h @@ -142,7 +142,12 @@ union kvm_smram { static inline int kvm_inject_smi(struct kvm_vcpu *vcpu) { - kvm_make_request(KVM_REQ_SMI, vcpu); + /* + * If SMM isn't supported (e.g. TDX), silently discard SMI request. + * Assume that SMM supported = MSR_IA32_SMBASE supported. + */ + if (static_call(kvm_x86_has_emulated_msr)(vcpu->kvm, MSR_IA32_SMBASE)) + kvm_make_request(KVM_REQ_SMI, vcpu); return 0; } diff --git a/arch/x86/kvm/vmx/main.c b/arch/x86/kvm/vmx/main.c index ed46e7e57c18..4f3b872cd401 100644 --- a/arch/x86/kvm/vmx/main.c +++ b/arch/x86/kvm/vmx/main.c @@ -283,6 +283,43 @@ static void vt_msr_filter_changed(struct kvm_vcpu *vcpu) vmx_msr_filter_changed(vcpu); } +#ifdef CONFIG_KVM_SMM +static int vt_smi_allowed(struct kvm_vcpu *vcpu, bool for_injection) +{ + if (is_td_vcpu(vcpu)) + return tdx_smi_allowed(vcpu, for_injection); + + return vmx_smi_allowed(vcpu, for_injection); +} + +static int vt_enter_smm(struct kvm_vcpu *vcpu, union kvm_smram *smram) +{ + if (unlikely(is_td_vcpu(vcpu))) + return tdx_enter_smm(vcpu, smram); + + return vmx_enter_smm(vcpu, smram); +} + +static int vt_leave_smm(struct kvm_vcpu *vcpu, const union kvm_smram *smram) +{ + if (unlikely(is_td_vcpu(vcpu))) + return tdx_leave_smm(vcpu, smram); + + return vmx_leave_smm(vcpu, smram); +} + +static void vt_enable_smi_window(struct kvm_vcpu *vcpu) +{ + if (is_td_vcpu(vcpu)) { + tdx_enable_smi_window(vcpu); + return; + } + + /* RSM will cause a vmexit anyway. */ + vmx_enable_smi_window(vcpu); +} +#endif + static void vt_apicv_pre_state_restore(struct kvm_vcpu *vcpu) { struct pi_desc *pi = vcpu_to_pi_desc(vcpu); @@ -700,10 +737,10 @@ struct kvm_x86_ops vt_x86_ops __initdata = { .setup_mce = vmx_setup_mce, #ifdef CONFIG_KVM_SMM - .smi_allowed = vmx_smi_allowed, - .enter_smm = vmx_enter_smm, - .leave_smm = vmx_leave_smm, - .enable_smi_window = vmx_enable_smi_window, + .smi_allowed = vt_smi_allowed, + .enter_smm = vt_enter_smm, + .leave_smm = vt_leave_smm, + .enable_smi_window = vt_enable_smi_window, #endif .check_emulate_instruction = vmx_check_emulate_instruction, diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c index 725cb40d0814..d9b36373e7d0 100644 --- a/arch/x86/kvm/vmx/tdx.c +++ b/arch/x86/kvm/vmx/tdx.c @@ -2044,6 +2044,35 @@ int tdx_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr) } } +#ifdef CONFIG_KVM_SMM +int tdx_smi_allowed(struct kvm_vcpu *vcpu, bool for_injection) +{ + /* SMI isn't supported for TDX. */ + WARN_ON_ONCE(1); + return false; +} + +int tdx_enter_smm(struct kvm_vcpu *vcpu, union kvm_smram *smram) +{ + /* smi_allowed() is always false for TDX as above. */ + WARN_ON_ONCE(1); + return 0; +} + +int tdx_leave_smm(struct kvm_vcpu *vcpu, const union kvm_smram *smram) +{ + WARN_ON_ONCE(1); + return 0; +} + +void tdx_enable_smi_window(struct kvm_vcpu *vcpu) +{ + /* SMI isn't supported for TDX. Silently discard SMI request. */ + WARN_ON_ONCE(1); + vcpu->arch.smi_pending = false; +} +#endif + static int tdx_get_capabilities(struct kvm_tdx_cmd *cmd) { struct kvm_tdx_capabilities __user *user_caps; diff --git a/arch/x86/kvm/vmx/x86_ops.h b/arch/x86/kvm/vmx/x86_ops.h index 017a73ab34bb..7c63b2b48125 100644 --- a/arch/x86/kvm/vmx/x86_ops.h +++ b/arch/x86/kvm/vmx/x86_ops.h @@ -241,4 +241,16 @@ int tdx_pre_memory_mapping(struct kvm_vcpu *vcpu, void tdx_post_memory_mapping(struct kvm_vcpu *vcpu, struct kvm_memory_mapping *mapping) {} #endif +#if defined(CONFIG_INTEL_TDX_HOST) && defined(CONFIG_KVM_SMM) +int tdx_smi_allowed(struct kvm_vcpu *vcpu, bool for_injection); +int tdx_enter_smm(struct kvm_vcpu *vcpu, union kvm_smram *smram); +int tdx_leave_smm(struct kvm_vcpu *vcpu, const union kvm_smram *smram); +void tdx_enable_smi_window(struct kvm_vcpu *vcpu); +#else +static inline int tdx_smi_allowed(struct kvm_vcpu *vcpu, bool for_injection) { return false; } +static inline int tdx_enter_smm(struct kvm_vcpu *vcpu, union kvm_smram *smram) { return 0; } +static inline int tdx_leave_smm(struct kvm_vcpu *vcpu, const union kvm_smram *smram) { return 0; } +static inline void tdx_enable_smi_window(struct kvm_vcpu *vcpu) {} +#endif + #endif /* __KVM_X86_VMX_X86_OPS_H */ From patchwork Mon Feb 26 08:26:59 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571551 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 85E7B12C554; Mon, 26 Feb 2024 08:29:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.8 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936154; cv=none; b=oyPLXDCLEMqt4qqz0N0wpWAjWuSA2vpGKMvRadRuULU2yJhA4F0JdiTqv30ada4h2icb9UZoj9oOQ+W68Q4KBwVq9CjuA0u+n0EbglREQ7xCnKVwuVRvuYVshonx4Alr5saL8+S/JVp4oH+jrAlH7heRAk8J66yxBKz1rm+QCUA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936154; c=relaxed/simple; bh=IanTwbMXzU7eQIlk95oQkjfIypB9z/y8b6X5LS33rsU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=MKkbhrCwOnILgY+Arpig4YCWsPfIuPT30LNerZz/V/1XCaaSC7jWslXWmEAOYrfcro1xjrToU0gg1kC2KJnNyqok/gV8kEB0JNeGOPGq5Hd8fJ8RsJbqrStS97bejVAsohqCyHIgkbt6BMKhNW5LrPWHqbcN7KmWw1HZZbIyDCw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=kCfUYNtK; arc=none smtp.client-ip=192.198.163.8 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="kCfUYNtK" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936152; x=1740472152; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=IanTwbMXzU7eQIlk95oQkjfIypB9z/y8b6X5LS33rsU=; b=kCfUYNtKb8swgtV9xtJ2f4n5fV0Ybbx4QDKV1XIv8s0KiEbFYX64JX0u wKIEnLziT3OyWh+4T1SCUgWk94T1JZeNKgx49AhFvC/FP/kylrB0h3++q wHuGTzPqt1T6VCzwGbJ1UGlMUslJbmJHhWTQuE5EEuozZD926cJgyHI8m 0higQPVAOwg0jMSJQrIgJ3Lj3xwb3m2/FnHRl68cS+1wQnpSzF/aE401j 2bmfFooNS9N0Dwth9dvf02cLy2j+579PxXH/9H1IJhYiMZDrLXu6YOx6s ZEFPbmaHHRHxZyaxvh8z38KsIaZT0CCw0TdUGLupEsSsHvJdnbo9Kdybh g==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="20751375" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="20751375" Received: from fmviesa008.fm.intel.com ([10.60.135.148]) by fmvoesa102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:29:08 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6735102" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by fmviesa008-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:29:07 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 117/130] KVM: TDX: Silently ignore INIT/SIPI Date: Mon, 26 Feb 2024 00:26:59 -0800 Message-Id: <4a4225de42be0f7568c5ecb5c22f2029f8e91d62.1708933498.git.isaku.yamahata@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata The TDX module API doesn't provide API for VMM to inject INIT IPI and SIPI. Instead it defines the different protocols to boot application processors. Ignore INIT and SIPI events for the TDX guest. There are two options. 1) (silently) ignore INIT/SIPI request or 2) return error to guest TDs somehow. Given that TDX guest is paravirtualized to boot AP, the option 1 is chosen for simplicity. Signed-off-by: Isaku Yamahata --- arch/x86/include/asm/kvm-x86-ops.h | 1 + arch/x86/include/asm/kvm_host.h | 2 ++ arch/x86/kvm/lapic.c | 19 +++++++++++------- arch/x86/kvm/svm/svm.c | 1 + arch/x86/kvm/vmx/main.c | 32 ++++++++++++++++++++++++++++-- arch/x86/kvm/vmx/tdx.c | 4 ++-- 6 files changed, 48 insertions(+), 11 deletions(-) diff --git a/arch/x86/include/asm/kvm-x86-ops.h b/arch/x86/include/asm/kvm-x86-ops.h index 22d93d4124c8..85c04aad6ab3 100644 --- a/arch/x86/include/asm/kvm-x86-ops.h +++ b/arch/x86/include/asm/kvm-x86-ops.h @@ -149,6 +149,7 @@ KVM_X86_OP_OPTIONAL(migrate_timers) KVM_X86_OP(msr_filter_changed) KVM_X86_OP(complete_emulated_msr) KVM_X86_OP(vcpu_deliver_sipi_vector) +KVM_X86_OP(vcpu_deliver_init) KVM_X86_OP_OPTIONAL_RET0(vcpu_get_apicv_inhibit_reasons); KVM_X86_OP_OPTIONAL(get_untagged_addr) KVM_X86_OP_OPTIONAL_RET0(gmem_max_level) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index bb8be091f996..2686c080820b 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1836,6 +1836,7 @@ struct kvm_x86_ops { int (*complete_emulated_msr)(struct kvm_vcpu *vcpu, int err); void (*vcpu_deliver_sipi_vector)(struct kvm_vcpu *vcpu, u8 vector); + void (*vcpu_deliver_init)(struct kvm_vcpu *vcpu); /* * Returns vCPU specific APICv inhibit reasons @@ -2092,6 +2093,7 @@ void kvm_get_segment(struct kvm_vcpu *vcpu, struct kvm_segment *var, int seg); void kvm_set_segment(struct kvm_vcpu *vcpu, struct kvm_segment *var, int seg); int kvm_load_segment_descriptor(struct kvm_vcpu *vcpu, u16 selector, int seg); void kvm_vcpu_deliver_sipi_vector(struct kvm_vcpu *vcpu, u8 vector); +void kvm_vcpu_deliver_init(struct kvm_vcpu *vcpu); int kvm_task_switch(struct kvm_vcpu *vcpu, u16 tss_selector, int idt_index, int reason, bool has_error_code, u32 error_code); diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c index 8025c7f614e0..431074679e83 100644 --- a/arch/x86/kvm/lapic.c +++ b/arch/x86/kvm/lapic.c @@ -3268,6 +3268,16 @@ int kvm_lapic_set_pv_eoi(struct kvm_vcpu *vcpu, u64 data, unsigned long len) return 0; } +void kvm_vcpu_deliver_init(struct kvm_vcpu *vcpu) +{ + kvm_vcpu_reset(vcpu, true); + if (kvm_vcpu_is_bsp(vcpu)) + vcpu->arch.mp_state = KVM_MP_STATE_RUNNABLE; + else + vcpu->arch.mp_state = KVM_MP_STATE_INIT_RECEIVED; +} +EXPORT_SYMBOL_GPL(kvm_vcpu_deliver_init); + int kvm_apic_accept_events(struct kvm_vcpu *vcpu) { struct kvm_lapic *apic = vcpu->arch.apic; @@ -3299,13 +3309,8 @@ int kvm_apic_accept_events(struct kvm_vcpu *vcpu) return 0; } - if (test_and_clear_bit(KVM_APIC_INIT, &apic->pending_events)) { - kvm_vcpu_reset(vcpu, true); - if (kvm_vcpu_is_bsp(apic->vcpu)) - vcpu->arch.mp_state = KVM_MP_STATE_RUNNABLE; - else - vcpu->arch.mp_state = KVM_MP_STATE_INIT_RECEIVED; - } + if (test_and_clear_bit(KVM_APIC_INIT, &apic->pending_events)) + static_call(kvm_x86_vcpu_deliver_init)(vcpu); if (test_and_clear_bit(KVM_APIC_SIPI, &apic->pending_events)) { if (vcpu->arch.mp_state == KVM_MP_STATE_INIT_RECEIVED) { /* evaluate pending_events before reading the vector */ diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c index f76dd52d29ba..27546d993809 100644 --- a/arch/x86/kvm/svm/svm.c +++ b/arch/x86/kvm/svm/svm.c @@ -5037,6 +5037,7 @@ static struct kvm_x86_ops svm_x86_ops __initdata = { .complete_emulated_msr = svm_complete_emulated_msr, .vcpu_deliver_sipi_vector = svm_vcpu_deliver_sipi_vector, + .vcpu_deliver_init = kvm_vcpu_deliver_init, .vcpu_get_apicv_inhibit_reasons = avic_vcpu_get_apicv_inhibit_reasons, }; diff --git a/arch/x86/kvm/vmx/main.c b/arch/x86/kvm/vmx/main.c index 4f3b872cd401..84d2dc818cf7 100644 --- a/arch/x86/kvm/vmx/main.c +++ b/arch/x86/kvm/vmx/main.c @@ -320,6 +320,14 @@ static void vt_enable_smi_window(struct kvm_vcpu *vcpu) } #endif +static bool vt_apic_init_signal_blocked(struct kvm_vcpu *vcpu) +{ + if (is_td_vcpu(vcpu)) + return true; + + return vmx_apic_init_signal_blocked(vcpu); +} + static void vt_apicv_pre_state_restore(struct kvm_vcpu *vcpu) { struct pi_desc *pi = vcpu_to_pi_desc(vcpu); @@ -348,6 +356,25 @@ static void vt_deliver_interrupt(struct kvm_lapic *apic, int delivery_mode, vmx_deliver_interrupt(apic, delivery_mode, trig_mode, vector); } +static void vt_vcpu_deliver_sipi_vector(struct kvm_vcpu *vcpu, u8 vector) +{ + if (is_td_vcpu(vcpu)) + return; + + kvm_vcpu_deliver_sipi_vector(vcpu, vector); +} + +static void vt_vcpu_deliver_init(struct kvm_vcpu *vcpu) +{ + if (is_td_vcpu(vcpu)) { + /* TDX doesn't support INIT. Ignore INIT event */ + vcpu->arch.mp_state = KVM_MP_STATE_RUNNABLE; + return; + } + + kvm_vcpu_deliver_init(vcpu); +} + static void vt_flush_tlb_all(struct kvm_vcpu *vcpu) { if (is_td_vcpu(vcpu)) { @@ -744,13 +771,14 @@ struct kvm_x86_ops vt_x86_ops __initdata = { #endif .check_emulate_instruction = vmx_check_emulate_instruction, - .apic_init_signal_blocked = vmx_apic_init_signal_blocked, + .apic_init_signal_blocked = vt_apic_init_signal_blocked, .migrate_timers = vmx_migrate_timers, .msr_filter_changed = vt_msr_filter_changed, .complete_emulated_msr = kvm_complete_insn_gp, - .vcpu_deliver_sipi_vector = kvm_vcpu_deliver_sipi_vector, + .vcpu_deliver_sipi_vector = vt_vcpu_deliver_sipi_vector, + .vcpu_deliver_init = vt_vcpu_deliver_init, .get_untagged_addr = vmx_get_untagged_addr, diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c index d9b36373e7d0..4c7c83105342 100644 --- a/arch/x86/kvm/vmx/tdx.c +++ b/arch/x86/kvm/vmx/tdx.c @@ -769,8 +769,8 @@ void tdx_vcpu_free(struct kvm_vcpu *vcpu) void tdx_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event) { - /* Ignore INIT silently because TDX doesn't support INIT event. */ - if (init_event) + /* vcpu_deliver_init method silently discards INIT event. */ + if (KVM_BUG_ON(init_event, vcpu->kvm)) return; if (KVM_BUG_ON(is_td_vcpu_created(to_tdx(vcpu)), vcpu->kvm)) return; From patchwork Mon Feb 26 08:27:00 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571552 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DD7AE12C7EE; Mon, 26 Feb 2024 08:29:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.8 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936155; cv=none; b=D5DYsdjTq32dzjqzlNrO/BKgXeLQaKNI239yO0sssEnzRaT6jOLWkzTRcboSsbd5sku6Afd5J3DWYdax13s6i/zI4v/ndiRzQxHa7f/5iksZYgaaH4mvM7VuGd5m/fGLerVrgBMsxBY30QvEDWresVW9KPxNGJBV6zFGRlvCS2s= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936155; c=relaxed/simple; bh=bP4SJ1xMJqPCNj8F3nXMMkFsZOZRA7vjzW3+nPuTcfs=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=J6yVJxVPefBnceurB17EmpNy/aKdHa5WBbk4Sx/um4YLGykdoAF4aPziiB6G8bBumXl6p5Mp2686AHRnwfDtGog2hPIrlozNOZ3RrXpr+022etf+hmP7dgwYayO9F3VxyoSkAlSx+uQBoFcy0cJ+6OZxvYcETsI2BGJEZLke6w8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=Z6qZyHh3; arc=none smtp.client-ip=192.198.163.8 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="Z6qZyHh3" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936153; x=1740472153; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=bP4SJ1xMJqPCNj8F3nXMMkFsZOZRA7vjzW3+nPuTcfs=; b=Z6qZyHh3gmigIFWLmM/3eMajFuQ15rY2DMCN6pqbwu4KISWFqmj3t8av nMysOr7NzJ9u7EYB7cxvP+egmYnfU2CvhqAIIfPph3MTilKQVoqJbypZu 43Nq55Jpj+LtGCYEHsxN5GsCAz7IMKD9zs/tgFG8LmoLjraAyoPT8yKbc isqmBe55E35Fb+GUPsD5qZLhQWvYC5zDjcy5sUj0BnX9BxLHa7GvaCc1v rrp/UAiXLvH16J3PpPuTO39VuY3cFexzz7CKWEOdN2D8SYKuare/9T29X 7ypgXsILmAQ0sFCVCow3yh7djymK6mdgfN8eR8HuMte9cRzA2huSH3BjN g==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="20751379" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="20751379" Received: from fmviesa008.fm.intel.com ([10.60.135.148]) by fmvoesa102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:29:08 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6735106" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by fmviesa008-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:29:08 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 118/130] KVM: TDX: Add methods to ignore accesses to CPU state Date: Mon, 26 Feb 2024 00:27:00 -0800 Message-Id: <9258b492295c0ef953f36b9fee60bf3a1873d71b.1708933498.git.isaku.yamahata@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata TDX protects TDX guest state from VMM. Implement access methods for TDX guest state to ignore them or return zero. Because those methods can be called by kvm ioctls to set/get cpu registers, they don't have KVM_BUG_ON except one method. Signed-off-by: Isaku Yamahata --- arch/x86/kvm/vmx/main.c | 289 +++++++++++++++++++++++++++++++++---- arch/x86/kvm/vmx/tdx.c | 48 +++++- arch/x86/kvm/vmx/x86_ops.h | 13 ++ 3 files changed, 321 insertions(+), 29 deletions(-) diff --git a/arch/x86/kvm/vmx/main.c b/arch/x86/kvm/vmx/main.c index 84d2dc818cf7..9fb3f28d8259 100644 --- a/arch/x86/kvm/vmx/main.c +++ b/arch/x86/kvm/vmx/main.c @@ -375,6 +375,200 @@ static void vt_vcpu_deliver_init(struct kvm_vcpu *vcpu) kvm_vcpu_deliver_init(vcpu); } +static void vt_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu) +{ + if (is_td_vcpu(vcpu)) + return; + + vmx_vcpu_after_set_cpuid(vcpu); +} + +static void vt_update_exception_bitmap(struct kvm_vcpu *vcpu) +{ + if (is_td_vcpu(vcpu)) + return; + + vmx_update_exception_bitmap(vcpu); +} + +static u64 vt_get_segment_base(struct kvm_vcpu *vcpu, int seg) +{ + if (is_td_vcpu(vcpu)) + return tdx_get_segment_base(vcpu, seg); + + return vmx_get_segment_base(vcpu, seg); +} + +static void vt_get_segment(struct kvm_vcpu *vcpu, struct kvm_segment *var, + int seg) +{ + if (is_td_vcpu(vcpu)) { + tdx_get_segment(vcpu, var, seg); + return; + } + + vmx_get_segment(vcpu, var, seg); +} + +static void vt_set_segment(struct kvm_vcpu *vcpu, struct kvm_segment *var, + int seg) +{ + if (is_td_vcpu(vcpu)) + return; + + vmx_set_segment(vcpu, var, seg); +} + +static int vt_get_cpl(struct kvm_vcpu *vcpu) +{ + if (is_td_vcpu(vcpu)) + return tdx_get_cpl(vcpu); + + return vmx_get_cpl(vcpu); +} + +static void vt_get_cs_db_l_bits(struct kvm_vcpu *vcpu, int *db, int *l) +{ + if (is_td_vcpu(vcpu)) { + *db = 0; + *l = 0; + return; + } + + vmx_get_cs_db_l_bits(vcpu, db, l); +} + +static bool vt_is_valid_cr0(struct kvm_vcpu *vcpu, unsigned long cr0) +{ + if (is_td_vcpu(vcpu)) + return true; + + return vmx_is_valid_cr0(vcpu, cr0); +} + +static void vt_set_cr0(struct kvm_vcpu *vcpu, unsigned long cr0) +{ + if (is_td_vcpu(vcpu)) + return; + + vmx_set_cr0(vcpu, cr0); +} + +static bool vt_is_valid_cr4(struct kvm_vcpu *vcpu, unsigned long cr4) +{ + if (is_td_vcpu(vcpu)) + return true; + + return vmx_is_valid_cr4(vcpu, cr4); +} + +static void vt_set_cr4(struct kvm_vcpu *vcpu, unsigned long cr4) +{ + if (is_td_vcpu(vcpu)) + return; + + vmx_set_cr4(vcpu, cr4); +} + +static int vt_set_efer(struct kvm_vcpu *vcpu, u64 efer) +{ + if (is_td_vcpu(vcpu)) + return 0; + + return vmx_set_efer(vcpu, efer); +} + +static void vt_get_idt(struct kvm_vcpu *vcpu, struct desc_ptr *dt) +{ + if (is_td_vcpu(vcpu)) { + memset(dt, 0, sizeof(*dt)); + return; + } + + vmx_get_idt(vcpu, dt); +} + +static void vt_set_idt(struct kvm_vcpu *vcpu, struct desc_ptr *dt) +{ + if (is_td_vcpu(vcpu)) + return; + + vmx_set_idt(vcpu, dt); +} + +static void vt_get_gdt(struct kvm_vcpu *vcpu, struct desc_ptr *dt) +{ + if (is_td_vcpu(vcpu)) { + memset(dt, 0, sizeof(*dt)); + return; + } + + vmx_get_gdt(vcpu, dt); +} + +static void vt_set_gdt(struct kvm_vcpu *vcpu, struct desc_ptr *dt) +{ + if (is_td_vcpu(vcpu)) + return; + + vmx_set_gdt(vcpu, dt); +} + +static void vt_set_dr7(struct kvm_vcpu *vcpu, unsigned long val) +{ + if (is_td_vcpu(vcpu)) + return; + + vmx_set_dr7(vcpu, val); +} + +static void vt_sync_dirty_debug_regs(struct kvm_vcpu *vcpu) +{ + /* + * MOV-DR exiting is always cleared for TD guest, even in debug mode. + * Thus KVM_DEBUGREG_WONT_EXIT can never be set and it should never + * reach here for TD vcpu. + */ + if (KVM_BUG_ON(is_td_vcpu(vcpu), vcpu->kvm)) + return; + + vmx_sync_dirty_debug_regs(vcpu); +} + +static void vt_cache_reg(struct kvm_vcpu *vcpu, enum kvm_reg reg) +{ + if (is_td_vcpu(vcpu)) { + tdx_cache_reg(vcpu, reg); + return; + } + + vmx_cache_reg(vcpu, reg); +} + +static unsigned long vt_get_rflags(struct kvm_vcpu *vcpu) +{ + if (is_td_vcpu(vcpu)) + return tdx_get_rflags(vcpu); + + return vmx_get_rflags(vcpu); +} + +static void vt_set_rflags(struct kvm_vcpu *vcpu, unsigned long rflags) +{ + if (is_td_vcpu(vcpu)) + return; + + vmx_set_rflags(vcpu, rflags); +} + +static bool vt_get_if_flag(struct kvm_vcpu *vcpu) +{ + if (is_td_vcpu(vcpu)) + return false; + + return vmx_get_if_flag(vcpu); +} + static void vt_flush_tlb_all(struct kvm_vcpu *vcpu) { if (is_td_vcpu(vcpu)) { @@ -521,6 +715,14 @@ static void vt_inject_irq(struct kvm_vcpu *vcpu, bool reinjected) vmx_inject_irq(vcpu, reinjected); } +static void vt_inject_exception(struct kvm_vcpu *vcpu) +{ + if (is_td_vcpu(vcpu)) + return; + + vmx_inject_exception(vcpu); +} + static void vt_cancel_injection(struct kvm_vcpu *vcpu) { if (is_td_vcpu(vcpu)) @@ -567,6 +769,39 @@ static void vt_get_exit_info(struct kvm_vcpu *vcpu, u32 *reason, vmx_get_exit_info(vcpu, reason, info1, info2, intr_info, error_code); } + +static void vt_update_cr8_intercept(struct kvm_vcpu *vcpu, int tpr, int irr) +{ + if (is_td_vcpu(vcpu)) + return; + + vmx_update_cr8_intercept(vcpu, tpr, irr); +} + +static void vt_load_eoi_exitmap(struct kvm_vcpu *vcpu, u64 *eoi_exit_bitmap) +{ + if (is_td_vcpu(vcpu)) + return; + + vmx_load_eoi_exitmap(vcpu, eoi_exit_bitmap); +} + +static int vt_set_tss_addr(struct kvm *kvm, unsigned int addr) +{ + if (is_td(kvm)) + return 0; + + return vmx_set_tss_addr(kvm, addr); +} + +static int vt_set_identity_map_addr(struct kvm *kvm, u64 ident_addr) +{ + if (is_td(kvm)) + return 0; + + return vmx_set_identity_map_addr(kvm, ident_addr); +} + static u8 vt_get_mt_mask(struct kvm_vcpu *vcpu, gfn_t gfn, bool is_mmio) { if (is_td_vcpu(vcpu)) @@ -661,30 +896,30 @@ struct kvm_x86_ops vt_x86_ops __initdata = { .vcpu_load = vt_vcpu_load, .vcpu_put = vt_vcpu_put, - .update_exception_bitmap = vmx_update_exception_bitmap, + .update_exception_bitmap = vt_update_exception_bitmap, .get_msr_feature = vmx_get_msr_feature, .get_msr = vt_get_msr, .set_msr = vt_set_msr, - .get_segment_base = vmx_get_segment_base, - .get_segment = vmx_get_segment, - .set_segment = vmx_set_segment, - .get_cpl = vmx_get_cpl, - .get_cs_db_l_bits = vmx_get_cs_db_l_bits, - .is_valid_cr0 = vmx_is_valid_cr0, - .set_cr0 = vmx_set_cr0, - .is_valid_cr4 = vmx_is_valid_cr4, - .set_cr4 = vmx_set_cr4, - .set_efer = vmx_set_efer, - .get_idt = vmx_get_idt, - .set_idt = vmx_set_idt, - .get_gdt = vmx_get_gdt, - .set_gdt = vmx_set_gdt, - .set_dr7 = vmx_set_dr7, - .sync_dirty_debug_regs = vmx_sync_dirty_debug_regs, - .cache_reg = vmx_cache_reg, - .get_rflags = vmx_get_rflags, - .set_rflags = vmx_set_rflags, - .get_if_flag = vmx_get_if_flag, + .get_segment_base = vt_get_segment_base, + .get_segment = vt_get_segment, + .set_segment = vt_set_segment, + .get_cpl = vt_get_cpl, + .get_cs_db_l_bits = vt_get_cs_db_l_bits, + .is_valid_cr0 = vt_is_valid_cr0, + .set_cr0 = vt_set_cr0, + .is_valid_cr4 = vt_is_valid_cr4, + .set_cr4 = vt_set_cr4, + .set_efer = vt_set_efer, + .get_idt = vt_get_idt, + .set_idt = vt_set_idt, + .get_gdt = vt_get_gdt, + .set_gdt = vt_set_gdt, + .set_dr7 = vt_set_dr7, + .sync_dirty_debug_regs = vt_sync_dirty_debug_regs, + .cache_reg = vt_cache_reg, + .get_rflags = vt_get_rflags, + .set_rflags = vt_set_rflags, + .get_if_flag = vt_get_if_flag, .flush_tlb_all = vt_flush_tlb_all, .flush_tlb_current = vt_flush_tlb_current, @@ -701,7 +936,7 @@ struct kvm_x86_ops vt_x86_ops __initdata = { .patch_hypercall = vmx_patch_hypercall, .inject_irq = vt_inject_irq, .inject_nmi = vt_inject_nmi, - .inject_exception = vmx_inject_exception, + .inject_exception = vt_inject_exception, .cancel_injection = vt_cancel_injection, .interrupt_allowed = vt_interrupt_allowed, .nmi_allowed = vt_nmi_allowed, @@ -709,11 +944,11 @@ struct kvm_x86_ops vt_x86_ops __initdata = { .set_nmi_mask = vt_set_nmi_mask, .enable_nmi_window = vt_enable_nmi_window, .enable_irq_window = vt_enable_irq_window, - .update_cr8_intercept = vmx_update_cr8_intercept, + .update_cr8_intercept = vt_update_cr8_intercept, .set_virtual_apic_mode = vmx_set_virtual_apic_mode, .set_apic_access_page_addr = vmx_set_apic_access_page_addr, .refresh_apicv_exec_ctrl = vmx_refresh_apicv_exec_ctrl, - .load_eoi_exitmap = vmx_load_eoi_exitmap, + .load_eoi_exitmap = vt_load_eoi_exitmap, .apicv_pre_state_restore = vt_apicv_pre_state_restore, .required_apicv_inhibits = VMX_REQUIRED_APICV_INHIBITS, .hwapic_irr_update = vmx_hwapic_irr_update, @@ -724,13 +959,13 @@ struct kvm_x86_ops vt_x86_ops __initdata = { .dy_apicv_has_pending_interrupt = pi_has_pending_interrupt, .protected_apic_has_interrupt = vt_protected_apic_has_interrupt, - .set_tss_addr = vmx_set_tss_addr, - .set_identity_map_addr = vmx_set_identity_map_addr, + .set_tss_addr = vt_set_tss_addr, + .set_identity_map_addr = vt_set_identity_map_addr, .get_mt_mask = vt_get_mt_mask, .get_exit_info = vt_get_exit_info, - .vcpu_after_set_cpuid = vmx_vcpu_after_set_cpuid, + .vcpu_after_set_cpuid = vt_vcpu_after_set_cpuid, .has_wbinvd_exit = cpu_has_vmx_wbinvd_exit, diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c index 4c7c83105342..fc78e9ca61c1 100644 --- a/arch/x86/kvm/vmx/tdx.c +++ b/arch/x86/kvm/vmx/tdx.c @@ -639,8 +639,15 @@ int tdx_vcpu_create(struct kvm_vcpu *vcpu) vcpu->arch.tsc_offset = to_kvm_tdx(vcpu->kvm)->tsc_offset; vcpu->arch.l1_tsc_offset = vcpu->arch.tsc_offset; - vcpu->arch.guest_state_protected = - !(to_kvm_tdx(vcpu->kvm)->attributes & TDX_TD_ATTRIBUTE_DEBUG); + /* + * TODO: support off-TD debug. If TD DEBUG is enabled, guest state + * can be accessed. guest_state_protected = false. and kvm ioctl to + * access CPU states should be usable for user space VMM (e.g. qemu). + * + * vcpu->arch.guest_state_protected = + * !(to_kvm_tdx(vcpu->kvm)->attributes & TDX_TD_ATTRIBUTE_DEBUG); + */ + vcpu->arch.guest_state_protected = true; if ((kvm_tdx->xfam & XFEATURE_MASK_XTILE) == XFEATURE_MASK_XTILE) vcpu->arch.xfd_no_write_intercept = true; @@ -2073,6 +2080,43 @@ void tdx_enable_smi_window(struct kvm_vcpu *vcpu) } #endif +int tdx_get_cpl(struct kvm_vcpu *vcpu) +{ + return 0; +} + +void tdx_cache_reg(struct kvm_vcpu *vcpu, enum kvm_reg reg) +{ + kvm_register_mark_available(vcpu, reg); + switch (reg) { + case VCPU_REGS_RSP: + case VCPU_REGS_RIP: + case VCPU_EXREG_PDPTR: + case VCPU_EXREG_CR0: + case VCPU_EXREG_CR3: + case VCPU_EXREG_CR4: + break; + default: + KVM_BUG_ON(1, vcpu->kvm); + break; + } +} + +unsigned long tdx_get_rflags(struct kvm_vcpu *vcpu) +{ + return 0; +} + +u64 tdx_get_segment_base(struct kvm_vcpu *vcpu, int seg) +{ + return 0; +} + +void tdx_get_segment(struct kvm_vcpu *vcpu, struct kvm_segment *var, int seg) +{ + memset(var, 0, sizeof(*var)); +} + static int tdx_get_capabilities(struct kvm_tdx_cmd *cmd) { struct kvm_tdx_capabilities __user *user_caps; diff --git a/arch/x86/kvm/vmx/x86_ops.h b/arch/x86/kvm/vmx/x86_ops.h index 7c63b2b48125..727c4d418601 100644 --- a/arch/x86/kvm/vmx/x86_ops.h +++ b/arch/x86/kvm/vmx/x86_ops.h @@ -169,6 +169,12 @@ bool tdx_has_emulated_msr(u32 index, bool write); int tdx_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr); int tdx_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr); +int tdx_get_cpl(struct kvm_vcpu *vcpu); +void tdx_cache_reg(struct kvm_vcpu *vcpu, enum kvm_reg reg); +unsigned long tdx_get_rflags(struct kvm_vcpu *vcpu); +u64 tdx_get_segment_base(struct kvm_vcpu *vcpu, int seg); +void tdx_get_segment(struct kvm_vcpu *vcpu, struct kvm_segment *var, int seg); + int tdx_vcpu_ioctl(struct kvm_vcpu *vcpu, void __user *argp); void tdx_flush_tlb(struct kvm_vcpu *vcpu); @@ -221,6 +227,13 @@ static inline bool tdx_has_emulated_msr(u32 index, bool write) { return false; } static inline int tdx_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr) { return 1; } static inline int tdx_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr) { return 1; } +static inline int tdx_get_cpl(struct kvm_vcpu *vcpu) { return 0; } +static inline void tdx_cache_reg(struct kvm_vcpu *vcpu, enum kvm_reg reg) {} +static inline unsigned long tdx_get_rflags(struct kvm_vcpu *vcpu) { return 0; } +static inline u64 tdx_get_segment_base(struct kvm_vcpu *vcpu, int seg) { return 0; } +static inline void tdx_get_segment(struct kvm_vcpu *vcpu, struct kvm_segment *var, + int seg) {} + static inline int tdx_vcpu_ioctl(struct kvm_vcpu *vcpu, void __user *argp) { return -EOPNOTSUPP; } static inline void tdx_flush_tlb(struct kvm_vcpu *vcpu) {} From patchwork Mon Feb 26 08:27:01 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571555 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C157912CDB7; Mon, 26 Feb 2024 08:29:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.8 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936158; cv=none; b=YzpKMa1qrlAago7RAXJP7/UyIj637MoAJmUoyg0u5rgLZBDuNtKpBdfPGAEdc5IPC6BcVlnvWs8IdqJVEvLzIsiFTrkszD280jC51V6A0q+5Xr/gMYOOQgjzqVRSpi3f8b7epE0bwg/AVfKVSj7wrOuA93PLvU7snLc1YBvIxao= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936158; c=relaxed/simple; bh=CFO7sbhVxx28cd10/6BY0pfLdF4OfRuPbI0qHxYmeIY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=FPO6oTKHlOWkCVrFewAXYylPRl14z7FT+QXl83HXvhEcUk85daDBWAeqlFcu3OB8d7BURIYaVKiTiaFcNjxYlIY8u/VvpGWjo+ZcXKmUIEJH65PA1Led1F82O1TL9a/9BpGv85t39h4RO6WA9fryrb7m15ECI4qEo0I/rDaJ060= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=emdlqhXg; arc=none smtp.client-ip=192.198.163.8 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="emdlqhXg" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936155; x=1740472155; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=CFO7sbhVxx28cd10/6BY0pfLdF4OfRuPbI0qHxYmeIY=; b=emdlqhXgkPffBQ+eavdxnX3TwtCPwRWdfTGljCK7jLsIjyTGaxZRame8 Ss89Vc4Oc5z1meMeKXlhMTIXFqhnJjJihdesEX+VQ5W0AJjgGSPTgn8no wF89RdZZJxAIo3zPFOwt+llM1yn/JdImlhPYYkRVqwoq7WSKUH4DrdvaO kO4NY+2JeT2mOvEIhSm/zCXPVHzcflrhJD/we4lQtzHtQBbKy7b1TtyiY z9fTeNWHHC1K0i5HCOeWTdnoecxmdCXJp1MfGyBf2kBgviwRF2hG50gwk fhmyAVc9VLnBy7BNY31ocTT1hSKMeQc/P3WpO+xpUJn0AMw7jg0s7YrWy w==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="20751383" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="20751383" Received: from fmviesa008.fm.intel.com ([10.60.135.148]) by fmvoesa102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:29:09 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6735113" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by fmviesa008-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:29:09 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 119/130] KVM: TDX: Add methods to ignore guest instruction emulation Date: Mon, 26 Feb 2024 00:27:01 -0800 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata Because TDX protects TDX guest state from VMM, instructions in guest memory cannot be emulated. Implement methods to ignore guest instruction emulator. Signed-off-by: Isaku Yamahata --- arch/x86/kvm/vmx/main.c | 28 ++++++++++++++++++++++++++-- 1 file changed, 26 insertions(+), 2 deletions(-) diff --git a/arch/x86/kvm/vmx/main.c b/arch/x86/kvm/vmx/main.c index 9fb3f28d8259..9aaa9bd7abf3 100644 --- a/arch/x86/kvm/vmx/main.c +++ b/arch/x86/kvm/vmx/main.c @@ -320,6 +320,30 @@ static void vt_enable_smi_window(struct kvm_vcpu *vcpu) } #endif +static int vt_check_emulate_instruction(struct kvm_vcpu *vcpu, int emul_type, + void *insn, int insn_len) +{ + if (is_td_vcpu(vcpu)) + return X86EMUL_RETRY_INSTR; + + return vmx_check_emulate_instruction(vcpu, emul_type, insn, insn_len); +} + +static int vt_check_intercept(struct kvm_vcpu *vcpu, + struct x86_instruction_info *info, + enum x86_intercept_stage stage, + struct x86_exception *exception) +{ + /* + * This call back is triggered by the x86 instruction emulator. TDX + * doesn't allow guest memory inspection. + */ + if (KVM_BUG_ON(is_td_vcpu(vcpu), vcpu->kvm)) + return X86EMUL_UNHANDLEABLE; + + return vmx_check_intercept(vcpu, info, stage, exception); +} + static bool vt_apic_init_signal_blocked(struct kvm_vcpu *vcpu) { if (is_td_vcpu(vcpu)) @@ -976,7 +1000,7 @@ struct kvm_x86_ops vt_x86_ops __initdata = { .load_mmu_pgd = vt_load_mmu_pgd, - .check_intercept = vmx_check_intercept, + .check_intercept = vt_check_intercept, .handle_exit_irqoff = vt_handle_exit_irqoff, .request_immediate_exit = vt_request_immediate_exit, @@ -1005,7 +1029,7 @@ struct kvm_x86_ops vt_x86_ops __initdata = { .enable_smi_window = vt_enable_smi_window, #endif - .check_emulate_instruction = vmx_check_emulate_instruction, + .check_emulate_instruction = vt_check_emulate_instruction, .apic_init_signal_blocked = vt_apic_init_signal_blocked, .migrate_timers = vmx_migrate_timers, From patchwork Mon Feb 26 08:27:02 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571553 Received: from mgamail.intel.com (unknown [192.198.163.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C152B12CDB6; Mon, 26 Feb 2024 08:29:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.8 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936156; cv=none; b=tGxRkKGUZSucO1Ky80xu3qxjmjgL53ZIx+RrDd2Igafzu4BiBnksw6pbDJFHrfilzjNMMed0OB5GFLarPcvJmOHv/f+q2r0N9tVuG7nLmvtborJPhyZa2Na8MV6kl+Da2ALpfDtUIzx6/S5kVmhCwE0EgX/AOXzaiaXfIInIwD0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936156; c=relaxed/simple; bh=TGyv4NcpisLqfQGp5DS6OP4mOwYZXVYUosr40c4DaB8=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=pGL7mOF3taQEW5odsgYbkwrvc4toHZezfVEbmlL12UJhHXUjPOIoAbFU2x3xKG3qVvZxdVgdyQ6rN9vs3K6o5kO4XHEGZW2bfIvvqA30t8N+QrhsX6dPboBea+L8vTEQbWCy4ubr2zLwtS73yX46YbRY0OXz6XgsrBl/0gflBaA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=YXl9J4E0; arc=none smtp.client-ip=192.198.163.8 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="YXl9J4E0" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936155; x=1740472155; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=TGyv4NcpisLqfQGp5DS6OP4mOwYZXVYUosr40c4DaB8=; b=YXl9J4E0ymHwNWqQG5EXaIl5CRCVWLYXWp6/32LgwB2ktOnl8RJt1lEv xxwNs/y2GqMj2NAd/cnU760vDji553ees3Z3aJKFlAoctBg6vYQNiN9ED +hNUCjuByAz5sHT2OmHsaZGY4CO3E+do/oqjJHW3tNh3MiktuU14s9r9L C/D0vZ5cnIuvaNNg/EvbP/B7uklsYDMIX63WYJrHLgVHsrBMDmt6Yh05p +Lcq8ts1TIJrqJVEoF/dnuLudAV0BGcCHTMw7EWeoa2qvFN4mJ9CBp/v+ 3LF3j6HqZhQGPvkXE+0qX6CVytOojSL0WjCj8/XvwUSSjvgeQE54qJxUL A==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="20751387" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="20751387" Received: from fmviesa008.fm.intel.com ([10.60.135.148]) by fmvoesa102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:29:09 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6735120" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by fmviesa008-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:29:09 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 120/130] KVM: TDX: Add a method to ignore dirty logging Date: Mon, 26 Feb 2024 00:27:02 -0800 Message-Id: <1491dd247829bf1a29df1904aeed5ed6b464d29c.1708933498.git.isaku.yamahata@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata Currently TDX KVM doesn't support tracking dirty pages (yet). Implement a method to ignore it. Because the flag for kvm memory slot to enable dirty logging isn't accepted for TDX, warn on the method is called for TDX. Signed-off-by: Isaku Yamahata --- arch/x86/kvm/vmx/main.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/arch/x86/kvm/vmx/main.c b/arch/x86/kvm/vmx/main.c index 9aaa9bd7abf3..991bfdecaed2 100644 --- a/arch/x86/kvm/vmx/main.c +++ b/arch/x86/kvm/vmx/main.c @@ -834,6 +834,14 @@ static u8 vt_get_mt_mask(struct kvm_vcpu *vcpu, gfn_t gfn, bool is_mmio) return vmx_get_mt_mask(vcpu, gfn, is_mmio); } +static void vt_update_cpu_dirty_logging(struct kvm_vcpu *vcpu) +{ + if (KVM_BUG_ON(is_td_vcpu(vcpu), vcpu->kvm)) + return; + + vmx_update_cpu_dirty_logging(vcpu); +} + static int vt_mem_enc_ioctl(struct kvm *kvm, void __user *argp) { if (!is_td(kvm)) @@ -1008,7 +1016,7 @@ struct kvm_x86_ops vt_x86_ops __initdata = { .sched_in = vt_sched_in, .cpu_dirty_log_size = PML_ENTITY_NUM, - .update_cpu_dirty_logging = vmx_update_cpu_dirty_logging, + .update_cpu_dirty_logging = vt_update_cpu_dirty_logging, .nested_ops = &vmx_nested_ops, From patchwork Mon Feb 26 08:27:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571554 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 328C712D75C; Mon, 26 Feb 2024 08:29:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.8 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936157; cv=none; b=HAYSw9L3xvvtn7J/Ty0JyqmgPGDUs0VxPc4x7hsypVZqFEFI2m02hY/83WXW/X/lin6f9oBUFPTBa/Rthgj3URsZHGlXwiYs7Pwjra4MFOFOaCmxknruNHvREg55I17S62241UJwOBKSZn3VhaGspJceTo7PwOlwoUtDjr9temc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936157; c=relaxed/simple; bh=0yh1shLe8hwMZzN1bzwtQ5WkdmEz/o+wucegI0YBDwc=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=HfExOVqAdJY5zDQPlZgVoCZMekCNM1rTmMMu2H0oZmTC+ySJan8ZIH2r0OQFzOZd0z/wpJ966RlmXYSzlxW37iaX+gIVbLqpa/YsN4w1llo7BAuHXgZdVkrsJ+1oXccyy60+rmoggVp4oZ3gr0ctWVQEuOraywVAhkjccQB3B/4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=UIza5fJt; arc=none smtp.client-ip=192.198.163.8 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="UIza5fJt" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936155; x=1740472155; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=0yh1shLe8hwMZzN1bzwtQ5WkdmEz/o+wucegI0YBDwc=; b=UIza5fJtF+MSSSNWEh+XJEcn10Yx+Unzbgm57cTWtbhiQQe/fmj4Q6r0 IFPYOpRu9f+t1l+SIY5TmM4FdgNH8DTBCuLEU+0Sfr2VgrNqBt7wuNDwy REo/m09TP3ydaojs3gvZu+M6vQhAmHA9FZEObN4AYwf9MLLbbMqgVQ1qi lmTq6eyYJAg0oDvBSizYNEzo62MXw7XMuiiV2Kh5AZhEnBQTJ9HzWdb9q BC7OeSGs980gUoIxyKheOiyXETtW/J5VOfoQS7DdwW7d6+b/DxfIpLccD 2d3cj0NqHypbx6shOuEuCp4yDCQzVTJFnuZ9QIGhVfgko2aqlNORMrIHp g==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="20751391" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="20751391" Received: from fmviesa008.fm.intel.com ([10.60.135.148]) by fmvoesa102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:29:10 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6735123" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by fmviesa008-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:29:10 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 121/130] KVM: TDX: Add methods to ignore VMX preemption timer Date: Mon, 26 Feb 2024 00:27:03 -0800 Message-Id: <30cd8696d4c6ab2e909b9ded93049ab217c713f8.1708933498.git.isaku.yamahata@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata TDX doesn't support VMX preemption timer. Implement access methods for VMM to ignore VMX preemption timer. Signed-off-by: Isaku Yamahata --- arch/x86/kvm/vmx/main.c | 25 +++++++++++++++++++++++-- 1 file changed, 23 insertions(+), 2 deletions(-) diff --git a/arch/x86/kvm/vmx/main.c b/arch/x86/kvm/vmx/main.c index 991bfdecaed2..ec5c0fda92e9 100644 --- a/arch/x86/kvm/vmx/main.c +++ b/arch/x86/kvm/vmx/main.c @@ -842,6 +842,27 @@ static void vt_update_cpu_dirty_logging(struct kvm_vcpu *vcpu) vmx_update_cpu_dirty_logging(vcpu); } +#ifdef CONFIG_X86_64 +static int vt_set_hv_timer(struct kvm_vcpu *vcpu, u64 guest_deadline_tsc, + bool *expired) +{ + /* VMX-preemption timer isn't available for TDX. */ + if (is_td_vcpu(vcpu)) + return -EINVAL; + + return vmx_set_hv_timer(vcpu, guest_deadline_tsc, expired); +} + +static void vt_cancel_hv_timer(struct kvm_vcpu *vcpu) +{ + /* VMX-preemption timer can't be set. See vt_set_hv_timer(). */ + if (KVM_BUG_ON(is_td_vcpu(vcpu), vcpu->kvm)) + return; + + vmx_cancel_hv_timer(vcpu); +} +#endif + static int vt_mem_enc_ioctl(struct kvm *kvm, void __user *argp) { if (!is_td(kvm)) @@ -1024,8 +1045,8 @@ struct kvm_x86_ops vt_x86_ops __initdata = { .pi_start_assignment = vmx_pi_start_assignment, #ifdef CONFIG_X86_64 - .set_hv_timer = vmx_set_hv_timer, - .cancel_hv_timer = vmx_cancel_hv_timer, + .set_hv_timer = vt_set_hv_timer, + .cancel_hv_timer = vt_cancel_hv_timer, #endif .setup_mce = vmx_setup_mce, From patchwork Mon Feb 26 08:27:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571557 Received: from mgamail.intel.com (unknown [192.198.163.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E591912EBC9; Mon, 26 Feb 2024 08:29:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.8 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936159; cv=none; b=SwR0F63Ihq7Yh44armnywC0qpynOCi1xgAM+BSEtSpgCuxw9hBWoew+VeOezOzPydgmJjXXEgo24tlcGGhr+NgshlL6FwPrJHCw7ynI0QH+AKy/Pu1r37p97TfiqPTb7jr4x3pokuIh5GcdPyTW/nEDl/k5vjC7KGNvgDGJ9MGo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936159; c=relaxed/simple; bh=m7n7i1eS4On2R241lFqJmq0+vkwcQBjfN5ADiMO6W+o=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=JiuN+Fyf2t/OP9h62xhwW7pBUy55Akea2xDSp/VJfZDF66hxaA1Mf7IqIp9DKhQgC6MsNj8Zn1A3NgWzHdNMDaQKJMTd7Q7hzqj8B7qJ9R83v1UT32rxX+zFvYlLwumxuafbPuDJbS8FGIY2XxQWtOOi62uPP1rdD9lAEEzD7Is= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=hSLTQWfo; arc=none smtp.client-ip=192.198.163.8 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="hSLTQWfo" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936157; x=1740472157; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=m7n7i1eS4On2R241lFqJmq0+vkwcQBjfN5ADiMO6W+o=; b=hSLTQWfojQc90/ad0OUazIBxZqbpipGJT4vsEsZ3pmh5yvyf3Twyb8aN XC0kXtFBlrNdQgt0HUKxt/88To1/5Os85T3O8EhAZLbTbyTzGk7entFAC OgVbu1Cxk0QYcvRLMma6OYBFiLstzpVYAOibZU8m1gQTDYQp0575mUkIL CpZc9plHOR2oFxLWRwbsfXfrQX96rw+vrylrYqFLDEmL8nzK3svAdj9Yq Q4/v5ZY0TPjlWlGcx+Zmn0WvwxlRmN6ASZZ4PwDmdsclI1sgUEYiWMxEM IhrKkzpnLNYoJlC2cNmaSsZVm6zzunUtoOSONzm7EcUCfj+yIjXCjJTRE g==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="20751396" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="20751396" Received: from fmviesa008.fm.intel.com ([10.60.135.148]) by fmvoesa102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:29:10 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6735128" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by fmviesa008-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:29:10 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 122/130] KVM: TDX: Add methods to ignore accesses to TSC Date: Mon, 26 Feb 2024 00:27:04 -0800 Message-Id: <59ff42a2dd3769de6127fde30c9b2b355a1a4c17.1708933498.git.isaku.yamahata@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata TDX protects TDX guest TSC state from VMM. Implement access methods to ignore guest TSC. Signed-off-by: Isaku Yamahata --- arch/x86/kvm/vmx/main.c | 44 +++++++++++++++++++++++++++++++++++++---- 1 file changed, 40 insertions(+), 4 deletions(-) diff --git a/arch/x86/kvm/vmx/main.c b/arch/x86/kvm/vmx/main.c index ec5c0fda92e9..9fcd71999bba 100644 --- a/arch/x86/kvm/vmx/main.c +++ b/arch/x86/kvm/vmx/main.c @@ -834,6 +834,42 @@ static u8 vt_get_mt_mask(struct kvm_vcpu *vcpu, gfn_t gfn, bool is_mmio) return vmx_get_mt_mask(vcpu, gfn, is_mmio); } +static u64 vt_get_l2_tsc_offset(struct kvm_vcpu *vcpu) +{ + /* TDX doesn't support L2 guest at the moment. */ + if (KVM_BUG_ON(is_td_vcpu(vcpu), vcpu->kvm)) + return 0; + + return vmx_get_l2_tsc_offset(vcpu); +} + +static u64 vt_get_l2_tsc_multiplier(struct kvm_vcpu *vcpu) +{ + /* TDX doesn't support L2 guest at the moment. */ + if (KVM_BUG_ON(is_td_vcpu(vcpu), vcpu->kvm)) + return 0; + + return vmx_get_l2_tsc_multiplier(vcpu); +} + +static void vt_write_tsc_offset(struct kvm_vcpu *vcpu) +{ + /* In TDX, tsc offset can't be changed. */ + if (is_td_vcpu(vcpu)) + return; + + vmx_write_tsc_offset(vcpu); +} + +static void vt_write_tsc_multiplier(struct kvm_vcpu *vcpu) +{ + /* In TDX, tsc multiplier can't be changed. */ + if (is_td_vcpu(vcpu)) + return; + + vmx_write_tsc_multiplier(vcpu); +} + static void vt_update_cpu_dirty_logging(struct kvm_vcpu *vcpu) { if (KVM_BUG_ON(is_td_vcpu(vcpu), vcpu->kvm)) @@ -1022,10 +1058,10 @@ struct kvm_x86_ops vt_x86_ops __initdata = { .has_wbinvd_exit = cpu_has_vmx_wbinvd_exit, - .get_l2_tsc_offset = vmx_get_l2_tsc_offset, - .get_l2_tsc_multiplier = vmx_get_l2_tsc_multiplier, - .write_tsc_offset = vmx_write_tsc_offset, - .write_tsc_multiplier = vmx_write_tsc_multiplier, + .get_l2_tsc_offset = vt_get_l2_tsc_offset, + .get_l2_tsc_multiplier = vt_get_l2_tsc_multiplier, + .write_tsc_offset = vt_write_tsc_offset, + .write_tsc_multiplier = vt_write_tsc_multiplier, .load_mmu_pgd = vt_load_mmu_pgd, From patchwork Mon Feb 26 08:27:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571556 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 84AD112EBEE; Mon, 26 Feb 2024 08:29:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.8 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936159; cv=none; b=mKkrRLQgtADNlIc5OpAflLQBfydzZTMU26uaZRvI6+Smnm2FxfZcUhnIanJbNlmcIgdP3JZHfWv0r+z8+sWi8owu/Y2/i1beAvLza3Jovw5cDPBuJy3OI6qjCVmgsCbVNNVv3srdhqV34oUxr/3de54PlKNJT8fuqyE5UIO3BbI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936159; c=relaxed/simple; bh=VTw9bzx7BOh5z8zeVoQCv87WXQmA2rn8M3JbvT++IJ4=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Eg82KTqev14zqxTO1OrBPQcH2ZF2DRXdZ5T6BWI4wr14W3Qz7/agPMmPUZTQf/KOEEAtwUyF1mgarfmSNFrvjZ5NbZyRMbwBF2++rL6j79bKzv7GVtsw5C44t5DPyPGloEeNKnbWvTTwS2KgXiNMi/JRU79YnFWFBEgEbn1ONBs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=E7g/7d/T; arc=none smtp.client-ip=192.198.163.8 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="E7g/7d/T" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936157; x=1740472157; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=VTw9bzx7BOh5z8zeVoQCv87WXQmA2rn8M3JbvT++IJ4=; b=E7g/7d/T7KT6NUDUMkVk6qcrhoR8jvLmq3tFPGcTagjl8wdihgJHNL6H W41QB9/8mMgIVeql9OI3T3c90hKmQK7XFRFvEBJ7nvIZACEStifKobWNH lZzTDGCO073r7Tc+pAKhYMwJDl1p4ogfXFpwCtNYW2OU2nx9VgTiRoBHw xnAgIQV6aUwB6DEhXCcc9NSv0pgmgD71DhxlZtbnWwYBwq5jCt5Cr8+5e Lj0U6KrFp6nnX2TprtPcnAg0DCnDmdGpKVYuExBLHQcOBT6XYlieEaUou QCn4+558BfFsjbZpDTt/s9a1e9hlAdZ2pSefA3qAdqK1IbPt61NGG6DZv Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="20751400" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="20751400" Received: from fmviesa008.fm.intel.com ([10.60.135.148]) by fmvoesa102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:29:11 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6735133" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by fmviesa008-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:29:10 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 123/130] KVM: TDX: Ignore setting up mce Date: Mon, 26 Feb 2024 00:27:05 -0800 Message-Id: <52df2fcce3f44911ee736f0abfc62c1ab05c8620.1708933498.git.isaku.yamahata@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata Because vmx_set_mce function is VMX specific and it cannot be used for TDX. Add vt stub to ignore setting up mce for TDX. Signed-off-by: Isaku Yamahata --- arch/x86/kvm/vmx/main.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/arch/x86/kvm/vmx/main.c b/arch/x86/kvm/vmx/main.c index 9fcd71999bba..7c47b02d88d8 100644 --- a/arch/x86/kvm/vmx/main.c +++ b/arch/x86/kvm/vmx/main.c @@ -899,6 +899,14 @@ static void vt_cancel_hv_timer(struct kvm_vcpu *vcpu) } #endif +static void vt_setup_mce(struct kvm_vcpu *vcpu) +{ + if (is_td_vcpu(vcpu)) + return; + + vmx_setup_mce(vcpu); +} + static int vt_mem_enc_ioctl(struct kvm *kvm, void __user *argp) { if (!is_td(kvm)) @@ -1085,7 +1093,7 @@ struct kvm_x86_ops vt_x86_ops __initdata = { .cancel_hv_timer = vt_cancel_hv_timer, #endif - .setup_mce = vmx_setup_mce, + .setup_mce = vt_setup_mce, #ifdef CONFIG_KVM_SMM .smi_allowed = vt_smi_allowed, From patchwork Mon Feb 26 08:27:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571558 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F1C7A12F38A; Mon, 26 Feb 2024 08:29:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.8 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936160; cv=none; b=jxGgeVJvBzwI84uTeVRW5yk0PZy1efzQRYTFyHtRLjAYfKQktB+Xb9SkVrIceLnyAwMc8e1xToDERQm56+csCiYf54Gflry8v9nG+nNsJXl4xZ+QZbPXQbbgr2TQsIiaWIsl84Pu0aT798txvhDWk/4RUh+rPm0oCUXtrqNabXc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936160; c=relaxed/simple; bh=riRqyN0CYgRzjdvvWWc6SvA55IJM4w/WwMYuoEgr/9o=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=peq1TJEJF0trjfSrovK5Nk72FNxprdJ26BtkAVPtkqRZe20bp5gdbZFGcWgNDjiiWopE98vO9NGpz5cxbs0D1u2Iec0TewnPRsRMcNcRRumf1UA8U73HrQtgwN4/j2WB+p64P75gv6+4PoSQz4zxO2h/gCmD30vwAbSbAqomn44= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=GPOfaT4V; arc=none smtp.client-ip=192.198.163.8 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="GPOfaT4V" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936159; x=1740472159; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=riRqyN0CYgRzjdvvWWc6SvA55IJM4w/WwMYuoEgr/9o=; b=GPOfaT4VxkV8Ej8PGWkoCqm+hr6we8qq8ckGhtwEvLUbDAeLoK6t4X1r KtXBsAmjSpPDMs6Rx4XhM0jnEEiuWATa52VOmzB8BoUUIyTxkE74WLA3z z3tqF/Z/a7lWU/mcluX4EgXSBzIzGZaJINhbK7JJjlqGbVem3pDxlYEv2 03BJjfkR0AX7BTqF4I6PREwOppHRq+5VWtj2OiieeJ8Mu87vn469aJ6pF 5oOF/erAkjZkbCZXNUbUeb6+1KWKMBRU350THkh/fBwEF2aax4YjEoORr eSv0IhMKr2z3VxzcGDvHj15i3tSJEEKgc8OcWwP7h9UM+qmL1P/BNzNhA Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="20751404" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="20751404" Received: from fmviesa008.fm.intel.com ([10.60.135.148]) by fmvoesa102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:29:11 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6735137" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by fmviesa008-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:29:11 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 124/130] KVM: TDX: Add a method to ignore for TDX to ignore hypercall patch Date: Mon, 26 Feb 2024 00:27:06 -0800 Message-Id: <7191f6856003e5315c72a9dffc53cafd6e529acd.1708933498.git.isaku.yamahata@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata Because guest TD memory is protected, VMM patching guest binary for hypercall instruction isn't possible. Add a method to ignore hypercall patching with a warning. Note: guest TD kernel needs to be modified to use TDG.VP.VMCALL for hypercall. Signed-off-by: Isaku Yamahata --- arch/x86/kvm/vmx/main.c | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/arch/x86/kvm/vmx/main.c b/arch/x86/kvm/vmx/main.c index 7c47b02d88d8..fae5a3668361 100644 --- a/arch/x86/kvm/vmx/main.c +++ b/arch/x86/kvm/vmx/main.c @@ -731,6 +731,19 @@ static u32 vt_get_interrupt_shadow(struct kvm_vcpu *vcpu) return vmx_get_interrupt_shadow(vcpu); } +static void vt_patch_hypercall(struct kvm_vcpu *vcpu, + unsigned char *hypercall) +{ + /* + * Because guest memory is protected, guest can't be patched. TD kernel + * is modified to use TDG.VP.VMCAL for hypercall. + */ + if (KVM_BUG_ON(is_td_vcpu(vcpu), vcpu->kvm)) + return; + + vmx_patch_hypercall(vcpu, hypercall); +} + static void vt_inject_irq(struct kvm_vcpu *vcpu, bool reinjected) { if (is_td_vcpu(vcpu)) @@ -1030,7 +1043,7 @@ struct kvm_x86_ops vt_x86_ops __initdata = { .update_emulated_instruction = vmx_update_emulated_instruction, .set_interrupt_shadow = vt_set_interrupt_shadow, .get_interrupt_shadow = vt_get_interrupt_shadow, - .patch_hypercall = vmx_patch_hypercall, + .patch_hypercall = vt_patch_hypercall, .inject_irq = vt_inject_irq, .inject_nmi = vt_inject_nmi, .inject_exception = vt_inject_exception, From patchwork Mon Feb 26 08:27:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571559 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 55BBD12F39B; Mon, 26 Feb 2024 08:29:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.8 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936161; cv=none; b=iKJigOhSelNwh3Gagm+ZnWhI89Zs0hId5K8HIG+7lxWdHwxqSMvjypWWRYDcWa6aCzxS0Tk57Q4iP64nFVf+VOQxns0gsnHvmCqMFE1bCa3mU2MWxNH9Wd9JLjNnpGlsNyD8jWve8YbNMAV80hscWSg6SlUst3lY9rLsEnFJe7w= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936161; c=relaxed/simple; bh=3EtzmmZ/aleau45nztUNKdmXOOZf4kna/f5LnDlJiNs=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=R4IPXgpF8RoWXxmiPSpp/EyZYsJl/6qqw2qzb802ZU6LXkVZMA0+DlSrlOfSwy0bvDT2Q473yWq6ShrupaWU/4QKBWUg60fV2QDtU8xnk4ER3w3lBBv8IoA4uAt36YbeiymIgeVI5X64pPkt11WuBoI3evI9WNBCGAkIh4ogG5U= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=Y0T53hPe; arc=none smtp.client-ip=192.198.163.8 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="Y0T53hPe" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936159; x=1740472159; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=3EtzmmZ/aleau45nztUNKdmXOOZf4kna/f5LnDlJiNs=; b=Y0T53hPebT+0Mde18P+DJZ0ZZ7MdJtyx+EZfDZoeZwQcoMaeBDppVXh8 LtT3wWFzm4+9rE/bd6NKUzl8gx+WRfqpH1mHFeAnzX4uondlk+zHVGUtz uUPkkAqBTrY2mgNqnEt6nBAU8UUlOJNCzs5AMf35OrTk1EZYobtNtr1mC uEsR9wbUkqCDUDSQoMcz3Gwi8CI4S0Hbb+U617Q6vy/s1z4T6twKRYlVA flzWr0IeTzFDpJHg4XXLDWOyNUiQnYPae3Pe0Co0Mrg9pEKqxjAx6zezr bbCHHol6WTupk7XT2dHdWVIVkuP23EGQypa/DykGY2+K+yN3Vju4/3T6F g==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="20751408" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="20751408" Received: from fmviesa008.fm.intel.com ([10.60.135.148]) by fmvoesa102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:29:11 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6735140" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by fmviesa008-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:29:11 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 125/130] KVM: TDX: Add methods to ignore virtual apic related operation Date: Mon, 26 Feb 2024 00:27:07 -0800 Message-Id: <52300c655b1e7d6cc0a13727d977f1f02729a4bb.1708933498.git.isaku.yamahata@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata TDX protects TDX guest APIC state from VMM. Implement access methods of TDX guest vAPIC state to ignore them or return zero. Signed-off-by: Isaku Yamahata --- arch/x86/kvm/vmx/main.c | 61 ++++++++++++++++++++++++++++++++++---- arch/x86/kvm/vmx/tdx.c | 6 ++++ arch/x86/kvm/vmx/x86_ops.h | 3 ++ 3 files changed, 64 insertions(+), 6 deletions(-) diff --git a/arch/x86/kvm/vmx/main.c b/arch/x86/kvm/vmx/main.c index fae5a3668361..c46c860be0f2 100644 --- a/arch/x86/kvm/vmx/main.c +++ b/arch/x86/kvm/vmx/main.c @@ -352,6 +352,14 @@ static bool vt_apic_init_signal_blocked(struct kvm_vcpu *vcpu) return vmx_apic_init_signal_blocked(vcpu); } +static void vt_set_virtual_apic_mode(struct kvm_vcpu *vcpu) +{ + if (is_td_vcpu(vcpu)) + return tdx_set_virtual_apic_mode(vcpu); + + return vmx_set_virtual_apic_mode(vcpu); +} + static void vt_apicv_pre_state_restore(struct kvm_vcpu *vcpu) { struct pi_desc *pi = vcpu_to_pi_desc(vcpu); @@ -360,6 +368,31 @@ static void vt_apicv_pre_state_restore(struct kvm_vcpu *vcpu) memset(pi->pir, 0, sizeof(pi->pir)); } +static void vt_hwapic_irr_update(struct kvm_vcpu *vcpu, int max_irr) +{ + if (is_td_vcpu(vcpu)) + return; + + return vmx_hwapic_irr_update(vcpu, max_irr); +} + +static void vt_hwapic_isr_update(int max_isr) +{ + if (is_td_vcpu(kvm_get_running_vcpu())) + return; + + return vmx_hwapic_isr_update(max_isr); +} + +static bool vt_guest_apic_has_interrupt(struct kvm_vcpu *vcpu) +{ + /* TDX doesn't support L2 at the moment. */ + if (WARN_ON_ONCE(is_td_vcpu(vcpu))) + return false; + + return vmx_guest_apic_has_interrupt(vcpu); +} + static int vt_sync_pir_to_irr(struct kvm_vcpu *vcpu) { if (is_td_vcpu(vcpu)) @@ -815,6 +848,22 @@ static void vt_update_cr8_intercept(struct kvm_vcpu *vcpu, int tpr, int irr) vmx_update_cr8_intercept(vcpu, tpr, irr); } +static void vt_set_apic_access_page_addr(struct kvm_vcpu *vcpu) +{ + if (is_td_vcpu(vcpu)) + return; + + vmx_set_apic_access_page_addr(vcpu); +} + +static void vt_refresh_apicv_exec_ctrl(struct kvm_vcpu *vcpu) +{ + if (WARN_ON_ONCE(is_td_vcpu(vcpu))) + return; + + vmx_refresh_apicv_exec_ctrl(vcpu); +} + static void vt_load_eoi_exitmap(struct kvm_vcpu *vcpu, u64 *eoi_exit_bitmap) { if (is_td_vcpu(vcpu)) @@ -1055,15 +1104,15 @@ struct kvm_x86_ops vt_x86_ops __initdata = { .enable_nmi_window = vt_enable_nmi_window, .enable_irq_window = vt_enable_irq_window, .update_cr8_intercept = vt_update_cr8_intercept, - .set_virtual_apic_mode = vmx_set_virtual_apic_mode, - .set_apic_access_page_addr = vmx_set_apic_access_page_addr, - .refresh_apicv_exec_ctrl = vmx_refresh_apicv_exec_ctrl, + .set_virtual_apic_mode = vt_set_virtual_apic_mode, + .set_apic_access_page_addr = vt_set_apic_access_page_addr, + .refresh_apicv_exec_ctrl = vt_refresh_apicv_exec_ctrl, .load_eoi_exitmap = vt_load_eoi_exitmap, .apicv_pre_state_restore = vt_apicv_pre_state_restore, .required_apicv_inhibits = VMX_REQUIRED_APICV_INHIBITS, - .hwapic_irr_update = vmx_hwapic_irr_update, - .hwapic_isr_update = vmx_hwapic_isr_update, - .guest_apic_has_interrupt = vmx_guest_apic_has_interrupt, + .hwapic_irr_update = vt_hwapic_irr_update, + .hwapic_isr_update = vt_hwapic_isr_update, + .guest_apic_has_interrupt = vt_guest_apic_has_interrupt, .sync_pir_to_irr = vt_sync_pir_to_irr, .deliver_interrupt = vt_deliver_interrupt, .dy_apicv_has_pending_interrupt = pi_has_pending_interrupt, diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c index fc78e9ca61c1..f706c346eea4 100644 --- a/arch/x86/kvm/vmx/tdx.c +++ b/arch/x86/kvm/vmx/tdx.c @@ -2080,6 +2080,12 @@ void tdx_enable_smi_window(struct kvm_vcpu *vcpu) } #endif +void tdx_set_virtual_apic_mode(struct kvm_vcpu *vcpu) +{ + /* Only x2APIC mode is supported for TD. */ + WARN_ON_ONCE(kvm_get_apic_mode(vcpu) != LAPIC_MODE_X2APIC); +} + int tdx_get_cpl(struct kvm_vcpu *vcpu) { return 0; diff --git a/arch/x86/kvm/vmx/x86_ops.h b/arch/x86/kvm/vmx/x86_ops.h index 727c4d418601..c507f1513dac 100644 --- a/arch/x86/kvm/vmx/x86_ops.h +++ b/arch/x86/kvm/vmx/x86_ops.h @@ -168,6 +168,7 @@ void tdx_get_exit_info(struct kvm_vcpu *vcpu, u32 *reason, bool tdx_has_emulated_msr(u32 index, bool write); int tdx_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr); int tdx_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr); +void tdx_set_virtual_apic_mode(struct kvm_vcpu *vcpu); int tdx_get_cpl(struct kvm_vcpu *vcpu); void tdx_cache_reg(struct kvm_vcpu *vcpu, enum kvm_reg reg); @@ -227,6 +228,8 @@ static inline bool tdx_has_emulated_msr(u32 index, bool write) { return false; } static inline int tdx_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr) { return 1; } static inline int tdx_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr) { return 1; } +static inline void tdx_set_virtual_apic_mode(struct kvm_vcpu *vcpu) {} + static inline int tdx_get_cpl(struct kvm_vcpu *vcpu) { return 0; } static inline void tdx_cache_reg(struct kvm_vcpu *vcpu, enum kvm_reg reg) {} static inline unsigned long tdx_get_rflags(struct kvm_vcpu *vcpu) { return 0; } From patchwork Mon Feb 26 08:27:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571560 Received: from mgamail.intel.com (unknown [192.198.163.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8C05512F588; Mon, 26 Feb 2024 08:29:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.8 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936161; cv=none; b=ulDsoXohb4wmaJFkOWnI1HjFokTPkfVNEQ+wsdbn4HTzB/WhgI7YmsNLTP2lEzDtLzN3zqsrY7Xglld5VDIJgGSceC7yRLtve+SwRi8mLKVcq74UNpd96pQwRyxkJYQ7Ndr9I9MCCTYGeJJgjjpcPN1BU9o+f88MUBI8ylmYRFc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936161; c=relaxed/simple; bh=kw0IabQRxVN3vncG2oiy3raswwUfUrRevEA6BZ0XDZg=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=q0R9sOZsIDpCU7Qh0GtbBzJhaSryEHg1+XkXsbFqJOJRcK6T5NpfWpsfplyMqKALxG4t/LG+NSg1MVo3jdNa0Jdkl/eUB3KVay8TIX3nMC6WKfIbVlcmYinSBDp6pCorKsEqXbLicPWIYoYKAiotHO3Y+7BmjBpfiSEE95RspQ8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=Dg823Kt2; arc=none smtp.client-ip=192.198.163.8 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="Dg823Kt2" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936159; x=1740472159; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=kw0IabQRxVN3vncG2oiy3raswwUfUrRevEA6BZ0XDZg=; b=Dg823Kt2rRgPRLYXh/A1O0ivGyIFfgq+3kNxqT1M6wyHySiX5iFk/8U+ o5BDAfqXOthgRblZLxJGmLjZ+g502oW+QRreAbiAA327yy+bgDDGFyTji OAU6xA3DXD7dO7YwvVKouf0cu3C+TPFHw2ryj2Y1TeofKsSO+icVGJ2bn EKGMVQ/xJNXXNieS8yn9CUEeUbU6lwDfUbH7cae5xek3JmLSevummLYVs yFMnAEMaGZumzidymGVGkDjbcFuGaIjhjOw6Oe7O01Z55CiP56+A7Li/G bI4L1zbcdCLsGyAHZFpYL8cZ2FxmNx14Cj5NyQ1VvVtfLaU4Wen6tbpYC Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="20751413" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="20751413" Received: from fmviesa008.fm.intel.com ([10.60.135.148]) by fmvoesa102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:29:12 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6735145" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by fmviesa008-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:29:11 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 126/130] KVM: TDX: Inhibit APICv for TDX guest Date: Mon, 26 Feb 2024 00:27:08 -0800 Message-Id: <38e2f8a77e89301534d82325946eb74db3e47815.1708933498.git.isaku.yamahata@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata TDX doesn't support APICV, inhibit APICv for TDX guest. Follow how SEV does it. Define a new inhibit reason for TDX, set it on TD initialization, and add the flag to kvm_x86_ops.required_apicv_inhibits. Signed-off-by: Isaku Yamahata --- arch/x86/include/asm/kvm_host.h | 9 +++++++++ arch/x86/kvm/vmx/main.c | 3 ++- arch/x86/kvm/vmx/tdx.c | 4 ++++ 3 files changed, 15 insertions(+), 1 deletion(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 2686c080820b..920fb771246b 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1300,6 +1300,15 @@ enum kvm_apicv_inhibit { * mapping between logical ID and vCPU. */ APICV_INHIBIT_REASON_LOGICAL_ID_ALIASED, + + /*********************************************************/ + /* INHIBITs that are relevant only to the Intel's APICv. */ + /*********************************************************/ + + /* + * APICv is disabled because TDX doesn't support it. + */ + APICV_INHIBIT_REASON_TDX, }; struct kvm_arch { diff --git a/arch/x86/kvm/vmx/main.c b/arch/x86/kvm/vmx/main.c index c46c860be0f2..2cd404fd7176 100644 --- a/arch/x86/kvm/vmx/main.c +++ b/arch/x86/kvm/vmx/main.c @@ -1022,7 +1022,8 @@ static void vt_post_memory_mapping(struct kvm_vcpu *vcpu, BIT(APICV_INHIBIT_REASON_BLOCKIRQ) | \ BIT(APICV_INHIBIT_REASON_PHYSICAL_ID_ALIASED) | \ BIT(APICV_INHIBIT_REASON_APIC_ID_MODIFIED) | \ - BIT(APICV_INHIBIT_REASON_APIC_BASE_MODIFIED)) + BIT(APICV_INHIBIT_REASON_APIC_BASE_MODIFIED) | \ + BIT(APICV_INHIBIT_REASON_TDX)) struct kvm_x86_ops vt_x86_ops __initdata = { .name = KBUILD_MODNAME, diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c index f706c346eea4..7be1be161dc2 100644 --- a/arch/x86/kvm/vmx/tdx.c +++ b/arch/x86/kvm/vmx/tdx.c @@ -2488,6 +2488,8 @@ static int __tdx_td_init(struct kvm *kvm, struct td_params *td_params, goto teardown; } + kvm_set_apicv_inhibit(kvm, APICV_INHIBIT_REASON_TDX); + return 0; /* @@ -2821,6 +2823,8 @@ static int tdx_td_vcpu_init(struct kvm_vcpu *vcpu, u64 vcpu_rcx) return -EIO; } + WARN_ON_ONCE(kvm_apicv_activated(vcpu->kvm)); + vcpu->arch.apic->apicv_active = false; vcpu->arch.mp_state = KVM_MP_STATE_RUNNABLE; tdx->td_vcpu_created = true; return 0; From patchwork Mon Feb 26 08:27:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571561 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E6F4A12FB34; Mon, 26 Feb 2024 08:29:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.8 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936163; cv=none; b=hCZ+r7hWE2vbJJdBOmGY6tj5rN+hnez3sZAPNtSt5CLuQoT4uhF2G/q0bPz+fHbp2s6j435EG5vG9+mtt4rV16KMM7gqlScnlPSy8cKDSWEM/jqLZsKpJQrlIAiTCmKP9vf6WJIwC4Pmtb8ynGoWTTCvzviNQUjwB4yUpkdCpsA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936163; c=relaxed/simple; bh=BDWpVPDb/o5eGg234cIsRAvf51uhkC79N+oxflW6L7E=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=rqDlxfCzMwwcwJYoxwPqjzHiNeboVBn4fgzIsEm8IObCalDYdpxiJwB1uBuVT5ns4fZTJHp8jijyrFaevXxpZyMujYiGVQb73wFVVwJRYvVYYX/+DC6eMBOKfi4/u6ePyC7xnoQ6c1hRc+w0iFncOF0JQrn/W9sbFLLBvF8ePSA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=WcvROSSa; arc=none smtp.client-ip=192.198.163.8 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="WcvROSSa" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936161; x=1740472161; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=BDWpVPDb/o5eGg234cIsRAvf51uhkC79N+oxflW6L7E=; b=WcvROSSa63Q0cm7yod+UcVuDo5eWFSsxCC4CV2RUdgJA+1XtlA88FsXn UTUe3kg7C3sXmaFfAFLCpDNIpnboenPcWXq/RR7D+krD7FtGYZObRnU7d Qz4kuhKFeQufzD0LLUoK5cnPMYJahI8cqSaycadEwOGJCQmT1IOtQnQky Ls29mjy6GsdzBqvZgYY1BpVPk7l1jhLi+DhvTwkRNxbJ1px/6e7sH7saa k5LVO6FYazbYvo5QfBCGrPqnI2nAzuPqp6pPaIQKJGn08Wta6uN4K5MKk +qRobQpKDftfQlkoXTmQDEjCTv5g7HNPq4+NrvRMaBNNff1L39PdFlcR1 w==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="20751417" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="20751417" Received: from fmviesa008.fm.intel.com ([10.60.135.148]) by fmvoesa102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:29:12 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6735149" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by fmviesa008-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:29:12 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 127/130] Documentation/virt/kvm: Document on Trust Domain Extensions(TDX) Date: Mon, 26 Feb 2024 00:27:09 -0800 Message-Id: <2d019d042874d28e4e881fb218d9bf86d3c73ba4.1708933498.git.isaku.yamahata@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata Add documentation to Intel Trusted Domain Extensions(TDX) support. Signed-off-by: Isaku Yamahata --- Documentation/virt/kvm/api.rst | 9 +- Documentation/virt/kvm/x86/index.rst | 1 + Documentation/virt/kvm/x86/intel-tdx.rst | 362 +++++++++++++++++++++++ 3 files changed, 371 insertions(+), 1 deletion(-) create mode 100644 Documentation/virt/kvm/x86/intel-tdx.rst diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst index 667dc58f7d2f..4b70d2b43532 100644 --- a/Documentation/virt/kvm/api.rst +++ b/Documentation/virt/kvm/api.rst @@ -1394,6 +1394,9 @@ the memory region are automatically reflected into the guest. For example, an mmap() that affects the region will be made visible immediately. Another example is madvise(MADV_DROP). +For TDX guest, deleting/moving memory region loses guest memory contents. +Read only region isn't supported. Only as-id 0 is supported. + Note: On arm64, a write generated by the page-table walker (to update the Access and Dirty flags, for example) never results in a KVM_EXIT_MMIO exit when the slot has the KVM_MEM_READONLY flag. This @@ -4734,7 +4737,7 @@ H_GET_CPU_CHARACTERISTICS hypercall. :Capability: basic :Architectures: x86 -:Type: vm +:Type: vm ioctl, vcpu ioctl :Parameters: an opaque platform specific structure (in/out) :Returns: 0 on success; -1 on error @@ -4746,6 +4749,10 @@ Currently, this ioctl is used for issuing Secure Encrypted Virtualization (SEV) commands on AMD Processors. The SEV commands are defined in Documentation/virt/kvm/x86/amd-memory-encryption.rst. +Currently, this ioctl is used for issuing Trusted Domain Extensions +(TDX) commands on Intel Processors. The TDX commands are defined in +Documentation/virt/kvm/x86/intel-tdx.rst. + 4.111 KVM_MEMORY_ENCRYPT_REG_REGION ----------------------------------- diff --git a/Documentation/virt/kvm/x86/index.rst b/Documentation/virt/kvm/x86/index.rst index 9ece6b8dc817..851e99174762 100644 --- a/Documentation/virt/kvm/x86/index.rst +++ b/Documentation/virt/kvm/x86/index.rst @@ -11,6 +11,7 @@ KVM for x86 systems cpuid errata hypercalls + intel-tdx mmu msr nested-vmx diff --git a/Documentation/virt/kvm/x86/intel-tdx.rst b/Documentation/virt/kvm/x86/intel-tdx.rst new file mode 100644 index 000000000000..a1b10e99c1ff --- /dev/null +++ b/Documentation/virt/kvm/x86/intel-tdx.rst @@ -0,0 +1,362 @@ +.. SPDX-License-Identifier: GPL-2.0 + +=================================== +Intel Trust Domain Extensions (TDX) +=================================== + +Overview +======== +TDX stands for Trust Domain Extensions which isolates VMs from +the virtual-machine manager (VMM)/hypervisor and any other software on +the platform. For details, see the specifications [1]_, whitepaper [2]_, +architectural extensions specification [3]_, module documentation [4]_, +loader interface specification [5]_, guest-hypervisor communication +interface [6]_, virtual firmware design guide [7]_, and other resources +([8]_, [9]_, [10]_, [11]_, and [12]_). + + +API description +=============== + +KVM_MEMORY_ENCRYPT_OP +--------------------- +:Type: vm ioctl, vcpu ioctl + +For TDX operations, KVM_MEMORY_ENCRYPT_OP is re-purposed to be generic +ioctl with TDX specific sub ioctl command. + +:: + + /* Trust Domain eXtension sub-ioctl() commands. */ + enum kvm_tdx_cmd_id { + KVM_TDX_CAPABILITIES = 0, + KVM_TDX_INIT_VM, + KVM_TDX_INIT_VCPU, + KVM_TDX_INIT_MEM_REGION, + KVM_TDX_FINALIZE_VM, + + KVM_TDX_CMD_NR_MAX, + }; + + struct kvm_tdx_cmd { + /* enum kvm_tdx_cmd_id */ + __u32 id; + /* flags for sub-commend. If sub-command doesn't use this, set zero. */ + __u32 flags; + /* + * data for each sub-command. An immediate or a pointer to the actual + * data in process virtual address. If sub-command doesn't use it, + * set zero. + */ + __u64 data; + /* + * Auxiliary error code. The sub-command may return TDX SEAMCALL + * status code in addition to -Exxx. + * Defined for consistency with struct kvm_sev_cmd. + */ + __u64 error; + /* Reserved: Defined for consistency with struct kvm_sev_cmd. */ + __u64 unused; + }; + +KVM_TDX_CAPABILITIES +-------------------- +:Type: vm ioctl + +Subset of TDSYSINFO_STRCUCT retrieved by TDH.SYS.INFO TDX SEAM call will be +returned. Which describes about Intel TDX module. + +- id: KVM_TDX_CAPABILITIES +- flags: must be 0 +- data: pointer to struct kvm_tdx_capabilities +- error: must be 0 +- unused: must be 0 + +:: + + struct kvm_tdx_cpuid_config { + __u32 leaf; + __u32 sub_leaf; + __u32 eax; + __u32 ebx; + __u32 ecx; + __u32 edx; + }; + + struct kvm_tdx_capabilities { + __u64 attrs_fixed0; + __u64 attrs_fixed1; + __u64 xfam_fixed0; + __u64 xfam_fixed1; + #define TDX_CAP_GPAW_48 (1 << 0) + #define TDX_CAP_GPAW_52 (1 << 1) + __u32 supported_gpaw; + __u32 padding; + __u64 reserved[251]; + + __u32 nr_cpuid_configs; + struct kvm_tdx_cpuid_config cpuid_configs[]; + }; + + +KVM_TDX_INIT_VM +--------------- +:Type: vm ioctl + +Does additional VM initialization specific to TDX which corresponds to +TDH.MNG.INIT TDX SEAM call. + +- id: KVM_TDX_INIT_VM +- flags: must be 0 +- data: pointer to struct kvm_tdx_init_vm +- error: must be 0 +- unused: must be 0 + +:: + + struct kvm_tdx_init_vm { + __u64 attributes; + __u64 mrconfigid[6]; /* sha384 digest */ + __u64 mrowner[6]; /* sha384 digest */ + __u64 mrownerconfig[6]; /* sha348 digest */ + __u64 reserved[1004]; /* must be zero for future extensibility */ + + struct kvm_cpuid2 cpuid; + }; + + +KVM_TDX_INIT_VCPU +----------------- +:Type: vcpu ioctl + +Does additional VCPU initialization specific to TDX which corresponds to +TDH.VP.INIT TDX SEAM call. + +- id: KVM_TDX_INIT_VCPU +- flags: must be 0 +- data: initial value of the guest TD VCPU RCX +- error: must be 0 +- unused: must be 0 + +KVM_TDX_INIT_MEM_REGION +----------------------- +:Type: vm ioctl + +Encrypt a memory continuous region which corresponding to TDH.MEM.PAGE.ADD +TDX SEAM call. +If KVM_TDX_MEASURE_MEMORY_REGION flag is specified, it also extends measurement +which corresponds to TDH.MR.EXTEND TDX SEAM call. + +- id: KVM_TDX_INIT_VCPU +- flags: flags + currently only KVM_TDX_MEASURE_MEMORY_REGION is defined +- data: pointer to struct kvm_tdx_init_mem_region +- error: must be 0 +- unused: must be 0 + +:: + + #define KVM_TDX_MEASURE_MEMORY_REGION (1UL << 0) + + struct kvm_tdx_init_mem_region { + __u64 source_addr; + __u64 gpa; + __u64 nr_pages; + }; + + +KVM_TDX_FINALIZE_VM +------------------- +:Type: vm ioctl + +Complete measurement of the initial TD contents and mark it ready to run +which corresponds to TDH.MR.FINALIZE + +- id: KVM_TDX_FINALIZE_VM +- flags: must be 0 +- data: must be 0 +- error: must be 0 +- unused: must be 0 + +KVM TDX creation flow +===================== +In addition to KVM normal flow, new TDX ioctls need to be called. The control flow +looks like as follows. + +#. system wide capability check + + * KVM_CAP_VM_TYPES: check if VM type is supported and if KVM_X86_TDX_VM + is supported. + +#. creating VM + + * KVM_CREATE_VM + * KVM_TDX_CAPABILITIES: query if TDX is supported on the platform. + * KVM_ENABLE_CAP_VM(KVM_CAP_MAX_VCPUS): set max_vcpus. KVM_MAX_VCPUS by + default. KVM_MAX_VCPUS is not a part of ABI, but kernel internal constant + that is subject to change. Because max vcpus is a part of attestation, max + vcpus should be explicitly set. + * KVM_SET_TSC_KHZ for vm. optional + * KVM_TDX_INIT_VM: pass TDX specific VM parameters. + +#. creating VCPU + + * KVM_CREATE_VCPU + * KVM_TDX_INIT_VCPU: pass TDX specific VCPU parameters. + * KVM_SET_CPUID2: Enable CPUID[0x1].ECX.X2APIC(bit 21)=1 so that the following + setting of MSR_IA32_APIC_BASE success. Without this, + KVM_SET_MSRS(MSR_IA32_APIC_BASE) fails. + * KVM_SET_MSRS: Set the initial reset value of MSR_IA32_APIC_BASE to + APIC_DEFAULT_ADDRESS(0xfee00000) | XAPIC_ENABLE(bit 10) | + X2APIC_ENABLE(bit 11) [| MSR_IA32_APICBASE_BSP(bit 8) optional] + +#. initializing guest memory + + * allocate guest memory and initialize page same to normal KVM case + In TDX case, parse and load TDVF into guest memory in addition. + * KVM_TDX_INIT_MEM_REGION to add and measure guest pages. + If the pages has contents above, those pages need to be added. + Otherwise the contents will be lost and guest sees zero pages. + * KVM_TDX_FINALIAZE_VM: Finalize VM and measurement + This must be after KVM_TDX_INIT_MEM_REGION. + +#. run vcpu + +Design discussion +================= + +Coexistence of normal(VMX) VM and TD VM +--------------------------------------- +It's required to allow both legacy(normal VMX) VMs and new TD VMs to +coexist. Otherwise the benefits of VM flexibility would be eliminated. +The main issue for it is that the logic of kvm_x86_ops callbacks for +TDX is different from VMX. On the other hand, the variable, +kvm_x86_ops, is global single variable. Not per-VM, not per-vcpu. + +Several points to be considered: + + * No or minimal overhead when TDX is disabled(CONFIG_INTEL_TDX_HOST=n). + * Avoid overhead of indirect call via function pointers. + * Contain the changes under arch/x86/kvm/vmx directory and share logic + with VMX for maintenance. + Even though the ways to operation on VM (VMX instruction vs TDX + SEAM call) are different, the basic idea remains the same. So, many + logic can be shared. + * Future maintenance + The huge change of kvm_x86_ops in (near) future isn't expected. + a centralized file is acceptable. + +- Wrapping kvm x86_ops: The current choice + + Introduce dedicated file for arch/x86/kvm/vmx/main.c (the name, + main.c, is just chosen to show main entry points for callbacks.) and + wrapper functions around all the callbacks with + "if (is-tdx) tdx-callback() else vmx-callback()". + + Pros: + + - No major change in common x86 KVM code. The change is (mostly) + contained under arch/x86/kvm/vmx/. + - When TDX is disabled(CONFIG_INTEL_TDX_HOST=n), the overhead is + optimized out. + - Micro optimization by avoiding function pointer. + + Cons: + + - Many boiler plates in arch/x86/kvm/vmx/main.c. + +KVM MMU Changes +--------------- +KVM MMU needs to be enhanced to handle Secure/Shared-EPT. The +high-level execution flow is mostly same to normal EPT case. +EPT violation/misconfiguration -> invoke TDP fault handler -> +resolve TDP fault -> resume execution. (or emulate MMIO) +The difference is, that S-EPT is operated(read/write) via TDX SEAM +call which is expensive instead of direct read/write EPT entry. +One bit of GPA (51 or 47 bit) is repurposed so that it means shared +with host(if set to 1) or private to TD(if cleared to 0). + +- The current implementation + + * Reuse the existing MMU code with minimal update. Because the + execution flow is mostly same. But additional operation, TDX call + for S-EPT, is needed. So add hooks for it to kvm_x86_ops. + * For performance, minimize TDX SEAM call to operate on S-EPT. When + getting corresponding S-EPT pages/entry from faulting GPA, don't + use TDX SEAM call to read S-EPT entry. Instead create shadow copy + in host memory. + Repurpose the existing kvm_mmu_page as shadow copy of S-EPT and + associate S-EPT to it. + * Treats share bit as attributes. mask/unmask the bit where + necessary to keep the existing traversing code works. + Introduce kvm.arch.gfn_shared_mask and use "if (gfn_share_mask)" + for special case. + + * 0 : for non-TDX case + * 51 or 47 bit set for TDX case. + + Pros: + + - Large code reuse with minimal new hooks. + - Execution path is same. + + Cons: + + - Complicates the existing code. + - Repurpose kvm_mmu_page as shadow of Secure-EPT can be confusing. + +New KVM API, ioctl (sub)command, to manage TD VMs +------------------------------------------------- +Additional KVM APIs are needed to control TD VMs. The operations on TD +VMs are specific to TDX. + +- Piggyback and repurpose KVM_MEMORY_ENCRYPT_OP + + Although operations for TD VMs aren't necessarily related to memory + encryption, define sub operations of KVM_MEMORY_ENCRYPT_OP for TDX specific + ioctls. + + Pros: + + - No major change in common x86 KVM code. + - Follows the SEV case. + + Cons: + + - The sub operations of KVM_MEMORY_ENCRYPT_OP aren't necessarily memory + encryption, but operations on TD VMs. + +References +========== + +.. [1] TDX specification + https://software.intel.com/content/www/us/en/develop/articles/intel-trust-domain-extensions.html +.. [2] Intel Trust Domain Extensions (Intel TDX) + https://software.intel.com/content/dam/develop/external/us/en/documents/tdx-whitepaper-final9-17.pdf +.. [3] Intel CPU Architectural Extensions Specification + https://software.intel.com/content/dam/develop/external/us/en/documents/intel-tdx-cpu-architectural-specification.pdf +.. [4] Intel TDX Module 1.0 EAS + https://software.intel.com/content/dam/develop/external/us/en/documents/intel-tdx-module-1eas.pdf +.. [5] Intel TDX Loader Interface Specification + https://software.intel.com/content/dam/develop/external/us/en/documents/intel-tdx-seamldr-interface-specification.pdf +.. [6] Intel TDX Guest-Hypervisor Communication Interface + https://software.intel.com/content/dam/develop/external/us/en/documents/intel-tdx-guest-hypervisor-communication-interface.pdf +.. [7] Intel TDX Virtual Firmware Design Guide + https://software.intel.com/content/dam/develop/external/us/en/documents/tdx-virtual-firmware-design-guide-rev-1. +.. [8] intel public github + + * kvm TDX branch: https://github.com/intel/tdx/tree/kvm + * TDX guest branch: https://github.com/intel/tdx/tree/guest + +.. [9] tdvf + https://github.com/tianocore/edk2-staging/tree/TDVF +.. [10] KVM forum 2020: Intel Virtualization Technology Extensions to + Enable Hardware Isolated VMs + https://osseu2020.sched.com/event/eDzm/intel-virtualization-technology-extensions-to-enable-hardware-isolated-vms-sean-christopherson-intel +.. [11] Linux Security Summit EU 2020: + Architectural Extensions for Hardware Virtual Machine Isolation + to Advance Confidential Computing in Public Clouds - Ravi Sahita + & Jun Nakajima, Intel Corporation + https://osseu2020.sched.com/event/eDOx/architectural-extensions-for-hardware-virtual-machine-isolation-to-advance-confidential-computing-in-public-clouds-ravi-sahita-jun-nakajima-intel-corporation +.. [12] [RFCv2,00/16] KVM protected memory extension + https://lore.kernel.org/all/20201020061859.18385-1-kirill.shutemov@linux.intel.com/ From patchwork Mon Feb 26 08:27:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571563 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 69CE612FF7B; Mon, 26 Feb 2024 08:29:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.8 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936164; cv=none; b=TCSX9VNiC65aHEdgQUJjnTBxpkcNl4koHCsqeW14c8Xn827HkHLDl/YJLJT59Ld16TWRZdsIdvzYpJbRBBqzcfgX9bFY0c0kT+4OOG+b2T4c7U1HGLeY6ee9NN0j8PdACRxix6xDW8d4s8jgJiQfFDVxwg5Rz+1g/HVEGU9uapI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936164; c=relaxed/simple; bh=gDn2qlctsRhiSfk4B8miWiyBiv/7pxkEkr9nT08qBkA=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=TN6WxTEO1cqt+8T8xByjTNN4JuOnq1RTF3U3A/GQSwr9+Ql+rHmW2lgOolHbw5yoqm0cUB3JRcItMa03gsh2X2jTYBs+Dr+zUgdZPpYXNABWDRYbqgUQAzwShcrVyLK20ivkIGJXxyLeAUqW66eEaPEzHytiaqI50kMbnw/UfsM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=HHDiJSqB; arc=none smtp.client-ip=192.198.163.8 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="HHDiJSqB" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936161; x=1740472161; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=gDn2qlctsRhiSfk4B8miWiyBiv/7pxkEkr9nT08qBkA=; b=HHDiJSqBf/5OhBevNzQnw6PyYwiCwjmuRA4H9zEnlAhfR6T7zx6AKRP7 ej/Bpbww+C/aHKYg40/+bcd5+IWUGQuByhQt+JFf15nP5gwy1SPEqsI9E +M/aV30MgfBSj18yPoF8vTj+MSGEg8EhS9zGfE/K0Vka3L014YRJpC0bg zPJJixVoFGWs4N8rFjlG1MvZJQUDTCZnuWpL9owpiSQudEJ3UQUjOjsZV +Lhjp+7Jq3/K4E9n8/n1J4N6Ut4huNXMgXS6xtlN9h+MM+RT1C51BasL4 KnA85K5iro26eajKOEQwe+/pvb7hHcm6I72ueYqMuPwxhM46e0muqVc06 Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="20751422" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="20751422" Received: from fmviesa008.fm.intel.com ([10.60.135.148]) by fmvoesa102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:29:12 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6735155" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by fmviesa008-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:29:12 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 128/130] KVM: x86: design documentation on TDX support of x86 KVM TDP MMU Date: Mon, 26 Feb 2024 00:27:10 -0800 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata Add a high level design document on TDX changes to TDP MMU. Signed-off-by: Isaku Yamahata --- Documentation/virt/kvm/x86/index.rst | 1 + Documentation/virt/kvm/x86/tdx-tdp-mmu.rst | 443 +++++++++++++++++++++ 2 files changed, 444 insertions(+) create mode 100644 Documentation/virt/kvm/x86/tdx-tdp-mmu.rst diff --git a/Documentation/virt/kvm/x86/index.rst b/Documentation/virt/kvm/x86/index.rst index 851e99174762..63a78bd41b16 100644 --- a/Documentation/virt/kvm/x86/index.rst +++ b/Documentation/virt/kvm/x86/index.rst @@ -16,4 +16,5 @@ KVM for x86 systems msr nested-vmx running-nested-guests + tdx-tdp-mmu timekeeping diff --git a/Documentation/virt/kvm/x86/tdx-tdp-mmu.rst b/Documentation/virt/kvm/x86/tdx-tdp-mmu.rst new file mode 100644 index 000000000000..49d103720272 --- /dev/null +++ b/Documentation/virt/kvm/x86/tdx-tdp-mmu.rst @@ -0,0 +1,443 @@ +.. SPDX-License-Identifier: GPL-2.0 + +Design of TDP MMU for TDX support +================================= +This document describes a (high level) design for TDX support of KVM TDP MMU of +x86 KVM. + +In this document, we use "TD" or "guest TD" to differentiate it from the current +"VM" (Virtual Machine), which is supported by KVM today. + + +Background of TDX +================= +TD private memory is designed to hold TD private content, encrypted by the CPU +using the TD ephemeral key. An encryption engine holds a table of encryption +keys, and an encryption key is selected for each memory transaction based on a +Host Key Identifier (HKID). By design, the host VMM does not have access to the +encryption keys. + +In the first generation of MKTME, HKID is "stolen" from the physical address by +allocating a configurable number of bits from the top of the physical address. +The HKID space is partitioned into shared HKIDs for legacy MKTME accesses and +private HKIDs for SEAM-mode-only accesses. We use 0 for the shared HKID on the +host so that MKTME can be opaque or bypassed on the host. + +During TDX non-root operation (i.e. guest TD), memory accesses can be qualified +as either shared or private, based on the value of a new SHARED bit in the Guest +Physical Address (GPA). The CPU translates shared GPAs using the usual VMX EPT +(Extended Page Table) or "Shared EPT" (in this document), which resides in the +host VMM memory. The Shared EPT is directly managed by the host VMM - the same +as with the current VMX. Since guest TDs usually require I/O, and the data +exchange needs to be done via shared memory, thus KVM needs to use the current +EPT functionality even for TDs. + +The CPU translates private GPAs using a separate Secure EPT. The Secure EPT +pages are encrypted and integrity-protected with the TD's ephemeral private key. +Secure EPT can be managed _indirectly_ by the host VMM, using the TDX interface +functions (SEAMCALLs), and thus conceptually Secure EPT is a subset of EPT +because not all functionalities are available. + +Since the execution of such interface functions takes much longer time than +accessing memory directly, in KVM we use the existing TDP code to mirror the +Secure EPT for the TD. And we think there are at least two options today in +terms of the timing for executing such SEAMCALLs: + +1. synchronous, i.e. while walking the TDP page tables, or +2. post-walk, i.e. record what needs to be done to the real Secure EPT during + the walk, and execute SEAMCALLs later. + +The option 1 seems to be more intuitive and simpler, but the Secure EPT +concurrency rules are different from the ones of the TDP or EPT. For example, +MEM.SEPT.RD acquire shared access to the whole Secure EPT tree of the target + +Secure EPT(SEPT) operations +--------------------------- +Secure EPT is an Extended Page Table for GPA-to-HPA translation of TD private +HPA. A Secure EPT is designed to be encrypted with the TD's ephemeral private +key. SEPT pages are allocated by the host VMM via Intel TDX functions, but their +content is intended to be hidden and is not architectural. + +Unlike the conventional EPT, the CPU can't directly read/write its entry. +Instead, TDX SEAMCALL API is used. Several SEAMCALLs correspond to operation on +the EPT entry. + +* TDH.MEM.SEPT.ADD(): + + Add a secure EPT page from the secure EPT tree. This corresponds to updating + the non-leaf EPT entry with present bit set + +* TDH.MEM.SEPT.REMOVE(): + + Remove the secure page from the secure EPT tree. There is no corresponding + to the EPT operation. + +* TDH.MEM.SEPT.RD(): + + Read the secure EPT entry. This corresponds to reading the EPT entry as + memory. Please note that this is much slower than direct memory reading. + +* TDH.MEM.PAGE.ADD() and TDH.MEM.PAGE.AUG(): + + Add a private page to the secure EPT tree. This corresponds to updating the + leaf EPT entry with present bit set. + +* THD.MEM.PAGE.REMOVE(): + + Remove a private page from the secure EPT tree. There is no corresponding + to the EPT operation. + +* TDH.MEM.RANGE.BLOCK(): + + This (mostly) corresponds to clearing the present bit of the leaf EPT entry. + Note that the private page is still linked in the secure EPT. To remove it + from the secure EPT, TDH.MEM.SEPT.REMOVE() and TDH.MEM.PAGE.REMOVE() needs to + be called. + +* TDH.MEM.TRACK(): + + Increment the TLB epoch counter. This (mostly) corresponds to EPT TLB flush. + Note that the private page is still linked in the secure EPT. To remove it + from the secure EPT, tdh_mem_page_remove() needs to be called. + + +Adding private page +------------------- +The procedure of populating the private page looks as follows. + +1. TDH.MEM.SEPT.ADD(512G level) +2. TDH.MEM.SEPT.ADD(1G level) +3. TDH.MEM.SEPT.ADD(2M level) +4. TDH.MEM.PAGE.AUG(4K level) + +Those operations correspond to updating the EPT entries. + +Dropping private page and TLB shootdown +--------------------------------------- +The procedure of dropping the private page looks as follows. + +1. TDH.MEM.RANGE.BLOCK(4K level) + + This mostly corresponds to clear the present bit in the EPT entry. This + prevents (or blocks) TLB entry from creating in the future. Note that the + private page is still linked in the secure EPT tree and the existing cache + entry in the TLB isn't flushed. + +2. TDH.MEM.TRACK(range) and TLB shootdown + + This mostly corresponds to the EPT TLB shootdown. Because all vcpus share + the same Secure EPT, all vcpus need to flush TLB. + + * TDH.MEM.TRACK(range) by one vcpu. It increments the global internal TLB + epoch counter. + + * send IPI to remote vcpus + * Other vcpu exits to VMM from guest TD and then re-enter. TDH.VP.ENTER(). + * TDH.VP.ENTER() checks the TLB epoch counter and If its TLB is old, flush + TLB. + + Note that only single vcpu issues tdh_mem_track(). + + Note that the private page is still linked in the secure EPT tree, unlike the + conventional EPT. + +3. TDH.MEM.PAGE.PROMOTE, TDH.MEM.PAGEDEMOTE(), TDH.MEM.PAGE.RELOCATE(), or + TDH.MEM.PAGE.REMOVE() + + There is no corresponding operation to the conventional EPT. + + * When changing page size (e.g. 4K <-> 2M) TDH.MEM.PAGE.PROMOTE() or + TDH.MEM.PAGE.DEMOTE() is used. During those operation, the guest page is + kept referenced in the Secure EPT. + + * When migrating page, TDH.MEM.PAGE.RELOCATE(). This requires both source + page and destination page. + * when destroying TD, TDH.MEM.PAGE.REMOVE() removes the private page from the + secure EPT tree. In this case TLB shootdown is not needed because vcpus + don't run any more. + +The basic idea for TDX support +============================== +Because shared EPT is the same as the existing EPT, use the existing logic for +shared EPT. On the other hand, secure EPT requires additional operations +instead of directly reading/writing of the EPT entry. + +On EPT violation, The KVM mmu walks down the EPT tree from the root, determines +the EPT entry to operate, and updates the entry. If necessary, a TLB shootdown +is done. Because it's very slow to directly walk secure EPT by TDX SEAMCALL, +TDH.MEM.SEPT.RD(), the mirror of secure EPT is created and maintained. Add +hooks to KVM MMU to reuse the existing code. + +EPT violation on shared GPA +--------------------------- +(1) EPT violation on shared GPA or zapping shared GPA + :: + + walk down shared EPT tree (the existing code) + | + | + V + shared EPT tree (CPU refers.) + +(2) update the EPT entry. (the existing code) + + TLB shootdown in the case of zapping. + + +EPT violation on private GPA +---------------------------- +(1) EPT violation on private GPA or zapping private GPA + :: + + walk down the mirror of secure EPT tree (mostly same as the existing code) + | + | + V + mirror of secure EPT tree (KVM MMU software only. reuse of the existing code) + +(2) update the (mirrored) EPT entry. (mostly same as the existing code) + +(3) call the hooks with what EPT entry is changed + :: + + | + NEW: hooks in KVM MMU + | + V + secure EPT root(CPU refers) + +(4) the TDX backend calls necessary TDX SEAMCALLs to update real secure EPT. + +The major modification is to add hooks for the TDX backend for additional +operations and to pass down which EPT, shared EPT, or private EPT is used, and +twist the behavior if we're operating on private EPT. + +The following depicts the relationship. +:: + + KVM | TDX module + | | | + -------------+---------- | | + | | | | + V V | | + shared GPA private GPA | V + CPU shared EPT pointer KVM private EPT pointer | CPU secure EPT pointer + | | | | + | | | | + V V | V + shared EPT private EPT<-------mirror----->Secure EPT + | | | | + | \--------------------+------\ | + | | | | + V | V V + shared guest page | private guest page + | + | + non-encrypted memory | encrypted memory + | + +shared EPT: CPU and KVM walk with shared GPA + Maintained by the existing code +private EPT: KVM walks with private GPA + Maintained by the twisted existing code +secure EPT: CPU walks with private GPA. + Maintained by TDX module with TDX SEAMCALLs via hooks + + +Tracking private EPT page +========================= +Shared EPT pages are managed by struct kvm_mmu_page. They are linked in a list +structure. When necessary, the list is traversed to operate on. Private EPT +pages have different characteristics. For example, private pages can't be +swapped out. When shrinking memory, we'd like to traverse only shared EPT pages +and skip private EPT pages. Likewise, page migration isn't supported for +private pages (yet). Introduce an additional list to track shared EPT pages and +track private EPT pages independently. + +At the beginning of EPT violation, the fault handler knows fault GPA, thus it +knows which EPT to operate on, private or shared. If it's private EPT, +an additional task is done. Something like "if (private) { callback a hook }". +Since the fault handler has deep function calls, it's cumbersome to hold the +information of which EPT is operating. Options to mitigate it are + +1. Pass the information as an argument for the function call. +2. Record the information in struct kvm_mmu_page somehow. +3. Record the information in vcpu structure. + +Option 2 was chosen. Because option 1 requires modifying all the functions. It +would affect badly to the normal case. Option 3 doesn't work well because in +some cases, we need to walk both private and shared EPT. + +The role of the EPT page can be utilized and one bit can be curved out from +unused bits in struct kvm_mmu_page_role. When allocating the EPT page, +initialize the information. Mostly struct kvm_mmu_page is available because +we're operating on EPT pages. + + +The conversion of private GPA and shared GPA +============================================ +A page of a given GPA can be assigned to only private GPA xor shared GPA at one +time. (This is the restriction by KVM implementation to avoid doubling guest +memory usage. Not by TDX architecture.) The GPA can't be accessed +simultaneously via both private GPA and shared GPA. On guest startup, all the +GPAs are assigned as private. Guest converts the range of GPA to shared (or +private) from private (or shared) by MapGPA hypercall. MapGPA hypercall takes +the start GPA and the size of the region. If the given start GPA is shared +(shared bit set), VMM converts the region into shared (if it's already shared, +nop). + +If the guest TD triggers an EPT violation on the already converted region, +i.e. EPT violation on private(or shared) GPA when page is shared(or private), +the access won't be allowed. KVM_EXIT_MEMORY_FAULT is triggered. The user +space VMM will decide how to handle it. + +If the guest access private (or shared) GPA after the conversion to shared (or +private), the following sequence will be observed + +1. MapGPA(shared GPA: shared bit set) hypercall +2. KVM cause KVM_TDX_EXIT with hypercall to the user space VMM. +3. The user space VMM converts the GPA with KVM_SET_MEMORY_ATTRIBUTES(shared). +4. The user space VMM resumes vcpu execution with KVM_VCPU_RUN +5. Guest TD accesses private GPA (shared bit cleared) +6. KVM gets EPT violation on private GPA (shared bit cleared) +7. KVM finds the GPA was set to be shared in the xarray while the faulting GPA + is private (shared bit cleared) +8. KVM_EXIT_MEMORY_FAULT. User space VMM, e.g. qemu, decide what to do. + Typically requests KVM conversion of GPA without MapGPA hypercall. +9. KVM converts GPA from shared to private with + KVM_SET_MEMORY_ATTRIBUTES(private) +10. Resume vcpu execution + +At step 9, user space VMM may think such memory access is due to race, let vcpu +resume without conversion with the expectation that other vcpu issues MapGPA. +Or user space VMM may think such memory access is doubtful and the guest is +trying to attack VMM. It may throttle vcpu execution as mitigation or finally +kill such a guest. Or user space VMM may think it's a bug of the guest TD, kill +the guest TD. + +This sequence is not efficient. Guest TD shouldn't access private (or shared) +GPA after converting GPA to shared (or private). Although KVM can handle it, +it's sub-optimal and won't be optimized. + +The original TDP MMU and race condition +======================================= +Because vcpus share the EPT, once the EPT entry is zapped, we need to shootdown +TLB. Send IPI to remote vcpus. Remote vcpus flush their down TLBs. Until TLB +shootdown is done, vcpus may reference the zapped guest page. + +TDP MMU uses read lock of mmu_lock to mitigate vcpu contention. When read lock +is obtained, it depends on the atomic update of the EPT entry. (On the other +hand legacy MMU uses write lock.) When vcpu is populating/zapping the EPT entry +with a read lock held, other vcpu may be populating or zapping the same EPT +entry at the same time. + +To avoid the race condition, the entry is frozen. It means the EPT entry is set +to the special value, REMOVED_SPTE which clears the present bit. And then after +TLB shootdown, update the EPT entry to the final value. + +Concurrent zapping +------------------ +1. read lock +2. freeze the EPT entry (atomically set the value to REMOVED_SPTE) + If other vcpu froze the entry, restart page fault. +3. TLB shootdown + + * send IPI to remote vcpus + * TLB flush (local and remote) + + For each entry update, TLB shootdown is needed because of the + concurrency. +4. atomically set the EPT entry to the final value +5. read unlock + +Concurrent populating +--------------------- +In the case of populating the non-present EPT entry, atomically update the EPT +entry. + +1. read lock + +2. atomically update the EPT entry + If other vcpu frozen the entry or updated the entry, restart page fault. + +3. read unlock + +In the case of updating the present EPT entry (e.g. page migration), the +operation is split into two. Zapping the entry and populating the entry. + +1. read lock +2. zap the EPT entry. follow the concurrent zapping case. +3. populate the non-present EPT entry. +4. read unlock + +Non-concurrent batched zapping +------------------------------ +In some cases, zapping the ranges is done exclusively with a write lock held. +In this case, the TLB shootdown is batched into one. + +1. write lock +2. zap the EPT entries by traversing them +3. TLB shootdown +4. write unlock + +For Secure EPT, TDX SEAMCALLs are needed in addition to updating the mirrored +EPT entry. + +TDX concurrent zapping +---------------------- +Add a hook for TDX SEAMCALLs at the step of the TLB shootdown. + +1. read lock +2. freeze the EPT entry(set the value to REMOVED_SPTE) +3. TLB shootdown via a hook + + * TLB.MEM.RANGE.BLOCK() + * TLB.MEM.TRACK() + * send IPI to remote vcpus + +4. set the EPT entry to the final value +5. read unlock + +TDX concurrent populating +------------------------- +TDX SEAMCALLs are required in addition to operating the mirrored EPT entry. The +frozen entry is utilized by following the zapping case to avoid the race +condition. A hook can be added. + +1. read lock +2. freeze the EPT entry +3. hook + + * TDH_MEM_SEPT_ADD() for non-leaf or TDH_MEM_PAGE_AUG() for leaf. + +4. set the EPT entry to the final value +5. read unlock + +Without freezing the entry, the following race can happen. Suppose two vcpus +are faulting on the same GPA and the 2M and 4K level entries aren't populated +yet. + +* vcpu 1: update 2M level EPT entry +* vcpu 2: update 4K level EPT entry +* vcpu 2: TDX SEAMCALL to update 4K secure EPT entry => error +* vcpu 1: TDX SEAMCALL to update 2M secure EPT entry + + +TDX non-concurrent batched zapping +---------------------------------- +For simplicity, the procedure of concurrent populating is utilized. The +procedure can be optimized later. + + +Co-existing with unmapping guest private memory +=============================================== +TODO. This needs to be addressed. + + +Restrictions or future work +=========================== +The following features aren't supported yet at the moment. + +* optimizing non-concurrent zap +* Large page +* Page migration From patchwork Mon Feb 26 08:27:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571562 Received: from mgamail.intel.com (unknown [192.198.163.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7436B12FF84; Mon, 26 Feb 2024 08:29:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.8 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936163; cv=none; b=cKYhP+UyqT8H6IRs7nW7mkoj8Qrc8TPkoVJbB/KWJzUw1SIHe808NAr37eT8COkagljaFFiqsuLGkepIoZoAdaRkHQzsr2GMeFYLrWcUjlt1F7HmBwwBBoVhXqOn8YU1jZzcXfJcS6CDE9nSeb7BtQCN3nNl01sfTWoOZhkyuB8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936163; c=relaxed/simple; bh=GYUoCTMLboGXsWuOy4m9bSJkkeOM9TwHQqtrMXvV0wo=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=qH2RQqojcNlmnvMT88Sauh5LuEJcSR2va/MnzgIoSDNLPVb7iflRIFMCTWC2x6QwGjthDqFI8r2ioF7ltWEp5+lHqAcrTfDaJUgPeJoI5SWPWB324w76eDZnHOuGybAzFwITyuISf9dsprL+tatI/iV32OMYVoXAVdn90KSkR74= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=AP+XfOpS; arc=none smtp.client-ip=192.198.163.8 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="AP+XfOpS" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936161; x=1740472161; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=GYUoCTMLboGXsWuOy4m9bSJkkeOM9TwHQqtrMXvV0wo=; b=AP+XfOpSMOZs4DCCiNDuOS71873UyxcHmpcYWtKPnpCDpkKho2fwE2bt VwbBWqBDndYh94aNJHqm4Daf0IqO3xX6IyJapPn8FrN6I4l1nPNFRUpnD irT6jJB5fDDZCZPdh03yY5U34Er4VRTXwj9Uh7pCY4IW7VcI/5XwRJBm5 jz3CCBAp1Cjrf3lBcpZ+D6regH1T4nPVWlCqlAPp+6Y8/DDgAuvaxjlM4 fDmWDmB+nGI5blH5JuaMs4YdpBwaa6bDQCJnOqWiAQon0nvTwzcf3pW2H 4wFkKdWz+/Np/ojO0P57OQ0Yyh70SKYFgL9IHwLr8KNyyg4WP+0c9zLBM Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="20751427" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="20751427" Received: from fmviesa008.fm.intel.com ([10.60.135.148]) by fmvoesa102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:29:13 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6735163" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by fmviesa008-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:29:13 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 129/130] RFC: KVM: x86: Add x86 callback to check cpuid Date: Mon, 26 Feb 2024 00:27:11 -0800 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata The x86 backend should check the consistency of KVM_SET_CPUID2 because it has its constraint. Add a callback for it. The backend code will come as another patch. Suggested-by: Sean Christopherson Link: https://lore.kernel.org/lkml/ZDiGpCkXOcCm074O@google.com/ Signed-off-by: Isaku Yamahata --- arch/x86/include/asm/kvm-x86-ops.h | 2 ++ arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/cpuid.c | 20 ++++++++++++-------- 3 files changed, 15 insertions(+), 8 deletions(-) diff --git a/arch/x86/include/asm/kvm-x86-ops.h b/arch/x86/include/asm/kvm-x86-ops.h index 85c04aad6ab3..3a7140129855 100644 --- a/arch/x86/include/asm/kvm-x86-ops.h +++ b/arch/x86/include/asm/kvm-x86-ops.h @@ -20,6 +20,8 @@ KVM_X86_OP(hardware_disable) KVM_X86_OP(hardware_unsetup) KVM_X86_OP_OPTIONAL_RET0(offline_cpu) KVM_X86_OP(has_emulated_msr) +/* TODO: Once all backend implemented this op, remove _OPTIONAL_RET0. */ +KVM_X86_OP_OPTIONAL_RET0(vcpu_check_cpuid) KVM_X86_OP(vcpu_after_set_cpuid) KVM_X86_OP(is_vm_type_supported) KVM_X86_OP_OPTIONAL(max_vcpus); diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 920fb771246b..e4d40e31fc31 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1638,6 +1638,7 @@ struct kvm_x86_ops { void (*hardware_unsetup)(void); int (*offline_cpu)(void); bool (*has_emulated_msr)(struct kvm *kvm, u32 index); + int (*vcpu_check_cpuid)(struct kvm_vcpu *vcpu, struct kvm_cpuid_entry2 *e2, int nent); void (*vcpu_after_set_cpuid)(struct kvm_vcpu *vcpu); bool (*is_vm_type_supported)(unsigned long vm_type); diff --git a/arch/x86/kvm/cpuid.c b/arch/x86/kvm/cpuid.c index 8cdcd6f406aa..b57006943247 100644 --- a/arch/x86/kvm/cpuid.c +++ b/arch/x86/kvm/cpuid.c @@ -136,6 +136,7 @@ static int kvm_check_cpuid(struct kvm_vcpu *vcpu, { struct kvm_cpuid_entry2 *best; u64 xfeatures; + int r; /* * The existing code assumes virtual address is 48-bit or 57-bit in the @@ -155,15 +156,18 @@ static int kvm_check_cpuid(struct kvm_vcpu *vcpu, * enabling in the FPU, e.g. to expand the guest XSAVE state size. */ best = cpuid_entry2_find(entries, nent, 0xd, 0); - if (!best) - return 0; - - xfeatures = best->eax | ((u64)best->edx << 32); - xfeatures &= XFEATURE_MASK_USER_DYNAMIC; - if (!xfeatures) - return 0; + if (best) { + xfeatures = best->eax | ((u64)best->edx << 32); + xfeatures &= XFEATURE_MASK_USER_DYNAMIC; + if (xfeatures) { + r = fpu_enable_guest_xfd_features(&vcpu->arch.guest_fpu, + xfeatures); + if (r) + return r; + } + } - return fpu_enable_guest_xfd_features(&vcpu->arch.guest_fpu, xfeatures); + return static_call(kvm_x86_vcpu_check_cpuid)(vcpu, entries, nent); } /* Check whether the supplied CPUID data is equal to what is already set for the vCPU. */ From patchwork Mon Feb 26 08:27:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Isaku Yamahata X-Patchwork-Id: 13571564 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8FB90130E4A; Mon, 26 Feb 2024 08:29:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.8 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936165; cv=none; b=lOMCA6EGVImwHFvxfOkObkyU28f2m2Ke1Fy8beUXCVpjRV2D9KghCNjMU3TXU5dAwFel64ptfkbxCIzBMd9QdYAbNViiwX+XOETD6J1LRqCkTxk738TeA47PADC/3iGNndHXB+6ryQ3w3ry/wCmwADES+8HPQZOudgpH/T2LwUI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708936165; c=relaxed/simple; bh=zdJVqbTu3xcs4a9uKIZqxUpxrTdKz6FK+bu86cLPFRI=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=t2J/co7IsMucmSiTyc0xEFw/VH8gtYHTUVyZeT0gI5Xzy5o52ZPrkzJ32SmPfmLPXIE20GReLpntNj5N3dqbl3+xgoLgJ4IifZPpXg2i0tLpbjB3SaV1pKj7/PA80U700WFoOn5vkUnEQSWLf8cli0MSo/PonHaQmRYSMvcK/z8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=Vhr1Tt0d; arc=none smtp.client-ip=192.198.163.8 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="Vhr1Tt0d" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1708936163; x=1740472163; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=zdJVqbTu3xcs4a9uKIZqxUpxrTdKz6FK+bu86cLPFRI=; b=Vhr1Tt0deJGEMHzpY5nm1ngfZ5yf9NT6GwCDwkhWzHwVBCygOEOqv3dT VbFKKrMJqL2D9+iyk9ErL2+6immqHnKRDGJRP18YZ0FhWfA7qMSlftFOz RDgt4hQHcveYeneP+RMW2sxJ+BMPFVurZ/mIHN0IZcYCoHdfErqHZXf3J Mi5eB5nIE8aKw0sqnYhbDkzUIGbsH8P8zg4+H9McCMcvTkNt4vqUdseQT Nji1lqKcllBjHH3tzjWUORfGtRmOrdSDepegDpGqz064/qknc04PwMm1c DYGARHSQ5yzOu5lATFAvhzK+dOEVBud5exfgr/XE0R0c1SCTsgM7R25f2 Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10995"; a="20751431" X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="20751431" Received: from fmviesa008.fm.intel.com ([10.60.135.148]) by fmvoesa102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:29:13 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.06,185,1705392000"; d="scan'208";a="6735168" Received: from ls.sc.intel.com (HELO localhost) ([172.25.112.31]) by fmviesa008-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Feb 2024 00:29:13 -0800 From: isaku.yamahata@intel.com To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: isaku.yamahata@intel.com, isaku.yamahata@gmail.com, Paolo Bonzini , erdemaktas@google.com, Sean Christopherson , Sagi Shahar , Kai Huang , chen.bo@intel.com, hang.yuan@intel.com, tina.zhang@intel.com Subject: [PATCH v19 130/130] RFC: KVM: x86, TDX: Add check for KVM_SET_CPUID2 Date: Mon, 26 Feb 2024 00:27:12 -0800 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Isaku Yamahata Implement a hook of KVM_SET_CPUID2 for additional consistency check. Intel TDX or AMD SEV has a restriction on the value of cpuid. For example, some values must be the same between all vcpus. Check if the new values are consistent with the old values. The check is light because the cpuid consistency is very model specific and complicated. The user space VMM should set cpuid and MSRs consistently. Suggested-by: Sean Christopherson Link: https://lore.kernel.org/lkml/ZDiGpCkXOcCm074O@google.com/ Signed-off-by: Isaku Yamahata --- v18: - Use TDH.SYS.RD() instead of struct tdsysinfo_struct --- arch/x86/kvm/vmx/main.c | 10 ++++++ arch/x86/kvm/vmx/tdx.c | 66 ++++++++++++++++++++++++++++++++++---- arch/x86/kvm/vmx/tdx.h | 7 ++++ arch/x86/kvm/vmx/x86_ops.h | 4 +++ 4 files changed, 81 insertions(+), 6 deletions(-) diff --git a/arch/x86/kvm/vmx/main.c b/arch/x86/kvm/vmx/main.c index 2cd404fd7176..1a979ec644d0 100644 --- a/arch/x86/kvm/vmx/main.c +++ b/arch/x86/kvm/vmx/main.c @@ -432,6 +432,15 @@ static void vt_vcpu_deliver_init(struct kvm_vcpu *vcpu) kvm_vcpu_deliver_init(vcpu); } +static int vt_vcpu_check_cpuid(struct kvm_vcpu *vcpu, + struct kvm_cpuid_entry2 *e2, int nent) +{ + if (is_td_vcpu(vcpu)) + return tdx_vcpu_check_cpuid(vcpu, e2, nent); + + return 0; +} + static void vt_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu) { if (is_td_vcpu(vcpu)) @@ -1125,6 +1134,7 @@ struct kvm_x86_ops vt_x86_ops __initdata = { .get_exit_info = vt_get_exit_info, + .vcpu_check_cpuid = vt_vcpu_check_cpuid, .vcpu_after_set_cpuid = vt_vcpu_after_set_cpuid, .has_wbinvd_exit = cpu_has_vmx_wbinvd_exit, diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c index 7be1be161dc2..a71093f7c3e3 100644 --- a/arch/x86/kvm/vmx/tdx.c +++ b/arch/x86/kvm/vmx/tdx.c @@ -547,6 +547,9 @@ void tdx_vm_free(struct kvm *kvm) free_page((unsigned long)__va(kvm_tdx->tdr_pa)); kvm_tdx->tdr_pa = 0; + + kfree(kvm_tdx->cpuid); + kvm_tdx->cpuid = NULL; } static int tdx_do_tdh_mng_key_config(void *param) @@ -661,6 +664,39 @@ int tdx_vcpu_create(struct kvm_vcpu *vcpu) return 0; } +int tdx_vcpu_check_cpuid(struct kvm_vcpu *vcpu, struct kvm_cpuid_entry2 *e2, int nent) +{ + struct kvm_tdx *kvm_tdx = to_kvm_tdx(vcpu->kvm); + int i; + + /* + * Simple check that new cpuid is consistent with created one. + * For simplicity, only trivial check. Don't try comprehensive checks + * with the cpuid virtualization table in the TDX module spec. + */ + for (i = 0; i < tdx_info->num_cpuid_config; i++) { + const struct kvm_tdx_cpuid_config *c = &tdx_info->cpuid_configs[i]; + u32 index = c->sub_leaf == KVM_TDX_CPUID_NO_SUBLEAF ? 0 : c->sub_leaf; + const struct kvm_cpuid_entry2 *old = + kvm_find_cpuid_entry2(kvm_tdx->cpuid, kvm_tdx->cpuid_nent, + c->leaf, index); + const struct kvm_cpuid_entry2 *new = kvm_find_cpuid_entry2(e2, nent, + c->leaf, index); + + if (!!old != !!new) + return -EINVAL; + if (!old && !new) + continue; + + if ((old->eax ^ new->eax) & c->eax || + (old->ebx ^ new->ebx) & c->ebx || + (old->ecx ^ new->ecx) & c->ecx || + (old->edx ^ new->edx) & c->edx) + return -EINVAL; + } + return 0; +} + void tdx_vcpu_load(struct kvm_vcpu *vcpu, int cpu) { struct vcpu_tdx *tdx = to_tdx(vcpu); @@ -2205,9 +2241,10 @@ static int setup_tdparams_eptp_controls(struct kvm_cpuid2 *cpuid, return 0; } -static void setup_tdparams_cpuids(struct kvm_cpuid2 *cpuid, +static void setup_tdparams_cpuids(struct kvm *kvm, struct kvm_cpuid2 *cpuid, struct td_params *td_params) { + struct kvm_tdx *kvm_tdx = to_kvm_tdx(kvm); int i; /* @@ -2215,6 +2252,7 @@ static void setup_tdparams_cpuids(struct kvm_cpuid2 *cpuid, * be same to the one of struct tdsysinfo.{num_cpuid_config, cpuid_configs} * It's assumed that td_params was zeroed. */ + kvm_tdx->cpuid_nent = 0; for (i = 0; i < tdx_info->num_cpuid_config; i++) { const struct kvm_tdx_cpuid_config *c = &tdx_info->cpuid_configs[i]; /* KVM_TDX_CPUID_NO_SUBLEAF means index = 0. */ @@ -2237,6 +2275,10 @@ static void setup_tdparams_cpuids(struct kvm_cpuid2 *cpuid, value->ebx = entry->ebx & c->ebx; value->ecx = entry->ecx & c->ecx; value->edx = entry->edx & c->edx; + + /* Remember the setting to check for KVM_SET_CPUID2. */ + kvm_tdx->cpuid[kvm_tdx->cpuid_nent] = *entry; + kvm_tdx->cpuid_nent++; } } @@ -2324,7 +2366,7 @@ static int setup_tdparams(struct kvm *kvm, struct td_params *td_params, ret = setup_tdparams_eptp_controls(cpuid, td_params); if (ret) return ret; - setup_tdparams_cpuids(cpuid, td_params); + setup_tdparams_cpuids(kvm, cpuid, td_params); ret = setup_tdparams_xfam(cpuid, td_params); if (ret) return ret; @@ -2548,11 +2590,18 @@ static int tdx_td_init(struct kvm *kvm, struct kvm_tdx_cmd *cmd) if (cmd->flags) return -EINVAL; - init_vm = kzalloc(sizeof(*init_vm) + - sizeof(init_vm->cpuid.entries[0]) * KVM_MAX_CPUID_ENTRIES, - GFP_KERNEL); - if (!init_vm) + WARN_ON_ONCE(kvm_tdx->cpuid); + kvm_tdx->cpuid = kzalloc(flex_array_size(init_vm, cpuid.entries, KVM_MAX_CPUID_ENTRIES), + GFP_KERNEL); + if (!kvm_tdx->cpuid) return -ENOMEM; + + init_vm = kzalloc(struct_size(init_vm, cpuid.entries, KVM_MAX_CPUID_ENTRIES), + GFP_KERNEL); + if (!init_vm) { + ret = -ENOMEM; + goto out; + } if (copy_from_user(init_vm, (void __user *)cmd->data, sizeof(*init_vm))) { ret = -EFAULT; goto out; @@ -2602,6 +2651,11 @@ static int tdx_td_init(struct kvm *kvm, struct kvm_tdx_cmd *cmd) out: /* kfree() accepts NULL. */ + if (ret) { + kfree(kvm_tdx->cpuid); + kvm_tdx->cpuid = NULL; + kvm_tdx->cpuid_nent = 0; + } kfree(init_vm); kfree(td_params); return ret; diff --git a/arch/x86/kvm/vmx/tdx.h b/arch/x86/kvm/vmx/tdx.h index 11c74c34555f..af3a2b8afee8 100644 --- a/arch/x86/kvm/vmx/tdx.h +++ b/arch/x86/kvm/vmx/tdx.h @@ -31,6 +31,13 @@ struct kvm_tdx { u64 tsc_offset; + /* + * For KVM_SET_CPUID to check consistency. Remember the one passed to + * TDH.MNG_INIT + */ + int cpuid_nent; + struct kvm_cpuid_entry2 *cpuid; + /* For KVM_MEMORY_MAPPING */ struct mutex source_lock; struct page *source_page; diff --git a/arch/x86/kvm/vmx/x86_ops.h b/arch/x86/kvm/vmx/x86_ops.h index c507f1513dac..6b067842a67f 100644 --- a/arch/x86/kvm/vmx/x86_ops.h +++ b/arch/x86/kvm/vmx/x86_ops.h @@ -162,6 +162,8 @@ u8 tdx_get_mt_mask(struct kvm_vcpu *vcpu, gfn_t gfn, bool is_mmio); void tdx_deliver_interrupt(struct kvm_lapic *apic, int delivery_mode, int trig_mode, int vector); +int tdx_vcpu_check_cpuid(struct kvm_vcpu *vcpu, struct kvm_cpuid_entry2 *e2, + int nent); void tdx_inject_nmi(struct kvm_vcpu *vcpu); void tdx_get_exit_info(struct kvm_vcpu *vcpu, u32 *reason, u64 *info1, u64 *info2, u32 *intr_info, u32 *error_code); @@ -221,6 +223,8 @@ static inline u8 tdx_get_mt_mask(struct kvm_vcpu *vcpu, gfn_t gfn, bool is_mmio) static inline void tdx_deliver_interrupt(struct kvm_lapic *apic, int delivery_mode, int trig_mode, int vector) {} +static inline int tdx_vcpu_check_cpuid(struct kvm_vcpu *vcpu, struct kvm_cpuid_entry2 *e2, + int nent) { return -EOPNOTSUPP; } static inline void tdx_inject_nmi(struct kvm_vcpu *vcpu) {} static inline void tdx_get_exit_info(struct kvm_vcpu *vcpu, u32 *reason, u64 *info1, u64 *info2, u32 *intr_info, u32 *error_code) {}