From patchwork Mon Apr 3 17:43:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tianyu Lan X-Patchwork-Id: 13198514 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6AEDCC761AF for ; Mon, 3 Apr 2023 17:44:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231932AbjDCRoS (ORCPT ); Mon, 3 Apr 2023 13:44:18 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50578 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231937AbjDCRoN (ORCPT ); Mon, 3 Apr 2023 13:44:13 -0400 Received: from mail-pl1-x629.google.com (mail-pl1-x629.google.com [IPv6:2607:f8b0:4864:20::629]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 113ECD9; Mon, 3 Apr 2023 10:44:12 -0700 (PDT) Received: by mail-pl1-x629.google.com with SMTP id c18so28790448ple.11; Mon, 03 Apr 2023 10:44:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1680543851; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=9a3JR7vmGcwJg3yramprshnzRS7TRETQfUc7rKzyISs=; b=o77BycsmUJd3tgRAtHCFRAgRLG5KI4CHgVSXZE1lz0jnsV5+iyfvhBWMLmiy+FrcKs qylR+OMJf8zJPwIwsC0y/Q8K9v5qugJyNhKVkfG37nUw7/SZieZ5rNftr9viE8H0Xaxs lynxk4lwNKytUxxDDcAp7JqUyOF1FQIm7FP2yMTT6nxcazpaJsGJ4ivLknVzOzdY10sz RKM1wk1QLr9qH4dmoLVxnmzj3dnvQcfDmB3u2Dw3cRNdFHJ0HuSfrd/73boG8ncea53C TRYiTrU7fIhYLoPkkNgebisPM81mjywnSDSV/c6gPc252T106k0sH4oQPQ/qs12JJNTe 02JQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680543851; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9a3JR7vmGcwJg3yramprshnzRS7TRETQfUc7rKzyISs=; b=Bn0Uak5+8LtxuYtQ3bCcbBd/0Fnyd9zvVk+tm5ko3fGti5Sol2QIHtcYxTQSAEj/F9 xvdN/YLhwzkQDiFROLFjRUXSlk42bm0sfDg12w9U684Pqi9mZlEgXxwqrypz2sYe9uqq c8jrz5yccPkVQamJ/swwiYrlf5AzZSKyc91cvn2EaITpc0cwDI9VoLsg5e0OV7sWNkVU RJJ52+BqOZYJ+XiML5yrnZ6T2Qnzgm5l/TdDSD6iMdwm3eQ8GWaRmqcu/XodKO0hBb9y S2HbDDnY0HbcJG4mmUu4xQp107hizS5pTadDz/2zo62YEgPbpggt7h6EhzRqDin/SBNI JG/A== X-Gm-Message-State: AAQBX9fRFxCzoBhauhRSNL3oC/BxP9qld71EbgYi4Yvwng9OE6l96C0Y fUHD/bc+n3/unHvkQyNOaZU= X-Google-Smtp-Source: AKy350aLHm4XZfaVrgKgPOZiQa6sMJpGyjAQVQHZ+r7zN0g4xwqJx45lr17zX1a/YHYi26pbECPktg== X-Received: by 2002:a17:902:f691:b0:1a2:8940:6dae with SMTP id l17-20020a170902f69100b001a289406daemr19939898plg.34.1680543851406; Mon, 03 Apr 2023 10:44:11 -0700 (PDT) Received: from ubuntu-Virtual-Machine.corp.microsoft.com ([2001:4898:80e8:2:8635:6e96:35c1:c560]) by smtp.gmail.com with ESMTPSA id jj21-20020a170903049500b001a19196af48sm6883803plb.64.2023.04.03.10.44.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 03 Apr 2023 10:44:11 -0700 (PDT) From: Tianyu Lan To: luto@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, seanjc@google.com, pbonzini@redhat.com, jgross@suse.com, tiala@microsoft.com, kirill@shutemov.name, jiangshan.ljs@antgroup.com, peterz@infradead.org, ashish.kalra@amd.com, srutherford@google.com, akpm@linux-foundation.org, anshuman.khandual@arm.com, pawan.kumar.gupta@linux.intel.com, adrian.hunter@intel.com, daniel.sneddon@linux.intel.com, alexander.shishkin@linux.intel.com, sandipan.das@amd.com, ray.huang@amd.com, brijesh.singh@amd.com, michael.roth@amd.com, thomas.lendacky@amd.com, venu.busireddy@oracle.com, sterritt@google.com, tony.luck@intel.com, samitolvanen@google.com, fenghua.yu@intel.com Cc: pangupta@amd.com, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-hyperv@vger.kernel.org, linux-arch@vger.kernel.org Subject: [RFC PATCH V4 01/17] x86/hyperv: Add sev-snp enlightened guest static key Date: Mon, 3 Apr 2023 13:43:49 -0400 Message-Id: <20230403174406.4180472-2-ltykernel@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230403174406.4180472-1-ltykernel@gmail.com> References: <20230403174406.4180472-1-ltykernel@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Tianyu Lan Introduce static key isolation_type_en_snp for enlightened sev-snp guest check. Signed-off-by: Tianyu Lan --- Change since RFC v3: * Remove some Hyper-V specific config setting --- arch/x86/hyperv/ivm.c | 11 +++++++++++ arch/x86/include/asm/mshyperv.h | 3 +++ arch/x86/kernel/cpu/mshyperv.c | 9 +++++++-- drivers/hv/hv_common.c | 6 ++++++ 4 files changed, 27 insertions(+), 2 deletions(-) diff --git a/arch/x86/hyperv/ivm.c b/arch/x86/hyperv/ivm.c index 127d5b7b63de..368b2731950e 100644 --- a/arch/x86/hyperv/ivm.c +++ b/arch/x86/hyperv/ivm.c @@ -409,3 +409,14 @@ bool hv_isolation_type_snp(void) { return static_branch_unlikely(&isolation_type_snp); } + +DEFINE_STATIC_KEY_FALSE(isolation_type_en_snp); +/* + * hv_isolation_type_en_snp - Check system runs in the AMD SEV-SNP based + * isolation enlightened VM. + */ +bool hv_isolation_type_en_snp(void) +{ + return static_branch_unlikely(&isolation_type_en_snp); +} + diff --git a/arch/x86/include/asm/mshyperv.h b/arch/x86/include/asm/mshyperv.h index e3cef98a0142..2fa40c6ca99f 100644 --- a/arch/x86/include/asm/mshyperv.h +++ b/arch/x86/include/asm/mshyperv.h @@ -22,6 +22,7 @@ union hv_ghcb; DECLARE_STATIC_KEY_FALSE(isolation_type_snp); +DECLARE_STATIC_KEY_FALSE(isolation_type_en_snp); typedef int (*hyperv_fill_flush_list_func)( struct hv_guest_mapping_flush_list *flush, @@ -36,6 +37,8 @@ extern void *hv_hypercall_pg; extern u64 hv_current_partition_id; +extern bool hv_isolation_type_en_snp(void); + extern union hv_ghcb * __percpu *hv_ghcb_pg; int hv_call_deposit_pages(int node, u64 partition_id, u32 num_pages); diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c index ff348ebb6ae2..2f2dcb2370b6 100644 --- a/arch/x86/kernel/cpu/mshyperv.c +++ b/arch/x86/kernel/cpu/mshyperv.c @@ -403,8 +403,12 @@ static void __init ms_hyperv_init_platform(void) pr_info("Hyper-V: Isolation Config: Group A 0x%x, Group B 0x%x\n", ms_hyperv.isolation_config_a, ms_hyperv.isolation_config_b); - if (hv_get_isolation_type() == HV_ISOLATION_TYPE_SNP) + + if (cc_platform_has(CC_ATTR_GUEST_SEV_SNP)) { + static_branch_enable(&isolation_type_en_snp); + } else if (hv_get_isolation_type() == HV_ISOLATION_TYPE_SNP) { static_branch_enable(&isolation_type_snp); + } } if (hv_max_functions_eax >= HYPERV_CPUID_NESTED_FEATURES) { @@ -474,7 +478,8 @@ static void __init ms_hyperv_init_platform(void) #if IS_ENABLED(CONFIG_HYPERV) if ((hv_get_isolation_type() == HV_ISOLATION_TYPE_VBS) || - (hv_get_isolation_type() == HV_ISOLATION_TYPE_SNP)) + (hv_get_isolation_type() == HV_ISOLATION_TYPE_SNP + && !cc_platform_has(CC_ATTR_GUEST_SEV_SNP))) hv_vtom_init(); /* * Setup the hook to get control post apic initialization. diff --git a/drivers/hv/hv_common.c b/drivers/hv/hv_common.c index 6d40b6c7b23b..e3b04e978932 100644 --- a/drivers/hv/hv_common.c +++ b/drivers/hv/hv_common.c @@ -271,6 +271,12 @@ bool __weak hv_isolation_type_snp(void) } EXPORT_SYMBOL_GPL(hv_isolation_type_snp); +bool __weak hv_isolation_type_en_snp(void) +{ + return false; +} +EXPORT_SYMBOL_GPL(hv_isolation_type_en_snp); + void __weak hv_setup_vmbus_handler(void (*handler)(void)) { } From patchwork Mon Apr 3 17:43:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tianyu Lan X-Patchwork-Id: 13198515 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CAB6DC77B6E for ; Mon, 3 Apr 2023 17:44:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231916AbjDCRoX (ORCPT ); Mon, 3 Apr 2023 13:44:23 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50614 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231974AbjDCRoO (ORCPT ); Mon, 3 Apr 2023 13:44:14 -0400 Received: from mail-pj1-x102c.google.com (mail-pj1-x102c.google.com [IPv6:2607:f8b0:4864:20::102c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7C2D11FFD; Mon, 3 Apr 2023 10:44:13 -0700 (PDT) Received: by mail-pj1-x102c.google.com with SMTP id gp15-20020a17090adf0f00b0023d1bbd9f9eso33452136pjb.0; Mon, 03 Apr 2023 10:44:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1680543853; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=UgABqQg7Pn/Hzm9HMPl9PubPVqr5Bo74B/K3wOK+7x0=; b=m6zyjDNmIujNACAnZPNBH+JtlTQO+NSM+A4PqzZtUsVcOHEUT0butES0y/9K5k88cN LbTNdMgXryXLF6Y6EinZ9dGvKjTzNDFQ402y/W/Bdg7TvdoDMs1UWsrGqWamPquMWAry xvholwMIbB3nU53gIiO6zqRUylVrWd3AB9plxiWrzk4JY3oUL3SS1q5ivQVM2iwmm/mm sYuzfn1UT/P2/irJMcICoXWCiRDc5DHkbZQ7Th75B/I0a2VKr2iqng7wCdvoQDmsOisZ Xcx2pmr8kY5D22AELD2kjXvEhT/u+h1qwCKQz+cdfnkUeZ70jkkTaDYh4aF7V5D3CETp 8F3g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680543853; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=UgABqQg7Pn/Hzm9HMPl9PubPVqr5Bo74B/K3wOK+7x0=; b=30WbAIAzbJL5L9HHasXpUL9Etw2KXJKsWR6Pv0id0gTUXFCuUzOCuuM7VY+LH8HxEG /55Ye7DOeSJ/w1cZ/ibWD0PTA4mM+R7lCTSU2pVTTZwFEMsgE74lAEHVBvgC+yGGCQDS HHPgQaySKi1NdO1u80cZZiejT4iClpw4BMSjoDzq+8luYNdPXcvk5Bjv71bZKowX2Rfi F/TybsXlTL+8Z7pxNDqempgctm0sfnvKaEzABaGOJURcr9Gp68DsP/QPQ495OTYMeaZW UMQ+kwpmHrxMxmVj2cHSMZC0kDyzYvLPZIkAY8ayd5D/Z1+TwiIYVUDoVLCP0zEWNWgq iUnw== X-Gm-Message-State: AAQBX9dvjZUhBzIFfT643vGm29RJ09wiE80YABmZz9sjZaI8UPEJwqXH sODkldbqWcr8/MbByoqm1Fs= X-Google-Smtp-Source: AKy350Z3mjEvpKV3whhnvT+S8Tg91v6ztFgOOYeKgpBXCLwcP5Ii/xxB1iaXadF2QcOQAtEZdo/hGg== X-Received: by 2002:a17:902:d411:b0:1a1:c8b3:3fe1 with SMTP id b17-20020a170902d41100b001a1c8b33fe1mr29560905ple.31.1680543852787; Mon, 03 Apr 2023 10:44:12 -0700 (PDT) Received: from ubuntu-Virtual-Machine.corp.microsoft.com ([2001:4898:80e8:2:8635:6e96:35c1:c560]) by smtp.gmail.com with ESMTPSA id jj21-20020a170903049500b001a19196af48sm6883803plb.64.2023.04.03.10.44.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 03 Apr 2023 10:44:12 -0700 (PDT) From: Tianyu Lan To: luto@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, seanjc@google.com, pbonzini@redhat.com, jgross@suse.com, tiala@microsoft.com, kirill@shutemov.name, jiangshan.ljs@antgroup.com, peterz@infradead.org, ashish.kalra@amd.com, srutherford@google.com, akpm@linux-foundation.org, anshuman.khandual@arm.com, pawan.kumar.gupta@linux.intel.com, adrian.hunter@intel.com, daniel.sneddon@linux.intel.com, alexander.shishkin@linux.intel.com, sandipan.das@amd.com, ray.huang@amd.com, brijesh.singh@amd.com, michael.roth@amd.com, thomas.lendacky@amd.com, venu.busireddy@oracle.com, sterritt@google.com, tony.luck@intel.com, samitolvanen@google.com, fenghua.yu@intel.com Cc: pangupta@amd.com, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-hyperv@vger.kernel.org, linux-arch@vger.kernel.org Subject: [RFC PATCH V4 02/17] Drivers: hv: vmbus: Decrypt vmbus ring buffer Date: Mon, 3 Apr 2023 13:43:50 -0400 Message-Id: <20230403174406.4180472-3-ltykernel@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230403174406.4180472-1-ltykernel@gmail.com> References: <20230403174406.4180472-1-ltykernel@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Tianyu Lan The ring buffer is remapped in the hv_ringbuffer_init() and it should be with decrypt flag in order to share it with hypervisor in sev-snp enlightened guest. Signed-off-by: Tianyu Lan --- arch/x86/hyperv/hv_init.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/arch/x86/hyperv/hv_init.c b/arch/x86/hyperv/hv_init.c index a5f9474f08e1..9f3e2d71d015 100644 --- a/arch/x86/hyperv/hv_init.c +++ b/arch/x86/hyperv/hv_init.c @@ -18,6 +18,7 @@ #include #include #include +#include #include #include #include @@ -113,6 +114,11 @@ static int hv_cpu_init(unsigned int cpu) } if (!WARN_ON(!(*hvp))) { + if (hv_isolation_type_en_snp()) { + WARN_ON_ONCE(set_memory_decrypted((unsigned long)(*hvp), 1)); + memset(*hvp, 0, PAGE_SIZE); + } + msr.enable = 1; wrmsrl(HV_X64_MSR_VP_ASSIST_PAGE, msr.as_uint64); } From patchwork Mon Apr 3 17:43:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tianyu Lan X-Patchwork-Id: 13198517 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8FA73C76196 for ; Mon, 3 Apr 2023 17:44:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231767AbjDCRo1 (ORCPT ); Mon, 3 Apr 2023 13:44:27 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50600 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232047AbjDCRoS (ORCPT ); Mon, 3 Apr 2023 13:44:18 -0400 Received: from mail-pj1-x1035.google.com (mail-pj1-x1035.google.com [IPv6:2607:f8b0:4864:20::1035]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DC6B02114; Mon, 3 Apr 2023 10:44:14 -0700 (PDT) Received: by mail-pj1-x1035.google.com with SMTP id f6-20020a17090ac28600b0023b9bf9eb63so31369029pjt.5; Mon, 03 Apr 2023 10:44:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1680543854; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=CizkYYSOlAsJPe03D8rmOV0gOBSN8nSrbNEPB/ot/CQ=; b=Hf9bG1s/shcJEEZGFOwmA96pHybN+72axE9tV+SYvePQjub19rmz7dUlO/mSUWuO4Q IkvilGjqL8wpqMm8XNw9absyyc4Wzno2YaW3DtPe5ajk7D8mh3JCxRRDrQH8p/yNmy3Q tu3idvP4NwClNZ5ObG0SMmmbtnATX+VLQmkLQkCY7pAcAdut0QSS8pDEwZJt9iCRCAia jamoT2yFcuNNpWwAHq9jnu8KD0IHYuDx+reiEwDKKV5TAuuXZzOSevUklgJElr7gRA5n jJvW8g9UIkG806oET4visFhJSMnQ5AahCTGOGUdQY2sEv/R7oZoaNa1pRDwhimPvDyzx QnsQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680543854; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=CizkYYSOlAsJPe03D8rmOV0gOBSN8nSrbNEPB/ot/CQ=; b=qLqG2t8Qtaaz9YuKD1xD1HkIYjX94wID2ivXKyMAvH/GETEd7fLQ0BpetaYOocIWgv wCFmwGswTPKga0zSjvptxzZKX71PfdleWutolnF8vuFrhkFJRbePGErJpbTp+0hID9lC MpQROF6YCyNAefzIul3E14qWjHM0mpl0M5FK37oOOBUV5DDxv+oGQdgnufkHxKYz+gz4 oTte24yDWNRTprIlbLL0mLiPj9w0u9bR6qceNhq6vjgs58md5z+m5Lw7ePfIPWPHIQjO K9Rt/3P9G7qpLAerTh34cw4AKQl0kfSU+ZV6oh69hDUeQP4G3+bZJHeVygCG7ZUXs6sh uRLA== X-Gm-Message-State: AAQBX9eqe36VRwFMt1DKpGKvXBDfjK4/2cx5+kp/ZmbEMusd+qacLQfO OGGOz+Hmse183jc8boyWLyA= X-Google-Smtp-Source: AKy350ahr1LccUTgtxrRakmD2iirEWBU/Y6C+6hduhZ/hoRdZ0l+1F/0dUh0cEDOJaFt8YS1rwaPlg== X-Received: by 2002:a17:902:c404:b0:19f:2328:beee with SMTP id k4-20020a170902c40400b0019f2328beeemr47498322plk.11.1680543854233; Mon, 03 Apr 2023 10:44:14 -0700 (PDT) Received: from ubuntu-Virtual-Machine.corp.microsoft.com ([2001:4898:80e8:2:8635:6e96:35c1:c560]) by smtp.gmail.com with ESMTPSA id jj21-20020a170903049500b001a19196af48sm6883803plb.64.2023.04.03.10.44.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 03 Apr 2023 10:44:13 -0700 (PDT) From: Tianyu Lan To: luto@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, seanjc@google.com, pbonzini@redhat.com, jgross@suse.com, tiala@microsoft.com, kirill@shutemov.name, jiangshan.ljs@antgroup.com, peterz@infradead.org, ashish.kalra@amd.com, srutherford@google.com, akpm@linux-foundation.org, anshuman.khandual@arm.com, pawan.kumar.gupta@linux.intel.com, adrian.hunter@intel.com, daniel.sneddon@linux.intel.com, alexander.shishkin@linux.intel.com, sandipan.das@amd.com, ray.huang@amd.com, brijesh.singh@amd.com, michael.roth@amd.com, thomas.lendacky@amd.com, venu.busireddy@oracle.com, sterritt@google.com, tony.luck@intel.com, samitolvanen@google.com, fenghua.yu@intel.com Cc: pangupta@amd.com, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-hyperv@vger.kernel.org, linux-arch@vger.kernel.org Subject: [RFC PATCH V4 03/17] x86/hyperv: Set Virtual Trust Level in VMBus init message Date: Mon, 3 Apr 2023 13:43:51 -0400 Message-Id: <20230403174406.4180472-4-ltykernel@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230403174406.4180472-1-ltykernel@gmail.com> References: <20230403174406.4180472-1-ltykernel@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Tianyu Lan sev-snp guest provides vtl(Virtual Trust Level) and get it from hyperv hvcall via HVCALL_GET_VP_REGISTERS. Set target vtl in the VMBus init message. Signed-off-by: Tianyu Lan --- Change since RFC v3: * Use the standard helper functions to check hypercall result * Fix coding style Change since RFC v2: * Rename get_current_vtl() to get_vtl() * Fix some coding style issues --- arch/x86/hyperv/hv_init.c | 36 ++++++++++++++++++++++++++++++ arch/x86/include/asm/hyperv-tlfs.h | 7 ++++++ drivers/hv/connection.c | 1 + include/asm-generic/mshyperv.h | 2 ++ include/linux/hyperv.h | 4 ++-- 5 files changed, 48 insertions(+), 2 deletions(-) diff --git a/arch/x86/hyperv/hv_init.c b/arch/x86/hyperv/hv_init.c index 9f3e2d71d015..7602db5ad4aa 100644 --- a/arch/x86/hyperv/hv_init.c +++ b/arch/x86/hyperv/hv_init.c @@ -384,6 +384,40 @@ static void __init hv_get_partition_id(void) local_irq_restore(flags); } +static u8 __init get_vtl(void) +{ + u64 control = HV_HYPERCALL_REP_COMP_1 | HVCALL_GET_VP_REGISTERS; + struct hv_get_vp_registers_input *input; + struct hv_get_vp_registers_output *output; + u64 vtl = 0; + u64 ret; + unsigned long flags; + + local_irq_save(flags); + input = *(struct hv_get_vp_registers_input **)this_cpu_ptr(hyperv_pcpu_input_arg); + output = (struct hv_get_vp_registers_output *)input; + if (!input) { + local_irq_restore(flags); + goto done; + } + + memset(input, 0, sizeof(*input) + sizeof(input->element[0])); + input->header.partitionid = HV_PARTITION_ID_SELF; + input->header.vpindex = HV_VP_INDEX_SELF; + input->header.inputvtl = 0; + input->element[0].name0 = HV_X64_REGISTER_VSM_VP_STATUS; + + ret = hv_do_hypercall(control, input, output); + if (hv_result_success(ret)) + vtl = output->as64.low & HV_X64_VTL_MASK; + else + pr_err("Hyper-V: failed to get VTL!"); + local_irq_restore(flags); + +done: + return vtl; +} + /* * This function is to be invoked early in the boot sequence after the * hypervisor has been detected. @@ -512,6 +546,8 @@ void __init hyperv_init(void) /* Query the VMs extended capability once, so that it can be cached. */ hv_query_ext_cap(0); + /* Find the VTL */ + ms_hyperv.vtl = get_vtl(); return; clean_guest_os_id: diff --git a/arch/x86/include/asm/hyperv-tlfs.h b/arch/x86/include/asm/hyperv-tlfs.h index b4fb75bd1013..3d3f014976e8 100644 --- a/arch/x86/include/asm/hyperv-tlfs.h +++ b/arch/x86/include/asm/hyperv-tlfs.h @@ -301,6 +301,13 @@ enum hv_isolation_type { #define HV_X64_MSR_TIME_REF_COUNT HV_REGISTER_TIME_REF_COUNT #define HV_X64_MSR_REFERENCE_TSC HV_REGISTER_REFERENCE_TSC +/* + * Registers are only accessible via HVCALL_GET_VP_REGISTERS hvcall and + * there is not associated MSR address. + */ +#define HV_X64_REGISTER_VSM_VP_STATUS 0x000D0003 +#define HV_X64_VTL_MASK GENMASK(3, 0) + /* Hyper-V memory host visibility */ enum hv_mem_host_visibility { VMBUS_PAGE_NOT_VISIBLE = 0, diff --git a/drivers/hv/connection.c b/drivers/hv/connection.c index f670cfd2e056..e4c39f4016ad 100644 --- a/drivers/hv/connection.c +++ b/drivers/hv/connection.c @@ -98,6 +98,7 @@ int vmbus_negotiate_version(struct vmbus_channel_msginfo *msginfo, u32 version) */ if (version >= VERSION_WIN10_V5) { msg->msg_sint = VMBUS_MESSAGE_SINT; + msg->msg_vtl = ms_hyperv.vtl; vmbus_connection.msg_conn_id = VMBUS_MESSAGE_CONNECTION_ID_4; } else { msg->interrupt_page = virt_to_phys(vmbus_connection.int_page); diff --git a/include/asm-generic/mshyperv.h b/include/asm-generic/mshyperv.h index afcd9ae9588c..f91fb06634f0 100644 --- a/include/asm-generic/mshyperv.h +++ b/include/asm-generic/mshyperv.h @@ -48,6 +48,7 @@ struct ms_hyperv_info { }; }; u64 shared_gpa_boundary; + u8 vtl; }; extern struct ms_hyperv_info ms_hyperv; extern bool hv_nested; @@ -58,6 +59,7 @@ extern void * __percpu *hyperv_pcpu_output_arg; extern u64 hv_do_hypercall(u64 control, void *inputaddr, void *outputaddr); extern u64 hv_do_fast_hypercall8(u16 control, u64 input8); extern bool hv_isolation_type_snp(void); +extern bool hv_isolation_type_en_snp(void); /* Helper functions that provide a consistent pattern for checking Hyper-V hypercall status. */ static inline int hv_result(u64 status) diff --git a/include/linux/hyperv.h b/include/linux/hyperv.h index bfbc37ce223b..1f2bfec4abde 100644 --- a/include/linux/hyperv.h +++ b/include/linux/hyperv.h @@ -665,8 +665,8 @@ struct vmbus_channel_initiate_contact { u64 interrupt_page; struct { u8 msg_sint; - u8 padding1[3]; - u32 padding2; + u8 msg_vtl; + u8 reserved[6]; }; }; u64 monitor_page1; From patchwork Mon Apr 3 17:43:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tianyu Lan X-Patchwork-Id: 13198516 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C4396C761A6 for ; Mon, 3 Apr 2023 17:44:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232353AbjDCRoZ (ORCPT ); Mon, 3 Apr 2023 13:44:25 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50602 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229780AbjDCRoS (ORCPT ); Mon, 3 Apr 2023 13:44:18 -0400 Received: from mail-pj1-x102f.google.com (mail-pj1-x102f.google.com [IPv6:2607:f8b0:4864:20::102f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 221462134; Mon, 3 Apr 2023 10:44:16 -0700 (PDT) Received: by mail-pj1-x102f.google.com with SMTP id x15so27969903pjk.2; Mon, 03 Apr 2023 10:44:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1680543855; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=OPkIOzll4BT9Ay9ERRkIKNiMh09YPuKAEEznSDGpKUM=; b=GnkvhrAoOuvq1qBFm7b02W8wKlDxceU3aehcnmEc00cKuV5OL2e5Faet0SPwn5Z1zx UR5yWDtNE1MFnT8NYHUCLkGrQuS5F0QgukO6CcK+ccUajPxMtFfaXBJJg2TV/3KdaNVT d9vDluaLz0va6QKE7l4pYkIU0otJND7BF26jDnyfUOFKlNgOfT0Sen39BjrbtkUSievb IJ2eDeg9m1FE8ygKMFuBxKk+UYCi55yiMHvhqd0NXfzPguDRh7S+a2DT47+S27aqE4Ih 5RDld1VifjupHqBi+LJh319PF8je5z5qoruAUFtLURs2jvOosBGfAD5SrtUfycpRZ1d1 1REQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680543855; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=OPkIOzll4BT9Ay9ERRkIKNiMh09YPuKAEEznSDGpKUM=; b=Br9U7pSQX9507SP/IMpVLC0mu1Bs7suf27qcmXjqiGv+7+jg3QXOFJ51DWmAAXnXNk X1IxLCF1GZVadM7Z0/TZHjpAoBo/ed4pEvkc3+seLXHwAQ1t9WsAuocNHA/Txqbcu1E6 Y8uRhhks7grNNxZ7fPxePYQg/A76l2OZleRBpYtqHGHkXvoHySqDEVeXuzJR5QZgN5Z7 Ynpx6NqQSV2K9Sq9TwhC2sX+BgePm+JT8gxMDb255McKbJkkqhlimv/p9F3Ld/pf5HEb TSlE0b1yd/OeT/W2h3Hh5AvCJJaIAHPbsnG1d4Lldoqc7CiYkQSQT9V/e+KdIq60Kaf2 a/Nw== X-Gm-Message-State: AAQBX9e3G1WlnNqTg4xeUySpYQgXj9ol6Xj4j2eg3uZo6Lb2ScfSHK3X H6vHF2Y4I+C4cdMnqpBCF9g= X-Google-Smtp-Source: AKy350ZX2/5cNiNJs7eqEeCmiv4eBiXo2CBjNP1uWsqfNGtesqYRU8AIYRwVeljZF08yiOkLqBIOrQ== X-Received: by 2002:a17:902:d2c1:b0:1a0:7425:4b73 with SMTP id n1-20020a170902d2c100b001a074254b73mr24670101plc.4.1680543855438; Mon, 03 Apr 2023 10:44:15 -0700 (PDT) Received: from ubuntu-Virtual-Machine.corp.microsoft.com ([2001:4898:80e8:2:8635:6e96:35c1:c560]) by smtp.gmail.com with ESMTPSA id jj21-20020a170903049500b001a19196af48sm6883803plb.64.2023.04.03.10.44.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 03 Apr 2023 10:44:15 -0700 (PDT) From: Tianyu Lan To: luto@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, seanjc@google.com, pbonzini@redhat.com, jgross@suse.com, tiala@microsoft.com, kirill@shutemov.name, jiangshan.ljs@antgroup.com, peterz@infradead.org, ashish.kalra@amd.com, srutherford@google.com, akpm@linux-foundation.org, anshuman.khandual@arm.com, pawan.kumar.gupta@linux.intel.com, adrian.hunter@intel.com, daniel.sneddon@linux.intel.com, alexander.shishkin@linux.intel.com, sandipan.das@amd.com, ray.huang@amd.com, brijesh.singh@amd.com, michael.roth@amd.com, thomas.lendacky@amd.com, venu.busireddy@oracle.com, sterritt@google.com, tony.luck@intel.com, samitolvanen@google.com, fenghua.yu@intel.com Cc: pangupta@amd.com, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-hyperv@vger.kernel.org, linux-arch@vger.kernel.org Subject: [RFC PATCH V4 04/17] x86/hyperv: Use vmmcall to implement Hyper-V hypercall in sev-snp enlightened guest Date: Mon, 3 Apr 2023 13:43:52 -0400 Message-Id: <20230403174406.4180472-5-ltykernel@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230403174406.4180472-1-ltykernel@gmail.com> References: <20230403174406.4180472-1-ltykernel@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Tianyu Lan In sev-snp enlightened guest, Hyper-V hypercall needs to use vmmcall to trigger vmexit and notify hypervisor to handle hypercall request. Signed-off-by: Tianyu Lan --- Change since RFC V2: * Fix indentation style --- arch/x86/include/asm/mshyperv.h | 44 ++++++++++++++++++++++++--------- 1 file changed, 33 insertions(+), 11 deletions(-) diff --git a/arch/x86/include/asm/mshyperv.h b/arch/x86/include/asm/mshyperv.h index 2fa40c6ca99f..3c15e23162e7 100644 --- a/arch/x86/include/asm/mshyperv.h +++ b/arch/x86/include/asm/mshyperv.h @@ -52,16 +52,25 @@ static inline u64 hv_do_hypercall(u64 control, void *input, void *output) u64 hv_status; #ifdef CONFIG_X86_64 - if (!hv_hypercall_pg) - return U64_MAX; + if (hv_isolation_type_en_snp()) { + __asm__ __volatile__("mov %4, %%r8\n" + "vmmcall" + : "=a" (hv_status), ASM_CALL_CONSTRAINT, + "+c" (control), "+d" (input_address) + : "r" (output_address) + : "cc", "memory", "r8", "r9", "r10", "r11"); + } else { + if (!hv_hypercall_pg) + return U64_MAX; - __asm__ __volatile__("mov %4, %%r8\n" - CALL_NOSPEC - : "=a" (hv_status), ASM_CALL_CONSTRAINT, - "+c" (control), "+d" (input_address) - : "r" (output_address), - THUNK_TARGET(hv_hypercall_pg) - : "cc", "memory", "r8", "r9", "r10", "r11"); + __asm__ __volatile__("mov %4, %%r8\n" + CALL_NOSPEC + : "=a" (hv_status), ASM_CALL_CONSTRAINT, + "+c" (control), "+d" (input_address) + : "r" (output_address), + THUNK_TARGET(hv_hypercall_pg) + : "cc", "memory", "r8", "r9", "r10", "r11"); + } #else u32 input_address_hi = upper_32_bits(input_address); u32 input_address_lo = lower_32_bits(input_address); @@ -95,7 +104,13 @@ static inline u64 _hv_do_fast_hypercall8(u64 control, u64 input1) u64 hv_status; #ifdef CONFIG_X86_64 - { + if (hv_isolation_type_en_snp()) { + __asm__ __volatile__( + "vmmcall" + : "=a" (hv_status), ASM_CALL_CONSTRAINT, + "+c" (control), "+d" (input1) + :: "cc", "r8", "r9", "r10", "r11"); + } else { __asm__ __volatile__(CALL_NOSPEC : "=a" (hv_status), ASM_CALL_CONSTRAINT, "+c" (control), "+d" (input1) @@ -140,7 +155,14 @@ static inline u64 _hv_do_fast_hypercall16(u64 control, u64 input1, u64 input2) u64 hv_status; #ifdef CONFIG_X86_64 - { + if (hv_isolation_type_en_snp()) { + __asm__ __volatile__("mov %4, %%r8\n" + "vmmcall" + : "=a" (hv_status), ASM_CALL_CONSTRAINT, + "+c" (control), "+d" (input1) + : "r" (input2) + : "cc", "r8", "r9", "r10", "r11"); + } else { __asm__ __volatile__("mov %4, %%r8\n" CALL_NOSPEC : "=a" (hv_status), ASM_CALL_CONSTRAINT, From patchwork Mon Apr 3 17:43:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tianyu Lan X-Patchwork-Id: 13198518 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5F5DCC761AF for ; Mon, 3 Apr 2023 17:44:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232380AbjDCRo2 (ORCPT ); Mon, 3 Apr 2023 13:44:28 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50728 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232054AbjDCRoS (ORCPT ); Mon, 3 Apr 2023 13:44:18 -0400 Received: from mail-pj1-x102b.google.com (mail-pj1-x102b.google.com [IPv6:2607:f8b0:4864:20::102b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 471212724; Mon, 3 Apr 2023 10:44:17 -0700 (PDT) Received: by mail-pj1-x102b.google.com with SMTP id q102so27956633pjq.3; Mon, 03 Apr 2023 10:44:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1680543857; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=d3G8RfDk2f7ftR+C1v/a0axqypneBk1zP9Qtg1LVHyA=; b=SisYW1Wmi4AST8NFhRKGsWMC2QoBuJyIC6l0wgsxsWGUzBwalQyzNsNoNyrf+WpDQ3 ggZLvMbvLgyuFj+SndN/CoKaBgJPjep6f5wHqIvP7c3K2UmrixL4u85HRFQeAD+M7Uct TcdRxdDlVJ8I1D9DJCnUe1SvQN+eMf9JUthe1j4dXLBNE/KyIixu6DninY+hKb80p/GK HTndq1yh4aVsXmWk1pMhT2Ht2DT45loXXhAM47JXJqy5V/GE8TYcGdg8RwM9Q4wPfmQt djx1k1hPi6YhzK6lCCJfj62KPsp1agf2ZWjRLxFS1cefgv59aGxGN6SRjw9gtw5DXt+b 6dqQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680543857; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=d3G8RfDk2f7ftR+C1v/a0axqypneBk1zP9Qtg1LVHyA=; b=3LUvJQNdaIcNvYAxuJwuy0mD/NUD8Yhi1wL8r91s42il6W+QINTzJc68ByCB7bpjzW psCtiOfDnQp98PN6xH2qETTyhe/nABR0ti+OVmEy9zvJyLJIPEVg4rsp8STeR0HjCe56 HtE6wjG4dfHzlHiLNhs1m8qeRZNCn/S4/KDjKAZ9IpE933kg6c7zmgr+SC7dFVKlyG5Y 8czEyZwwc5mbzoRZL981tQgLU/S3f5m1VP3HgNqV3LCYcexgtTBN6KkWaeWwwPCp+P2z Gz7UzTSkN/FCjV66+n9Zq7w5Nra9b88TaKfp5S/6Jpn7MfCyHjoVFPWhWSaTtndFiSkj Pwig== X-Gm-Message-State: AAQBX9fQauoHqNKcaAkPzcdrjmA89l1o4JCBZuoYek4K0vMWiJYbVSQ7 TGzQ4YVYGOq7nOZfQSsVVL8= X-Google-Smtp-Source: AKy350ZDL0d0QwCZQluEybHBjQnkyvlYmiXMNtLiPXZIizX8eaYVsaZeFAonglLEHSkaIAqZUE5l2A== X-Received: by 2002:a17:902:cecb:b0:1a1:c82d:dcf3 with SMTP id d11-20020a170902cecb00b001a1c82ddcf3mr21052092plg.12.1680543856749; Mon, 03 Apr 2023 10:44:16 -0700 (PDT) Received: from ubuntu-Virtual-Machine.corp.microsoft.com ([2001:4898:80e8:2:8635:6e96:35c1:c560]) by smtp.gmail.com with ESMTPSA id jj21-20020a170903049500b001a19196af48sm6883803plb.64.2023.04.03.10.44.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 03 Apr 2023 10:44:16 -0700 (PDT) From: Tianyu Lan To: luto@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, seanjc@google.com, pbonzini@redhat.com, jgross@suse.com, tiala@microsoft.com, kirill@shutemov.name, jiangshan.ljs@antgroup.com, peterz@infradead.org, ashish.kalra@amd.com, srutherford@google.com, akpm@linux-foundation.org, anshuman.khandual@arm.com, pawan.kumar.gupta@linux.intel.com, adrian.hunter@intel.com, daniel.sneddon@linux.intel.com, alexander.shishkin@linux.intel.com, sandipan.das@amd.com, ray.huang@amd.com, brijesh.singh@amd.com, michael.roth@amd.com, thomas.lendacky@amd.com, venu.busireddy@oracle.com, sterritt@google.com, tony.luck@intel.com, samitolvanen@google.com, fenghua.yu@intel.com Cc: pangupta@amd.com, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-hyperv@vger.kernel.org, linux-arch@vger.kernel.org Subject: [RFC PATCH V4 05/17] clocksource/drivers/hyper-v: decrypt hyperv tsc page in sev-snp enlightened guest Date: Mon, 3 Apr 2023 13:43:53 -0400 Message-Id: <20230403174406.4180472-6-ltykernel@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230403174406.4180472-1-ltykernel@gmail.com> References: <20230403174406.4180472-1-ltykernel@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Tianyu Lan Hyper-V tsc page is shared with hypervisor and it should be decrypted in sev-snp enlightened guest when it's used. Signed-off-by: Tianyu Lan --- Change since RFC V2: * Change the Subject line prefix --- drivers/clocksource/hyperv_timer.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/clocksource/hyperv_timer.c b/drivers/clocksource/hyperv_timer.c index c0cef92b12b8..44da16ca203c 100644 --- a/drivers/clocksource/hyperv_timer.c +++ b/drivers/clocksource/hyperv_timer.c @@ -365,7 +365,7 @@ EXPORT_SYMBOL_GPL(hv_stimer_global_cleanup); static union { struct ms_hyperv_tsc_page page; u8 reserved[PAGE_SIZE]; -} tsc_pg __aligned(PAGE_SIZE); +} tsc_pg __bss_decrypted __aligned(PAGE_SIZE); static struct ms_hyperv_tsc_page *tsc_page = &tsc_pg.page; static unsigned long tsc_pfn; From patchwork Mon Apr 3 17:43:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tianyu Lan X-Patchwork-Id: 13198519 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B3F41C76196 for ; Mon, 3 Apr 2023 17:44:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232116AbjDCRoi (ORCPT ); Mon, 3 Apr 2023 13:44:38 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50804 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232109AbjDCRoT (ORCPT ); Mon, 3 Apr 2023 13:44:19 -0400 Received: from mail-pj1-x1034.google.com (mail-pj1-x1034.google.com [IPv6:2607:f8b0:4864:20::1034]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CA0642D74; Mon, 3 Apr 2023 10:44:18 -0700 (PDT) Received: by mail-pj1-x1034.google.com with SMTP id j13so27980318pjd.1; Mon, 03 Apr 2023 10:44:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1680543858; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=IOp+J/MlPfipkIt8hJ1o3RldVOHyWVonLkOqsh7nnxE=; b=cNHdFhG1ku6DUDGidsm1yJOL/aAIk1oglkPnxeY7og8ZgkiM+b3Pv24z5fzscyI+os qzhTSEuFFXMfpi09zy7sPw8B6SXXsLMWC3+SHcg0WImPp+0Sz3TbPyySkpGliNacOBRX /PL69a5eFxhM+r2UHZGd7le+Mvs76NrUJxg0yXEbRpru7AFS3LV45Bc6S0KHvn2Rmko8 dMHFTU9IDosPlLpsldPWD0M0bHltEFRmpnaGECmcRFlB/nq78Du61dNx0ERORHQKsufp NSbGnU0V3q10kfriRRm8zOL8hjiJMQ3H6pn7c5VdClkRM3CApaxFZLKxOOKovWXwHvr7 +lLQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680543858; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=IOp+J/MlPfipkIt8hJ1o3RldVOHyWVonLkOqsh7nnxE=; b=yYoCWEFGzBP1fsQtlclz0KCkA5OHe0amxTK42FYOWOOg8t+Xqfbf7dZ9Srj6mONd6v YXzV+WPMQE/zmyR6jBo6zPA2Sfwx6PtAGwd1nqbbwP+bUubF88e5FJJbOrCY8J1OsVJy CGewKSFeqgjzrZPxpFuXAyWH1/8qvtrjVsI5BA+3GEsCQPzuo9qid5NcY6vLbtKKwaQF FH7utvCP1GO/xjPD9hPbZQHyTKw0UIG5WtIRmrVM4IDpMKvBjlMaUskIDZcia6LnYbsN Cj5vNDSpdQ00dhjqbNQDRvaxVnK0BXMVHgRsSfYLUKYM9kQAFm5ZH+17Wk35qyWfiYdV eoBA== X-Gm-Message-State: AAQBX9cur3AZJ5SkuTQ9QjR++KaJiiGsxpNKzT1twNljQjmj/XGQJIec C04XcIf5BMp35L0QOuBMNFY= X-Google-Smtp-Source: AKy350bgSpE7FqsEYwJ33BV67GQv+DWoN9DsIYPXLnrVVPKzO31Hv3TYn0cY+TJ77q81lu98kJcYaw== X-Received: by 2002:a17:90b:1e43:b0:23f:8752:98be with SMTP id pi3-20020a17090b1e4300b0023f875298bemr43385277pjb.4.1680543858196; Mon, 03 Apr 2023 10:44:18 -0700 (PDT) Received: from ubuntu-Virtual-Machine.corp.microsoft.com ([2001:4898:80e8:2:8635:6e96:35c1:c560]) by smtp.gmail.com with ESMTPSA id jj21-20020a170903049500b001a19196af48sm6883803plb.64.2023.04.03.10.44.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 03 Apr 2023 10:44:17 -0700 (PDT) From: Tianyu Lan To: luto@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, seanjc@google.com, pbonzini@redhat.com, jgross@suse.com, tiala@microsoft.com, kirill@shutemov.name, jiangshan.ljs@antgroup.com, peterz@infradead.org, ashish.kalra@amd.com, srutherford@google.com, akpm@linux-foundation.org, anshuman.khandual@arm.com, pawan.kumar.gupta@linux.intel.com, adrian.hunter@intel.com, daniel.sneddon@linux.intel.com, alexander.shishkin@linux.intel.com, sandipan.das@amd.com, ray.huang@amd.com, brijesh.singh@amd.com, michael.roth@amd.com, thomas.lendacky@amd.com, venu.busireddy@oracle.com, sterritt@google.com, tony.luck@intel.com, samitolvanen@google.com, fenghua.yu@intel.com Cc: pangupta@amd.com, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-hyperv@vger.kernel.org, linux-arch@vger.kernel.org Subject: [RFC PATCH V4 06/17] x86/hyperv: decrypt VMBus pages for sev-snp enlightened guest Date: Mon, 3 Apr 2023 13:43:54 -0400 Message-Id: <20230403174406.4180472-7-ltykernel@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230403174406.4180472-1-ltykernel@gmail.com> References: <20230403174406.4180472-1-ltykernel@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Tianyu Lan VMBus post msg, synic event and message pages are shared with hypervisor and so decrypt these pages in the sev-snp guest. Signed-off-by: Tianyu Lan --- Change since RFC V3: * Set encrypt page back in the hv_synic_free() Change since RFC V2: * Fix error in the error code path and encrypt pages correctly when decryption failure happens. --- drivers/hv/hv.c | 42 +++++++++++++++++++++++++++++++++++++++++- 1 file changed, 41 insertions(+), 1 deletion(-) diff --git a/drivers/hv/hv.c b/drivers/hv/hv.c index 008234894d28..e09cea8f2f04 100644 --- a/drivers/hv/hv.c +++ b/drivers/hv/hv.c @@ -20,6 +20,7 @@ #include #include #include +#include #include "hyperv_vmbus.h" /* The one and only */ @@ -117,7 +118,7 @@ int hv_post_message(union hv_connection_id connection_id, int hv_synic_alloc(void) { - int cpu; + int cpu, ret; struct hv_per_cpu_context *hv_cpu; /* @@ -168,9 +169,39 @@ int hv_synic_alloc(void) pr_err("Unable to allocate post msg page\n"); goto err; } + + if (hv_isolation_type_en_snp()) { + ret = set_memory_decrypted((unsigned long) + hv_cpu->synic_message_page, 1); + if (ret) + goto err; + + ret = set_memory_decrypted((unsigned long) + hv_cpu->synic_event_page, 1); + if (ret) + goto err_decrypt_event_page; + + ret = set_memory_decrypted((unsigned long) + hv_cpu->post_msg_page, 1); + if (ret) + goto err_decrypt_msg_page; + + memset(hv_cpu->synic_message_page, 0, PAGE_SIZE); + memset(hv_cpu->synic_event_page, 0, PAGE_SIZE); + memset(hv_cpu->post_msg_page, 0, PAGE_SIZE); + } } return 0; + +err_decrypt_msg_page: + set_memory_encrypted((unsigned long) + hv_cpu->synic_event_page, 1); + +err_decrypt_event_page: + set_memory_encrypted((unsigned long) + hv_cpu->synic_message_page, 1); + err: /* * Any memory allocations that succeeded will be freed when @@ -191,6 +222,15 @@ void hv_synic_free(void) free_page((unsigned long)hv_cpu->synic_event_page); free_page((unsigned long)hv_cpu->synic_message_page); free_page((unsigned long)hv_cpu->post_msg_page); + + if (hv_isolation_type_en_snp()) { + set_memory_encrypted((unsigned long) + hv_cpu->synic_message_page, 1); + set_memory_encrypted((unsigned long) + hv_cpu->synic_event_page, 1); + set_memory_encrypted((unsigned long) + hv_cpu->post_msg_page, 1); + } } kfree(hv_context.hv_numa_map); From patchwork Mon Apr 3 17:43:55 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tianyu Lan X-Patchwork-Id: 13198520 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C5BA7C76196 for ; Mon, 3 Apr 2023 17:44:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232135AbjDCRoy (ORCPT ); Mon, 3 Apr 2023 13:44:54 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50964 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232145AbjDCRoW (ORCPT ); Mon, 3 Apr 2023 13:44:22 -0400 Received: from mail-pl1-x632.google.com (mail-pl1-x632.google.com [IPv6:2607:f8b0:4864:20::632]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 135402688; Mon, 3 Apr 2023 10:44:20 -0700 (PDT) Received: by mail-pl1-x632.google.com with SMTP id w4so28787938plg.9; Mon, 03 Apr 2023 10:44:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1680543859; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=fRBMiXzZNygTcZn2nWWcUVPl/gZC7IQbi/zy9Q0QWG8=; b=Vn/Lo0p4Nv0x5bLf7vQUWrfkkMLVlX6jfveMHNj0Uti93dq8W3ItJV4FlSUdH0o1Dl jdHp4yXrT8HX4TG/Kw7MWRZfxnlwTRtrIhT88kRY2QvyW9Na9P6mKsue5w+IAgPMgLmw RfYMEQ3Pja6wz/35rnzn8aXIRPy2U2nJeRezAplaGXNEXCVpH8rXlCR3iBNmYIwH2t0w wrUZ5PpPFQrINNYhD5pSUvQZkoLrx75iWUCmVxyUc4p3+/LvLDPKmoz98zX0efRybiED PsLn6IufclLcNHByQWLcGCDsjaGyiMUG0aZ73r8Neop2kM6RCTuT5V5CfpiiR4cbcg6Q NyCg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680543859; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=fRBMiXzZNygTcZn2nWWcUVPl/gZC7IQbi/zy9Q0QWG8=; b=0zaSfYqIiTK5Pv8YDCxycIYISWzkb4xZ3IzccFrihl5G43b8pdYXwLBlydwrFIs0xl Rq6IFqLx67EWSPRh0+J1grVfU6nX4lNPxWxTV/yvOt9mFWO2J1TTxROe/eDlcgwFmMwk UDxDCqQZ+L0pmsENsi/eFUEr8t9LfWU3HfOrBrnl5+JdENoJ3FlEQW8lC8DNT97UIRKv eEXqZ1irq20gd+9hLfF8D/j3pRjZAnNZ7MWVL9ZlIhsF8Fzr1D7z4nO/c1QOZYA/4Jor ygmQAxPdQLhjmqhTn+ev54Dci6zcPsQDwwZAGzrWUcNY/ucGY8Ohl1zIWUIvuy3d761x 61Lg== X-Gm-Message-State: AAQBX9fa9iJzO9Bsdi6STon9/MawenfK/o4y9kgpM3M8w+QDVpoav8ab uiUSZgwj8vbgqYk2bM5b/fw= X-Google-Smtp-Source: AKy350bgaRlI+pSd0yR6OkLeUp0nfBqWDPsMadzvxbDT8HCL58L5FJrQtgajuf32tRHeSjojFkjJfA== X-Received: by 2002:a17:902:f687:b0:19c:be0c:738 with SMTP id l7-20020a170902f68700b0019cbe0c0738mr48077342plg.59.1680543859450; Mon, 03 Apr 2023 10:44:19 -0700 (PDT) Received: from ubuntu-Virtual-Machine.corp.microsoft.com ([2001:4898:80e8:2:8635:6e96:35c1:c560]) by smtp.gmail.com with ESMTPSA id jj21-20020a170903049500b001a19196af48sm6883803plb.64.2023.04.03.10.44.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 03 Apr 2023 10:44:19 -0700 (PDT) From: Tianyu Lan To: luto@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, seanjc@google.com, pbonzini@redhat.com, jgross@suse.com, tiala@microsoft.com, kirill@shutemov.name, jiangshan.ljs@antgroup.com, peterz@infradead.org, ashish.kalra@amd.com, srutherford@google.com, akpm@linux-foundation.org, anshuman.khandual@arm.com, pawan.kumar.gupta@linux.intel.com, adrian.hunter@intel.com, daniel.sneddon@linux.intel.com, alexander.shishkin@linux.intel.com, sandipan.das@amd.com, ray.huang@amd.com, brijesh.singh@amd.com, michael.roth@amd.com, thomas.lendacky@amd.com, venu.busireddy@oracle.com, sterritt@google.com, tony.luck@intel.com, samitolvanen@google.com, fenghua.yu@intel.com Cc: pangupta@amd.com, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-hyperv@vger.kernel.org, linux-arch@vger.kernel.org Subject: [RFC PATCH V4 07/17] drivers: hv: Decrypt percpu hvcall input arg page in sev-snp enlightened guest Date: Mon, 3 Apr 2023 13:43:55 -0400 Message-Id: <20230403174406.4180472-8-ltykernel@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230403174406.4180472-1-ltykernel@gmail.com> References: <20230403174406.4180472-1-ltykernel@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Tianyu Lan Hypervisor needs to access iput arg page and guest should decrypt the page. Signed-off-by: Tianyu Lan --- Change since RFC V3: * Use pgcount to decrypt memory. Change since RFC V2: * Set inputarg to be zero after kfree() * Not free mem when fail to encrypt mem in the hv_common_cpu_die(). --- drivers/hv/hv_common.c | 21 ++++++++++++++++++++- 1 file changed, 20 insertions(+), 1 deletion(-) diff --git a/drivers/hv/hv_common.c b/drivers/hv/hv_common.c index e3b04e978932..455432d49fd3 100644 --- a/drivers/hv/hv_common.c +++ b/drivers/hv/hv_common.c @@ -21,6 +21,7 @@ #include #include #include +#include #include #include @@ -128,6 +129,7 @@ int hv_common_cpu_init(unsigned int cpu) u64 msr_vp_index; gfp_t flags; int pgcount = hv_root_partition ? 2 : 1; + int ret; /* hv_cpu_init() can be called with IRQs disabled from hv_resume() */ flags = irqs_disabled() ? GFP_ATOMIC : GFP_KERNEL; @@ -137,6 +139,17 @@ int hv_common_cpu_init(unsigned int cpu) if (!(*inputarg)) return -ENOMEM; + if (hv_isolation_type_en_snp()) { + ret = set_memory_decrypted((unsigned long)*inputarg, pgcount); + if (ret) { + kfree(*inputarg); + *inputarg = NULL; + return ret; + } + + memset(*inputarg, 0x00, PAGE_SIZE); + } + if (hv_root_partition) { outputarg = (void **)this_cpu_ptr(hyperv_pcpu_output_arg); *outputarg = (char *)(*inputarg) + HV_HYP_PAGE_SIZE; @@ -156,6 +169,7 @@ int hv_common_cpu_die(unsigned int cpu) { unsigned long flags; void **inputarg, **outputarg; + int pgcount = hv_root_partition ? 2 : 1; void *mem; local_irq_save(flags); @@ -171,7 +185,12 @@ int hv_common_cpu_die(unsigned int cpu) local_irq_restore(flags); - kfree(mem); + if (hv_isolation_type_en_snp()) { + if (!set_memory_encrypted((unsigned long)mem, pgcount)) + kfree(mem); + } else { + kfree(mem); + } return 0; } From patchwork Mon Apr 3 17:43:56 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tianyu Lan X-Patchwork-Id: 13198523 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 31ED3C76188 for ; Mon, 3 Apr 2023 17:45:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232493AbjDCRpB (ORCPT ); Mon, 3 Apr 2023 13:45:01 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50304 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232276AbjDCRoY (ORCPT ); Mon, 3 Apr 2023 13:44:24 -0400 Received: from mail-pj1-x102d.google.com (mail-pj1-x102d.google.com [IPv6:2607:f8b0:4864:20::102d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F1E572D79; Mon, 3 Apr 2023 10:44:21 -0700 (PDT) Received: by mail-pj1-x102d.google.com with SMTP id lr16-20020a17090b4b9000b0023f187954acso31375857pjb.2; Mon, 03 Apr 2023 10:44:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1680543861; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=0EILUdjprvW3YCHuPVyOQT4JHVYk96Hi+rg/ChRnI6Q=; b=jPWllsA8sC/dmhbOuAo2Qx+3epE0R3wQgHMAcuJnzsLaz/a11VO0LTijIT+QuuDB9Z yG4qRWkEFTp/wR2EZevUqd1Kgrr7xy6KdAgr06eqR4Jdvdagb6g1RVoAXCj1olIlgf/k zt+EKUlufTHJ5U5NnUhrtI/7nX/vehXZTDywjDkP3kX5LdsxTMSLjOvRdxLHmG8ol6Uk 1LKdEqyWzbzneDb094Mv/hvP32pymymL1nq6ICP2Qls/W3y/ZTvfb3pyXwt5Mi4zwczt vGTD9+7kntgIpUAfkYpUNVFEZ7dAu7+chyuW+ZIyANejzXY3ENb9Y/PBha4zQqAl7MSu ISKQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680543861; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=0EILUdjprvW3YCHuPVyOQT4JHVYk96Hi+rg/ChRnI6Q=; b=ed8b3LudEnDktTTJDzVKailTt+XjiMzNsi7lNZdyL3iC4OYBSy4zTV+dzbGr+lBJGj Vz4FcwtXm2wAU3pmztaPYlfAtLHetw6uA69ShzrLD6ZzbwEiozurv1mTmbeJX1PmOv8/ 1DXeZOVzgjE4o/s4E7F/D7KCiQVkFbQz2lBNxi4fEv98ca7PIDlUCdPGkApBizMD6bGf /1oYO9okwulknjE5PqhT7YQyoQiYB3j7worYrjiZCTazw6QepAdBvOXNjQwyz6Hn0c5q SU5McAfg3PVPxxZaqf6ifMqr+KiMPxOdodhbhZqxGX+C1scivlZfdpsSydlCgYmZ8gxY ca/Q== X-Gm-Message-State: AAQBX9eDLgmJ6BwmNvo591PgPBqt8xYD6L7Te1cc2sqVaE2qgfsCqJnP UGUNpAU5TEKx5uYH1UAEvsU= X-Google-Smtp-Source: AKy350Z0owT7mIkpcsyOL9tzWiFhFZY6B1QNzoj6nkv9SjCb9v7wJAwc1HXbKbL/RUnzRDBDTqIQoQ== X-Received: by 2002:a17:903:32cd:b0:1a2:9ce6:648b with SMTP id i13-20020a17090332cd00b001a29ce6648bmr17755487plr.12.1680543860731; Mon, 03 Apr 2023 10:44:20 -0700 (PDT) Received: from ubuntu-Virtual-Machine.corp.microsoft.com ([2001:4898:80e8:2:8635:6e96:35c1:c560]) by smtp.gmail.com with ESMTPSA id jj21-20020a170903049500b001a19196af48sm6883803plb.64.2023.04.03.10.44.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 03 Apr 2023 10:44:20 -0700 (PDT) From: Tianyu Lan To: luto@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, seanjc@google.com, pbonzini@redhat.com, jgross@suse.com, tiala@microsoft.com, kirill@shutemov.name, jiangshan.ljs@antgroup.com, peterz@infradead.org, ashish.kalra@amd.com, srutherford@google.com, akpm@linux-foundation.org, anshuman.khandual@arm.com, pawan.kumar.gupta@linux.intel.com, adrian.hunter@intel.com, daniel.sneddon@linux.intel.com, alexander.shishkin@linux.intel.com, sandipan.das@amd.com, ray.huang@amd.com, brijesh.singh@amd.com, michael.roth@amd.com, thomas.lendacky@amd.com, venu.busireddy@oracle.com, sterritt@google.com, tony.luck@intel.com, samitolvanen@google.com, fenghua.yu@intel.com Cc: pangupta@amd.com, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-hyperv@vger.kernel.org, linux-arch@vger.kernel.org Subject: [RFC PATCH V4 08/17] x86/hyperv: Initialize cpu and memory for sev-snp enlightened guest Date: Mon, 3 Apr 2023 13:43:56 -0400 Message-Id: <20230403174406.4180472-9-ltykernel@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230403174406.4180472-1-ltykernel@gmail.com> References: <20230403174406.4180472-1-ltykernel@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Tianyu Lan Read processor amd memory info from specific address which are populated by Hyper-V. Initialize smp cpu related ops, pvalidate system memory and add it into e820 table. Signed-off-by: Tianyu Lan --- arch/x86/hyperv/ivm.c | 78 +++++++++++++++++++++++++++++++++ arch/x86/include/asm/mshyperv.h | 16 +++++++ arch/x86/kernel/cpu/mshyperv.c | 3 ++ 3 files changed, 97 insertions(+) diff --git a/arch/x86/hyperv/ivm.c b/arch/x86/hyperv/ivm.c index 368b2731950e..fa4de2761460 100644 --- a/arch/x86/hyperv/ivm.c +++ b/arch/x86/hyperv/ivm.c @@ -17,6 +17,11 @@ #include #include #include +#include +#include +#include +#include +#include #ifdef CONFIG_AMD_MEM_ENCRYPT @@ -57,6 +62,22 @@ union hv_ghcb { static u16 hv_ghcb_version __ro_after_init; +static u32 processor_count; + +static __init void hv_snp_get_smp_config(unsigned int early) +{ + if (!early) { + while (num_processors < processor_count) { + early_per_cpu(x86_cpu_to_apicid, num_processors) = num_processors; + early_per_cpu(x86_bios_cpu_apicid, num_processors) = num_processors; + physid_set(num_processors, phys_cpu_present_map); + set_cpu_possible(num_processors, true); + set_cpu_present(num_processors, true); + num_processors++; + } + } +} + u64 hv_ghcb_hypercall(u64 control, void *input, void *output, u32 input_size) { union hv_ghcb *hv_ghcb; @@ -356,6 +377,63 @@ static bool hv_is_private_mmio(u64 addr) return false; } +__init void hv_sev_init_mem_and_cpu(void) +{ + struct memory_map_entry *entry; + struct e820_entry *e820_entry; + u64 e820_end; + u64 ram_end; + u64 page; + + /* + * Hyper-V enlightened snp guest boots kernel + * directly without bootloader and so roms, + * bios regions and reserve resources are not + * available. Set these callback to NULL. + */ + x86_platform.legacy.reserve_bios_regions = 0; + x86_init.resources.probe_roms = x86_init_noop; + x86_init.resources.reserve_resources = x86_init_noop; + x86_init.mpparse.find_smp_config = x86_init_noop; + x86_init.mpparse.get_smp_config = hv_snp_get_smp_config; + + /* + * Hyper-V SEV-SNP enlightened guest doesn't support ioapic + * and legacy APIC page read/write. Switch to hv apic here. + */ + disable_ioapic_support(); + + /* Read processor number and memory layout. */ + processor_count = *(u32 *)__va(EN_SEV_SNP_PROCESSOR_INFO_ADDR); + entry = (struct memory_map_entry *)(__va(EN_SEV_SNP_PROCESSOR_INFO_ADDR) + + sizeof(struct memory_map_entry)); + + /* + * E820 table in the memory just describes memory for + * kernel, ACPI table, cmdline, boot params and ramdisk. + * Hyper-V popoulates the rest memory layout in the EN_SEV_ + * SNP_PROCESSOR_INFO_ADDR. + */ + for (; entry->numpages != 0; entry++) { + e820_entry = &e820_table->entries[ + e820_table->nr_entries - 1]; + e820_end = e820_entry->addr + e820_entry->size; + ram_end = (entry->starting_gpn + + entry->numpages) * PAGE_SIZE; + + if (e820_end < entry->starting_gpn * PAGE_SIZE) + e820_end = entry->starting_gpn * PAGE_SIZE; + + if (e820_end < ram_end) { + pr_info("Hyper-V: add e820 entry [mem %#018Lx-%#018Lx]\n", e820_end, ram_end - 1); + e820__range_add(e820_end, ram_end - e820_end, + E820_TYPE_RAM); + for (page = e820_end; page < ram_end; page += PAGE_SIZE) + pvalidate((unsigned long)__va(page), RMP_PG_SIZE_4K, true); + } + } +} + void __init hv_vtom_init(void) { /* diff --git a/arch/x86/include/asm/mshyperv.h b/arch/x86/include/asm/mshyperv.h index 3c15e23162e7..a4a59007b5f2 100644 --- a/arch/x86/include/asm/mshyperv.h +++ b/arch/x86/include/asm/mshyperv.h @@ -41,6 +41,20 @@ extern bool hv_isolation_type_en_snp(void); extern union hv_ghcb * __percpu *hv_ghcb_pg; +/* + * Hyper-V puts processor and memory layout info + * to this address in SEV-SNP enlightened guest. + */ +#define EN_SEV_SNP_PROCESSOR_INFO_ADDR 0x802000 + +struct memory_map_entry { + u64 starting_gpn; + u64 numpages; + u16 type; + u16 flags; + u32 reserved; +}; + int hv_call_deposit_pages(int node, u64 partition_id, u32 num_pages); int hv_call_add_logical_proc(int node, u32 lp_index, u32 acpi_id); int hv_call_create_vp(int node, u64 partition_id, u32 vp_index, u32 flags); @@ -246,12 +260,14 @@ void hv_ghcb_msr_read(u64 msr, u64 *value); bool hv_ghcb_negotiate_protocol(void); void hv_ghcb_terminate(unsigned int set, unsigned int reason); void hv_vtom_init(void); +void hv_sev_init_mem_and_cpu(void); #else static inline void hv_ghcb_msr_write(u64 msr, u64 value) {} static inline void hv_ghcb_msr_read(u64 msr, u64 *value) {} static inline bool hv_ghcb_negotiate_protocol(void) { return false; } static inline void hv_ghcb_terminate(unsigned int set, unsigned int reason) {} static inline void hv_vtom_init(void) {} +static inline void hv_sev_init_mem_and_cpu(void) {} #endif extern bool hv_isolation_type_snp(void); diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c index 2f2dcb2370b6..71820bbf9e90 100644 --- a/arch/x86/kernel/cpu/mshyperv.c +++ b/arch/x86/kernel/cpu/mshyperv.c @@ -529,6 +529,9 @@ static void __init ms_hyperv_init_platform(void) if (!(ms_hyperv.features & HV_ACCESS_TSC_INVARIANT)) mark_tsc_unstable("running on Hyper-V"); + if (hv_isolation_type_en_snp()) + hv_sev_init_mem_and_cpu(); + hardlockup_detector_disable(); } From patchwork Mon Apr 3 17:43:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tianyu Lan X-Patchwork-Id: 13198522 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2EA68C761A6 for ; Mon, 3 Apr 2023 17:45:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232147AbjDCRo7 (ORCPT ); Mon, 3 Apr 2023 13:44:59 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50760 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232321AbjDCRoY (ORCPT ); Mon, 3 Apr 2023 13:44:24 -0400 Received: from mail-pj1-x102f.google.com (mail-pj1-x102f.google.com [IPv6:2607:f8b0:4864:20::102f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9E4BB1BE4; Mon, 3 Apr 2023 10:44:22 -0700 (PDT) Received: by mail-pj1-x102f.google.com with SMTP id om3-20020a17090b3a8300b0023efab0e3bfso33392482pjb.3; Mon, 03 Apr 2023 10:44:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1680543862; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=IsgF+fleeMgXLp1N/FMCJ16iEQR8VAMoZKtj7f5mA3o=; b=ATr4I3dpzC3Rv85kph1+DyvP/Xloo8gEXAVXj2pB2xgpmn8x4gqj9HLVDwaALBpBy6 PBbosgvXZaNsXzZjYK+ihXBaamk6u5FJCBENQPf9smRkAtpvOqupBkdPXvvvjPN8kjsl ubhXjgZiwOime42tJtk1gcB8B5A9q1yfLWn8rOSAdnr6WaKoXpcye22w/5zOj5Uzk/y7 KwewgboeilHtseZtdn6OmJvyUoelZvEClIAxiSmsbof8W4xO4cDz2GHf7R9AN55mM+qk GPBRi7Ahh7Lh5H4VxWPSvm6HQSBFGjjNpekjWyKGFCwv1NA68kBuXvQ0a6+0lXq9KA4w F2KA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680543862; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=IsgF+fleeMgXLp1N/FMCJ16iEQR8VAMoZKtj7f5mA3o=; b=5Sy0grfkS5Y0S/jUCofZXoRvmrDtns2KDgA9v2TU0GbMMl7nu/lLMBWFu5h5Szz9/D Pb0ZtEG6CmgiER90uP78CRD/JEB9u3Pw38tVhQywl2Ogz9ryMDFHGwWKNTiv9qUUHlPh jJ7XHLbyQvBqVIb//rLZRRN0nsTCxR16BQlikbAqRoDN7JlOSh+ECJJ0Gdp8mvQvYFDe obNwr6eMXiy03x2bkh+S74KGWE1EIPtdP1eZfEYdsWcAfTVYzMYFHZrLtnblzjDHYfLs RbJU8aeIj+M3W6tG4iZ77XCgRWi9T4rIIv1JfSoreh/lZF8XEUlHQ/FOy89CH8+0YLad 6+2Q== X-Gm-Message-State: AAQBX9cuQCcst5bMggA6JgDZjOKwFaWO7H2VxlO6Iq0Yv8AkPl5LO8WR eaJmnTajCnw7M29M2EXMNHw= X-Google-Smtp-Source: AKy350aQjZRe/EhCQZ0EpcgcQEibFK4Vfiii/+iB44zCAGt9hyxo/gC92OGQRZWtdLbas9BUxvMl2A== X-Received: by 2002:a17:902:e882:b0:1a1:f413:70b1 with SMTP id w2-20020a170902e88200b001a1f41370b1mr45509775plg.18.1680543861931; Mon, 03 Apr 2023 10:44:21 -0700 (PDT) Received: from ubuntu-Virtual-Machine.corp.microsoft.com ([2001:4898:80e8:2:8635:6e96:35c1:c560]) by smtp.gmail.com with ESMTPSA id jj21-20020a170903049500b001a19196af48sm6883803plb.64.2023.04.03.10.44.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 03 Apr 2023 10:44:21 -0700 (PDT) From: Tianyu Lan To: luto@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, seanjc@google.com, pbonzini@redhat.com, jgross@suse.com, tiala@microsoft.com, kirill@shutemov.name, jiangshan.ljs@antgroup.com, peterz@infradead.org, ashish.kalra@amd.com, srutherford@google.com, akpm@linux-foundation.org, anshuman.khandual@arm.com, pawan.kumar.gupta@linux.intel.com, adrian.hunter@intel.com, daniel.sneddon@linux.intel.com, alexander.shishkin@linux.intel.com, sandipan.das@amd.com, ray.huang@amd.com, brijesh.singh@amd.com, michael.roth@amd.com, thomas.lendacky@amd.com, venu.busireddy@oracle.com, sterritt@google.com, tony.luck@intel.com, samitolvanen@google.com, fenghua.yu@intel.com Cc: pangupta@amd.com, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-hyperv@vger.kernel.org, linux-arch@vger.kernel.org Subject: [RFC PATCH V4 09/17] x86/hyperv: SEV-SNP enlightened guest don't support legacy rtc Date: Mon, 3 Apr 2023 13:43:57 -0400 Message-Id: <20230403174406.4180472-10-ltykernel@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230403174406.4180472-1-ltykernel@gmail.com> References: <20230403174406.4180472-1-ltykernel@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Tianyu Lan SEV-SNP enlightened guest doesn't support legacy rtc. Set legacy.rtc, x86_platform.set_wallclock and get_wallclock to 0 or noop(). Make get/set_rtc_noop() to be public and reuse them in the ms_hyperv_init_platform(). Signed-off-by: Tianyu Lan --- arch/x86/hyperv/ivm.c | 3 +++ arch/x86/include/asm/x86_init.h | 2 ++ arch/x86/kernel/x86_init.c | 4 ++-- 3 files changed, 7 insertions(+), 2 deletions(-) diff --git a/arch/x86/hyperv/ivm.c b/arch/x86/hyperv/ivm.c index fa4de2761460..c0f3fa924163 100644 --- a/arch/x86/hyperv/ivm.c +++ b/arch/x86/hyperv/ivm.c @@ -391,6 +391,9 @@ __init void hv_sev_init_mem_and_cpu(void) * bios regions and reserve resources are not * available. Set these callback to NULL. */ + x86_platform.legacy.rtc = 0; + x86_platform.set_wallclock = set_rtc_noop; + x86_platform.get_wallclock = get_rtc_noop; x86_platform.legacy.reserve_bios_regions = 0; x86_init.resources.probe_roms = x86_init_noop; x86_init.resources.reserve_resources = x86_init_noop; diff --git a/arch/x86/include/asm/x86_init.h b/arch/x86/include/asm/x86_init.h index 6f873c686cd3..e5a6f9baec67 100644 --- a/arch/x86/include/asm/x86_init.h +++ b/arch/x86/include/asm/x86_init.h @@ -330,5 +330,7 @@ extern void x86_init_uint_noop(unsigned int unused); extern bool bool_x86_init_noop(void); extern void x86_op_int_noop(int cpu); extern bool x86_pnpbios_disabled(void); +extern int set_rtc_noop(const struct timespec64 *now); +extern void get_rtc_noop(struct timespec64 *now); #endif diff --git a/arch/x86/kernel/x86_init.c b/arch/x86/kernel/x86_init.c index 95be3831df73..d82f4fa2f1bf 100644 --- a/arch/x86/kernel/x86_init.c +++ b/arch/x86/kernel/x86_init.c @@ -33,8 +33,8 @@ static int __init iommu_init_noop(void) { return 0; } static void iommu_shutdown_noop(void) { } bool __init bool_x86_init_noop(void) { return false; } void x86_op_int_noop(int cpu) { } -static __init int set_rtc_noop(const struct timespec64 *now) { return -EINVAL; } -static __init void get_rtc_noop(struct timespec64 *now) { } +int set_rtc_noop(const struct timespec64 *now) { return -EINVAL; } +void get_rtc_noop(struct timespec64 *now) { } static __initconst const struct of_device_id of_cmos_match[] = { { .compatible = "motorola,mc146818" }, From patchwork Mon Apr 3 17:43:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tianyu Lan X-Patchwork-Id: 13198524 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AD18FC76188 for ; Mon, 3 Apr 2023 17:45:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232520AbjDCRpD (ORCPT ); Mon, 3 Apr 2023 13:45:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51918 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232439AbjDCRoh (ORCPT ); Mon, 3 Apr 2023 13:44:37 -0400 Received: from mail-pj1-x1033.google.com (mail-pj1-x1033.google.com [IPv6:2607:f8b0:4864:20::1033]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 34E212D4B; Mon, 3 Apr 2023 10:44:24 -0700 (PDT) Received: by mail-pj1-x1033.google.com with SMTP id r7-20020a17090b050700b002404be7920aso29377036pjz.5; Mon, 03 Apr 2023 10:44:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1680543863; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Efu5rhOggzlI0kcT150AQduXmzuRafJrSYFqNtQftrI=; b=gQ/WrzwVffRoPeh+XgYJeBIyAd1XhgHgkvI44mtrOyV0nJmBs+PM7ErifjlKkjnli7 /fcZFSJW4X9o6A1t4+VrpuoXRelg2CdaS8i/q0yx6uYYbb8DjSAtIF7I+aAyalemtaF5 rI/CepYF3bjYAmrkvm263Ldxh+yLisZ8JffYyl5XKHigosYttCmR45CjCHxyh1vwLDxY s022Cu2AvpJzj2IU3ABd42dFGLc6t+zKcIAx75oM1NDoKjzIUhzDzsAoclu4+Bq3uoRa RoYF19AxDbfeGy4U47l22rLdupg+wC8ZchXmQjM0EG7Ynf2spNmuHamGRVF6KX8L+Mxg iWMw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680543863; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Efu5rhOggzlI0kcT150AQduXmzuRafJrSYFqNtQftrI=; b=hglE5vco06cEAmlIgu56cyi1Iktpw+oM7+3vYrGdmS6sJ1/fTHdWgtNa57Xj9ywUcU PPamSolIyXiZJP5m5qAbNtZTVWSzIqBvsTsQBWPv1lISsM79g1MiMZ1Kl+pJ2Imxd2D4 MNGQSLeT+UPHcr3jv8keW76e1vyp3UVfX/ju9usruCIR3d1ilmNp+7O+4DRO3M7RR99l yGxOMDe+k64Bf8O2zv10pyW0jm48oqrWphBzMo1ZGylNDHAaecOwAln4C7qzCKxHOVVq eDkTY0DQA9vi714ZOvq3FG2TPaBDuteNz4e9yzyjnC/1+efYTD2/Uer+gB+WusYetuy0 7qwA== X-Gm-Message-State: AAQBX9f21HBltvrUxusQpqr92ygQ8HiEpESsMeYq+6Jst7JYHR1CRQGl lM7vJ+2UwYHWHNjfbKaiqp0= X-Google-Smtp-Source: AKy350aFdkCUQuqf3H8aJn3BQxPFh1mNDKkOo4VE2zYt41+OzuCzgh401vVBvzTJWng/+jZlsVNOzQ== X-Received: by 2002:a17:902:ec8b:b0:1a2:87a2:c91a with SMTP id x11-20020a170902ec8b00b001a287a2c91amr21118850plg.34.1680543863287; Mon, 03 Apr 2023 10:44:23 -0700 (PDT) Received: from ubuntu-Virtual-Machine.corp.microsoft.com ([2001:4898:80e8:2:8635:6e96:35c1:c560]) by smtp.gmail.com with ESMTPSA id jj21-20020a170903049500b001a19196af48sm6883803plb.64.2023.04.03.10.44.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 03 Apr 2023 10:44:22 -0700 (PDT) From: Tianyu Lan To: luto@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, seanjc@google.com, pbonzini@redhat.com, jgross@suse.com, tiala@microsoft.com, kirill@shutemov.name, jiangshan.ljs@antgroup.com, peterz@infradead.org, ashish.kalra@amd.com, srutherford@google.com, akpm@linux-foundation.org, anshuman.khandual@arm.com, pawan.kumar.gupta@linux.intel.com, adrian.hunter@intel.com, daniel.sneddon@linux.intel.com, alexander.shishkin@linux.intel.com, sandipan.das@amd.com, ray.huang@amd.com, brijesh.singh@amd.com, michael.roth@amd.com, thomas.lendacky@amd.com, venu.busireddy@oracle.com, sterritt@google.com, tony.luck@intel.com, samitolvanen@google.com, fenghua.yu@intel.com Cc: pangupta@amd.com, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-hyperv@vger.kernel.org, linux-arch@vger.kernel.org Subject: [RFC PATCH V4 10/17] x86/hyperv: Add smp support for sev-snp guest Date: Mon, 3 Apr 2023 13:43:58 -0400 Message-Id: <20230403174406.4180472-11-ltykernel@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230403174406.4180472-1-ltykernel@gmail.com> References: <20230403174406.4180472-1-ltykernel@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Tianyu Lan The wakeup_secondary_cpu callback was populated with wakeup_ cpu_via_vmgexit() which doesn't work for Hyper-V and Hyper-V requires to call Hyper-V specific hvcall to start APs. So override it with Hyper-V specific hook to start AP sev_es_save_area data structure. Signed-off-by: Tianyu Lan --- Change sicne RFC v3: * Replace struct sev_es_save_area with struct vmcb_save_area * Move code from mshyperv.c to ivm.c Change since RFC v2: * Add helper function to initialize segment * Fix some coding style --- arch/x86/hyperv/ivm.c | 89 +++++++++++++++++++++++++++++++ arch/x86/include/asm/mshyperv.h | 18 +++++++ arch/x86/include/asm/sev.h | 13 +++++ arch/x86/include/asm/svm.h | 15 +++++- arch/x86/kernel/cpu/mshyperv.c | 13 ++++- arch/x86/kernel/sev.c | 4 +- include/asm-generic/hyperv-tlfs.h | 19 +++++++ 7 files changed, 166 insertions(+), 5 deletions(-) diff --git a/arch/x86/hyperv/ivm.c b/arch/x86/hyperv/ivm.c index c0f3fa924163..51243148b8e6 100644 --- a/arch/x86/hyperv/ivm.c +++ b/arch/x86/hyperv/ivm.c @@ -22,11 +22,15 @@ #include #include #include +#include #ifdef CONFIG_AMD_MEM_ENCRYPT #define GHCB_USAGE_HYPERV_CALL 1 +static u8 ap_start_input_arg[PAGE_SIZE] __bss_decrypted __aligned(PAGE_SIZE); +static u8 ap_start_stack[PAGE_SIZE] __aligned(PAGE_SIZE); + union hv_ghcb { struct ghcb ghcb; struct { @@ -437,6 +441,91 @@ __init void hv_sev_init_mem_and_cpu(void) } } +#define hv_populate_vmcb_seg(seg, gdtr_base) \ +do { \ + if (seg.selector) { \ + seg.base = 0; \ + seg.limit = HV_AP_SEGMENT_LIMIT; \ + seg.attrib = *(u16 *)(gdtr_base + seg.selector + 5); \ + seg.attrib = (seg.attrib & 0xFF) | ((seg.attrib >> 4) & 0xF00); \ + } \ +} while (0) \ + +int hv_snp_boot_ap(int cpu, unsigned long start_ip) +{ + struct sev_es_save_area *vmsa = (struct sev_es_save_area *) + __get_free_page(GFP_KERNEL | __GFP_ZERO); + struct desc_ptr gdtr; + u64 ret, retry = 5; + struct hv_start_virtual_processor_input *start_vp_input; + union sev_rmp_adjust rmp_adjust; + unsigned long flags; + + native_store_gdt(&gdtr); + + vmsa->gdtr.base = gdtr.address; + vmsa->gdtr.limit = gdtr.size; + + asm volatile("movl %%es, %%eax;" : "=a" (vmsa->es.selector)); + hv_populate_vmcb_seg(vmsa->es, vmsa->gdtr.base); + + asm volatile("movl %%cs, %%eax;" : "=a" (vmsa->cs.selector)); + hv_populate_vmcb_seg(vmsa->cs, vmsa->gdtr.base); + + asm volatile("movl %%ss, %%eax;" : "=a" (vmsa->ss.selector)); + hv_populate_vmcb_seg(vmsa->ss, vmsa->gdtr.base); + + asm volatile("movl %%ds, %%eax;" : "=a" (vmsa->ds.selector)); + hv_populate_vmcb_seg(vmsa->ds, vmsa->gdtr.base); + + vmsa->efer = native_read_msr(MSR_EFER); + + asm volatile("movq %%cr4, %%rax;" : "=a" (vmsa->cr4)); + asm volatile("movq %%cr3, %%rax;" : "=a" (vmsa->cr3)); + asm volatile("movq %%cr0, %%rax;" : "=a" (vmsa->cr0)); + + vmsa->xcr0 = 1; + vmsa->g_pat = HV_AP_INIT_GPAT_DEFAULT; + vmsa->rip = (u64)secondary_startup_64_no_verify; + vmsa->rsp = (u64)&ap_start_stack[PAGE_SIZE]; + + vmsa->sev_features.snp = 1; + vmsa->sev_features.restrict_injection = 1; + + rmp_adjust.as_uint64 = 0; + rmp_adjust.target_vmpl = 1; + rmp_adjust.vmsa = 1; + ret = rmpadjust((unsigned long)vmsa, RMP_PG_SIZE_4K, + rmp_adjust.as_uint64); + if (ret != 0) { + pr_err("RMPADJUST(%llx) failed: %llx\n", (u64)vmsa, ret); + return ret; + } + + local_irq_save(flags); + start_vp_input = + (struct hv_start_virtual_processor_input *)ap_start_input_arg; + memset(start_vp_input, 0, sizeof(*start_vp_input)); + start_vp_input->partitionid = -1; + start_vp_input->vpindex = cpu; + start_vp_input->targetvtl = ms_hyperv.vtl; + *(u64 *)&start_vp_input->context[0] = __pa(vmsa) | 1; + + do { + ret = hv_do_hypercall(HVCALL_START_VIRTUAL_PROCESSOR, + start_vp_input, NULL); + } while (hv_result(ret) == HV_STATUS_TIME_OUT && retry--); + + if (!hv_result_success(ret)) { + pr_err("HvCallStartVirtualProcessor failed: %llx\n", ret); + goto done; + } + +done: + local_irq_restore(flags); + return ret; +} + void __init hv_vtom_init(void) { /* diff --git a/arch/x86/include/asm/mshyperv.h b/arch/x86/include/asm/mshyperv.h index a4a59007b5f2..e23c987deb7a 100644 --- a/arch/x86/include/asm/mshyperv.h +++ b/arch/x86/include/asm/mshyperv.h @@ -55,6 +55,20 @@ struct memory_map_entry { u32 reserved; }; +/* + * DEFAULT INIT GPAT and SEGMENT LIMIT value in struct VMSA + * to start AP in enlightened SEV guest. + */ +#define HV_AP_INIT_GPAT_DEFAULT 0x0007040600070406ULL +#define HV_AP_SEGMENT_LIMIT 0xffffffff + +/* + * DEFAULT INIT GPAT and SEGMENT LIMIT value in struct VMSA + * to start AP in enlightened SEV guest. + */ +#define HV_AP_INIT_GPAT_DEFAULT 0x0007040600070406ULL +#define HV_AP_SEGMENT_LIMIT 0xffffffff + int hv_call_deposit_pages(int node, u64 partition_id, u32 num_pages); int hv_call_add_logical_proc(int node, u32 lp_index, u32 acpi_id); int hv_call_create_vp(int node, u64 partition_id, u32 vp_index, u32 flags); @@ -253,6 +267,8 @@ struct irq_domain *hv_create_pci_msi_domain(void); int hv_map_ioapic_interrupt(int ioapic_id, bool level, int vcpu, int vector, struct hv_interrupt_entry *entry); int hv_unmap_ioapic_interrupt(int ioapic_id, struct hv_interrupt_entry *entry); +int hv_set_mem_host_visibility(unsigned long addr, int numpages, bool visible); +int hv_snp_boot_ap(int cpu, unsigned long start_ip); #ifdef CONFIG_AMD_MEM_ENCRYPT void hv_ghcb_msr_write(u64 msr, u64 value); @@ -261,6 +277,7 @@ bool hv_ghcb_negotiate_protocol(void); void hv_ghcb_terminate(unsigned int set, unsigned int reason); void hv_vtom_init(void); void hv_sev_init_mem_and_cpu(void); +int hv_snp_boot_ap(int cpu, unsigned long start_ip); #else static inline void hv_ghcb_msr_write(u64 msr, u64 value) {} static inline void hv_ghcb_msr_read(u64 msr, u64 *value) {} @@ -268,6 +285,7 @@ static inline bool hv_ghcb_negotiate_protocol(void) { return false; } static inline void hv_ghcb_terminate(unsigned int set, unsigned int reason) {} static inline void hv_vtom_init(void) {} static inline void hv_sev_init_mem_and_cpu(void) {} +static int hv_snp_boot_ap(int cpu, unsigned long start_ip) {} #endif extern bool hv_isolation_type_snp(void); diff --git a/arch/x86/include/asm/sev.h b/arch/x86/include/asm/sev.h index ebc271bb6d8e..e34aaf730220 100644 --- a/arch/x86/include/asm/sev.h +++ b/arch/x86/include/asm/sev.h @@ -86,6 +86,19 @@ extern bool handle_vc_boot_ghcb(struct pt_regs *regs); #define RMPADJUST_VMSA_PAGE_BIT BIT(16) +union sev_rmp_adjust { + u64 as_uint64; + struct { + unsigned long target_vmpl : 8; + unsigned long enable_read : 1; + unsigned long enable_write : 1; + unsigned long enable_user_execute : 1; + unsigned long enable_kernel_execute : 1; + unsigned long reserved1 : 4; + unsigned long vmsa : 1; + }; +}; + /* SNP Guest message request */ struct snp_req_data { unsigned long req_gpa; diff --git a/arch/x86/include/asm/svm.h b/arch/x86/include/asm/svm.h index cb1ee53ad3b1..bcf970901512 100644 --- a/arch/x86/include/asm/svm.h +++ b/arch/x86/include/asm/svm.h @@ -423,7 +423,20 @@ struct sev_es_save_area { u64 guest_exit_info_2; u64 guest_exit_int_info; u64 guest_nrip; - u64 sev_features; + union { + struct { + u64 snp : 1; + u64 vtom : 1; + u64 reflectvc : 1; + u64 restrict_injection : 1; + u64 alternate_injection : 1; + u64 full_debug : 1; + u64 reserved1 : 1; + u64 snpbtb_isolation : 1; + u64 resrved2 : 56; + }; + u64 val; + } sev_features; u64 vintr_ctrl; u64 guest_exit_code; u64 virtual_tom; diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c index 71820bbf9e90..829234ba8da5 100644 --- a/arch/x86/kernel/cpu/mshyperv.c +++ b/arch/x86/kernel/cpu/mshyperv.c @@ -300,6 +300,16 @@ static void __init hv_smp_prepare_cpus(unsigned int max_cpus) native_smp_prepare_cpus(max_cpus); + /* + * Override wakeup_secondary_cpu callback for SEV-SNP + * enlightened guest. + */ + if (hv_isolation_type_en_snp()) + apic->wakeup_secondary_cpu = hv_snp_boot_ap; + + if (!hv_root_partition) + return; + #ifdef CONFIG_X86_64 for_each_present_cpu(i) { if (i == 0) @@ -503,8 +513,7 @@ static void __init ms_hyperv_init_platform(void) # ifdef CONFIG_SMP smp_ops.smp_prepare_boot_cpu = hv_smp_prepare_boot_cpu; - if (hv_root_partition) - smp_ops.smp_prepare_cpus = hv_smp_prepare_cpus; + smp_ops.smp_prepare_cpus = hv_smp_prepare_cpus; # endif /* diff --git a/arch/x86/kernel/sev.c b/arch/x86/kernel/sev.c index 679026a640ef..b3f95fcb8b18 100644 --- a/arch/x86/kernel/sev.c +++ b/arch/x86/kernel/sev.c @@ -1080,7 +1080,7 @@ static int wakeup_cpu_via_vmgexit(int apic_id, unsigned long start_ip) * SEV_FEATURES (matches the SEV STATUS MSR right shifted 2 bits) */ vmsa->vmpl = 0; - vmsa->sev_features = sev_status >> 2; + vmsa->sev_features.val = sev_status >> 2; /* Switch the page over to a VMSA page now that it is initialized */ ret = snp_set_vmsa(vmsa, true); @@ -1097,7 +1097,7 @@ static int wakeup_cpu_via_vmgexit(int apic_id, unsigned long start_ip) ghcb = __sev_get_ghcb(&state); vc_ghcb_invalidate(ghcb); - ghcb_set_rax(ghcb, vmsa->sev_features); + ghcb_set_rax(ghcb, vmsa->sev_features.val); ghcb_set_sw_exit_code(ghcb, SVM_VMGEXIT_AP_CREATION); ghcb_set_sw_exit_info_1(ghcb, ((u64)apic_id << 32) | SVM_VMGEXIT_AP_CREATE); ghcb_set_sw_exit_info_2(ghcb, __pa(vmsa)); diff --git a/include/asm-generic/hyperv-tlfs.h b/include/asm-generic/hyperv-tlfs.h index ea406e901469..b2f14aa608f7 100644 --- a/include/asm-generic/hyperv-tlfs.h +++ b/include/asm-generic/hyperv-tlfs.h @@ -148,6 +148,7 @@ union hv_reference_tsc_msr { #define HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST 0x0003 #define HVCALL_NOTIFY_LONG_SPIN_WAIT 0x0008 #define HVCALL_SEND_IPI 0x000b +#define HVCALL_ENABLE_VP_VTL 0x000f #define HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE_EX 0x0013 #define HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST_EX 0x0014 #define HVCALL_SEND_IPI_EX 0x0015 @@ -165,6 +166,7 @@ union hv_reference_tsc_msr { #define HVCALL_MAP_DEVICE_INTERRUPT 0x007c #define HVCALL_UNMAP_DEVICE_INTERRUPT 0x007d #define HVCALL_RETARGET_INTERRUPT 0x007e +#define HVCALL_START_VIRTUAL_PROCESSOR 0x0099 #define HVCALL_FLUSH_GUEST_PHYSICAL_ADDRESS_SPACE 0x00af #define HVCALL_FLUSH_GUEST_PHYSICAL_ADDRESS_LIST 0x00b0 #define HVCALL_MODIFY_SPARSE_GPA_PAGE_HOST_VISIBILITY 0x00db @@ -220,6 +222,7 @@ enum HV_GENERIC_SET_FORMAT { #define HV_STATUS_INVALID_PORT_ID 17 #define HV_STATUS_INVALID_CONNECTION_ID 18 #define HV_STATUS_INSUFFICIENT_BUFFERS 19 +#define HV_STATUS_TIME_OUT 0x78 /* * The Hyper-V TimeRefCount register and the TSC @@ -779,6 +782,22 @@ struct hv_input_unmap_device_interrupt { struct hv_interrupt_entry interrupt_entry; } __packed; +struct hv_enable_vp_vtl_input { + u64 partitionid; + u32 vpindex; + u8 targetvtl; + u8 padding[3]; + u8 context[0xe0]; +} __packed; + +struct hv_start_virtual_processor_input { + u64 partitionid; + u32 vpindex; + u8 targetvtl; + u8 padding[3]; + u8 context[0xe0]; +} __packed; + #define HV_SOURCE_SHADOW_NONE 0x0 #define HV_SOURCE_SHADOW_BRIDGE_BUS_RANGE 0x1 From patchwork Mon Apr 3 17:43:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tianyu Lan X-Patchwork-Id: 13198521 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DCFABC77B70 for ; Mon, 3 Apr 2023 17:44:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232463AbjDCRo4 (ORCPT ); Mon, 3 Apr 2023 13:44:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52222 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232476AbjDCRop (ORCPT ); Mon, 3 Apr 2023 13:44:45 -0400 Received: from mail-pl1-x62c.google.com (mail-pl1-x62c.google.com [IPv6:2607:f8b0:4864:20::62c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4C0183588; Mon, 3 Apr 2023 10:44:25 -0700 (PDT) Received: by mail-pl1-x62c.google.com with SMTP id kc4so28791105plb.10; Mon, 03 Apr 2023 10:44:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1680543865; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=wR0AzrPzYG8OJqqCg/MeK2RAGKVdybTSl0W3QOrUJm4=; b=SY9MAvUMv0Z6B9kvhH9fJTPTFmqJueX5AK4+E8LO50Nza65kNzX6ERGapqjeiZxU6z 2531ngDn6jSFzmuGYp4z2ECysW9mulBaLiDMP7qwOc50HTQ/n7gar9Nwpo75Lpa3Q7DY OaM+PEYGs20eN5RaNUIR3lmvVUJFl+b/3i8CddQqB4mP/6+orFhbDViWs+3lC0KZTM3G v3nF6MCFW2Fa1vD9TYhNEqKRurm9sV+fW6Fuwb8L+2NCVA1DYw8v6qI4JE/zUmHphrqE P7Nx18D/4bKXevy8r6xXO8qOLCOy2jNkueGLcVrl9CXs++nGPg/Flej+NY3mEFjfP3lF YUTQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680543865; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=wR0AzrPzYG8OJqqCg/MeK2RAGKVdybTSl0W3QOrUJm4=; b=i8aVRY9ryP2W2Oj+sBmQiTyNqzXoljr6BywlP+IvsL9bIcqN97F5BWOPqPPrI4WwyT /7fZTfWEQgrGz/wl4S91AFgwRlLMKVzwQ0BE1owUKKHxUIIOmDe38wL6eKIdPEQMp31v 1lAuPPHxie+ikHNFZQWa1m08tCcBUJJR67KyPXwFf15zRUKA66Ocuc/Wp2iOSR/rY/XJ htsiFbo//JgWBjKhKRpFlvJgia650AG5VhdDZjPTTw+0F0zpvBS8o6q+ieLzZnXGshFL leAoSDYtCtwstDA6D6m2puE7IS5utM3gxNinz7TuTX/SR2CnOMt09BxngRgIyAayh1Ne FcXg== X-Gm-Message-State: AAQBX9c5Et99zlc5l96P/0707f4hPy0h5VuQ4c/pqqQp92WSvYb0lfeY Kn5LgkWkTNmtWWTwx49Wk9s= X-Google-Smtp-Source: AKy350Ymu8g42Sls7u3/3eR7Wme8+uJHZttkV5AzLFy780UTDLk8y3o19x52ab6AosSxwAna7OMxQg== X-Received: by 2002:a17:902:f283:b0:1a0:7065:e000 with SMTP id k3-20020a170902f28300b001a07065e000mr29673761plc.14.1680543864770; Mon, 03 Apr 2023 10:44:24 -0700 (PDT) Received: from ubuntu-Virtual-Machine.corp.microsoft.com ([2001:4898:80e8:2:8635:6e96:35c1:c560]) by smtp.gmail.com with ESMTPSA id jj21-20020a170903049500b001a19196af48sm6883803plb.64.2023.04.03.10.44.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 03 Apr 2023 10:44:24 -0700 (PDT) From: Tianyu Lan To: luto@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, seanjc@google.com, pbonzini@redhat.com, jgross@suse.com, tiala@microsoft.com, kirill@shutemov.name, jiangshan.ljs@antgroup.com, peterz@infradead.org, ashish.kalra@amd.com, srutherford@google.com, akpm@linux-foundation.org, anshuman.khandual@arm.com, pawan.kumar.gupta@linux.intel.com, adrian.hunter@intel.com, daniel.sneddon@linux.intel.com, alexander.shishkin@linux.intel.com, sandipan.das@amd.com, ray.huang@amd.com, brijesh.singh@amd.com, michael.roth@amd.com, thomas.lendacky@amd.com, venu.busireddy@oracle.com, sterritt@google.com, tony.luck@intel.com, samitolvanen@google.com, fenghua.yu@intel.com Cc: pangupta@amd.com, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-hyperv@vger.kernel.org, linux-arch@vger.kernel.org Subject: [RFC PATCH V4 11/17] x86/hyperv: Add hyperv-specific handling for VMMCALL under SEV-ES Date: Mon, 3 Apr 2023 13:43:59 -0400 Message-Id: <20230403174406.4180472-12-ltykernel@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230403174406.4180472-1-ltykernel@gmail.com> References: <20230403174406.4180472-1-ltykernel@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Tianyu Lan Add Hyperv-specific handling for faults caused by VMMCALL instructions. Signed-off-by: Tianyu Lan --- arch/x86/kernel/cpu/mshyperv.c | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c index 829234ba8da5..69ca4b8f615a 100644 --- a/arch/x86/kernel/cpu/mshyperv.c +++ b/arch/x86/kernel/cpu/mshyperv.c @@ -32,6 +32,7 @@ #include #include #include +#include /* Is Linux running as the root partition? */ bool hv_root_partition; @@ -577,6 +578,20 @@ static bool __init ms_hyperv_msi_ext_dest_id(void) return eax & HYPERV_VS_PROPERTIES_EAX_EXTENDED_IOAPIC_RTE; } +static void hv_sev_es_hcall_prepare(struct ghcb *ghcb, struct pt_regs *regs) +{ + /* RAX and CPL are already in the GHCB */ + ghcb_set_rcx(ghcb, regs->cx); + ghcb_set_rdx(ghcb, regs->dx); + ghcb_set_r8(ghcb, regs->r8); +} + +static bool hv_sev_es_hcall_finish(struct ghcb *ghcb, struct pt_regs *regs) +{ + /* No checking of the return state needed */ + return true; +} + const __initconst struct hypervisor_x86 x86_hyper_ms_hyperv = { .name = "Microsoft Hyper-V", .detect = ms_hyperv_platform, @@ -584,4 +599,6 @@ const __initconst struct hypervisor_x86 x86_hyper_ms_hyperv = { .init.x2apic_available = ms_hyperv_x2apic_available, .init.msi_ext_dest_id = ms_hyperv_msi_ext_dest_id, .init.init_platform = ms_hyperv_init_platform, + .runtime.sev_es_hcall_prepare = hv_sev_es_hcall_prepare, + .runtime.sev_es_hcall_finish = hv_sev_es_hcall_finish, }; From patchwork Mon Apr 3 17:44:00 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tianyu Lan X-Patchwork-Id: 13198525 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CA92AC761A6 for ; Mon, 3 Apr 2023 17:45:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232755AbjDCRpl (ORCPT ); Mon, 3 Apr 2023 13:45:41 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50726 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232555AbjDCRos (ORCPT ); Mon, 3 Apr 2023 13:44:48 -0400 Received: from mail-pj1-x102b.google.com (mail-pj1-x102b.google.com [IPv6:2607:f8b0:4864:20::102b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 855A630FE; Mon, 3 Apr 2023 10:44:26 -0700 (PDT) Received: by mail-pj1-x102b.google.com with SMTP id q102so27957004pjq.3; Mon, 03 Apr 2023 10:44:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1680543866; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=npHkfV75SrGd+ID1/np7JLThVcJL7MPD/f19211QO/Y=; b=dbLkI+rW0HjOiPmwXbvaXA0ZCIGY7L7Pb3l+7QobXmSh8nogPpnHvhrpIGZdt7sZ0A rAxev5UhR7rsQDsx7mXLa7q638AMZ1JVIDv62K1xQD80jTi2aAA8zCntdOI34cmI/EqA KB0uotrSIYnuBSzCU1hDkSOvH9cPHctYJrGm8NN8n1MgrGU7KO/CCgQgEtX0vfmOqTVy FUrIxisY6S5/yOH75ZY9ef5QIcnwhW4Jhzi9CsnZfocGm7eaiYl9/Pd13eV8aRTSRvsl J/Sa2eI9t3aE8a8jlbnybUR0OwA/kg6vLU2Ql8e6HuHU/YImHL9lNWo2+vLA6tTFToCq 0A0A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680543866; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=npHkfV75SrGd+ID1/np7JLThVcJL7MPD/f19211QO/Y=; b=y2t/mr8YIJgAle5ODMagZqsWIwlnRvCo7lMzTMTdabKY6uGv1WbUuHSttGoJyba/kO hVf/H9eHEoS4r3mufKo5YZ3VUNS2XS7kyfCbsUg4QvuUZY45DQ4hCHyjT5hZ4mdtvZY3 Tru19Q1FUm9Q8tjS7e+OGHpgsaKyS7zBp7jUzxmhDK4Rz/ZeawEfrPBtxHPxBXqK6NRt ybVH24m4GZhsknTTpm/AwdnyK/quPOO2GhuLTxQVln4xSKh+8ejAECdo3HI46QD7By9w bbc5SDuojUp1a9fVvRtsZ8iCAql7mZNGzd7zi/0Dl8MkSevaOGn3KKCdwCSMDucvQWl6 3rhQ== X-Gm-Message-State: AAQBX9dtHFPYp5J6q3pg9481hWBKSENib94sXsz3Wr/T3/SSl/3+CpP/ GVak+uDrx4obCFFwep6ImA0= X-Google-Smtp-Source: AKy350Y9duylew26ufz4xRmLJaoEPWrGcjw6DJvDoldNNH0kne+yGLmBb/uNqdNPJmd+OqcxlYfmhA== X-Received: by 2002:a17:902:ea10:b0:19a:a520:b203 with SMTP id s16-20020a170902ea1000b0019aa520b203mr45989716plg.25.1680543865951; Mon, 03 Apr 2023 10:44:25 -0700 (PDT) Received: from ubuntu-Virtual-Machine.corp.microsoft.com ([2001:4898:80e8:2:8635:6e96:35c1:c560]) by smtp.gmail.com with ESMTPSA id jj21-20020a170903049500b001a19196af48sm6883803plb.64.2023.04.03.10.44.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 03 Apr 2023 10:44:25 -0700 (PDT) From: Tianyu Lan To: luto@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, seanjc@google.com, pbonzini@redhat.com, jgross@suse.com, tiala@microsoft.com, kirill@shutemov.name, jiangshan.ljs@antgroup.com, peterz@infradead.org, ashish.kalra@amd.com, srutherford@google.com, akpm@linux-foundation.org, anshuman.khandual@arm.com, pawan.kumar.gupta@linux.intel.com, adrian.hunter@intel.com, daniel.sneddon@linux.intel.com, alexander.shishkin@linux.intel.com, sandipan.das@amd.com, ray.huang@amd.com, brijesh.singh@amd.com, michael.roth@amd.com, thomas.lendacky@amd.com, venu.busireddy@oracle.com, sterritt@google.com, tony.luck@intel.com, samitolvanen@google.com, fenghua.yu@intel.com Cc: pangupta@amd.com, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-hyperv@vger.kernel.org, linux-arch@vger.kernel.org Subject: [RFC PATCH V4 12/17] x86/sev: Add a #HV exception handler Date: Mon, 3 Apr 2023 13:44:00 -0400 Message-Id: <20230403174406.4180472-13-ltykernel@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230403174406.4180472-1-ltykernel@gmail.com> References: <20230403174406.4180472-1-ltykernel@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Tianyu Lan Add a #HV exception handler that uses IST stack. Signed-off-by: Tianyu Lan --- Change since RFC V2: * Remove unnecessary line in the change log. --- arch/x86/entry/entry_64.S | 60 +++++++++++++++++++++++++++ arch/x86/include/asm/cpu_entry_area.h | 6 +++ arch/x86/include/asm/idtentry.h | 39 ++++++++++++++++- arch/x86/include/asm/page_64_types.h | 1 + arch/x86/include/asm/trapnr.h | 1 + arch/x86/include/asm/traps.h | 1 + arch/x86/kernel/cpu/common.c | 1 + arch/x86/kernel/dumpstack_64.c | 9 +++- arch/x86/kernel/idt.c | 1 + arch/x86/kernel/sev.c | 53 +++++++++++++++++++++++ arch/x86/kernel/traps.c | 40 ++++++++++++++++++ arch/x86/mm/cpu_entry_area.c | 2 + 12 files changed, 211 insertions(+), 3 deletions(-) diff --git a/arch/x86/entry/entry_64.S b/arch/x86/entry/entry_64.S index eccc3431e515..d877774c3141 100644 --- a/arch/x86/entry/entry_64.S +++ b/arch/x86/entry/entry_64.S @@ -570,6 +570,66 @@ SYM_CODE_START(\asmsym) .Lfrom_usermode_switch_stack_\@: idtentry_body user_\cfunc, has_error_code=1 +_ASM_NOKPROBE(\asmsym) +SYM_CODE_END(\asmsym) +.endm +/* + * idtentry_hv - Macro to generate entry stub for #HV + * @vector: Vector number + * @asmsym: ASM symbol for the entry point + * @cfunc: C function to be called + * + * The macro emits code to set up the kernel context for #HV. The #HV handler + * runs on an IST stack and needs to be able to support nested #HV exceptions. + * + * To make this work the #HV entry code tries its best to pretend it doesn't use + * an IST stack by switching to the task stack if coming from user-space (which + * includes early SYSCALL entry path) or back to the stack in the IRET frame if + * entered from kernel-mode. + * + * If entered from kernel-mode the return stack is validated first, and if it is + * not safe to use (e.g. because it points to the entry stack) the #HV handler + * will switch to a fall-back stack (HV2) and call a special handler function. + * + * The macro is only used for one vector, but it is planned to be extended in + * the future for the #HV exception. + */ +.macro idtentry_hv vector asmsym cfunc +SYM_CODE_START(\asmsym) + UNWIND_HINT_IRET_REGS + ENDBR + ASM_CLAC + pushq $-1 /* ORIG_RAX: no syscall to restart */ + + testb $3, CS-ORIG_RAX(%rsp) + jnz .Lfrom_usermode_switch_stack_\@ + + call paranoid_entry + + UNWIND_HINT_REGS + + /* + * Switch off the IST stack to make it free for nested exceptions. + */ + movq %rsp, %rdi /* pt_regs pointer */ + call hv_switch_off_ist + movq %rax, %rsp /* Switch to new stack */ + + ENCODE_FRAME_POINTER + UNWIND_HINT_REGS + + /* Update pt_regs */ + movq ORIG_RAX(%rsp), %rsi /* get error code into 2nd argument*/ + movq $-1, ORIG_RAX(%rsp) /* no syscall to restart */ + + movq %rsp, %rdi /* pt_regs pointer */ + call kernel_\cfunc + + jmp paranoid_exit + +.Lfrom_usermode_switch_stack_\@: + idtentry_body user_\cfunc, has_error_code=1 + _ASM_NOKPROBE(\asmsym) SYM_CODE_END(\asmsym) .endm diff --git a/arch/x86/include/asm/cpu_entry_area.h b/arch/x86/include/asm/cpu_entry_area.h index 462fc34f1317..2186ed601b4a 100644 --- a/arch/x86/include/asm/cpu_entry_area.h +++ b/arch/x86/include/asm/cpu_entry_area.h @@ -30,6 +30,10 @@ char VC_stack[optional_stack_size]; \ char VC2_stack_guard[guardsize]; \ char VC2_stack[optional_stack_size]; \ + char HV_stack_guard[guardsize]; \ + char HV_stack[optional_stack_size]; \ + char HV2_stack_guard[guardsize]; \ + char HV2_stack[optional_stack_size]; \ char IST_top_guard[guardsize]; \ /* The exception stacks' physical storage. No guard pages required */ @@ -52,6 +56,8 @@ enum exception_stack_ordering { ESTACK_MCE, ESTACK_VC, ESTACK_VC2, + ESTACK_HV, + ESTACK_HV2, N_EXCEPTION_STACKS }; diff --git a/arch/x86/include/asm/idtentry.h b/arch/x86/include/asm/idtentry.h index b241af4ce9b4..22ac10a7b34f 100644 --- a/arch/x86/include/asm/idtentry.h +++ b/arch/x86/include/asm/idtentry.h @@ -317,6 +317,19 @@ static __always_inline void __##func(struct pt_regs *regs) __visible noinstr void kernel_##func(struct pt_regs *regs, unsigned long error_code); \ __visible noinstr void user_##func(struct pt_regs *regs, unsigned long error_code) + +/** + * DECLARE_IDTENTRY_HV - Declare functions for the HV entry point + * @vector: Vector number (ignored for C) + * @func: Function name of the entry point + * + * Maps to DECLARE_IDTENTRY_RAW, but declares also the user C handler. + */ +#define DECLARE_IDTENTRY_HV(vector, func) \ + DECLARE_IDTENTRY_RAW_ERRORCODE(vector, func); \ + __visible noinstr void kernel_##func(struct pt_regs *regs); \ + __visible noinstr void user_##func(struct pt_regs *regs) + /** * DEFINE_IDTENTRY_IST - Emit code for IST entry points * @func: Function name of the entry point @@ -376,6 +389,26 @@ static __always_inline void __##func(struct pt_regs *regs) #define DEFINE_IDTENTRY_VC_USER(func) \ DEFINE_IDTENTRY_RAW_ERRORCODE(user_##func) +/** + * DEFINE_IDTENTRY_HV_KERNEL - Emit code for HV injection handler + * when raised from kernel mode + * @func: Function name of the entry point + * + * Maps to DEFINE_IDTENTRY_RAW + */ +#define DEFINE_IDTENTRY_HV_KERNEL(func) \ + DEFINE_IDTENTRY_RAW(kernel_##func) + +/** + * DEFINE_IDTENTRY_HV_USER - Emit code for HV injection handler + * when raised from user mode + * @func: Function name of the entry point + * + * Maps to DEFINE_IDTENTRY_RAW + */ +#define DEFINE_IDTENTRY_HV_USER(func) \ + DEFINE_IDTENTRY_RAW(user_##func) + #else /* CONFIG_X86_64 */ /** @@ -465,6 +498,9 @@ __visible noinstr void func(struct pt_regs *regs, \ # define DECLARE_IDTENTRY_VC(vector, func) \ idtentry_vc vector asm_##func func +# define DECLARE_IDTENTRY_HV(vector, func) \ + idtentry_hv vector asm_##func func + #else # define DECLARE_IDTENTRY_MCE(vector, func) \ DECLARE_IDTENTRY(vector, func) @@ -618,9 +654,10 @@ DECLARE_IDTENTRY_RAW_ERRORCODE(X86_TRAP_DF, xenpv_exc_double_fault); DECLARE_IDTENTRY_ERRORCODE(X86_TRAP_CP, exc_control_protection); #endif -/* #VC */ +/* #VC & #HV */ #ifdef CONFIG_AMD_MEM_ENCRYPT DECLARE_IDTENTRY_VC(X86_TRAP_VC, exc_vmm_communication); +DECLARE_IDTENTRY_HV(X86_TRAP_HV, exc_hv_injection); #endif #ifdef CONFIG_XEN_PV diff --git a/arch/x86/include/asm/page_64_types.h b/arch/x86/include/asm/page_64_types.h index e9e2c3ba5923..0bd7dab676c5 100644 --- a/arch/x86/include/asm/page_64_types.h +++ b/arch/x86/include/asm/page_64_types.h @@ -29,6 +29,7 @@ #define IST_INDEX_DB 2 #define IST_INDEX_MCE 3 #define IST_INDEX_VC 4 +#define IST_INDEX_HV 5 /* * Set __PAGE_OFFSET to the most negative possible address + diff --git a/arch/x86/include/asm/trapnr.h b/arch/x86/include/asm/trapnr.h index f5d2325aa0b7..c6583631cecb 100644 --- a/arch/x86/include/asm/trapnr.h +++ b/arch/x86/include/asm/trapnr.h @@ -26,6 +26,7 @@ #define X86_TRAP_XF 19 /* SIMD Floating-Point Exception */ #define X86_TRAP_VE 20 /* Virtualization Exception */ #define X86_TRAP_CP 21 /* Control Protection Exception */ +#define X86_TRAP_HV 28 /* HV injected exception in SNP restricted mode */ #define X86_TRAP_VC 29 /* VMM Communication Exception */ #define X86_TRAP_IRET 32 /* IRET Exception */ diff --git a/arch/x86/include/asm/traps.h b/arch/x86/include/asm/traps.h index 47ecfff2c83d..6795d3e517d6 100644 --- a/arch/x86/include/asm/traps.h +++ b/arch/x86/include/asm/traps.h @@ -16,6 +16,7 @@ asmlinkage __visible notrace struct pt_regs *fixup_bad_iret(struct pt_regs *bad_regs); void __init trap_init(void); asmlinkage __visible noinstr struct pt_regs *vc_switch_off_ist(struct pt_regs *eregs); +asmlinkage __visible noinstr struct pt_regs *hv_switch_off_ist(struct pt_regs *eregs); #endif extern bool ibt_selftest(void); diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c index 8cd4126d8253..5bc44bcf6e48 100644 --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -2172,6 +2172,7 @@ static inline void tss_setup_ist(struct tss_struct *tss) tss->x86_tss.ist[IST_INDEX_MCE] = __this_cpu_ist_top_va(MCE); /* Only mapped when SEV-ES is active */ tss->x86_tss.ist[IST_INDEX_VC] = __this_cpu_ist_top_va(VC); + tss->x86_tss.ist[IST_INDEX_HV] = __this_cpu_ist_top_va(HV); } #else /* CONFIG_X86_64 */ diff --git a/arch/x86/kernel/dumpstack_64.c b/arch/x86/kernel/dumpstack_64.c index f05339fee778..6d8f8864810c 100644 --- a/arch/x86/kernel/dumpstack_64.c +++ b/arch/x86/kernel/dumpstack_64.c @@ -26,11 +26,14 @@ static const char * const exception_stack_names[] = { [ ESTACK_MCE ] = "#MC", [ ESTACK_VC ] = "#VC", [ ESTACK_VC2 ] = "#VC2", + [ ESTACK_HV ] = "#HV", + [ ESTACK_HV2 ] = "#HV2", + }; const char *stack_type_name(enum stack_type type) { - BUILD_BUG_ON(N_EXCEPTION_STACKS != 6); + BUILD_BUG_ON(N_EXCEPTION_STACKS != 8); if (type == STACK_TYPE_TASK) return "TASK"; @@ -89,6 +92,8 @@ struct estack_pages estack_pages[CEA_ESTACK_PAGES] ____cacheline_aligned = { EPAGERANGE(MCE), EPAGERANGE(VC), EPAGERANGE(VC2), + EPAGERANGE(HV), + EPAGERANGE(HV2), }; static __always_inline bool in_exception_stack(unsigned long *stack, struct stack_info *info) @@ -98,7 +103,7 @@ static __always_inline bool in_exception_stack(unsigned long *stack, struct stac struct pt_regs *regs; unsigned int k; - BUILD_BUG_ON(N_EXCEPTION_STACKS != 6); + BUILD_BUG_ON(N_EXCEPTION_STACKS != 8); begin = (unsigned long)__this_cpu_read(cea_exception_stacks); /* diff --git a/arch/x86/kernel/idt.c b/arch/x86/kernel/idt.c index a58c6bc1cd68..48c0a7e1dbcb 100644 --- a/arch/x86/kernel/idt.c +++ b/arch/x86/kernel/idt.c @@ -113,6 +113,7 @@ static const __initconst struct idt_data def_idts[] = { #ifdef CONFIG_AMD_MEM_ENCRYPT ISTG(X86_TRAP_VC, asm_exc_vmm_communication, IST_INDEX_VC), + ISTG(X86_TRAP_HV, asm_exc_hv_injection, IST_INDEX_HV), #endif SYSG(X86_TRAP_OF, asm_exc_overflow), diff --git a/arch/x86/kernel/sev.c b/arch/x86/kernel/sev.c index b3f95fcb8b18..2684a45b50a6 100644 --- a/arch/x86/kernel/sev.c +++ b/arch/x86/kernel/sev.c @@ -2004,6 +2004,59 @@ DEFINE_IDTENTRY_VC_USER(exc_vmm_communication) irqentry_exit_to_user_mode(regs); } +static bool hv_raw_handle_exception(struct pt_regs *regs) +{ + return false; +} + +static __always_inline bool on_hv_fallback_stack(struct pt_regs *regs) +{ + unsigned long sp = (unsigned long)regs; + + return (sp >= __this_cpu_ist_bottom_va(HV2) && sp < __this_cpu_ist_top_va(HV2)); +} + +DEFINE_IDTENTRY_HV_USER(exc_hv_injection) +{ + irqentry_enter_from_user_mode(regs); + instrumentation_begin(); + + if (!hv_raw_handle_exception(regs)) { + /* + * Do not kill the machine if user-space triggered the + * exception. Send SIGBUS instead and let user-space deal + * with it. + */ + force_sig_fault(SIGBUS, BUS_OBJERR, (void __user *)0); + } + + instrumentation_end(); + irqentry_exit_to_user_mode(regs); +} + +DEFINE_IDTENTRY_HV_KERNEL(exc_hv_injection) +{ + irqentry_state_t irq_state; + + irq_state = irqentry_nmi_enter(regs); + instrumentation_begin(); + + if (!hv_raw_handle_exception(regs)) { + pr_emerg("PANIC: Unhandled #HV exception in kernel space\n"); + + /* Show some debug info */ + show_regs(regs); + + /* Ask hypervisor to sev_es_terminate */ + sev_es_terminate(SEV_TERM_SET_GEN, GHCB_SEV_ES_GEN_REQ); + + panic("Returned from Terminate-Request to Hypervisor\n"); + } + + instrumentation_end(); + irqentry_nmi_exit(regs, irq_state); +} + bool __init handle_vc_boot_ghcb(struct pt_regs *regs) { unsigned long exit_code = regs->orig_ax; diff --git a/arch/x86/kernel/traps.c b/arch/x86/kernel/traps.c index d317dc3d06a3..d29debec8134 100644 --- a/arch/x86/kernel/traps.c +++ b/arch/x86/kernel/traps.c @@ -905,6 +905,46 @@ asmlinkage __visible noinstr struct pt_regs *vc_switch_off_ist(struct pt_regs *r return regs_ret; } + +asmlinkage __visible noinstr struct pt_regs *hv_switch_off_ist(struct pt_regs *regs) +{ + unsigned long sp, *stack; + struct stack_info info; + struct pt_regs *regs_ret; + + /* + * In the SYSCALL entry path the RSP value comes from user-space - don't + * trust it and switch to the current kernel stack + */ + if (ip_within_syscall_gap(regs)) { + sp = this_cpu_read(pcpu_hot.top_of_stack); + goto sync; + } + + /* + * From here on the RSP value is trusted. Now check whether entry + * happened from a safe stack. Not safe are the entry or unknown stacks, + * use the fall-back stack instead in this case. + */ + sp = regs->sp; + stack = (unsigned long *)sp; + + if (!get_stack_info_noinstr(stack, current, &info) || info.type == STACK_TYPE_ENTRY || + info.type > STACK_TYPE_EXCEPTION_LAST) + sp = __this_cpu_ist_top_va(HV2); +sync: + /* + * Found a safe stack - switch to it as if the entry didn't happen via + * IST stack. The code below only copies pt_regs, the real switch happens + * in assembly code. + */ + sp = ALIGN_DOWN(sp, 8) - sizeof(*regs_ret); + + regs_ret = (struct pt_regs *)sp; + *regs_ret = *regs; + + return regs_ret; +} #endif asmlinkage __visible noinstr struct pt_regs *fixup_bad_iret(struct pt_regs *bad_regs) diff --git a/arch/x86/mm/cpu_entry_area.c b/arch/x86/mm/cpu_entry_area.c index 7316a8224259..3ec844cef652 100644 --- a/arch/x86/mm/cpu_entry_area.c +++ b/arch/x86/mm/cpu_entry_area.c @@ -153,6 +153,8 @@ static void __init percpu_setup_exception_stacks(unsigned int cpu) if (cc_platform_has(CC_ATTR_GUEST_STATE_ENCRYPT)) { cea_map_stack(VC); cea_map_stack(VC2); + cea_map_stack(HV); + cea_map_stack(HV2); } } } From patchwork Mon Apr 3 17:44:01 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tianyu Lan X-Patchwork-Id: 13198526 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 711C7C77B6C for ; Mon, 3 Apr 2023 17:45:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232782AbjDCRp4 (ORCPT ); Mon, 3 Apr 2023 13:45:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52316 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232628AbjDCRot (ORCPT ); Mon, 3 Apr 2023 13:44:49 -0400 Received: from mail-pj1-x1035.google.com (mail-pj1-x1035.google.com [IPv6:2607:f8b0:4864:20::1035]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EBD0330E3; Mon, 3 Apr 2023 10:44:27 -0700 (PDT) Received: by mail-pj1-x1035.google.com with SMTP id f6-20020a17090ac28600b0023b9bf9eb63so31369605pjt.5; Mon, 03 Apr 2023 10:44:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1680543867; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=JCnmOn8eALVJCVo8Tdsddc5boZ5t7pqgIErQC6wwwQk=; b=WOoo+RlPGJwCaMPKG0qkXj9WrFgkZ46HvLWXK0sYjrTLZ8Lwzu0U0g/fMyypNWxcXQ FUPM3xfOv36KkDEkGEF85kv9Qfqf6lFCG7LsgB/sS3S279Z4jeYfnv+ioZIYpRMh7b83 bc6WFMUReZyqHbyPJgrzG9IRCqKgSSFsIAQcTf8zExx5SbQUMnr4rGdCkCKehIlcfZED XI0p/yBGVB1/ecEO0fGRPuOaj2cUumPYMbafYH4CIYFaUo8l6+TZz9p7SQv7Id0O/gpY yY/hqSZFPoh2rS6maguZFiNU+HdYjC2jUb16opx42XFP+cIqx9ILEpSS5Q9WI0OTsKpa TkRw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680543867; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=JCnmOn8eALVJCVo8Tdsddc5boZ5t7pqgIErQC6wwwQk=; b=HrD6WfDCOtG1JclR9Jydy6mfqnpRa3Q9OJRCdix/8bW6BVQJl8zY8cFE37j2idumov WqrlPVabBhkwymLtXzAlWQOXoJK4dGpWMHs5Bx1N8nBxwz68qU7xiCyUElVizEBC1tui WdRCEPeXIAOXhPMrv6AxKodxcuWLm/62p9JcMqTFswQqpgdK92hbwGrlazwwccFBAlGD DI/tCXJndQsJQFfXJIm0qJkpzN9PutHuTXiVCANV3TH7aKOeipuPAZPJ7Y4/t1uTwgO6 l6B5YnY4rqrjH9uqnkx7FzGAIYbOSI/TmyCgmLm1cN6nGDDaTUpLzq7nvnhlUMQUDPmE Dodg== X-Gm-Message-State: AAQBX9eyz9Gz7hraYSz6pHe37ncd2aK+7CFtfPRyTGmToss9mlb/hMiH XYBPDR+QphlNxAJ9UQfHLGHuNMNHVSHrIBRB X-Google-Smtp-Source: AKy350aca/70/Ld9Mf/GNADUkLESox8Os9PI7C3dj/wE46nTR0Ur/A2gUYAWlfJBUj25XGPUixH8Tw== X-Received: by 2002:a17:903:283:b0:1a2:1922:9850 with SMTP id j3-20020a170903028300b001a219229850mr43163575plr.67.1680543867357; Mon, 03 Apr 2023 10:44:27 -0700 (PDT) Received: from ubuntu-Virtual-Machine.corp.microsoft.com ([2001:4898:80e8:2:8635:6e96:35c1:c560]) by smtp.gmail.com with ESMTPSA id jj21-20020a170903049500b001a19196af48sm6883803plb.64.2023.04.03.10.44.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 03 Apr 2023 10:44:26 -0700 (PDT) From: Tianyu Lan To: luto@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, seanjc@google.com, pbonzini@redhat.com, jgross@suse.com, tiala@microsoft.com, kirill@shutemov.name, jiangshan.ljs@antgroup.com, peterz@infradead.org, ashish.kalra@amd.com, srutherford@google.com, akpm@linux-foundation.org, anshuman.khandual@arm.com, pawan.kumar.gupta@linux.intel.com, adrian.hunter@intel.com, daniel.sneddon@linux.intel.com, alexander.shishkin@linux.intel.com, sandipan.das@amd.com, ray.huang@amd.com, brijesh.singh@amd.com, michael.roth@amd.com, thomas.lendacky@amd.com, venu.busireddy@oracle.com, sterritt@google.com, tony.luck@intel.com, samitolvanen@google.com, fenghua.yu@intel.com Cc: pangupta@amd.com, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-hyperv@vger.kernel.org, linux-arch@vger.kernel.org Subject: [RFC PATCH V4 13/17] x86/sev: Add Check of #HV event in path Date: Mon, 3 Apr 2023 13:44:01 -0400 Message-Id: <20230403174406.4180472-14-ltykernel@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230403174406.4180472-1-ltykernel@gmail.com> References: <20230403174406.4180472-1-ltykernel@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Tianyu Lan Add check_hv_pending() and check_hv_pending_after_irq() to check queued #HV event when irq is disabled. Signed-off-by: Tianyu Lan --- arch/x86/entry/entry_64.S | 18 ++++++++++++++++ arch/x86/include/asm/irqflags.h | 11 ++++++++++ arch/x86/kernel/sev.c | 38 +++++++++++++++++++++++++++++++++ 3 files changed, 67 insertions(+) diff --git a/arch/x86/entry/entry_64.S b/arch/x86/entry/entry_64.S index d877774c3141..efa56dfde19e 100644 --- a/arch/x86/entry/entry_64.S +++ b/arch/x86/entry/entry_64.S @@ -1073,6 +1073,15 @@ SYM_CODE_END(paranoid_entry) * R15 - old SPEC_CTRL */ SYM_CODE_START_LOCAL(paranoid_exit) +#ifdef CONFIG_AMD_MEM_ENCRYPT + /* + * If a #HV was delivered during execution and interrupts were + * disabled, then check if it can be handled before the iret + * (which may re-enable interrupts). + */ + mov %rsp, %rdi + call check_hv_pending +#endif UNWIND_HINT_REGS /* @@ -1197,6 +1206,15 @@ SYM_CODE_START(error_entry) SYM_CODE_END(error_entry) SYM_CODE_START_LOCAL(error_return) +#ifdef CONFIG_AMD_MEM_ENCRYPT + /* + * If a #HV was delivered during execution and interrupts were + * disabled, then check if it can be handled before the iret + * (which may re-enable interrupts). + */ + mov %rsp, %rdi + call check_hv_pending +#endif UNWIND_HINT_REGS DEBUG_ENTRY_ASSERT_IRQS_OFF testb $3, CS(%rsp) diff --git a/arch/x86/include/asm/irqflags.h b/arch/x86/include/asm/irqflags.h index 8c5ae649d2df..8368e3fe2d36 100644 --- a/arch/x86/include/asm/irqflags.h +++ b/arch/x86/include/asm/irqflags.h @@ -11,6 +11,10 @@ /* * Interrupt control: */ +#ifdef CONFIG_AMD_MEM_ENCRYPT +void check_hv_pending(struct pt_regs *regs); +void check_hv_pending_irq_enable(void); +#endif /* Declaration required for gcc < 4.9 to prevent -Werror=missing-prototypes */ extern inline unsigned long native_save_fl(void); @@ -40,12 +44,19 @@ static __always_inline void native_irq_disable(void) static __always_inline void native_irq_enable(void) { asm volatile("sti": : :"memory"); +#ifdef CONFIG_AMD_MEM_ENCRYPT + check_hv_pending_irq_enable(); +#endif } static __always_inline void native_safe_halt(void) { mds_idle_clear_cpu_buffers(); asm volatile("sti; hlt": : :"memory"); + +#ifdef CONFIG_AMD_MEM_ENCRYPT + check_hv_pending_irq_enable(); +#endif } static __always_inline void native_halt(void) diff --git a/arch/x86/kernel/sev.c b/arch/x86/kernel/sev.c index 2684a45b50a6..6445f5356c45 100644 --- a/arch/x86/kernel/sev.c +++ b/arch/x86/kernel/sev.c @@ -179,6 +179,44 @@ void noinstr __sev_es_ist_enter(struct pt_regs *regs) this_cpu_write(cpu_tss_rw.x86_tss.ist[IST_INDEX_VC], new_ist); } +static void do_exc_hv(struct pt_regs *regs) +{ + /* Handle #HV exception. */ +} + +void check_hv_pending(struct pt_regs *regs) +{ + if (!cc_platform_has(CC_ATTR_GUEST_SEV_SNP)) + return; + + if ((regs->flags & X86_EFLAGS_IF) == 0) + return; + + do_exc_hv(regs); +} + +void check_hv_pending_irq_enable(void) +{ + struct pt_regs regs; + + if (!cc_platform_has(CC_ATTR_GUEST_SEV_SNP)) + return; + + memset(®s, 0, sizeof(struct pt_regs)); + asm volatile("movl %%cs, %%eax;" : "=a" (regs.cs)); + asm volatile("movl %%ss, %%eax;" : "=a" (regs.ss)); + regs.orig_ax = 0xffffffff; + regs.flags = native_save_fl(); + + /* + * Disable irq when handle pending #HV events after + * re-enabling irq. + */ + asm volatile("cli" : : : "memory"); + do_exc_hv(®s); + asm volatile("sti" : : : "memory"); +} + void noinstr __sev_es_ist_exit(void) { unsigned long ist; From patchwork Mon Apr 3 17:44:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tianyu Lan X-Patchwork-Id: 13198528 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5EECAC761AF for ; Mon, 3 Apr 2023 17:46:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232206AbjDCRqN (ORCPT ); Mon, 3 Apr 2023 13:46:13 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50966 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232016AbjDCRoz (ORCPT ); Mon, 3 Apr 2023 13:44:55 -0400 Received: from mail-pj1-x102e.google.com (mail-pj1-x102e.google.com [IPv6:2607:f8b0:4864:20::102e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 830613AA5; Mon, 3 Apr 2023 10:44:29 -0700 (PDT) Received: by mail-pj1-x102e.google.com with SMTP id p13-20020a17090a284d00b0023d2e945aebso14457383pjf.0; Mon, 03 Apr 2023 10:44:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1680543869; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=LU2XDrJ3Un64sfFrjwv6qJG+nmpNESwNtX/Lvqu0+x0=; b=HHXLp6dko+hnaG/fOsuc38cmcR/JtFt3AG0R7K7tX8m72mduc7B0Rwcwqp2ulhio4f iio5NceLXdTltX/lA7k0Hu44bioSOX8l0zUUcT+z81d/tsxQ/st6gUreFTU4AwXd+UEp SqFR/OwOL/r6d7e/zuUWKHXDyk/KNW3YAMpnTP/RN3dSgtblQC1QfHDT0uXzdReMR4Ov BN/ULdQ/J/jRH1JBDlty6AnMw3LcbLG8ZuFhud6wdtYo2rthpBA45nToi5iFHRBJ+dex v3Zh3hW2XBGkvKQ9tKoCZHRJAtopVbY3zWuwgaRWygr8MyRATsC5yXWq76w/CA93B97U v7Qg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680543869; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=LU2XDrJ3Un64sfFrjwv6qJG+nmpNESwNtX/Lvqu0+x0=; b=r2UNDsFz4jax/GkJoB8UJXuXicRDx4Nc7Ue/riBmpvgVJ0OPBOtjTiz1F3lod7T1MD yqVcFA3fWGzCRwCzqYA+XPOdNoIO09J8c/INB77h/kxzIz4yU7q1xQ0NhldMTpEFRsqc /IYDfvZVj0wzirnEppezWFZacsBCZt2J6tD9se0sSU/DhnYW5qLAfSYgdtxs2XCEStto qGQNmuYTG53c5bbD6vm9BLWCGUBrWzmaI8do25W4CmiOXsVBvZqQbCs3U0KFVB56b+s2 wRDQsnu5RmJMa48qmWYfbqh3lFlFz90aqzGFAfMKa5qKREX5VZDg8rF6VgIp9rlEIFWP 2iLA== X-Gm-Message-State: AAQBX9eLXER/VU2Ta5f/gmEIf3G6+74yAqPM4SYhmoz7+LHRCJFhOcKh IizPZD96cx2ZUK3nE8YDuow= X-Google-Smtp-Source: AKy350b1AygEIDPp0c+Uz33dG3LsrRD/5DPPJWUS32lZluaja4f86BReIhNKGFwxESQ3GeiLkRM4+g== X-Received: by 2002:a17:902:ecc4:b0:1a1:d949:a52d with SMTP id a4-20020a170902ecc400b001a1d949a52dmr46465677plh.65.1680543868585; Mon, 03 Apr 2023 10:44:28 -0700 (PDT) Received: from ubuntu-Virtual-Machine.corp.microsoft.com ([2001:4898:80e8:2:8635:6e96:35c1:c560]) by smtp.gmail.com with ESMTPSA id jj21-20020a170903049500b001a19196af48sm6883803plb.64.2023.04.03.10.44.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 03 Apr 2023 10:44:28 -0700 (PDT) From: Tianyu Lan To: luto@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, seanjc@google.com, pbonzini@redhat.com, jgross@suse.com, tiala@microsoft.com, kirill@shutemov.name, jiangshan.ljs@antgroup.com, peterz@infradead.org, ashish.kalra@amd.com, srutherford@google.com, akpm@linux-foundation.org, anshuman.khandual@arm.com, pawan.kumar.gupta@linux.intel.com, adrian.hunter@intel.com, daniel.sneddon@linux.intel.com, alexander.shishkin@linux.intel.com, sandipan.das@amd.com, ray.huang@amd.com, brijesh.singh@amd.com, michael.roth@amd.com, thomas.lendacky@amd.com, venu.busireddy@oracle.com, sterritt@google.com, tony.luck@intel.com, samitolvanen@google.com, fenghua.yu@intel.com Cc: pangupta@amd.com, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-hyperv@vger.kernel.org, linux-arch@vger.kernel.org Subject: [RFC PATCH V4 14/17] x86/hyperv/sev: Add AMD sev-snp enlightened guest support on hyperv Date: Mon, 3 Apr 2023 13:44:02 -0400 Message-Id: <20230403174406.4180472-15-ltykernel@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230403174406.4180472-1-ltykernel@gmail.com> References: <20230403174406.4180472-1-ltykernel@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Tianyu Lan Enable #HV exception to handle interrupt requests from hypervisor. Co-developed-by: Lendacky Thomas Co-developed-by: Kalra Ashish Signed-off-by: Tianyu Lan --- Change since RFC V3: * Check NMI event when irq is disabled. * Remove redundant variable --- arch/x86/include/asm/mem_encrypt.h | 2 + arch/x86/include/uapi/asm/svm.h | 4 + arch/x86/kernel/sev.c | 314 ++++++++++++++++++++++++----- arch/x86/kernel/traps.c | 2 + 4 files changed, 266 insertions(+), 56 deletions(-) diff --git a/arch/x86/include/asm/mem_encrypt.h b/arch/x86/include/asm/mem_encrypt.h index b7126701574c..9299caeca69f 100644 --- a/arch/x86/include/asm/mem_encrypt.h +++ b/arch/x86/include/asm/mem_encrypt.h @@ -50,6 +50,7 @@ void __init early_set_mem_enc_dec_hypercall(unsigned long vaddr, int npages, void __init mem_encrypt_free_decrypted_mem(void); void __init sev_es_init_vc_handling(void); +void __init sev_snp_init_hv_handling(void); #define __bss_decrypted __section(".bss..decrypted") @@ -73,6 +74,7 @@ static inline void __init sme_encrypt_kernel(struct boot_params *bp) { } static inline void __init sme_enable(struct boot_params *bp) { } static inline void sev_es_init_vc_handling(void) { } +static inline void sev_snp_init_hv_handling(void) { } static inline int __init early_set_memory_decrypted(unsigned long vaddr, unsigned long size) { return 0; } diff --git a/arch/x86/include/uapi/asm/svm.h b/arch/x86/include/uapi/asm/svm.h index 80e1df482337..828d624a38cf 100644 --- a/arch/x86/include/uapi/asm/svm.h +++ b/arch/x86/include/uapi/asm/svm.h @@ -115,6 +115,10 @@ #define SVM_VMGEXIT_AP_CREATE_ON_INIT 0 #define SVM_VMGEXIT_AP_CREATE 1 #define SVM_VMGEXIT_AP_DESTROY 2 +#define SVM_VMGEXIT_HV_DOORBELL_PAGE 0x80000014 +#define SVM_VMGEXIT_GET_PREFERRED_HV_DOORBELL_PAGE 0 +#define SVM_VMGEXIT_SET_HV_DOORBELL_PAGE 1 +#define SVM_VMGEXIT_QUERY_HV_DOORBELL_PAGE 2 #define SVM_VMGEXIT_HV_FEATURES 0x8000fffd #define SVM_VMGEXIT_TERM_REQUEST 0x8000fffe #define SVM_VMGEXIT_TERM_REASON(reason_set, reason_code) \ diff --git a/arch/x86/kernel/sev.c b/arch/x86/kernel/sev.c index 6445f5356c45..7fcb3b548215 100644 --- a/arch/x86/kernel/sev.c +++ b/arch/x86/kernel/sev.c @@ -122,6 +122,152 @@ struct sev_config { static struct sev_config sev_cfg __read_mostly; +static noinstr struct ghcb *__sev_get_ghcb(struct ghcb_state *state); +static noinstr void __sev_put_ghcb(struct ghcb_state *state); +static int vmgexit_hv_doorbell_page(struct ghcb *ghcb, u64 op, u64 pa); +static void sev_snp_setup_hv_doorbell_page(struct ghcb *ghcb); + +union hv_pending_events { + u16 events; + struct { + u8 vector; + u8 nmi : 1; + u8 mc : 1; + u8 reserved1 : 5; + u8 no_further_signal : 1; + }; +}; + +struct sev_hv_doorbell_page { + union hv_pending_events pending_events; + u8 no_eoi_required; + u8 reserved2[61]; + u8 padding[4032]; +}; + +struct sev_snp_runtime_data { + struct sev_hv_doorbell_page hv_doorbell_page; +}; + +static DEFINE_PER_CPU(struct sev_snp_runtime_data*, snp_runtime_data); + +static inline u64 sev_es_rd_ghcb_msr(void) +{ + return __rdmsr(MSR_AMD64_SEV_ES_GHCB); +} + +static __always_inline void sev_es_wr_ghcb_msr(u64 val) +{ + u32 low, high; + + low = (u32)(val); + high = (u32)(val >> 32); + + native_wrmsr(MSR_AMD64_SEV_ES_GHCB, low, high); +} + +struct sev_hv_doorbell_page *sev_snp_current_doorbell_page(void) +{ + return &this_cpu_read(snp_runtime_data)->hv_doorbell_page; +} + +static u8 sev_hv_pending(void) +{ + return sev_snp_current_doorbell_page()->pending_events.events; +} + +#define sev_hv_pending_nmi \ + sev_snp_current_doorbell_page()->pending_events.nmi + +static void hv_doorbell_apic_eoi_write(u32 reg, u32 val) +{ + if (xchg(&sev_snp_current_doorbell_page()->no_eoi_required, 0) & 0x1) + return; + + BUG_ON(reg != APIC_EOI); + apic->write(reg, val); +} + +static void do_exc_hv(struct pt_regs *regs) +{ + union hv_pending_events pending_events; + + while (sev_hv_pending()) { + pending_events.events = xchg( + &sev_snp_current_doorbell_page()->pending_events.events, + 0); + + if (pending_events.nmi) + exc_nmi(regs); + +#ifdef CONFIG_X86_MCE + if (pending_events.mc) + exc_machine_check(regs); +#endif + + if (!pending_events.vector) + return; + + if (pending_events.vector < FIRST_EXTERNAL_VECTOR) { + /* Exception vectors */ + WARN(1, "exception shouldn't happen\n"); + } else if (pending_events.vector == FIRST_EXTERNAL_VECTOR) { + sysvec_irq_move_cleanup(regs); + } else if (pending_events.vector == IA32_SYSCALL_VECTOR) { + WARN(1, "syscall shouldn't happen\n"); + } else if (pending_events.vector >= FIRST_SYSTEM_VECTOR) { + switch (pending_events.vector) { +#if IS_ENABLED(CONFIG_HYPERV) + case HYPERV_STIMER0_VECTOR: + sysvec_hyperv_stimer0(regs); + break; + case HYPERVISOR_CALLBACK_VECTOR: + sysvec_hyperv_callback(regs); + break; +#endif +#ifdef CONFIG_SMP + case RESCHEDULE_VECTOR: + sysvec_reschedule_ipi(regs); + break; + case IRQ_MOVE_CLEANUP_VECTOR: + sysvec_irq_move_cleanup(regs); + break; + case REBOOT_VECTOR: + sysvec_reboot(regs); + break; + case CALL_FUNCTION_SINGLE_VECTOR: + sysvec_call_function_single(regs); + break; + case CALL_FUNCTION_VECTOR: + sysvec_call_function(regs); + break; +#endif +#ifdef CONFIG_X86_LOCAL_APIC + case ERROR_APIC_VECTOR: + sysvec_error_interrupt(regs); + break; + case SPURIOUS_APIC_VECTOR: + sysvec_spurious_apic_interrupt(regs); + break; + case LOCAL_TIMER_VECTOR: + sysvec_apic_timer_interrupt(regs); + break; + case X86_PLATFORM_IPI_VECTOR: + sysvec_x86_platform_ipi(regs); + break; +#endif + case 0x0: + break; + default: + panic("Unexpected vector %d\n", vector); + unreachable(); + } + } else { + common_interrupt(regs, pending_events.vector); + } + } +} + static __always_inline bool on_vc_stack(struct pt_regs *regs) { unsigned long sp = regs->sp; @@ -179,18 +325,19 @@ void noinstr __sev_es_ist_enter(struct pt_regs *regs) this_cpu_write(cpu_tss_rw.x86_tss.ist[IST_INDEX_VC], new_ist); } -static void do_exc_hv(struct pt_regs *regs) -{ - /* Handle #HV exception. */ -} - void check_hv_pending(struct pt_regs *regs) { if (!cc_platform_has(CC_ATTR_GUEST_SEV_SNP)) return; - if ((regs->flags & X86_EFLAGS_IF) == 0) + /* Handle NMI when irq is disabled. */ + if ((regs->flags & X86_EFLAGS_IF) == 0) { + if (sev_hv_pending_nmi) { + exc_nmi(regs); + sev_hv_pending_nmi = 0; + } return; + } do_exc_hv(regs); } @@ -231,68 +378,35 @@ void noinstr __sev_es_ist_exit(void) this_cpu_write(cpu_tss_rw.x86_tss.ist[IST_INDEX_VC], *(unsigned long *)ist); } -/* - * Nothing shall interrupt this code path while holding the per-CPU - * GHCB. The backup GHCB is only for NMIs interrupting this path. - * - * Callers must disable local interrupts around it. - */ -static noinstr struct ghcb *__sev_get_ghcb(struct ghcb_state *state) +static bool sev_restricted_injection_enabled(void) +{ + return sev_status & MSR_AMD64_SNP_RESTRICTED_INJ; +} + +void __init sev_snp_init_hv_handling(void) { struct sev_es_runtime_data *data; + struct ghcb_state state; struct ghcb *ghcb; + unsigned long flags; WARN_ON(!irqs_disabled()); + if (!cc_platform_has(CC_ATTR_GUEST_SEV_SNP) || !sev_restricted_injection_enabled()) + return; data = this_cpu_read(runtime_data); - ghcb = &data->ghcb_page; - - if (unlikely(data->ghcb_active)) { - /* GHCB is already in use - save its contents */ - - if (unlikely(data->backup_ghcb_active)) { - /* - * Backup-GHCB is also already in use. There is no way - * to continue here so just kill the machine. To make - * panic() work, mark GHCBs inactive so that messages - * can be printed out. - */ - data->ghcb_active = false; - data->backup_ghcb_active = false; - - instrumentation_begin(); - panic("Unable to handle #VC exception! GHCB and Backup GHCB are already in use"); - instrumentation_end(); - } - - /* Mark backup_ghcb active before writing to it */ - data->backup_ghcb_active = true; - state->ghcb = &data->backup_ghcb; + local_irq_save(flags); - /* Backup GHCB content */ - *state->ghcb = *ghcb; - } else { - state->ghcb = NULL; - data->ghcb_active = true; - } + ghcb = __sev_get_ghcb(&state); - return ghcb; -} + sev_snp_setup_hv_doorbell_page(ghcb); -static inline u64 sev_es_rd_ghcb_msr(void) -{ - return __rdmsr(MSR_AMD64_SEV_ES_GHCB); -} - -static __always_inline void sev_es_wr_ghcb_msr(u64 val) -{ - u32 low, high; + __sev_put_ghcb(&state); - low = (u32)(val); - high = (u32)(val >> 32); + apic_set_eoi_write(hv_doorbell_apic_eoi_write); - native_wrmsr(MSR_AMD64_SEV_ES_GHCB, low, high); + local_irq_restore(flags); } static int vc_fetch_insn_kernel(struct es_em_ctxt *ctxt, @@ -553,6 +667,69 @@ static enum es_result vc_slow_virt_to_phys(struct ghcb *ghcb, struct es_em_ctxt /* Include code shared with pre-decompression boot stage */ #include "sev-shared.c" +/* + * Nothing shall interrupt this code path while holding the per-CPU + * GHCB. The backup GHCB is only for NMIs interrupting this path. + * + * Callers must disable local interrupts around it. + */ +static noinstr struct ghcb *__sev_get_ghcb(struct ghcb_state *state) +{ + struct sev_es_runtime_data *data; + struct ghcb *ghcb; + + WARN_ON(!irqs_disabled()); + + data = this_cpu_read(runtime_data); + ghcb = &data->ghcb_page; + + if (unlikely(data->ghcb_active)) { + /* GHCB is already in use - save its contents */ + + if (unlikely(data->backup_ghcb_active)) { + /* + * Backup-GHCB is also already in use. There is no way + * to continue here so just kill the machine. To make + * panic() work, mark GHCBs inactive so that messages + * can be printed out. + */ + data->ghcb_active = false; + data->backup_ghcb_active = false; + + instrumentation_begin(); + panic("Unable to handle #VC exception! GHCB and Backup GHCB are already in use"); + instrumentation_end(); + } + + /* Mark backup_ghcb active before writing to it */ + data->backup_ghcb_active = true; + + state->ghcb = &data->backup_ghcb; + + /* Backup GHCB content */ + *state->ghcb = *ghcb; + } else { + state->ghcb = NULL; + data->ghcb_active = true; + } + + return ghcb; +} + +static void sev_snp_setup_hv_doorbell_page(struct ghcb *ghcb) +{ + u64 pa; + enum es_result ret; + + pa = __pa(sev_snp_current_doorbell_page()); + vc_ghcb_invalidate(ghcb); + ret = vmgexit_hv_doorbell_page(ghcb, + SVM_VMGEXIT_SET_HV_DOORBELL_PAGE, + pa); + if (ret != ES_OK) + panic("SEV-SNP: failed to set up #HV doorbell page"); +} + static noinstr void __sev_put_ghcb(struct ghcb_state *state) { struct sev_es_runtime_data *data; @@ -1281,6 +1458,7 @@ static void snp_register_per_cpu_ghcb(void) ghcb = &data->ghcb_page; snp_register_ghcb_early(__pa(ghcb)); + sev_snp_setup_hv_doorbell_page(ghcb); } void setup_ghcb(void) @@ -1320,6 +1498,11 @@ void setup_ghcb(void) snp_register_ghcb_early(__pa(&boot_ghcb_page)); } +int vmgexit_hv_doorbell_page(struct ghcb *ghcb, u64 op, u64 pa) +{ + return sev_es_ghcb_hv_call(ghcb, NULL, SVM_VMGEXIT_HV_DOORBELL_PAGE, op, pa); +} + #ifdef CONFIG_HOTPLUG_CPU static void sev_es_ap_hlt_loop(void) { @@ -1393,6 +1576,7 @@ static void __init alloc_runtime_data(int cpu) static void __init init_ghcb(int cpu) { struct sev_es_runtime_data *data; + struct sev_snp_runtime_data *snp_data; int err; data = per_cpu(runtime_data, cpu); @@ -1404,6 +1588,19 @@ static void __init init_ghcb(int cpu) memset(&data->ghcb_page, 0, sizeof(data->ghcb_page)); + snp_data = memblock_alloc(sizeof(*snp_data), PAGE_SIZE); + if (!snp_data) + panic("Can't allocate SEV-SNP runtime data"); + + err = early_set_memory_decrypted((unsigned long)&snp_data->hv_doorbell_page, + sizeof(snp_data->hv_doorbell_page)); + if (err) + panic("Can't map #HV doorbell pages unencrypted"); + + memset(&snp_data->hv_doorbell_page, 0, sizeof(snp_data->hv_doorbell_page)); + + per_cpu(snp_runtime_data, cpu) = snp_data; + data->ghcb_active = false; data->backup_ghcb_active = false; } @@ -2044,7 +2241,12 @@ DEFINE_IDTENTRY_VC_USER(exc_vmm_communication) static bool hv_raw_handle_exception(struct pt_regs *regs) { - return false; + /* Clear the no_further_signal bit */ + sev_snp_current_doorbell_page()->pending_events.events &= 0x7fff; + + check_hv_pending(regs); + + return true; } static __always_inline bool on_hv_fallback_stack(struct pt_regs *regs) diff --git a/arch/x86/kernel/traps.c b/arch/x86/kernel/traps.c index d29debec8134..1aa6cab2394b 100644 --- a/arch/x86/kernel/traps.c +++ b/arch/x86/kernel/traps.c @@ -1503,5 +1503,7 @@ void __init trap_init(void) cpu_init_exception_handling(); /* Setup traps as cpu_init() might #GP */ idt_setup_traps(); + sev_snp_init_hv_handling(); + cpu_init(); } From patchwork Mon Apr 3 17:44:03 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tianyu Lan X-Patchwork-Id: 13198527 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0F4F3C761AF for ; Mon, 3 Apr 2023 17:46:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232823AbjDCRqK (ORCPT ); Mon, 3 Apr 2023 13:46:10 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50684 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231579AbjDCRow (ORCPT ); Mon, 3 Apr 2023 13:44:52 -0400 Received: from mail-pl1-x632.google.com (mail-pl1-x632.google.com [IPv6:2607:f8b0:4864:20::632]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 694A830DD; Mon, 3 Apr 2023 10:44:30 -0700 (PDT) Received: by mail-pl1-x632.google.com with SMTP id w4so28788424plg.9; Mon, 03 Apr 2023 10:44:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1680543870; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=dLIZsKwb/Fiqfvq7w/cI7yvHscawpRIWUMLmHd0mf5o=; b=fjf92n0dw0W70GKRm2BWERl2DClz3HWPZdXaAZrCpsGGp1/qKQjcrMcBf/68QUfoep KhTF2nrpMZ/aztYQVk0xk1diDf4ByJbqj9S0O7qi6ooTjs7pVhBrCX1QOfIBExHJhFtP R5PCeLyy8rqXPcNruoB/1ns6fxMSlrb+h86N5W0DxNeyvR/ErM8G3F7YlbwcK4lwNtI6 79tJpJvGwOyXc8S83U4MDuCHd7dvJOh/7TvSCrGP/budNZ62Q0n2rBsPG3H+VAm5buKg 7pZThddZN3D0k+ZtKe/rEL0AFYqkDiwvBK6Num+zN19GkcEI4Qy6Jtu+qVcq+KLjxxRs tiBw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680543870; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=dLIZsKwb/Fiqfvq7w/cI7yvHscawpRIWUMLmHd0mf5o=; b=O2Rrx63N6Q/TOwCv5O5VZuPS5aVl/BeUEbmhY7Ey0ZNrBiF84RWkoFkptlZVAk1qx1 JcKwetx4CfS42sLdeKkt3dUrsoYH+mRO+s6CYGyuNhZVCeTdvbBTuJMPL5U/czrcN5VC fXyXfEBs7D6sKlH5axyxK0oiX6IvFJ+oJ3yBcbTw18zMAa2UG8w6e4Q1fJPiawRriEdO hYVhPiOKvv9aQGe1vve4sZvcWy0PiSji2SEJdz+1ejyD3xPIg9fusjmYnJLD9K5658LA gbwyf99seyTn3sl3WQJ1mVtbY/epROV4m4LyfMHfGSvgeefkVuS+3xD5Wo8bl7On30nK b4Iw== X-Gm-Message-State: AAQBX9dgrA0iI8L5ScYP2an5A/0Ju+X8uLGNO+flc9EI+dPfappycq35 BqfZDFy4D7TruejEeZzbdq0= X-Google-Smtp-Source: AKy350YHd0a+6UzrP8X+j2amAzAEzbG1MK2R+yLhVtWdREMB7NF1P6VBcNU7EOa8tGEveSjLCMRFnw== X-Received: by 2002:a17:903:32c7:b0:1a2:8871:b419 with SMTP id i7-20020a17090332c700b001a28871b419mr21645144plr.23.1680543869792; Mon, 03 Apr 2023 10:44:29 -0700 (PDT) Received: from ubuntu-Virtual-Machine.corp.microsoft.com ([2001:4898:80e8:2:8635:6e96:35c1:c560]) by smtp.gmail.com with ESMTPSA id jj21-20020a170903049500b001a19196af48sm6883803plb.64.2023.04.03.10.44.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 03 Apr 2023 10:44:29 -0700 (PDT) From: Tianyu Lan To: luto@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, seanjc@google.com, pbonzini@redhat.com, jgross@suse.com, tiala@microsoft.com, kirill@shutemov.name, jiangshan.ljs@antgroup.com, peterz@infradead.org, ashish.kalra@amd.com, srutherford@google.com, akpm@linux-foundation.org, anshuman.khandual@arm.com, pawan.kumar.gupta@linux.intel.com, adrian.hunter@intel.com, daniel.sneddon@linux.intel.com, alexander.shishkin@linux.intel.com, sandipan.das@amd.com, ray.huang@amd.com, brijesh.singh@amd.com, michael.roth@amd.com, thomas.lendacky@amd.com, venu.busireddy@oracle.com, sterritt@google.com, tony.luck@intel.com, samitolvanen@google.com, fenghua.yu@intel.com Cc: pangupta@amd.com, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-hyperv@vger.kernel.org, linux-arch@vger.kernel.org Subject: [RFC PATCH V4 15/17] x86/sev: optimize system vector processing invoked from #HV exception Date: Mon, 3 Apr 2023 13:44:03 -0400 Message-Id: <20230403174406.4180472-16-ltykernel@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230403174406.4180472-1-ltykernel@gmail.com> References: <20230403174406.4180472-1-ltykernel@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Ashish Kalra Construct system vector table and dispatch system vector exceptions through sysvec_table from #HV exception handler instead of explicitly calling each system vector. The system vector table is created dynamically and is placed in a new named ELF section. Signed-off-by: Ashish Kalra --- arch/x86/entry/entry_64.S | 6 +++ arch/x86/kernel/sev.c | 70 +++++++++++++---------------------- arch/x86/kernel/vmlinux.lds.S | 7 ++++ 3 files changed, 38 insertions(+), 45 deletions(-) diff --git a/arch/x86/entry/entry_64.S b/arch/x86/entry/entry_64.S index efa56dfde19e..802fb6dec244 100644 --- a/arch/x86/entry/entry_64.S +++ b/arch/x86/entry/entry_64.S @@ -419,6 +419,12 @@ SYM_CODE_START(\asmsym) _ASM_NOKPROBE(\asmsym) SYM_CODE_END(\asmsym) + .if \vector >= FIRST_SYSTEM_VECTOR && \vector < NR_VECTORS + .section .system_vectors, "aw" + .byte \vector + .quad \cfunc + .previous + .endif .endm /* diff --git a/arch/x86/kernel/sev.c b/arch/x86/kernel/sev.c index 7fcb3b548215..2c964f7ac7dc 100644 --- a/arch/x86/kernel/sev.c +++ b/arch/x86/kernel/sev.c @@ -151,6 +151,16 @@ struct sev_snp_runtime_data { static DEFINE_PER_CPU(struct sev_snp_runtime_data*, snp_runtime_data); +static void (*sysvec_table[NR_VECTORS - FIRST_SYSTEM_VECTOR]) + (struct pt_regs *regs) __ro_after_init; + +struct sysvec_entry { + unsigned char vector; + void (*sysvec_func)(struct pt_regs *regs); +} __packed; + +extern struct sysvec_entry __system_vectors[], __system_vectors_end[]; + static inline u64 sev_es_rd_ghcb_msr(void) { return __rdmsr(MSR_AMD64_SEV_ES_GHCB); @@ -216,51 +226,11 @@ static void do_exc_hv(struct pt_regs *regs) } else if (pending_events.vector == IA32_SYSCALL_VECTOR) { WARN(1, "syscall shouldn't happen\n"); } else if (pending_events.vector >= FIRST_SYSTEM_VECTOR) { - switch (pending_events.vector) { -#if IS_ENABLED(CONFIG_HYPERV) - case HYPERV_STIMER0_VECTOR: - sysvec_hyperv_stimer0(regs); - break; - case HYPERVISOR_CALLBACK_VECTOR: - sysvec_hyperv_callback(regs); - break; -#endif -#ifdef CONFIG_SMP - case RESCHEDULE_VECTOR: - sysvec_reschedule_ipi(regs); - break; - case IRQ_MOVE_CLEANUP_VECTOR: - sysvec_irq_move_cleanup(regs); - break; - case REBOOT_VECTOR: - sysvec_reboot(regs); - break; - case CALL_FUNCTION_SINGLE_VECTOR: - sysvec_call_function_single(regs); - break; - case CALL_FUNCTION_VECTOR: - sysvec_call_function(regs); - break; -#endif -#ifdef CONFIG_X86_LOCAL_APIC - case ERROR_APIC_VECTOR: - sysvec_error_interrupt(regs); - break; - case SPURIOUS_APIC_VECTOR: - sysvec_spurious_apic_interrupt(regs); - break; - case LOCAL_TIMER_VECTOR: - sysvec_apic_timer_interrupt(regs); - break; - case X86_PLATFORM_IPI_VECTOR: - sysvec_x86_platform_ipi(regs); - break; -#endif - case 0x0: - break; - default: - panic("Unexpected vector %d\n", vector); - unreachable(); + if (!(sysvec_table[pending_events.vector - FIRST_SYSTEM_VECTOR])) { + WARN(1, "system vector entry 0x%x is NULL\n", + pending_events.vector); + } else { + (*sysvec_table[pending_events.vector - FIRST_SYSTEM_VECTOR])(regs); } } else { common_interrupt(regs, pending_events.vector); @@ -383,6 +353,14 @@ static bool sev_restricted_injection_enabled(void) return sev_status & MSR_AMD64_SNP_RESTRICTED_INJ; } +static void __init construct_sysvec_table(void) +{ + struct sysvec_entry *p; + + for (p = __system_vectors; p < __system_vectors_end; p++) + sysvec_table[p->vector - FIRST_SYSTEM_VECTOR] = p->sysvec_func; +} + void __init sev_snp_init_hv_handling(void) { struct sev_es_runtime_data *data; @@ -407,6 +385,8 @@ void __init sev_snp_init_hv_handling(void) apic_set_eoi_write(hv_doorbell_apic_eoi_write); local_irq_restore(flags); + + construct_sysvec_table(); } static int vc_fetch_insn_kernel(struct es_em_ctxt *ctxt, diff --git a/arch/x86/kernel/vmlinux.lds.S b/arch/x86/kernel/vmlinux.lds.S index 25f155205770..c37165d8e877 100644 --- a/arch/x86/kernel/vmlinux.lds.S +++ b/arch/x86/kernel/vmlinux.lds.S @@ -338,6 +338,13 @@ SECTIONS *(.altinstr_replacement) } + . = ALIGN(8); + .system_vectors : AT(ADDR(.system_vectors) - LOAD_OFFSET) { + __system_vectors = .; + *(.system_vectors) + __system_vectors_end = .; + } + . = ALIGN(8); .apicdrivers : AT(ADDR(.apicdrivers) - LOAD_OFFSET) { __apicdrivers = .; From patchwork Mon Apr 3 17:44:04 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tianyu Lan X-Patchwork-Id: 13198529 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AA824C76196 for ; Mon, 3 Apr 2023 17:46:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232855AbjDCRqY (ORCPT ); Mon, 3 Apr 2023 13:46:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52266 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231906AbjDCRpX (ORCPT ); Mon, 3 Apr 2023 13:45:23 -0400 Received: from mail-pj1-x102f.google.com (mail-pj1-x102f.google.com [IPv6:2607:f8b0:4864:20::102f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DAEF13C10; Mon, 3 Apr 2023 10:44:31 -0700 (PDT) Received: by mail-pj1-x102f.google.com with SMTP id om3-20020a17090b3a8300b0023efab0e3bfso33392930pjb.3; Mon, 03 Apr 2023 10:44:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1680543871; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Bgf457aunDrkl2WhwoRG8QIPPft59HyZBgW7yaFAyA4=; b=OMShyxyPTVdfPfhFnqcPIKKLJd2IIWaJNF8XCVmJ3jVqpq3RDw2zJCNHAZrZd7kV/g cMCkpcLcVznjFi8A5KmvMhRuVJ8Flr/3oSRaQaCSuCwGnUBJozqaiXaX1SovzkmEMQNm XERfLsjYJrV+lOrQpVdQfrMBkH8rxgczU9jxj1v09GNDz32YNr+FVEOhBR2DpFGJ983c HifoDUEDYfaBo6JVBgjfmY3uK+ek4KBaPOSQnh17oO/DHnWkhUN63VZ6e1o8gF561fww vXwGzKxvMYylLW8dADI7aW4TQpzTyrMURnF3mcXG/FKqE6fruUBJ5Y223x2uwwfROYDC DZ1w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680543871; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Bgf457aunDrkl2WhwoRG8QIPPft59HyZBgW7yaFAyA4=; b=wNO9hJUqjFCw4EyYYyBuyo1MkNOL69lzbWdR0S0g8GIidLIaU+P6YMdIMUzLFn6q5R GXCZDMTVkilpF35xmUCNcAcRm7jzMVmMgoDjE/ht2YtGR147wUnXx1nuUrp6JvSYns5I Y1C8ylr9rUJs5rpbMVdlKMyPPkh97WBmvd2qOdY2riYZzKtRch8c57dmJNX73fBWf8ja x7dUQD4l1cTZHI1EnZMCDyQ90TjrSNWs5epPuFg56FV1jWifdA8U//AFNPEabeuL+jW1 uy/lyDacFAM0s7maMa0qgW6GRcQma5CY94mPlZYkMAVQuKW80fi2ftXgrV6D+rjdSS/S ncGA== X-Gm-Message-State: AAQBX9dNkizFL0wkdTkf7Ea2aG1xejuir4LWDGY6ovCC6Z05boyCdf8Z 1FH+uMKqI/5j91hN9anbQYw= X-Google-Smtp-Source: AKy350ap5d9DGz1XnWBU56RZbP9TcA2E7/HgF6MA7iTiEtbMMw/LoWZiaPBQElRAKwCvxW9xWzeefA== X-Received: by 2002:a17:902:7c11:b0:1a0:566c:aaf8 with SMTP id x17-20020a1709027c1100b001a0566caaf8mr31283646pll.27.1680543871035; Mon, 03 Apr 2023 10:44:31 -0700 (PDT) Received: from ubuntu-Virtual-Machine.corp.microsoft.com ([2001:4898:80e8:2:8635:6e96:35c1:c560]) by smtp.gmail.com with ESMTPSA id jj21-20020a170903049500b001a19196af48sm6883803plb.64.2023.04.03.10.44.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 03 Apr 2023 10:44:30 -0700 (PDT) From: Tianyu Lan To: luto@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, seanjc@google.com, pbonzini@redhat.com, jgross@suse.com, tiala@microsoft.com, kirill@shutemov.name, jiangshan.ljs@antgroup.com, peterz@infradead.org, ashish.kalra@amd.com, srutherford@google.com, akpm@linux-foundation.org, anshuman.khandual@arm.com, pawan.kumar.gupta@linux.intel.com, adrian.hunter@intel.com, daniel.sneddon@linux.intel.com, alexander.shishkin@linux.intel.com, sandipan.das@amd.com, ray.huang@amd.com, brijesh.singh@amd.com, michael.roth@amd.com, thomas.lendacky@amd.com, venu.busireddy@oracle.com, sterritt@google.com, tony.luck@intel.com, samitolvanen@google.com, fenghua.yu@intel.com Cc: pangupta@amd.com, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-hyperv@vger.kernel.org, linux-arch@vger.kernel.org Subject: [RFC PATCH V4 16/17] x86/sev: Fix interrupt exit code paths from #HV exception Date: Mon, 3 Apr 2023 13:44:04 -0400 Message-Id: <20230403174406.4180472-17-ltykernel@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230403174406.4180472-1-ltykernel@gmail.com> References: <20230403174406.4180472-1-ltykernel@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Ashish Kalra Add checks in interrupt exit code paths in case of returns to user mode to check if currently executing the #HV handler then don't follow the irqentry_exit_to_user_mode path as that can potentially cause the #HV handler to be preempted and rescheduled on another CPU. Rescheduled #HV handler on another cpu will cause interrupts to be handled on a different cpu than the injected one, causing invalid EOIs and missed/lost guest interrupts and corresponding hangs and/or per-cpu IRQs handled on non-intended cpu. Signed-off-by: Ashish Kalra --- Change since RFC v3: * Add check of hv_handling_events in the do_exc_hv() to avoid nested entry. --- arch/x86/include/asm/idtentry.h | 66 +++++++++++++++++++++++++++++++++ arch/x86/kernel/sev.c | 34 +++++++++++++++++ 2 files changed, 100 insertions(+) diff --git a/arch/x86/include/asm/idtentry.h b/arch/x86/include/asm/idtentry.h index 22ac10a7b34f..08cc99936531 100644 --- a/arch/x86/include/asm/idtentry.h +++ b/arch/x86/include/asm/idtentry.h @@ -13,6 +13,10 @@ #include +#ifdef CONFIG_AMD_MEM_ENCRYPT +noinstr void irqentry_exit_hv_cond(struct pt_regs *regs, irqentry_state_t state); +#endif + /** * DECLARE_IDTENTRY - Declare functions for simple IDT entry points * No error code pushed by hardware @@ -176,6 +180,7 @@ __visible noinstr void func(struct pt_regs *regs, unsigned long error_code) #define DECLARE_IDTENTRY_IRQ(vector, func) \ DECLARE_IDTENTRY_ERRORCODE(vector, func) +#ifndef CONFIG_AMD_MEM_ENCRYPT /** * DEFINE_IDTENTRY_IRQ - Emit code for device interrupt IDT entry points * @func: Function name of the entry point @@ -205,6 +210,26 @@ __visible noinstr void func(struct pt_regs *regs, \ } \ \ static noinline void __##func(struct pt_regs *regs, u32 vector) +#else + +#define DEFINE_IDTENTRY_IRQ(func) \ +static void __##func(struct pt_regs *regs, u32 vector); \ + \ +__visible noinstr void func(struct pt_regs *regs, \ + unsigned long error_code) \ +{ \ + irqentry_state_t state = irqentry_enter(regs); \ + u32 vector = (u32)(u8)error_code; \ + \ + instrumentation_begin(); \ + kvm_set_cpu_l1tf_flush_l1d(); \ + run_irq_on_irqstack_cond(__##func, regs, vector); \ + instrumentation_end(); \ + irqentry_exit_hv_cond(regs, state); \ +} \ + \ +static noinline void __##func(struct pt_regs *regs, u32 vector) +#endif /** * DECLARE_IDTENTRY_SYSVEC - Declare functions for system vector entry points @@ -221,6 +246,7 @@ static noinline void __##func(struct pt_regs *regs, u32 vector) #define DECLARE_IDTENTRY_SYSVEC(vector, func) \ DECLARE_IDTENTRY(vector, func) +#ifndef CONFIG_AMD_MEM_ENCRYPT /** * DEFINE_IDTENTRY_SYSVEC - Emit code for system vector IDT entry points * @func: Function name of the entry point @@ -245,6 +271,26 @@ __visible noinstr void func(struct pt_regs *regs) \ } \ \ static noinline void __##func(struct pt_regs *regs) +#else + +#define DEFINE_IDTENTRY_SYSVEC(func) \ +static void __##func(struct pt_regs *regs); \ + \ +__visible noinstr void func(struct pt_regs *regs) \ +{ \ + irqentry_state_t state = irqentry_enter(regs); \ + \ + instrumentation_begin(); \ + kvm_set_cpu_l1tf_flush_l1d(); \ + run_sysvec_on_irqstack_cond(__##func, regs); \ + instrumentation_end(); \ + irqentry_exit_hv_cond(regs, state); \ +} \ + \ +static noinline void __##func(struct pt_regs *regs) +#endif + +#ifndef CONFIG_AMD_MEM_ENCRYPT /** * DEFINE_IDTENTRY_SYSVEC_SIMPLE - Emit code for simple system vector IDT @@ -274,6 +320,26 @@ __visible noinstr void func(struct pt_regs *regs) \ } \ \ static __always_inline void __##func(struct pt_regs *regs) +#else + +#define DEFINE_IDTENTRY_SYSVEC_SIMPLE(func) \ +static __always_inline void __##func(struct pt_regs *regs); \ + \ +__visible noinstr void func(struct pt_regs *regs) \ +{ \ + irqentry_state_t state = irqentry_enter(regs); \ + \ + instrumentation_begin(); \ + __irq_enter_raw(); \ + kvm_set_cpu_l1tf_flush_l1d(); \ + __##func(regs); \ + __irq_exit_raw(); \ + instrumentation_end(); \ + irqentry_exit_hv_cond(regs, state); \ +} \ + \ +static __always_inline void __##func(struct pt_regs *regs) +#endif /** * DECLARE_IDTENTRY_XENCB - Declare functions for XEN HV callback entry point diff --git a/arch/x86/kernel/sev.c b/arch/x86/kernel/sev.c index 2c964f7ac7dc..32d7dd0159ac 100644 --- a/arch/x86/kernel/sev.c +++ b/arch/x86/kernel/sev.c @@ -147,6 +147,10 @@ struct sev_hv_doorbell_page { struct sev_snp_runtime_data { struct sev_hv_doorbell_page hv_doorbell_page; + /* + * Indication that we are currently handling #HV events. + */ + bool hv_handling_events; }; static DEFINE_PER_CPU(struct sev_snp_runtime_data*, snp_runtime_data); @@ -202,6 +206,12 @@ static void do_exc_hv(struct pt_regs *regs) { union hv_pending_events pending_events; + /* Avoid nested entry. */ + if (this_cpu_read(snp_runtime_data)->hv_handling_events) + return; + + this_cpu_read(snp_runtime_data)->hv_handling_events = true; + while (sev_hv_pending()) { pending_events.events = xchg( &sev_snp_current_doorbell_page()->pending_events.events, @@ -236,6 +246,8 @@ static void do_exc_hv(struct pt_regs *regs) common_interrupt(regs, pending_events.vector); } } + + this_cpu_read(snp_runtime_data)->hv_handling_events = false; } static __always_inline bool on_vc_stack(struct pt_regs *regs) @@ -2533,3 +2545,25 @@ static int __init snp_init_platform_device(void) return 0; } device_initcall(snp_init_platform_device); + +noinstr void irqentry_exit_hv_cond(struct pt_regs *regs, irqentry_state_t state) +{ + /* + * Check whether this returns to user mode, if so and if + * we are currently executing the #HV handler then we don't + * want to follow the irqentry_exit_to_user_mode path as + * that can potentially cause the #HV handler to be + * preempted and rescheduled on another CPU. Rescheduled #HV + * handler on another cpu will cause interrupts to be handled + * on a different cpu than the injected one, causing + * invalid EOIs and missed/lost guest interrupts and + * corresponding hangs and/or per-cpu IRQs handled on + * non-intended cpu. + */ + if (user_mode(regs) && + this_cpu_read(snp_runtime_data)->hv_handling_events) + return; + + /* follow normal interrupt return/exit path */ + irqentry_exit(regs, state); +} From patchwork Mon Apr 3 17:44:05 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tianyu Lan X-Patchwork-Id: 13198530 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 36FA7C761AF for ; Mon, 3 Apr 2023 17:46:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232889AbjDCRqg (ORCPT ); Mon, 3 Apr 2023 13:46:36 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51104 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232600AbjDCRpz (ORCPT ); Mon, 3 Apr 2023 13:45:55 -0400 Received: from mail-pj1-x1029.google.com (mail-pj1-x1029.google.com [IPv6:2607:f8b0:4864:20::1029]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A855740C4; Mon, 3 Apr 2023 10:44:33 -0700 (PDT) Received: by mail-pj1-x1029.google.com with SMTP id mp3-20020a17090b190300b0023fcc8ce113so33394278pjb.4; Mon, 03 Apr 2023 10:44:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1680543872; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=lMejaISgZo/X14mzAcVEleMXRw+S9JCsqKPVR2Pqb9E=; b=MN5PWqJJcDeLW7Mi7FTOujqRf1HNSdLofD2KRByg6HjTFDDpA569Ig5os+InWesOcr Gu9BwpU4YuWOyhRdGJsYB4gg5WMgjptMoOACYsSbtkdOq5Px+rdh3yqGH0ebDii4ZlXQ Q5KV2JNP8TH4KJ/kpKR8yvF8f0ZdEXLiFRQ0Y0jm1oJgoiV9H7Lsp4PC+aJ53U1MfEti OoIS9TkBFM3AoKnKPgwQixLNQYDQHSAvoVWn8mp6Ki3yAvAp2kIRAcN+/hbG2TOSxZ4m aLctnjnogBaiJhmkmLLkF878etMUtUZjoGxEMgo1NuH/dt1ERAw/pN7lKh0Vx/rp8w8b n6qQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680543872; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=lMejaISgZo/X14mzAcVEleMXRw+S9JCsqKPVR2Pqb9E=; b=wb+W+StugMHXt+kiCuMi/4E7BFhINXGCyVraJ/pBmgqEiVKdDMTO9AfON9RLxcvO4X 5xgZqBlud3vFmcV7iR6cJnA1siePrdxfINpFe3KDnAjeTBmciL9Jd5T3BWJiH5cKWmb/ 6TnbTIwIVLaG48zOEJXPhG5O7zuUEx4oGswr6Z3Wt4ydi8ZmhMRu1hXD4zd6Ycfsm8vZ 77JXWutg+it7jcy7A8algRHL6UOoJaoeC86evfTj1/syCiwwfBKszD4G/plv5LlLZk98 RaSIPSVp+wIKtUOKfsIasyG/Rh3MjdsZ7uWHMwp+JKCKZfhPGmdW2fjFL9QdMB0+PmSB ktuQ== X-Gm-Message-State: AAQBX9fuiXFg6XzNCXaKUM0k+6ZuMvPeS/kKCGK7S6gvpToqhfwYJaJ4 VrKd14V3lXCerUvsbEhLdM8= X-Google-Smtp-Source: AKy350YtXShHvctrBUnf1VOJUMw1RbAjDXOMG87QyIRxfxsZNlIUqA3IijXEYlvxwureT0bkJrmQvw== X-Received: by 2002:a17:902:e847:b0:19e:d6f2:feea with SMTP id t7-20020a170902e84700b0019ed6f2feeamr46661447plg.9.1680543872252; Mon, 03 Apr 2023 10:44:32 -0700 (PDT) Received: from ubuntu-Virtual-Machine.corp.microsoft.com ([2001:4898:80e8:2:8635:6e96:35c1:c560]) by smtp.gmail.com with ESMTPSA id jj21-20020a170903049500b001a19196af48sm6883803plb.64.2023.04.03.10.44.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 03 Apr 2023 10:44:31 -0700 (PDT) From: Tianyu Lan To: luto@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, seanjc@google.com, pbonzini@redhat.com, jgross@suse.com, tiala@microsoft.com, kirill@shutemov.name, jiangshan.ljs@antgroup.com, peterz@infradead.org, ashish.kalra@amd.com, srutherford@google.com, akpm@linux-foundation.org, anshuman.khandual@arm.com, pawan.kumar.gupta@linux.intel.com, adrian.hunter@intel.com, daniel.sneddon@linux.intel.com, alexander.shishkin@linux.intel.com, sandipan.das@amd.com, ray.huang@amd.com, brijesh.singh@amd.com, michael.roth@amd.com, thomas.lendacky@amd.com, venu.busireddy@oracle.com, sterritt@google.com, tony.luck@intel.com, samitolvanen@google.com, fenghua.yu@intel.com Cc: pangupta@amd.com, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-hyperv@vger.kernel.org, linux-arch@vger.kernel.org Subject: [RFC PATCH V4 17/17] x86/sev: Remove restrict interrupt injection from SNP_FEATURES_IMPL_REQ Date: Mon, 3 Apr 2023 13:44:05 -0400 Message-Id: <20230403174406.4180472-18-ltykernel@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230403174406.4180472-1-ltykernel@gmail.com> References: <20230403174406.4180472-1-ltykernel@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Tianyu Lan Enabled restrict interrupt injection function. Remove MSR_AMD64_ SNP_RESTRICTED_INJ from SNP_FEATURES_IMPL_REQ to let kernel boot up with this function. --- arch/x86/boot/compressed/sev.c | 1 - 1 file changed, 1 deletion(-) diff --git a/arch/x86/boot/compressed/sev.c b/arch/x86/boot/compressed/sev.c index d63ad8f99f83..a5f41301a600 100644 --- a/arch/x86/boot/compressed/sev.c +++ b/arch/x86/boot/compressed/sev.c @@ -299,7 +299,6 @@ static void enforce_vmpl0(void) */ #define SNP_FEATURES_IMPL_REQ (MSR_AMD64_SNP_VTOM | \ MSR_AMD64_SNP_REFLECT_VC | \ - MSR_AMD64_SNP_RESTRICTED_INJ | \ MSR_AMD64_SNP_ALT_INJ | \ MSR_AMD64_SNP_DEBUG_SWAP | \ MSR_AMD64_SNP_VMPL_SSS | \