From patchwork Sat Nov 19 03:46:15 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tianyu Lan X-Patchwork-Id: 13049556 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D0F77C43219 for ; Sat, 19 Nov 2022 03:46:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231543AbiKSDqq (ORCPT ); Fri, 18 Nov 2022 22:46:46 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36714 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231171AbiKSDqk (ORCPT ); Fri, 18 Nov 2022 22:46:40 -0500 Received: from mail-pf1-x42a.google.com (mail-pf1-x42a.google.com [IPv6:2607:f8b0:4864:20::42a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 12013BF5BA; Fri, 18 Nov 2022 19:46:40 -0800 (PST) Received: by mail-pf1-x42a.google.com with SMTP id q9so6648003pfg.5; Fri, 18 Nov 2022 19:46:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=DP2gDnxC84ZwxSYEmb2xdHigIgTSb4Jlj1QnGwsCAFw=; b=VizoFiyC6m0+qVewF22y7z3UPlL0Ni2vLQg44N7CDbbM8xWC44Wzz15t4B3kp75117 q/kbcD8R8RLvwVmn9KJkqrHZ/kSPStVNsr4jndlK992jfN81t3oHKdAaf8trVXNIwUt4 gigRs6KKVFk2FXwtkJPdHjXZSt3QEjYXj1Z0yVDVVzflI22FH5121EDud+AwikWRW/7p dmnxmcMHcpi9xZhH+Rprv/ga9TzGoxzIQEGK7huomFGj5jtgFzQTqmEZkt3hPCQ7Vps7 mnZRj478hBuFuTtSc84OM/3JO2lewa3X5X/1Y1RyQZ2VvQDlBJkXe+CslumMiMJVwK0x yR8Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=DP2gDnxC84ZwxSYEmb2xdHigIgTSb4Jlj1QnGwsCAFw=; b=hrQsPLEVM9nBOKZZ1Pbjzyl2R/wA1c/4Fsob4D5fE/hcxak6rX7p0RmHC2XvI+N+SN QWGFGonea6slURPaHc+sRzmkkkgHG1DpEkBz8Wj7QkOXV9AExbehOEKTUFS5IQ+Gul7J iDp5vI4sXxiu6P5ERghv+l5xVzXdcURlYQI7uE2k359vwBZSY8qO8I8g3HcbTadqXY+d ojlUjZSVmpwnlru43u3M6asKRQnggvhbeM+xCC9JiII/lFMJHIT0jE8Pmm1c/Qdp7koP QWkUx6phedCHchc3FWgjh5od69DfUyv0dBYPzzSFDMIpwHxvXDdvThoSLy+kmAfyAWAy KmJA== X-Gm-Message-State: ANoB5pnu48szRAasAriB5HJjhzZBq3qZASBm3oIIzoYTFoc/NQRgrkra IgiRQi54SogIKbsjISuOt0M= X-Google-Smtp-Source: AA0mqf6EXaF7823hWGBs45o8b/pvnR4W7OZ3VExiL7yrwZVbS8Ia7qNxtQyccE2mZAEkUrUgF69C2A== X-Received: by 2002:a62:874c:0:b0:556:e951:f8de with SMTP id i73-20020a62874c000000b00556e951f8demr10830626pfe.59.1668829599447; Fri, 18 Nov 2022 19:46:39 -0800 (PST) Received: from ubuntu-Virtual-Machine.corp.microsoft.com ([2001:4898:80e8:38:f087:1794:92c5:f8f0]) by smtp.gmail.com with ESMTPSA id e5-20020a056a0000c500b005360da6b26bsm3913892pfj.159.2022.11.18.19.46.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Nov 2022 19:46:38 -0800 (PST) From: Tianyu Lan To: luto@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, seanjc@google.com, pbonzini@redhat.com, jgross@suse.com, tiala@microsoft.com, kirill@shutemov.name, jiangshan.ljs@antgroup.com, peterz@infradead.org, ashish.kalra@amd.com, srutherford@google.com, akpm@linux-foundation.org, anshuman.khandual@arm.com, pawan.kumar.gupta@linux.intel.com, adrian.hunter@intel.com, daniel.sneddon@linux.intel.com, alexander.shishkin@linux.intel.com, sandipan.das@amd.com, ray.huang@amd.com, brijesh.singh@amd.com, michael.roth@amd.com, thomas.lendacky@amd.com, venu.busireddy@oracle.com, sterritt@google.com, tony.luck@intel.com, samitolvanen@google.com, fenghua.yu@intel.com Cc: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-hyperv@vger.kernel.org, linux-arch@vger.kernel.org Subject: [RFC PATCH V2 01/18] x86/sev: Pvalidate memory gab for decompressing kernel Date: Fri, 18 Nov 2022 22:46:15 -0500 Message-Id: <20221119034633.1728632-2-ltykernel@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221119034633.1728632-1-ltykernel@gmail.com> References: <20221119034633.1728632-1-ltykernel@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Tianyu Lan Pvalidate needed pages for decompressing kernel. The E820_TYPE_RAM entry includes only validated memory. The kernel expects that the RAM entry's addr is fixed while the entry size is to be extended to cover addresses to the start of next entry. This patch increases the RAM entry size to cover all possilble memory addresses until init_size. Signed-off-by: Tianyu Lan --- arch/x86/boot/compressed/head_64.S | 8 +++ arch/x86/boot/compressed/sev.c | 84 ++++++++++++++++++++++++++++++ 2 files changed, 92 insertions(+) diff --git a/arch/x86/boot/compressed/head_64.S b/arch/x86/boot/compressed/head_64.S index d33f060900d2..818edaf5d0cf 100644 --- a/arch/x86/boot/compressed/head_64.S +++ b/arch/x86/boot/compressed/head_64.S @@ -348,6 +348,14 @@ SYM_CODE_START(startup_64) cld cli +#ifdef CONFIG_AMD_MEM_ENCRYPT + /* pvalidate memory on demand if SNP is enabled. */ + pushq %rsi + movq %rsi, %rdi + call pvalidate_for_startup_64 + popq %rsi +#endif + /* Setup data segments. */ xorl %eax, %eax movl %eax, %ds diff --git a/arch/x86/boot/compressed/sev.c b/arch/x86/boot/compressed/sev.c index 960968f8bf75..3a5a1ab16095 100644 --- a/arch/x86/boot/compressed/sev.c +++ b/arch/x86/boot/compressed/sev.c @@ -12,8 +12,10 @@ */ #include "misc.h" +#include #include #include +#include #include #include #include @@ -21,6 +23,7 @@ #include #include #include +#include #include "error.h" #include "../msr.h" @@ -117,6 +120,22 @@ static enum es_result vc_read_mem(struct es_em_ctxt *ctxt, /* Include code for early handlers */ #include "../../kernel/sev-shared.c" +/* Check SEV-SNP via MSR */ +static bool sev_snp_runtime_check(void) +{ + unsigned long low, high; + u64 val; + + asm volatile("rdmsr\n" : "=a" (low), "=d" (high) : + "c" (MSR_AMD64_SEV)); + + val = (high << 32) | low; + if (val & MSR_AMD64_SEV_SNP_ENABLED) + return true; + + return false; +} + static inline bool sev_snp_enabled(void) { return sev_status & MSR_AMD64_SEV_SNP_ENABLED; @@ -456,3 +475,68 @@ void sev_prep_identity_maps(unsigned long top_level_pgt) sev_verify_cbit(top_level_pgt); } + +static void extend_e820_on_demand(struct boot_e820_entry *e820_entry, + u64 needed_ram_end) +{ + u64 end, paddr; + unsigned long eflags; + int rc; + + if (!e820_entry) + return; + + /* Validated memory must be aligned by PAGE_SIZE. */ + end = ALIGN(e820_entry->addr + e820_entry->size, PAGE_SIZE); + if (needed_ram_end > end && e820_entry->type == E820_TYPE_RAM) { + for (paddr = end; paddr < needed_ram_end; paddr += PAGE_SIZE) { + rc = pvalidate(paddr, RMP_PG_SIZE_4K, true); + if (rc) { + error("Failed to validate address.n"); + return; + } + } + e820_entry->size = needed_ram_end - e820_entry->addr; + } +} + +/* + * Explicitly pvalidate needed pages for decompressing the kernel. + * The E820_TYPE_RAM entry includes only validated memory. The kernel + * expects that the RAM entry's addr is fixed while the entry size is to be + * extended to cover addresses to the start of next entry. + * The function increases the RAM entry size to cover all possible memory + * addresses until init_size. + * For example, init_end = 0x4000000, + * [RAM: 0x0 - 0x0], M[RAM: 0x0 - 0xa0000] + * [RSVD: 0xa0000 - 0x10000] [RSVD: 0xa0000 - 0x10000] + * [ACPI: 0x10000 - 0x20000] ==> [ACPI: 0x10000 - 0x20000] + * [RSVD: 0x800000 - 0x900000] [RSVD: 0x800000 - 0x900000] + * [RAM: 0x1000000 - 0x2000000] M[RAM: 0x1000000 - 0x2001000] + * [RAM: 0x2001000 - 0x2007000] M[RAM: 0x2001000 - 0x4000000] + * Other RAM memory after init_end is pvalidated by ms_hyperv_init_platform + */ +__visible void pvalidate_for_startup_64(struct boot_params *boot_params) +{ + struct boot_e820_entry *e820_entry; + u64 init_end = + boot_params->hdr.pref_address + boot_params->hdr.init_size; + u8 i, nr_entries = boot_params->e820_entries; + u64 needed_end; + + if (!sev_snp_runtime_check()) + return; + + for (i = 0; i < nr_entries; ++i) { + /* Pvalidate memory holes in e820 RAM entries. */ + e820_entry = &boot_params->e820_table[i]; + if (i < nr_entries - 1) { + needed_end = boot_params->e820_table[i + 1].addr; + if (needed_end < e820_entry->addr) + error("e820 table is not sorted.\n"); + } else { + needed_end = init_end; + } + extend_e820_on_demand(e820_entry, needed_end); + } +} From patchwork Sat Nov 19 03:46:16 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tianyu Lan X-Patchwork-Id: 13049557 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 89B21C4167D for ; Sat, 19 Nov 2022 03:47:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231877AbiKSDq6 (ORCPT ); Fri, 18 Nov 2022 22:46:58 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36730 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231297AbiKSDqm (ORCPT ); Fri, 18 Nov 2022 22:46:42 -0500 Received: from mail-pg1-x52d.google.com (mail-pg1-x52d.google.com [IPv6:2607:f8b0:4864:20::52d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id ED154BF5BC; Fri, 18 Nov 2022 19:46:41 -0800 (PST) Received: by mail-pg1-x52d.google.com with SMTP id b62so6658386pgc.0; Fri, 18 Nov 2022 19:46:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=DTm6TpUl+FGxnM+t+ZkzqsU9392mhy5cn6Qp9D3tMP0=; b=FQ3RUShhbQAmafQLUbeCf2qo6LowXIyhJnOvf8A9T10NHQXV6Sq63hWSlLW5Zn1v9O lJaYn78rKGdCVXO9ZUcNtc8BDourqLnganEm8UBr8xLilSVgWvKHE9BtF26VRmTgz+zm b/lq+VRM1PDlBuSPszoCJ+fygLYX9SGAGpx9I+Jc1D2pD7B1CJdIzLD/f4v+WNimEjan 73zEeFZcFHxZFnhaEm4QP9h1VRt+CYBZuAibbQUjwPqTMVFvSgKIPwuJdeN+Nq+0zP4/ VvlWAL8PvAAbHJcATGIfqpQO8/0Zgcb10os0GTmfi3MzOMX9Jg/5UrA5Dysatm98GoHS 3dYQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=DTm6TpUl+FGxnM+t+ZkzqsU9392mhy5cn6Qp9D3tMP0=; b=FtMTq9KXasbm8ANZFIeTJShXdBUm9jEP03JMLFBFFivuL6YndAh6WwQ+JCdd4KEvaE swmZLzX3bSTYQEF9O4XCYswF417tVPMdxToMl2+V1Ckk6KD5plgLq/INnZ8KSc/os4lE Y0/roJj2Is5fGJlccIrV5ugXKlzlTmXNEq+O25puydozeqV8GvvNKVX31vKWiIYcOkd5 Cpnm1Y0OT+OcewqBzyHNKFGlzMWfNKKg9Cl2iCoGOgk8HhcJG8uxQ5grsIPcyrV/Lutw GCcZs15UIAiXxxO5vcWPm+TcJYA0m0k8drSoef2UYPK4rqqSM0FY8LzV2GR9/GC58JcP DgVw== X-Gm-Message-State: ANoB5pnNpa0PVuwTAJ/Di34gKfu6LpwHcovBDJP2gaAbGkLdnxF3YlFg D4Fm+4PTKHYcPhhxh/DXrbE= X-Google-Smtp-Source: AA0mqf4WZ3NgfARIEipwLiBHu7/1+/tfKIxfmtmEYszYgl01Kfq4fztmGsjmDAF4acSgwZNpP2YTiw== X-Received: by 2002:aa7:9508:0:b0:56d:3180:e885 with SMTP id b8-20020aa79508000000b0056d3180e885mr10812364pfp.82.1668829601461; Fri, 18 Nov 2022 19:46:41 -0800 (PST) Received: from ubuntu-Virtual-Machine.corp.microsoft.com ([2001:4898:80e8:38:f087:1794:92c5:f8f0]) by smtp.gmail.com with ESMTPSA id e5-20020a056a0000c500b005360da6b26bsm3913892pfj.159.2022.11.18.19.46.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Nov 2022 19:46:40 -0800 (PST) From: Tianyu Lan To: luto@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, seanjc@google.com, pbonzini@redhat.com, jgross@suse.com, tiala@microsoft.com, kirill@shutemov.name, jiangshan.ljs@antgroup.com, peterz@infradead.org, ashish.kalra@amd.com, srutherford@google.com, akpm@linux-foundation.org, anshuman.khandual@arm.com, pawan.kumar.gupta@linux.intel.com, adrian.hunter@intel.com, daniel.sneddon@linux.intel.com, alexander.shishkin@linux.intel.com, sandipan.das@amd.com, ray.huang@amd.com, brijesh.singh@amd.com, michael.roth@amd.com, thomas.lendacky@amd.com, venu.busireddy@oracle.com, sterritt@google.com, tony.luck@intel.com, samitolvanen@google.com, fenghua.yu@intel.com Cc: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-hyperv@vger.kernel.org, linux-arch@vger.kernel.org Subject: [RFC PATCH V2 02/18] x86/hyperv: Add sev-snp enlightened guest specific config Date: Fri, 18 Nov 2022 22:46:16 -0500 Message-Id: <20221119034633.1728632-3-ltykernel@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221119034633.1728632-1-ltykernel@gmail.com> References: <20221119034633.1728632-1-ltykernel@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Tianyu Lan Introduce static key isolation_type_en_snp for enlightened guest check and add some specific options in ms_hyperv_init_ platform(). Signed-off-by: Tianyu Lan --- arch/x86/hyperv/ivm.c | 12 +++++++++++- arch/x86/include/asm/mshyperv.h | 2 ++ arch/x86/kernel/cpu/mshyperv.c | 29 ++++++++++++++++++++++++----- drivers/hv/hv_common.c | 7 +++++++ 4 files changed, 44 insertions(+), 6 deletions(-) diff --git a/arch/x86/hyperv/ivm.c b/arch/x86/hyperv/ivm.c index 1dbcbd9da74d..e9c30dad3419 100644 --- a/arch/x86/hyperv/ivm.c +++ b/arch/x86/hyperv/ivm.c @@ -259,10 +259,20 @@ bool hv_is_isolation_supported(void) } DEFINE_STATIC_KEY_FALSE(isolation_type_snp); +DEFINE_STATIC_KEY_FALSE(isolation_type_en_snp); + +/* + * hv_isolation_type_en_snp - Check system runs in the AMD SEV-SNP based + * isolation enlightened VM. + */ +bool hv_isolation_type_en_snp(void) +{ + return static_branch_unlikely(&isolation_type_en_snp); +} /* * hv_isolation_type_snp - Check system runs in the AMD SEV-SNP based - * isolation VM. + * isolation VM with vTOM support. */ bool hv_isolation_type_snp(void) { diff --git a/arch/x86/include/asm/mshyperv.h b/arch/x86/include/asm/mshyperv.h index 61f0c206bff0..9b8c3f638845 100644 --- a/arch/x86/include/asm/mshyperv.h +++ b/arch/x86/include/asm/mshyperv.h @@ -14,6 +14,7 @@ union hv_ghcb; DECLARE_STATIC_KEY_FALSE(isolation_type_snp); +DECLARE_STATIC_KEY_FALSE(isolation_type_en_snp); typedef int (*hyperv_fill_flush_list_func)( struct hv_guest_mapping_flush_list *flush, @@ -32,6 +33,7 @@ extern u64 hv_current_partition_id; extern union hv_ghcb * __percpu *hv_ghcb_pg; +extern bool hv_isolation_type_en_snp(void); int hv_call_deposit_pages(int node, u64 partition_id, u32 num_pages); int hv_call_add_logical_proc(int node, u32 lp_index, u32 acpi_id); int hv_call_create_vp(int node, u64 partition_id, u32 vp_index, u32 flags); diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c index 831613959a92..2ea4f21c6172 100644 --- a/arch/x86/kernel/cpu/mshyperv.c +++ b/arch/x86/kernel/cpu/mshyperv.c @@ -273,6 +273,21 @@ static void __init ms_hyperv_init_platform(void) ms_hyperv.misc_features = cpuid_edx(HYPERV_CPUID_FEATURES); ms_hyperv.hints = cpuid_eax(HYPERV_CPUID_ENLIGHTMENT_INFO); + /* + * Add custom configuration for SEV-SNP Enlightened guest + */ + if (cc_platform_has(CC_ATTR_GUEST_SEV_SNP)) { + ms_hyperv.features |= HV_ACCESS_FREQUENCY_MSRS; + ms_hyperv.misc_features |= HV_FEATURE_FREQUENCY_MSRS_AVAILABLE; + ms_hyperv.misc_features &= ~HV_FEATURE_GUEST_CRASH_MSR_AVAILABLE; + ms_hyperv.hints |= HV_DEPRECATING_AEOI_RECOMMENDED; + ms_hyperv.hints |= HV_X64_APIC_ACCESS_RECOMMENDED; + ms_hyperv.hints |= HV_X64_CLUSTER_IPI_RECOMMENDED; + } + + pr_info("Hyper-V: enlightment features 0x%x, hints 0x%x, misc 0x%x\n", + ms_hyperv.features, ms_hyperv.hints, ms_hyperv.misc_features); + hv_max_functions_eax = cpuid_eax(HYPERV_CPUID_VENDOR_AND_MAX_FUNCTIONS); pr_info("Hyper-V: privilege flags low 0x%x, high 0x%x, hints 0x%x, misc 0x%x\n", @@ -328,18 +343,22 @@ static void __init ms_hyperv_init_platform(void) ms_hyperv.shared_gpa_boundary = BIT_ULL(ms_hyperv.shared_gpa_boundary_bits); - pr_info("Hyper-V: Isolation Config: Group A 0x%x, Group B 0x%x\n", - ms_hyperv.isolation_config_a, ms_hyperv.isolation_config_b); - - if (hv_get_isolation_type() == HV_ISOLATION_TYPE_SNP) { + if (cc_platform_has(CC_ATTR_GUEST_SEV_SNP)) { + static_branch_enable(&isolation_type_en_snp); + } else if (hv_get_isolation_type() == HV_ISOLATION_TYPE_SNP) { static_branch_enable(&isolation_type_snp); #ifdef CONFIG_SWIOTLB swiotlb_unencrypted_base = ms_hyperv.shared_gpa_boundary; #endif } + + pr_info("Hyper-V: Isolation Config: Group A 0x%x, Group B 0x%x\n", + ms_hyperv.isolation_config_a, ms_hyperv.isolation_config_b); + /* Isolation VMs are unenlightened SEV-based VMs, thus this check: */ if (IS_ENABLED(CONFIG_AMD_MEM_ENCRYPT)) { - if (hv_get_isolation_type() != HV_ISOLATION_TYPE_NONE) + if (hv_get_isolation_type() != HV_ISOLATION_TYPE_NONE + && !cc_platform_has(CC_ATTR_GUEST_SEV_SNP)) cc_set_vendor(CC_VENDOR_HYPERV); } } diff --git a/drivers/hv/hv_common.c b/drivers/hv/hv_common.c index ae68298c0dca..2c6602571c47 100644 --- a/drivers/hv/hv_common.c +++ b/drivers/hv/hv_common.c @@ -268,6 +268,13 @@ bool __weak hv_isolation_type_snp(void) } EXPORT_SYMBOL_GPL(hv_isolation_type_snp); +bool __weak hv_isolation_type_en_snp(void) +{ + return false; +} +EXPORT_SYMBOL_GPL(hv_isolation_type_en_snp); + + void __weak hv_setup_vmbus_handler(void (*handler)(void)) { } From patchwork Sat Nov 19 03:46:17 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tianyu Lan X-Patchwork-Id: 13049558 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 98C2BC4167B for ; Sat, 19 Nov 2022 03:47:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229935AbiKSDrD (ORCPT ); Fri, 18 Nov 2022 22:47:03 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36748 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231483AbiKSDqo (ORCPT ); Fri, 18 Nov 2022 22:46:44 -0500 Received: from mail-pg1-x532.google.com (mail-pg1-x532.google.com [IPv6:2607:f8b0:4864:20::532]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 21A6ABF803; Fri, 18 Nov 2022 19:46:43 -0800 (PST) Received: by mail-pg1-x532.google.com with SMTP id v3so6617132pgh.4; Fri, 18 Nov 2022 19:46:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=0l9afkkKeSOzI6mzq1htMqRSlzdNXqGpUKUcuz2i9ek=; b=qE2HYPAuJ52VU6dEZOdfLmCANGmCXWcs45kfZvjcuh8A7Vy3KgZQY0ydGY0R+OE3ci 0Z9JXSwEvMHH+EV78Y2ljOhU0hm9mvG3huO30M8+m0BW7i4vg06gBz59D7/Oj/sdWg27 dE2EwofvGxhgCDMAagxzQmCQdIXH7D44YUVFEmVNk6zNFqMIbBLRDQwgdJRmz5it8LP+ CHsMuvAR8m2xjZzJarHYpYWOa8cuQHPlgiTMBfz/dBR17MD0vC5Bv9wDI9hOofgAN4TJ mOgtjWDdGSeYz1oWmMI42WK0zI86yTyCqi7MtlPtLsKeMJIpZJKZi+n5xYE8SVh7EThR /bAg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=0l9afkkKeSOzI6mzq1htMqRSlzdNXqGpUKUcuz2i9ek=; b=8FV2t0jwnMWu/L5/fqm2SxDjA0xqdZ5SivuOFLKxvhRQilUm4kZjzBVn+VFqQnqiVZ SzXlTBxcCDj26Khdgwoh2qPJAVNwJV64pPux5VqRyv/XDP9RQsjWcJneiNRZ48tlUn+s 9AJpfcm44A3nQLJjnmDWt08o/1KGHdMa4uYjyUg8igTC8RwSuH6nMDW3nw1H9Hou1hlh 3XS/NGJTfbEsEQkE3PjHyRF2wzs4gS3kt3wg4snkuMPiRIhdVC7nF0d0Bd393V/tJtak 7EYp7yptQiQ2LtduQC4UYPxMBfikozQ8oJJxUfTqKuy3/VuCI+vhckjPVwsMU2Qa0i5M O9oA== X-Gm-Message-State: ANoB5pln0BvB9akNG7Y6GpcRVA8dZJE5m8B65WMY4TMgS8Dz4W5kogNr IqgBndci57M9/ogPfh2arYE= X-Google-Smtp-Source: AA0mqf7UDVjSG+tLndDVGuKUx7C5w+cTSrsMn+v0cthIG7y8DQpEL+oKgXf10/A+TKGKUcFV3Gj2qQ== X-Received: by 2002:aa7:8054:0:b0:56c:4303:a93d with SMTP id y20-20020aa78054000000b0056c4303a93dmr10662237pfm.73.1668829602636; Fri, 18 Nov 2022 19:46:42 -0800 (PST) Received: from ubuntu-Virtual-Machine.corp.microsoft.com ([2001:4898:80e8:38:f087:1794:92c5:f8f0]) by smtp.gmail.com with ESMTPSA id e5-20020a056a0000c500b005360da6b26bsm3913892pfj.159.2022.11.18.19.46.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Nov 2022 19:46:42 -0800 (PST) From: Tianyu Lan To: luto@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, seanjc@google.com, pbonzini@redhat.com, jgross@suse.com, tiala@microsoft.com, kirill@shutemov.name, jiangshan.ljs@antgroup.com, peterz@infradead.org, ashish.kalra@amd.com, srutherford@google.com, akpm@linux-foundation.org, anshuman.khandual@arm.com, pawan.kumar.gupta@linux.intel.com, adrian.hunter@intel.com, daniel.sneddon@linux.intel.com, alexander.shishkin@linux.intel.com, sandipan.das@amd.com, ray.huang@amd.com, brijesh.singh@amd.com, michael.roth@amd.com, thomas.lendacky@amd.com, venu.busireddy@oracle.com, sterritt@google.com, tony.luck@intel.com, samitolvanen@google.com, fenghua.yu@intel.com Cc: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-hyperv@vger.kernel.org, linux-arch@vger.kernel.org Subject: [RFC PATCH V2 03/18] x86/hyperv: apic change for sev-snp enlightened guest Date: Fri, 18 Nov 2022 22:46:17 -0500 Message-Id: <20221119034633.1728632-4-ltykernel@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221119034633.1728632-1-ltykernel@gmail.com> References: <20221119034633.1728632-1-ltykernel@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Tianyu Lan Hyperv sev-snp enlightened guest doesn't support x2apic and apic page read/write operation. Bypass these requests. ipi request maybe returned with timeout error code and add retry mechanism. Signed-off-by: Tianyu Lan --- arch/x86/hyperv/hv_apic.c | 79 ++++++++++++++++++++++++------- include/asm-generic/hyperv-tlfs.h | 1 + 2 files changed, 63 insertions(+), 17 deletions(-) diff --git a/arch/x86/hyperv/hv_apic.c b/arch/x86/hyperv/hv_apic.c index fb8b2c088681..214354d20833 100644 --- a/arch/x86/hyperv/hv_apic.c +++ b/arch/x86/hyperv/hv_apic.c @@ -66,9 +66,15 @@ static u32 hv_apic_read(u32 reg) rdmsr(HV_X64_MSR_TPR, reg_val, hi); (void)hi; return reg_val; - + case APIC_ID: + if (hv_isolation_type_en_snp()) + return smp_processor_id(); + fallthrough; default: - return native_apic_mem_read(reg); + if (!hv_isolation_type_en_snp()) + return native_apic_mem_read(reg); + else + return 0; } } @@ -82,7 +88,8 @@ static void hv_apic_write(u32 reg, u32 val) wrmsr(HV_X64_MSR_TPR, val, 0); break; default: - native_apic_mem_write(reg, val); + if (!hv_isolation_type_en_snp()) + native_apic_mem_write(reg, val); } } @@ -106,6 +113,7 @@ static bool __send_ipi_mask_ex(const struct cpumask *mask, int vector, struct hv_send_ipi_ex *ipi_arg; unsigned long flags; int nr_bank = 0; + int retry = 5; u64 status = HV_STATUS_INVALID_PARAMETER; if (!(ms_hyperv.hints & HV_X64_EX_PROCESSOR_MASKS_RECOMMENDED)) @@ -144,8 +152,10 @@ static bool __send_ipi_mask_ex(const struct cpumask *mask, int vector, ipi_arg->vp_set.format = HV_GENERIC_SET_ALL; } - status = hv_do_rep_hypercall(HVCALL_SEND_IPI_EX, 0, nr_bank, + do { + status = hv_do_rep_hypercall(HVCALL_SEND_IPI_EX, 0, nr_bank, ipi_arg, NULL); + } while (status == HV_STATUS_TIME_OUT && retry--); ipi_mask_ex_done: local_irq_restore(flags); @@ -159,6 +169,7 @@ static bool __send_ipi_mask(const struct cpumask *mask, int vector, struct hv_send_ipi ipi_arg; u64 status; unsigned int weight; + int retry = 5; trace_hyperv_send_ipi_mask(mask, vector); @@ -212,8 +223,11 @@ static bool __send_ipi_mask(const struct cpumask *mask, int vector, __set_bit(vcpu, (unsigned long *)&ipi_arg.cpu_mask); } - status = hv_do_fast_hypercall16(HVCALL_SEND_IPI, ipi_arg.vector, - ipi_arg.cpu_mask); + do { + status = hv_do_fast_hypercall16(HVCALL_SEND_IPI, ipi_arg.vector, + ipi_arg.cpu_mask); + } while (status == HV_STATUS_TIME_OUT && retry--); + return hv_result_success(status); do_ex_hypercall: @@ -224,6 +238,7 @@ static bool __send_ipi_one(int cpu, int vector) { int vp = hv_cpu_number_to_vp_number(cpu); u64 status; + int retry = 5; trace_hyperv_send_ipi_one(cpu, vector); @@ -236,26 +251,48 @@ static bool __send_ipi_one(int cpu, int vector) if (vp >= 64) return __send_ipi_mask_ex(cpumask_of(cpu), vector, false); - status = hv_do_fast_hypercall16(HVCALL_SEND_IPI, vector, BIT_ULL(vp)); + do { + status = hv_do_fast_hypercall16(HVCALL_SEND_IPI, vector, BIT_ULL(vp)); + } while (status == HV_STATUS_TIME_OUT || retry--); + return hv_result_success(status); } static void hv_send_ipi(int cpu, int vector) { - if (!__send_ipi_one(cpu, vector)) - orig_apic.send_IPI(cpu, vector); + if (!__send_ipi_one(cpu, vector)) { + if (!hv_isolation_type_en_snp()) + orig_apic.send_IPI(cpu, vector); + else + WARN_ON_ONCE(1); + } } static void hv_send_ipi_mask(const struct cpumask *mask, int vector) { - if (!__send_ipi_mask(mask, vector, false)) - orig_apic.send_IPI_mask(mask, vector); + if (!__send_ipi_mask(mask, vector, false)) { + if (!hv_isolation_type_en_snp()) + orig_apic.send_IPI_mask(mask, vector); + else + WARN_ON_ONCE(1); + } } static void hv_send_ipi_mask_allbutself(const struct cpumask *mask, int vector) { - if (!__send_ipi_mask(mask, vector, true)) - orig_apic.send_IPI_mask_allbutself(mask, vector); + unsigned int this_cpu = smp_processor_id(); + struct cpumask new_mask; + const struct cpumask *local_mask; + + cpumask_copy(&new_mask, mask); + cpumask_clear_cpu(this_cpu, &new_mask); + local_mask = &new_mask; + if (!__send_ipi_mask(local_mask, vector, true)) { + if (!hv_isolation_type_en_snp()) + orig_apic.send_IPI_mask_allbutself(mask, vector); + else + WARN_ON_ONCE(1); + } } static void hv_send_ipi_allbutself(int vector) @@ -265,14 +302,22 @@ static void hv_send_ipi_allbutself(int vector) static void hv_send_ipi_all(int vector) { - if (!__send_ipi_mask(cpu_online_mask, vector, false)) - orig_apic.send_IPI_all(vector); + if (!__send_ipi_mask(cpu_online_mask, vector, false)) { + if (!hv_isolation_type_en_snp()) + orig_apic.send_IPI_all(vector); + else + WARN_ON_ONCE(1); + } } static void hv_send_ipi_self(int vector) { - if (!__send_ipi_one(smp_processor_id(), vector)) - orig_apic.send_IPI_self(vector); + if (!__send_ipi_one(smp_processor_id(), vector)) { + if (!hv_isolation_type_en_snp()) + orig_apic.send_IPI_self(vector); + else + WARN_ON_ONCE(1); + } } void __init hv_apic_init(void) diff --git a/include/asm-generic/hyperv-tlfs.h b/include/asm-generic/hyperv-tlfs.h index fdce7a4cfc6f..6e2a090e2649 100644 --- a/include/asm-generic/hyperv-tlfs.h +++ b/include/asm-generic/hyperv-tlfs.h @@ -208,6 +208,7 @@ enum HV_GENERIC_SET_FORMAT { #define HV_STATUS_INVALID_PORT_ID 17 #define HV_STATUS_INVALID_CONNECTION_ID 18 #define HV_STATUS_INSUFFICIENT_BUFFERS 19 +#define HV_STATUS_TIME_OUT 0x78 /* * The Hyper-V TimeRefCount register and the TSC From patchwork Sat Nov 19 03:46:18 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tianyu Lan X-Patchwork-Id: 13049559 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B316AC4332F for ; Sat, 19 Nov 2022 03:47:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232251AbiKSDrF (ORCPT ); Fri, 18 Nov 2022 22:47:05 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36766 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230472AbiKSDqp (ORCPT ); Fri, 18 Nov 2022 22:46:45 -0500 Received: from mail-pj1-x102a.google.com (mail-pj1-x102a.google.com [IPv6:2607:f8b0:4864:20::102a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 43958BF805; Fri, 18 Nov 2022 19:46:44 -0800 (PST) Received: by mail-pj1-x102a.google.com with SMTP id a22-20020a17090a6d9600b0021896eb5554so996093pjk.1; Fri, 18 Nov 2022 19:46:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=1OVwfeSJzfcwy29SuQO6DhipoyIsaZzVhoPd/85YD4c=; b=LD3AT/BYpwC/HqxCDdRm//s50GtDWZorc/UFGpuAPu2xhi34RQwp7GRQMvFxTdVWxu bowWb+WTPbjeO6ovgkk0OQiz1021DOUHT+z3MrH8Cp/evCzdkACdEjNxS5TFmn01h/Gf bjifJRtK6TTofF0a3iOJyICNN87CncDHEk9RbcOY8PsXhAwPBGaQL17oxZLxG2nTrDw3 C9Kwp6fkFT6bZrS8ifcIKXWIl0bVtrxRmRnRPMJWOsO6TqVZo/WFYP8dLZGgZ3XBcn9M xzrmErYHuy5zG9qpTydaAxyiNngOtPd+mLCaj1zGNtOTliv8a52uUeSp1Dsn2z1CVkw5 wByg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=1OVwfeSJzfcwy29SuQO6DhipoyIsaZzVhoPd/85YD4c=; b=k6ohULoEMtQy3ANL5uwaeB9gx7CpR2zHRRGnpUdL9Y8AmQR19BoAgIx8c/12yVOCKQ +l23LXeuZeCSKfbDelUhSLaVCUH7nYkZX4WmDXBuHUj1E86nYSbtEij5tfmlUhXA/Chk xON4in3HnRaex6eWvLfgt6w5JN3XCHYizIYxAt5aVsB1awNRthUYfm9z8MMMPJF0DJlV AplsnIIN23SfoCVgnVvdsAgNfqrguoj0Jvx2WNxZH41jTkT51zQFFmAv3AkK4gR4EP5U qgV/67gqD6BixOjkkauSABHf9VhDIPQuRnp8/C4RD5VNHkiPUQiMwmJ7vNqoA3omyWjB MYXQ== X-Gm-Message-State: ANoB5pkwZiUm4WdG/cGIGkASrf/aDx6S5JvK1dT5M4+xnEm90vdgzW/K tPEukkhbJX030qVZ6OWwP3Q= X-Google-Smtp-Source: AA0mqf7hiJVtgg1DhDW42yAAa9DrEzz1sqT6RtyJ+nXa4uL21WoINwaxnUnf1gQokdJat9PnbUylWQ== X-Received: by 2002:a17:90a:d50c:b0:218:722e:cac7 with SMTP id t12-20020a17090ad50c00b00218722ecac7mr10860624pju.47.1668829603776; Fri, 18 Nov 2022 19:46:43 -0800 (PST) Received: from ubuntu-Virtual-Machine.corp.microsoft.com ([2001:4898:80e8:38:f087:1794:92c5:f8f0]) by smtp.gmail.com with ESMTPSA id e5-20020a056a0000c500b005360da6b26bsm3913892pfj.159.2022.11.18.19.46.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Nov 2022 19:46:43 -0800 (PST) From: Tianyu Lan To: luto@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, seanjc@google.com, pbonzini@redhat.com, jgross@suse.com, tiala@microsoft.com, kirill@shutemov.name, jiangshan.ljs@antgroup.com, peterz@infradead.org, ashish.kalra@amd.com, srutherford@google.com, akpm@linux-foundation.org, anshuman.khandual@arm.com, pawan.kumar.gupta@linux.intel.com, adrian.hunter@intel.com, daniel.sneddon@linux.intel.com, alexander.shishkin@linux.intel.com, sandipan.das@amd.com, ray.huang@amd.com, brijesh.singh@amd.com, michael.roth@amd.com, thomas.lendacky@amd.com, venu.busireddy@oracle.com, sterritt@google.com, tony.luck@intel.com, samitolvanen@google.com, fenghua.yu@intel.com Cc: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-hyperv@vger.kernel.org, linux-arch@vger.kernel.org Subject: [RFC PATCH V2 04/18] x86/hyperv: Decrypt hv vp assist page in sev-snp enlightened guest Date: Fri, 18 Nov 2022 22:46:18 -0500 Message-Id: <20221119034633.1728632-5-ltykernel@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221119034633.1728632-1-ltykernel@gmail.com> References: <20221119034633.1728632-1-ltykernel@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Tianyu Lan hv vp assist page is shared between sev snp guest and hyperv. Decrypt the page when use it. Signed-off-by: Tianyu Lan --- arch/x86/hyperv/hv_init.c | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/arch/x86/hyperv/hv_init.c b/arch/x86/hyperv/hv_init.c index 29774126e931..4600c5941957 100644 --- a/arch/x86/hyperv/hv_init.c +++ b/arch/x86/hyperv/hv_init.c @@ -30,6 +30,7 @@ #include #include #include +#include int hyperv_init_cpuhp; u64 hv_current_partition_id = ~0ull; @@ -112,6 +113,11 @@ static int hv_cpu_init(unsigned int cpu) } WARN_ON(!(*hvp)); if (*hvp) { + if (hv_isolation_type_en_snp()) { + WARN_ON_ONCE(set_memory_decrypted((unsigned long)(*hvp), 1) != 0); + memset(*hvp, 0, PAGE_SIZE); + } + msr.enable = 1; wrmsrl(HV_X64_MSR_VP_ASSIST_PAGE, msr.as_uint64); } @@ -228,6 +234,12 @@ static int hv_cpu_die(unsigned int cpu) if (hv_vp_assist_page && hv_vp_assist_page[cpu]) { union hv_vp_assist_msr_contents msr = { 0 }; + + if (hv_isolation_type_en_snp()) + WARN_ON_ONCE(set_memory_encrypted( + (unsigned long)hv_vp_assist_page[cpu], + 1) != 0); + if (hv_root_partition) { /* * For root partition the VP assist page is mapped to From patchwork Sat Nov 19 03:46:19 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tianyu Lan X-Patchwork-Id: 13049564 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B4421C43219 for ; Sat, 19 Nov 2022 03:48:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232213AbiKSDrj (ORCPT ); Fri, 18 Nov 2022 22:47:39 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37080 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230058AbiKSDq5 (ORCPT ); Fri, 18 Nov 2022 22:46:57 -0500 Received: from mail-pg1-x52d.google.com (mail-pg1-x52d.google.com [IPv6:2607:f8b0:4864:20::52d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B1B8ABF80B; Fri, 18 Nov 2022 19:46:45 -0800 (PST) Received: by mail-pg1-x52d.google.com with SMTP id b62so6658466pgc.0; Fri, 18 Nov 2022 19:46:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ZHdN7biDTMCbZsH86bq9wEVAHh9Jk5WDAS/BJmNojPQ=; b=TONekkcAfBmXRry37Wbi51cr2toA2jiBUJMasTmc7l2fvIja1wdaN2X24ZFp2YVWpn steUrrkufUn0wEzm57h+/JnRnmvpOChMAFwykT8+bgus9Kj3Hm20/b96RiQ/kyf8WeS/ G3aUroC/Aq7tA5KejVGVa2bztjFmKadFznW/m/goy045L8PHEnEJaOznmLBunZd2zL+J AH/ZONaDjpHjlJPcUSlykncMcLEnDOijSC/TYynTSJdphSB25OW8JuIA8gme1eAL7yXn uI8QDS75KwmAYCGnJs533NogBgneAQrRmcio34vIt0Op30bIh3icBuOYVDnScTYy1jiI H3pg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ZHdN7biDTMCbZsH86bq9wEVAHh9Jk5WDAS/BJmNojPQ=; b=sVfHsa3o3FTXqskn9v9ITGXO3M5w6FQCGrPqIktf/M5qk1bhuMLlxHspzUfAJ9EthF tK7XHG4wvRfC2XfEUvz7H3JZ/23k7GupFJYbs7nhSLjkEMx24ElDNFs++FKLRT4iVy8C 0G82neDxSBdscGffl+YVYGH4xGSRKNQLMrLCMtCExaYilt6HZKLx8ip8IC4pxOvhA781 LzgD0AADHN9b6gBECI2l2LlA7TSzhT9guWdpnVzFGLMfmac4VkIAvFza6vMuyNZpzHNA +MjdE/vQuGm7OlcngHHXQF16P6538MZ/sXBz6oi/op+YoelwanuWz+7V1H1skeU1dDx8 7U8g== X-Gm-Message-State: ANoB5pmdrpIqD1lp0HLJ/SYNZsqftNTa0Cc7AiYrqNBOdO8FrSosP6Or bfSZeDEZoPgKs1MEC2rKbs6kPkQQjlnDsw== X-Google-Smtp-Source: AA0mqf4W9C8O5R6U3ygETad3CLbfYrIKVfeSRJhpnQRqtB9YGtpcZtnOxW+pV8nSo49Drl7yTT0g+Q== X-Received: by 2002:a63:5502:0:b0:470:3e8:6f25 with SMTP id j2-20020a635502000000b0047003e86f25mr9319719pgb.294.1668829605426; Fri, 18 Nov 2022 19:46:45 -0800 (PST) Received: from ubuntu-Virtual-Machine.corp.microsoft.com ([2001:4898:80e8:38:f087:1794:92c5:f8f0]) by smtp.gmail.com with ESMTPSA id e5-20020a056a0000c500b005360da6b26bsm3913892pfj.159.2022.11.18.19.46.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Nov 2022 19:46:44 -0800 (PST) From: Tianyu Lan To: luto@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, seanjc@google.com, pbonzini@redhat.com, jgross@suse.com, tiala@microsoft.com, kirill@shutemov.name, jiangshan.ljs@antgroup.com, peterz@infradead.org, ashish.kalra@amd.com, srutherford@google.com, akpm@linux-foundation.org, anshuman.khandual@arm.com, pawan.kumar.gupta@linux.intel.com, adrian.hunter@intel.com, daniel.sneddon@linux.intel.com, alexander.shishkin@linux.intel.com, sandipan.das@amd.com, ray.huang@amd.com, brijesh.singh@amd.com, michael.roth@amd.com, thomas.lendacky@amd.com, venu.busireddy@oracle.com, sterritt@google.com, tony.luck@intel.com, samitolvanen@google.com, fenghua.yu@intel.com Cc: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-hyperv@vger.kernel.org, linux-arch@vger.kernel.org Subject: [RFC PATCH V2 05/18] x86/hyperv: Get Virtual Trust Level via hvcall Date: Fri, 18 Nov 2022 22:46:19 -0500 Message-Id: <20221119034633.1728632-6-ltykernel@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221119034633.1728632-1-ltykernel@gmail.com> References: <20221119034633.1728632-1-ltykernel@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Tianyu Lan sev-snp guest provides vtl(Virtual Trust Level) and get it from hyperv hvcall via HVCALL_GET_VP_REGISTERS. Signed-off-by: Tianyu Lan --- arch/x86/hyperv/hv_init.c | 35 ++++++++++++++++++++++++++++++++++ include/asm-generic/mshyperv.h | 2 ++ 2 files changed, 37 insertions(+) diff --git a/arch/x86/hyperv/hv_init.c b/arch/x86/hyperv/hv_init.c index 4600c5941957..5b919d4d24c0 100644 --- a/arch/x86/hyperv/hv_init.c +++ b/arch/x86/hyperv/hv_init.c @@ -390,6 +390,39 @@ static void __init hv_get_partition_id(void) local_irq_restore(flags); } +static u8 __init get_current_vtl(void) +{ + u64 control = ((u64)1 << HV_HYPERCALL_REP_COMP_OFFSET) | HVCALL_GET_VP_REGISTERS; + struct hv_get_vp_registers_input *input = NULL; + struct hv_get_vp_registers_output *output = NULL; + u8 vtl = 0; + int ret; + unsigned long flags; + + local_irq_save(flags); + input = *(struct hv_get_vp_registers_input **)this_cpu_ptr(hyperv_pcpu_input_arg); + output = (struct hv_get_vp_registers_output *)input; + if (!input || !output) { + pr_err("Hyper-V: cannot allocate a shared page!"); + goto done; + } + + memset(input, 0, sizeof(*input) + sizeof(input->element[0])); + input->header.partitionid = HV_PARTITION_ID_SELF; + input->header.inputvtl = 0; + input->element[0].name0 = 0x000D0003; + + ret = hv_do_hypercall(control, input, output); + if (ret == 0) + vtl = output->as64.low & 0xf; + else + pr_err("Hyper-V: failed to get the current VTL!"); + local_irq_restore(flags); + +done: + return vtl; +} + /* * This function is to be invoked early in the boot sequence after the * hypervisor has been detected. @@ -527,6 +560,8 @@ void __init hyperv_init(void) if (hv_is_isolation_supported()) swiotlb_update_mem_attributes(); #endif + /* Find the current VTL */ + ms_hyperv.vtl = get_current_vtl(); return; diff --git a/include/asm-generic/mshyperv.h b/include/asm-generic/mshyperv.h index bfb9eb9d7215..68133de044ec 100644 --- a/include/asm-generic/mshyperv.h +++ b/include/asm-generic/mshyperv.h @@ -46,6 +46,7 @@ struct ms_hyperv_info { }; }; u64 shared_gpa_boundary; + u8 vtl; }; extern struct ms_hyperv_info ms_hyperv; @@ -55,6 +56,7 @@ extern void * __percpu *hyperv_pcpu_output_arg; extern u64 hv_do_hypercall(u64 control, void *inputaddr, void *outputaddr); extern u64 hv_do_fast_hypercall8(u16 control, u64 input8); extern bool hv_isolation_type_snp(void); +extern bool hv_isolation_type_en_snp(void); /* Helper functions that provide a consistent pattern for checking Hyper-V hypercall status. */ static inline int hv_result(u64 status) From patchwork Sat Nov 19 03:46:20 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tianyu Lan X-Patchwork-Id: 13049560 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D60C7C4321E for ; Sat, 19 Nov 2022 03:47:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231445AbiKSDrK (ORCPT ); Fri, 18 Nov 2022 22:47:10 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37120 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231837AbiKSDq6 (ORCPT ); Fri, 18 Nov 2022 22:46:58 -0500 Received: from mail-pl1-x632.google.com (mail-pl1-x632.google.com [IPv6:2607:f8b0:4864:20::632]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 82B53BF81A; Fri, 18 Nov 2022 19:46:47 -0800 (PST) Received: by mail-pl1-x632.google.com with SMTP id d20so6166878plr.10; Fri, 18 Nov 2022 19:46:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=loq3axK/RYrhXox7s9iRtCxZB75cuqQoJg9zTfXxZb4=; b=GllTYzWqVeBfD05vmI3Kf6GQCRffUlQTKvgReqPUs8bFAsnamfS/bSPbWcMdZIz2On xZY3KrIbQwunMR9cxZZDrR+/49IOPL1TzdJcRBUxkciMza7kYtr3Vxkzv8HF9F/zPoMV zryjUzHKcaq9Ml2m+ql3StS+qHYKftVUqHeqDRcBAa5QOx7r3etLKD7YsXOhUpEvgsMi gR346QLh3kwgCl726R/h5BMr8Gf6fYP5acx9USQ3yTNDknhLocyRNXs7gB0pTVD9J5NQ wd4vIF9JBinej/tPBZCt5HX3cjSIbmfXGRwiDvJbjcxL9HxRHu1NGhyQrxC1nhGAReP9 Mf+A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=loq3axK/RYrhXox7s9iRtCxZB75cuqQoJg9zTfXxZb4=; b=MdidGG+BnIa8MpeOjH5Z9WPu8AmXpfmKccWH9YqnCW9ulPB0uBfbUIom8zwnrW6WAr A4vdnF0J1As8bkGwHZD5mKnxt6cTsfSJO9haGzvz5Ec4bWoQxAVjlZ324cFKBPWCElFh p3kzZhCJxjRShhpEqNYe2swLlUrUMyhMwwa1xgyTFkAPZUdf/4kLiYyxwHWMyFBLTxe2 g4UurQUuW1EMEBv0gasg1fTYg1jZm5oct7XoPgDgRgm/eTxOee6dHM+FNF13zNBOFP/i WwjcIVLbqy0Ok6KxH9cUS5DPh2PCYuXo4cKnaJHwIbUf5L2I+LNid/ilmtCLf76XnXs/ KuIw== X-Gm-Message-State: ANoB5pk8cCzD+CoPsc1J+riYtFJrRIvCHlixYDPdvT4GT1nf0RV5C8d4 nIfK3aqAaeeM9Bu0c851FGTU/lBNovk9Rg== X-Google-Smtp-Source: AA0mqf4Yr80lVxobHyYze1nJWwmu9pREn8Lqn0OrbCGwLv5PSXDahvLnr3CQjNI90RQVOeSyl8FSVA== X-Received: by 2002:a17:902:cac4:b0:188:51c0:705 with SMTP id y4-20020a170902cac400b0018851c00705mr2411353pld.62.1668829606972; Fri, 18 Nov 2022 19:46:46 -0800 (PST) Received: from ubuntu-Virtual-Machine.corp.microsoft.com ([2001:4898:80e8:38:f087:1794:92c5:f8f0]) by smtp.gmail.com with ESMTPSA id e5-20020a056a0000c500b005360da6b26bsm3913892pfj.159.2022.11.18.19.46.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Nov 2022 19:46:46 -0800 (PST) From: Tianyu Lan To: luto@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, seanjc@google.com, pbonzini@redhat.com, jgross@suse.com, tiala@microsoft.com, kirill@shutemov.name, jiangshan.ljs@antgroup.com, peterz@infradead.org, ashish.kalra@amd.com, srutherford@google.com, akpm@linux-foundation.org, anshuman.khandual@arm.com, pawan.kumar.gupta@linux.intel.com, adrian.hunter@intel.com, daniel.sneddon@linux.intel.com, alexander.shishkin@linux.intel.com, sandipan.das@amd.com, ray.huang@amd.com, brijesh.singh@amd.com, michael.roth@amd.com, thomas.lendacky@amd.com, venu.busireddy@oracle.com, sterritt@google.com, tony.luck@intel.com, samitolvanen@google.com, fenghua.yu@intel.com Cc: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-hyperv@vger.kernel.org, linux-arch@vger.kernel.org Subject: [RFC PATCH V2 06/18] x86/hyperv: Use vmmcall to implement hvcall in sev-snp enlightened guest Date: Fri, 18 Nov 2022 22:46:20 -0500 Message-Id: <20221119034633.1728632-7-ltykernel@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221119034633.1728632-1-ltykernel@gmail.com> References: <20221119034633.1728632-1-ltykernel@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Tianyu Lan In sev-snp enlightened guest, hvcall needs to use vmmcall to trigger vmexit and notify hypervisor to handle hypercall request. Signed-off-by: Tianyu Lan --- arch/x86/include/asm/mshyperv.h | 66 ++++++++++++++++++++++----------- 1 file changed, 45 insertions(+), 21 deletions(-) diff --git a/arch/x86/include/asm/mshyperv.h b/arch/x86/include/asm/mshyperv.h index 9b8c3f638845..28d5429e33c9 100644 --- a/arch/x86/include/asm/mshyperv.h +++ b/arch/x86/include/asm/mshyperv.h @@ -45,16 +45,25 @@ static inline u64 hv_do_hypercall(u64 control, void *input, void *output) u64 hv_status; #ifdef CONFIG_X86_64 - if (!hv_hypercall_pg) - return U64_MAX; + if (hv_isolation_type_en_snp()) { + __asm__ __volatile__("mov %4, %%r8\n" + "vmmcall" + : "=a" (hv_status), ASM_CALL_CONSTRAINT, + "+c" (control), "+d" (input_address) + : "r" (output_address) + : "cc", "memory", "r8", "r9", "r10", "r11"); + } else { + if (!hv_hypercall_pg) + return U64_MAX; - __asm__ __volatile__("mov %4, %%r8\n" - CALL_NOSPEC - : "=a" (hv_status), ASM_CALL_CONSTRAINT, - "+c" (control), "+d" (input_address) - : "r" (output_address), - THUNK_TARGET(hv_hypercall_pg) - : "cc", "memory", "r8", "r9", "r10", "r11"); + __asm__ __volatile__("mov %4, %%r8\n" + CALL_NOSPEC + : "=a" (hv_status), ASM_CALL_CONSTRAINT, + "+c" (control), "+d" (input_address) + : "r" (output_address), + THUNK_TARGET(hv_hypercall_pg) + : "cc", "memory", "r8", "r9", "r10", "r11"); + } #else u32 input_address_hi = upper_32_bits(input_address); u32 input_address_lo = lower_32_bits(input_address); @@ -82,12 +91,18 @@ static inline u64 hv_do_fast_hypercall8(u16 code, u64 input1) u64 hv_status, control = (u64)code | HV_HYPERCALL_FAST_BIT; #ifdef CONFIG_X86_64 - { + if (hv_isolation_type_en_snp()) { + __asm__ __volatile__( + "vmmcall" + : "=a" (hv_status), ASM_CALL_CONSTRAINT, + "+c" (control), "+d" (input1) + :: "cc", "r8", "r9", "r10", "r11"); + } else { __asm__ __volatile__(CALL_NOSPEC - : "=a" (hv_status), ASM_CALL_CONSTRAINT, - "+c" (control), "+d" (input1) - : THUNK_TARGET(hv_hypercall_pg) - : "cc", "r8", "r9", "r10", "r11"); + : "=a" (hv_status), ASM_CALL_CONSTRAINT, + "+c" (control), "+d" (input1) + : THUNK_TARGET(hv_hypercall_pg) + : "cc", "r8", "r9", "r10", "r11"); } #else { @@ -113,14 +128,21 @@ static inline u64 hv_do_fast_hypercall16(u16 code, u64 input1, u64 input2) u64 hv_status, control = (u64)code | HV_HYPERCALL_FAST_BIT; #ifdef CONFIG_X86_64 - { + if (hv_isolation_type_en_snp()) { __asm__ __volatile__("mov %4, %%r8\n" - CALL_NOSPEC - : "=a" (hv_status), ASM_CALL_CONSTRAINT, - "+c" (control), "+d" (input1) - : "r" (input2), - THUNK_TARGET(hv_hypercall_pg) - : "cc", "r8", "r9", "r10", "r11"); + "vmmcall" + : "=a" (hv_status), ASM_CALL_CONSTRAINT, + "+c" (control), "+d" (input1) + : "r" (input2) + : "cc", "r8", "r9", "r10", "r11"); + } else { + __asm__ __volatile__("mov %4, %%r8\n" + CALL_NOSPEC + : "=a" (hv_status), ASM_CALL_CONSTRAINT, + "+c" (control), "+d" (input1) + : "r" (input2), + THUNK_TARGET(hv_hypercall_pg) + : "cc", "r8", "r9", "r10", "r11"); } #else { @@ -177,6 +199,7 @@ int hv_map_ioapic_interrupt(int ioapic_id, bool level, int vcpu, int vector, struct hv_interrupt_entry *entry); int hv_unmap_ioapic_interrupt(int ioapic_id, struct hv_interrupt_entry *entry); int hv_set_mem_host_visibility(unsigned long addr, int numpages, bool visible); +int hv_snp_boot_ap(int cpu, unsigned long start_ip); #ifdef CONFIG_AMD_MEM_ENCRYPT void hv_ghcb_msr_write(u64 msr, u64 value); @@ -191,6 +214,7 @@ static inline void hv_ghcb_terminate(unsigned int set, unsigned int reason) {} #endif extern bool hv_isolation_type_snp(void); +extern bool hv_isolation_type_en_snp(void); static inline bool hv_is_synic_reg(unsigned int reg) { From patchwork Sat Nov 19 03:46:21 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tianyu Lan X-Patchwork-Id: 13049562 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4018FC4332F for ; Sat, 19 Nov 2022 03:47:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232705AbiKSDrf (ORCPT ); Fri, 18 Nov 2022 22:47:35 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37082 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231894AbiKSDq6 (ORCPT ); Fri, 18 Nov 2022 22:46:58 -0500 Received: from mail-pg1-x533.google.com (mail-pg1-x533.google.com [IPv6:2607:f8b0:4864:20::533]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D7485BF827; Fri, 18 Nov 2022 19:46:48 -0800 (PST) Received: by mail-pg1-x533.google.com with SMTP id s196so6625641pgs.3; Fri, 18 Nov 2022 19:46:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=HkcTs/ArLjoTKDcTyzMU889nkY7DdVdas4PocmXRhm8=; b=mqftbILMyrMMSY0sRSnUOUKl4NdLFeLmvTIprm5l3ulZjOduQa+DKnjzXatyDsfod2 5aMn9BhYRf4D7b3w6+k+76w5t/0HejkZAI76dYCA8Wk/r4lyaFcV54HtbPgllMGWHd6n G9UoXvtLawT5Jo/k3vj2l0JXU0KG+vo6jPGXihU6Z4mVDqj4y0IBpTVPcezUXYEA6Z6V cJuM5W6P/srgHZ328qfssz5u3cQCDzXXEWdhCkqf6JrXPuzveOosoYIxPREjoMgzQIP9 QzfDes422LSjz86opbhLBm/QIXXRyGbJXmXvMJwHfD2VlJSjQatUdtiEOc8CmS54YSR3 Vqnw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=HkcTs/ArLjoTKDcTyzMU889nkY7DdVdas4PocmXRhm8=; b=Q8EIYKAjGLtazXN2TjnoHVeHcx7At7JnNrw4ca7aJbgU5XyOrvHUHQI28CVqdQE/Di Wp+1oSSJpAmCfr2gdbi9ZWnDvPb/I7rlIcd3x5+a18rLfpqyqrXquL9s5vH/JXC70nhP qjQ0XW7IXXeo56uQ65p5EsCP03CtanpPnNg90pSIGqDYQFqYkQFbxAAxY13yneXnZ2ad Iz2LPrSp/Xlo69yMNFrO68rvl45G+1eGqAMeleXP98wtQZhGxUHD7QCtFOBnlTGSVxM+ PNjasMSkxnkxP3FIiYBAtj2MDtsNkpwJIrTBqgYg+ioEd9COKf+C6HScU7lZ9qcKNTB/ fcDg== X-Gm-Message-State: ANoB5pmOqGI1VGeHAr/PuxHu7E9XXRO0SzMRutBZrASRH5QmPoVIqJaJ XxEEu0pgs8ILnWs3Vw+DhBc= X-Google-Smtp-Source: AA0mqf4HTkeLEW7VU8wU9ICZbtzKyS4ZtKTO5JSSusDifWlB3yEYFfUMwdwkdGtSgPsQRfXJ5zEblQ== X-Received: by 2002:a63:5308:0:b0:46f:cec6:8805 with SMTP id h8-20020a635308000000b0046fcec68805mr9278748pgb.612.1668829608401; Fri, 18 Nov 2022 19:46:48 -0800 (PST) Received: from ubuntu-Virtual-Machine.corp.microsoft.com ([2001:4898:80e8:38:f087:1794:92c5:f8f0]) by smtp.gmail.com with ESMTPSA id e5-20020a056a0000c500b005360da6b26bsm3913892pfj.159.2022.11.18.19.46.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Nov 2022 19:46:47 -0800 (PST) From: Tianyu Lan To: luto@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, seanjc@google.com, pbonzini@redhat.com, jgross@suse.com, tiala@microsoft.com, kirill@shutemov.name, jiangshan.ljs@antgroup.com, peterz@infradead.org, ashish.kalra@amd.com, srutherford@google.com, akpm@linux-foundation.org, anshuman.khandual@arm.com, pawan.kumar.gupta@linux.intel.com, adrian.hunter@intel.com, daniel.sneddon@linux.intel.com, alexander.shishkin@linux.intel.com, sandipan.das@amd.com, ray.huang@amd.com, brijesh.singh@amd.com, michael.roth@amd.com, thomas.lendacky@amd.com, venu.busireddy@oracle.com, sterritt@google.com, tony.luck@intel.com, samitolvanen@google.com, fenghua.yu@intel.com Cc: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-hyperv@vger.kernel.org, linux-arch@vger.kernel.org Subject: [RFC PATCH V2 07/18] clocksource: hyper-v: decrypt hyperv tsc page in sev-snp enlightened guest Date: Fri, 18 Nov 2022 22:46:21 -0500 Message-Id: <20221119034633.1728632-8-ltykernel@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221119034633.1728632-1-ltykernel@gmail.com> References: <20221119034633.1728632-1-ltykernel@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Tianyu Lan Hyper-V tsc page is shared with hypervisor and it should be decrypted in sev-snp enlightened guest when it's used. Signed-off-by: Tianyu Lan --- drivers/clocksource/hyperv_timer.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/clocksource/hyperv_timer.c b/drivers/clocksource/hyperv_timer.c index bb47610bbd1c..aa68eebed5ee 100644 --- a/drivers/clocksource/hyperv_timer.c +++ b/drivers/clocksource/hyperv_timer.c @@ -364,7 +364,7 @@ EXPORT_SYMBOL_GPL(hv_stimer_global_cleanup); static union { struct ms_hyperv_tsc_page page; u8 reserved[PAGE_SIZE]; -} tsc_pg __aligned(PAGE_SIZE); +} tsc_pg __bss_decrypted __aligned(PAGE_SIZE); struct ms_hyperv_tsc_page *hv_get_tsc_page(void) { From patchwork Sat Nov 19 03:46:22 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tianyu Lan X-Patchwork-Id: 13049563 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9D462C433FE for ; Sat, 19 Nov 2022 03:48:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232052AbiKSDrh (ORCPT ); Fri, 18 Nov 2022 22:47:37 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37136 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231940AbiKSDq7 (ORCPT ); Fri, 18 Nov 2022 22:46:59 -0500 Received: from mail-pj1-x102d.google.com (mail-pj1-x102d.google.com [IPv6:2607:f8b0:4864:20::102d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4A98EBF822; Fri, 18 Nov 2022 19:46:50 -0800 (PST) Received: by mail-pj1-x102d.google.com with SMTP id w15-20020a17090a380f00b0021873113cb4so6349431pjb.0; Fri, 18 Nov 2022 19:46:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ItxDpPRpAqbvZZHr07qhhxQmPJehVCNLqSw71HrxTxc=; b=VQxEqC5qot3L/gBM3Sf1bbsdOF9if2kV/RVHJFP65XEk0ifY/2XASgPM++aXUyKchT TQ4efq06/HtUT3jAbNbtFLIfIf2BhwxPbGubtO8Lxstay5MYzdNJASfgRUVsk6pJuv4d eARufqyIzHmJx5HwxkSvlSj6Em9vvSblPFjpnYdQOPvNDBOztA790QOnHRQKPSP/Tr4q T8PsDn+w+vtzBVqab8aWwu/NJ9p/o2ERcKATOb8/5jkGJfuD7QBF92OjaSh9ov4DAHYc hK5NqZyFTjbSJKEGCAGtiYgRad7Xe8T44UBi/KUH+DKTVndUSMgTI0UKcEAsmHT1YK5Q d0bg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ItxDpPRpAqbvZZHr07qhhxQmPJehVCNLqSw71HrxTxc=; b=dnfmHBgAb65b9zcUKSjWjsu/mMx1d7kNJ8crRNBmsTZA9JXK6kOj2TdJjSWNikROGT +816c9XCTMjJEouoOmCep1mfjAfUI/R0dfw21mj4othhPJoaD+YTmnO/ePQn8dmHUzM2 HUOUF7mOJvDioPEtH9hF/fBvYqp04jfxcNC/5NxJ534KMkpuS6wk6oe9FxbCBQ2xHo1+ TXsNv5MshdUIwPhARdPfoZyaU+GH2Qy1jdhC5q15EF8J+jH+gDuiJvOk6q991u8INMQQ 4OOKqWkbI02eg7fi/r/c7nvm3laSj8uAhl8QBkDOGGOLjx70INCguxblz+WpzEvJRiyE g5dA== X-Gm-Message-State: ANoB5pkUm9KqpnCJOqexPNHPIHj5Kr/FVNT6+4FMfI38Qrnu+uW3+4fX cC2Sjq94hSbxu1PmUnd/I2o= X-Google-Smtp-Source: AA0mqf6Qw5HeeCOlmu7rZ59aqHFSQUAoRvby6UhqkbdaM9NOcB3rWY+HvxJyUP1+EtFsq1YlG5DDMg== X-Received: by 2002:a17:90a:bb86:b0:218:8d27:cdae with SMTP id v6-20020a17090abb8600b002188d27cdaemr4403165pjr.244.1668829609794; Fri, 18 Nov 2022 19:46:49 -0800 (PST) Received: from ubuntu-Virtual-Machine.corp.microsoft.com ([2001:4898:80e8:38:f087:1794:92c5:f8f0]) by smtp.gmail.com with ESMTPSA id e5-20020a056a0000c500b005360da6b26bsm3913892pfj.159.2022.11.18.19.46.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Nov 2022 19:46:49 -0800 (PST) From: Tianyu Lan To: luto@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, seanjc@google.com, pbonzini@redhat.com, jgross@suse.com, tiala@microsoft.com, kirill@shutemov.name, jiangshan.ljs@antgroup.com, peterz@infradead.org, ashish.kalra@amd.com, srutherford@google.com, akpm@linux-foundation.org, anshuman.khandual@arm.com, pawan.kumar.gupta@linux.intel.com, adrian.hunter@intel.com, daniel.sneddon@linux.intel.com, alexander.shishkin@linux.intel.com, sandipan.das@amd.com, ray.huang@amd.com, brijesh.singh@amd.com, michael.roth@amd.com, thomas.lendacky@amd.com, venu.busireddy@oracle.com, sterritt@google.com, tony.luck@intel.com, samitolvanen@google.com, fenghua.yu@intel.com Cc: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-hyperv@vger.kernel.org, linux-arch@vger.kernel.org Subject: [RFC PATCH V2 08/18] x86/hyperv: decrypt vmbus pages for sev-snp enlightened guest Date: Fri, 18 Nov 2022 22:46:22 -0500 Message-Id: <20221119034633.1728632-9-ltykernel@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221119034633.1728632-1-ltykernel@gmail.com> References: <20221119034633.1728632-1-ltykernel@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Tianyu Lan Vmbus int, synic and post message pages are shared with hypervisor and so decrypt these pages in the sev-snp guest. Signed-off-by: Tianyu Lan --- drivers/hv/connection.c | 13 +++++++++++++ drivers/hv/hv.c | 32 +++++++++++++++++++++++++++++++- 2 files changed, 44 insertions(+), 1 deletion(-) diff --git a/drivers/hv/connection.c b/drivers/hv/connection.c index 9dc27e5d367a..43141225ea15 100644 --- a/drivers/hv/connection.c +++ b/drivers/hv/connection.c @@ -215,6 +215,15 @@ int vmbus_connect(void) (void *)((unsigned long)vmbus_connection.int_page + (HV_HYP_PAGE_SIZE >> 1)); + if (hv_isolation_type_snp() || hv_isolation_type_en_snp()) { + ret = set_memory_decrypted((unsigned long) + vmbus_connection.int_page, 1); + if (ret) + goto cleanup; + + memset(vmbus_connection.int_page, 0, PAGE_SIZE); + } + /* * Setup the monitor notification facility. The 1st page for * parent->child and the 2nd page for child->parent @@ -372,6 +381,10 @@ void vmbus_disconnect(void) destroy_workqueue(vmbus_connection.work_queue); if (vmbus_connection.int_page) { + if (hv_isolation_type_en_snp()) + set_memory_encrypted((unsigned long) + vmbus_connection.int_page, 1); + hv_free_hyperv_page((unsigned long)vmbus_connection.int_page); vmbus_connection.int_page = NULL; } diff --git a/drivers/hv/hv.c b/drivers/hv/hv.c index 4d6480d57546..f9111eb32739 100644 --- a/drivers/hv/hv.c +++ b/drivers/hv/hv.c @@ -20,6 +20,7 @@ #include #include #include +#include #include "hyperv_vmbus.h" /* The one and only */ @@ -117,7 +118,7 @@ int hv_post_message(union hv_connection_id connection_id, int hv_synic_alloc(void) { - int cpu; + int cpu, ret; struct hv_per_cpu_context *hv_cpu; /* @@ -168,6 +169,29 @@ int hv_synic_alloc(void) pr_err("Unable to allocate post msg page\n"); goto err; } + + if (hv_isolation_type_en_snp()) { + ret = set_memory_decrypted((unsigned long) + hv_cpu->synic_message_page, 1); + ret |= set_memory_decrypted((unsigned long) + hv_cpu->synic_event_page, 1); + ret |= set_memory_decrypted((unsigned long) + hv_cpu->post_msg_page, 1); + + if (ret) { + set_memory_encrypted((unsigned long) + hv_cpu->synic_message_page, 1); + set_memory_encrypted((unsigned long) + hv_cpu->synic_event_page, 1); + set_memory_encrypted((unsigned long) + hv_cpu->post_msg_page, 1); + goto err; + } + + memset(hv_cpu->synic_message_page, 0, PAGE_SIZE); + memset(hv_cpu->synic_event_page, 0, PAGE_SIZE); + memset(hv_cpu->post_msg_page, 0, PAGE_SIZE); + } } return 0; @@ -188,6 +212,12 @@ void hv_synic_free(void) struct hv_per_cpu_context *hv_cpu = per_cpu_ptr(hv_context.cpu_context, cpu); + if (hv_isolation_type_en_snp()) { + set_memory_encrypted((unsigned long)hv_cpu->synic_message_page, 1); + set_memory_encrypted((unsigned long)hv_cpu->synic_event_page, 1); + set_memory_encrypted((unsigned long)hv_cpu->post_msg_page, 1); + } + free_page((unsigned long)hv_cpu->synic_event_page); free_page((unsigned long)hv_cpu->synic_message_page); free_page((unsigned long)hv_cpu->post_msg_page); From patchwork Sat Nov 19 03:46:23 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tianyu Lan X-Patchwork-Id: 13049561 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A0168C07E9D for ; Sat, 19 Nov 2022 03:47:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232315AbiKSDrc (ORCPT ); Fri, 18 Nov 2022 22:47:32 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37184 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231963AbiKSDrA (ORCPT ); Fri, 18 Nov 2022 22:47:00 -0500 Received: from mail-pj1-x102a.google.com (mail-pj1-x102a.google.com [IPv6:2607:f8b0:4864:20::102a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4EA32BF830; Fri, 18 Nov 2022 19:46:51 -0800 (PST) Received: by mail-pj1-x102a.google.com with SMTP id a22-20020a17090a6d9600b0021896eb5554so996231pjk.1; Fri, 18 Nov 2022 19:46:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=wN6eB0wxJRf6zzRxY6T7pa65W2+E1oedx78oWdQ5Oug=; b=NBt4Xls1RZOBtmtBilmybh/XvJM5mSsTG2HAd6utN6D5Mu9JgAYK1Az3jDz+NYOUmf 4YyOzRgIifgQxbVi8a3V1ZLzsJr06l37H6yxvSDAXYuMsNNFqighPMYkMgq0dgHn2kJU OgBeCVSscvZ8ZJHXAQ4JOxi9n/Aqm8cNgstROCJtG7OFxP0f32WsSMW76UDYZxTAMS8+ 1RRhWDB22k3c5cOsw7iJko8FYwxpoLFc1P6kznPmTcVWHu+pqUqvX816WtU4+frGBEq7 GXPI/SGzZmV73ZhFlsV/IBtG/InWq+hDEEy6mgIKE4Pz2CXDme4mvwdTC1hlcUhD1xvp CTlA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=wN6eB0wxJRf6zzRxY6T7pa65W2+E1oedx78oWdQ5Oug=; b=NmJGObYoNeMjrVmnwC+yWe/TjlsuuqI2TV4JaXyoVNzHRlnb0PUZRBpRyZjEnyM0Ih Jpi0N/7qFF15N5/eGl7tV4ttsWOzyJDTO4ADNa7cWm8114ID3uIO1uhykHmuPPcCpXqD cOUC/5Useu1KY6jwAT4ml4xATZXtzwq7LeoUKlSW2xdHawhvDb6QmM1ukmZjrhSddJ/N rRxEGsqY8Hcrn+03u9io0w3jZtzGXXB60ireqDMiiE5MlDWCviE5QpuwcpdiDDXTEllH NuQVBZOK+UqR35nsO4KtTQswboqdKx+eSO4/WsU/7K0tqaeCGjwL/t7x9FxgkoMqw7Ug rKEg== X-Gm-Message-State: ANoB5pl1mQYI4jXtDRUql0QA/oYIy3hXGGw71Wv5vBFI/CzxtPlZ9ipq nz9FY7vJrasLtgBXlV0S8+Y= X-Google-Smtp-Source: AA0mqf5zIYZtw6iVkCd7uQV2a6WMjWTxhRQw0MJkAjBeqXqboLVNOYh8+nY30Xi18DY1G3MYt/fMIg== X-Received: by 2002:a17:902:6847:b0:183:6555:38ef with SMTP id f7-20020a170902684700b00183655538efmr2340783pln.157.1668829611039; Fri, 18 Nov 2022 19:46:51 -0800 (PST) Received: from ubuntu-Virtual-Machine.corp.microsoft.com ([2001:4898:80e8:38:f087:1794:92c5:f8f0]) by smtp.gmail.com with ESMTPSA id e5-20020a056a0000c500b005360da6b26bsm3913892pfj.159.2022.11.18.19.46.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Nov 2022 19:46:50 -0800 (PST) From: Tianyu Lan To: luto@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, seanjc@google.com, pbonzini@redhat.com, jgross@suse.com, tiala@microsoft.com, kirill@shutemov.name, jiangshan.ljs@antgroup.com, peterz@infradead.org, ashish.kalra@amd.com, srutherford@google.com, akpm@linux-foundation.org, anshuman.khandual@arm.com, pawan.kumar.gupta@linux.intel.com, adrian.hunter@intel.com, daniel.sneddon@linux.intel.com, alexander.shishkin@linux.intel.com, sandipan.das@amd.com, ray.huang@amd.com, brijesh.singh@amd.com, michael.roth@amd.com, thomas.lendacky@amd.com, venu.busireddy@oracle.com, sterritt@google.com, tony.luck@intel.com, samitolvanen@google.com, fenghua.yu@intel.com Cc: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-hyperv@vger.kernel.org, linux-arch@vger.kernel.org Subject: [RFC PATCH V2 09/18] x86/hyperv: set target vtl in the vmbus init message Date: Fri, 18 Nov 2022 22:46:23 -0500 Message-Id: <20221119034633.1728632-10-ltykernel@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221119034633.1728632-1-ltykernel@gmail.com> References: <20221119034633.1728632-1-ltykernel@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Tianyu Lan Set target vtl (Virtual Trust Level) in the vmbus init message. Signed-off-by: Tianyu Lan --- drivers/hv/connection.c | 1 + include/linux/hyperv.h | 4 ++-- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/drivers/hv/connection.c b/drivers/hv/connection.c index 43141225ea15..09a1253b539a 100644 --- a/drivers/hv/connection.c +++ b/drivers/hv/connection.c @@ -98,6 +98,7 @@ int vmbus_negotiate_version(struct vmbus_channel_msginfo *msginfo, u32 version) */ if (version >= VERSION_WIN10_V5) { msg->msg_sint = VMBUS_MESSAGE_SINT; + msg->msg_vtl = ms_hyperv.vtl; vmbus_connection.msg_conn_id = VMBUS_MESSAGE_CONNECTION_ID_4; } else { msg->interrupt_page = virt_to_phys(vmbus_connection.int_page); diff --git a/include/linux/hyperv.h b/include/linux/hyperv.h index 3b42264333ef..2be0b5efd1ea 100644 --- a/include/linux/hyperv.h +++ b/include/linux/hyperv.h @@ -665,8 +665,8 @@ struct vmbus_channel_initiate_contact { u64 interrupt_page; struct { u8 msg_sint; - u8 padding1[3]; - u32 padding2; + u8 msg_vtl; + u8 reserved[6]; }; }; u64 monitor_page1; From patchwork Sat Nov 19 03:46:24 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tianyu Lan X-Patchwork-Id: 13049565 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D48FCC4167B for ; Sat, 19 Nov 2022 03:48:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232089AbiKSDrl (ORCPT ); Fri, 18 Nov 2022 22:47:41 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37136 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232216AbiKSDrF (ORCPT ); Fri, 18 Nov 2022 22:47:05 -0500 Received: from mail-pj1-x102a.google.com (mail-pj1-x102a.google.com [IPv6:2607:f8b0:4864:20::102a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 00DF1BFF46; Fri, 18 Nov 2022 19:46:52 -0800 (PST) Received: by mail-pj1-x102a.google.com with SMTP id b1-20020a17090a7ac100b00213fde52d49so6773362pjl.3; Fri, 18 Nov 2022 19:46:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=xAXRqP2y00DrPAfpogwk3TLv9UhC4S/D5/TlIyE91Hw=; b=ClwKlnSy/Mz6sk0P1y3kITeMuLjvpRy0L67pKK4fRHpHSolrEEDdsUI8Y20sPFG+7F aA6QaIZ52/Y6wT4KHTuSaz6J4S581VQH1ExKZRz3qcZr1jp5kNLLxw5EFxc1EelDP1sD 7O7HCK0yC7QmOqc5BebQe15FUSnCBBSUXCapZ18gkI+bEAeX2dsf/MK3V7GG3020IeEj 9RI091r5EyrpPdbEi5E0mciPoKmPis4EU2iJr55IXHvGS+V1Iu3EeDkupNAeftDCtvWR v8ABBQLDsXLOz1GjuGO+6apEtUTRvYy6wbZWQ4GW7nLzmBMnbFZJtvcSqYknPCh00wce NNDQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=xAXRqP2y00DrPAfpogwk3TLv9UhC4S/D5/TlIyE91Hw=; b=BOnml5AwKKLp1njyz44x2w1l1JNKASLakeTY5tEPClkhYCbJM3h4Nyw7wOcS8ZqayF hWjhLPuWUAtFyXzMeHA66LC1GloQMyKd027/yJrLjbqE4dM88tr9z7fS8D96/pnfrobP 1O+rl8CiGVgyxuRQqXA1yWCkrowEc7viDFzgLliA3sfeAbjDiuIK7sLlxRHpbNT17qJA r4sSvWbJV7S2un8HaSN5Vcnf1397PsPAsbtp89pbrsONqNWM2ac2YNtXM/SHTDGN5g0R AiHaHQSj1C8Bro1jldEMvG5Mxq+2UIufoxXSU0iVWxJoKBB78mdLVtxRGTBI42ePE6NY Wcqw== X-Gm-Message-State: ANoB5pkCC1rNTnxg93+Z8QqCEjFFMEtuuAtepnO3Je2Eff9UKuGg7N0a bf1gia6AqzddqqccN8VFnn4= X-Google-Smtp-Source: AA0mqf7C4HWWj9/I41fGzMZ4RjAnGmUbCLdi2fhrOpVXAY0OkMXcLrJbuauBk8W5UthPygU54iZXSw== X-Received: by 2002:a17:90a:a60c:b0:213:2e97:5ea4 with SMTP id c12-20020a17090aa60c00b002132e975ea4mr16715811pjq.92.1668829612431; Fri, 18 Nov 2022 19:46:52 -0800 (PST) Received: from ubuntu-Virtual-Machine.corp.microsoft.com ([2001:4898:80e8:38:f087:1794:92c5:f8f0]) by smtp.gmail.com with ESMTPSA id e5-20020a056a0000c500b005360da6b26bsm3913892pfj.159.2022.11.18.19.46.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Nov 2022 19:46:51 -0800 (PST) From: Tianyu Lan To: luto@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, seanjc@google.com, pbonzini@redhat.com, jgross@suse.com, tiala@microsoft.com, kirill@shutemov.name, jiangshan.ljs@antgroup.com, peterz@infradead.org, ashish.kalra@amd.com, srutherford@google.com, akpm@linux-foundation.org, anshuman.khandual@arm.com, pawan.kumar.gupta@linux.intel.com, adrian.hunter@intel.com, daniel.sneddon@linux.intel.com, alexander.shishkin@linux.intel.com, sandipan.das@amd.com, ray.huang@amd.com, brijesh.singh@amd.com, michael.roth@amd.com, thomas.lendacky@amd.com, venu.busireddy@oracle.com, sterritt@google.com, tony.luck@intel.com, samitolvanen@google.com, fenghua.yu@intel.com Cc: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-hyperv@vger.kernel.org, linux-arch@vger.kernel.org Subject: [RFC PATCH V2 10/18] drivers: hv: Decrypt percpu hvcall input arg page in sev-snp enlightened guest Date: Fri, 18 Nov 2022 22:46:24 -0500 Message-Id: <20221119034633.1728632-11-ltykernel@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221119034633.1728632-1-ltykernel@gmail.com> References: <20221119034633.1728632-1-ltykernel@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Tianyu Lan Hypervisor needs to access iput arg page and guest should decrypt the page. Signed-off-by: Tianyu Lan --- drivers/hv/hv_common.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/drivers/hv/hv_common.c b/drivers/hv/hv_common.c index 2c6602571c47..c16961e686a0 100644 --- a/drivers/hv/hv_common.c +++ b/drivers/hv/hv_common.c @@ -21,6 +21,7 @@ #include #include #include +#include #include #include @@ -125,6 +126,7 @@ int hv_common_cpu_init(unsigned int cpu) u64 msr_vp_index; gfp_t flags; int pgcount = hv_root_partition ? 2 : 1; + int ret; /* hv_cpu_init() can be called with IRQs disabled from hv_resume() */ flags = irqs_disabled() ? GFP_ATOMIC : GFP_KERNEL; @@ -134,6 +136,16 @@ int hv_common_cpu_init(unsigned int cpu) if (!(*inputarg)) return -ENOMEM; + if (hv_isolation_type_en_snp()) { + ret = set_memory_decrypted((unsigned long)*inputarg, 1); + if (ret) { + kfree(*inputarg); + return ret; + } + + memset(*inputarg, 0x00, PAGE_SIZE); + } + if (hv_root_partition) { outputarg = (void **)this_cpu_ptr(hyperv_pcpu_output_arg); *outputarg = (char *)(*inputarg) + HV_HYP_PAGE_SIZE; @@ -168,6 +180,9 @@ int hv_common_cpu_die(unsigned int cpu) local_irq_restore(flags); + if (hv_isolation_type_en_snp()) + set_memory_encrypted((unsigned long)mem, 1); + kfree(mem); return 0; From patchwork Sat Nov 19 03:46:25 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tianyu Lan X-Patchwork-Id: 13049566 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 727D8C4332F for ; Sat, 19 Nov 2022 03:48:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233217AbiKSDsE (ORCPT ); Fri, 18 Nov 2022 22:48:04 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37372 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231864AbiKSDrG (ORCPT ); Fri, 18 Nov 2022 22:47:06 -0500 Received: from mail-pj1-x102c.google.com (mail-pj1-x102c.google.com [IPv6:2607:f8b0:4864:20::102c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5CB88BFF67; Fri, 18 Nov 2022 19:46:54 -0800 (PST) Received: by mail-pj1-x102c.google.com with SMTP id m14so6119104pji.0; Fri, 18 Nov 2022 19:46:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=eHfQ2xg7TSoUMJoNFBa3Rt6P1l7onXyfXsIWqOdHwnM=; b=p7wzlQcXuRaSIC6PeeE++lmR7XnrDFVT5YTYvnC4lth3AZIUOaMSkbgVGFQqOKOtz1 q/kfQynPzZLbZtNfW+R3Vs6Uprf/d1Ub4tp0hdsMRVMHNBNiXVVdoldrZN6vD7leJ46N ks3cVYzKSAEozqNDHUx0mQZnacluhbffwbLvSF58+MQkXZO3xFecGg9/xhPTfMm9oBVh lQHQ6zon6yB6Ve8DW352Gxi85rH6/MSzEdWS7rpv3BxYYvd5hbrFaGPGMSO24pUhqrLa nzZGkTN9VwCi+nojCKLDkLlUsvyPaaZR3NfA0KCUsn/xkOyfgRvz2dLA4GS/mGffCLcb fm7A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=eHfQ2xg7TSoUMJoNFBa3Rt6P1l7onXyfXsIWqOdHwnM=; b=cay6VYGaXIabscLFgUGcnVM4RmP0AiKFKfXXC7InbUBAr8Z5TaTSxMJq5zEl3UbZWE rzU8A7JltSZivoC/i391Y2a3VKz1wfun2LI30CgugS1wHp6vQ8i32WDM3fZBm5cGySGd eGNCPBA8ELx97GUcjiksctWk6nuHHM2wCv3QthElGyKO4/vrdtyTzoHP1XyXcDL8hxjs AygV52KgAmE42McMPh9i0DfmKZxNUA4pmEY/3dWQq2U7f5cqJoVVMb6UIgpOLG9aidCm bWUl3+8XgUv/C7Nlpb2oj4bxAmMz7uTKizcpH08a1ndlqmbCVK1wTX2F1KS8xTNEq3BW 2/iQ== X-Gm-Message-State: ANoB5plTy0SW38+qQJTnbAPBNc5SuPazyfdTU6eFUS1zgnh6VtoZQ6fY EAnSQs266Ufe/Z5UhEcZdIQ= X-Google-Smtp-Source: AA0mqf7Klwt7zuSS5Bc0Ck/gVlzg6wPps8ApHYVA792N13f9ScivIPhHN4zDt0S1CpfGCmrimlbxdw== X-Received: by 2002:a17:902:d88d:b0:186:cb27:4e01 with SMTP id b13-20020a170902d88d00b00186cb274e01mr2334256plz.139.1668829613896; Fri, 18 Nov 2022 19:46:53 -0800 (PST) Received: from ubuntu-Virtual-Machine.corp.microsoft.com ([2001:4898:80e8:38:f087:1794:92c5:f8f0]) by smtp.gmail.com with ESMTPSA id e5-20020a056a0000c500b005360da6b26bsm3913892pfj.159.2022.11.18.19.46.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Nov 2022 19:46:53 -0800 (PST) From: Tianyu Lan To: luto@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, seanjc@google.com, pbonzini@redhat.com, jgross@suse.com, tiala@microsoft.com, kirill@shutemov.name, jiangshan.ljs@antgroup.com, peterz@infradead.org, ashish.kalra@amd.com, srutherford@google.com, akpm@linux-foundation.org, anshuman.khandual@arm.com, pawan.kumar.gupta@linux.intel.com, adrian.hunter@intel.com, daniel.sneddon@linux.intel.com, alexander.shishkin@linux.intel.com, sandipan.das@amd.com, ray.huang@amd.com, brijesh.singh@amd.com, michael.roth@amd.com, thomas.lendacky@amd.com, venu.busireddy@oracle.com, sterritt@google.com, tony.luck@intel.com, samitolvanen@google.com, fenghua.yu@intel.com Cc: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-hyperv@vger.kernel.org, linux-arch@vger.kernel.org Subject: [RFC PATCH V2 11/18] Drivers: hv: vmbus: Decrypt vmbus ring buffer Date: Fri, 18 Nov 2022 22:46:25 -0500 Message-Id: <20221119034633.1728632-12-ltykernel@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221119034633.1728632-1-ltykernel@gmail.com> References: <20221119034633.1728632-1-ltykernel@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Tianyu Lan The ring buffer is remapped in the hv_ringbuffer_init() and it should be with decrypt flag in order to share it with hypervisor in sev-snp enlightened guest. Signed-off-by: Tianyu Lan --- drivers/hv/ring_buffer.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/drivers/hv/ring_buffer.c b/drivers/hv/ring_buffer.c index 59a4aa86d1f3..391995c76be7 100644 --- a/drivers/hv/ring_buffer.c +++ b/drivers/hv/ring_buffer.c @@ -18,6 +18,7 @@ #include #include #include +#include #include #include "hyperv_vmbus.h" @@ -233,14 +234,18 @@ int hv_ringbuffer_init(struct hv_ring_buffer_info *ring_info, ring_info->ring_buffer = (struct hv_ring_buffer *) vmap(pages_wraparound, page_cnt * 2 - 1, VM_MAP, + hv_isolation_type_en_snp() ? + pgprot_decrypted(PAGE_KERNEL_NOENC) : PAGE_KERNEL); + if (hv_isolation_type_en_snp()) + memset(ring_info->ring_buffer, 0x00, page_cnt * PAGE_SIZE); + kfree(pages_wraparound); if (!ring_info->ring_buffer) return -ENOMEM; } - ring_info->ring_buffer->read_index = ring_info->ring_buffer->write_index = 0; From patchwork Sat Nov 19 03:46:26 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tianyu Lan X-Patchwork-Id: 13049567 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0BC2BC433FE for ; Sat, 19 Nov 2022 03:48:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232388AbiKSDsJ (ORCPT ); Fri, 18 Nov 2022 22:48:09 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37380 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230118AbiKSDrH (ORCPT ); Fri, 18 Nov 2022 22:47:07 -0500 Received: from mail-pf1-x42a.google.com (mail-pf1-x42a.google.com [IPv6:2607:f8b0:4864:20::42a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6EF31BFF7B; Fri, 18 Nov 2022 19:46:55 -0800 (PST) Received: by mail-pf1-x42a.google.com with SMTP id q9so6648336pfg.5; Fri, 18 Nov 2022 19:46:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=w0YgLiOl0S0PjO9RGU6dEILmddfdRljq1tR8KTSAHbs=; b=cVPhlzU8eK/Pt3xDnHgsJ/EHF7L2/ZjJbr+pz3d5HqMIYeseRIZI1jgD6y0biQtosF rtieszzeSK7hKuyl2HXeDG4Mrwi/qYTrDyrOpoRqjlJ9OCEO/gZ0t36hMi7G7I5Y6dTj MoJ+c32mi1FDPxQdD/0+oaI+GcXzbze8q+bhg60VRpC0McbMEtP/RgOk0BNMhOeYLdsH lDL7T8zBTE8mVFKGK7I00Nizec9pcy6C36Q4BMB/5vjKDUSS1pZfvjJFIHqc5/jGbP2e trK5AoClykpbtJhpDVGaRNSyKMA0i4lSlX74HZwdOmswu+YOpcuTuJRZ5sQn+XZvNEKJ DE0g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=w0YgLiOl0S0PjO9RGU6dEILmddfdRljq1tR8KTSAHbs=; b=47yInzjKn1nxI5LL8RFim1P4KZku5EwKLsUS683dhBrr5aqsqP01rGsAzOE7LgI8SI 0rwyAX5VPdsT6FKTmUO+0zA2fUuwiYQpAq1hMIAXuv9hXBd7RVSHKNlILheXA/Z2gekQ pVZrswupKZt4sKmXVvhdoZ+ByxdfsVLWW1U6CBMuV/Oe7HZVj5XjkHWUnGpbWypj3+yz xoK4OlU1aFP3N3QQbMTYr1TEv2zhlPWa+tQKyoINHG9dLSBfvmvjWd4LddTFsIqRL8qc Mvtbqb0l9/Eu4Vzl78LaHY0+gt/+5xTD5m4sVvOeKZJR/KxGLhXZQhQGHm54G+fnEHI+ To9g== X-Gm-Message-State: ANoB5pkDNJvfuu/yJ16zOufayeRpiepujlKjn1WRc5jr9qXZgYpLx26T OOBkUp19cyz1LE5dMajPJ8hGEb3YI1X3VA== X-Google-Smtp-Source: AA0mqf56dEhRWdRWvzLyGPnEqRp10609+I0je6wOhyvoa8UBAoHSVxZY65PfscIPqv8kvCLC9fZVMA== X-Received: by 2002:a63:4e4c:0:b0:46f:dc59:aab6 with SMTP id o12-20020a634e4c000000b0046fdc59aab6mr9347136pgl.35.1668829615127; Fri, 18 Nov 2022 19:46:55 -0800 (PST) Received: from ubuntu-Virtual-Machine.corp.microsoft.com ([2001:4898:80e8:38:f087:1794:92c5:f8f0]) by smtp.gmail.com with ESMTPSA id e5-20020a056a0000c500b005360da6b26bsm3913892pfj.159.2022.11.18.19.46.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Nov 2022 19:46:54 -0800 (PST) From: Tianyu Lan To: luto@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, seanjc@google.com, pbonzini@redhat.com, jgross@suse.com, tiala@microsoft.com, kirill@shutemov.name, jiangshan.ljs@antgroup.com, peterz@infradead.org, ashish.kalra@amd.com, srutherford@google.com, akpm@linux-foundation.org, anshuman.khandual@arm.com, pawan.kumar.gupta@linux.intel.com, adrian.hunter@intel.com, daniel.sneddon@linux.intel.com, alexander.shishkin@linux.intel.com, sandipan.das@amd.com, ray.huang@amd.com, brijesh.singh@amd.com, michael.roth@amd.com, thomas.lendacky@amd.com, venu.busireddy@oracle.com, sterritt@google.com, tony.luck@intel.com, samitolvanen@google.com, fenghua.yu@intel.com Cc: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-hyperv@vger.kernel.org, linux-arch@vger.kernel.org Subject: [RFC PATCH V2 12/18] x86/hyperv: Initialize cpu and memory for sev-snp enlightened guest Date: Fri, 18 Nov 2022 22:46:26 -0500 Message-Id: <20221119034633.1728632-13-ltykernel@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221119034633.1728632-1-ltykernel@gmail.com> References: <20221119034633.1728632-1-ltykernel@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Tianyu Lan Read processor amd memory info from specific address which are populated by Hyper-V. Initialize smp cpu related ops, pvalidate system memory and add it into e820 table. Signed-off-by: Tianyu Lan --- arch/x86/kernel/cpu/mshyperv.c | 75 ++++++++++++++++++++++++++++++++++ 1 file changed, 75 insertions(+) diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c index 2ea4f21c6172..f0c97210c64a 100644 --- a/arch/x86/kernel/cpu/mshyperv.c +++ b/arch/x86/kernel/cpu/mshyperv.c @@ -34,6 +34,12 @@ #include #include #include +#include +#include +#include +#include +#include +#include /* Is Linux running as the root partition? */ bool hv_root_partition; @@ -253,6 +259,33 @@ static void __init hv_smp_prepare_cpus(unsigned int max_cpus) } #endif +static __init int hv_snp_set_rtc_noop(const struct timespec64 *now) { return -EINVAL; } +static __init void hv_snp_get_rtc_noop(struct timespec64 *now) { } + +static u32 processor_count; + +static __init void hv_snp_get_smp_config(unsigned int early) +{ + if (!early) { + while (num_processors < processor_count) { + early_per_cpu(x86_cpu_to_apicid, num_processors) = num_processors; + early_per_cpu(x86_bios_cpu_apicid, num_processors) = num_processors; + physid_set(num_processors, phys_cpu_present_map); + set_cpu_possible(num_processors, true); + set_cpu_present(num_processors, true); + num_processors++; + } + } +} + +struct memory_map_entry { + u64 starting_gpn; + u64 numpages; + u16 type; + u16 flags; + u32 reserved; +}; + static void __init ms_hyperv_init_platform(void) { int hv_max_functions_eax; @@ -260,6 +293,11 @@ static void __init ms_hyperv_init_platform(void) int hv_host_info_ebx; int hv_host_info_ecx; int hv_host_info_edx; + struct memory_map_entry *entry; + struct e820_entry *e820_entry; + u64 e820_end; + u64 ram_end; + u64 page; #ifdef CONFIG_PARAVIRT pv_info.name = "Hyper-V"; @@ -477,6 +515,43 @@ static void __init ms_hyperv_init_platform(void) if (!(ms_hyperv.features & HV_ACCESS_TSC_INVARIANT)) mark_tsc_unstable("running on Hyper-V"); + if (cc_platform_has(CC_ATTR_GUEST_SEV_SNP)) { + x86_platform.legacy.reserve_bios_regions = 0; + x86_platform.set_wallclock = hv_snp_set_rtc_noop; + x86_platform.get_wallclock = hv_snp_get_rtc_noop; + x86_init.resources.probe_roms = x86_init_noop; + x86_init.resources.reserve_resources = x86_init_noop; + x86_init.mpparse.find_smp_config = x86_init_noop; + x86_init.mpparse.get_smp_config = hv_snp_get_smp_config; + + /* + * Hyper-V SEV-SNP enlightened guest doesn't support ioapic + * and legacy APIC page read/write. Switch to hv apic here. + */ + disable_ioapic_support(); + hv_apic_init(); + + processor_count = *(u32 *)__va(EN_SEV_SNP_PROCESSOR_INFO_ADDR); + + entry = (struct memory_map_entry *)(__va(EN_SEV_SNP_PROCESSOR_INFO_ADDR) + + sizeof(struct memory_map_entry)); + + for (; entry->numpages != 0; entry++) { + e820_entry = &e820_table->entries[e820_table->nr_entries - 1]; + e820_end = e820_entry->addr + e820_entry->size; + ram_end = (entry->starting_gpn + entry->numpages) * PAGE_SIZE; + + if (e820_end < entry->starting_gpn * PAGE_SIZE) + e820_end = entry->starting_gpn * PAGE_SIZE; + if (e820_end < ram_end) { + pr_info("Hyper-V: add [mem %#018Lx-%#018Lx]\n", e820_end, ram_end - 1); + e820__range_add(e820_end, ram_end - e820_end, E820_TYPE_RAM); + for (page = e820_end; page < ram_end; page += PAGE_SIZE) + pvalidate((unsigned long)__va(page), RMP_PG_SIZE_4K, true); + } + } + } + hardlockup_detector_disable(); } From patchwork Sat Nov 19 03:46:27 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tianyu Lan X-Patchwork-Id: 13049568 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3D555C4332F for ; Sat, 19 Nov 2022 03:48:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233369AbiKSDsM (ORCPT ); Fri, 18 Nov 2022 22:48:12 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37812 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232440AbiKSDrY (ORCPT ); Fri, 18 Nov 2022 22:47:24 -0500 Received: from mail-pj1-x102e.google.com (mail-pj1-x102e.google.com [IPv6:2607:f8b0:4864:20::102e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 75F8EC0509; Fri, 18 Nov 2022 19:46:57 -0800 (PST) Received: by mail-pj1-x102e.google.com with SMTP id u6-20020a17090a5e4600b0021881a8d264so4439856pji.4; Fri, 18 Nov 2022 19:46:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=KHQxOX05VuPCuHlsy0vXM0F/fWibSFir6W5PfPRogHE=; b=LurCtGOTimpppiFgfFLC+mvau7w+LbSFf7nML82heGkU4TpPwT6r4Z5Dluce9mOMIO Baa7ezuu/z3QvfpMKzXcywlfMMEPwxmf+nOCPtcsR2NgzZzZmRNeJSBJ6O/Psi5eWLjT 3MstR0iFkuCGpspoSRDTrOoRwGs5kBs47mkrKGDzBKaiy5Ua/F99LcG5OKTRkES9yEHW FpPAziI1jvoB1nLFE2rfz2eVDCBNJudgIlvh63/ArxnOmbxeOq3pLCvVnvgsae5Y2r1o ZnDR2SZcDUhXZujg+qBgBKbszn85MJpp5nX8IqF3uSQjGPjvJYnKo6EDdEShFVyB3op6 dF8Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=KHQxOX05VuPCuHlsy0vXM0F/fWibSFir6W5PfPRogHE=; b=djjtNqCgvhZDGdQ3b6w/I963lajcedw5kQznX+1fhcNp+LWnExH9CSlrbkg8pYYcdw DnLEVzv6iLkUHDs9KW0v/EM93EFlmZVdVKISQuVmB66VixaeVJa8wiY/ZVWFB9mPPvrX c7uDvakhbP+eQi8upIIHryDR+VCP+03BkBLq+sQfnwEevv+KF5hii5IcRQS+GJgzBHQX SSZF9vBoY8VU9V3cIKJHYddsxW+rZGu2PTdopxaXYf2b4mRyfbgKO+PotKWzxyhiBPi8 C0WNzwZwkWMcMbmH5a1Lyjt5+tjRaEj7bD5tYSL6SemghLcobsHJs55DezsB1wA8Z7Sq uKuA== X-Gm-Message-State: ANoB5pnJG0UrmJ2WmUgtkoBFyXImpAczwYH9wimRHVDFJOFcULkXLBQo cVPMKtB9PHqe/rV48zP5U5k= X-Google-Smtp-Source: AA0mqf651nM/dyyHrkwYXVpAbqGU5wBTXfmI0Wb5XQxX19TGXSUlTgSes/dumy4Q3TYpahrP8DSQXA== X-Received: by 2002:a17:902:820b:b0:188:9ae7:bb7d with SMTP id x11-20020a170902820b00b001889ae7bb7dmr2432017pln.113.1668829616513; Fri, 18 Nov 2022 19:46:56 -0800 (PST) Received: from ubuntu-Virtual-Machine.corp.microsoft.com ([2001:4898:80e8:38:f087:1794:92c5:f8f0]) by smtp.gmail.com with ESMTPSA id e5-20020a056a0000c500b005360da6b26bsm3913892pfj.159.2022.11.18.19.46.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Nov 2022 19:46:55 -0800 (PST) From: Tianyu Lan To: luto@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, seanjc@google.com, pbonzini@redhat.com, jgross@suse.com, tiala@microsoft.com, kirill@shutemov.name, jiangshan.ljs@antgroup.com, peterz@infradead.org, ashish.kalra@amd.com, srutherford@google.com, akpm@linux-foundation.org, anshuman.khandual@arm.com, pawan.kumar.gupta@linux.intel.com, adrian.hunter@intel.com, daniel.sneddon@linux.intel.com, alexander.shishkin@linux.intel.com, sandipan.das@amd.com, ray.huang@amd.com, brijesh.singh@amd.com, michael.roth@amd.com, thomas.lendacky@amd.com, venu.busireddy@oracle.com, sterritt@google.com, tony.luck@intel.com, samitolvanen@google.com, fenghua.yu@intel.com Cc: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-hyperv@vger.kernel.org, linux-arch@vger.kernel.org Subject: [RFC PATCH V2 13/18] x86/hyperv: Add smp support for sev-snp guest Date: Fri, 18 Nov 2022 22:46:27 -0500 Message-Id: <20221119034633.1728632-14-ltykernel@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221119034633.1728632-1-ltykernel@gmail.com> References: <20221119034633.1728632-1-ltykernel@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Tianyu Lan The wakeup_secondary_cpu callback was populated with wakeup_ cpu_via_vmgexit() which doesn't work for Hyper-V. Override it with Hyper-V specific hook which uses HVCALL_START_VIRTUAL_ PROCESSOR hvcall to start AP with vmsa data structure. Signed-off-by: Tianyu Lan --- arch/x86/include/asm/sev.h | 13 +++ arch/x86/include/asm/svm.h | 55 ++++++++++- arch/x86/kernel/cpu/mshyperv.c | 147 +++++++++++++++++++++++++++++- include/asm-generic/hyperv-tlfs.h | 18 ++++ 4 files changed, 230 insertions(+), 3 deletions(-) diff --git a/arch/x86/include/asm/sev.h b/arch/x86/include/asm/sev.h index ebc271bb6d8e..e34aaf730220 100644 --- a/arch/x86/include/asm/sev.h +++ b/arch/x86/include/asm/sev.h @@ -86,6 +86,19 @@ extern bool handle_vc_boot_ghcb(struct pt_regs *regs); #define RMPADJUST_VMSA_PAGE_BIT BIT(16) +union sev_rmp_adjust { + u64 as_uint64; + struct { + unsigned long target_vmpl : 8; + unsigned long enable_read : 1; + unsigned long enable_write : 1; + unsigned long enable_user_execute : 1; + unsigned long enable_kernel_execute : 1; + unsigned long reserved1 : 4; + unsigned long vmsa : 1; + }; +}; + /* SNP Guest message request */ struct snp_req_data { unsigned long req_gpa; diff --git a/arch/x86/include/asm/svm.h b/arch/x86/include/asm/svm.h index 0361626841bc..fc54d3e7f817 100644 --- a/arch/x86/include/asm/svm.h +++ b/arch/x86/include/asm/svm.h @@ -328,8 +328,61 @@ struct vmcb_save_area { u64 br_to; u64 last_excp_from; u64 last_excp_to; - u8 reserved_6[72]; + + /* + * The following part of the save area is valid only for + * SEV-ES guests when referenced through the GHCB or for + * saving to the host save area. + */ + u8 reserved_7[72]; u32 spec_ctrl; /* Guest version of SPEC_CTRL at 0x2E0 */ + u8 reserved_7b[4]; + u32 pkru; + u8 reserved_7a[20]; + u64 reserved_8; /* rax already available at 0x01f8 */ + u64 rcx; + u64 rdx; + u64 rbx; + u64 reserved_9; /* rsp already available at 0x01d8 */ + u64 rbp; + u64 rsi; + u64 rdi; + u64 r8; + u64 r9; + u64 r10; + u64 r11; + u64 r12; + u64 r13; + u64 r14; + u64 r15; + u8 reserved_10[16]; + u64 sw_exit_code; + u64 sw_exit_info_1; + u64 sw_exit_info_2; + u64 sw_scratch; + union { + u64 sev_features; + struct { + u64 sev_feature_snp : 1; + u64 sev_feature_vtom : 1; + u64 sev_feature_reflectvc : 1; + u64 sev_feature_restrict_injection : 1; + u64 sev_feature_alternate_injection : 1; + u64 sev_feature_full_debug : 1; + u64 sev_feature_reserved1 : 1; + u64 sev_feature_snpbtb_isolation : 1; + u64 sev_feature_resrved2 : 56; + }; + }; + u64 vintr_ctrl; + u64 guest_error_code; + u64 virtual_tom; + u64 tlb_id; + u64 pcpu_id; + u64 event_inject; + u64 xcr0; + u8 valid_bitmap[16]; + u64 x87_state_gpa; } __packed; /* Save area definition for SEV-ES and SEV-SNP guests */ diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c index f0c97210c64a..b266f648e5cd 100644 --- a/arch/x86/kernel/cpu/mshyperv.c +++ b/arch/x86/kernel/cpu/mshyperv.c @@ -41,6 +41,10 @@ #include #include +#define EN_SEV_SNP_PROCESSOR_INFO_ADDR 0x802000 +#define HV_AP_INIT_GPAT_DEFAULT 0x0007040600070406ULL +#define HV_AP_SEGMENT_LIMIT 0xffffffff + /* Is Linux running as the root partition? */ bool hv_root_partition; struct ms_hyperv_info ms_hyperv; @@ -232,6 +236,136 @@ static void __init hv_smp_prepare_boot_cpu(void) #endif } +static u8 ap_start_input_arg[PAGE_SIZE] __bss_decrypted __aligned(PAGE_SIZE); +static u8 ap_start_stack[PAGE_SIZE] __aligned(PAGE_SIZE); + +int hv_snp_boot_ap(int cpu, unsigned long start_ip) +{ + struct vmcb_save_area *vmsa = (struct vmcb_save_area *) + __get_free_page(GFP_KERNEL | __GFP_ZERO); + struct desc_ptr gdtr; + u64 ret, retry = 5; + struct hv_enable_vp_vtl_input *enable_vtl_input; + struct hv_start_virtual_processor_input *start_vp_input; + union sev_rmp_adjust rmp_adjust; + void **arg; + unsigned long flags; + + *(void **)per_cpu_ptr(hyperv_pcpu_input_arg, cpu) = ap_start_input_arg; + + hv_vp_index[cpu] = cpu; + + /* Prevent APs from entering busy calibration loop */ + preset_lpj = lpj_fine; + + /* Replace the provided real-mode start_ip */ + start_ip = (unsigned long)secondary_startup_64_no_verify; + + native_store_gdt(&gdtr); + + vmsa->gdtr.base = gdtr.address; + vmsa->gdtr.limit = gdtr.size; + + asm volatile("movl %%es, %%eax;" : "=a" (vmsa->es.selector)); + if (vmsa->es.selector) { + vmsa->es.base = 0; + vmsa->es.limit = HV_AP_SEGMENT_LIMIT; + vmsa->es.attrib = *(u16 *)(vmsa->gdtr.base + vmsa->es.selector + 5); + vmsa->es.attrib = (vmsa->es.attrib & 0xFF) | ((vmsa->es.attrib >> 4) & 0xF00); + } + + asm volatile("movl %%cs, %%eax;" : "=a" (vmsa->cs.selector)); + if (vmsa->cs.selector) { + vmsa->cs.base = 0; + vmsa->cs.limit = HV_AP_SEGMENT_LIMIT; + vmsa->cs.attrib = *(u16 *)(vmsa->gdtr.base + vmsa->cs.selector + 5); + vmsa->cs.attrib = (vmsa->cs.attrib & 0xFF) | ((vmsa->cs.attrib >> 4) & 0xF00); + } + + asm volatile("movl %%ss, %%eax;" : "=a" (vmsa->ss.selector)); + if (vmsa->ss.selector) { + vmsa->ss.base = 0; + vmsa->ss.limit = HV_AP_SEGMENT_LIMIT; + vmsa->ss.attrib = *(u16 *)(vmsa->gdtr.base + vmsa->ss.selector + 5); + vmsa->ss.attrib = (vmsa->ss.attrib & 0xFF) | ((vmsa->ss.attrib >> 4) & 0xF00); + } + + asm volatile("movl %%ds, %%eax;" : "=a" (vmsa->ds.selector)); + if (vmsa->ds.selector) { + vmsa->ds.base = 0; + vmsa->ds.limit = HV_AP_SEGMENT_LIMIT; + vmsa->ds.attrib = *(u16 *)(vmsa->gdtr.base + vmsa->ds.selector + 5); + vmsa->ds.attrib = (vmsa->ds.attrib & 0xFF) | ((vmsa->ds.attrib >> 4) & 0xF00); + } + + vmsa->efer = native_read_msr(MSR_EFER); + + asm volatile("movq %%cr4, %%rax;" : "=a" (vmsa->cr4)); + asm volatile("movq %%cr3, %%rax;" : "=a" (vmsa->cr3)); + asm volatile("movq %%cr0, %%rax;" : "=a" (vmsa->cr0)); + + vmsa->xcr0 = 1; + vmsa->g_pat = HV_AP_INIT_GPAT_DEFAULT; + vmsa->rip = (u64)start_ip; + vmsa->rsp = (u64)&ap_start_stack[PAGE_SIZE]; + + vmsa->sev_feature_snp = 1; + vmsa->sev_feature_restrict_injection = 1; + + rmp_adjust.as_uint64 = 0; + rmp_adjust.target_vmpl = 1; + rmp_adjust.vmsa = 1; + ret = rmpadjust((unsigned long)vmsa, RMP_PG_SIZE_4K, + rmp_adjust.as_uint64); + if (ret != 0) { + pr_err("RMPADJUST(%llx) failed: %llx\n", (u64)vmsa, ret); + return ret; + } + + local_irq_save(flags); + arg = (void **)this_cpu_ptr(hyperv_pcpu_input_arg); + if (unlikely(!*arg)) { + ret = -ENOMEM; + goto done; + } + + if (ms_hyperv.vtl != 0) { + enable_vtl_input = (struct hv_enable_vp_vtl_input *)*arg; + memset(enable_vtl_input, 0, sizeof(*enable_vtl_input)); + enable_vtl_input->partitionid = -1; + enable_vtl_input->vpindex = cpu; + enable_vtl_input->targetvtl = ms_hyperv.vtl; + *(u64 *)&enable_vtl_input->context[0] = __pa(vmsa) | 1; + + ret = hv_do_hypercall(HVCALL_ENABLE_VP_VTL, enable_vtl_input, NULL); + if (ret != 0) { + pr_err("HvCallEnableVpVtl failed: %llx\n", ret); + goto done; + } + } + + start_vp_input = (struct hv_start_virtual_processor_input *)*arg; + memset(start_vp_input, 0, sizeof(*start_vp_input)); + start_vp_input->partitionid = -1; + start_vp_input->vpindex = cpu; + start_vp_input->targetvtl = ms_hyperv.vtl; + *(u64 *)&start_vp_input->context[0] = __pa(vmsa) | 1; + + do { + ret = hv_do_hypercall(HVCALL_START_VIRTUAL_PROCESSOR, + start_vp_input, NULL); + } while (ret == HV_STATUS_TIME_OUT && retry--); + + if (ret != 0) { + pr_err("HvCallStartVirtualProcessor failed: %llx\n", ret); + goto done; + } + +done: + local_irq_restore(flags); + return ret; +} + static void __init hv_smp_prepare_cpus(unsigned int max_cpus) { #ifdef CONFIG_X86_64 @@ -241,6 +375,16 @@ static void __init hv_smp_prepare_cpus(unsigned int max_cpus) native_smp_prepare_cpus(max_cpus); + /* + * Override wakeup_secondary_cpu callback for SEV-SNP + * enlightened guest. + */ + if (hv_isolation_type_en_snp()) + apic->wakeup_secondary_cpu = hv_snp_boot_ap; + + if (!hv_root_partition) + return; + #ifdef CONFIG_X86_64 for_each_present_cpu(i) { if (i == 0) @@ -489,8 +633,7 @@ static void __init ms_hyperv_init_platform(void) # ifdef CONFIG_SMP smp_ops.smp_prepare_boot_cpu = hv_smp_prepare_boot_cpu; - if (hv_root_partition) - smp_ops.smp_prepare_cpus = hv_smp_prepare_cpus; + smp_ops.smp_prepare_cpus = hv_smp_prepare_cpus; # endif /* diff --git a/include/asm-generic/hyperv-tlfs.h b/include/asm-generic/hyperv-tlfs.h index 6e2a090e2649..7072adbf5540 100644 --- a/include/asm-generic/hyperv-tlfs.h +++ b/include/asm-generic/hyperv-tlfs.h @@ -139,6 +139,7 @@ struct ms_hyperv_tsc_page { #define HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST 0x0003 #define HVCALL_NOTIFY_LONG_SPIN_WAIT 0x0008 #define HVCALL_SEND_IPI 0x000b +#define HVCALL_ENABLE_VP_VTL 0x000f #define HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE_EX 0x0013 #define HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST_EX 0x0014 #define HVCALL_SEND_IPI_EX 0x0015 @@ -156,6 +157,7 @@ struct ms_hyperv_tsc_page { #define HVCALL_MAP_DEVICE_INTERRUPT 0x007c #define HVCALL_UNMAP_DEVICE_INTERRUPT 0x007d #define HVCALL_RETARGET_INTERRUPT 0x007e +#define HVCALL_START_VIRTUAL_PROCESSOR 0x0099 #define HVCALL_FLUSH_GUEST_PHYSICAL_ADDRESS_SPACE 0x00af #define HVCALL_FLUSH_GUEST_PHYSICAL_ADDRESS_LIST 0x00b0 #define HVCALL_MODIFY_SPARSE_GPA_PAGE_HOST_VISIBILITY 0x00db @@ -763,6 +765,22 @@ struct hv_input_unmap_device_interrupt { struct hv_interrupt_entry interrupt_entry; } __packed; +struct hv_enable_vp_vtl_input { + u64 partitionid; + u32 vpindex; + u8 targetvtl; + u8 padding[3]; + u8 context[0xe0]; +} __packed; + +struct hv_start_virtual_processor_input { + u64 partitionid; + u32 vpindex; + u8 targetvtl; + u8 padding[3]; + u8 context[0xe0]; +} __packed; + #define HV_SOURCE_SHADOW_NONE 0x0 #define HV_SOURCE_SHADOW_BRIDGE_BUS_RANGE 0x1 From patchwork Sat Nov 19 03:46:28 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tianyu Lan X-Patchwork-Id: 13049569 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BA8E1C433FE for ; Sat, 19 Nov 2022 03:48:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229670AbiKSDsi (ORCPT ); Fri, 18 Nov 2022 22:48:38 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37954 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232544AbiKSDra (ORCPT ); Fri, 18 Nov 2022 22:47:30 -0500 Received: from mail-pf1-x433.google.com (mail-pf1-x433.google.com [IPv6:2607:f8b0:4864:20::433]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7DFF5BF803; Fri, 18 Nov 2022 19:46:58 -0800 (PST) Received: by mail-pf1-x433.google.com with SMTP id d192so6681472pfd.0; Fri, 18 Nov 2022 19:46:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Dn38defXSzafrqFyjuRL6cm4llsqPTH+TKtsPBjqhmM=; b=U9SmzWmMPdi+GTCOWAVMDuYMa2ra4/VmRWt71/144RtiJ/lAeGqiEUf9zUG9O851GN TJTrQSmKkbDgG/zbFAqoVnUE47fp9hSqWBdJQ6Fv//j/0fK8ihzrAneaDGhjiq7frp5M BsyLUZGfsDZWfuQ0XBXbjpn/lAcw+PpQBFDlfCq/3XnaRu4NVibrG5xg2/87bDApaG+M MZI+e9CkPlIpOjgQ6vQELW1E1GVk/PdEN1xDcCy9P/gOPJJkUxelSA9qe3lJOc0FMnq+ 6Jl2i1NzBcSaiRzkNyEIKXum6edHYgSDykoZklpFl2nInrceTYIdMfvfSOpN3HutCEGK moWg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Dn38defXSzafrqFyjuRL6cm4llsqPTH+TKtsPBjqhmM=; b=HhbLCJDiTGgDWgmMaQQoZIjDmE0h6Dngl4GQDTTe457bmZvDlwoG2vGgmizjtZ0koa iuVssxUCUtTGfEJ+lDFwZXn8bb9s+jwdn/D7r5HzdY1erC/QnpNF2nyh21VBAb+r3Kbd NtNxtXhozTjxEGeuRWQ9kncrOEc930VnGYeHhSQfdJFvstAdyIKnk/4K5XhuygYQ7YCJ 252yYFxjS3vSCMfswDYKD2ttsUg7WVGwdhLwgvhCxu6vI2ADHYmWRL9UGRyeqFoeHlQn wtNcfXx/9iVy/qAclBNd+3CNAMc+YEXMwB4fDygCwwMbpDeMwoQRusSrH+7Z8ylYt1Be DWZg== X-Gm-Message-State: ANoB5pkfK+wjjK36MbohjrCeu7D0FZv4AcfuBVkERGIXMMUHs1ubhSbl XIkL/RclgllYCt8tAdlKmVc= X-Google-Smtp-Source: AA0mqf5+5sUTrnmfWZqryMJVyu9lhL8pbYtiSGhKCtpMEVtjuBaAVD20Qoa02B7aO8gczuxUSv/olg== X-Received: by 2002:a63:48b:0:b0:43b:f6df:3356 with SMTP id 133-20020a63048b000000b0043bf6df3356mr9461903pge.352.1668829617938; Fri, 18 Nov 2022 19:46:57 -0800 (PST) Received: from ubuntu-Virtual-Machine.corp.microsoft.com ([2001:4898:80e8:38:f087:1794:92c5:f8f0]) by smtp.gmail.com with ESMTPSA id e5-20020a056a0000c500b005360da6b26bsm3913892pfj.159.2022.11.18.19.46.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Nov 2022 19:46:57 -0800 (PST) From: Tianyu Lan To: luto@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, seanjc@google.com, pbonzini@redhat.com, jgross@suse.com, tiala@microsoft.com, kirill@shutemov.name, jiangshan.ljs@antgroup.com, peterz@infradead.org, ashish.kalra@amd.com, srutherford@google.com, akpm@linux-foundation.org, anshuman.khandual@arm.com, pawan.kumar.gupta@linux.intel.com, adrian.hunter@intel.com, daniel.sneddon@linux.intel.com, alexander.shishkin@linux.intel.com, sandipan.das@amd.com, ray.huang@amd.com, brijesh.singh@amd.com, michael.roth@amd.com, thomas.lendacky@amd.com, venu.busireddy@oracle.com, sterritt@google.com, tony.luck@intel.com, samitolvanen@google.com, fenghua.yu@intel.com Cc: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-hyperv@vger.kernel.org, linux-arch@vger.kernel.org Subject: [RFC PATCH V2 14/18] x86/hyperv: Add hyperv-specific hadling for VMMCALL under SEV-ES Date: Fri, 18 Nov 2022 22:46:28 -0500 Message-Id: <20221119034633.1728632-15-ltykernel@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221119034633.1728632-1-ltykernel@gmail.com> References: <20221119034633.1728632-1-ltykernel@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Tianyu Lan Add Hyperv-specific handling for faults caused by VMMCALL instructions. Signed-off-by: Tianyu Lan --- arch/x86/kernel/cpu/mshyperv.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/arch/x86/kernel/cpu/mshyperv.c b/arch/x86/kernel/cpu/mshyperv.c index b266f648e5cd..a4e526378603 100644 --- a/arch/x86/kernel/cpu/mshyperv.c +++ b/arch/x86/kernel/cpu/mshyperv.c @@ -725,6 +725,20 @@ static bool __init ms_hyperv_msi_ext_dest_id(void) return eax & HYPERV_VS_PROPERTIES_EAX_EXTENDED_IOAPIC_RTE; } +static void hv_sev_es_hcall_prepare(struct ghcb *ghcb, struct pt_regs *regs) +{ + /* RAX and CPL are already in the GHCB */ + ghcb_set_rcx(ghcb, regs->cx); + ghcb_set_rdx(ghcb, regs->dx); + ghcb_set_r8(ghcb, regs->r8); +} + +static bool hv_sev_es_hcall_finish(struct ghcb *ghcb, struct pt_regs *regs) +{ + /* No checking of the return state needed */ + return true; +} + const __initconst struct hypervisor_x86 x86_hyper_ms_hyperv = { .name = "Microsoft Hyper-V", .detect = ms_hyperv_platform, @@ -732,4 +746,6 @@ const __initconst struct hypervisor_x86 x86_hyper_ms_hyperv = { .init.x2apic_available = ms_hyperv_x2apic_available, .init.msi_ext_dest_id = ms_hyperv_msi_ext_dest_id, .init.init_platform = ms_hyperv_init_platform, + .runtime.sev_es_hcall_prepare = hv_sev_es_hcall_prepare, + .runtime.sev_es_hcall_finish = hv_sev_es_hcall_finish, }; From patchwork Sat Nov 19 03:46:29 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tianyu Lan X-Patchwork-Id: 13049571 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id EDC5DC43217 for ; Sat, 19 Nov 2022 03:49:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233847AbiKSDtO (ORCPT ); Fri, 18 Nov 2022 22:49:14 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37998 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232385AbiKSDrc (ORCPT ); Fri, 18 Nov 2022 22:47:32 -0500 Received: from mail-pl1-x62d.google.com (mail-pl1-x62d.google.com [IPv6:2607:f8b0:4864:20::62d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DF09FC0530; Fri, 18 Nov 2022 19:46:59 -0800 (PST) Received: by mail-pl1-x62d.google.com with SMTP id j12so6190252plj.5; Fri, 18 Nov 2022 19:46:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=QBd9L0jrDkd3hAnI+wytLhl/qcmfoCXwePOoFw321CQ=; b=mOdH3S6ZgmtKKqMF3U3fs7M9wFXmEyvmMXv41DgT1UPkknA87jfp0qXM4XhMEKYjH9 +hhwo8SSyujF6XIaqli95AdEIw3DywecOIPJry3oUJ5Vg1QxSVdZNl83RkLB6cJCn4Eo lNTGvvHDdCyttvNsXfs6435tRjiyK0MTsd320lpqxe1P7YImZ68ZNcolH4DKDVPMF4+w XrYzH9S9IF2l9i/KrL3TEPxM48Q8UBBu7VFl7jcw5OPpx7zcMD8R8CbR1vVderj/kdto +Tu8WqHrkyPz6t+7OsPsyZDUFXbYIQsjbF3zLNZVEqSVa11Jh14O0NKIjk+5XwnbhEtO g3Iw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=QBd9L0jrDkd3hAnI+wytLhl/qcmfoCXwePOoFw321CQ=; b=CEZM0RZhrskFUxiL62tqBdocSealZZrTcMmE60X7RCuF8/3zqr0fy9vvVqA05a9GNj UjBTaywMf10ZQN5ooAMxv19MdXwdc+WABatm2JP7trh9r9JqrnXP4uRij4zhlUSPkTZD MxyOSAQtWA80H3GPVbGle9nL4iZUVgO4j5oxeMfzYiXSKDOwnz3i1ZY+zJKIeiNbXM0F qq/0uHfSmQbZf4/mfhRtyqSu9pK1t3hfgUZMrLbpNpgUSeyB9aFcdSPWYCOINCmawKq2 RzgT/6//vtRYXaCilYGLR2JS6KFR891WCsfQb7UvPPWFzz0ZqlVWLwdIz6QOPb9uPvZF DiSA== X-Gm-Message-State: ANoB5pkpmkjVaB/ITdTt1pwrMQ3iS/lEBA6EKVCZyWiYfEtusTw6w085 m3imfgLgcuT01IvE+0K/zg8= X-Google-Smtp-Source: AA0mqf6CTGiRx8Dpplql0ByG/d67Os8GZ+U85qbbYDNmMzmWrpQ9dLghk/iZv7aPzk5QFLYhyzVDDw== X-Received: by 2002:a17:902:f08c:b0:17c:892e:7c8e with SMTP id p12-20020a170902f08c00b0017c892e7c8emr2467334pla.92.1668829619173; Fri, 18 Nov 2022 19:46:59 -0800 (PST) Received: from ubuntu-Virtual-Machine.corp.microsoft.com ([2001:4898:80e8:38:f087:1794:92c5:f8f0]) by smtp.gmail.com with ESMTPSA id e5-20020a056a0000c500b005360da6b26bsm3913892pfj.159.2022.11.18.19.46.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Nov 2022 19:46:58 -0800 (PST) From: Tianyu Lan To: luto@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, seanjc@google.com, pbonzini@redhat.com, jgross@suse.com, tiala@microsoft.com, kirill@shutemov.name, jiangshan.ljs@antgroup.com, peterz@infradead.org, ashish.kalra@amd.com, srutherford@google.com, akpm@linux-foundation.org, anshuman.khandual@arm.com, pawan.kumar.gupta@linux.intel.com, adrian.hunter@intel.com, daniel.sneddon@linux.intel.com, alexander.shishkin@linux.intel.com, sandipan.das@amd.com, ray.huang@amd.com, brijesh.singh@amd.com, michael.roth@amd.com, thomas.lendacky@amd.com, venu.busireddy@oracle.com, sterritt@google.com, tony.luck@intel.com, samitolvanen@google.com, fenghua.yu@intel.com Cc: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-hyperv@vger.kernel.org, linux-arch@vger.kernel.org Subject: [RFC PATCH V2 15/18] x86/sev: Add a #HV exception handler Date: Fri, 18 Nov 2022 22:46:29 -0500 Message-Id: <20221119034633.1728632-16-ltykernel@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221119034633.1728632-1-ltykernel@gmail.com> References: <20221119034633.1728632-1-ltykernel@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Tianyu Lan Add a #HV exception handler that uses IST stack. Signed-off-by: Tianyu Lan --- arch/x86/entry/entry_64.S | 58 +++++++++++++++++++++++++++ arch/x86/include/asm/cpu_entry_area.h | 6 +++ arch/x86/include/asm/idtentry.h | 39 +++++++++++++++++- arch/x86/include/asm/page_64_types.h | 1 + arch/x86/include/asm/trapnr.h | 1 + arch/x86/include/asm/traps.h | 1 + arch/x86/kernel/cpu/common.c | 1 + arch/x86/kernel/dumpstack_64.c | 9 ++++- arch/x86/kernel/idt.c | 1 + arch/x86/kernel/sev.c | 53 ++++++++++++++++++++++++ arch/x86/mm/cpu_entry_area.c | 2 + 11 files changed, 169 insertions(+), 3 deletions(-) diff --git a/arch/x86/entry/entry_64.S b/arch/x86/entry/entry_64.S index 9953d966d124..b2059df43c57 100644 --- a/arch/x86/entry/entry_64.S +++ b/arch/x86/entry/entry_64.S @@ -560,6 +560,64 @@ SYM_CODE_START(\asmsym) .Lfrom_usermode_switch_stack_\@: idtentry_body user_\cfunc, has_error_code=1 +_ASM_NOKPROBE(\asmsym) +SYM_CODE_END(\asmsym) +.endm +/* + * idtentry_hv - Macro to generate entry stub for #HV + * @vector: Vector number + * @asmsym: ASM symbol for the entry point + * @cfunc: C function to be called + * + * The macro emits code to set up the kernel context for #HV. The #HV handler + * runs on an IST stack and needs to be able to support nested #HV exceptions. + * + * To make this work the #HV entry code tries its best to pretend it doesn't use + * an IST stack by switching to the task stack if coming from user-space (which + * includes early SYSCALL entry path) or back to the stack in the IRET frame if + * entered from kernel-mode. + * + * If entered from kernel-mode the return stack is validated first, and if it is + * not safe to use (e.g. because it points to the entry stack) the #HV handler + * will switch to a fall-back stack (HV2) and call a special handler function. + * + * The macro is only used for one vector, but it is planned to be extended in + * the future for the #HV exception. + */ +.macro idtentry_hv vector asmsym cfunc +SYM_CODE_START(\asmsym) + UNWIND_HINT_IRET_REGS + ASM_CLAC + pushq $-1 /* ORIG_RAX: no syscall to restart */ + + testb $3, CS-ORIG_RAX(%rsp) + jnz .Lfrom_usermode_switch_stack_\@ + + call paranoid_entry + + UNWIND_HINT_REGS + + /* + * Switch off the IST stack to make it free for nested exceptions. + */ + movq %rsp, %rdi /* pt_regs pointer */ + call hv_switch_off_ist + movq %rax, %rsp /* Switch to new stack */ + + UNWIND_HINT_REGS + + /* Update pt_regs */ + movq ORIG_RAX(%rsp), %rsi /* get error code into 2nd argument*/ + movq $-1, ORIG_RAX(%rsp) /* no syscall to restart */ + + movq %rsp, %rdi /* pt_regs pointer */ + call kernel_\cfunc + + jmp paranoid_exit + +.Lfrom_usermode_switch_stack_\@: + idtentry_body user_\cfunc, has_error_code=1 + _ASM_NOKPROBE(\asmsym) SYM_CODE_END(\asmsym) .endm diff --git a/arch/x86/include/asm/cpu_entry_area.h b/arch/x86/include/asm/cpu_entry_area.h index 75efc4c6f076..f173a16cfc59 100644 --- a/arch/x86/include/asm/cpu_entry_area.h +++ b/arch/x86/include/asm/cpu_entry_area.h @@ -30,6 +30,10 @@ char VC_stack[optional_stack_size]; \ char VC2_stack_guard[guardsize]; \ char VC2_stack[optional_stack_size]; \ + char HV_stack_guard[guardsize]; \ + char HV_stack[optional_stack_size]; \ + char HV2_stack_guard[guardsize]; \ + char HV2_stack[optional_stack_size]; \ char IST_top_guard[guardsize]; \ /* The exception stacks' physical storage. No guard pages required */ @@ -52,6 +56,8 @@ enum exception_stack_ordering { ESTACK_MCE, ESTACK_VC, ESTACK_VC2, + ESTACK_HV, + ESTACK_HV2, N_EXCEPTION_STACKS }; diff --git a/arch/x86/include/asm/idtentry.h b/arch/x86/include/asm/idtentry.h index 72184b0b2219..652fea10d377 100644 --- a/arch/x86/include/asm/idtentry.h +++ b/arch/x86/include/asm/idtentry.h @@ -317,6 +317,19 @@ static __always_inline void __##func(struct pt_regs *regs) __visible noinstr void kernel_##func(struct pt_regs *regs, unsigned long error_code); \ __visible noinstr void user_##func(struct pt_regs *regs, unsigned long error_code) + +/** + * DECLARE_IDTENTRY_HV - Declare functions for the HV entry point + * @vector: Vector number (ignored for C) + * @func: Function name of the entry point + * + * Maps to DECLARE_IDTENTRY_RAW, but declares also the user C handler. + */ +#define DECLARE_IDTENTRY_HV(vector, func) \ + DECLARE_IDTENTRY_RAW_ERRORCODE(vector, func); \ + __visible noinstr void kernel_##func(struct pt_regs *regs); \ + __visible noinstr void user_##func(struct pt_regs *regs) + /** * DEFINE_IDTENTRY_IST - Emit code for IST entry points * @func: Function name of the entry point @@ -376,6 +389,26 @@ static __always_inline void __##func(struct pt_regs *regs) #define DEFINE_IDTENTRY_VC_USER(func) \ DEFINE_IDTENTRY_RAW_ERRORCODE(user_##func) +/** + * DEFINE_IDTENTRY_HV_KERNEL - Emit code for HV injection handler + * when raised from kernel mode + * @func: Function name of the entry point + * + * Maps to DEFINE_IDTENTRY_RAW + */ +#define DEFINE_IDTENTRY_HV_KERNEL(func) \ + DEFINE_IDTENTRY_RAW(kernel_##func) + +/** + * DEFINE_IDTENTRY_HV_USER - Emit code for HV injection handler + * when raised from user mode + * @func: Function name of the entry point + * + * Maps to DEFINE_IDTENTRY_RAW + */ +#define DEFINE_IDTENTRY_HV_USER(func) \ + DEFINE_IDTENTRY_RAW(user_##func) + #else /* CONFIG_X86_64 */ /** @@ -465,6 +498,9 @@ __visible noinstr void func(struct pt_regs *regs, \ # define DECLARE_IDTENTRY_VC(vector, func) \ idtentry_vc vector asm_##func func +# define DECLARE_IDTENTRY_HV(vector, func) \ + idtentry_hv vector asm_##func func + #else # define DECLARE_IDTENTRY_MCE(vector, func) \ DECLARE_IDTENTRY(vector, func) @@ -622,9 +658,10 @@ DECLARE_IDTENTRY_RAW_ERRORCODE(X86_TRAP_DF, xenpv_exc_double_fault); DECLARE_IDTENTRY_ERRORCODE(X86_TRAP_CP, exc_control_protection); #endif -/* #VC */ +/* #VC & #HV */ #ifdef CONFIG_AMD_MEM_ENCRYPT DECLARE_IDTENTRY_VC(X86_TRAP_VC, exc_vmm_communication); +DECLARE_IDTENTRY_HV(X86_TRAP_HV, exc_hv_injection); #endif #ifdef CONFIG_XEN_PV diff --git a/arch/x86/include/asm/page_64_types.h b/arch/x86/include/asm/page_64_types.h index e9e2c3ba5923..0bd7dab676c5 100644 --- a/arch/x86/include/asm/page_64_types.h +++ b/arch/x86/include/asm/page_64_types.h @@ -29,6 +29,7 @@ #define IST_INDEX_DB 2 #define IST_INDEX_MCE 3 #define IST_INDEX_VC 4 +#define IST_INDEX_HV 5 /* * Set __PAGE_OFFSET to the most negative possible address + diff --git a/arch/x86/include/asm/trapnr.h b/arch/x86/include/asm/trapnr.h index f5d2325aa0b7..c6583631cecb 100644 --- a/arch/x86/include/asm/trapnr.h +++ b/arch/x86/include/asm/trapnr.h @@ -26,6 +26,7 @@ #define X86_TRAP_XF 19 /* SIMD Floating-Point Exception */ #define X86_TRAP_VE 20 /* Virtualization Exception */ #define X86_TRAP_CP 21 /* Control Protection Exception */ +#define X86_TRAP_HV 28 /* HV injected exception in SNP restricted mode */ #define X86_TRAP_VC 29 /* VMM Communication Exception */ #define X86_TRAP_IRET 32 /* IRET Exception */ diff --git a/arch/x86/include/asm/traps.h b/arch/x86/include/asm/traps.h index 47ecfff2c83d..6795d3e517d6 100644 --- a/arch/x86/include/asm/traps.h +++ b/arch/x86/include/asm/traps.h @@ -16,6 +16,7 @@ asmlinkage __visible notrace struct pt_regs *fixup_bad_iret(struct pt_regs *bad_regs); void __init trap_init(void); asmlinkage __visible noinstr struct pt_regs *vc_switch_off_ist(struct pt_regs *eregs); +asmlinkage __visible noinstr struct pt_regs *hv_switch_off_ist(struct pt_regs *eregs); #endif extern bool ibt_selftest(void); diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c index 3e508f239098..87afa3a4c8b1 100644 --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -2165,6 +2165,7 @@ static inline void tss_setup_ist(struct tss_struct *tss) tss->x86_tss.ist[IST_INDEX_MCE] = __this_cpu_ist_top_va(MCE); /* Only mapped when SEV-ES is active */ tss->x86_tss.ist[IST_INDEX_VC] = __this_cpu_ist_top_va(VC); + tss->x86_tss.ist[IST_INDEX_HV] = __this_cpu_ist_top_va(HV); } #else /* CONFIG_X86_64 */ diff --git a/arch/x86/kernel/dumpstack_64.c b/arch/x86/kernel/dumpstack_64.c index 6c5defd6569a..23aa5912c87a 100644 --- a/arch/x86/kernel/dumpstack_64.c +++ b/arch/x86/kernel/dumpstack_64.c @@ -26,11 +26,14 @@ static const char * const exception_stack_names[] = { [ ESTACK_MCE ] = "#MC", [ ESTACK_VC ] = "#VC", [ ESTACK_VC2 ] = "#VC2", + [ ESTACK_HV ] = "#HV", + [ ESTACK_HV2 ] = "#HV2", + }; const char *stack_type_name(enum stack_type type) { - BUILD_BUG_ON(N_EXCEPTION_STACKS != 6); + BUILD_BUG_ON(N_EXCEPTION_STACKS != 8); if (type == STACK_TYPE_TASK) return "TASK"; @@ -89,6 +92,8 @@ struct estack_pages estack_pages[CEA_ESTACK_PAGES] ____cacheline_aligned = { EPAGERANGE(MCE), EPAGERANGE(VC), EPAGERANGE(VC2), + EPAGERANGE(HV), + EPAGERANGE(HV2), }; static __always_inline bool in_exception_stack(unsigned long *stack, struct stack_info *info) @@ -98,7 +103,7 @@ static __always_inline bool in_exception_stack(unsigned long *stack, struct stac struct pt_regs *regs; unsigned int k; - BUILD_BUG_ON(N_EXCEPTION_STACKS != 6); + BUILD_BUG_ON(N_EXCEPTION_STACKS != 8); begin = (unsigned long)__this_cpu_read(cea_exception_stacks); /* diff --git a/arch/x86/kernel/idt.c b/arch/x86/kernel/idt.c index a58c6bc1cd68..48c0a7e1dbcb 100644 --- a/arch/x86/kernel/idt.c +++ b/arch/x86/kernel/idt.c @@ -113,6 +113,7 @@ static const __initconst struct idt_data def_idts[] = { #ifdef CONFIG_AMD_MEM_ENCRYPT ISTG(X86_TRAP_VC, asm_exc_vmm_communication, IST_INDEX_VC), + ISTG(X86_TRAP_HV, asm_exc_hv_injection, IST_INDEX_HV), #endif SYSG(X86_TRAP_OF, asm_exc_overflow), diff --git a/arch/x86/kernel/sev.c b/arch/x86/kernel/sev.c index a428c62330d3..b54ee3ba37b0 100644 --- a/arch/x86/kernel/sev.c +++ b/arch/x86/kernel/sev.c @@ -2004,6 +2004,59 @@ DEFINE_IDTENTRY_VC_USER(exc_vmm_communication) irqentry_exit_to_user_mode(regs); } +static bool hv_raw_handle_exception(struct pt_regs *regs) +{ + return false; +} + +static __always_inline bool on_hv_fallback_stack(struct pt_regs *regs) +{ + unsigned long sp = (unsigned long)regs; + + return (sp >= __this_cpu_ist_bottom_va(HV2) && sp < __this_cpu_ist_top_va(HV2)); +} + +DEFINE_IDTENTRY_HV_USER(exc_hv_injection) +{ + irqentry_enter_from_user_mode(regs); + instrumentation_begin(); + + if (!hv_raw_handle_exception(regs)) { + /* + * Do not kill the machine if user-space triggered the + * exception. Send SIGBUS instead and let user-space deal + * with it. + */ + force_sig_fault(SIGBUS, BUS_OBJERR, (void __user *)0); + } + + instrumentation_end(); + irqentry_exit_to_user_mode(regs); +} + +DEFINE_IDTENTRY_HV_KERNEL(exc_hv_injection) +{ + irqentry_state_t irq_state; + + irq_state = irqentry_nmi_enter(regs); + instrumentation_begin(); + + if (!hv_raw_handle_exception(regs)) { + pr_emerg("PANIC: Unhandled #HV exception in kernel space\n"); + + /* Show some debug info */ + show_regs(regs); + + /* Ask hypervisor to sev_es_terminate */ + sev_es_terminate(SEV_TERM_SET_GEN, GHCB_SEV_ES_GEN_REQ); + + panic("Returned from Terminate-Request to Hypervisor\n"); + } + + instrumentation_end(); + irqentry_nmi_exit(regs, irq_state); +} + bool __init handle_vc_boot_ghcb(struct pt_regs *regs) { unsigned long exit_code = regs->orig_ax; diff --git a/arch/x86/mm/cpu_entry_area.c b/arch/x86/mm/cpu_entry_area.c index 6c2f1b76a0b6..8b1670fa2b81 100644 --- a/arch/x86/mm/cpu_entry_area.c +++ b/arch/x86/mm/cpu_entry_area.c @@ -115,6 +115,8 @@ static void __init percpu_setup_exception_stacks(unsigned int cpu) if (cc_platform_has(CC_ATTR_GUEST_STATE_ENCRYPT)) { cea_map_stack(VC); cea_map_stack(VC2); + cea_map_stack(HV); + cea_map_stack(HV2); } } } From patchwork Sat Nov 19 03:46:30 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tianyu Lan X-Patchwork-Id: 13049570 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C36EDC433FE for ; Sat, 19 Nov 2022 03:49:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231609AbiKSDtK (ORCPT ); Fri, 18 Nov 2022 22:49:10 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37380 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231940AbiKSDrl (ORCPT ); Fri, 18 Nov 2022 22:47:41 -0500 Received: from mail-pj1-x102d.google.com (mail-pj1-x102d.google.com [IPv6:2607:f8b0:4864:20::102d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 139F7BF817; Fri, 18 Nov 2022 19:47:01 -0800 (PST) Received: by mail-pj1-x102d.google.com with SMTP id w15-20020a17090a380f00b0021873113cb4so6349640pjb.0; Fri, 18 Nov 2022 19:47:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=wrCNSp1R8NV5FdK0kNmKFSIJaMlv9YlqQVpAgSEGk/U=; b=qmJOHex8TgtJHTc4EJCANfTxPKFE0DFT5WQlikkpW6/E95emJ3yJ9+AEtYJ4XFdGtK iFgiaNoT2c2I9E+J25QtjSP2TvxfmkvQzD5g0OEcXeUS+iP8ivjKh7NV6r2KYnPEouOF 1G4Yu64TnEF56COKbdtp4R0ye54Mrut3tHOELS/XQDpSWisoKtkPBB8SXuJhO1xJqbE3 z07+IqnkM8FFqjXz8tIlVol+sYXbJ4VdcGUq2VJzOgONPCznGhfetM5st0l/3ZMHOOxt y0OdrE0T6HStGcYgyTeCBljn+f5U9ezXTKsElLr7LuEf/WnNLZlf+kR8cFHHyve6WevH 1YHw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=wrCNSp1R8NV5FdK0kNmKFSIJaMlv9YlqQVpAgSEGk/U=; b=Il3Lml8XgqXYsVnpW3vi2Z0jeOnoSrcGK/tcmUKWV61TcRhbf9w3qstiGVHAgY2Uhb fR5BRQ9VoNb6ex78hsqhTS6jpZlIy3vBTUIXQDCUrB3t+i/O/xh0cq+khZaasXYOrm/H FVhkp7KJ3PjamAohHuBiFISWm+Yd2y9RR1sAjgPgK70MMCz7+m2NYSp9j37MLI1+Zy3b WDmx5TYNHpKIpHlYbo0Xhu+vAjBYPFBiDAZxM785RiKQcGgUQuXdqqZWtSA/ob6P5GCT wbbcSQ4/aVwC1oVIKux3GLuONphj4/FMCizmgAfQxjVDWYlgk0nBk6300kfe+Yn+caLx w4NA== X-Gm-Message-State: ANoB5pnTAQIiaXvunK0IiYT01TY0+NKs+XKo35cQUqbgNoFSMSamcqub QhqIHZLzCX8i0MQOyxR/eVo= X-Google-Smtp-Source: AA0mqf5QAQbK9WuXaHspsVhhKfQtAZfD/7+0PUmpk+48x6SP83pquo+unM2BTvhoLowpkBMy2jZwsQ== X-Received: by 2002:a17:902:a386:b0:189:1076:7dbe with SMTP id x6-20020a170902a38600b0018910767dbemr192864pla.68.1668829620470; Fri, 18 Nov 2022 19:47:00 -0800 (PST) Received: from ubuntu-Virtual-Machine.corp.microsoft.com ([2001:4898:80e8:38:f087:1794:92c5:f8f0]) by smtp.gmail.com with ESMTPSA id e5-20020a056a0000c500b005360da6b26bsm3913892pfj.159.2022.11.18.19.46.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Nov 2022 19:47:00 -0800 (PST) From: Tianyu Lan To: luto@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, seanjc@google.com, pbonzini@redhat.com, jgross@suse.com, tiala@microsoft.com, kirill@shutemov.name, jiangshan.ljs@antgroup.com, peterz@infradead.org, ashish.kalra@amd.com, srutherford@google.com, akpm@linux-foundation.org, anshuman.khandual@arm.com, pawan.kumar.gupta@linux.intel.com, adrian.hunter@intel.com, daniel.sneddon@linux.intel.com, alexander.shishkin@linux.intel.com, sandipan.das@amd.com, ray.huang@amd.com, brijesh.singh@amd.com, michael.roth@amd.com, thomas.lendacky@amd.com, venu.busireddy@oracle.com, sterritt@google.com, tony.luck@intel.com, samitolvanen@google.com, fenghua.yu@intel.com Cc: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-hyperv@vger.kernel.org, linux-arch@vger.kernel.org Subject: [RFC PATCH V2 16/18] x86/sev: Initialize #HV doorbell and handle interrupt requests Date: Fri, 18 Nov 2022 22:46:30 -0500 Message-Id: <20221119034633.1728632-17-ltykernel@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221119034633.1728632-1-ltykernel@gmail.com> References: <20221119034633.1728632-1-ltykernel@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Tianyu Lan Enable #HV exception to handle interrupt requests from hypervisor. Co-developed-by: Lendacky Thomas Co-developed-by: Kalra Ashish Signed-off-by: Tianyu Lan --- arch/x86/entry/entry_64.S | 18 ++ arch/x86/include/asm/irqflags.h | 19 ++ arch/x86/include/asm/mem_encrypt.h | 2 + arch/x86/include/asm/msr-index.h | 6 + arch/x86/include/uapi/asm/svm.h | 4 + arch/x86/kernel/sev.c | 354 ++++++++++++++++++++++++----- arch/x86/kernel/traps.c | 50 ++++ 7 files changed, 400 insertions(+), 53 deletions(-) diff --git a/arch/x86/entry/entry_64.S b/arch/x86/entry/entry_64.S index b2059df43c57..fe460cf44ab5 100644 --- a/arch/x86/entry/entry_64.S +++ b/arch/x86/entry/entry_64.S @@ -1058,6 +1058,15 @@ SYM_CODE_END(paranoid_entry) * R15 - old SPEC_CTRL */ SYM_CODE_START_LOCAL(paranoid_exit) +#ifdef CONFIG_AMD_MEM_ENCRYPT + /* + * If a #HV was delivered during execution and interrupts were + * disabled, then check if it can be handled before the iret + * (which may re-enable interrupts). + */ + mov %rsp, %rdi + call check_hv_pending +#endif UNWIND_HINT_REGS /* @@ -1183,6 +1192,15 @@ SYM_CODE_START_LOCAL(error_entry) SYM_CODE_END(error_entry) SYM_CODE_START_LOCAL(error_return) +#ifdef CONFIG_AMD_MEM_ENCRYPT + /* + * If a #HV was delivered during execution and interrupts were + * disabled, then check if it can be handled before the iret + * (which may re-enable interrupts). + */ + mov %rsp, %rdi + call check_hv_pending +#endif UNWIND_HINT_REGS DEBUG_ENTRY_ASSERT_IRQS_OFF testb $3, CS(%rsp) diff --git a/arch/x86/include/asm/irqflags.h b/arch/x86/include/asm/irqflags.h index 7793e52d6237..e0730d8bc0ac 100644 --- a/arch/x86/include/asm/irqflags.h +++ b/arch/x86/include/asm/irqflags.h @@ -14,6 +14,9 @@ /* * Interrupt control: */ +#ifdef CONFIG_AMD_MEM_ENCRYPT +void check_hv_pending(struct pt_regs *regs); +#endif /* Declaration required for gcc < 4.9 to prevent -Werror=missing-prototypes */ extern inline unsigned long native_save_fl(void); @@ -35,6 +38,19 @@ extern __always_inline unsigned long native_save_fl(void) return flags; } +extern inline void native_restore_fl(unsigned long flags) +{ + asm volatile("push %0 ; popf" + : /* no output */ + : "g" (flags) + : "memory", "cc"); +#ifdef CONFIG_AMD_MEM_ENCRYPT + if ((flags & X86_EFLAGS_IF)) { + check_hv_pending(NULL); + } +#endif +} + static __always_inline void native_irq_disable(void) { asm volatile("cli": : :"memory"); @@ -43,6 +59,9 @@ static __always_inline void native_irq_disable(void) static __always_inline void native_irq_enable(void) { asm volatile("sti": : :"memory"); +#ifdef CONFIG_AMD_MEM_ENCRYPT + check_hv_pending(NULL); +#endif } static inline __cpuidle void native_safe_halt(void) diff --git a/arch/x86/include/asm/mem_encrypt.h b/arch/x86/include/asm/mem_encrypt.h index 72ca90552b6a..7264ca5f5b2d 100644 --- a/arch/x86/include/asm/mem_encrypt.h +++ b/arch/x86/include/asm/mem_encrypt.h @@ -50,6 +50,7 @@ void __init early_set_mem_enc_dec_hypercall(unsigned long vaddr, int npages, void __init mem_encrypt_free_decrypted_mem(void); void __init sev_es_init_vc_handling(void); +void __init sev_snp_init_hv_handling(void); #define __bss_decrypted __section(".bss..decrypted") @@ -72,6 +73,7 @@ static inline void __init sme_encrypt_kernel(struct boot_params *bp) { } static inline void __init sme_enable(struct boot_params *bp) { } static inline void sev_es_init_vc_handling(void) { } +static inline void sev_snp_init_hv_handling(void) { } static inline int __init early_set_memory_decrypted(unsigned long vaddr, unsigned long size) { return 0; } diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h index 10ac52705892..6fe25a6e325f 100644 --- a/arch/x86/include/asm/msr-index.h +++ b/arch/x86/include/asm/msr-index.h @@ -562,10 +562,16 @@ #define MSR_AMD64_SEV_ENABLED_BIT 0 #define MSR_AMD64_SEV_ES_ENABLED_BIT 1 #define MSR_AMD64_SEV_SNP_ENABLED_BIT 2 +#define MSR_AMD64_SEV_REFLECTVC_ENABLED_BIT 4 +#define MSR_AMD64_SEV_RESTRICTED_INJECTION_ENABLED_BIT 5 +#define MSR_AMD64_SEV_ALTERNATE_INJECTION_ENABLED_BIT 6 #define MSR_AMD64_SEV_ENABLED BIT_ULL(MSR_AMD64_SEV_ENABLED_BIT) #define MSR_AMD64_SEV_ES_ENABLED BIT_ULL(MSR_AMD64_SEV_ES_ENABLED_BIT) #define MSR_AMD64_SEV_SNP_ENABLED BIT_ULL(MSR_AMD64_SEV_SNP_ENABLED_BIT) +#define MSR_AMD64_SEV_REFLECTVC_ENABLED BIT_ULL(MSR_AMD64_SEV_REFLECTVC_ENABLED_BIT) +#define MSR_AMD64_SEV_RESTRICTED_INJECTION_ENABLED BIT_ULL(MSR_AMD64_SEV_RESTRICTED_INJECTION_ENABLED_BIT) +#define MSR_AMD64_SEV_ALTERNATE_INJECTION_ENABLED BIT_ULL(MSR_AMD64_SEV_ALTERNATE_INJECTION_ENABLED_BIT) #define MSR_AMD64_VIRT_SPEC_CTRL 0xc001011f /* AMD Collaborative Processor Performance Control MSRs */ diff --git a/arch/x86/include/uapi/asm/svm.h b/arch/x86/include/uapi/asm/svm.h index f69c168391aa..85d6882262e7 100644 --- a/arch/x86/include/uapi/asm/svm.h +++ b/arch/x86/include/uapi/asm/svm.h @@ -115,6 +115,10 @@ #define SVM_VMGEXIT_AP_CREATE_ON_INIT 0 #define SVM_VMGEXIT_AP_CREATE 1 #define SVM_VMGEXIT_AP_DESTROY 2 +#define SVM_VMGEXIT_HV_DOORBELL_PAGE 0x80000014 +#define SVM_VMGEXIT_GET_PREFERRED_HV_DOORBELL_PAGE 0 +#define SVM_VMGEXIT_SET_HV_DOORBELL_PAGE 1 +#define SVM_VMGEXIT_QUERY_HV_DOORBELL_PAGE 2 #define SVM_VMGEXIT_HV_FEATURES 0x8000fffd #define SVM_VMGEXIT_UNSUPPORTED_EVENT 0x8000ffff diff --git a/arch/x86/kernel/sev.c b/arch/x86/kernel/sev.c index b54ee3ba37b0..23cd025f97dc 100644 --- a/arch/x86/kernel/sev.c +++ b/arch/x86/kernel/sev.c @@ -104,6 +104,12 @@ struct sev_es_runtime_data { * is currently unsupported in SEV-ES guests. */ unsigned long dr7; + /* + * SEV-SNP requires that the GHCB must be registered before using it. + * The flag below will indicate whether the GHCB is registered, if its + * not registered then sev_es_get_ghcb() will perform the registration. + */ + bool ghcb_registered; }; struct ghcb_state { @@ -122,6 +128,183 @@ struct sev_config { static struct sev_config sev_cfg __read_mostly; +static noinstr struct ghcb *__sev_get_ghcb(struct ghcb_state *state); +static noinstr void __sev_put_ghcb(struct ghcb_state *state); +static int vmgexit_hv_doorbell_page(struct ghcb *ghcb, u64 op, u64 pa); +static void sev_snp_setup_hv_doorbell_page(struct ghcb *ghcb); + +union hv_pending_events { + u16 events; + struct { + u8 vector; + u8 nmi : 1; + u8 mc : 1; + u8 reserved1 : 5; + u8 no_further_signal : 1; + }; +}; + +struct sev_hv_doorbell_page { + union hv_pending_events pending_events; + u8 no_eoi_required; + u8 reserved2[61]; + u8 padding[4032]; +}; + +struct sev_snp_runtime_data { + struct sev_hv_doorbell_page hv_doorbell_page; +}; + +static DEFINE_PER_CPU(struct sev_snp_runtime_data*, snp_runtime_data); + +static inline u64 sev_es_rd_ghcb_msr(void) +{ + return __rdmsr(MSR_AMD64_SEV_ES_GHCB); +} + +static __always_inline void sev_es_wr_ghcb_msr(u64 val) +{ + u32 low, high; + + low = (u32)(val); + high = (u32)(val >> 32); + + native_wrmsr(MSR_AMD64_SEV_ES_GHCB, low, high); +} + +struct sev_hv_doorbell_page *sev_snp_current_doorbell_page(void) +{ + return &this_cpu_read(snp_runtime_data)->hv_doorbell_page; +} + +static u8 sev_hv_pending(void) +{ + return sev_snp_current_doorbell_page()->pending_events.events; +} + +static void hv_doorbell_apic_eoi_write(u32 reg, u32 val) +{ + if (xchg(&sev_snp_current_doorbell_page()->no_eoi_required, 0) & 0x1) + return; + + BUG_ON(reg != APIC_EOI); + apic->write(reg, val); +} + +static void do_exc_hv(struct pt_regs *regs) +{ + union hv_pending_events pending_events; + u8 vector; + + while (sev_hv_pending()) { + asm volatile("cli" : : : "memory"); + + pending_events.events = xchg( + &sev_snp_current_doorbell_page()->pending_events.events, + 0); + + if (pending_events.nmi) + exc_nmi(regs); + +#ifdef CONFIG_X86_MCE + if (pending_events.mc) + exc_machine_check(regs); +#endif + + if (!pending_events.vector) + return; + + if (pending_events.vector < FIRST_EXTERNAL_VECTOR) { + /* Exception vectors */ + WARN(1, "exception shouldn't happen\n"); + } else if (pending_events.vector == FIRST_EXTERNAL_VECTOR) { + sysvec_irq_move_cleanup(regs); + } else if (pending_events.vector == IA32_SYSCALL_VECTOR) { + WARN(1, "syscall shouldn't happen\n"); + } else if (pending_events.vector >= FIRST_SYSTEM_VECTOR) { + switch (pending_events.vector) { +#if IS_ENABLED(CONFIG_HYPERV) + case HYPERV_STIMER0_VECTOR: + sysvec_hyperv_stimer0(regs); + break; + case HYPERVISOR_CALLBACK_VECTOR: + sysvec_hyperv_callback(regs); + break; +#endif +#ifdef CONFIG_SMP + case RESCHEDULE_VECTOR: + sysvec_reschedule_ipi(regs); + break; + case IRQ_MOVE_CLEANUP_VECTOR: + sysvec_irq_move_cleanup(regs); + break; + case REBOOT_VECTOR: + sysvec_reboot(regs); + break; + case CALL_FUNCTION_SINGLE_VECTOR: + sysvec_call_function_single(regs); + break; + case CALL_FUNCTION_VECTOR: + sysvec_call_function(regs); + break; +#endif +#ifdef CONFIG_X86_LOCAL_APIC + case ERROR_APIC_VECTOR: + sysvec_error_interrupt(regs); + break; + case SPURIOUS_APIC_VECTOR: + sysvec_spurious_apic_interrupt(regs); + break; + case LOCAL_TIMER_VECTOR: + sysvec_apic_timer_interrupt(regs); + break; + case X86_PLATFORM_IPI_VECTOR: + sysvec_x86_platform_ipi(regs); + break; +#endif + case 0x0: + break; + default: + panic("Unexpected vector %d\n", vector); + unreachable(); + } + } else { + common_interrupt(regs, pending_events.vector); + } + + asm volatile("sti" : : : "memory"); + } +} + +void check_hv_pending(struct pt_regs *regs) +{ + struct pt_regs local_regs; + + if (!cc_platform_has(CC_ATTR_GUEST_SEV_SNP)) + return; + + if (regs) { + if ((regs->flags & X86_EFLAGS_IF) == 0) + return; + + if (!sev_hv_pending()) + return; + + do_exc_hv(regs); + } else { + if (sev_hv_pending()) { + memset(&local_regs, 0, sizeof(struct pt_regs)); + regs = &local_regs; + asm volatile("movl %%cs, %%eax;" : "=a" (regs->cs)); + asm volatile("movl %%ss, %%eax;" : "=a" (regs->ss)); + regs->orig_ax = 0xffffffff; + regs->flags = native_save_fl(); + do_exc_hv(regs); + } + } +} +EXPORT_SYMBOL_GPL(check_hv_pending); + static __always_inline bool on_vc_stack(struct pt_regs *regs) { unsigned long sp = regs->sp; @@ -193,68 +376,35 @@ void noinstr __sev_es_ist_exit(void) this_cpu_write(cpu_tss_rw.x86_tss.ist[IST_INDEX_VC], *(unsigned long *)ist); } -/* - * Nothing shall interrupt this code path while holding the per-CPU - * GHCB. The backup GHCB is only for NMIs interrupting this path. - * - * Callers must disable local interrupts around it. - */ -static noinstr struct ghcb *__sev_get_ghcb(struct ghcb_state *state) +static bool sev_restricted_injection_enabled(void) { - struct sev_es_runtime_data *data; + return sev_status & MSR_AMD64_SEV_RESTRICTED_INJECTION_ENABLED; +} + +void __init sev_snp_init_hv_handling(void) +{ + struct sev_snp_runtime_data *snp_data; + struct ghcb_state state; struct ghcb *ghcb; + unsigned long flags; + int cpu; + int err; WARN_ON(!irqs_disabled()); + if (!cc_platform_has(CC_ATTR_GUEST_SEV_SNP) || !sev_restricted_injection_enabled()) + return; - data = this_cpu_read(runtime_data); - ghcb = &data->ghcb_page; - - if (unlikely(data->ghcb_active)) { - /* GHCB is already in use - save its contents */ - - if (unlikely(data->backup_ghcb_active)) { - /* - * Backup-GHCB is also already in use. There is no way - * to continue here so just kill the machine. To make - * panic() work, mark GHCBs inactive so that messages - * can be printed out. - */ - data->ghcb_active = false; - data->backup_ghcb_active = false; - - instrumentation_begin(); - panic("Unable to handle #VC exception! GHCB and Backup GHCB are already in use"); - instrumentation_end(); - } - - /* Mark backup_ghcb active before writing to it */ - data->backup_ghcb_active = true; - - state->ghcb = &data->backup_ghcb; - - /* Backup GHCB content */ - *state->ghcb = *ghcb; - } else { - state->ghcb = NULL; - data->ghcb_active = true; - } + local_irq_save(flags); - return ghcb; -} + ghcb = __sev_get_ghcb(&state); -static inline u64 sev_es_rd_ghcb_msr(void) -{ - return __rdmsr(MSR_AMD64_SEV_ES_GHCB); -} + sev_snp_setup_hv_doorbell_page(ghcb); -static __always_inline void sev_es_wr_ghcb_msr(u64 val) -{ - u32 low, high; + __sev_put_ghcb(&state); - low = (u32)(val); - high = (u32)(val >> 32); + apic_set_eoi_write(hv_doorbell_apic_eoi_write); - native_wrmsr(MSR_AMD64_SEV_ES_GHCB, low, high); + local_irq_restore(flags); } static int vc_fetch_insn_kernel(struct es_em_ctxt *ctxt, @@ -515,6 +665,79 @@ static enum es_result vc_slow_virt_to_phys(struct ghcb *ghcb, struct es_em_ctxt /* Include code shared with pre-decompression boot stage */ #include "sev-shared.c" +/* + * Nothing shall interrupt this code path while holding the per-CPU + * GHCB. The backup GHCB is only for NMIs interrupting this path. + * + * Callers must disable local interrupts around it. + */ +static noinstr struct ghcb *__sev_get_ghcb(struct ghcb_state *state) +{ + struct sev_es_runtime_data *data; + struct ghcb *ghcb; + + WARN_ON(!irqs_disabled()); + + data = this_cpu_read(runtime_data); + ghcb = &data->ghcb_page; + + if (unlikely(data->ghcb_active)) { + /* GHCB is already in use - save its contents */ + + if (unlikely(data->backup_ghcb_active)) { + /* + * Backup-GHCB is also already in use. There is no way + * to continue here so just kill the machine. To make + * panic() work, mark GHCBs inactive so that messages + * can be printed out. + */ + data->ghcb_active = false; + data->backup_ghcb_active = false; + + instrumentation_begin(); + panic("Unable to handle #VC exception! GHCB and Backup GHCB are already in use"); + instrumentation_end(); + } + + /* Mark backup_ghcb active before writing to it */ + data->backup_ghcb_active = true; + + state->ghcb = &data->backup_ghcb; + + /* Backup GHCB content */ + *state->ghcb = *ghcb; + } else { + state->ghcb = NULL; + data->ghcb_active = true; + } + + /* SEV-SNP guest requires that GHCB must be registered before using it. */ + if (!data->ghcb_registered) { + if (cc_platform_has(CC_ATTR_GUEST_SEV_SNP)) { + snp_register_ghcb_early(__pa(ghcb)); + sev_snp_setup_hv_doorbell_page(ghcb); + } else { + sev_es_wr_ghcb_msr(__pa(ghcb)); + } + data->ghcb_registered = true; + } + + return ghcb; +} + +static void sev_snp_setup_hv_doorbell_page(struct ghcb *ghcb) +{ + u64 pa; + enum es_result ret; + + pa = __pa(sev_snp_current_doorbell_page()); + vc_ghcb_invalidate(ghcb); + ret = vmgexit_hv_doorbell_page(ghcb, + SVM_VMGEXIT_SET_HV_DOORBELL_PAGE, pa); + if (ret != ES_OK) + panic("SEV-SNP: failed to set up #HV doorbell page"); +} + static noinstr void __sev_put_ghcb(struct ghcb_state *state) { struct sev_es_runtime_data *data; @@ -1282,6 +1505,11 @@ void setup_ghcb(void) snp_register_ghcb_early(__pa(&boot_ghcb_page)); } +int vmgexit_hv_doorbell_page(struct ghcb *ghcb, u64 op, u64 pa) +{ + return sev_es_ghcb_hv_call(ghcb, NULL, SVM_VMGEXIT_HV_DOORBELL_PAGE, op, pa); +} + #ifdef CONFIG_HOTPLUG_CPU static void sev_es_ap_hlt_loop(void) { @@ -1355,6 +1583,7 @@ static void __init alloc_runtime_data(int cpu) static void __init init_ghcb(int cpu) { struct sev_es_runtime_data *data; + struct sev_snp_runtime_data *snp_data; int err; data = per_cpu(runtime_data, cpu); @@ -1366,8 +1595,22 @@ static void __init init_ghcb(int cpu) memset(&data->ghcb_page, 0, sizeof(data->ghcb_page)); + snp_data = memblock_alloc(sizeof(*snp_data), PAGE_SIZE); + if (!snp_data) + panic("Can't allocate SEV-SNP runtime data"); + + err = early_set_memory_decrypted((unsigned long)&snp_data->hv_doorbell_page, + sizeof(snp_data->hv_doorbell_page)); + if (err) + panic("Can't map #HV doorbell pages unencrypted"); + + memset(&snp_data->hv_doorbell_page, 0, sizeof(snp_data->hv_doorbell_page)); + + per_cpu(snp_runtime_data, cpu) = snp_data; + data->ghcb_active = false; data->backup_ghcb_active = false; + data->ghcb_registered = false; } void __init sev_es_init_vc_handling(void) @@ -2006,7 +2249,12 @@ DEFINE_IDTENTRY_VC_USER(exc_vmm_communication) static bool hv_raw_handle_exception(struct pt_regs *regs) { - return false; + /* Clear the no_further_signal bit */ + sev_snp_current_doorbell_page()->pending_events.events &= 0x7fff; + + check_hv_pending(regs); + + return true; } static __always_inline bool on_hv_fallback_stack(struct pt_regs *regs) diff --git a/arch/x86/kernel/traps.c b/arch/x86/kernel/traps.c index 178015a820f0..af97e6610fbb 100644 --- a/arch/x86/kernel/traps.c +++ b/arch/x86/kernel/traps.c @@ -898,6 +898,53 @@ asmlinkage __visible noinstr struct pt_regs *vc_switch_off_ist(struct pt_regs *r return regs_ret; } + +asmlinkage __visible noinstr struct pt_regs *hv_switch_off_ist(struct pt_regs *regs) +{ + unsigned long sp, *stack; + struct stack_info info; + struct pt_regs *regs_ret; + + /* + * A malicious hypervisor can inject 2 HVs in a row, which will corrupt + * the trap frame on our IST stack. We add a defensive check here to + * catch such behavior. + */ + BUG_ON(regs->sp >= __this_cpu_ist_bottom_va(HV) && regs->sp < __this_cpu_ist_top_va(HV)); + + /* + * In the SYSCALL entry path the RSP value comes from user-space - don't + * trust it and switch to the current kernel stack + */ + if (ip_within_syscall_gap(regs)) { + sp = this_cpu_read(cpu_current_top_of_stack); + goto sync; + } + + /* + * From here on the RSP value is trusted. Now check whether entry + * happened from a safe stack. Not safe are the entry or unknown stacks, + * use the fall-back stack instead in this case. + */ + sp = regs->sp; + stack = (unsigned long *)sp; + + if (!get_stack_info_noinstr(stack, current, &info) || info.type == STACK_TYPE_ENTRY || + info.type > STACK_TYPE_EXCEPTION_LAST) + sp = __this_cpu_ist_top_va(HV2); +sync: + /* + * Found a safe stack - switch to it as if the entry didn't happen via + * IST stack. The code below only copies pt_regs, the real switch happens + * in assembly code. + */ + sp = ALIGN_DOWN(sp, 8) - sizeof(*regs_ret); + + regs_ret = (struct pt_regs *)sp; + *regs_ret = *regs; + + return regs_ret; +} #endif asmlinkage __visible noinstr struct pt_regs *fixup_bad_iret(struct pt_regs *bad_regs) @@ -1457,4 +1504,7 @@ void __init trap_init(void) /* Setup traps as cpu_init() might #GP */ idt_setup_traps(); cpu_init(); + + /* Init #HV doorbell pages when running as an SEV-SNP guest */ + sev_snp_init_hv_handling(); } From patchwork Sat Nov 19 03:46:31 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tianyu Lan X-Patchwork-Id: 13049572 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 123FBC43219 for ; Sat, 19 Nov 2022 03:49:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232831AbiKSDt0 (ORCPT ); Fri, 18 Nov 2022 22:49:26 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37504 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232812AbiKSDrm (ORCPT ); Fri, 18 Nov 2022 22:47:42 -0500 Received: from mail-pj1-x102b.google.com (mail-pj1-x102b.google.com [IPv6:2607:f8b0:4864:20::102b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4AAEDC1F4D; Fri, 18 Nov 2022 19:47:02 -0800 (PST) Received: by mail-pj1-x102b.google.com with SMTP id b11so6099620pjp.2; Fri, 18 Nov 2022 19:47:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=3W/ga+rb85UmynzkXZMkLPydLwV3k2KVfEjUszZdz7c=; b=ebrGewTU3dV0JdfmvjHowccZWPmRVcYBZLJunNbUni+y/kT50wvLLKV9Pc/OY4a8lr GOx6f5iB6ijRkAP0uriXtwYPffZHvK9fD7YsVskKhD35gwgEjRsYzqenxgKQmM826+zA PBqZTkQlJROx7D+Od3YosIJ94/vnUQzF5rh+lz/ZCd+O6j3k2I5VI78JDFDCq+ZsF1Bo CeLLzO4KKZKB0CO9S+7eFSblaC05lgOCBpooWLkxjC0RGi2lRJJDc8HiwkNIXzqu1tGe BQbp1EtamcjmGw02qyFTpK5rcMFqSdJmYzyoni8rzy1vXhgjLz8W80yVgSgffLuckJ33 O8eA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3W/ga+rb85UmynzkXZMkLPydLwV3k2KVfEjUszZdz7c=; b=sf6rq4Cjm8/aQrY+Mm6BaWUtxF9UkQe5MOAzPY9MNDv7B5xKE/K9se6uV501BAYbtl ZUoRmAasGH6YbLrCHE5hLi1T2dlVzvUU6wcxbzjcAv3tdNRAwnxE00dUNodzveRKhNg3 ptXvCGt6AO/+vLy248g5Z4MtL0M0/PsteHKiGZKnN9TH4t3H1t6kCW5+xJ4G1zVKYj/9 bBO66KRpJkDv7lwA2Q4h9bNxsAgxVkcX1biTcfM7mKlwydavnSEwLG+mN0+kq1pzVK1S vO8Cxv0g29gb7wwFZdOoOyOP6B7RDHAhkT6FTIkD8aMTHfoyIWKkDlDfWBUIOWtLNSAZ /tjQ== X-Gm-Message-State: ANoB5pmq0SQimO7EFRGjeN4HP7HrkIyj2Eax1gB4u4lMk5nNosf3zSHZ UsZBxumixxwNB18LAxR9EUw= X-Google-Smtp-Source: AA0mqf6oEXz+pia7JgE20F2ViUK4iORI4m8VEQPx8Gm11vkvXbxytlN/XHDGShW1ePq7w3AsYYvORA== X-Received: by 2002:a17:902:748b:b0:186:8111:adff with SMTP id h11-20020a170902748b00b001868111adffmr2423710pll.81.1668829621710; Fri, 18 Nov 2022 19:47:01 -0800 (PST) Received: from ubuntu-Virtual-Machine.corp.microsoft.com ([2001:4898:80e8:38:f087:1794:92c5:f8f0]) by smtp.gmail.com with ESMTPSA id e5-20020a056a0000c500b005360da6b26bsm3913892pfj.159.2022.11.18.19.47.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Nov 2022 19:47:01 -0800 (PST) From: Tianyu Lan To: luto@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, seanjc@google.com, pbonzini@redhat.com, jgross@suse.com, tiala@microsoft.com, kirill@shutemov.name, jiangshan.ljs@antgroup.com, peterz@infradead.org, ashish.kalra@amd.com, srutherford@google.com, akpm@linux-foundation.org, anshuman.khandual@arm.com, pawan.kumar.gupta@linux.intel.com, adrian.hunter@intel.com, daniel.sneddon@linux.intel.com, alexander.shishkin@linux.intel.com, sandipan.das@amd.com, ray.huang@amd.com, brijesh.singh@amd.com, michael.roth@amd.com, thomas.lendacky@amd.com, venu.busireddy@oracle.com, sterritt@google.com, tony.luck@intel.com, samitolvanen@google.com, fenghua.yu@intel.com Cc: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-hyperv@vger.kernel.org, linux-arch@vger.kernel.org Subject: [RFC PATCH V2 17/18] x86/sev: optimize system vector processing invoked from #HV exception Date: Fri, 18 Nov 2022 22:46:31 -0500 Message-Id: <20221119034633.1728632-18-ltykernel@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221119034633.1728632-1-ltykernel@gmail.com> References: <20221119034633.1728632-1-ltykernel@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Ashish Kalra Construct system vector table and dispatch system vector exceptions through sysvec_table from #HV exception handler instead of explicitly calling each system vector. The system vector table is created dynamically and is placed in a new named ELF section. Signed-off-by: Ashish Kalra --- arch/x86/entry/entry_64.S | 6 +++ arch/x86/kernel/sev.c | 70 +++++++++++++---------------------- arch/x86/kernel/vmlinux.lds.S | 7 ++++ 3 files changed, 38 insertions(+), 45 deletions(-) diff --git a/arch/x86/entry/entry_64.S b/arch/x86/entry/entry_64.S index fe460cf44ab5..05f4fcc60652 100644 --- a/arch/x86/entry/entry_64.S +++ b/arch/x86/entry/entry_64.S @@ -409,6 +409,12 @@ SYM_CODE_START(\asmsym) _ASM_NOKPROBE(\asmsym) SYM_CODE_END(\asmsym) + .if \vector >= FIRST_SYSTEM_VECTOR && \vector < NR_VECTORS + .section .system_vectors, "aw" + .byte \vector + .quad \cfunc + .previous + .endif .endm /* diff --git a/arch/x86/kernel/sev.c b/arch/x86/kernel/sev.c index 23cd025f97dc..5a2f59022c98 100644 --- a/arch/x86/kernel/sev.c +++ b/arch/x86/kernel/sev.c @@ -157,6 +157,16 @@ struct sev_snp_runtime_data { static DEFINE_PER_CPU(struct sev_snp_runtime_data*, snp_runtime_data); +static void (*sysvec_table[NR_VECTORS - FIRST_SYSTEM_VECTOR]) + (struct pt_regs *regs) __ro_after_init; + +struct sysvec_entry { + unsigned char vector; + void (*sysvec_func)(struct pt_regs *regs); +} __packed; + +extern struct sysvec_entry __system_vectors[], __system_vectors_end[]; + static inline u64 sev_es_rd_ghcb_msr(void) { return __rdmsr(MSR_AMD64_SEV_ES_GHCB); @@ -222,51 +232,11 @@ static void do_exc_hv(struct pt_regs *regs) } else if (pending_events.vector == IA32_SYSCALL_VECTOR) { WARN(1, "syscall shouldn't happen\n"); } else if (pending_events.vector >= FIRST_SYSTEM_VECTOR) { - switch (pending_events.vector) { -#if IS_ENABLED(CONFIG_HYPERV) - case HYPERV_STIMER0_VECTOR: - sysvec_hyperv_stimer0(regs); - break; - case HYPERVISOR_CALLBACK_VECTOR: - sysvec_hyperv_callback(regs); - break; -#endif -#ifdef CONFIG_SMP - case RESCHEDULE_VECTOR: - sysvec_reschedule_ipi(regs); - break; - case IRQ_MOVE_CLEANUP_VECTOR: - sysvec_irq_move_cleanup(regs); - break; - case REBOOT_VECTOR: - sysvec_reboot(regs); - break; - case CALL_FUNCTION_SINGLE_VECTOR: - sysvec_call_function_single(regs); - break; - case CALL_FUNCTION_VECTOR: - sysvec_call_function(regs); - break; -#endif -#ifdef CONFIG_X86_LOCAL_APIC - case ERROR_APIC_VECTOR: - sysvec_error_interrupt(regs); - break; - case SPURIOUS_APIC_VECTOR: - sysvec_spurious_apic_interrupt(regs); - break; - case LOCAL_TIMER_VECTOR: - sysvec_apic_timer_interrupt(regs); - break; - case X86_PLATFORM_IPI_VECTOR: - sysvec_x86_platform_ipi(regs); - break; -#endif - case 0x0: - break; - default: - panic("Unexpected vector %d\n", vector); - unreachable(); + if (!(sysvec_table[pending_events.vector - FIRST_SYSTEM_VECTOR])) { + WARN(1, "system vector entry 0x%x is NULL\n", + pending_events.vector); + } else { + (*sysvec_table[pending_events.vector - FIRST_SYSTEM_VECTOR])(regs); } } else { common_interrupt(regs, pending_events.vector); @@ -381,6 +351,14 @@ static bool sev_restricted_injection_enabled(void) return sev_status & MSR_AMD64_SEV_RESTRICTED_INJECTION_ENABLED; } +static void __init construct_sysvec_table(void) +{ + struct sysvec_entry *p; + + for (p = __system_vectors; p < __system_vectors_end; p++) + sysvec_table[p->vector - FIRST_SYSTEM_VECTOR] = p->sysvec_func; +} + void __init sev_snp_init_hv_handling(void) { struct sev_snp_runtime_data *snp_data; @@ -405,6 +383,8 @@ void __init sev_snp_init_hv_handling(void) apic_set_eoi_write(hv_doorbell_apic_eoi_write); local_irq_restore(flags); + + construct_sysvec_table(); } static int vc_fetch_insn_kernel(struct es_em_ctxt *ctxt, diff --git a/arch/x86/kernel/vmlinux.lds.S b/arch/x86/kernel/vmlinux.lds.S index 15f29053cec4..160c7cdaa3de 100644 --- a/arch/x86/kernel/vmlinux.lds.S +++ b/arch/x86/kernel/vmlinux.lds.S @@ -322,6 +322,13 @@ SECTIONS *(.altinstr_replacement) } + . = ALIGN(8); + .system_vectors : AT(ADDR(.system_vectors) - LOAD_OFFSET) { + __system_vectors = .; + *(.system_vectors) + __system_vectors_end = .; + } + . = ALIGN(8); .apicdrivers : AT(ADDR(.apicdrivers) - LOAD_OFFSET) { __apicdrivers = .; From patchwork Sat Nov 19 03:46:32 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tianyu Lan X-Patchwork-Id: 13049573 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 62BCEC4332F for ; Sat, 19 Nov 2022 03:49:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234034AbiKSDt2 (ORCPT ); Fri, 18 Nov 2022 22:49:28 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38224 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232849AbiKSDrm (ORCPT ); Fri, 18 Nov 2022 22:47:42 -0500 Received: from mail-pj1-x1036.google.com (mail-pj1-x1036.google.com [IPv6:2607:f8b0:4864:20::1036]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AF5ACC1F5E; Fri, 18 Nov 2022 19:47:03 -0800 (PST) Received: by mail-pj1-x1036.google.com with SMTP id r61-20020a17090a43c300b00212f4e9cccdso9929199pjg.5; Fri, 18 Nov 2022 19:47:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=+3qjpotohnIDBJGiUkElMdGRD5fwHCHM0g3ZymBFFdM=; b=KqwZEV8a2kCExG25rGD2s+E9r29CMfOLWEI86bfNZ028Q2ODuWjiRqtXxz4Yfqe2Gk Et1ESN8bYgHU+nb6o7DcYUyN3PhSMWQImHQGTMOZq9Zo9bVeZFM1AU86+p6yTHSklC5K nyk8O2oV6uxtfDyOieVgf1ytD7VJs85KpeV3VItmutk9G0vw8ouxH+4k/oQW+7eYEvrS KR4Lt86o15OJX4h6H7OWrFggi0tBzWmd6BZUPJDVKUs+5Ak454+507zJ82JpCDyTa/BH JgA3m/cApWx9CPYK8mPram4PJPfsfyrAmW38sEqJXwIQ5Zr+s891l5EPiaej/htuWFQy kRxQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=+3qjpotohnIDBJGiUkElMdGRD5fwHCHM0g3ZymBFFdM=; b=SMN7xx0jGwWIxV/0gin8HOG9eUjTNF1ekA9zBqU7BONP6X/3n97GQviKmPFDgJNpgF EDfIEIJ/PUDa0DJ/dku3E3iuy+L4D6iqNV4n7ZZAUgcUtctPz4t8gUY/zRw0rb6Mq5wN tkbwUFbsKH2V7wCRPxD52JjCRZ3QvrrL9k+Uy9/L8KJCrXoVwmgG4+jebLPs/DgqC6Dg rk/UDV2LoZHbAPcqNpbDkgEpTjSJONq8fhXao6uYbOiF4mszWMhKgRaC8WjFRXYhba+e aICI54+Dyh0QBPkmkCeLY92wsw1rTg8jWltjs4m52pj2WzWC+8mgI2BBG3PSQ1rbYdxi Sdsg== X-Gm-Message-State: ANoB5plYSg8gVa9fml9i0OCFRbJmDnXst6dW0SU8klZg3hlurNVvvurI 0JDf2Ouj3VtCCFjQ/9l00MJIxxkAVevkhA== X-Google-Smtp-Source: AA0mqf7qGf73NCz68C5Wm69ao01VLtP2h0joAu+fhEKObXUUR3k8EPAUdcAla2AKrtCDtOPmEuEYlw== X-Received: by 2002:a17:90a:39c9:b0:218:499b:bee9 with SMTP id k9-20020a17090a39c900b00218499bbee9mr11018497pjf.171.1668829622985; Fri, 18 Nov 2022 19:47:02 -0800 (PST) Received: from ubuntu-Virtual-Machine.corp.microsoft.com ([2001:4898:80e8:38:f087:1794:92c5:f8f0]) by smtp.gmail.com with ESMTPSA id e5-20020a056a0000c500b005360da6b26bsm3913892pfj.159.2022.11.18.19.47.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Nov 2022 19:47:02 -0800 (PST) From: Tianyu Lan To: luto@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, seanjc@google.com, pbonzini@redhat.com, jgross@suse.com, tiala@microsoft.com, kirill@shutemov.name, jiangshan.ljs@antgroup.com, peterz@infradead.org, ashish.kalra@amd.com, srutherford@google.com, akpm@linux-foundation.org, anshuman.khandual@arm.com, pawan.kumar.gupta@linux.intel.com, adrian.hunter@intel.com, daniel.sneddon@linux.intel.com, alexander.shishkin@linux.intel.com, sandipan.das@amd.com, ray.huang@amd.com, brijesh.singh@amd.com, michael.roth@amd.com, thomas.lendacky@amd.com, venu.busireddy@oracle.com, sterritt@google.com, tony.luck@intel.com, samitolvanen@google.com, fenghua.yu@intel.com Cc: linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-hyperv@vger.kernel.org, linux-arch@vger.kernel.org Subject: [RFC PATCH V2 18/18] x86/sev: Fix interrupt exit code paths from #HV exception Date: Fri, 18 Nov 2022 22:46:32 -0500 Message-Id: <20221119034633.1728632-19-ltykernel@gmail.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221119034633.1728632-1-ltykernel@gmail.com> References: <20221119034633.1728632-1-ltykernel@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Ashish Kalra Add checks in interrupt exit code paths in case of returns to user mode to check if currently executing the #HV handler then don't follow the irqentry_exit_to_user_mode path as that can potentially cause the #HV handler to be preempted and rescheduled on another CPU. Rescheduled #HV handler on another cpu will cause interrupts to be handled on a different cpu than the injected one, causing invalid EOIs and missed/lost guest interrupts and corresponding hangs and/or per-cpu IRQs handled on non-intended cpu. Signed-off-by: Ashish Kalra --- arch/x86/include/asm/idtentry.h | 66 +++++++++++++++++++++++++++++++++ arch/x86/kernel/sev.c | 30 +++++++++++++++ 2 files changed, 96 insertions(+) diff --git a/arch/x86/include/asm/idtentry.h b/arch/x86/include/asm/idtentry.h index 652fea10d377..45b47132be7c 100644 --- a/arch/x86/include/asm/idtentry.h +++ b/arch/x86/include/asm/idtentry.h @@ -13,6 +13,10 @@ #include +#ifdef CONFIG_AMD_MEM_ENCRYPT +noinstr void irqentry_exit_hv_cond(struct pt_regs *regs, irqentry_state_t state); +#endif + /** * DECLARE_IDTENTRY - Declare functions for simple IDT entry points * No error code pushed by hardware @@ -176,6 +180,7 @@ __visible noinstr void func(struct pt_regs *regs, unsigned long error_code) #define DECLARE_IDTENTRY_IRQ(vector, func) \ DECLARE_IDTENTRY_ERRORCODE(vector, func) +#ifndef CONFIG_AMD_MEM_ENCRYPT /** * DEFINE_IDTENTRY_IRQ - Emit code for device interrupt IDT entry points * @func: Function name of the entry point @@ -205,6 +210,26 @@ __visible noinstr void func(struct pt_regs *regs, \ } \ \ static noinline void __##func(struct pt_regs *regs, u32 vector) +#else + +#define DEFINE_IDTENTRY_IRQ(func) \ +static void __##func(struct pt_regs *regs, u32 vector); \ + \ +__visible noinstr void func(struct pt_regs *regs, \ + unsigned long error_code) \ +{ \ + irqentry_state_t state = irqentry_enter(regs); \ + u32 vector = (u32)(u8)error_code; \ + \ + instrumentation_begin(); \ + kvm_set_cpu_l1tf_flush_l1d(); \ + run_irq_on_irqstack_cond(__##func, regs, vector); \ + instrumentation_end(); \ + irqentry_exit_hv_cond(regs, state); \ +} \ + \ +static noinline void __##func(struct pt_regs *regs, u32 vector) +#endif /** * DECLARE_IDTENTRY_SYSVEC - Declare functions for system vector entry points @@ -221,6 +246,7 @@ static noinline void __##func(struct pt_regs *regs, u32 vector) #define DECLARE_IDTENTRY_SYSVEC(vector, func) \ DECLARE_IDTENTRY(vector, func) +#ifndef CONFIG_AMD_MEM_ENCRYPT /** * DEFINE_IDTENTRY_SYSVEC - Emit code for system vector IDT entry points * @func: Function name of the entry point @@ -245,6 +271,26 @@ __visible noinstr void func(struct pt_regs *regs) \ } \ \ static noinline void __##func(struct pt_regs *regs) +#else + +#define DEFINE_IDTENTRY_SYSVEC(func) \ +static void __##func(struct pt_regs *regs); \ + \ +__visible noinstr void func(struct pt_regs *regs) \ +{ \ + irqentry_state_t state = irqentry_enter(regs); \ + \ + instrumentation_begin(); \ + kvm_set_cpu_l1tf_flush_l1d(); \ + run_sysvec_on_irqstack_cond(__##func, regs); \ + instrumentation_end(); \ + irqentry_exit_hv_cond(regs, state); \ +} \ + \ +static noinline void __##func(struct pt_regs *regs) +#endif + +#ifndef CONFIG_AMD_MEM_ENCRYPT /** * DEFINE_IDTENTRY_SYSVEC_SIMPLE - Emit code for simple system vector IDT @@ -274,6 +320,26 @@ __visible noinstr void func(struct pt_regs *regs) \ } \ \ static __always_inline void __##func(struct pt_regs *regs) +#else + +#define DEFINE_IDTENTRY_SYSVEC_SIMPLE(func) \ +static __always_inline void __##func(struct pt_regs *regs); \ + \ +__visible noinstr void func(struct pt_regs *regs) \ +{ \ + irqentry_state_t state = irqentry_enter(regs); \ + \ + instrumentation_begin(); \ + __irq_enter_raw(); \ + kvm_set_cpu_l1tf_flush_l1d(); \ + __##func(regs); \ + __irq_exit_raw(); \ + instrumentation_end(); \ + irqentry_exit_hv_cond(regs, state); \ +} \ + \ +static __always_inline void __##func(struct pt_regs *regs) +#endif /** * DECLARE_IDTENTRY_XENCB - Declare functions for XEN HV callback entry point diff --git a/arch/x86/kernel/sev.c b/arch/x86/kernel/sev.c index 5a2f59022c98..ef6a123c50fe 100644 --- a/arch/x86/kernel/sev.c +++ b/arch/x86/kernel/sev.c @@ -153,6 +153,10 @@ struct sev_hv_doorbell_page { struct sev_snp_runtime_data { struct sev_hv_doorbell_page hv_doorbell_page; + /* + * Indication that we are currently handling #HV events. + */ + bool hv_handling_events; }; static DEFINE_PER_CPU(struct sev_snp_runtime_data*, snp_runtime_data); @@ -206,6 +210,8 @@ static void do_exc_hv(struct pt_regs *regs) union hv_pending_events pending_events; u8 vector; + this_cpu_read(snp_runtime_data)->hv_handling_events = true; + while (sev_hv_pending()) { asm volatile("cli" : : : "memory"); @@ -244,6 +250,8 @@ static void do_exc_hv(struct pt_regs *regs) asm volatile("sti" : : : "memory"); } + + this_cpu_read(snp_runtime_data)->hv_handling_events = false; } void check_hv_pending(struct pt_regs *regs) @@ -2541,3 +2549,25 @@ static int __init snp_init_platform_device(void) return 0; } device_initcall(snp_init_platform_device); + +noinstr void irqentry_exit_hv_cond(struct pt_regs *regs, irqentry_state_t state) +{ + /* + * Check whether this returns to user mode, if so and if + * we are currently executing the #HV handler then we don't + * want to follow the irqentry_exit_to_user_mode path as + * that can potentially cause the #HV handler to be + * preempted and rescheduled on another CPU. Rescheduled #HV + * handler on another cpu will cause interrupts to be handled + * on a different cpu than the injected one, causing + * invalid EOIs and missed/lost guest interrupts and + * corresponding hangs and/or per-cpu IRQs handled on + * non-intended cpu. + */ + if (user_mode(regs) && + this_cpu_read(snp_runtime_data)->hv_handling_events) + return; + + /* follow normal interrupt return/exit path */ + irqentry_exit(regs, state); +}