From patchwork Mon Sep 14 20:15:20 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tom Lendacky X-Patchwork-Id: 11774775 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E40B86CA for ; Mon, 14 Sep 2020 20:18:06 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id BC8AA21741 for ; Mon, 14 Sep 2020 20:18:06 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=amdcloud.onmicrosoft.com header.i=@amdcloud.onmicrosoft.com header.b="OVV/sTKx" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726125AbgINUSC (ORCPT ); Mon, 14 Sep 2020 16:18:02 -0400 Received: from mail-bn8nam12on2061.outbound.protection.outlook.com ([40.107.237.61]:36897 "EHLO NAM12-BN8-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1726100AbgINUQ7 (ORCPT ); Mon, 14 Sep 2020 16:16:59 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=RR5JPj/TwHsJJRJl/QQK+PexB9f4yWo2L2uciDLxaH3r73PDTb5hwc/EfxpV13dU++a6i9NUFdAIoSf1/+hBjgewAsA2o+evpqDYnz6gSUa7SGschRnZfIEOjt9ICNldpmqsz/FUihAhVSRKIVMOvsbEUfQNMUAJDmJwohNT8/4pA6IGOAKFhU8iqYXia3QFf34wH03G58lfrG+ySn55rxXnhFMforAqsk4uYM8BMokKnkOrw+3hshZi2x+0Gea+gS5fVYS58LW9PYsCv0rO7HV5nZcsFZykLBv4rZ1PsIIfD8SacRWcOJQo9GhptWjPlMmsPQ/YoH2c9jGFEO/QQQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=taxoIxZp0bZp/NVXjXBqJPvClCBO1J+EfzAt7HtG68A=; b=BheN7//+kuB/Ca1LXhFkJqIO3QpvTvrGzH47pi9ylGF1WnyS2uWrzeL8yzkELNrblcXnzPDzKhjz+AozPClG+beDTkwKzr5gtfLsESu+F5RSQU64KKdzKaD9JF9uj4NmERZUI5+U9rm9OM6l8uzyBSey6ojq5adfeDaC8xi3RM//PSy22BcI74lM9wvEGfxSnDElxYrDGmUUQisy5sXIKZtq2q+GdJHSH3VkanintgI/oZIGR0CqAldgk5Pe+J9ddWASBQNLgwGV+8Aykt4Z/Z0XTSfMG5Ysvom23dsbG15oJXr+1rSLw6inkpXUgw6LiHWbNRCT/ImvQiPWCn69DQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amdcloud.onmicrosoft.com; s=selector2-amdcloud-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=taxoIxZp0bZp/NVXjXBqJPvClCBO1J+EfzAt7HtG68A=; b=OVV/sTKxozHjxVoefKjwCf/1/F7Qg+S4MGUqeR+QCpRjfk5jqCJPyzmIhjp+Pdo3wrcyu/G3UJ8wG7IAPabm0uW2RM2EoYbeagRUcLZqp5csj7A4QrMWwgo8KfDGnF71RfUqA3b+Hr4/HWM34A/sOPEuyYyBP77aIFELBSfR9pU= Authentication-Results: vger.kernel.org; dkim=none (message not signed) header.d=none;vger.kernel.org; dmarc=none action=none header.from=amd.com; Received: from DM5PR12MB1355.namprd12.prod.outlook.com (2603:10b6:3:6e::7) by DM5PR12MB1163.namprd12.prod.outlook.com (2603:10b6:3:7a::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3370.16; Mon, 14 Sep 2020 20:16:47 +0000 Received: from DM5PR12MB1355.namprd12.prod.outlook.com ([fe80::299a:8ed2:23fc:6346]) by DM5PR12MB1355.namprd12.prod.outlook.com ([fe80::299a:8ed2:23fc:6346%3]) with mapi id 15.20.3370.019; Mon, 14 Sep 2020 20:16:47 +0000 From: Tom Lendacky To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, x86@kernel.org Cc: Paolo Bonzini , Jim Mattson , Joerg Roedel , Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Borislav Petkov , Ingo Molnar , Thomas Gleixner , Brijesh Singh Subject: [RFC PATCH 06/35] KVM: SVM: Add required changes to support intercepts under SEV-ES Date: Mon, 14 Sep 2020 15:15:20 -0500 Message-Id: <16838d177e7f12eb4666bb55e14763970aa6552a.1600114548.git.thomas.lendacky@amd.com> X-Mailer: git-send-email 2.28.0 In-Reply-To: References: X-ClientProxiedBy: DM5PR07CA0146.namprd07.prod.outlook.com (2603:10b6:3:ee::12) To DM5PR12MB1355.namprd12.prod.outlook.com (2603:10b6:3:6e::7) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from tlendack-t1.amd.com (165.204.77.1) by DM5PR07CA0146.namprd07.prod.outlook.com (2603:10b6:3:ee::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3370.16 via Frontend Transport; Mon, 14 Sep 2020 20:16:46 +0000 X-Mailer: git-send-email 2.28.0 X-Originating-IP: [165.204.77.1] X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: bb29c9c9-0bb6-48dd-b015-08d858eb1b4c X-MS-TrafficTypeDiagnostic: DM5PR12MB1163: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:8882; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: EPgKqFIT5ljo3sPSFAfBRybbXyTW/mkau5zjgPfYb6C4Mp1vKSS0s8ibk83f+dwgL2EG2s+em0vaiQGiW5bIpQPwxFwYxRb9VYG2FPkFow2UV0U5qYT162wRPQMYwTonHWDPl7iWEG7kyJ0DEir7Ul619XvhdMDAKf915OtCOww/tnMxoaS1HbChEY28CsTnmrCWxP3B1N1FeALopeTkO4xfFtf9/U1z8psQHpFgsDvmsVBs83TA+/SwQCC3tvk88JIVtm6RvMDYWBLG/Gddz8s96qrVMrJFeBdsgfaQsMNfwLAhUnVSfLutXroT633000prrZBnDzUxVvsr2q4ljQ== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM5PR12MB1355.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(136003)(396003)(366004)(376002)(39860400002)(346002)(8676002)(478600001)(83380400001)(26005)(316002)(7416002)(2906002)(5660300002)(6666004)(956004)(86362001)(186003)(16526019)(52116002)(7696005)(66556008)(66476007)(66946007)(4326008)(8936002)(54906003)(36756003)(2616005)(6486002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: vdPDMqD9uazl8tZnOomBQbAZp9mKu6ZJMrXYDeI/CcUcmAqpT4Ei+J1hOQPFjf96RGrozAgjxuzjg0QBd9sq91SwuC7X1kX1a0MDOdGffet9Q7I3eKh4jANRkA4uk+iTR4CZvXAAtThsXTSQVeRcXAxeE4LqeqeXzXV8uf0v670Ju+CwEj0KRKLq72xZiY+cIyI89uoKuzdO3l8Xvtqyir32xwlUF2OElAZTltPOQ1YD79RXHucxAh7uqUZwG2gMPoh/4eE7rEJmDOXNmtQyHj1F8DEL6hi392j1q9Car3O1+PCI8VMjqNprfEQ2nvnWUQ3ZWjuppry74/xG7+Dc2/saCEZ8Y68PxaJxmoMw3djzZuuX72Tg9GUcU4u6oJBj8vuAYQbuYRhfayV1UnsaQWjaHZEMJYD0iJ3xpdkwcJFXeGpzC6Co65avi7sI2f1qQSnUZw9bqvIUvnjTNfgfSjJSOceARbt6Io/EHf4k/+OQC9lILunTEaSD6vMgFNJvE9ek35ZWBcOlz2MYwQKjqy07rcJ/aO1oHo421/iGedJZxhh8w75Ks3Mbhpz4FY1bq1lVnsei1sgOZVaZkv5G03KsiLU8lsPBVf+XbuvFfxdIUcEsKLVtQfmcV4gW2UTuGOpAxWvi4cxaGUDCMJtfeQ== X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: bb29c9c9-0bb6-48dd-b015-08d858eb1b4c X-MS-Exchange-CrossTenant-AuthSource: DM5PR12MB1355.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Sep 2020 20:16:47.4153 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: gi0YHbLQo5GCqXfUQGXZVuQqP9uiMFQuErVW+q61sx/m9EOgB55vZnYi+q3mBPEO/oxEJKcsII/XEG0WFhXH8g== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR12MB1163 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Tom Lendacky When a guest is running under SEV-ES, the hypervisor cannot access the guest register state. There are numerous places in the KVM code where certain registers are accessed that are not allowed to be accessed (e.g. RIP, CR0, etc). Add checks to prevent register accesses and intercept updates at various points within the KVM code. Signed-off-by: Tom Lendacky --- arch/x86/include/asm/svm.h | 3 +- arch/x86/kvm/cpuid.c | 1 + arch/x86/kvm/svm/svm.c | 114 ++++++++++++++++++++++++++++++++++--- arch/x86/kvm/x86.c | 6 +- 4 files changed, 113 insertions(+), 11 deletions(-) diff --git a/arch/x86/include/asm/svm.h b/arch/x86/include/asm/svm.h index c112207c201b..ed03d23f56fe 100644 --- a/arch/x86/include/asm/svm.h +++ b/arch/x86/include/asm/svm.h @@ -130,7 +130,8 @@ struct __attribute__ ((__packed__)) vmcb_control_area { #define LBR_CTL_ENABLE_MASK BIT_ULL(0) #define VIRTUAL_VMLOAD_VMSAVE_ENABLE_MASK BIT_ULL(1) -#define SVM_INTERRUPT_SHADOW_MASK 1 +#define SVM_INTERRUPT_SHADOW_MASK BIT_ULL(0) +#define SVM_GUEST_INTERRUPT_MASK BIT_ULL(1) #define SVM_IOIO_STR_SHIFT 2 #define SVM_IOIO_REP_SHIFT 3 diff --git a/arch/x86/kvm/cpuid.c b/arch/x86/kvm/cpuid.c index 3fd6eec202d7..15f2b2365936 100644 --- a/arch/x86/kvm/cpuid.c +++ b/arch/x86/kvm/cpuid.c @@ -115,6 +115,7 @@ void kvm_update_cpuid_runtime(struct kvm_vcpu *vcpu) MSR_IA32_MISC_ENABLE_MWAIT); } } +EXPORT_SYMBOL_GPL(kvm_update_cpuid_runtime); static void kvm_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu) { diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c index d1f52211627a..f8a5b7164008 100644 --- a/arch/x86/kvm/svm/svm.c +++ b/arch/x86/kvm/svm/svm.c @@ -36,6 +36,7 @@ #include #include #include +#include #include #include "trace.h" @@ -320,6 +321,13 @@ static int skip_emulated_instruction(struct kvm_vcpu *vcpu) { struct vcpu_svm *svm = to_svm(vcpu); + /* + * SEV-ES does not expose the next RIP. The RIP update is controlled by + * the type of exit and the #VC handler in the guest. + */ + if (sev_es_guest(vcpu->kvm)) + goto done; + if (nrips && svm->vmcb->control.next_rip != 0) { WARN_ON_ONCE(!static_cpu_has(X86_FEATURE_NRIPS)); svm->next_rip = svm->vmcb->control.next_rip; @@ -331,6 +339,8 @@ static int skip_emulated_instruction(struct kvm_vcpu *vcpu) } else { kvm_rip_write(vcpu, svm->next_rip); } + +done: svm_set_interrupt_shadow(vcpu, 0); return 1; @@ -1578,9 +1588,17 @@ static void svm_set_gdt(struct kvm_vcpu *vcpu, struct desc_ptr *dt) static void update_cr0_intercept(struct vcpu_svm *svm) { - ulong gcr0 = svm->vcpu.arch.cr0; + ulong gcr0; u64 hcr0; + /* + * SEV-ES guests must always keep the CR intercepts cleared. CR + * tracking is done using the CR write traps. + */ + if (sev_es_guest(svm->vcpu.kvm)) + return; + + gcr0 = svm->vcpu.arch.cr0; hcr0 = (svm_cr0_read(svm) & ~SVM_CR0_SELECTIVE_MASK) | (gcr0 & SVM_CR0_SELECTIVE_MASK); @@ -2209,6 +2227,17 @@ static int task_switch_interception(struct vcpu_svm *svm) static int cpuid_interception(struct vcpu_svm *svm) { + /* + * SEV-ES guests require the vCPU arch registers to be populated via + * the GHCB. + */ + if (sev_es_guest(svm->vcpu.kvm)) { + if (kvm_register_read(&svm->vcpu, VCPU_REGS_RAX) == 0x0d) { + svm->vcpu.arch.xcr0 = svm_xcr0_read(svm); + kvm_update_cpuid_runtime(&svm->vcpu); + } + } + return kvm_emulate_cpuid(&svm->vcpu); } @@ -2527,7 +2556,28 @@ static int svm_get_msr(struct kvm_vcpu *vcpu, struct msr_data *msr_info) static int rdmsr_interception(struct vcpu_svm *svm) { - return kvm_emulate_rdmsr(&svm->vcpu); + u32 ecx = kvm_rcx_read(&svm->vcpu); + u64 data; + + if (kvm_get_msr(&svm->vcpu, ecx, &data)) { + trace_kvm_msr_read_ex(ecx); + if (sev_es_guest(svm->vcpu.kvm)) { + ghcb_set_sw_exit_info_1(svm->ghcb, 1); + ghcb_set_sw_exit_info_2(svm->ghcb, + X86_TRAP_GP | + SVM_EVTINJ_TYPE_EXEPT | + SVM_EVTINJ_VALID); + } else { + kvm_inject_gp(&svm->vcpu, 0); + } + return 1; + } + + trace_kvm_msr_read(ecx, data); + + kvm_rax_write(&svm->vcpu, data & 0xffffffff); + kvm_rdx_write(&svm->vcpu, data >> 32); + return kvm_skip_emulated_instruction(&svm->vcpu); } static int svm_set_vm_cr(struct kvm_vcpu *vcpu, u64 data) @@ -2716,7 +2766,25 @@ static int svm_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr) static int wrmsr_interception(struct vcpu_svm *svm) { - return kvm_emulate_wrmsr(&svm->vcpu); + u32 ecx = kvm_rcx_read(&svm->vcpu); + u64 data = kvm_read_edx_eax(&svm->vcpu); + + if (kvm_set_msr(&svm->vcpu, ecx, data)) { + trace_kvm_msr_write_ex(ecx, data); + if (sev_es_guest(svm->vcpu.kvm)) { + ghcb_set_sw_exit_info_1(svm->ghcb, 1); + ghcb_set_sw_exit_info_2(svm->ghcb, + X86_TRAP_GP | + SVM_EVTINJ_TYPE_EXEPT | + SVM_EVTINJ_VALID); + } else { + kvm_inject_gp(&svm->vcpu, 0); + } + return 1; + } + + trace_kvm_msr_write(ecx, data); + return kvm_skip_emulated_instruction(&svm->vcpu); } static int msr_interception(struct vcpu_svm *svm) @@ -2746,7 +2814,14 @@ static int interrupt_window_interception(struct vcpu_svm *svm) static int pause_interception(struct vcpu_svm *svm) { struct kvm_vcpu *vcpu = &svm->vcpu; - bool in_kernel = (svm_get_cpl(vcpu) == 0); + bool in_kernel; + + /* + * CPL is not made available for an SEV-ES guest, so just set in_kernel + * to true. + */ + in_kernel = (sev_es_guest(svm->vcpu.kvm)) ? true + : (svm_get_cpl(vcpu) == 0); if (!kvm_pause_in_guest(vcpu->kvm)) grow_ple_window(vcpu); @@ -2972,10 +3047,13 @@ static int handle_exit(struct kvm_vcpu *vcpu, fastpath_t exit_fastpath) trace_kvm_exit(exit_code, vcpu, KVM_ISA_SVM); - if (!is_cr_intercept(svm, INTERCEPT_CR0_WRITE)) - vcpu->arch.cr0 = svm_cr0_read(svm); - if (npt_enabled) - vcpu->arch.cr3 = svm_cr3_read(svm); + /* SEV-ES guests must use the CR write traps to track CR registers. */ + if (!sev_es_guest(vcpu->kvm)) { + if (!is_cr_intercept(svm, INTERCEPT_CR0_WRITE)) + vcpu->arch.cr0 = svm_cr0_read(svm); + if (npt_enabled) + vcpu->arch.cr3 = svm_cr3_read(svm); + } svm_complete_interrupts(svm); @@ -3094,6 +3172,13 @@ static void update_cr8_intercept(struct kvm_vcpu *vcpu, int tpr, int irr) { struct vcpu_svm *svm = to_svm(vcpu); + /* + * SEV-ES guests must always keep the CR intercepts cleared. CR + * tracking is done using the CR write traps. + */ + if (sev_es_guest(vcpu->kvm)) + return; + if (nested_svm_virtualize_tpr(vcpu)) return; @@ -3162,6 +3247,13 @@ bool svm_interrupt_blocked(struct kvm_vcpu *vcpu) struct vcpu_svm *svm = to_svm(vcpu); struct vmcb *vmcb = svm->vmcb; + /* + * SEV-ES guests to not expose RFLAGS. Use the VMCB interrupt mask + * bit to determine the state of the IF flag. + */ + if (sev_es_guest(svm->vcpu.kvm)) + return !(vmcb->control.int_state & SVM_GUEST_INTERRUPT_MASK); + if (!gif_set(svm)) return true; @@ -3347,6 +3439,12 @@ static void svm_complete_interrupts(struct vcpu_svm *svm) svm->vcpu.arch.nmi_injected = true; break; case SVM_EXITINTINFO_TYPE_EXEPT: + /* + * Never re-inject a #VC exception. + */ + if (vector == X86_TRAP_VC) + break; + /* * In case of software exceptions, do not reinject the vector, * but re-execute the instruction instead. Rewind RIP first diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 539ea1cd6020..a5afdccb6c17 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -3771,7 +3771,7 @@ void kvm_arch_vcpu_put(struct kvm_vcpu *vcpu) { int idx; - if (vcpu->preempted) + if (vcpu->preempted && !vcpu->arch.vmsa_encrypted) vcpu->arch.preempted_in_kernel = !kvm_x86_ops.get_cpl(vcpu); /* @@ -7774,7 +7774,9 @@ static void post_kvm_run_save(struct kvm_vcpu *vcpu) { struct kvm_run *kvm_run = vcpu->run; - kvm_run->if_flag = (kvm_get_rflags(vcpu) & X86_EFLAGS_IF) != 0; + kvm_run->if_flag = (vcpu->arch.vmsa_encrypted) + ? kvm_arch_interrupt_allowed(vcpu) + : (kvm_get_rflags(vcpu) & X86_EFLAGS_IF) != 0; kvm_run->flags = is_smm(vcpu) ? KVM_RUN_X86_SMM : 0; kvm_run->cr8 = kvm_get_cr8(vcpu); kvm_run->apic_base = kvm_get_apic_base(vcpu);