From patchwork Wed Oct 24 08:28:56 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Julian Stecklina X-Patchwork-Id: 10654387 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B900E13A4 for ; Wed, 24 Oct 2018 08:30:04 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A2E272A19D for ; Wed, 24 Oct 2018 08:30:04 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 96F182A1F0; Wed, 24 Oct 2018 08:30:04 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3D8962A19D for ; Wed, 24 Oct 2018 08:30:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727204AbeJXQ4s (ORCPT ); Wed, 24 Oct 2018 12:56:48 -0400 Received: from smtp-fw-6002.amazon.com ([52.95.49.90]:62250 "EHLO smtp-fw-6002.amazon.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726267AbeJXQ4s (ORCPT ); Wed, 24 Oct 2018 12:56:48 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.de; i=@amazon.de; q=dns/txt; s=amazon201209; t=1540369778; x=1571905778; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=tdB5rCYLJAIR6t6/83w13JTk6bLB2tkIQ4irXe8NrL4=; b=p0d1P78Wo46NFOezgxO2gbErUCCyc3n5RMJpskokiXEG+IynMbYvAn4z P9BrOdu16KwUwv2ncAiBiJ7TFFCPpo754U91zJ9swrwR3mApRoDPE4nhB Simg8Fe1/buIDaXWBkNDV7gWYlDTHZO48La55d3t0AxwRcCXkiSQ2dRs6 g=; X-IronPort-AV: E=Sophos;i="5.54,419,1534809600"; d="scan'208";a="370227152" Received: from iad6-co-svc-p1-lb1-vlan3.amazon.com (HELO email-inbound-relay-1a-16acd5e0.us-east-1.amazon.com) ([10.124.125.6]) by smtp-border-fw-out-6002.iad6.amazon.com with ESMTP/TLS/DHE-RSA-AES256-SHA; 24 Oct 2018 08:29:37 +0000 Received: from u54ee758033e858cfa736.ant.amazon.com (iad7-ws-svc-lb50-vlan2.amazon.com [10.0.93.210]) by email-inbound-relay-1a-16acd5e0.us-east-1.amazon.com (8.14.7/8.14.7) with ESMTP id w9O8TUfr031968 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 24 Oct 2018 08:29:33 GMT Received: from u54ee758033e858cfa736.ant.amazon.com (localhost [127.0.0.1]) by u54ee758033e858cfa736.ant.amazon.com (8.15.2/8.15.2/Debian-3) with ESMTP id w9O8TT2d029692; Wed, 24 Oct 2018 10:29:29 +0200 Received: (from jsteckli@localhost) by u54ee758033e858cfa736.ant.amazon.com (8.15.2/8.15.2/Submit) id w9O8TTvq029688; Wed, 24 Oct 2018 10:29:29 +0200 From: Julian Stecklina To: kvm@vger.kernel.org, Paolo Bonzini Cc: Julian Stecklina , js@alien8.de, linux-kernel@vger.kernel.org Subject: [PATCH 1/4] kvm, vmx: move CR2 context switch out of assembly path Date: Wed, 24 Oct 2018 10:28:56 +0200 Message-Id: <09986c98c9655f1542768ecfda644ac821e67a57.1540369608.git.jsteckli@amazon.de> X-Mailer: git-send-email 2.7.4 MIME-Version: 1.0 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The VM entry/exit path is a giant inline assembly statement. Simplify it by doing CR2 context switching in plain C. Move CR2 restore behind IBRS clearing, so we reduce the amount of code we execute with IBRS on. Signed-off-by: Julian Stecklina Reviewed-by: Jan H. Schönherr Reviewed-by: Konrad Jan Miller Reviewed-by: Jim Mattson --- arch/x86/kvm/vmx.c | 15 +++++---------- 1 file changed, 5 insertions(+), 10 deletions(-) diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c index e665aa7..93562d5 100644 --- a/arch/x86/kvm/vmx.c +++ b/arch/x86/kvm/vmx.c @@ -10728,6 +10728,9 @@ static void __noclone vmx_vcpu_run(struct kvm_vcpu *vcpu) evmcs_rsp = static_branch_unlikely(&enable_evmcs) ? (unsigned long)¤t_evmcs->host_rsp : 0; + if (read_cr2() != vcpu->arch.cr2) + write_cr2(vcpu->arch.cr2); + if (static_branch_unlikely(&vmx_l1d_should_flush)) vmx_l1d_flush(vcpu); @@ -10747,13 +10750,6 @@ static void __noclone vmx_vcpu_run(struct kvm_vcpu *vcpu) "2: \n\t" __ex(ASM_VMX_VMWRITE_RSP_RDX) "\n\t" "1: \n\t" - /* Reload cr2 if changed */ - "mov %c[cr2](%0), %%" _ASM_AX " \n\t" - "mov %%cr2, %%" _ASM_DX " \n\t" - "cmp %%" _ASM_AX ", %%" _ASM_DX " \n\t" - "je 3f \n\t" - "mov %%" _ASM_AX", %%cr2 \n\t" - "3: \n\t" /* Check if vmlaunch of vmresume is needed */ "cmpl $0, %c[launched](%0) \n\t" /* Load guest registers. Don't clobber flags. */ @@ -10810,8 +10806,6 @@ static void __noclone vmx_vcpu_run(struct kvm_vcpu *vcpu) "xor %%r14d, %%r14d \n\t" "xor %%r15d, %%r15d \n\t" #endif - "mov %%cr2, %%" _ASM_AX " \n\t" - "mov %%" _ASM_AX ", %c[cr2](%0) \n\t" "xor %%eax, %%eax \n\t" "xor %%ebx, %%ebx \n\t" @@ -10843,7 +10837,6 @@ static void __noclone vmx_vcpu_run(struct kvm_vcpu *vcpu) [r14]"i"(offsetof(struct vcpu_vmx, vcpu.arch.regs[VCPU_REGS_R14])), [r15]"i"(offsetof(struct vcpu_vmx, vcpu.arch.regs[VCPU_REGS_R15])), #endif - [cr2]"i"(offsetof(struct vcpu_vmx, vcpu.arch.cr2)), [wordsize]"i"(sizeof(ulong)) : "cc", "memory" #ifdef CONFIG_X86_64 @@ -10877,6 +10870,8 @@ static void __noclone vmx_vcpu_run(struct kvm_vcpu *vcpu) /* Eliminate branch target predictions from guest mode */ vmexit_fill_RSB(); + vcpu->arch.cr2 = read_cr2(); + /* All fields are clean at this point */ if (static_branch_unlikely(&enable_evmcs)) current_evmcs->hv_clean_fields |= From patchwork Wed Oct 24 08:28:57 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Julian Stecklina X-Patchwork-Id: 10654381 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AAE0E13A4 for ; Wed, 24 Oct 2018 08:29:51 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 956F92A170 for ; Wed, 24 Oct 2018 08:29:51 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 89BB72A1C0; Wed, 24 Oct 2018 08:29:51 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 61BFC2A19D for ; Wed, 24 Oct 2018 08:29:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727444AbeJXQ4v (ORCPT ); Wed, 24 Oct 2018 12:56:51 -0400 Received: from smtp-fw-2101.amazon.com ([72.21.196.25]:47755 "EHLO smtp-fw-2101.amazon.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726267AbeJXQ4v (ORCPT ); Wed, 24 Oct 2018 12:56:51 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.de; i=@amazon.de; q=dns/txt; s=amazon201209; t=1540369782; x=1571905782; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=5sqDDn7QP7sBKIfJPTSAIDBRqBS3u0CYMIP5f/m9YTo=; b=H8fIi0NQckGtDMIdojY9H6PbWpp9t59nOMM8mx3cC/6JecjoWMetEwu5 HYpf2105umSBBMp92wYHj4SztDiqR+3Kd2W70sIgvE3cwkRisuz775P19 YlQ1ZXs93AiZFGYn4c6buhiJLDdKyH6IbUHLUclYV+S24hR4Gb3c3+4Ph 0=; X-IronPort-AV: E=Sophos;i="5.54,419,1534809600"; d="scan'208";a="701226933" Received: from iad6-co-svc-p1-lb1-vlan2.amazon.com (HELO email-inbound-relay-2b-2eab95aa.us-west-2.amazon.com) ([10.124.125.2]) by smtp-border-fw-out-2101.iad2.amazon.com with ESMTP/TLS/DHE-RSA-AES256-SHA; 24 Oct 2018 08:29:41 +0000 Received: from u54ee758033e858cfa736.ant.amazon.com (pdx2-ws-svc-lb17-vlan2.amazon.com [10.247.140.66]) by email-inbound-relay-2b-2eab95aa.us-west-2.amazon.com (8.14.7/8.14.7) with ESMTP id w9O8TbpF022039 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 24 Oct 2018 08:29:38 GMT Received: from u54ee758033e858cfa736.ant.amazon.com (localhost [127.0.0.1]) by u54ee758033e858cfa736.ant.amazon.com (8.15.2/8.15.2/Debian-3) with ESMTP id w9O8Ta9A030178; Wed, 24 Oct 2018 10:29:36 +0200 Received: (from jsteckli@localhost) by u54ee758033e858cfa736.ant.amazon.com (8.15.2/8.15.2/Submit) id w9O8TaIe030177; Wed, 24 Oct 2018 10:29:36 +0200 From: Julian Stecklina To: kvm@vger.kernel.org, Paolo Bonzini Cc: Julian Stecklina , js@alien8.de, linux-kernel@vger.kernel.org Subject: [PATCH 2/4] kvm, vmx: move register clearing out of assembly path Date: Wed, 24 Oct 2018 10:28:57 +0200 Message-Id: <558fea0b4df498eefcaea5ae07a089ad9706c1a2.1540369608.git.jsteckli@amazon.de> X-Mailer: git-send-email 2.7.4 In-Reply-To: <09986c98c9655f1542768ecfda644ac821e67a57.1540369608.git.jsteckli@amazon.de> References: <09986c98c9655f1542768ecfda644ac821e67a57.1540369608.git.jsteckli@amazon.de> In-Reply-To: <09986c98c9655f1542768ecfda644ac821e67a57.1540369608.git.jsteckli@amazon.de> References: <09986c98c9655f1542768ecfda644ac821e67a57.1540369608.git.jsteckli@amazon.de> MIME-Version: 1.0 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Split the security related register clearing out of the large inline assembly VM entry path. This results in two slightly less complicated inline assembly statements, where it is clearer what each one does. Signed-off-by: Julian Stecklina Reviewed-by: Jan H. Schönherr Reviewed-by: Konrad Jan Miller Reviewed-by: Jim Mattson --- arch/x86/kvm/vmx.c | 33 ++++++++++++++++++++------------- 1 file changed, 20 insertions(+), 13 deletions(-) diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c index 93562d5..9225099 100644 --- a/arch/x86/kvm/vmx.c +++ b/arch/x86/kvm/vmx.c @@ -10797,20 +10797,7 @@ static void __noclone vmx_vcpu_run(struct kvm_vcpu *vcpu) "mov %%r13, %c[r13](%0) \n\t" "mov %%r14, %c[r14](%0) \n\t" "mov %%r15, %c[r15](%0) \n\t" - "xor %%r8d, %%r8d \n\t" - "xor %%r9d, %%r9d \n\t" - "xor %%r10d, %%r10d \n\t" - "xor %%r11d, %%r11d \n\t" - "xor %%r12d, %%r12d \n\t" - "xor %%r13d, %%r13d \n\t" - "xor %%r14d, %%r14d \n\t" - "xor %%r15d, %%r15d \n\t" #endif - - "xor %%eax, %%eax \n\t" - "xor %%ebx, %%ebx \n\t" - "xor %%esi, %%esi \n\t" - "xor %%edi, %%edi \n\t" "pop %%" _ASM_BP "; pop %%" _ASM_DX " \n\t" ".pushsection .rodata \n\t" ".global vmx_return \n\t" @@ -10847,6 +10834,26 @@ static void __noclone vmx_vcpu_run(struct kvm_vcpu *vcpu) #endif ); + /* Don't let guest register values survive. */ + asm volatile ( + "" +#ifdef CONFIG_X86_64 + "xor %%r8d, %%r8d \n\t" + "xor %%r9d, %%r9d \n\t" + "xor %%r10d, %%r10d \n\t" + "xor %%r11d, %%r11d \n\t" + "xor %%r12d, %%r12d \n\t" + "xor %%r13d, %%r13d \n\t" + "xor %%r14d, %%r14d \n\t" + "xor %%r15d, %%r15d \n\t" +#endif + :: "a" (0), "b" (0), "S" (0), "D" (0) + : "cc" +#ifdef CONFIG_X86_64 + , "r8", "r9", "r10", "r11", "r12", "r13", "r14", "r15" +#endif + ); + /* * We do not use IBRS in the kernel. If this vCPU has used the * SPEC_CTRL MSR it may have left it on; save the value and From patchwork Wed Oct 24 08:28:58 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Julian Stecklina X-Patchwork-Id: 10654385 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D372A13BF for ; Wed, 24 Oct 2018 08:29:55 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C06572A170 for ; Wed, 24 Oct 2018 08:29:55 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B45F82A1C0; Wed, 24 Oct 2018 08:29:55 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 675BC2A170 for ; Wed, 24 Oct 2018 08:29:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727478AbeJXQ4x (ORCPT ); Wed, 24 Oct 2018 12:56:53 -0400 Received: from smtp-fw-33001.amazon.com ([207.171.190.10]:43720 "EHLO smtp-fw-33001.amazon.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727356AbeJXQ4w (ORCPT ); Wed, 24 Oct 2018 12:56:52 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.de; i=@amazon.de; q=dns/txt; s=amazon201209; t=1540369783; x=1571905783; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=Ia1bA2EwUCIicadXyP2boItppd2fNM58Z5bAjO88G64=; b=IUzGmazBE2smmVykcUPZXSByg+CdWDh9Bo1SNCLjZ9Cd6N3CCNxvJmkW d9PyEC6NSr9nYajrP1FsqxXD9NGIf0JVhNSrUS6dXKFIvUT9LiGoPOpVA 17R8psUJoeSfHM7wWlJcMy8G1D1mjHZFWvfCQf33db0PT+sjp1PiW5CC4 M=; X-IronPort-AV: E=Sophos;i="5.54,419,1534809600"; d="scan'208";a="760955894" Received: from sea3-co-svc-lb6-vlan2.sea.amazon.com (HELO email-inbound-relay-2a-c5104f52.us-west-2.amazon.com) ([10.47.22.34]) by smtp-border-fw-out-33001.sea14.amazon.com with ESMTP/TLS/DHE-RSA-AES256-SHA; 24 Oct 2018 08:29:41 +0000 Received: from u54ee758033e858cfa736.ant.amazon.com (pdx2-ws-svc-lb17-vlan2.amazon.com [10.247.140.66]) by email-inbound-relay-2a-c5104f52.us-west-2.amazon.com (8.14.7/8.14.7) with ESMTP id w9O8TcJ7080732 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 24 Oct 2018 08:29:39 GMT Received: from u54ee758033e858cfa736.ant.amazon.com (localhost [127.0.0.1]) by u54ee758033e858cfa736.ant.amazon.com (8.15.2/8.15.2/Debian-3) with ESMTP id w9O8Tbe2030182; Wed, 24 Oct 2018 10:29:37 +0200 Received: (from jsteckli@localhost) by u54ee758033e858cfa736.ant.amazon.com (8.15.2/8.15.2/Submit) id w9O8Tbjs030181; Wed, 24 Oct 2018 10:29:37 +0200 From: Julian Stecklina To: kvm@vger.kernel.org, Paolo Bonzini Cc: Julian Stecklina , js@alien8.de, linux-kernel@vger.kernel.org Subject: [PATCH 3/4] kvm, vmx: fix __invvpid style Date: Wed, 24 Oct 2018 10:28:58 +0200 Message-Id: <16eda1c463f357607b6d7ae5cfe967055fc17ec5.1540369608.git.jsteckli@amazon.de> X-Mailer: git-send-email 2.7.4 In-Reply-To: <09986c98c9655f1542768ecfda644ac821e67a57.1540369608.git.jsteckli@amazon.de> References: <09986c98c9655f1542768ecfda644ac821e67a57.1540369608.git.jsteckli@amazon.de> In-Reply-To: <09986c98c9655f1542768ecfda644ac821e67a57.1540369608.git.jsteckli@amazon.de> References: <09986c98c9655f1542768ecfda644ac821e67a57.1540369608.git.jsteckli@amazon.de> MIME-Version: 1.0 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The code violated the coding style. Fixed by using tabs instead of spaces. There are only whitespace changes here. Signed-off-by: Julian Stecklina Reviewed-by: Jan H. Schönherr Reviewed-by: Konrad Jan Miller --- arch/x86/kvm/vmx.c | 22 +++++++++++----------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c index 9225099..82cfb909 100644 --- a/arch/x86/kvm/vmx.c +++ b/arch/x86/kvm/vmx.c @@ -2079,17 +2079,17 @@ static int __find_msr_index(struct vcpu_vmx *vmx, u32 msr) static inline void __invvpid(int ext, u16 vpid, gva_t gva) { - struct { - u64 vpid : 16; - u64 rsvd : 48; - u64 gva; - } operand = { vpid, 0, gva }; - bool error; - - asm volatile (__ex(ASM_VMX_INVVPID) CC_SET(na) - : CC_OUT(na) (error) : "a"(&operand), "c"(ext) - : "memory"); - BUG_ON(error); + struct { + u64 vpid : 16; + u64 rsvd : 48; + u64 gva; + } operand = { vpid, 0, gva }; + bool error; + + asm volatile (__ex(ASM_VMX_INVVPID) CC_SET(na) + : CC_OUT(na) (error) : "a"(&operand), "c"(ext) + : "memory"); + BUG_ON(error); } static inline void __invept(int ext, u64 eptp, gpa_t gpa) From patchwork Wed Oct 24 08:28:59 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Julian Stecklina X-Patchwork-Id: 10654383 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 15B2F13BF for ; Wed, 24 Oct 2018 08:29:54 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 010892A170 for ; Wed, 24 Oct 2018 08:29:54 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E95352A1C0; Wed, 24 Oct 2018 08:29:53 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 585472A170 for ; Wed, 24 Oct 2018 08:29:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727521AbeJXQ4y (ORCPT ); Wed, 24 Oct 2018 12:56:54 -0400 Received: from smtp-fw-9101.amazon.com ([207.171.184.25]:56614 "EHLO smtp-fw-9101.amazon.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727356AbeJXQ4y (ORCPT ); Wed, 24 Oct 2018 12:56:54 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.de; i=@amazon.de; q=dns/txt; s=amazon201209; t=1540369786; x=1571905786; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=r9pFQ2+Kis40MqGPKw3NvesWLLEYany1nU1/R2zjamA=; b=CbhgQUnm1UL0B8nq7nYAE/w3q+Q7uJeygT6nR9iQcFDqfbdGHTUpI6AS k5ZIAUbgzi9c8zJumUsOqbvTVV5I+7P6J0AhTQV3gAUL6BoBEMJWH4xN+ khHnM5SmQs7wBHdfVsXcUxZLMo+OjijGD2eKa739mSeinZUn3+T/sbAZC s=; X-IronPort-AV: E=Sophos;i="5.54,419,1534809600"; d="scan'208";a="766031222" Received: from sea3-co-svc-lb6-vlan3.sea.amazon.com (HELO email-inbound-relay-1d-5dd976cd.us-east-1.amazon.com) ([10.47.22.38]) by smtp-border-fw-out-9101.sea19.amazon.com with ESMTP/TLS/DHE-RSA-AES256-SHA; 24 Oct 2018 08:29:45 +0000 Received: from u54ee758033e858cfa736.ant.amazon.com (iad7-ws-svc-lb50-vlan3.amazon.com [10.0.93.214]) by email-inbound-relay-1d-5dd976cd.us-east-1.amazon.com (8.14.7/8.14.7) with ESMTP id w9O8TcdU044798 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 24 Oct 2018 08:29:40 GMT Received: from u54ee758033e858cfa736.ant.amazon.com (localhost [127.0.0.1]) by u54ee758033e858cfa736.ant.amazon.com (8.15.2/8.15.2/Debian-3) with ESMTP id w9O8TcPL030186; Wed, 24 Oct 2018 10:29:38 +0200 Received: (from jsteckli@localhost) by u54ee758033e858cfa736.ant.amazon.com (8.15.2/8.15.2/Submit) id w9O8TcFB030185; Wed, 24 Oct 2018 10:29:38 +0200 From: Julian Stecklina To: kvm@vger.kernel.org, Paolo Bonzini Cc: Julian Stecklina , js@alien8.de, linux-kernel@vger.kernel.org Subject: [PATCH 4/4] kvm, vmx: remove manually coded vmx instructions Date: Wed, 24 Oct 2018 10:28:59 +0200 Message-Id: X-Mailer: git-send-email 2.7.4 In-Reply-To: <09986c98c9655f1542768ecfda644ac821e67a57.1540369608.git.jsteckli@amazon.de> References: <09986c98c9655f1542768ecfda644ac821e67a57.1540369608.git.jsteckli@amazon.de> In-Reply-To: <09986c98c9655f1542768ecfda644ac821e67a57.1540369608.git.jsteckli@amazon.de> References: <09986c98c9655f1542768ecfda644ac821e67a57.1540369608.git.jsteckli@amazon.de> MIME-Version: 1.0 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP So far the VMX code relied on manually assembled VMX instructions. This was apparently done to ensure compatibility with old binutils. VMX instructions were introduced with binutils 2.19 and the kernel currently requires binutils 2.20. Remove the manually assembled versions and replace them with the proper inline assembly. This improves code generation (and source code readability). According to the bloat-o-meter this change removes ~1300 bytes from the text segment. Signed-off-by: Julian Stecklina Reviewed-by: Jan H. Schönherr Reviewed-by: Konrad Jan Miller Reviewed-by: Razvan-Alin Ghitulete --- arch/x86/include/asm/virtext.h | 2 +- arch/x86/include/asm/vmx.h | 13 ------------- arch/x86/kvm/vmx.c | 39 ++++++++++++++++++--------------------- 3 files changed, 19 insertions(+), 35 deletions(-) diff --git a/arch/x86/include/asm/virtext.h b/arch/x86/include/asm/virtext.h index 0116b2e..c5395b3 100644 --- a/arch/x86/include/asm/virtext.h +++ b/arch/x86/include/asm/virtext.h @@ -40,7 +40,7 @@ static inline int cpu_has_vmx(void) */ static inline void cpu_vmxoff(void) { - asm volatile (ASM_VMX_VMXOFF : : : "cc"); + asm volatile ("vmxoff" : : : "cc"); cr4_clear_bits(X86_CR4_VMXE); } diff --git a/arch/x86/include/asm/vmx.h b/arch/x86/include/asm/vmx.h index 9527ba5..ade0f15 100644 --- a/arch/x86/include/asm/vmx.h +++ b/arch/x86/include/asm/vmx.h @@ -503,19 +503,6 @@ enum vmcs_field { #define VMX_EPT_IDENTITY_PAGETABLE_ADDR 0xfffbc000ul - -#define ASM_VMX_VMCLEAR_RAX ".byte 0x66, 0x0f, 0xc7, 0x30" -#define ASM_VMX_VMLAUNCH ".byte 0x0f, 0x01, 0xc2" -#define ASM_VMX_VMRESUME ".byte 0x0f, 0x01, 0xc3" -#define ASM_VMX_VMPTRLD_RAX ".byte 0x0f, 0xc7, 0x30" -#define ASM_VMX_VMREAD_RDX_RAX ".byte 0x0f, 0x78, 0xd0" -#define ASM_VMX_VMWRITE_RAX_RDX ".byte 0x0f, 0x79, 0xd0" -#define ASM_VMX_VMWRITE_RSP_RDX ".byte 0x0f, 0x79, 0xd4" -#define ASM_VMX_VMXOFF ".byte 0x0f, 0x01, 0xc4" -#define ASM_VMX_VMXON_RAX ".byte 0xf3, 0x0f, 0xc7, 0x30" -#define ASM_VMX_INVEPT ".byte 0x66, 0x0f, 0x38, 0x80, 0x08" -#define ASM_VMX_INVVPID ".byte 0x66, 0x0f, 0x38, 0x81, 0x08" - struct vmx_msr_entry { u32 index; u32 reserved; diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c index 82cfb909..bbbdccb 100644 --- a/arch/x86/kvm/vmx.c +++ b/arch/x86/kvm/vmx.c @@ -2077,7 +2077,7 @@ static int __find_msr_index(struct vcpu_vmx *vmx, u32 msr) return -1; } -static inline void __invvpid(int ext, u16 vpid, gva_t gva) +static inline void __invvpid(long ext, u16 vpid, gva_t gva) { struct { u64 vpid : 16; @@ -2086,21 +2086,21 @@ static inline void __invvpid(int ext, u16 vpid, gva_t gva) } operand = { vpid, 0, gva }; bool error; - asm volatile (__ex(ASM_VMX_INVVPID) CC_SET(na) - : CC_OUT(na) (error) : "a"(&operand), "c"(ext) + asm volatile ("invvpid %1, %2" CC_SET(na) + : CC_OUT(na) (error) : "m"(operand), "r"(ext) : "memory"); BUG_ON(error); } -static inline void __invept(int ext, u64 eptp, gpa_t gpa) +static inline void __invept(long ext, u64 eptp, gpa_t gpa) { struct { u64 eptp, gpa; } operand = {eptp, gpa}; bool error; - asm volatile (__ex(ASM_VMX_INVEPT) CC_SET(na) - : CC_OUT(na) (error) : "a" (&operand), "c" (ext) + asm volatile ("invept %1, %2" CC_SET(na) + : CC_OUT(na) (error) : "m" (operand), "r" (ext) : "memory"); BUG_ON(error); } @@ -2120,8 +2120,8 @@ static void vmcs_clear(struct vmcs *vmcs) u64 phys_addr = __pa(vmcs); bool error; - asm volatile (__ex(ASM_VMX_VMCLEAR_RAX) CC_SET(na) - : CC_OUT(na) (error) : "a"(&phys_addr), "m"(phys_addr) + asm volatile ("vmclear %1" CC_SET(na) + : CC_OUT(na) (error) : "m"(phys_addr) : "memory"); if (unlikely(error)) printk(KERN_ERR "kvm: vmclear fail: %p/%llx\n", @@ -2145,8 +2145,8 @@ static void vmcs_load(struct vmcs *vmcs) if (static_branch_unlikely(&enable_evmcs)) return evmcs_load(phys_addr); - asm volatile (__ex(ASM_VMX_VMPTRLD_RAX) CC_SET(na) - : CC_OUT(na) (error) : "a"(&phys_addr), "m"(phys_addr) + asm volatile ("vmptrld %1" CC_SET(na) + : CC_OUT(na) (error) : "m"(phys_addr) : "memory"); if (unlikely(error)) printk(KERN_ERR "kvm: vmptrld %p/%llx failed\n", @@ -2323,8 +2323,7 @@ static __always_inline unsigned long __vmcs_readl(unsigned long field) { unsigned long value; - asm volatile (__ex_clear(ASM_VMX_VMREAD_RDX_RAX, "%0") - : "=a"(value) : "d"(field) : "cc"); + asm volatile ("vmread %1, %0" : "=rm"(value) : "r"(field) : "cc"); return value; } @@ -2375,8 +2374,8 @@ static __always_inline void __vmcs_writel(unsigned long field, unsigned long val { bool error; - asm volatile (__ex(ASM_VMX_VMWRITE_RAX_RDX) CC_SET(na) - : CC_OUT(na) (error) : "a"(value), "d"(field)); + asm volatile ("vmwrite %1, %2" CC_SET(na) + : CC_OUT(na) (error) : "rm"(value), "r"(field)); if (unlikely(error)) vmwrite_error(field, value); } @@ -4397,9 +4396,7 @@ static void kvm_cpu_vmxon(u64 addr) cr4_set_bits(X86_CR4_VMXE); intel_pt_handle_vmx(1); - asm volatile (ASM_VMX_VMXON_RAX - : : "a"(&addr), "m"(addr) - : "memory", "cc"); + asm volatile ("vmxon %0" : : "m"(addr) : "memory", "cc"); } static int hardware_enable(void) @@ -4468,7 +4465,7 @@ static void vmclear_local_loaded_vmcss(void) */ static void kvm_cpu_vmxoff(void) { - asm volatile (__ex(ASM_VMX_VMXOFF) : : : "cc"); + asm volatile ("vmxoff" : : : "cc"); intel_pt_handle_vmx(0); cr4_clear_bits(X86_CR4_VMXE); @@ -10748,7 +10745,7 @@ static void __noclone vmx_vcpu_run(struct kvm_vcpu *vcpu) "mov %%" _ASM_SP ", (%%" _ASM_SI ") \n\t" "jmp 1f \n\t" "2: \n\t" - __ex(ASM_VMX_VMWRITE_RSP_RDX) "\n\t" + "vmwrite %%" _ASM_SP ", %%" _ASM_DX "\n\t" "1: \n\t" /* Check if vmlaunch of vmresume is needed */ "cmpl $0, %c[launched](%0) \n\t" @@ -10773,9 +10770,9 @@ static void __noclone vmx_vcpu_run(struct kvm_vcpu *vcpu) /* Enter guest mode */ "jne 1f \n\t" - __ex(ASM_VMX_VMLAUNCH) "\n\t" + "vmlaunch \n\t" "jmp 2f \n\t" - "1: " __ex(ASM_VMX_VMRESUME) "\n\t" + "1: vmresume \n\t" "2: " /* Save guest registers, load host registers, keep flags */ "mov %0, %c[wordsize](%%" _ASM_SP ") \n\t"