From patchwork Wed Jun 19 16:21:31 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Liran Alon X-Patchwork-Id: 11004613 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9092F1580 for ; Wed, 19 Jun 2019 16:23:03 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 811BB2844E for ; Wed, 19 Jun 2019 16:23:03 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 74971284BD; Wed, 19 Jun 2019 16:23:03 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EE422284C9 for ; Wed, 19 Jun 2019 16:23:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726839AbfFSQXB (ORCPT ); Wed, 19 Jun 2019 12:23:01 -0400 Received: from aserp2120.oracle.com ([141.146.126.78]:49576 "EHLO aserp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725843AbfFSQXB (ORCPT ); Wed, 19 Jun 2019 12:23:01 -0400 Received: from pps.filterd (aserp2120.oracle.com [127.0.0.1]) by aserp2120.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x5JGDhOF030766; Wed, 19 Jun 2019 16:22:05 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2018-07-02; bh=L4C5DBKuJWcIqdSV5rV5f253I/cdGNQGbqVgYGJvinQ=; b=OUpZCnQHLkeSSEwhQmRgYD9qhhXAn64dgooovuQLAn9Wlhq4MbhABk8E2g1Q5NvzrKWS cXba/t1D6RJmfwyCZq1GA/u8NtLSMJa6qv5GLGz/1W7edvo3bGgBKVg2QRaZM9BMFtgS yaxgv4QCoA1YCdSt+dwQb6UlO7rhtdFZrLUmP64idqPx/G6aWieh2/qxxmVEMTkrNwTi w+XpdhvYm/wVCS1bccDdDVIgx5e0hKoxWe7SaBPZzifg/mWQbdlhbabDmNIxsE68+90n aWOWdu2hd4BMoY7wAo0Ut8P86Br6geLNIdGakiutM65tAnZKSozCtjjhc0E2+Rrwvib5 Gw== Received: from userp3020.oracle.com (userp3020.oracle.com [156.151.31.79]) by aserp2120.oracle.com with ESMTP id 2t7809cek0-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 19 Jun 2019 16:22:04 +0000 Received: from pps.filterd (userp3020.oracle.com [127.0.0.1]) by userp3020.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x5JGLE4q187184; Wed, 19 Jun 2019 16:22:04 GMT Received: from aserv0122.oracle.com (aserv0122.oracle.com [141.146.126.236]) by userp3020.oracle.com with ESMTP id 2t77yn6nu6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 19 Jun 2019 16:22:04 +0000 Received: from abhmp0008.oracle.com (abhmp0008.oracle.com [141.146.116.14]) by aserv0122.oracle.com (8.14.4/8.14.4) with ESMTP id x5JGM2Zg028417; Wed, 19 Jun 2019 16:22:02 GMT Received: from spark.ravello.local (/213.57.127.2) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 19 Jun 2019 09:22:02 -0700 From: Liran Alon To: qemu-devel@nongnu.org Cc: pbonzini@redhat.com, mtosatti@redhat.com, rth@twiddle.net, ehabkost@redhat.com, kvm@vger.kernel.org, jmattson@google.com, maran.wilson@oracle.com, dgilbert@redhat.com, Liran Alon Subject: [QEMU PATCH v4 01/10] target/i386: kvm: Delete VMX migration blocker on vCPU init failure Date: Wed, 19 Jun 2019 19:21:31 +0300 Message-Id: <20190619162140.133674-2-liran.alon@oracle.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190619162140.133674-1-liran.alon@oracle.com> References: <20190619162140.133674-1-liran.alon@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9293 signatures=668687 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=3 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=905 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1906190131 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9293 signatures=668687 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=3 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=948 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1906190131 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Commit d98f26073beb ("target/i386: kvm: add VMX migration blocker") added migration blocker for vCPU exposed with Intel VMX because QEMU doesn't yet contain code to support migration of nested virtualization workloads. However, that commit missed adding deletion of the migration blocker in case init of vCPU failed. Similar to invtsc_mig_blocker. This commit fix that issue. Fixes: d98f26073beb ("target/i386: kvm: add VMX migration blocker") Signed-off-by: Liran Alon Reviewed-by: Maran Wilson --- target/i386/kvm.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/target/i386/kvm.c b/target/i386/kvm.c index 3b29ce5c0d08..7aa7914a498c 100644 --- a/target/i386/kvm.c +++ b/target/i386/kvm.c @@ -940,7 +940,7 @@ int kvm_arch_init_vcpu(CPUState *cs) r = kvm_arch_set_tsc_khz(cs); if (r < 0) { - goto fail; + return r; } /* vcpu's TSC frequency is either specified by user, or following @@ -1295,7 +1295,7 @@ int kvm_arch_init_vcpu(CPUState *cs) if (local_err) { error_report_err(local_err); error_free(invtsc_mig_blocker); - return r; + goto fail2; } } } @@ -1346,6 +1346,9 @@ int kvm_arch_init_vcpu(CPUState *cs) fail: migrate_del_blocker(invtsc_mig_blocker); + fail2: + migrate_del_blocker(vmx_mig_blocker); + return r; } From patchwork Wed Jun 19 16:21:32 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Liran Alon X-Patchwork-Id: 11004623 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4A4861395 for ; Wed, 19 Jun 2019 16:23:22 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3B5A52842B for ; Wed, 19 Jun 2019 16:23:22 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2FD9D284D2; Wed, 19 Jun 2019 16:23:22 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B117E2844E for ; Wed, 19 Jun 2019 16:23:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729825AbfFSQXU (ORCPT ); Wed, 19 Jun 2019 12:23:20 -0400 Received: from userp2130.oracle.com ([156.151.31.86]:35864 "EHLO userp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729325AbfFSQXU (ORCPT ); Wed, 19 Jun 2019 12:23:20 -0400 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x5JGDi8a121580; Wed, 19 Jun 2019 16:22:08 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2018-07-02; bh=UXe0e1TANKFu82uWO3nVvsP+tdQVKk7/VXjuuh4srmg=; b=kZ7xEsTS2gBeDq867/ec/BhbjZA47VEjsS/bI2qmmtk3ZespQqnQo2Z6Wr5lfTIPuqiU oedXCPeTtRNIdcT+sExgGwHke5O0JijX5wrclssAM8ZsHZxOeJmXVDAZ7ms+xGTJlkv1 xL+oTzfhrlni6by7khWTJmIvtxtKfJEjgy07kFxAXakMVOEvaL6pkLf8hvWdQ8aHqqiQ ehU+2hnhgdi6dP2SOxBJVC5BlNyEy0lzY9avemBcoPslQF9Md2ayakzfIFYzlgZ+CDsq viNLoBcEzileNyXBJZ23x7bikuABkMrRZfie1J1n+0UZ7hmyONWGf8nLa/3L44wmDk4y Tw== Received: from aserp3030.oracle.com (aserp3030.oracle.com [141.146.126.71]) by userp2130.oracle.com with ESMTP id 2t7809cgec-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 19 Jun 2019 16:22:07 +0000 Received: from pps.filterd (aserp3030.oracle.com [127.0.0.1]) by aserp3030.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x5JGLFLZ060021; Wed, 19 Jun 2019 16:22:07 GMT Received: from userv0121.oracle.com (userv0121.oracle.com [156.151.31.72]) by aserp3030.oracle.com with ESMTP id 2t7rdwr2rg-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 19 Jun 2019 16:22:06 +0000 Received: from abhmp0008.oracle.com (abhmp0008.oracle.com [141.146.116.14]) by userv0121.oracle.com (8.14.4/8.13.8) with ESMTP id x5JGM5xg003070; Wed, 19 Jun 2019 16:22:05 GMT Received: from spark.ravello.local (/213.57.127.2) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 19 Jun 2019 09:22:04 -0700 From: Liran Alon To: qemu-devel@nongnu.org Cc: pbonzini@redhat.com, mtosatti@redhat.com, rth@twiddle.net, ehabkost@redhat.com, kvm@vger.kernel.org, jmattson@google.com, maran.wilson@oracle.com, dgilbert@redhat.com, Liran Alon Subject: [QEMU PATCH v4 02/10] KVM: Introduce kvm_arch_destroy_vcpu() Date: Wed, 19 Jun 2019 19:21:32 +0300 Message-Id: <20190619162140.133674-3-liran.alon@oracle.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190619162140.133674-1-liran.alon@oracle.com> References: <20190619162140.133674-1-liran.alon@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9293 signatures=668687 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=3 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=806 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1906190131 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9293 signatures=668687 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=3 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=849 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1906190131 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Simiar to how kvm_init_vcpu() calls kvm_arch_init_vcpu() to perform arch-dependent initialisation, introduce kvm_arch_destroy_vcpu() to be called from kvm_destroy_vcpu() to perform arch-dependent destruction. This was added because some architectures (Such as i386) currently do not free memory that it have allocated in kvm_arch_init_vcpu(). Suggested-by: Maran Wilson Reviewed-by: Maran Wilson Signed-off-by: Liran Alon --- accel/kvm/kvm-all.c | 5 +++++ include/sysemu/kvm.h | 1 + target/arm/kvm32.c | 5 +++++ target/arm/kvm64.c | 5 +++++ target/i386/kvm.c | 12 ++++++++++++ target/mips/kvm.c | 5 +++++ target/ppc/kvm.c | 5 +++++ target/s390x/kvm.c | 10 ++++++++++ 8 files changed, 48 insertions(+) diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c index 524c4ddfbd0f..59a3aa3a40da 100644 --- a/accel/kvm/kvm-all.c +++ b/accel/kvm/kvm-all.c @@ -292,6 +292,11 @@ int kvm_destroy_vcpu(CPUState *cpu) DPRINTF("kvm_destroy_vcpu\n"); + ret = kvm_arch_destroy_vcpu(cpu); + if (ret < 0) { + goto err; + } + mmap_size = kvm_ioctl(s, KVM_GET_VCPU_MMAP_SIZE, 0); if (mmap_size < 0) { ret = mmap_size; diff --git a/include/sysemu/kvm.h b/include/sysemu/kvm.h index a6d1cd190fed..64f55e519df7 100644 --- a/include/sysemu/kvm.h +++ b/include/sysemu/kvm.h @@ -371,6 +371,7 @@ int kvm_arch_put_registers(CPUState *cpu, int level); int kvm_arch_init(MachineState *ms, KVMState *s); int kvm_arch_init_vcpu(CPUState *cpu); +int kvm_arch_destroy_vcpu(CPUState *cpu); bool kvm_vcpu_id_is_valid(int vcpu_id); diff --git a/target/arm/kvm32.c b/target/arm/kvm32.c index 4e54e372a668..51f78f722b18 100644 --- a/target/arm/kvm32.c +++ b/target/arm/kvm32.c @@ -240,6 +240,11 @@ int kvm_arch_init_vcpu(CPUState *cs) return kvm_arm_init_cpreg_list(cpu); } +int kvm_arch_destroy_vcpu(CPUState *cs) +{ + return 0; +} + typedef struct Reg { uint64_t id; int offset; diff --git a/target/arm/kvm64.c b/target/arm/kvm64.c index 998d21f399f4..22d19c9aec6f 100644 --- a/target/arm/kvm64.c +++ b/target/arm/kvm64.c @@ -654,6 +654,11 @@ int kvm_arch_init_vcpu(CPUState *cs) return kvm_arm_init_cpreg_list(cpu); } +int kvm_arch_destroy_vcpu(CPUState *cs) +{ + return 0; +} + bool kvm_arm_reg_syncs_via_cpreg_list(uint64_t regidx) { /* Return true if the regidx is a register we should synchronize diff --git a/target/i386/kvm.c b/target/i386/kvm.c index 7aa7914a498c..efbecfc9d7f0 100644 --- a/target/i386/kvm.c +++ b/target/i386/kvm.c @@ -1352,6 +1352,18 @@ int kvm_arch_init_vcpu(CPUState *cs) return r; } +int kvm_arch_destroy_vcpu(CPUState *cs) +{ + X86CPU *cpu = X86_CPU(cs); + + if (cpu->kvm_msr_buf) { + g_free(cpu->kvm_msr_buf); + cpu->kvm_msr_buf = NULL; + } + + return 0; +} + void kvm_arch_reset_vcpu(X86CPU *cpu) { CPUX86State *env = &cpu->env; diff --git a/target/mips/kvm.c b/target/mips/kvm.c index 8e72850962e1..938f8f144b74 100644 --- a/target/mips/kvm.c +++ b/target/mips/kvm.c @@ -91,6 +91,11 @@ int kvm_arch_init_vcpu(CPUState *cs) return ret; } +int kvm_arch_destroy_vcpu(CPUState *cs) +{ + return 0; +} + void kvm_mips_reset_vcpu(MIPSCPU *cpu) { CPUMIPSState *env = &cpu->env; diff --git a/target/ppc/kvm.c b/target/ppc/kvm.c index 3bf0a46c3352..1967ccc51791 100644 --- a/target/ppc/kvm.c +++ b/target/ppc/kvm.c @@ -521,6 +521,11 @@ int kvm_arch_init_vcpu(CPUState *cs) return ret; } +int kvm_arch_destroy_vcpu(CPUState *cs) +{ + return 0; +} + static void kvm_sw_tlb_put(PowerPCCPU *cpu) { CPUPPCState *env = &cpu->env; diff --git a/target/s390x/kvm.c b/target/s390x/kvm.c index e5e2b691f253..c2747c31649b 100644 --- a/target/s390x/kvm.c +++ b/target/s390x/kvm.c @@ -368,6 +368,16 @@ int kvm_arch_init_vcpu(CPUState *cs) return 0; } +int kvm_arch_destroy_vcpu(CPUState *cs) +{ + S390CPU *cpu = S390_CPU(cs); + + g_free(cpu->irqstate); + cpu->irqstate = NULL; + + return 0; +} + void kvm_s390_reset_vcpu(S390CPU *cpu) { CPUState *cs = CPU(cpu); From patchwork Wed Jun 19 16:21:33 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Liran Alon X-Patchwork-Id: 11004619 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 695A81580 for ; Wed, 19 Jun 2019 16:23:17 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 59B17283AD for ; Wed, 19 Jun 2019 16:23:17 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 46749284C3; Wed, 19 Jun 2019 16:23:17 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EB2A72848B for ; Wed, 19 Jun 2019 16:23:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729142AbfFSQXP (ORCPT ); Wed, 19 Jun 2019 12:23:15 -0400 Received: from userp2120.oracle.com ([156.151.31.85]:36568 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729061AbfFSQXP (ORCPT ); Wed, 19 Jun 2019 12:23:15 -0400 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x5JGDmuS130849; Wed, 19 Jun 2019 16:22:10 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2018-07-02; bh=F+Gcfp9dlPhla7i45Van9Vb6VGMrUmjoVKTqLpj7sQE=; b=Bsul6HY4C0qKV6VafRfLbcqHCusygoQDzerPepsKKJui7qEfZvahulBsQ0Bzko5uc/x9 k0ra+naBc2nphYJmHK8UOql5MyGfVLxFEhIAy60Kr00om6LxO32uOZVSrLMVYCP8hVBH 0+/zcpZeBMAJQofIiHF3AKYuMg0WoRuX27kko7hvzN2FwpAll5EKyoR205SQnBl5yQM6 7OuQGfBzV5yuIXpNhX3jUNRj3Te+/KwnNgGoQReZZyPySLOdYRkPe2vTh2563rMtnSbS wdra7zSBvzahL1x8i9iSav0G6ODxHz8MnqG0rZgmi/KqbbQPX2cTu0ihn6jK/SeHGd7B 4Q== Received: from userp3020.oracle.com (userp3020.oracle.com [156.151.31.79]) by userp2120.oracle.com with ESMTP id 2t7809cfpx-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 19 Jun 2019 16:22:10 +0000 Received: from pps.filterd (userp3020.oracle.com [127.0.0.1]) by userp3020.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x5JGLER3187116; Wed, 19 Jun 2019 16:22:09 GMT Received: from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235]) by userp3020.oracle.com with ESMTP id 2t77yn6nwp-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 19 Jun 2019 16:22:09 +0000 Received: from abhmp0008.oracle.com (abhmp0008.oracle.com [141.146.116.14]) by aserv0121.oracle.com (8.14.4/8.13.8) with ESMTP id x5JGM8Yu000727; Wed, 19 Jun 2019 16:22:08 GMT Received: from spark.ravello.local (/213.57.127.2) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 19 Jun 2019 09:22:08 -0700 From: Liran Alon To: qemu-devel@nongnu.org Cc: pbonzini@redhat.com, mtosatti@redhat.com, rth@twiddle.net, ehabkost@redhat.com, kvm@vger.kernel.org, jmattson@google.com, maran.wilson@oracle.com, dgilbert@redhat.com, Liran Alon , Nikita Leshenko , Krish Sadhukhan Subject: [QEMU PATCH v4 03/10] target/i386: kvm: Use symbolic constant for #DB/#BP exception constants Date: Wed, 19 Jun 2019 19:21:33 +0300 Message-Id: <20190619162140.133674-4-liran.alon@oracle.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190619162140.133674-1-liran.alon@oracle.com> References: <20190619162140.133674-1-liran.alon@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9293 signatures=668687 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=1 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=913 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1906190131 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9293 signatures=668687 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=952 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1906190131 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Reviewed-by: Nikita Leshenko Reviewed-by: Krish Sadhukhan Signed-off-by: Liran Alon --- target/i386/kvm.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/target/i386/kvm.c b/target/i386/kvm.c index efbecfc9d7f0..7b3b80833fdd 100644 --- a/target/i386/kvm.c +++ b/target/i386/kvm.c @@ -3009,9 +3009,9 @@ static int kvm_guest_debug_workarounds(X86CPU *cpu) unsigned long reinject_trap = 0; if (!kvm_has_vcpu_events()) { - if (env->exception_injected == 1) { + if (env->exception_injected == EXCP01_DB) { reinject_trap = KVM_GUESTDBG_INJECT_DB; - } else if (env->exception_injected == 3) { + } else if (env->exception_injected == EXCP03_INT3) { reinject_trap = KVM_GUESTDBG_INJECT_BP; } env->exception_injected = -1; @@ -3523,8 +3523,8 @@ static int kvm_handle_debug(X86CPU *cpu, int ret = 0; int n; - if (arch_info->exception == 1) { - if (arch_info->dr6 & (1 << 14)) { + if (arch_info->exception == EXCP01_DB) { + if (arch_info->dr6 & DR6_BS) { if (cs->singlestep_enabled) { ret = EXCP_DEBUG; } From patchwork Wed Jun 19 16:21:34 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Liran Alon X-Patchwork-Id: 11004621 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DBCD91395 for ; Wed, 19 Jun 2019 16:23:19 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CC0102842B for ; Wed, 19 Jun 2019 16:23:19 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C0869284C5; Wed, 19 Jun 2019 16:23:19 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E2D532848B for ; Wed, 19 Jun 2019 16:23:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729188AbfFSQXR (ORCPT ); Wed, 19 Jun 2019 12:23:17 -0400 Received: from aserp2120.oracle.com ([141.146.126.78]:49896 "EHLO aserp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726518AbfFSQXR (ORCPT ); Wed, 19 Jun 2019 12:23:17 -0400 Received: from pps.filterd (aserp2120.oracle.com [127.0.0.1]) by aserp2120.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x5JGDn2w030848; Wed, 19 Jun 2019 16:22:13 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2018-07-02; bh=DBjYNF4/pkt24mrwuEAr3h34TQt4sLfFgFuRGeEq4+Q=; b=OJCEG7AifwUmQe9qTLr7tlefubpKQPr8SqsrsVuiAJlEeRhmS/Emlk23CrlgbA01vhYJ /15+DqaREVuxVr0Y1EiyoPKPk7irXuGdj6Cfdk5iahCe9eI3r63/cLITi6FkYIjl87o5 FcHtb4eZ9ND64IeTBogHmQ0SsAanRkoe7n6Dw5qhfbYPZny5j0tcKr04GBicRYoP+pR1 QbwQ9m6HqLWqvj6a4zlHI4SXMQjENL2dzJ1532/Ef24xHAYlia84naRAG6egRThdQcEb o/PG7EmOCmNHkNKz77mnMe9woNbvruShHeyLenBkJ+DhJNF/m3uew/hZ+uGF7QUP/4Oc Ug== Received: from userp3030.oracle.com (userp3030.oracle.com [156.151.31.80]) by aserp2120.oracle.com with ESMTP id 2t7809cekq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 19 Jun 2019 16:22:13 +0000 Received: from pps.filterd (userp3030.oracle.com [127.0.0.1]) by userp3030.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x5JGLbur133107; Wed, 19 Jun 2019 16:22:12 GMT Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by userp3030.oracle.com with ESMTP id 2t77yn6pwk-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 19 Jun 2019 16:22:12 +0000 Received: from abhmp0008.oracle.com (abhmp0008.oracle.com [141.146.116.14]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id x5JGMBrI024629; Wed, 19 Jun 2019 16:22:11 GMT Received: from spark.ravello.local (/213.57.127.2) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 19 Jun 2019 09:22:11 -0700 From: Liran Alon To: qemu-devel@nongnu.org Cc: pbonzini@redhat.com, mtosatti@redhat.com, rth@twiddle.net, ehabkost@redhat.com, kvm@vger.kernel.org, jmattson@google.com, maran.wilson@oracle.com, dgilbert@redhat.com, Liran Alon , Nikita Leshenko , Krish Sadhukhan Subject: [QEMU PATCH v4 04/10] target/i386: kvm: Re-inject #DB to guest with updated DR6 Date: Wed, 19 Jun 2019 19:21:34 +0300 Message-Id: <20190619162140.133674-5-liran.alon@oracle.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190619162140.133674-1-liran.alon@oracle.com> References: <20190619162140.133674-1-liran.alon@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9293 signatures=668687 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=1 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1906190131 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9293 signatures=668687 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1906190131 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP If userspace (QEMU) debug guest, when #DB is raised in guest and intercepted by KVM, KVM forwards information on #DB to userspace instead of injecting #DB to guest. While doing so, KVM don't update vCPU DR6 but instead report the #DB DR6 value to userspace for further handling. See KVM's handle_exception() DB_VECTOR handler. QEMU handler for this case is kvm_handle_debug(). This handler basically checks if #DB is related to one of user set hardware breakpoints and if not, it re-inject #DB into guest. The re-injection is done by setting env->exception_injected to #DB which will later be passed as events.exception.nr to KVM_SET_VCPU_EVENTS ioctl by kvm_put_vcpu_events(). However, in case userspace re-injects #DB, KVM expects userspace to set vCPU DR6 as reported to userspace when #DB was intercepted! Otherwise, KVM_REQ_EVENT handler will inject #DB with wrong DR6 to guest. Fix this issue by updating vCPU DR6 appropriately when re-inject #DB to guest. Reviewed-by: Nikita Leshenko Reviewed-by: Krish Sadhukhan Signed-off-by: Liran Alon --- target/i386/kvm.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/target/i386/kvm.c b/target/i386/kvm.c index 7b3b80833fdd..fa01f18b28f0 100644 --- a/target/i386/kvm.c +++ b/target/i386/kvm.c @@ -3561,6 +3561,9 @@ static int kvm_handle_debug(X86CPU *cpu, /* pass to guest */ env->exception_injected = arch_info->exception; env->has_error_code = 0; + if (arch_info->exception == EXCP01_DB) { + env->dr[6] = arch_info->dr6; + } } return ret; From patchwork Wed Jun 19 16:21:35 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Liran Alon X-Patchwork-Id: 11004627 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EE7B41580 for ; Wed, 19 Jun 2019 16:23:25 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DCB42283AD for ; Wed, 19 Jun 2019 16:23:25 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D0EC5284C9; Wed, 19 Jun 2019 16:23:25 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6081D283AD for ; Wed, 19 Jun 2019 16:23:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729890AbfFSQXY (ORCPT ); Wed, 19 Jun 2019 12:23:24 -0400 Received: from aserp2120.oracle.com ([141.146.126.78]:50070 "EHLO aserp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729670AbfFSQXX (ORCPT ); Wed, 19 Jun 2019 12:23:23 -0400 Received: from pps.filterd (aserp2120.oracle.com [127.0.0.1]) by aserp2120.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x5JGDiCv030793; Wed, 19 Jun 2019 16:22:16 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2018-07-02; bh=5+VDtEHpYCvXTvJzElp/9F5GUAkU9Kk5VcCDhZwomMI=; b=FxzokixQ03Pj1zbmRAjnhM3BdKOVjkDlE8xNcYqSg5v4PlJ5VbqGsToZCDJPtuBgSuOn OfFdZ4aqQk2RK7WgWGibmb7kP/bllezYP0IuiTc2p5EshwG014HxPJ8mb/VWnSE2d7cR /xvYPyvgFYc19aEuiqEzgZVnyO66K7BqZQXhEuqrT4EkIGy+VBthkrl1YxJasHT+Fq2B Zseyby1AOk8WjKwokOWmXUJju7LXr6A0sTAAHMEpezWsnjveiTF/P5Lw45NImSKVnNlE +m9a4PX7u2zEi2VQucZ37woPdx4eeqMNV+F2ONp/CULOVvUz7uOsE0LZTtccllNyCNsB HQ== Received: from userp3020.oracle.com (userp3020.oracle.com [156.151.31.79]) by aserp2120.oracle.com with ESMTP id 2t7809cem3-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 19 Jun 2019 16:22:16 +0000 Received: from pps.filterd (userp3020.oracle.com [127.0.0.1]) by userp3020.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x5JGLEHp187176; Wed, 19 Jun 2019 16:22:15 GMT Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by userp3020.oracle.com with ESMTP id 2t77yn6nxq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 19 Jun 2019 16:22:15 +0000 Received: from abhmp0008.oracle.com (abhmp0008.oracle.com [141.146.116.14]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id x5JGMEMO024652; Wed, 19 Jun 2019 16:22:14 GMT Received: from spark.ravello.local (/213.57.127.2) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 19 Jun 2019 09:22:14 -0700 From: Liran Alon To: qemu-devel@nongnu.org Cc: pbonzini@redhat.com, mtosatti@redhat.com, rth@twiddle.net, ehabkost@redhat.com, kvm@vger.kernel.org, jmattson@google.com, maran.wilson@oracle.com, dgilbert@redhat.com, Liran Alon Subject: [QEMU PATCH v4 05/10] target/i386: kvm: Block migration for vCPUs exposed with nested virtualization Date: Wed, 19 Jun 2019 19:21:35 +0300 Message-Id: <20190619162140.133674-6-liran.alon@oracle.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190619162140.133674-1-liran.alon@oracle.com> References: <20190619162140.133674-1-liran.alon@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9293 signatures=668687 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=3 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1906190131 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9293 signatures=668687 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=3 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1906190131 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Commit d98f26073beb ("target/i386: kvm: add VMX migration blocker") added a migration blocker for vCPU exposed with Intel VMX. However, migration should also be blocked for vCPU exposed with AMD SVM. Both cases should be blocked because QEMU should extract additional vCPU state from KVM that should be migrated as part of vCPU VMState. E.g. Whether vCPU is running in guest-mode or host-mode. Fixes: d98f26073beb ("target/i386: kvm: add VMX migration blocker") Reviewed-by: Maran Wilson Signed-off-by: Liran Alon --- target/i386/cpu.c | 6 ------ target/i386/cpu.h | 22 ++++++++++++++++++++++ target/i386/kvm.c | 14 +++++++------- 3 files changed, 29 insertions(+), 13 deletions(-) diff --git a/target/i386/cpu.c b/target/i386/cpu.c index 536d7d152044..197201087e65 100644 --- a/target/i386/cpu.c +++ b/target/i386/cpu.c @@ -5170,12 +5170,6 @@ static int x86_cpu_filter_features(X86CPU *cpu) return rv; } -#define IS_INTEL_CPU(env) ((env)->cpuid_vendor1 == CPUID_VENDOR_INTEL_1 && \ - (env)->cpuid_vendor2 == CPUID_VENDOR_INTEL_2 && \ - (env)->cpuid_vendor3 == CPUID_VENDOR_INTEL_3) -#define IS_AMD_CPU(env) ((env)->cpuid_vendor1 == CPUID_VENDOR_AMD_1 && \ - (env)->cpuid_vendor2 == CPUID_VENDOR_AMD_2 && \ - (env)->cpuid_vendor3 == CPUID_VENDOR_AMD_3) static void x86_cpu_realizefn(DeviceState *dev, Error **errp) { CPUState *cs = CPU(dev); diff --git a/target/i386/cpu.h b/target/i386/cpu.h index fce6660bac00..64bb7fdcb231 100644 --- a/target/i386/cpu.h +++ b/target/i386/cpu.h @@ -728,6 +728,13 @@ typedef uint32_t FeatureWordArray[FEATURE_WORDS]; #define CPUID_VENDOR_HYGON "HygonGenuine" +#define IS_INTEL_CPU(env) ((env)->cpuid_vendor1 == CPUID_VENDOR_INTEL_1 && \ + (env)->cpuid_vendor2 == CPUID_VENDOR_INTEL_2 && \ + (env)->cpuid_vendor3 == CPUID_VENDOR_INTEL_3) +#define IS_AMD_CPU(env) ((env)->cpuid_vendor1 == CPUID_VENDOR_AMD_1 && \ + (env)->cpuid_vendor2 == CPUID_VENDOR_AMD_2 && \ + (env)->cpuid_vendor3 == CPUID_VENDOR_AMD_3) + #define CPUID_MWAIT_IBE (1U << 1) /* Interrupts can exit capability */ #define CPUID_MWAIT_EMX (1U << 0) /* enumeration supported */ @@ -1866,6 +1873,21 @@ static inline int32_t x86_get_a20_mask(CPUX86State *env) } } +static inline bool cpu_has_vmx(CPUX86State *env) +{ + return (env->features[FEAT_1_ECX] & CPUID_EXT_VMX); +} + +static inline bool cpu_has_svm(CPUX86State *env) +{ + return (env->features[FEAT_8000_0001_ECX] & CPUID_EXT3_SVM); +} + +static inline bool cpu_has_nested_virt(CPUX86State *env) +{ + return (cpu_has_vmx(env) || cpu_has_svm(env)); +} + /* fpu_helper.c */ void update_fp_status(CPUX86State *env); void update_mxcsr_status(CPUX86State *env); diff --git a/target/i386/kvm.c b/target/i386/kvm.c index fa01f18b28f0..b2041ce5ec3c 100644 --- a/target/i386/kvm.c +++ b/target/i386/kvm.c @@ -906,7 +906,7 @@ static int hyperv_init_vcpu(X86CPU *cpu) } static Error *invtsc_mig_blocker; -static Error *vmx_mig_blocker; +static Error *nested_virt_mig_blocker; #define KVM_MAX_CPUID_ENTRIES 100 @@ -1270,13 +1270,13 @@ int kvm_arch_init_vcpu(CPUState *cs) !!(c->ecx & CPUID_EXT_SMX); } - if ((env->features[FEAT_1_ECX] & CPUID_EXT_VMX) && !vmx_mig_blocker) { - error_setg(&vmx_mig_blocker, - "Nested VMX virtualization does not support live migration yet"); - r = migrate_add_blocker(vmx_mig_blocker, &local_err); + if (cpu_has_nested_virt(env) && !nested_virt_mig_blocker) { + error_setg(&nested_virt_mig_blocker, + "Nested virtualization does not support live migration yet"); + r = migrate_add_blocker(nested_virt_mig_blocker, &local_err); if (local_err) { error_report_err(local_err); - error_free(vmx_mig_blocker); + error_free(nested_virt_mig_blocker); return r; } } @@ -1347,7 +1347,7 @@ int kvm_arch_init_vcpu(CPUState *cs) fail: migrate_del_blocker(invtsc_mig_blocker); fail2: - migrate_del_blocker(vmx_mig_blocker); + migrate_del_blocker(nested_virt_mig_blocker); return r; } From patchwork Wed Jun 19 16:21:36 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Liran Alon X-Patchwork-Id: 11004633 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 89CEE1395 for ; Wed, 19 Jun 2019 16:23:38 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7A02E2848B for ; Wed, 19 Jun 2019 16:23:38 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 64F00284D4; Wed, 19 Jun 2019 16:23:38 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FUZZY_XPILL,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 01973284D2 for ; Wed, 19 Jun 2019 16:23:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729957AbfFSQXg (ORCPT ); Wed, 19 Jun 2019 12:23:36 -0400 Received: from aserp2120.oracle.com ([141.146.126.78]:50354 "EHLO aserp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727134AbfFSQXg (ORCPT ); Wed, 19 Jun 2019 12:23:36 -0400 Received: from pps.filterd (aserp2120.oracle.com [127.0.0.1]) by aserp2120.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x5JGDhnv030761; Wed, 19 Jun 2019 16:22:19 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2018-07-02; bh=7JwQ662wv964waE5io4PviuPTvJ3njoDQMuOoUYwh78=; b=yVlGhYH9s4cju6A+RfLtAizkhN3IAtnIvsI1mE9YF2txcACfSChxOvw0ZCBrPCLD+20P pBQNeN+Vmd7A/ICbklTWjqwz2Aetm73duDv8Q+Oi2UifJmHr7/edRpla4ZBhZLpuecrn iVmCI2nSNQlVx26mgsgHkWxSvpSORbXqzTiEBF4aOT/KMpI21imUa5uQJBr/+rUliwgv li5Sq4dYeFGStH4ePHvHJNBwcfrjRtEVMSAvLaV35eZvOnjyVzybkjOvram9FQaeNGd/ CuhxEiBuMZit/VqIHRZs5/qwTbRKGM3oMPoFT/gqdFLxVcOpuhIFpUfoTzFScWG9o5RX AA== Received: from userp3030.oracle.com (userp3030.oracle.com [156.151.31.80]) by aserp2120.oracle.com with ESMTP id 2t7809cem9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 19 Jun 2019 16:22:18 +0000 Received: from pps.filterd (userp3030.oracle.com [127.0.0.1]) by userp3030.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x5JGLbw1132953; Wed, 19 Jun 2019 16:22:18 GMT Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by userp3030.oracle.com with ESMTP id 2t77yn6pxn-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 19 Jun 2019 16:22:18 +0000 Received: from abhmp0008.oracle.com (abhmp0008.oracle.com [141.146.116.14]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id x5JGMHmV024699; Wed, 19 Jun 2019 16:22:17 GMT Received: from spark.ravello.local (/213.57.127.2) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 19 Jun 2019 09:22:16 -0700 From: Liran Alon To: qemu-devel@nongnu.org Cc: pbonzini@redhat.com, mtosatti@redhat.com, rth@twiddle.net, ehabkost@redhat.com, kvm@vger.kernel.org, jmattson@google.com, maran.wilson@oracle.com, dgilbert@redhat.com, Liran Alon Subject: [QEMU PATCH v4 06/10] linux-headers: i386: Modify struct kvm_nested_state to have explicit fields for data Date: Wed, 19 Jun 2019 19:21:36 +0300 Message-Id: <20190619162140.133674-7-liran.alon@oracle.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190619162140.133674-1-liran.alon@oracle.com> References: <20190619162140.133674-1-liran.alon@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9293 signatures=668687 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=1 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=798 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1906190131 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9293 signatures=668687 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=847 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1906190131 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Improve the KVM_{GET,SET}_NESTED_STATE structs by detailing the format of VMX nested state data in a struct. In order to avoid changing the ioctl values of KVM_{GET,SET}_NESTED_STATE, there is a need to preserve sizeof(struct kvm_nested_state). This is done by defining the data struct as "data.vmx[0]". It was the most elegant way I found to preserve struct size while still keeping struct readable and easy to maintain. It does have a misfortunate side-effect that now it has to be accessed as "data.vmx[0]" rather than just "data.vmx". Because we are already modifying these structs, I also modified the following: * Define the "format" field values as macros. * Rename vmcs_pa to vmcs12_pa for better readability. Signed-off-by: Liran Alon Reviewed-by: Maran Wilson --- linux-headers/asm-x86/kvm.h | 33 ++++++++++++++++++++++----------- 1 file changed, 22 insertions(+), 11 deletions(-) diff --git a/linux-headers/asm-x86/kvm.h b/linux-headers/asm-x86/kvm.h index 7a0e64ccd6ff..6e7dd792e448 100644 --- a/linux-headers/asm-x86/kvm.h +++ b/linux-headers/asm-x86/kvm.h @@ -383,16 +383,26 @@ struct kvm_sync_regs { #define KVM_X86_QUIRK_LAPIC_MMIO_HOLE (1 << 2) #define KVM_X86_QUIRK_OUT_7E_INC_RIP (1 << 3) +#define KVM_STATE_NESTED_FORMAT_VMX 0 +#define KVM_STATE_NESTED_FORMAT_SVM 1 + #define KVM_STATE_NESTED_GUEST_MODE 0x00000001 #define KVM_STATE_NESTED_RUN_PENDING 0x00000002 #define KVM_STATE_NESTED_EVMCS 0x00000004 +#define KVM_STATE_NESTED_VMX_VMCS_SIZE 0x1000 + #define KVM_STATE_NESTED_SMM_GUEST_MODE 0x00000001 #define KVM_STATE_NESTED_SMM_VMXON 0x00000002 -struct kvm_vmx_nested_state { +struct kvm_vmx_nested_state_data { + __u8 vmcs12[KVM_STATE_NESTED_VMX_VMCS_SIZE]; + __u8 shadow_vmcs12[KVM_STATE_NESTED_VMX_VMCS_SIZE]; +}; + +struct kvm_vmx_nested_state_hdr { __u64 vmxon_pa; - __u64 vmcs_pa; + __u64 vmcs12_pa; struct { __u16 flags; @@ -401,24 +411,25 @@ struct kvm_vmx_nested_state { /* for KVM_CAP_NESTED_STATE */ struct kvm_nested_state { - /* KVM_STATE_* flags */ __u16 flags; - - /* 0 for VMX, 1 for SVM. */ __u16 format; - - /* 128 for SVM, 128 + VMCS size for VMX. */ __u32 size; union { - /* VMXON, VMCS */ - struct kvm_vmx_nested_state vmx; + struct kvm_vmx_nested_state_hdr vmx; /* Pad the header to 128 bytes. */ __u8 pad[120]; - }; + } hdr; - __u8 data[0]; + /* + * Define data region as 0 bytes to preserve backwards-compatability + * to old definition of kvm_nested_state in order to avoid changing + * KVM_{GET,PUT}_NESTED_STATE ioctl values. + */ + union { + struct kvm_vmx_nested_state_data vmx[0]; + } data; }; #endif /* _ASM_X86_KVM_H */ From patchwork Wed Jun 19 16:21:37 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Liran Alon X-Patchwork-Id: 11004617 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3FED01395 for ; Wed, 19 Jun 2019 16:23:12 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 304C42848B for ; Wed, 19 Jun 2019 16:23:12 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 247FD284D4; Wed, 19 Jun 2019 16:23:12 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BE8EE2848B for ; Wed, 19 Jun 2019 16:23:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727818AbfFSQXK (ORCPT ); Wed, 19 Jun 2019 12:23:10 -0400 Received: from aserp2120.oracle.com ([141.146.126.78]:49784 "EHLO aserp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726865AbfFSQXK (ORCPT ); Wed, 19 Jun 2019 12:23:10 -0400 Received: from pps.filterd (aserp2120.oracle.com [127.0.0.1]) by aserp2120.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x5JGDhCo030752; Wed, 19 Jun 2019 16:22:22 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2018-07-02; bh=BWi35KhlexcWG60HSgIA6eOXZUlIDa6NkY02SiUCb8o=; b=uc/h6i/RICxxocMpbyap1Dq6BrvtbJ95h8CG94lhCAvhPW0m2nOH3jgUHDBkKPogRZ88 vjGgYj76w6RfXZ9pic671YpQG6pxnr0u8cHQ4bI190I+1TSTAssy8ofRc4ncjUHb43Rp MUf5Lmnk5iyImpH62rvscB0zyAmM46slVBC5x0TYXtOytly+t6Antnvi9atfZrnMR9jY /7wqyxLmzvD+3gbTIC02WDiPTRUbn/PeyCrMwRr45oMjuoKfw6myVdVhIZ6JWEhIG3kC uxwg/wxsERsjxkvUK4NfsbxoqG3RtuRObFmzvG8k9UCjhZ7hkyUGN5ZN08Ggcink+yDr 0g== Received: from userp3030.oracle.com (userp3030.oracle.com [156.151.31.80]) by aserp2120.oracle.com with ESMTP id 2t7809cemk-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 19 Jun 2019 16:22:21 +0000 Received: from pps.filterd (userp3030.oracle.com [127.0.0.1]) by userp3030.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x5JGLbao133007; Wed, 19 Jun 2019 16:22:21 GMT Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by userp3030.oracle.com with ESMTP id 2t77yn6py7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 19 Jun 2019 16:22:21 +0000 Received: from abhmp0008.oracle.com (abhmp0008.oracle.com [141.146.116.14]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id x5JGMK4C024737; Wed, 19 Jun 2019 16:22:20 GMT Received: from spark.ravello.local (/213.57.127.2) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 19 Jun 2019 09:22:19 -0700 From: Liran Alon To: qemu-devel@nongnu.org Cc: pbonzini@redhat.com, mtosatti@redhat.com, rth@twiddle.net, ehabkost@redhat.com, kvm@vger.kernel.org, jmattson@google.com, maran.wilson@oracle.com, dgilbert@redhat.com, Liran Alon , Nikita Leshenko Subject: [QEMU PATCH v4 07/10] vmstate: Add support for kernel integer types Date: Wed, 19 Jun 2019 19:21:37 +0300 Message-Id: <20190619162140.133674-8-liran.alon@oracle.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190619162140.133674-1-liran.alon@oracle.com> References: <20190619162140.133674-1-liran.alon@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9293 signatures=668687 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=1 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1906190131 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9293 signatures=668687 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1906190131 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Reviewed-by: Nikita Leshenko Reviewed-by: Maran Wilson Signed-off-by: Liran Alon Reviewed-by: Dr. David Alan Gilbert --- include/migration/vmstate.h | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) diff --git a/include/migration/vmstate.h b/include/migration/vmstate.h index 9224370ed59a..ca68584eba4d 100644 --- a/include/migration/vmstate.h +++ b/include/migration/vmstate.h @@ -797,6 +797,19 @@ extern const VMStateInfo vmstate_info_qtailq; #define VMSTATE_UINT64_V(_f, _s, _v) \ VMSTATE_SINGLE(_f, _s, _v, vmstate_info_uint64, uint64_t) +#ifdef CONFIG_LINUX + +#define VMSTATE_U8_V(_f, _s, _v) \ + VMSTATE_SINGLE(_f, _s, _v, vmstate_info_uint8, __u8) +#define VMSTATE_U16_V(_f, _s, _v) \ + VMSTATE_SINGLE(_f, _s, _v, vmstate_info_uint16, __u16) +#define VMSTATE_U32_V(_f, _s, _v) \ + VMSTATE_SINGLE(_f, _s, _v, vmstate_info_uint32, __u32) +#define VMSTATE_U64_V(_f, _s, _v) \ + VMSTATE_SINGLE(_f, _s, _v, vmstate_info_uint64, __u64) + +#endif + #define VMSTATE_BOOL(_f, _s) \ VMSTATE_BOOL_V(_f, _s, 0) @@ -818,6 +831,19 @@ extern const VMStateInfo vmstate_info_qtailq; #define VMSTATE_UINT64(_f, _s) \ VMSTATE_UINT64_V(_f, _s, 0) +#ifdef CONFIG_LINUX + +#define VMSTATE_U8(_f, _s) \ + VMSTATE_U8_V(_f, _s, 0) +#define VMSTATE_U16(_f, _s) \ + VMSTATE_U16_V(_f, _s, 0) +#define VMSTATE_U32(_f, _s) \ + VMSTATE_U32_V(_f, _s, 0) +#define VMSTATE_U64(_f, _s) \ + VMSTATE_U64_V(_f, _s, 0) + +#endif + #define VMSTATE_UINT8_EQUAL(_f, _s, _err_hint) \ VMSTATE_SINGLE_FULL(_f, _s, 0, 0, \ vmstate_info_uint8_equal, uint8_t, _err_hint) From patchwork Wed Jun 19 16:21:38 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Liran Alon X-Patchwork-Id: 11004631 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9B7F91580 for ; Wed, 19 Jun 2019 16:23:37 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 88E4A2842A for ; Wed, 19 Jun 2019 16:23:37 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7D0A52848B; Wed, 19 Jun 2019 16:23:37 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4EED7284C5 for ; Wed, 19 Jun 2019 16:23:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729098AbfFSQXe (ORCPT ); Wed, 19 Jun 2019 12:23:34 -0400 Received: from aserp2120.oracle.com ([141.146.126.78]:50316 "EHLO aserp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729907AbfFSQXe (ORCPT ); Wed, 19 Jun 2019 12:23:34 -0400 Received: from pps.filterd (aserp2120.oracle.com [127.0.0.1]) by aserp2120.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x5JGDrlD030908; Wed, 19 Jun 2019 16:22:25 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2018-07-02; bh=xmlQIZjdzCbiOWyoW3XP6RdI+XBL3vEFkP3dwbFS9ak=; b=JerOC0Ym/dsbk70/zrI2UzOmt9neFOuVX0t3FQTCDAvMt+ZYfH/WsALSVHVDqBA1K3Va 8HqI8K3kFNfpN2cuLEI9QB/4/tfxa2NkLwrovm69D7FntT3MsXNH6Pff65vDtvnWG3e2 HJ2jSdHpbPPO9JGRiy8Dgxv8jQMGpH+WRcIRJFzB3vkJ4H1jq9NTL6cEXoShjMQ631Um etuQFllAW0PVuDsJyvx5X5nuWpq43PEXBcMkzaP59ewaG4y65GnZRMbAwHfDf1E3ZHij FU36qinEp1+WMioXA9ZjcDFaua8cpxh7747iT4siHW7kqA3c4QbqRvXqGT5A7L07Fy+F kQ== Received: from aserp3030.oracle.com (aserp3030.oracle.com [141.146.126.71]) by aserp2120.oracle.com with ESMTP id 2t7809cemv-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 19 Jun 2019 16:22:25 +0000 Received: from pps.filterd (aserp3030.oracle.com [127.0.0.1]) by aserp3030.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x5JGLEtm059888; Wed, 19 Jun 2019 16:22:25 GMT Received: from userv0121.oracle.com (userv0121.oracle.com [156.151.31.72]) by aserp3030.oracle.com with ESMTP id 2t7rdwr2vv-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 19 Jun 2019 16:22:24 +0000 Received: from abhmp0008.oracle.com (abhmp0008.oracle.com [141.146.116.14]) by userv0121.oracle.com (8.14.4/8.13.8) with ESMTP id x5JGMNaw003201; Wed, 19 Jun 2019 16:22:23 GMT Received: from spark.ravello.local (/213.57.127.2) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 19 Jun 2019 09:22:22 -0700 From: Liran Alon To: qemu-devel@nongnu.org Cc: pbonzini@redhat.com, mtosatti@redhat.com, rth@twiddle.net, ehabkost@redhat.com, kvm@vger.kernel.org, jmattson@google.com, maran.wilson@oracle.com, dgilbert@redhat.com, Liran Alon , Nikita Leshenko Subject: [QEMU PATCH v4 08/10] target/i386: kvm: Add support for save and restore nested state Date: Wed, 19 Jun 2019 19:21:38 +0300 Message-Id: <20190619162140.133674-9-liran.alon@oracle.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190619162140.133674-1-liran.alon@oracle.com> References: <20190619162140.133674-1-liran.alon@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9293 signatures=668687 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=3 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1906190131 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9293 signatures=668687 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=3 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1906190131 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Kernel commit 8fcc4b5923af ("kvm: nVMX: Introduce KVM_CAP_NESTED_STATE") introduced new IOCTLs to extract and restore vCPU state related to Intel VMX & AMD SVM. Utilize these IOCTLs to add support for migration of VMs which are running nested hypervisors. Reviewed-by: Nikita Leshenko Reviewed-by: Maran Wilson Tested-by: Maran Wilson Signed-off-by: Liran Alon --- accel/kvm/kvm-all.c | 8 ++ include/sysemu/kvm.h | 1 + target/i386/cpu.h | 3 + target/i386/kvm.c | 80 +++++++++++++++++ target/i386/machine.c | 198 ++++++++++++++++++++++++++++++++++++++++++ 5 files changed, 290 insertions(+) diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c index 59a3aa3a40da..4fdf5b04b131 100644 --- a/accel/kvm/kvm-all.c +++ b/accel/kvm/kvm-all.c @@ -88,6 +88,7 @@ struct KVMState #ifdef KVM_CAP_SET_GUEST_DEBUG QTAILQ_HEAD(, kvm_sw_breakpoint) kvm_sw_breakpoints; #endif + int max_nested_state_len; int many_ioeventfds; int intx_set_mask; bool sync_mmu; @@ -1678,6 +1679,8 @@ static int kvm_init(MachineState *ms) s->debugregs = kvm_check_extension(s, KVM_CAP_DEBUGREGS); #endif + s->max_nested_state_len = kvm_check_extension(s, KVM_CAP_NESTED_STATE); + #ifdef KVM_CAP_IRQ_ROUTING kvm_direct_msi_allowed = (kvm_check_extension(s, KVM_CAP_SIGNAL_MSI) > 0); #endif @@ -2245,6 +2248,11 @@ int kvm_has_debugregs(void) return kvm_state->debugregs; } +int kvm_max_nested_state_length(void) +{ + return kvm_state->max_nested_state_len; +} + int kvm_has_many_ioeventfds(void) { if (!kvm_enabled()) { diff --git a/include/sysemu/kvm.h b/include/sysemu/kvm.h index 64f55e519df7..acd90aebb6c4 100644 --- a/include/sysemu/kvm.h +++ b/include/sysemu/kvm.h @@ -210,6 +210,7 @@ bool kvm_has_sync_mmu(void); int kvm_has_vcpu_events(void); int kvm_has_robust_singlestep(void); int kvm_has_debugregs(void); +int kvm_max_nested_state_length(void); int kvm_has_pit_state2(void); int kvm_has_many_ioeventfds(void); int kvm_has_gsi_routing(void); diff --git a/target/i386/cpu.h b/target/i386/cpu.h index 64bb7fdcb231..c1a42aa8851a 100644 --- a/target/i386/cpu.h +++ b/target/i386/cpu.h @@ -1350,6 +1350,9 @@ typedef struct CPUX86State { #if defined(CONFIG_KVM) || defined(CONFIG_HVF) void *xsave_buf; #endif +#if defined(CONFIG_KVM) + struct kvm_nested_state *nested_state; +#endif #if defined(CONFIG_HVF) HVFX86EmulatorState *hvf_emul; #endif diff --git a/target/i386/kvm.c b/target/i386/kvm.c index b2041ce5ec3c..8ae7598aac3d 100644 --- a/target/i386/kvm.c +++ b/target/i386/kvm.c @@ -931,6 +931,7 @@ int kvm_arch_init_vcpu(CPUState *cs) struct kvm_cpuid_entry2 *c; uint32_t signature[3]; int kvm_base = KVM_CPUID_SIGNATURE; + int max_nested_state_len; int r; Error *local_err = NULL; @@ -1331,6 +1332,24 @@ int kvm_arch_init_vcpu(CPUState *cs) if (has_xsave) { env->xsave_buf = qemu_memalign(4096, sizeof(struct kvm_xsave)); } + + max_nested_state_len = kvm_max_nested_state_length(); + if (max_nested_state_len > 0) { + assert(max_nested_state_len >= offsetof(struct kvm_nested_state, data)); + env->nested_state = g_malloc0(max_nested_state_len); + + env->nested_state->size = max_nested_state_len; + + if (IS_INTEL_CPU(env)) { + struct kvm_vmx_nested_state_hdr *vmx_hdr = + &env->nested_state->hdr.vmx; + + vmx_hdr->vmxon_pa = -1ull; + vmx_hdr->vmcs12_pa = -1ull; + } + + } + cpu->kvm_msr_buf = g_malloc0(MSR_BUF_SIZE); if (!(env->features[FEAT_8000_0001_EDX] & CPUID_EXT2_RDTSCP)) { @@ -1355,12 +1374,18 @@ int kvm_arch_init_vcpu(CPUState *cs) int kvm_arch_destroy_vcpu(CPUState *cs) { X86CPU *cpu = X86_CPU(cs); + CPUX86State *env = &cpu->env; if (cpu->kvm_msr_buf) { g_free(cpu->kvm_msr_buf); cpu->kvm_msr_buf = NULL; } + if (env->nested_state) { + g_free(env->nested_state); + env->nested_state = NULL; + } + return 0; } @@ -3075,6 +3100,52 @@ static int kvm_get_debugregs(X86CPU *cpu) return 0; } +static int kvm_put_nested_state(X86CPU *cpu) +{ + CPUX86State *env = &cpu->env; + int max_nested_state_len = kvm_max_nested_state_length(); + + if (max_nested_state_len <= 0) { + return 0; + } + + assert(env->nested_state->size <= max_nested_state_len); + return kvm_vcpu_ioctl(CPU(cpu), KVM_SET_NESTED_STATE, env->nested_state); +} + +static int kvm_get_nested_state(X86CPU *cpu) +{ + CPUX86State *env = &cpu->env; + int max_nested_state_len = kvm_max_nested_state_length(); + int ret; + + if (max_nested_state_len <= 0) { + return 0; + } + + /* + * It is possible that migration restored a smaller size into + * nested_state->hdr.size than what our kernel support. + * We preserve migration origin nested_state->hdr.size for + * call to KVM_SET_NESTED_STATE but wish that our next call + * to KVM_GET_NESTED_STATE will use max size our kernel support. + */ + env->nested_state->size = max_nested_state_len; + + ret = kvm_vcpu_ioctl(CPU(cpu), KVM_GET_NESTED_STATE, env->nested_state); + if (ret < 0) { + return ret; + } + + if (env->nested_state->flags & KVM_STATE_NESTED_GUEST_MODE) { + env->hflags |= HF_GUEST_MASK; + } else { + env->hflags &= ~HF_GUEST_MASK; + } + + return ret; +} + int kvm_arch_put_registers(CPUState *cpu, int level) { X86CPU *x86_cpu = X86_CPU(cpu); @@ -3082,6 +3153,11 @@ int kvm_arch_put_registers(CPUState *cpu, int level) assert(cpu_is_stopped(cpu) || qemu_cpu_is_self(cpu)); + ret = kvm_put_nested_state(x86_cpu); + if (ret < 0) { + return ret; + } + if (level >= KVM_PUT_RESET_STATE) { ret = kvm_put_msr_feature_control(x86_cpu); if (ret < 0) { @@ -3197,6 +3273,10 @@ int kvm_arch_get_registers(CPUState *cs) if (ret < 0) { goto out; } + ret = kvm_get_nested_state(cpu); + if (ret < 0) { + goto out; + } ret = 0; out: cpu_sync_bndcs_hflags(&cpu->env); diff --git a/target/i386/machine.c b/target/i386/machine.c index 225b5d433bc4..e27115fdb756 100644 --- a/target/i386/machine.c +++ b/target/i386/machine.c @@ -231,6 +231,15 @@ static int cpu_pre_save(void *opaque) env->segs[R_SS].flags &= ~(env->segs[R_SS].flags & DESC_DPL_MASK); } +#ifdef CONFIG_KVM + /* Verify we have nested virtualization state from kernel if required */ + if (cpu_has_nested_virt(env) && !env->nested_state) { + error_report("Guest enabled nested virtualization but kernel " + "does not support saving of nested state"); + return -EINVAL; + } +#endif + return 0; } @@ -278,6 +287,16 @@ static int cpu_post_load(void *opaque, int version_id) env->hflags &= ~HF_CPL_MASK; env->hflags |= (env->segs[R_SS].flags >> DESC_DPL_SHIFT) & HF_CPL_MASK; +#ifdef CONFIG_KVM + if ((env->hflags & HF_GUEST_MASK) && + (!env->nested_state || + !(env->nested_state->flags & KVM_STATE_NESTED_GUEST_MODE))) { + error_report("vCPU set in guest-mode inconsistent with " + "migrated kernel nested state"); + return -EINVAL; + } +#endif + env->fpstt = (env->fpus_vmstate >> 11) & 7; env->fpus = env->fpus_vmstate & ~0x3800; env->fptag_vmstate ^= 0xff; @@ -851,6 +870,182 @@ static const VMStateDescription vmstate_tsc_khz = { } }; +#ifdef CONFIG_KVM + +static bool vmx_vmcs12_needed(void *opaque) +{ + struct kvm_nested_state *nested_state = opaque; + return (nested_state->size > + offsetof(struct kvm_nested_state, data.vmx[0].vmcs12)); +} + +static const VMStateDescription vmstate_vmx_vmcs12 = { + .name = "cpu/kvm_nested_state/vmx/vmcs12", + .version_id = 1, + .minimum_version_id = 1, + .needed = vmx_vmcs12_needed, + .fields = (VMStateField[]) { + VMSTATE_UINT8_ARRAY(data.vmx[0].vmcs12, + struct kvm_nested_state, + KVM_STATE_NESTED_VMX_VMCS_SIZE), + VMSTATE_END_OF_LIST() + } +}; + +static bool vmx_shadow_vmcs12_needed(void *opaque) +{ + struct kvm_nested_state *nested_state = opaque; + return (nested_state->size > + offsetof(struct kvm_nested_state, data.vmx[0].shadow_vmcs12)); +} + +static const VMStateDescription vmstate_vmx_shadow_vmcs12 = { + .name = "cpu/kvm_nested_state/vmx/shadow_vmcs12", + .version_id = 1, + .minimum_version_id = 1, + .needed = vmx_shadow_vmcs12_needed, + .fields = (VMStateField[]) { + VMSTATE_UINT8_ARRAY(data.vmx[0].shadow_vmcs12, + struct kvm_nested_state, + KVM_STATE_NESTED_VMX_VMCS_SIZE), + VMSTATE_END_OF_LIST() + } +}; + +static bool vmx_nested_state_needed(void *opaque) +{ + struct kvm_nested_state *nested_state = opaque; + + return ((nested_state->format == KVM_STATE_NESTED_FORMAT_VMX) && + ((nested_state->hdr.vmx.vmxon_pa != -1ull) || + (nested_state->hdr.vmx.smm.flags & KVM_STATE_NESTED_SMM_VMXON))); +} + +static const VMStateDescription vmstate_vmx_nested_state = { + .name = "cpu/kvm_nested_state/vmx", + .version_id = 1, + .minimum_version_id = 1, + .needed = vmx_nested_state_needed, + .fields = (VMStateField[]) { + VMSTATE_U64(hdr.vmx.vmxon_pa, struct kvm_nested_state), + VMSTATE_U64(hdr.vmx.vmcs12_pa, struct kvm_nested_state), + VMSTATE_U16(hdr.vmx.smm.flags, struct kvm_nested_state), + VMSTATE_END_OF_LIST() + }, + .subsections = (const VMStateDescription*[]) { + &vmstate_vmx_vmcs12, + &vmstate_vmx_shadow_vmcs12, + NULL, + } +}; + +static bool svm_nested_state_needed(void *opaque) +{ + struct kvm_nested_state *nested_state = opaque; + + return (nested_state->format == KVM_STATE_NESTED_FORMAT_SVM); +} + +static const VMStateDescription vmstate_svm_nested_state = { + .name = "cpu/kvm_nested_state/svm", + .version_id = 1, + .minimum_version_id = 1, + .needed = svm_nested_state_needed, + .fields = (VMStateField[]) { + VMSTATE_END_OF_LIST() + } +}; + +static bool nested_state_needed(void *opaque) +{ + X86CPU *cpu = opaque; + CPUX86State *env = &cpu->env; + + return (env->nested_state && + (vmx_nested_state_needed(env->nested_state) || + svm_nested_state_needed(env->nested_state))); +} + +static int nested_state_post_load(void *opaque, int version_id) +{ + X86CPU *cpu = opaque; + CPUX86State *env = &cpu->env; + struct kvm_nested_state *nested_state = env->nested_state; + int min_nested_state_len = offsetof(struct kvm_nested_state, data); + int max_nested_state_len = kvm_max_nested_state_length(); + + /* + * If our kernel don't support setting nested state + * and we have received nested state from migration stream, + * we need to fail migration + */ + if (max_nested_state_len <= 0) { + error_report("Received nested state when kernel cannot restore it"); + return -EINVAL; + } + + /* + * Verify that the size of received nested_state struct + * at least cover required header and is not larger + * than the max size that our kernel support + */ + if (nested_state->size < min_nested_state_len) { + error_report("Received nested state size less than min: " + "len=%d, min=%d", + nested_state->size, min_nested_state_len); + return -EINVAL; + } + if (nested_state->size > max_nested_state_len) { + error_report("Recieved unsupported nested state size: " + "nested_state->size=%d, max=%d", + nested_state->size, max_nested_state_len); + return -EINVAL; + } + + /* Verify format is valid */ + if ((nested_state->format != KVM_STATE_NESTED_FORMAT_VMX) && + (nested_state->format != KVM_STATE_NESTED_FORMAT_SVM)) { + error_report("Received invalid nested state format: %d", + nested_state->format); + return -EINVAL; + } + + return 0; +} + +static const VMStateDescription vmstate_kvm_nested_state = { + .name = "cpu/kvm_nested_state", + .version_id = 1, + .minimum_version_id = 1, + .fields = (VMStateField[]) { + VMSTATE_U16(flags, struct kvm_nested_state), + VMSTATE_U16(format, struct kvm_nested_state), + VMSTATE_U32(size, struct kvm_nested_state), + VMSTATE_END_OF_LIST() + }, + .subsections = (const VMStateDescription*[]) { + &vmstate_vmx_nested_state, + &vmstate_svm_nested_state, + NULL + } +}; + +static const VMStateDescription vmstate_nested_state = { + .name = "cpu/nested_state", + .version_id = 1, + .minimum_version_id = 1, + .needed = nested_state_needed, + .post_load = nested_state_post_load, + .fields = (VMStateField[]) { + VMSTATE_STRUCT_POINTER(env.nested_state, X86CPU, + vmstate_kvm_nested_state, + struct kvm_nested_state), + VMSTATE_END_OF_LIST() + } +}; + +#endif + static bool mcg_ext_ctl_needed(void *opaque) { X86CPU *cpu = opaque; @@ -1089,6 +1284,9 @@ VMStateDescription vmstate_x86_cpu = { &vmstate_msr_intel_pt, &vmstate_msr_virt_ssbd, &vmstate_svm_npt, +#ifdef CONFIG_KVM + &vmstate_nested_state, +#endif NULL } }; From patchwork Wed Jun 19 16:21:39 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Liran Alon X-Patchwork-Id: 11004629 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E34F81395 for ; Wed, 19 Jun 2019 16:23:36 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D30982848B for ; Wed, 19 Jun 2019 16:23:36 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id BEBFC284D2; Wed, 19 Jun 2019 16:23:36 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7B366284D2 for ; Wed, 19 Jun 2019 16:23:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729955AbfFSQXe (ORCPT ); Wed, 19 Jun 2019 12:23:34 -0400 Received: from userp2120.oracle.com ([156.151.31.85]:36898 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729902AbfFSQXe (ORCPT ); Wed, 19 Jun 2019 12:23:34 -0400 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x5JGDhh2130736; Wed, 19 Jun 2019 16:22:28 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2018-07-02; bh=4P0sajkI2HEl+1nJlO7RmwQiRpx3s9JbynLs8+wcZLk=; b=pomzwoTb1sESq6iqUo27liy2anI1TFLfWr9GMsHPSg69VxGIECawMwluoA8G7sHK3X0U r236zdRZ+xIqfgz8YECq9riuWkPOh3TQX08WOxXVx0OPi1mwtrQt0mLc+2oo84OifUNM +psAMdgarZkwxJCQGuiZdlYKA1QZiAACDWbGWeixju4/G/LV5nlS1ly9UiWUQ3NertyM QZ4Iv8oeKWKE8RSJtHAzymsdyuP/utHxlWcdg8f3oo98LV0Yb1ZZ3lOz3d2vE+zQDGaa ZjVSBnWRstn/AqeCNDLEeis32ctSX2aowgY8mh1w+WajSINGGLpJvmmVZK0MQOmM1Y7U sQ== Received: from userp3020.oracle.com (userp3020.oracle.com [156.151.31.79]) by userp2120.oracle.com with ESMTP id 2t7809cfre-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 19 Jun 2019 16:22:27 +0000 Received: from pps.filterd (userp3020.oracle.com [127.0.0.1]) by userp3020.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x5JGLExp187119; Wed, 19 Jun 2019 16:22:27 GMT Received: from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235]) by userp3020.oracle.com with ESMTP id 2t77yn6p0w-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 19 Jun 2019 16:22:27 +0000 Received: from abhmp0008.oracle.com (abhmp0008.oracle.com [141.146.116.14]) by aserv0121.oracle.com (8.14.4/8.13.8) with ESMTP id x5JGMQlJ000894; Wed, 19 Jun 2019 16:22:26 GMT Received: from spark.ravello.local (/213.57.127.2) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 19 Jun 2019 09:22:26 -0700 From: Liran Alon To: qemu-devel@nongnu.org Cc: pbonzini@redhat.com, mtosatti@redhat.com, rth@twiddle.net, ehabkost@redhat.com, kvm@vger.kernel.org, jmattson@google.com, maran.wilson@oracle.com, dgilbert@redhat.com, Liran Alon , Nikita Leshenko Subject: [QEMU PATCH v4 09/10] target/i386: kvm: Add support for KVM_CAP_EXCEPTION_PAYLOAD Date: Wed, 19 Jun 2019 19:21:39 +0300 Message-Id: <20190619162140.133674-10-liran.alon@oracle.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190619162140.133674-1-liran.alon@oracle.com> References: <20190619162140.133674-1-liran.alon@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9293 signatures=668687 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=1 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1906190131 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9293 signatures=668687 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1906190131 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Kernel commit c4f55198c7c2 ("kvm: x86: Introduce KVM_CAP_EXCEPTION_PAYLOAD") introduced a new KVM capability which allows userspace to correctly distinguish between pending and injected exceptions. This distinguish is important in case of nested virtualization scenarios because a L2 pending exception can still be intercepted by the L1 hypervisor while a L2 injected exception cannot. Furthermore, when an exception is attempted to be injected by QEMU, QEMU should specify the exception payload (CR2 in case of #PF or DR6 in case of #DB) instead of having the payload already delivered in the respective vCPU register. Because in case exception is injected to L2 guest and is intercepted by L1 hypervisor, then payload needs to be reported to L1 intercept (VMExit handler) while still preserving respective vCPU register unchanged. This commit adds support for QEMU to properly utilise this new KVM capability (KVM_CAP_EXCEPTION_PAYLOAD). Reviewed-by: Nikita Leshenko Signed-off-by: Liran Alon --- target/i386/cpu.c | 6 ++- target/i386/cpu.h | 6 ++- target/i386/hvf/hvf.c | 10 ++-- target/i386/hvf/x86hvf.c | 4 +- target/i386/kvm.c | 101 ++++++++++++++++++++++++++++++++------- target/i386/machine.c | 84 +++++++++++++++++++++++++++++++- 6 files changed, 187 insertions(+), 24 deletions(-) diff --git a/target/i386/cpu.c b/target/i386/cpu.c index 197201087e65..a026e49f5c0d 100644 --- a/target/i386/cpu.c +++ b/target/i386/cpu.c @@ -4774,7 +4774,11 @@ static void x86_cpu_reset(CPUState *s) memset(env->mtrr_fixed, 0, sizeof(env->mtrr_fixed)); env->interrupt_injected = -1; - env->exception_injected = -1; + env->exception_nr = -1; + env->exception_pending = 0; + env->exception_injected = 0; + env->exception_has_payload = false; + env->exception_payload = 0; env->nmi_injected = false; #if !defined(CONFIG_USER_ONLY) /* We hard-wire the BSP to the first CPU. */ diff --git a/target/i386/cpu.h b/target/i386/cpu.h index c1a42aa8851a..24543dc188f8 100644 --- a/target/i386/cpu.h +++ b/target/i386/cpu.h @@ -1338,10 +1338,14 @@ typedef struct CPUX86State { /* For KVM */ uint32_t mp_state; - int32_t exception_injected; + int32_t exception_nr; int32_t interrupt_injected; uint8_t soft_interrupt; + uint8_t exception_pending; + uint8_t exception_injected; uint8_t has_error_code; + uint8_t exception_has_payload; + uint64_t exception_payload; uint32_t ins_len; uint32_t sipi_vector; bool tsc_valid; diff --git a/target/i386/hvf/hvf.c b/target/i386/hvf/hvf.c index 2751c8125ca2..dc4bb63536c8 100644 --- a/target/i386/hvf/hvf.c +++ b/target/i386/hvf/hvf.c @@ -605,7 +605,9 @@ static void hvf_store_events(CPUState *cpu, uint32_t ins_len, uint64_t idtvec_in X86CPU *x86_cpu = X86_CPU(cpu); CPUX86State *env = &x86_cpu->env; - env->exception_injected = -1; + env->exception_nr = -1; + env->exception_pending = 0; + env->exception_injected = 0; env->interrupt_injected = -1; env->nmi_injected = false; if (idtvec_info & VMCS_IDT_VEC_VALID) { @@ -619,7 +621,8 @@ static void hvf_store_events(CPUState *cpu, uint32_t ins_len, uint64_t idtvec_in break; case VMCS_IDT_VEC_HWEXCEPTION: case VMCS_IDT_VEC_SWEXCEPTION: - env->exception_injected = idtvec_info & VMCS_IDT_VEC_VECNUM; + env->exception_nr = idtvec_info & VMCS_IDT_VEC_VECNUM; + env->exception_injected = 1; break; case VMCS_IDT_VEC_PRIV_SWEXCEPTION: default: @@ -912,7 +915,8 @@ int hvf_vcpu_exec(CPUState *cpu) macvm_set_rip(cpu, rip + ins_len); break; case VMX_REASON_VMCALL: - env->exception_injected = EXCP0D_GPF; + env->exception_nr = EXCP0D_GPF; + env->exception_injected = 1; env->has_error_code = true; env->error_code = 0; break; diff --git a/target/i386/hvf/x86hvf.c b/target/i386/hvf/x86hvf.c index df8e946fbcde..e0ea02d631e6 100644 --- a/target/i386/hvf/x86hvf.c +++ b/target/i386/hvf/x86hvf.c @@ -362,8 +362,8 @@ bool hvf_inject_interrupts(CPUState *cpu_state) if (env->interrupt_injected != -1) { vector = env->interrupt_injected; intr_type = VMCS_INTR_T_SWINTR; - } else if (env->exception_injected != -1) { - vector = env->exception_injected; + } else if (env->exception_nr != -1) { + vector = env->exception_nr; if (vector == EXCP03_INT3 || vector == EXCP04_INTO) { intr_type = VMCS_INTR_T_SWEXCEPTION; } else { diff --git a/target/i386/kvm.c b/target/i386/kvm.c index 8ae7598aac3d..99480a52ad33 100644 --- a/target/i386/kvm.c +++ b/target/i386/kvm.c @@ -104,6 +104,7 @@ static uint32_t num_architectural_pmu_fixed_counters; static int has_xsave; static int has_xcrs; static int has_pit_state2; +static int has_exception_payload; static bool has_msr_mcg_ext_ctl; @@ -584,15 +585,56 @@ void kvm_arch_on_sigbus_vcpu(CPUState *c, int code, void *addr) /* Hope we are lucky for AO MCE */ } +static void kvm_reset_exception(CPUX86State *env) +{ + env->exception_nr = -1; + env->exception_pending = 0; + env->exception_injected = 0; + env->exception_has_payload = false; + env->exception_payload = 0; +} + +static void kvm_queue_exception(CPUX86State *env, + int32_t exception_nr, + uint8_t exception_has_payload, + uint64_t exception_payload) +{ + assert(env->exception_nr == -1); + assert(!env->exception_pending); + assert(!env->exception_injected); + assert(!env->exception_has_payload); + + env->exception_nr = exception_nr; + + if (has_exception_payload) { + env->exception_pending = 1; + + env->exception_has_payload = exception_has_payload; + env->exception_payload = exception_payload; + } else { + env->exception_injected = 1; + + if (exception_nr == EXCP01_DB) { + assert(exception_has_payload); + env->dr[6] = exception_payload; + } else if (exception_nr == EXCP0E_PAGE) { + assert(exception_has_payload); + env->cr[2] = exception_payload; + } else { + assert(!exception_has_payload); + } + } +} + static int kvm_inject_mce_oldstyle(X86CPU *cpu) { CPUX86State *env = &cpu->env; - if (!kvm_has_vcpu_events() && env->exception_injected == EXCP12_MCHK) { + if (!kvm_has_vcpu_events() && env->exception_nr == EXCP12_MCHK) { unsigned int bank, bank_num = env->mcg_cap & 0xff; struct kvm_x86_mce mce; - env->exception_injected = -1; + kvm_reset_exception(env); /* * There must be at least one bank in use if an MCE is pending. @@ -1613,6 +1655,16 @@ int kvm_arch_init(MachineState *ms, KVMState *s) hv_vpindex_settable = kvm_check_extension(s, KVM_CAP_HYPERV_VP_INDEX); + has_exception_payload = kvm_check_extension(s, KVM_CAP_EXCEPTION_PAYLOAD); + if (has_exception_payload) { + ret = kvm_vm_enable_cap(s, KVM_CAP_EXCEPTION_PAYLOAD, 0, true); + if (ret < 0) { + error_report("kvm: Failed to enable exception payload cap: %s", + strerror(-ret)); + return ret; + } + } + ret = kvm_get_supported_msrs(s); if (ret < 0) { return ret; @@ -2917,8 +2969,16 @@ static int kvm_put_vcpu_events(X86CPU *cpu, int level) return 0; } - events.exception.injected = (env->exception_injected >= 0); - events.exception.nr = env->exception_injected; + events.flags = 0; + + if (has_exception_payload) { + events.flags |= KVM_VCPUEVENT_VALID_PAYLOAD; + events.exception.pending = env->exception_pending; + events.exception_has_payload = env->exception_has_payload; + events.exception_payload = env->exception_payload; + } + events.exception.nr = env->exception_nr; + events.exception.injected = env->exception_injected; events.exception.has_error_code = env->has_error_code; events.exception.error_code = env->error_code; @@ -2931,7 +2991,6 @@ static int kvm_put_vcpu_events(X86CPU *cpu, int level) events.nmi.masked = !!(env->hflags2 & HF2_NMI_MASK); events.sipi_vector = env->sipi_vector; - events.flags = 0; if (has_msr_smbase) { events.smi.smm = !!(env->hflags & HF_SMM_MASK); @@ -2981,8 +3040,19 @@ static int kvm_get_vcpu_events(X86CPU *cpu) if (ret < 0) { return ret; } - env->exception_injected = - events.exception.injected ? events.exception.nr : -1; + + if (events.flags & KVM_VCPUEVENT_VALID_PAYLOAD) { + env->exception_pending = events.exception.pending; + env->exception_has_payload = events.exception_has_payload; + env->exception_payload = events.exception_payload; + } else { + env->exception_pending = 0; + env->exception_has_payload = false; + } + env->exception_injected = events.exception.injected; + env->exception_nr = + (env->exception_pending || env->exception_injected) ? + events.exception.nr : -1; env->has_error_code = events.exception.has_error_code; env->error_code = events.exception.error_code; @@ -3034,12 +3104,12 @@ static int kvm_guest_debug_workarounds(X86CPU *cpu) unsigned long reinject_trap = 0; if (!kvm_has_vcpu_events()) { - if (env->exception_injected == EXCP01_DB) { + if (env->exception_nr == EXCP01_DB) { reinject_trap = KVM_GUESTDBG_INJECT_DB; } else if (env->exception_injected == EXCP03_INT3) { reinject_trap = KVM_GUESTDBG_INJECT_BP; } - env->exception_injected = -1; + kvm_reset_exception(env); } /* @@ -3415,13 +3485,13 @@ int kvm_arch_process_async_events(CPUState *cs) kvm_cpu_synchronize_state(cs); - if (env->exception_injected == EXCP08_DBLE) { + if (env->exception_nr == EXCP08_DBLE) { /* this means triple fault */ qemu_system_reset_request(SHUTDOWN_CAUSE_GUEST_RESET); cs->exit_request = 1; return 0; } - env->exception_injected = EXCP12_MCHK; + kvm_queue_exception(env, EXCP12_MCHK, 0, 0); env->has_error_code = 0; cs->halted = 0; @@ -3636,14 +3706,13 @@ static int kvm_handle_debug(X86CPU *cpu, } if (ret == 0) { cpu_synchronize_state(cs); - assert(env->exception_injected == -1); + assert(env->exception_nr == -1); /* pass to guest */ - env->exception_injected = arch_info->exception; + kvm_queue_exception(env, arch_info->exception, + arch_info->exception == EXCP01_DB, + arch_info->dr6); env->has_error_code = 0; - if (arch_info->exception == EXCP01_DB) { - env->dr[6] = arch_info->dr6; - } } return ret; diff --git a/target/i386/machine.c b/target/i386/machine.c index e27115fdb756..4f292f0660ca 100644 --- a/target/i386/machine.c +++ b/target/i386/machine.c @@ -240,6 +240,41 @@ static int cpu_pre_save(void *opaque) } #endif + /* + * When vCPU is running L2 and exception is still pending, + * it can potentially be intercepted by L1 hypervisor. + * In contrast to an injected exception which cannot be + * intercepted anymore. + * + * Furthermore, when a L2 exception is intercepted by L1 + * hypervisor, it's exception payload (CR2/DR6 on #PF/#DB) + * should not be set yet in the respective vCPU register. + * Thus, in case an exception is pending, it is + * important to save the exception payload seperately. + * + * Therefore, if an exception is not in a pending state + * or vCPU is not in guest-mode, it is not important to + * distinguish between a pending and injected exception + * and we don't need to store seperately the exception payload. + * + * In order to preserve better backwards-compatabile migration, + * convert a pending exception to an injected exception in + * case it is not important to distingiush between them + * as described above. + */ + if (env->exception_pending && !(env->hflags & HF_GUEST_MASK)) { + env->exception_pending = 0; + env->exception_injected = 1; + + if (env->exception_has_payload) { + if (env->exception_nr == EXCP01_DB) { + env->dr[6] = env->exception_payload; + } else if (env->exception_nr == EXCP0E_PAGE) { + env->cr[2] = env->exception_payload; + } + } + } + return 0; } @@ -297,6 +332,23 @@ static int cpu_post_load(void *opaque, int version_id) } #endif + /* + * There are cases that we can get valid exception_nr with both + * exception_pending and exception_injected being cleared. + * This can happen in one of the following scenarios: + * 1) Source is older QEMU without KVM_CAP_EXCEPTION_PAYLOAD support. + * 2) Source is running on kernel without KVM_CAP_EXCEPTION_PAYLOAD support. + * 3) "cpu/exception_info" subsection not sent because there is no exception + * pending or guest wasn't running L2 (See comment in cpu_pre_save()). + * + * In those cases, we can just deduce that a valid exception_nr means + * we can treat the exception as already injected. + */ + if ((env->exception_nr != -1) && + !env->exception_pending && !env->exception_injected) { + env->exception_injected = 1; + } + env->fpstt = (env->fpus_vmstate >> 11) & 7; env->fpus = env->fpus_vmstate & ~0x3800; env->fptag_vmstate ^= 0xff; @@ -342,6 +394,35 @@ static bool steal_time_msr_needed(void *opaque) return cpu->env.steal_time_msr != 0; } +static bool exception_info_needed(void *opaque) +{ + X86CPU *cpu = opaque; + CPUX86State *env = &cpu->env; + + /* + * It is important to save exception-info only in case + * we need to distingiush between a pending and injected + * exception. Which is only required in case there is a + * pending exception and vCPU is running L2. + * For more info, refer to comment in cpu_pre_save(). + */ + return (env->exception_pending && (env->hflags & HF_GUEST_MASK)); +} + +static const VMStateDescription vmstate_exception_info = { + .name = "cpu/exception_info", + .version_id = 1, + .minimum_version_id = 1, + .needed = exception_info_needed, + .fields = (VMStateField[]) { + VMSTATE_UINT8(env.exception_pending, X86CPU), + VMSTATE_UINT8(env.exception_injected, X86CPU), + VMSTATE_UINT8(env.exception_has_payload, X86CPU), + VMSTATE_UINT64(env.exception_payload, X86CPU), + VMSTATE_END_OF_LIST() + } +}; + static const VMStateDescription vmstate_steal_time_msr = { .name = "cpu/steal_time_msr", .version_id = 1, @@ -1230,7 +1311,7 @@ VMStateDescription vmstate_x86_cpu = { VMSTATE_INT32(env.interrupt_injected, X86CPU), VMSTATE_UINT32(env.mp_state, X86CPU), VMSTATE_UINT64(env.tsc, X86CPU), - VMSTATE_INT32(env.exception_injected, X86CPU), + VMSTATE_INT32(env.exception_nr, X86CPU), VMSTATE_UINT8(env.soft_interrupt, X86CPU), VMSTATE_UINT8(env.nmi_injected, X86CPU), VMSTATE_UINT8(env.nmi_pending, X86CPU), @@ -1254,6 +1335,7 @@ VMStateDescription vmstate_x86_cpu = { /* The above list is not sorted /wrt version numbers, watch out! */ }, .subsections = (const VMStateDescription*[]) { + &vmstate_exception_info, &vmstate_async_pf_msr, &vmstate_pv_eoi_msr, &vmstate_steal_time_msr, From patchwork Wed Jun 19 16:21:40 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Liran Alon X-Patchwork-Id: 11004625 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 174DE1580 for ; Wed, 19 Jun 2019 16:23:25 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 02B6421FAC for ; Wed, 19 Jun 2019 16:23:25 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id EB35F28478; Wed, 19 Jun 2019 16:23:24 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9ACD4284C5 for ; Wed, 19 Jun 2019 16:23:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729841AbfFSQXX (ORCPT ); Wed, 19 Jun 2019 12:23:23 -0400 Received: from aserp2120.oracle.com ([141.146.126.78]:50030 "EHLO aserp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727134AbfFSQXX (ORCPT ); Wed, 19 Jun 2019 12:23:23 -0400 Received: from pps.filterd (aserp2120.oracle.com [127.0.0.1]) by aserp2120.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x5JGE3Hm031298; Wed, 19 Jun 2019 16:22:32 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2018-07-02; bh=1GmbOAxbteTLbo8yPsg654GIHeNPBlXklMYNzYSRJGI=; b=2y9zWccHfH3kWAQzAcrFYezZMN1DDpfHfK31m8W9DQ17db+iIkhhFjukMgU80VsnsiYh CUuT0gpJwZxwu6mZ4AjkqGke67VbGHtZT+XvCUggxr56lRmdHrrN4fc+gT5NXLetKhAm 5cbIyCW2iy4kveC1fh3JJgK2F4WGobuCTxO4MmZu8A84A3fk7LcNaNRnq23nMkALK21z gi7XFj1zB86Rk4JgoU2rFBVeY2OTQOG8g1LhlCSBogSqxVxdKylJWufQ4wKNC4Tvv8Nx Bf6kxUHyHIdzCSKxaEL3AoW0vDOliC/jLM7iKdOjzcaJXjdrbnF3AqJ6CKjmg5mhb7PF 9w== Received: from userp3030.oracle.com (userp3030.oracle.com [156.151.31.80]) by aserp2120.oracle.com with ESMTP id 2t7809ceng-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 19 Jun 2019 16:22:32 +0000 Received: from pps.filterd (userp3030.oracle.com [127.0.0.1]) by userp3030.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x5JGLbuv133107; Wed, 19 Jun 2019 16:22:31 GMT Received: from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235]) by userp3030.oracle.com with ESMTP id 2t77yn6q1n-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 19 Jun 2019 16:22:31 +0000 Received: from abhmp0008.oracle.com (abhmp0008.oracle.com [141.146.116.14]) by aserv0121.oracle.com (8.14.4/8.13.8) with ESMTP id x5JGMUhF000913; Wed, 19 Jun 2019 16:22:31 GMT Received: from spark.ravello.local (/213.57.127.2) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 19 Jun 2019 09:22:30 -0700 From: Liran Alon To: qemu-devel@nongnu.org Cc: pbonzini@redhat.com, mtosatti@redhat.com, rth@twiddle.net, ehabkost@redhat.com, kvm@vger.kernel.org, jmattson@google.com, maran.wilson@oracle.com, dgilbert@redhat.com, Liran Alon Subject: [QEMU PATCH v4 10/10] target/i386: kvm: Add nested migration blocker only when kernel lacks required capabilities Date: Wed, 19 Jun 2019 19:21:40 +0300 Message-Id: <20190619162140.133674-11-liran.alon@oracle.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190619162140.133674-1-liran.alon@oracle.com> References: <20190619162140.133674-1-liran.alon@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9293 signatures=668687 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=1 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=895 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1906190131 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9293 signatures=668687 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=1 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=937 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1906190131 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Previous commits have added support for migration of nested virtualization workloads. This was done by utilising two new KVM capabilities: KVM_CAP_NESTED_STATE and KVM_CAP_EXCEPTION_PAYLOAD. Both which are required in order to correctly migrate such workloads. Therefore, change code to add a migration blocker for vCPUs exposed with Intel VMX or AMD SVM in case one of these kernel capabilities is missing. Signed-off-by: Liran Alon Reviewed-by: Maran Wilson --- target/i386/kvm.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/target/i386/kvm.c b/target/i386/kvm.c index 99480a52ad33..a3d0fbed3b35 100644 --- a/target/i386/kvm.c +++ b/target/i386/kvm.c @@ -1313,9 +1313,14 @@ int kvm_arch_init_vcpu(CPUState *cs) !!(c->ecx & CPUID_EXT_SMX); } - if (cpu_has_nested_virt(env) && !nested_virt_mig_blocker) { + if (cpu_has_nested_virt(env) && !nested_virt_mig_blocker && + ((kvm_max_nested_state_length() <= 0) || !has_exception_payload)) { error_setg(&nested_virt_mig_blocker, - "Nested virtualization does not support live migration yet"); + "Kernel do not provide required capabilities for " + "nested virtualization migration. " + "(CAP_NESTED_STATE=%d, CAP_EXCEPTION_PAYLOAD=%d)", + kvm_max_nested_state_length() > 0, + has_exception_payload); r = migrate_add_blocker(nested_virt_mig_blocker, &local_err); if (local_err) { error_report_err(local_err);