From patchwork Sun May 12 08:36:12 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Jones X-Patchwork-Id: 10939927 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3EB2E13AD for ; Sun, 12 May 2019 08:39:57 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2BB4C1FFDB for ; Sun, 12 May 2019 08:39:57 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1756E232A7; Sun, 12 May 2019 08:39:57 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id C3DF71FFDB for ; Sun, 12 May 2019 08:39:56 +0000 (UTC) Received: from localhost ([127.0.0.1]:40487 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hPk1T-0005x0-Ve for patchwork-qemu-devel@patchwork.kernel.org; Sun, 12 May 2019 04:39:56 -0400 Received: from eggs.gnu.org ([209.51.188.92]:38139) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hPjyM-0003cD-4i for qemu-devel@nongnu.org; Sun, 12 May 2019 04:36:42 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hPjyL-0003Gl-9i for qemu-devel@nongnu.org; Sun, 12 May 2019 04:36:42 -0400 Received: from mx1.redhat.com ([209.132.183.28]:53146) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hPjyI-0003Fp-WC; Sun, 12 May 2019 04:36:39 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 40A748667B; Sun, 12 May 2019 08:36:38 +0000 (UTC) Received: from kamzik.brq.redhat.com (ovpn-116-87.ams2.redhat.com [10.36.116.87]) by smtp.corp.redhat.com (Postfix) with ESMTP id 03F115D706; Sun, 12 May 2019 08:36:33 +0000 (UTC) From: Andrew Jones To: qemu-devel@nongnu.org, qemu-arm@nongnu.org Date: Sun, 12 May 2019 10:36:12 +0200 Message-Id: <20190512083624.8916-2-drjones@redhat.com> In-Reply-To: <20190512083624.8916-1-drjones@redhat.com> References: <20190512083624.8916-1-drjones@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.26]); Sun, 12 May 2019 08:36:38 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH 01/13] target/arm/kvm64: fix error returns X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, richard.henderson@linaro.org, armbru@redhat.com, abologna@redhat.com, alex.bennee@linaro.org, Dave.Martin@arm.com Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP A couple return -EINVAL's forget their '-'s. Signed-off-by: Andrew Jones Reviewed-by: Eric Auger --- target/arm/kvm64.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/target/arm/kvm64.c b/target/arm/kvm64.c index e3ba1492482f..ba232b27a6d3 100644 --- a/target/arm/kvm64.c +++ b/target/arm/kvm64.c @@ -841,7 +841,7 @@ int kvm_arch_put_registers(CPUState *cs, int level) write_cpustate_to_list(cpu, true); if (!write_list_to_kvmstate(cpu, level)) { - return EINVAL; + return -EINVAL; } kvm_arm_sync_mpstate_to_kvm(cpu); @@ -982,7 +982,7 @@ int kvm_arch_get_registers(CPUState *cs) } if (!write_kvmstate_to_list(cpu)) { - return EINVAL; + return -EINVAL; } /* Note that it's OK to have registers which aren't in CPUState, * so we can ignore a failure return here. From patchwork Sun May 12 08:36:13 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Jones X-Patchwork-Id: 10939939 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5EAF31515 for ; Sun, 12 May 2019 08:43:35 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4BF5921F61 for ; Sun, 12 May 2019 08:43:35 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3D2DE20564; Sun, 12 May 2019 08:43:35 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id EA3E120564 for ; Sun, 12 May 2019 08:43:34 +0000 (UTC) Received: from localhost ([127.0.0.1]:40559 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hPk50-0000mW-6l for patchwork-qemu-devel@patchwork.kernel.org; Sun, 12 May 2019 04:43:34 -0400 Received: from eggs.gnu.org ([209.51.188.92]:38368) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hPjzW-0004IA-9D for qemu-devel@nongnu.org; Sun, 12 May 2019 04:37:55 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hPjzU-0003kX-Tv for qemu-devel@nongnu.org; Sun, 12 May 2019 04:37:54 -0400 Received: from mx1.redhat.com ([209.132.183.28]:39280) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hPjyN-0003HM-Hi; Sun, 12 May 2019 04:37:49 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id D0F633081244; Sun, 12 May 2019 08:36:42 +0000 (UTC) Received: from kamzik.brq.redhat.com (ovpn-116-87.ams2.redhat.com [10.36.116.87]) by smtp.corp.redhat.com (Postfix) with ESMTP id ADB865D706; Sun, 12 May 2019 08:36:38 +0000 (UTC) From: Andrew Jones To: qemu-devel@nongnu.org, qemu-arm@nongnu.org Date: Sun, 12 May 2019 10:36:13 +0200 Message-Id: <20190512083624.8916-3-drjones@redhat.com> In-Reply-To: <20190512083624.8916-1-drjones@redhat.com> References: <20190512083624.8916-1-drjones@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.49]); Sun, 12 May 2019 08:36:42 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH 02/13] update-linux-headers: Add sve_context.h to asm-arm64 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, richard.henderson@linaro.org, armbru@redhat.com, abologna@redhat.com, alex.bennee@linaro.org, Dave.Martin@arm.com Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP Signed-off-by: Andrew Jones Reviewed-by: Eric Auger --- scripts/update-linux-headers.sh | 3 +++ 1 file changed, 3 insertions(+) diff --git a/scripts/update-linux-headers.sh b/scripts/update-linux-headers.sh index c3819d2b983d..e1fce54f8aa3 100755 --- a/scripts/update-linux-headers.sh +++ b/scripts/update-linux-headers.sh @@ -99,6 +99,9 @@ for arch in $ARCHLIST; do cp "$tmpdir/include/asm/$header" "$output/linux-headers/asm-$arch" done + if [ $arch = arm64 ]; then + cp "$tmpdir/include/asm/sve_context.h" "$output/linux-headers/asm-arm64/" + fi if [ $arch = mips ]; then cp "$tmpdir/include/asm/sgidefs.h" "$output/linux-headers/asm-mips/" cp "$tmpdir/include/asm/unistd_o32.h" "$output/linux-headers/asm-mips/" From patchwork Sun May 12 08:36:14 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Jones X-Patchwork-Id: 10939949 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A6A64933 for ; Sun, 12 May 2019 08:49:57 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 83C2620564 for ; Sun, 12 May 2019 08:49:57 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 5DA0122BF1; Sun, 12 May 2019 08:49:57 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id BEA6C20564 for ; Sun, 12 May 2019 08:49:56 +0000 (UTC) Received: from localhost ([127.0.0.1]:40640 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hPkBA-0005NG-1Q for patchwork-qemu-devel@patchwork.kernel.org; Sun, 12 May 2019 04:49:56 -0400 Received: from eggs.gnu.org ([209.51.188.92]:38528) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hPjza-0004NP-2e for qemu-devel@nongnu.org; Sun, 12 May 2019 04:37:59 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hPjzY-0003pQ-Cr for qemu-devel@nongnu.org; Sun, 12 May 2019 04:37:57 -0400 Received: from mx1.redhat.com ([209.132.183.28]:58296) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hPjyR-0003IX-RN; Sun, 12 May 2019 04:37:49 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 23A4C3086201; Sun, 12 May 2019 08:36:47 +0000 (UTC) Received: from kamzik.brq.redhat.com (ovpn-116-87.ams2.redhat.com [10.36.116.87]) by smtp.corp.redhat.com (Postfix) with ESMTP id A0D535D706; Sun, 12 May 2019 08:36:43 +0000 (UTC) From: Andrew Jones To: qemu-devel@nongnu.org, qemu-arm@nongnu.org Date: Sun, 12 May 2019 10:36:14 +0200 Message-Id: <20190512083624.8916-4-drjones@redhat.com> In-Reply-To: <20190512083624.8916-1-drjones@redhat.com> References: <20190512083624.8916-1-drjones@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.42]); Sun, 12 May 2019 08:36:47 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH 03/13] HACK: linux header update X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, richard.henderson@linaro.org, armbru@redhat.com, abologna@redhat.com, alex.bennee@linaro.org, Dave.Martin@arm.com Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP --- linux-headers/asm-arm64/kvm.h | 41 +++++++++++++++++++++ linux-headers/asm-arm64/sve_context.h | 53 +++++++++++++++++++++++++++ linux-headers/linux/kvm.h | 5 +++ 3 files changed, 99 insertions(+) create mode 100644 linux-headers/asm-arm64/sve_context.h diff --git a/linux-headers/asm-arm64/kvm.h b/linux-headers/asm-arm64/kvm.h index e6a98c14c848..3e73a4c67b67 100644 --- a/linux-headers/asm-arm64/kvm.h +++ b/linux-headers/asm-arm64/kvm.h @@ -35,6 +35,7 @@ #include #include #include +#include #define __KVM_HAVE_GUEST_DEBUG #define __KVM_HAVE_IRQ_LINE @@ -102,6 +103,7 @@ struct kvm_regs { #define KVM_ARM_VCPU_EL1_32BIT 1 /* CPU running a 32bit VM */ #define KVM_ARM_VCPU_PSCI_0_2 2 /* CPU uses PSCI v0.2 */ #define KVM_ARM_VCPU_PMU_V3 3 /* Support guest PMUv3 */ +#define KVM_ARM_VCPU_SVE 4 /* enable SVE for this CPU */ struct kvm_vcpu_init { __u32 target; @@ -226,6 +228,45 @@ struct kvm_vcpu_events { KVM_REG_ARM_FW | ((r) & 0xffff)) #define KVM_REG_ARM_PSCI_VERSION KVM_REG_ARM_FW_REG(0) +/* SVE registers */ +#define KVM_REG_ARM64_SVE (0x15 << KVM_REG_ARM_COPROC_SHIFT) + +/* Z- and P-regs occupy blocks at the following offsets within this range: */ +#define KVM_REG_ARM64_SVE_ZREG_BASE 0 +#define KVM_REG_ARM64_SVE_PREG_BASE 0x400 +#define KVM_REG_ARM64_SVE_FFR_BASE 0x600 + +#define KVM_ARM64_SVE_NUM_ZREGS __SVE_NUM_ZREGS +#define KVM_ARM64_SVE_NUM_PREGS __SVE_NUM_PREGS + +#define KVM_ARM64_SVE_MAX_SLICES 32 + +#define KVM_REG_ARM64_SVE_ZREG(n, i) \ + (KVM_REG_ARM64 | KVM_REG_ARM64_SVE | KVM_REG_ARM64_SVE_ZREG_BASE | \ + KVM_REG_SIZE_U2048 | \ + (((n) & (KVM_ARM64_SVE_NUM_ZREGS - 1)) << 5) | \ + ((i) & (KVM_ARM64_SVE_MAX_SLICES - 1))) + +#define KVM_REG_ARM64_SVE_PREG(n, i) \ + (KVM_REG_ARM64 | KVM_REG_ARM64_SVE | KVM_REG_ARM64_SVE_PREG_BASE | \ + KVM_REG_SIZE_U256 | \ + (((n) & (KVM_ARM64_SVE_NUM_PREGS - 1)) << 5) | \ + ((i) & (KVM_ARM64_SVE_MAX_SLICES - 1))) + +#define KVM_REG_ARM64_SVE_FFR(i) \ + (KVM_REG_ARM64 | KVM_REG_ARM64_SVE | KVM_REG_ARM64_SVE_FFR_BASE | \ + KVM_REG_SIZE_U256 | \ + ((i) & (KVM_ARM64_SVE_MAX_SLICES - 1))) + +#define KVM_ARM64_SVE_VQ_MIN __SVE_VQ_MIN +#define KVM_ARM64_SVE_VQ_MAX __SVE_VQ_MAX + +/* Vector lengths pseudo-register: */ +#define KVM_REG_ARM64_SVE_VLS (KVM_REG_ARM64 | KVM_REG_ARM64_SVE | \ + KVM_REG_SIZE_U512 | 0xffff) +#define KVM_ARM64_SVE_VLS_WORDS \ + ((KVM_ARM64_SVE_VQ_MAX - KVM_ARM64_SVE_VQ_MIN) / 64 + 1) + /* Device Control API: ARM VGIC */ #define KVM_DEV_ARM_VGIC_GRP_ADDR 0 #define KVM_DEV_ARM_VGIC_GRP_DIST_REGS 1 diff --git a/linux-headers/asm-arm64/sve_context.h b/linux-headers/asm-arm64/sve_context.h new file mode 100644 index 000000000000..1d0e3e1d0950 --- /dev/null +++ b/linux-headers/asm-arm64/sve_context.h @@ -0,0 +1,53 @@ +/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */ +/* Copyright (C) 2017-2018 ARM Limited */ + +/* + * For use by other UAPI headers only. + * Do not make direct use of header or its definitions. + */ + +#ifndef __ASM_SVE_CONTEXT_H +#define __ASM_SVE_CONTEXT_H + +#include + +#define __SVE_VQ_BYTES 16 /* number of bytes per quadword */ + +#define __SVE_VQ_MIN 1 +#define __SVE_VQ_MAX 512 + +#define __SVE_VL_MIN (__SVE_VQ_MIN * __SVE_VQ_BYTES) +#define __SVE_VL_MAX (__SVE_VQ_MAX * __SVE_VQ_BYTES) + +#define __SVE_NUM_ZREGS 32 +#define __SVE_NUM_PREGS 16 + +#define __sve_vl_valid(vl) \ + ((vl) % __SVE_VQ_BYTES == 0 && \ + (vl) >= __SVE_VL_MIN && \ + (vl) <= __SVE_VL_MAX) + +#define __sve_vq_from_vl(vl) ((vl) / __SVE_VQ_BYTES) +#define __sve_vl_from_vq(vq) ((vq) * __SVE_VQ_BYTES) + +#define __SVE_ZREG_SIZE(vq) ((__u32)(vq) * __SVE_VQ_BYTES) +#define __SVE_PREG_SIZE(vq) ((__u32)(vq) * (__SVE_VQ_BYTES / 8)) +#define __SVE_FFR_SIZE(vq) __SVE_PREG_SIZE(vq) + +#define __SVE_ZREGS_OFFSET 0 +#define __SVE_ZREG_OFFSET(vq, n) \ + (__SVE_ZREGS_OFFSET + __SVE_ZREG_SIZE(vq) * (n)) +#define __SVE_ZREGS_SIZE(vq) \ + (__SVE_ZREG_OFFSET(vq, __SVE_NUM_ZREGS) - __SVE_ZREGS_OFFSET) + +#define __SVE_PREGS_OFFSET(vq) \ + (__SVE_ZREGS_OFFSET + __SVE_ZREGS_SIZE(vq)) +#define __SVE_PREG_OFFSET(vq, n) \ + (__SVE_PREGS_OFFSET(vq) + __SVE_PREG_SIZE(vq) * (n)) +#define __SVE_PREGS_SIZE(vq) \ + (__SVE_PREG_OFFSET(vq, __SVE_NUM_PREGS) - __SVE_PREGS_OFFSET(vq)) + +#define __SVE_FFR_OFFSET(vq) \ + (__SVE_PREGS_OFFSET(vq) + __SVE_PREGS_SIZE(vq)) + +#endif /* ! _UAPI__ASM_SVE_CONTEXT_H */ diff --git a/linux-headers/linux/kvm.h b/linux-headers/linux/kvm.h index b53ee5974802..1a639ae0ceed 100644 --- a/linux-headers/linux/kvm.h +++ b/linux-headers/linux/kvm.h @@ -988,6 +988,7 @@ struct kvm_ppc_resize_hpt { #define KVM_CAP_ARM_VM_IPA_SIZE 165 #define KVM_CAP_MANUAL_DIRTY_LOG_PROTECT 166 #define KVM_CAP_HYPERV_CPUID 167 +#define KVM_CAP_ARM_SVE 168 #ifdef KVM_CAP_IRQ_ROUTING @@ -1145,6 +1146,7 @@ struct kvm_dirty_tlb { #define KVM_REG_SIZE_U256 0x0050000000000000ULL #define KVM_REG_SIZE_U512 0x0060000000000000ULL #define KVM_REG_SIZE_U1024 0x0070000000000000ULL +#define KVM_REG_SIZE_U2048 0x0080000000000000ULL struct kvm_reg_list { __u64 n; /* number of regs */ @@ -1440,6 +1442,9 @@ struct kvm_enc_region { /* Available with KVM_CAP_HYPERV_CPUID */ #define KVM_GET_SUPPORTED_HV_CPUID _IOWR(KVMIO, 0xc1, struct kvm_cpuid2) +/* Available with KVM_CAP_ARM_SVE */ +#define KVM_ARM_VCPU_FINALIZE _IOW(KVMIO, 0xc2, int) + /* Secure Encrypted Virtualization command */ enum sev_cmd_id { /* Guest initialization commands */ From patchwork Sun May 12 08:36:15 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Jones X-Patchwork-Id: 10939947 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 687D31515 for ; Sun, 12 May 2019 08:48:13 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 56E8120564 for ; Sun, 12 May 2019 08:48:13 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 4AEFE232A7; Sun, 12 May 2019 08:48:13 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id AAF9D20564 for ; Sun, 12 May 2019 08:48:12 +0000 (UTC) Received: from localhost ([127.0.0.1]:40626 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hPk9T-0004M6-So for patchwork-qemu-devel@patchwork.kernel.org; Sun, 12 May 2019 04:48:11 -0400 Received: from eggs.gnu.org ([209.51.188.92]:38509) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hPjzZ-0004Mc-GN for qemu-devel@nongnu.org; Sun, 12 May 2019 04:37:58 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hPjzX-0003nY-83 for qemu-devel@nongnu.org; Sun, 12 May 2019 04:37:57 -0400 Received: from mx1.redhat.com ([209.132.183.28]:56678) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hPjyV-0003KL-CM; Sun, 12 May 2019 04:37:49 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 9E6E03082E25; Sun, 12 May 2019 08:36:50 +0000 (UTC) Received: from kamzik.brq.redhat.com (ovpn-116-87.ams2.redhat.com [10.36.116.87]) by smtp.corp.redhat.com (Postfix) with ESMTP id E4DDC5D706; Sun, 12 May 2019 08:36:47 +0000 (UTC) From: Andrew Jones To: qemu-devel@nongnu.org, qemu-arm@nongnu.org Date: Sun, 12 May 2019 10:36:15 +0200 Message-Id: <20190512083624.8916-5-drjones@redhat.com> In-Reply-To: <20190512083624.8916-1-drjones@redhat.com> References: <20190512083624.8916-1-drjones@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.46]); Sun, 12 May 2019 08:36:50 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH 04/13] target/arm/kvm: Move the get/put of fpsimd registers out X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, richard.henderson@linaro.org, armbru@redhat.com, abologna@redhat.com, alex.bennee@linaro.org, Dave.Martin@arm.com Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP Move the getting/putting of the fpsimd registers out of kvm_arch_get/put_registers() into their own helper functions to prepare for alternatively getting/putting SVE registers. No functional change. Signed-off-by: Andrew Jones Reviewed-by: Eric Auger --- target/arm/kvm64.c | 148 +++++++++++++++++++++++++++------------------ 1 file changed, 88 insertions(+), 60 deletions(-) diff --git a/target/arm/kvm64.c b/target/arm/kvm64.c index ba232b27a6d3..61947f3716e1 100644 --- a/target/arm/kvm64.c +++ b/target/arm/kvm64.c @@ -706,13 +706,53 @@ int kvm_arm_cpreg_level(uint64_t regidx) #define AARCH64_SIMD_CTRL_REG(x) (KVM_REG_ARM64 | KVM_REG_SIZE_U32 | \ KVM_REG_ARM_CORE | KVM_REG_ARM_CORE_REG(x)) +static int kvm_arch_put_fpsimd(CPUState *cs) +{ + ARMCPU *cpu = ARM_CPU(cs); + CPUARMState *env = &cpu->env; + struct kvm_one_reg reg; + uint32_t fpr; + int i, ret; + + for (i = 0; i < 32; i++) { + uint64_t *q = aa64_vfp_qreg(env, i); +#ifdef HOST_WORDS_BIGENDIAN + uint64_t fp_val[2] = { q[1], q[0] }; + reg.addr = (uintptr_t)fp_val; +#else + reg.addr = (uintptr_t)q; +#endif + reg.id = AARCH64_SIMD_CORE_REG(fp_regs.vregs[i]); + ret = kvm_vcpu_ioctl(cs, KVM_SET_ONE_REG, ®); + if (ret) { + return ret; + } + } + + reg.addr = (uintptr_t)(&fpr); + fpr = vfp_get_fpsr(env); + reg.id = AARCH64_SIMD_CTRL_REG(fp_regs.fpsr); + ret = kvm_vcpu_ioctl(cs, KVM_SET_ONE_REG, ®); + if (ret) { + return ret; + } + + reg.addr = (uintptr_t)(&fpr); + fpr = vfp_get_fpcr(env); + reg.id = AARCH64_SIMD_CTRL_REG(fp_regs.fpcr); + ret = kvm_vcpu_ioctl(cs, KVM_SET_ONE_REG, ®); + if (ret) { + return ret; + } + + return 0; +} + int kvm_arch_put_registers(CPUState *cs, int level) { struct kvm_one_reg reg; - uint32_t fpr; uint64_t val; - int i; - int ret; + int i, ret; unsigned int el; ARMCPU *cpu = ARM_CPU(cs); @@ -802,33 +842,7 @@ int kvm_arch_put_registers(CPUState *cs, int level) } } - /* Advanced SIMD and FP registers. */ - for (i = 0; i < 32; i++) { - uint64_t *q = aa64_vfp_qreg(env, i); -#ifdef HOST_WORDS_BIGENDIAN - uint64_t fp_val[2] = { q[1], q[0] }; - reg.addr = (uintptr_t)fp_val; -#else - reg.addr = (uintptr_t)q; -#endif - reg.id = AARCH64_SIMD_CORE_REG(fp_regs.vregs[i]); - ret = kvm_vcpu_ioctl(cs, KVM_SET_ONE_REG, ®); - if (ret) { - return ret; - } - } - - reg.addr = (uintptr_t)(&fpr); - fpr = vfp_get_fpsr(env); - reg.id = AARCH64_SIMD_CTRL_REG(fp_regs.fpsr); - ret = kvm_vcpu_ioctl(cs, KVM_SET_ONE_REG, ®); - if (ret) { - return ret; - } - - fpr = vfp_get_fpcr(env); - reg.id = AARCH64_SIMD_CTRL_REG(fp_regs.fpcr); - ret = kvm_vcpu_ioctl(cs, KVM_SET_ONE_REG, ®); + ret = kvm_arch_put_fpsimd(cs); if (ret) { return ret; } @@ -849,14 +863,54 @@ int kvm_arch_put_registers(CPUState *cs, int level) return ret; } +static int kvm_arch_get_fpsimd(CPUState *cs) +{ + ARMCPU *cpu = ARM_CPU(cs); + CPUARMState *env = &cpu->env; + struct kvm_one_reg reg; + uint32_t fpr; + int i, ret; + + for (i = 0; i < 32; i++) { + uint64_t *q = aa64_vfp_qreg(env, i); + reg.id = AARCH64_SIMD_CORE_REG(fp_regs.vregs[i]); + reg.addr = (uintptr_t)q; + ret = kvm_vcpu_ioctl(cs, KVM_GET_ONE_REG, ®); + if (ret) { + return ret; + } else { +#ifdef HOST_WORDS_BIGENDIAN + uint64_t t; + t = q[0], q[0] = q[1], q[1] = t; +#endif + } + } + + reg.addr = (uintptr_t)(&fpr); + reg.id = AARCH64_SIMD_CTRL_REG(fp_regs.fpsr); + ret = kvm_vcpu_ioctl(cs, KVM_GET_ONE_REG, ®); + if (ret) { + return ret; + } + vfp_set_fpsr(env, fpr); + + reg.addr = (uintptr_t)(&fpr); + reg.id = AARCH64_SIMD_CTRL_REG(fp_regs.fpcr); + ret = kvm_vcpu_ioctl(cs, KVM_GET_ONE_REG, ®); + if (ret) { + return ret; + } + vfp_set_fpcr(env, fpr); + + return 0; +} + int kvm_arch_get_registers(CPUState *cs) { struct kvm_one_reg reg; uint64_t val; - uint32_t fpr; unsigned int el; - int i; - int ret; + int i, ret; ARMCPU *cpu = ARM_CPU(cs); CPUARMState *env = &cpu->env; @@ -945,36 +999,10 @@ int kvm_arch_get_registers(CPUState *cs) env->spsr = env->banked_spsr[i]; } - /* Advanced SIMD and FP registers */ - for (i = 0; i < 32; i++) { - uint64_t *q = aa64_vfp_qreg(env, i); - reg.id = AARCH64_SIMD_CORE_REG(fp_regs.vregs[i]); - reg.addr = (uintptr_t)q; - ret = kvm_vcpu_ioctl(cs, KVM_GET_ONE_REG, ®); - if (ret) { - return ret; - } else { -#ifdef HOST_WORDS_BIGENDIAN - uint64_t t; - t = q[0], q[0] = q[1], q[1] = t; -#endif - } - } - - reg.addr = (uintptr_t)(&fpr); - reg.id = AARCH64_SIMD_CTRL_REG(fp_regs.fpsr); - ret = kvm_vcpu_ioctl(cs, KVM_GET_ONE_REG, ®); - if (ret) { - return ret; - } - vfp_set_fpsr(env, fpr); - - reg.id = AARCH64_SIMD_CTRL_REG(fp_regs.fpcr); - ret = kvm_vcpu_ioctl(cs, KVM_GET_ONE_REG, ®); + ret = kvm_arch_get_fpsimd(cs); if (ret) { return ret; } - vfp_set_fpcr(env, fpr); ret = kvm_get_vcpu_events(cpu); if (ret) { From patchwork Sun May 12 08:36:16 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Jones X-Patchwork-Id: 10939945 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2E0BD933 for ; Sun, 12 May 2019 08:47:42 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1D0E120564 for ; Sun, 12 May 2019 08:47:42 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1032E232A7; Sun, 12 May 2019 08:47:42 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 6ABC720564 for ; Sun, 12 May 2019 08:47:41 +0000 (UTC) Received: from localhost ([127.0.0.1]:40622 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hPk8y-0003xy-QK for patchwork-qemu-devel@patchwork.kernel.org; Sun, 12 May 2019 04:47:40 -0400 Received: from eggs.gnu.org ([209.51.188.92]:38518) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hPjzZ-0004Mr-NM for qemu-devel@nongnu.org; Sun, 12 May 2019 04:37:59 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hPjzX-0003op-W1 for qemu-devel@nongnu.org; Sun, 12 May 2019 04:37:57 -0400 Received: from mx1.redhat.com ([209.132.183.28]:58308) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hPjyZ-0003LM-EK; Sun, 12 May 2019 04:37:49 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id B98283086212; Sun, 12 May 2019 08:36:54 +0000 (UTC) Received: from kamzik.brq.redhat.com (ovpn-116-87.ams2.redhat.com [10.36.116.87]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3376E5D706; Sun, 12 May 2019 08:36:50 +0000 (UTC) From: Andrew Jones To: qemu-devel@nongnu.org, qemu-arm@nongnu.org Date: Sun, 12 May 2019 10:36:16 +0200 Message-Id: <20190512083624.8916-6-drjones@redhat.com> In-Reply-To: <20190512083624.8916-1-drjones@redhat.com> References: <20190512083624.8916-1-drjones@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.42]); Sun, 12 May 2019 08:36:54 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH 05/13] target/arm/kvm: Add kvm_arch_get/put_sve X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, richard.henderson@linaro.org, armbru@redhat.com, abologna@redhat.com, alex.bennee@linaro.org, Dave.Martin@arm.com Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP These are the SVE equivalents to kvm_arch_get/put_fpsimd. Signed-off-by: Andrew Jones --- target/arm/kvm64.c | 127 +++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 123 insertions(+), 4 deletions(-) diff --git a/target/arm/kvm64.c b/target/arm/kvm64.c index 61947f3716e1..86362f4cd7d0 100644 --- a/target/arm/kvm64.c +++ b/target/arm/kvm64.c @@ -658,11 +658,12 @@ int kvm_arch_init_vcpu(CPUState *cs) bool kvm_arm_reg_syncs_via_cpreg_list(uint64_t regidx) { /* Return true if the regidx is a register we should synchronize - * via the cpreg_tuples array (ie is not a core reg we sync by - * hand in kvm_arch_get/put_registers()) + * via the cpreg_tuples array (ie is not a core or sve reg that + * we sync by hand in kvm_arch_get/put_registers()) */ switch (regidx & KVM_REG_ARM_COPROC_MASK) { case KVM_REG_ARM_CORE: + case KVM_REG_ARM64_SVE: return false; default: return true; @@ -748,6 +749,61 @@ static int kvm_arch_put_fpsimd(CPUState *cs) return 0; } +static int kvm_arch_put_sve(CPUState *cs) +{ + ARMCPU *cpu = ARM_CPU(cs); + CPUARMState *env = &cpu->env; + struct kvm_one_reg reg; + int n, ret; + + for (n = 0; n < KVM_ARM64_SVE_NUM_ZREGS; n++) { + uint64_t *q = aa64_vfp_qreg(env, n); +#ifdef HOST_WORDS_BIGENDIAN + uint64_t d[ARM_MAX_VQ * 2]; + int i; + for (i = 0; i < cpu->sve_max_vq * 2; i++) { + d[i] = q[cpu->sve_max_vq * 2 - 1 - i]; + } + reg.addr = (uintptr_t)d; +#else + reg.addr = (uintptr_t)q; +#endif + reg.id = KVM_REG_ARM64_SVE_ZREG(n, 0); + ret = kvm_vcpu_ioctl(cs, KVM_SET_ONE_REG, ®); + if (ret) { + return ret; + } + } + + for (n = 0; n < KVM_ARM64_SVE_NUM_PREGS; n++) { + uint64_t *p = &env->vfp.pregs[n].p[0]; +#ifdef HOST_WORDS_BIGENDIAN + uint64_t d[ARM_MAX_VQ * 2]; + int i; + for (i = 0; i < cpu->sve_max_vq * 2 / 8; i++) { + d[i] = p[cpu->sve_max_vq * 2 / 8 - 1 - i]; + } + reg.addr = (uintptr_t)d; +#else + reg.addr = (uintptr_t)p; +#endif + reg.id = KVM_REG_ARM64_SVE_PREG(n, 0); + ret = kvm_vcpu_ioctl(cs, KVM_SET_ONE_REG, ®); + if (ret) { + return ret; + } + } + + reg.addr = (uintptr_t)&env->vfp.pregs[FFR_PRED_NUM].p[0]; + reg.id = KVM_REG_ARM64_SVE_FFR(0); + ret = kvm_vcpu_ioctl(cs, KVM_SET_ONE_REG, ®); + if (ret) { + return ret; + } + + return 0; +} + int kvm_arch_put_registers(CPUState *cs, int level) { struct kvm_one_reg reg; @@ -842,7 +898,11 @@ int kvm_arch_put_registers(CPUState *cs, int level) } } - ret = kvm_arch_put_fpsimd(cs); + if (!cpu_isar_feature(aa64_sve, cpu)) { + ret = kvm_arch_put_fpsimd(cs); + } else { + ret = kvm_arch_put_sve(cs); + } if (ret) { return ret; } @@ -905,6 +965,61 @@ static int kvm_arch_get_fpsimd(CPUState *cs) return 0; } +static int kvm_arch_get_sve(CPUState *cs) +{ + ARMCPU *cpu = ARM_CPU(cs); + CPUARMState *env = &cpu->env; + struct kvm_one_reg reg; + int n, ret; + + for (n = 0; n < KVM_ARM64_SVE_NUM_ZREGS; n++) { + uint64_t *q = aa64_vfp_qreg(env, n); + reg.id = KVM_REG_ARM64_SVE_ZREG(n, 0); + reg.addr = (uintptr_t)q; + ret = kvm_vcpu_ioctl(cs, KVM_GET_ONE_REG, ®); + if (ret) { + return ret; + } else { +#ifdef HOST_WORDS_BIGENDIAN + int i = 0, j = cpu->sve_max_vq * 2 - 1; + while (i < j) { + uint64_t t; + t = q[i], q[i] = q[j], q[j] = t; + ++i, --j; + } +#endif + } + } + + for (n = 0; n < KVM_ARM64_SVE_NUM_PREGS; n++) { + uint64_t *p = &env->vfp.pregs[n].p[0]; + reg.id = KVM_REG_ARM64_SVE_PREG(n, 0); + reg.addr = (uintptr_t)p; + ret = kvm_vcpu_ioctl(cs, KVM_GET_ONE_REG, ®); + if (ret) { + return ret; + } else { +#ifdef HOST_WORDS_BIGENDIAN + int i = 0, j = cpu->sve_max_vq * 2 / 8 - 1; + while (i < j) { + uint64_t t; + t = q[i], q[i] = q[j], q[j] = t; + ++i, --j; + } +#endif + } + } + + reg.addr = (uintptr_t)&env->vfp.pregs[FFR_PRED_NUM].p[0]; + reg.id = KVM_REG_ARM64_SVE_FFR(0); + ret = kvm_vcpu_ioctl(cs, KVM_GET_ONE_REG, ®); + if (ret) { + return ret; + } + + return 0; +} + int kvm_arch_get_registers(CPUState *cs) { struct kvm_one_reg reg; @@ -999,7 +1114,11 @@ int kvm_arch_get_registers(CPUState *cs) env->spsr = env->banked_spsr[i]; } - ret = kvm_arch_get_fpsimd(cs); + if (!cpu_isar_feature(aa64_sve, cpu)) { + ret = kvm_arch_get_fpsimd(cs); + } else { + ret = kvm_arch_get_sve(cs); + } if (ret) { return ret; } From patchwork Sun May 12 08:36:17 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Jones X-Patchwork-Id: 10939933 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9497D13AD for ; Sun, 12 May 2019 08:41:08 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 830D9232A7 for ; Sun, 12 May 2019 08:41:08 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 75ECB2623D; Sun, 12 May 2019 08:41:08 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 94A2D232A7 for ; Sun, 12 May 2019 08:41:06 +0000 (UTC) Received: from localhost ([127.0.0.1]:40523 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hPk2b-00078t-Tp for patchwork-qemu-devel@patchwork.kernel.org; Sun, 12 May 2019 04:41:05 -0400 Received: from eggs.gnu.org ([209.51.188.92]:38474) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hPjzY-0004La-RX for qemu-devel@nongnu.org; Sun, 12 May 2019 04:37:58 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hPjzX-0003nm-Cq for qemu-devel@nongnu.org; Sun, 12 May 2019 04:37:56 -0400 Received: from mx1.redhat.com ([209.132.183.28]:50770) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hPjyf-0003Mq-CN; Sun, 12 May 2019 04:37:49 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id A7EFAB0AA6; Sun, 12 May 2019 08:37:00 +0000 (UTC) Received: from kamzik.brq.redhat.com (ovpn-116-87.ams2.redhat.com [10.36.116.87]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7FE2E5D706; Sun, 12 May 2019 08:36:55 +0000 (UTC) From: Andrew Jones To: qemu-devel@nongnu.org, qemu-arm@nongnu.org Date: Sun, 12 May 2019 10:36:17 +0200 Message-Id: <20190512083624.8916-7-drjones@redhat.com> In-Reply-To: <20190512083624.8916-1-drjones@redhat.com> References: <20190512083624.8916-1-drjones@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Sun, 12 May 2019 08:37:00 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH 06/13] target/arm/kvm: max cpu: Enable SVE when available X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, richard.henderson@linaro.org, armbru@redhat.com, abologna@redhat.com, alex.bennee@linaro.org, Dave.Martin@arm.com Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP Enable SVE in the KVM guest when the 'max' cpu type is configured and KVM supports it. KVM SVE requires use of the new finalize vcpu ioctl, so we add that now too. Signed-off-by: Andrew Jones --- target/arm/cpu64.c | 1 + target/arm/kvm.c | 5 +++++ target/arm/kvm64.c | 16 +++++++++++++++- target/arm/kvm_arm.h | 12 ++++++++++++ 4 files changed, 33 insertions(+), 1 deletion(-) diff --git a/target/arm/cpu64.c b/target/arm/cpu64.c index 228906f26786..6c19ef6837d5 100644 --- a/target/arm/cpu64.c +++ b/target/arm/cpu64.c @@ -292,6 +292,7 @@ static void aarch64_max_initfn(Object *obj) if (kvm_enabled()) { kvm_arm_set_cpu_features_from_host(cpu); + cpu->sve_max_vq = ARM_MAX_VQ; } else { uint64_t t; uint32_t u; diff --git a/target/arm/kvm.c b/target/arm/kvm.c index 599563461264..c51db4229d0f 100644 --- a/target/arm/kvm.c +++ b/target/arm/kvm.c @@ -50,6 +50,11 @@ int kvm_arm_vcpu_init(CPUState *cs) return kvm_vcpu_ioctl(cs, KVM_ARM_VCPU_INIT, &init); } +int kvm_arm_vcpu_finalize(CPUState *cs, int feature) +{ + return kvm_vcpu_ioctl(cs, KVM_ARM_VCPU_FINALIZE, &feature); +} + void kvm_arm_init_serror_injection(CPUState *cs) { cap_has_inject_serror_esr = kvm_check_extension(cs->kvm_state, diff --git a/target/arm/kvm64.c b/target/arm/kvm64.c index 86362f4cd7d0..c2d92df75353 100644 --- a/target/arm/kvm64.c +++ b/target/arm/kvm64.c @@ -622,13 +622,20 @@ int kvm_arch_init_vcpu(CPUState *cs) cpu->kvm_init_features[0] |= 1 << KVM_ARM_VCPU_EL1_32BIT; } if (!kvm_check_extension(cs->kvm_state, KVM_CAP_ARM_PMU_V3)) { - cpu->has_pmu = false; + cpu->has_pmu = false; } if (cpu->has_pmu) { cpu->kvm_init_features[0] |= 1 << KVM_ARM_VCPU_PMU_V3; } else { unset_feature(&env->features, ARM_FEATURE_PMU); } + if (cpu->sve_max_vq) { + if (!kvm_check_extension(cs->kvm_state, KVM_CAP_ARM_SVE)) { + cpu->sve_max_vq = 0; + } else { + cpu->kvm_init_features[0] |= 1 << KVM_ARM_VCPU_SVE; + } + } /* Do KVM_ARM_VCPU_INIT ioctl */ ret = kvm_arm_vcpu_init(cs); @@ -636,6 +643,13 @@ int kvm_arch_init_vcpu(CPUState *cs) return ret; } + if (cpu->sve_max_vq) { + ret = kvm_arm_vcpu_finalize(cs, KVM_ARM_VCPU_SVE); + if (ret) { + return ret; + } + } + /* * When KVM is in use, PSCI is emulated in-kernel and not by qemu. * Currently KVM has its own idea about MPIDR assignment, so we diff --git a/target/arm/kvm_arm.h b/target/arm/kvm_arm.h index 2a07333c615f..c488ec3ab410 100644 --- a/target/arm/kvm_arm.h +++ b/target/arm/kvm_arm.h @@ -27,6 +27,18 @@ */ int kvm_arm_vcpu_init(CPUState *cs); +/** + * kvm_arm_vcpu_finalize + * @cs: CPUState + * @feature: int + * + * Finalizes the configuration of the specified VCPU feature + * by invoking the KVM_ARM_VCPU_FINALIZE ioctl. + * + * Returns: 0 if success else < 0 error code + */ +int kvm_arm_vcpu_finalize(CPUState *cs, int feature); + /** * kvm_arm_register_device: * @mr: memory region for this device From patchwork Sun May 12 08:36:18 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Jones X-Patchwork-Id: 10939931 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5070113AD for ; Sun, 12 May 2019 08:40:09 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4203E1FF13 for ; Sun, 12 May 2019 08:40:09 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 36375201B1; Sun, 12 May 2019 08:40:09 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id AA5641FF13 for ; Sun, 12 May 2019 08:40:08 +0000 (UTC) Received: from localhost ([127.0.0.1]:40502 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hPk1f-00064m-W1 for patchwork-qemu-devel@patchwork.kernel.org; Sun, 12 May 2019 04:40:08 -0400 Received: from eggs.gnu.org ([209.51.188.92]:38454) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hPjzY-0004L8-FF for qemu-devel@nongnu.org; Sun, 12 May 2019 04:37:58 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hPjzW-0003mG-4g for qemu-devel@nongnu.org; Sun, 12 May 2019 04:37:56 -0400 Received: from mx1.redhat.com ([209.132.183.28]:50790) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hPjyj-0003O5-IP; Sun, 12 May 2019 04:37:49 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id D7F50DC8EF; Sun, 12 May 2019 08:37:04 +0000 (UTC) Received: from kamzik.brq.redhat.com (ovpn-116-87.ams2.redhat.com [10.36.116.87]) by smtp.corp.redhat.com (Postfix) with ESMTP id 237625D706; Sun, 12 May 2019 08:37:00 +0000 (UTC) From: Andrew Jones To: qemu-devel@nongnu.org, qemu-arm@nongnu.org Date: Sun, 12 May 2019 10:36:18 +0200 Message-Id: <20190512083624.8916-8-drjones@redhat.com> In-Reply-To: <20190512083624.8916-1-drjones@redhat.com> References: <20190512083624.8916-1-drjones@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Sun, 12 May 2019 08:37:04 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH 07/13] target/arm/kvm: max cpu: Allow sve max vector length setting X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, richard.henderson@linaro.org, armbru@redhat.com, abologna@redhat.com, alex.bennee@linaro.org, Dave.Martin@arm.com Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP Allow the cpu type 'max' sve-max-vq property to work with kvm too. If the property is not specified then the maximum kvm supports is used. If it is specified we check that kvm supports that exact length or error out if it doesn't. Signed-off-by: Andrew Jones --- target/arm/cpu.h | 4 +++ target/arm/cpu64.c | 7 ++-- target/arm/kvm64.c | 80 ++++++++++++++++++++++++++++++++++++++++++++-- 3 files changed, 86 insertions(+), 5 deletions(-) diff --git a/target/arm/cpu.h b/target/arm/cpu.h index 733b840a7127..8292d547e8f9 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -3122,6 +3122,10 @@ static inline uint64_t arm_sctlr(CPUARMState *env, int el) } } +static inline int arm_cpu_fls64(uint64_t v) +{ + return !v ? 0 : 64 - clz64(v); +} /* Return true if the processor is in big-endian mode. */ static inline bool arm_cpu_data_is_big_endian(CPUARMState *env) diff --git a/target/arm/cpu64.c b/target/arm/cpu64.c index 6c19ef6837d5..3756e7e2a3e5 100644 --- a/target/arm/cpu64.c +++ b/target/arm/cpu64.c @@ -292,7 +292,7 @@ static void aarch64_max_initfn(Object *obj) if (kvm_enabled()) { kvm_arm_set_cpu_features_from_host(cpu); - cpu->sve_max_vq = ARM_MAX_VQ; + cpu->sve_max_vq = -1; /* set in kvm_arch_init_vcpu() */ } else { uint64_t t; uint32_t u; @@ -374,9 +374,10 @@ static void aarch64_max_initfn(Object *obj) #endif cpu->sve_max_vq = ARM_MAX_VQ; - object_property_add(obj, "sve-max-vq", "uint32", cpu_max_get_sve_vq, - cpu_max_set_sve_vq, NULL, NULL, &error_fatal); } + + object_property_add(obj, "sve-max-vq", "uint32", cpu_max_get_sve_vq, + cpu_max_set_sve_vq, NULL, NULL, &error_fatal); } struct ARMCPUInfo { diff --git a/target/arm/kvm64.c b/target/arm/kvm64.c index c2d92df75353..0c666e405357 100644 --- a/target/arm/kvm64.c +++ b/target/arm/kvm64.c @@ -446,6 +446,59 @@ void kvm_arm_pmu_set_irq(CPUState *cs, int irq) } } +static int kvm_arm_get_sve_vls(CPUState *cs, uint64_t sve_vls[]) +{ + struct kvm_one_reg reg = { + .id = KVM_REG_ARM64_SVE_VLS, + .addr = (uint64_t)&sve_vls[0], + }; + int i, ret; + + ret = kvm_vcpu_ioctl(cs, KVM_GET_ONE_REG, ®); + if (ret) { + return ret; + } + + ret = 0; + for (i = KVM_ARM64_SVE_VLS_WORDS - 1; i >= 0; --i) { + if (sve_vls[i]) { + ret = arm_cpu_fls64(sve_vls[i]) + i * 64; + break; + } + } + + return ret; +} + +static int kvm_arm_set_sve_vls(CPUState *cs, uint64_t sve_vls[], int max_vq) +{ + struct kvm_one_reg reg = { + .id = KVM_REG_ARM64_SVE_VLS, + .addr = (uint64_t)&sve_vls[0], + }; + int i; + + for (i = KVM_ARM64_SVE_VLS_WORDS - 1; i >= 0; --i) { + if (sve_vls[i]) { + int vq = arm_cpu_fls64(sve_vls[i]) + i * 64; + while (vq > max_vq) { + sve_vls[i] &= ~BIT_MASK(vq - 1); + vq = arm_cpu_fls64(sve_vls[i]) + i * 64; + } + if (vq < max_vq) { + error_report("sve-max-vq=%d is not a valid length", max_vq); + error_printf("next lowest is %d\n", vq); + return -EINVAL; + } + if (vq == max_vq) { + break; + } + } + } + + return kvm_vcpu_ioctl(cs, KVM_SET_ONE_REG, ®); +} + static inline void set_feature(uint64_t *features, int feature) { *features |= 1ULL << feature; @@ -605,7 +658,7 @@ int kvm_arch_init_vcpu(CPUState *cs) if (cpu->kvm_target == QEMU_KVM_ARM_TARGET_NONE || !object_dynamic_cast(OBJECT(cpu), TYPE_AARCH64_CPU)) { - fprintf(stderr, "KVM is not supported for this guest CPU type\n"); + error_report("KVM is not supported for this guest CPU type"); return -EINVAL; } @@ -631,7 +684,12 @@ int kvm_arch_init_vcpu(CPUState *cs) } if (cpu->sve_max_vq) { if (!kvm_check_extension(cs->kvm_state, KVM_CAP_ARM_SVE)) { - cpu->sve_max_vq = 0; + if (cpu->sve_max_vq == -1) { + cpu->sve_max_vq = 0; + } else { + error_report("This KVM host does not support SVE"); + return -EINVAL; + } } else { cpu->kvm_init_features[0] |= 1 << KVM_ARM_VCPU_SVE; } @@ -644,6 +702,24 @@ int kvm_arch_init_vcpu(CPUState *cs) } if (cpu->sve_max_vq) { + uint64_t sve_vls[KVM_ARM64_SVE_VLS_WORDS]; + ret = kvm_arm_get_sve_vls(cs, sve_vls); + if (ret < 0) { + return ret; + } + if (cpu->sve_max_vq == -1) { + cpu->sve_max_vq = ret; + } else if (cpu->sve_max_vq > ret) { + error_report("This KVM host does not support SVE vectors " + "of length %d quadwords (%d bytes)", + cpu->sve_max_vq, cpu->sve_max_vq * 16); + return -EINVAL; + } else { + ret = kvm_arm_set_sve_vls(cs, sve_vls, cpu->sve_max_vq); + if (ret < 0) { + return ret; + } + } ret = kvm_arm_vcpu_finalize(cs, KVM_ARM_VCPU_SVE); if (ret) { return ret; From patchwork Sun May 12 08:36:19 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Jones X-Patchwork-Id: 10939941 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 944A1933 for ; Sun, 12 May 2019 08:45:36 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7D75F20564 for ; Sun, 12 May 2019 08:45:36 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 57ED3232A7; Sun, 12 May 2019 08:45:36 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id C682220564 for ; Sun, 12 May 2019 08:45:35 +0000 (UTC) Received: from localhost ([127.0.0.1]:40608 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hPk6w-0002Rs-Ph for patchwork-qemu-devel@patchwork.kernel.org; Sun, 12 May 2019 04:45:34 -0400 Received: from eggs.gnu.org ([209.51.188.92]:38493) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hPjzZ-0004M6-5u for qemu-devel@nongnu.org; Sun, 12 May 2019 04:37:58 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hPjzX-0003oK-Kz for qemu-devel@nongnu.org; Sun, 12 May 2019 04:37:57 -0400 Received: from mx1.redhat.com ([209.132.183.28]:38624) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hPjyo-0003PZ-Cz; Sun, 12 May 2019 04:37:49 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id AB1753092662; Sun, 12 May 2019 08:37:09 +0000 (UTC) Received: from kamzik.brq.redhat.com (ovpn-116-87.ams2.redhat.com [10.36.116.87]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5168E5D706; Sun, 12 May 2019 08:37:05 +0000 (UTC) From: Andrew Jones To: qemu-devel@nongnu.org, qemu-arm@nongnu.org Date: Sun, 12 May 2019 10:36:19 +0200 Message-Id: <20190512083624.8916-9-drjones@redhat.com> In-Reply-To: <20190512083624.8916-1-drjones@redhat.com> References: <20190512083624.8916-1-drjones@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.43]); Sun, 12 May 2019 08:37:09 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH 08/13] target/arm/monitor: Add query-sve-vector-lengths X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, richard.henderson@linaro.org, armbru@redhat.com, abologna@redhat.com, alex.bennee@linaro.org, Dave.Martin@arm.com Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP Provide a QMP interface to query the supported SVE vector lengths. A migratable guest will need to explicitly specify a valid set of lengths on the command line and that set can be obtained from the list returned with this QMP command. This patch only introduces the QMP command with the TCG implementation. The result may not yet be correct for KVM. Following patches ensure the KVM result is correct. Signed-off-by: Andrew Jones --- qapi/target.json | 34 ++++++++++++++++++++++++ target/arm/monitor.c | 62 ++++++++++++++++++++++++++++++++++++++++++++ tests/qmp-cmd-test.c | 1 + 3 files changed, 97 insertions(+) diff --git a/qapi/target.json b/qapi/target.json index 1d4d54b6002e..ca1e85254780 100644 --- a/qapi/target.json +++ b/qapi/target.json @@ -397,6 +397,40 @@ { 'command': 'query-gic-capabilities', 'returns': ['GICCapability'], 'if': 'defined(TARGET_ARM)' } +## +# @SVEVectorLengths: +# +# The struct contains a list of integers where each integer is a valid +# SVE vector length for a KVM guest on this host. The vector lengths +# are in quadword (128-bit) units, e.g. '4' means 512 bits (64 bytes). +# +# @vls: list of vector lengths in quadwords. +# +# Since: 4.1 +## +{ 'struct': 'SVEVectorLengths', + 'data': { 'vls': ['int'] }, + 'if': 'defined(TARGET_ARM)' } + +## +# @query-sve-vector-lengths: +# +# This command is ARM-only. It will return a list of SVEVectorLengths +# objects. The list describes all valid SVE vector length sets. +# +# Returns: a list of SVEVectorLengths objects +# +# Since: 4.1 +# +# -> { "execute": "query-sve-vector-lengths" } +# <- { "return": [ { "vls": [ 1 ] }, +# { "vls": [ 1, 2 ] }, +# { "vls": [ 1, 2, 4 ] } ] } +# +## +{ 'command': 'query-sve-vector-lengths', 'returns': ['SVEVectorLengths'], + 'if': 'defined(TARGET_ARM)' } + ## # @CpuModelExpansionInfo: # diff --git a/target/arm/monitor.c b/target/arm/monitor.c index 41b32b94b258..8b2afa255c92 100644 --- a/target/arm/monitor.c +++ b/target/arm/monitor.c @@ -24,6 +24,7 @@ #include "hw/boards.h" #include "kvm_arm.h" #include "qapi/qapi-commands-target.h" +#include "monitor/hmp-target.h" static GICCapability *gic_cap_new(int version) { @@ -82,3 +83,64 @@ GICCapabilityList *qmp_query_gic_capabilities(Error **errp) return head; } + +static SVEVectorLengths *qmp_sve_vls_get(void) +{ + CPUArchState *env = mon_get_cpu_env(); + ARMCPU *cpu = arm_env_get_cpu(env); + SVEVectorLengths *vls = g_new(SVEVectorLengths, 1); + intList **v = &vls->vls; + int i; + + if (cpu->sve_max_vq == 0) { + *v = g_new0(intList, 1); /* one vl of 0 means none supported */ + return vls; + } + + for (i = 1; i <= cpu->sve_max_vq; ++i) { + *v = g_new0(intList, 1); + (*v)->value = i; + v = &(*v)->next; + } + + return vls; +} + +static SVEVectorLengths *qmp_sve_vls_dup_and_truncate(SVEVectorLengths *vls) +{ + SVEVectorLengths *trunc_vls; + intList **v, *p = vls->vls; + + if (!p->next) { + return NULL; + } + + trunc_vls = g_new(SVEVectorLengths, 1); + v = &trunc_vls->vls; + + for (; p->next; p = p->next) { + *v = g_new0(intList, 1); + (*v)->value = p->value; + v = &(*v)->next; + } + + return trunc_vls; +} + +SVEVectorLengthsList *qmp_query_sve_vector_lengths(Error **errp) +{ + SVEVectorLengthsList *vls_list = g_new0(SVEVectorLengthsList, 1); + SVEVectorLengths *vls = qmp_sve_vls_get(); + + while (vls) { + vls_list->value = vls; + vls = qmp_sve_vls_dup_and_truncate(vls); + if (vls) { + SVEVectorLengthsList *next = vls_list; + vls_list = g_new0(SVEVectorLengthsList, 1); + vls_list->next = next; + } + } + + return vls_list; +} diff --git a/tests/qmp-cmd-test.c b/tests/qmp-cmd-test.c index 9f5228cd9951..3d714dbc6a4a 100644 --- a/tests/qmp-cmd-test.c +++ b/tests/qmp-cmd-test.c @@ -90,6 +90,7 @@ static bool query_is_blacklisted(const char *cmd) /* Success depends on target arch: */ "query-cpu-definitions", /* arm, i386, ppc, s390x */ "query-gic-capabilities", /* arm */ + "query-sve-vector-lengths", /* arm */ /* Success depends on target-specific build configuration: */ "query-pci", /* CONFIG_PCI */ /* Success depends on launching SEV guest */ From patchwork Sun May 12 08:36:20 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Jones X-Patchwork-Id: 10939929 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4B24B13AD for ; Sun, 12 May 2019 08:40:03 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 37CCC1FF13 for ; Sun, 12 May 2019 08:40:03 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2544B201B1; Sun, 12 May 2019 08:40:03 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id C1D011FF13 for ; Sun, 12 May 2019 08:40:02 +0000 (UTC) Received: from localhost ([127.0.0.1]:40489 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hPk1Z-00061v-To for patchwork-qemu-devel@patchwork.kernel.org; Sun, 12 May 2019 04:40:01 -0400 Received: from eggs.gnu.org ([209.51.188.92]:38413) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hPjzX-0004J8-3Y for qemu-devel@nongnu.org; Sun, 12 May 2019 04:37:56 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hPjzV-0003l9-Fc for qemu-devel@nongnu.org; Sun, 12 May 2019 04:37:55 -0400 Received: from mx1.redhat.com ([209.132.183.28]:48314) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hPjyv-0003SA-Vp; Sun, 12 May 2019 04:37:49 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 6F69E811DC; Sun, 12 May 2019 08:37:16 +0000 (UTC) Received: from kamzik.brq.redhat.com (ovpn-116-87.ams2.redhat.com [10.36.116.87]) by smtp.corp.redhat.com (Postfix) with ESMTP id 72C0E5D706; Sun, 12 May 2019 08:37:10 +0000 (UTC) From: Andrew Jones To: qemu-devel@nongnu.org, qemu-arm@nongnu.org Date: Sun, 12 May 2019 10:36:20 +0200 Message-Id: <20190512083624.8916-10-drjones@redhat.com> In-Reply-To: <20190512083624.8916-1-drjones@redhat.com> References: <20190512083624.8916-1-drjones@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.27]); Sun, 12 May 2019 08:37:16 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH 09/13] target/arm/kvm: Export kvm_arm_get_sve_vls X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, richard.henderson@linaro.org, armbru@redhat.com, abologna@redhat.com, alex.bennee@linaro.org, Dave.Martin@arm.com Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP Signed-off-by: Andrew Jones --- target/arm/kvm64.c | 7 +++++-- target/arm/kvm_arm.h | 20 ++++++++++++++++++++ 2 files changed, 25 insertions(+), 2 deletions(-) diff --git a/target/arm/kvm64.c b/target/arm/kvm64.c index 0c666e405357..11c6334a7c08 100644 --- a/target/arm/kvm64.c +++ b/target/arm/kvm64.c @@ -446,7 +446,8 @@ void kvm_arm_pmu_set_irq(CPUState *cs, int irq) } } -static int kvm_arm_get_sve_vls(CPUState *cs, uint64_t sve_vls[]) +int kvm_arm_get_sve_vls(CPUState *cs, + uint64_t sve_vls[KVM_ARM64_SVE_VLS_WORDS]) { struct kvm_one_reg reg = { .id = KVM_REG_ARM64_SVE_VLS, @@ -470,7 +471,9 @@ static int kvm_arm_get_sve_vls(CPUState *cs, uint64_t sve_vls[]) return ret; } -static int kvm_arm_set_sve_vls(CPUState *cs, uint64_t sve_vls[], int max_vq) +static int kvm_arm_set_sve_vls(CPUState *cs, + uint64_t sve_vls[KVM_ARM64_SVE_VLS_WORDS], + int max_vq) { struct kvm_one_reg reg = { .id = KVM_REG_ARM64_SVE_VLS, diff --git a/target/arm/kvm_arm.h b/target/arm/kvm_arm.h index c488ec3ab410..748ed8d54985 100644 --- a/target/arm/kvm_arm.h +++ b/target/arm/kvm_arm.h @@ -248,6 +248,26 @@ int kvm_arm_vgic_probe(void); void kvm_arm_pmu_set_irq(CPUState *cs, int irq); void kvm_arm_pmu_init(CPUState *cs); +/** + * kvm_arm_get_sve_vls + * @cs: CPUState + * @sve_vls: valid vector length bitmap + * + * Get the valid vector length bitmap. If a bit 'bit' is set + * then the host supports a vector length of (bit * 16) bytes. + * + * For example, if + * + * sve_vls[0] = 0xb and + * sve_vls[1 ... KVM_ARM64_SVE_VLS_WORDS-1] = 0, + * + * then the host supports 16, 32, and 64 byte vector lengths. + * + * Returns: the highest set bit if successful else < 0 error code + */ +int kvm_arm_get_sve_vls(CPUState *cs, + uint64_t sve_vls[KVM_ARM64_SVE_VLS_WORDS]); + #else static inline void kvm_arm_set_cpu_features_from_host(ARMCPU *cpu) From patchwork Sun May 12 08:36:21 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Jones X-Patchwork-Id: 10939943 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3013A933 for ; Sun, 12 May 2019 08:45:56 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1E08120564 for ; Sun, 12 May 2019 08:45:56 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 0EA3B232A7; Sun, 12 May 2019 08:45:56 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id AC85720564 for ; Sun, 12 May 2019 08:45:55 +0000 (UTC) Received: from localhost ([127.0.0.1]:40610 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hPk7H-0002ky-2J for patchwork-qemu-devel@patchwork.kernel.org; Sun, 12 May 2019 04:45:55 -0400 Received: from eggs.gnu.org ([209.51.188.92]:38365) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hPjzW-0004I5-6E for qemu-devel@nongnu.org; Sun, 12 May 2019 04:37:55 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hPjzU-0003kD-Ff for qemu-devel@nongnu.org; Sun, 12 May 2019 04:37:54 -0400 Received: from mx1.redhat.com ([209.132.183.28]:52478) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hPjz1-0003UD-Lz; Sun, 12 May 2019 04:37:49 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id A24F03084248; Sun, 12 May 2019 08:37:21 +0000 (UTC) Received: from kamzik.brq.redhat.com (ovpn-116-87.ams2.redhat.com [10.36.116.87]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3754E5D706; Sun, 12 May 2019 08:37:16 +0000 (UTC) From: Andrew Jones To: qemu-devel@nongnu.org, qemu-arm@nongnu.org Date: Sun, 12 May 2019 10:36:21 +0200 Message-Id: <20190512083624.8916-11-drjones@redhat.com> In-Reply-To: <20190512083624.8916-1-drjones@redhat.com> References: <20190512083624.8916-1-drjones@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.40]); Sun, 12 May 2019 08:37:21 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH 10/13] target/arm/monitor: kvm: only return valid sve vector sets X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, richard.henderson@linaro.org, armbru@redhat.com, abologna@redhat.com, alex.bennee@linaro.org, Dave.Martin@arm.com Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP While the TCG SVE implementation can implement all vector lengths which are a quadword multiple, up to some maximum length, KVM can only provide what the host supports, and not all multiples are required to be supported by the architecture. With this patch we extend the QMP query to ask KVM for the valid vectors when KVM is enabled. Signed-off-by: Andrew Jones --- target/arm/monitor.c | 43 ++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 42 insertions(+), 1 deletion(-) diff --git a/target/arm/monitor.c b/target/arm/monitor.c index 8b2afa255c92..3e13dd7c7b7a 100644 --- a/target/arm/monitor.c +++ b/target/arm/monitor.c @@ -84,6 +84,41 @@ GICCapabilityList *qmp_query_gic_capabilities(Error **errp) return head; } +#ifdef CONFIG_KVM +static SVEVectorLengths *qmp_kvm_sve_vls_get(void) +{ + CPUArchState *env = mon_get_cpu_env(); + ARMCPU *cpu = arm_env_get_cpu(env); + uint64_t sve_vls[KVM_ARM64_SVE_VLS_WORDS]; + SVEVectorLengths *vls = g_new(SVEVectorLengths, 1); + intList **v = &vls->vls; + int ret, i; + + ret = kvm_arm_get_sve_vls(CPU(cpu), sve_vls); + if (ret <= 0) { + *v = g_new0(intList, 1); /* one vl of 0 means none supported */ + return vls; + } + + for (i = KVM_ARM64_SVE_VQ_MIN; i <= ret; ++i) { + int bitval = (sve_vls[(i - KVM_ARM64_SVE_VQ_MIN) / 64] >> + ((i - KVM_ARM64_SVE_VQ_MIN) % 64)) & 1; + if (bitval) { + *v = g_new0(intList, 1); + (*v)->value = i; + v = &(*v)->next; + } + } + + return vls; +} +#else +static SVEVectorLengths *qmp_kvm_sve_vls_get(void) +{ + return NULL; +} +#endif + static SVEVectorLengths *qmp_sve_vls_get(void) { CPUArchState *env = mon_get_cpu_env(); @@ -130,7 +165,13 @@ static SVEVectorLengths *qmp_sve_vls_dup_and_truncate(SVEVectorLengths *vls) SVEVectorLengthsList *qmp_query_sve_vector_lengths(Error **errp) { SVEVectorLengthsList *vls_list = g_new0(SVEVectorLengthsList, 1); - SVEVectorLengths *vls = qmp_sve_vls_get(); + SVEVectorLengths *vls; + + if (kvm_enabled()) { + vls = qmp_kvm_sve_vls_get(); + } else { + vls = qmp_sve_vls_get(); + } while (vls) { vls_list->value = vls; From patchwork Sun May 12 08:36:22 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Jones X-Patchwork-Id: 10939951 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B7924933 for ; Sun, 12 May 2019 08:51:17 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A73DD21BED for ; Sun, 12 May 2019 08:51:17 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 9913A22BF1; Sun, 12 May 2019 08:51:17 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id F12F021BED for ; Sun, 12 May 2019 08:51:16 +0000 (UTC) Received: from localhost ([127.0.0.1]:40678 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hPkCS-0006P7-A8 for patchwork-qemu-devel@patchwork.kernel.org; Sun, 12 May 2019 04:51:16 -0400 Received: from eggs.gnu.org ([209.51.188.92]:38554) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hPjza-0004OL-Ps for qemu-devel@nongnu.org; Sun, 12 May 2019 04:38:00 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hPjzZ-0003ql-7n for qemu-devel@nongnu.org; Sun, 12 May 2019 04:37:58 -0400 Received: from mx1.redhat.com ([209.132.183.28]:52490) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hPjz4-0003VK-5n; Sun, 12 May 2019 04:37:49 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 75341308427E; Sun, 12 May 2019 08:37:25 +0000 (UTC) Received: from kamzik.brq.redhat.com (ovpn-116-87.ams2.redhat.com [10.36.116.87]) by smtp.corp.redhat.com (Postfix) with ESMTP id 077FA5D706; Sun, 12 May 2019 08:37:21 +0000 (UTC) From: Andrew Jones To: qemu-devel@nongnu.org, qemu-arm@nongnu.org Date: Sun, 12 May 2019 10:36:22 +0200 Message-Id: <20190512083624.8916-12-drjones@redhat.com> In-Reply-To: <20190512083624.8916-1-drjones@redhat.com> References: <20190512083624.8916-1-drjones@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.40]); Sun, 12 May 2019 08:37:25 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH 11/13] target/arm/cpu64: max cpu: Introduce sve-vls-map X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, richard.henderson@linaro.org, armbru@redhat.com, abologna@redhat.com, alex.bennee@linaro.org, Dave.Martin@arm.com Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP Introduce another cpu property to control SVE vector lengths, sve-vls-map, which allows the user to explicitly select the set of vector lengths the guest can use. The map must conform to QEMU's limits and architectural constraints, checked when the property is set. Inconsistencies with sve-max-vq are also checked. The bit number of a set bit in the map represents the allowed vector length in number of quadwords. Note, as the map is implemented with a single 64-bit word we currently only support up to 8192-bit vectors. As QEMU and KVM only support up to 2048-bit vectors then this sufficient now, and probably for some time. Extending the bitmap beyond a single word will likely require changing the property to a string and adding yet another parser to QEMU. Signed-off-by: Andrew Jones --- target/arm/cpu.c | 4 +++ target/arm/cpu.h | 3 ++ target/arm/cpu64.c | 70 +++++++++++++++++++++++++++++++++++++++++--- target/arm/helper.c | 10 ++++++- target/arm/monitor.c | 9 ++++-- 5 files changed, 88 insertions(+), 8 deletions(-) diff --git a/target/arm/cpu.c b/target/arm/cpu.c index a181fa8dc1a7..ea0e24bba8b6 100644 --- a/target/arm/cpu.c +++ b/target/arm/cpu.c @@ -1188,6 +1188,10 @@ static void arm_cpu_realizefn(DeviceState *dev, Error **errp) set_feature(env, ARM_FEATURE_VBAR); } + if (!kvm_enabled() && !cpu->sve_vls_map) { + cpu->sve_vls_map = BIT_MASK(cpu->sve_max_vq) - 1; + } + register_cp_regs_for_features(cpu); arm_cpu_register_gdb_regs_for_features(cpu); diff --git a/target/arm/cpu.h b/target/arm/cpu.h index 8292d547e8f9..f0d0ce759ba8 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -920,6 +920,9 @@ struct ARMCPU { /* Used to set the maximum vector length the cpu will support. */ uint32_t sve_max_vq; + + /* Each bit represents a supported vector length of (bitnum * 16) bytes */ + uint64_t sve_vls_map; }; static inline ARMCPU *arm_env_get_cpu(CPUARMState *env) diff --git a/target/arm/cpu64.c b/target/arm/cpu64.c index 3756e7e2a3e5..9ac702d54136 100644 --- a/target/arm/cpu64.c +++ b/target/arm/cpu64.c @@ -273,14 +273,73 @@ static void cpu_max_set_sve_vq(Object *obj, Visitor *v, const char *name, visit_type_uint32(v, name, &cpu->sve_max_vq, &err); - if (!err && (cpu->sve_max_vq == 0 || cpu->sve_max_vq > ARM_MAX_VQ)) { - error_setg(&err, "unsupported SVE vector length"); - error_append_hint(&err, "Valid sve-max-vq in range [1-%d]\n", - ARM_MAX_VQ); + if (!err) { + if (cpu->sve_max_vq == 0 || cpu->sve_max_vq > ARM_MAX_VQ) { + error_setg(&err, "unsupported SVE vector length"); + error_append_hint(&err, "Valid sve-max-vq in range [1-%d]\n", + ARM_MAX_VQ); + } else if (cpu->sve_vls_map && + cpu->sve_max_vq != arm_cpu_fls64(cpu->sve_vls_map)) { + error_setg(&err, "sve-vls-map and sve-max-vq are inconsistent"); + } } error_propagate(errp, err); } +static void cpu_get_sve_vls_map(Object *obj, Visitor *v, const char *name, + void *opaque, Error **errp) +{ + ARMCPU *cpu = ARM_CPU(obj); + visit_type_uint64(v, name, &cpu->sve_vls_map, errp); +} + +static void cpu_set_sve_vls_map(Object *obj, Visitor *v, const char *name, + void *opaque, Error **errp) +{ + ARMCPU *cpu = ARM_CPU(obj); + Error *err = NULL; + uint64_t mask = ~(BIT_MASK(ARM_MAX_VQ - 1) - 1); + int i; + + visit_type_uint64(v, name, &cpu->sve_vls_map, errp); + + if (!err) { + if (cpu->sve_vls_map == 0) { + error_setg(&err, "SVE vector length map cannot be zero"); + } else if (cpu->sve_vls_map & mask) { + error_setg(&err, "SVE vector length map has unsupported lengths"); + error_append_hint(&err, "Valid vector lengths in range [1-%d]\n", + ARM_MAX_VQ); + } else if (cpu->sve_max_vq != ARM_MAX_VQ && + cpu->sve_max_vq != arm_cpu_fls64(cpu->sve_vls_map)) { + /* + * If the user provides both sve-max-vq and sve-vls-map, with + * sve-max-vq first, then, unless ARM_MAX_VQ is selected for + * sve-max-vq, we can catch inconsistencies. If ARM_MAX_VQ was + * selected then sve-max-vq will get silently overwritten with + * the max sve-vls-map provides. This is the best we can do + * without initially setting sve-max-vq to -1 like we do for KVM. + */ + error_setg(&err, "sve-vls-map and sve-max-vq are inconsistent"); + } else { + cpu->sve_max_vq = arm_cpu_fls64(cpu->sve_vls_map); + mask = 0; + for (i = 1; i < cpu->sve_max_vq; ++i) { + if (is_power_of_2(i)) { + mask |= BIT_MASK(i - 1); + } + } + if ((cpu->sve_vls_map & mask) != mask) { + error_setg(&err, "SVE vector length map is missing lengths"); + error_append_hint(&err, "All power-of-2 vector lengths up to " + "the max supported length are required.\n"); + } + } + } + + error_propagate(errp, err); +} + /* -cpu max: if KVM is enabled, like -cpu host (best possible with this host); * otherwise, a CPU with as many features enabled as our emulation supports. * The version of '-cpu max' for qemu-system-arm is defined in cpu.c; @@ -374,6 +433,9 @@ static void aarch64_max_initfn(Object *obj) #endif cpu->sve_max_vq = ARM_MAX_VQ; + + object_property_add(obj, "sve-vls-map", "uint64", cpu_get_sve_vls_map, + cpu_set_sve_vls_map, NULL, NULL, &error_fatal); } object_property_add(obj, "sve-max-vq", "uint32", cpu_max_get_sve_vq, diff --git a/target/arm/helper.c b/target/arm/helper.c index 1e6eb0d0f360..bedec1ea0b27 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -5254,12 +5254,20 @@ uint32_t sve_zcr_len_for_el(CPUARMState *env, int el) static void zcr_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value) { + ARMCPU *cpu = arm_env_get_cpu(env); int cur_el = arm_current_el(env); int old_len = sve_zcr_len_for_el(env, cur_el); int new_len; /* Bits other than [3:0] are RAZ/WI. */ - raw_write(env, ri, value & 0xf); + value &= 0xf; + + if (value && !(BIT_MASK(value) & cpu->sve_vls_map)) { + uint64_t map = cpu->sve_vls_map & (BIT_MASK(value) - 1); + value = arm_cpu_fls64(map) - 1; + } + + raw_write(env, ri, value); /* * Because we arrived here, we know both FP and SVE are enabled; diff --git a/target/arm/monitor.c b/target/arm/monitor.c index 3e13dd7c7b7a..192012659e36 100644 --- a/target/arm/monitor.c +++ b/target/arm/monitor.c @@ -133,9 +133,12 @@ static SVEVectorLengths *qmp_sve_vls_get(void) } for (i = 1; i <= cpu->sve_max_vq; ++i) { - *v = g_new0(intList, 1); - (*v)->value = i; - v = &(*v)->next; + int bitval = (cpu->sve_vls_map >> (i - 1)) & 1; + if (bitval) { + *v = g_new0(intList, 1); + (*v)->value = i; + v = &(*v)->next; + } } return vls; From patchwork Sun May 12 08:36:23 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Jones X-Patchwork-Id: 10939937 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3FF9013AD for ; Sun, 12 May 2019 08:43:05 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2F9DB20564 for ; Sun, 12 May 2019 08:43:05 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 23C5F232A7; Sun, 12 May 2019 08:43:05 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 01DC720564 for ; Sun, 12 May 2019 08:43:04 +0000 (UTC) Received: from localhost ([127.0.0.1]:40557 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hPk4V-0000QR-AY for patchwork-qemu-devel@patchwork.kernel.org; Sun, 12 May 2019 04:43:03 -0400 Received: from eggs.gnu.org ([209.51.188.92]:38514) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hPjzZ-0004Mj-L3 for qemu-devel@nongnu.org; Sun, 12 May 2019 04:37:58 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hPjzX-0003nr-E2 for qemu-devel@nongnu.org; Sun, 12 May 2019 04:37:57 -0400 Received: from mx1.redhat.com ([209.132.183.28]:56744) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hPjz9-0003X3-9p; Sun, 12 May 2019 04:37:49 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 91DED3082E4C; Sun, 12 May 2019 08:37:30 +0000 (UTC) Received: from kamzik.brq.redhat.com (ovpn-116-87.ams2.redhat.com [10.36.116.87]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3D26D5D706; Sun, 12 May 2019 08:37:25 +0000 (UTC) From: Andrew Jones To: qemu-devel@nongnu.org, qemu-arm@nongnu.org Date: Sun, 12 May 2019 10:36:23 +0200 Message-Id: <20190512083624.8916-13-drjones@redhat.com> In-Reply-To: <20190512083624.8916-1-drjones@redhat.com> References: <20190512083624.8916-1-drjones@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.46]); Sun, 12 May 2019 08:37:30 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH 12/13] target/arm/kvm: max cpu: Add support for sve-vls-map X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, richard.henderson@linaro.org, armbru@redhat.com, abologna@redhat.com, alex.bennee@linaro.org, Dave.Martin@arm.com Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP The max cpu type can have its SVE vector lengths explicitly set with the sve-vls-map property. This patch allows that property to work when KVM is in use. The map must conform to additional constraints for KVM which are checked at vcpu init. Signed-off-by: Andrew Jones --- target/arm/cpu64.c | 7 +++--- target/arm/kvm64.c | 56 +++++++++++++++++++++++++++++++++++++++++++--- 2 files changed, 56 insertions(+), 7 deletions(-) diff --git a/target/arm/cpu64.c b/target/arm/cpu64.c index 9ac702d54136..94f3dd5b51e5 100644 --- a/target/arm/cpu64.c +++ b/target/arm/cpu64.c @@ -310,7 +310,7 @@ static void cpu_set_sve_vls_map(Object *obj, Visitor *v, const char *name, error_setg(&err, "SVE vector length map has unsupported lengths"); error_append_hint(&err, "Valid vector lengths in range [1-%d]\n", ARM_MAX_VQ); - } else if (cpu->sve_max_vq != ARM_MAX_VQ && + } else if (cpu->sve_max_vq != ARM_MAX_VQ && cpu->sve_max_vq != -1 && cpu->sve_max_vq != arm_cpu_fls64(cpu->sve_vls_map)) { /* * If the user provides both sve-max-vq and sve-vls-map, with @@ -433,13 +433,12 @@ static void aarch64_max_initfn(Object *obj) #endif cpu->sve_max_vq = ARM_MAX_VQ; - - object_property_add(obj, "sve-vls-map", "uint64", cpu_get_sve_vls_map, - cpu_set_sve_vls_map, NULL, NULL, &error_fatal); } object_property_add(obj, "sve-max-vq", "uint32", cpu_max_get_sve_vq, cpu_max_set_sve_vq, NULL, NULL, &error_fatal); + object_property_add(obj, "sve-vls-map", "uint64", cpu_get_sve_vls_map, + cpu_set_sve_vls_map, NULL, NULL, &error_fatal); } struct ARMCPUInfo { diff --git a/target/arm/kvm64.c b/target/arm/kvm64.c index 11c6334a7c08..5506f019c190 100644 --- a/target/arm/kvm64.c +++ b/target/arm/kvm64.c @@ -685,9 +685,9 @@ int kvm_arch_init_vcpu(CPUState *cs) } else { unset_feature(&env->features, ARM_FEATURE_PMU); } - if (cpu->sve_max_vq) { + if (cpu->sve_max_vq || cpu->sve_vls_map) { if (!kvm_check_extension(cs->kvm_state, KVM_CAP_ARM_SVE)) { - if (cpu->sve_max_vq == -1) { + if (cpu->sve_max_vq == -1 && !cpu->sve_vls_map) { cpu->sve_max_vq = 0; } else { error_report("This KVM host does not support SVE"); @@ -704,12 +704,62 @@ int kvm_arch_init_vcpu(CPUState *cs) return ret; } - if (cpu->sve_max_vq) { + if (cpu->sve_max_vq || cpu->sve_vls_map) { uint64_t sve_vls[KVM_ARM64_SVE_VLS_WORDS]; ret = kvm_arm_get_sve_vls(cs, sve_vls); if (ret < 0) { return ret; } + if (cpu->sve_vls_map) { + uint64_t ovls; + int i; + + /* + * We currently only support a single VLS word, as that should + * be sufficient for some time (vq=64 means a 8192-bit vector + * and KVM currently only supports up to 2048-bit vectors). + * The choice to only support a single word for now is due to + * the need to input it on the command line. It's much simpler + * to input a word as a cpu property than an array of words. + * So for now just warn if we detect our assumption was wrong. + */ + for (i = 1; i < KVM_ARM64_SVE_VLS_WORDS; ++i) { + if (sve_vls[i]) { + warn_report("KVM supports vector lengths larger than " + "sve-vls-map can select"); + sve_vls[i] = 0; + } + } + + ovls = sve_vls[0]; + sve_vls[0] = cpu->sve_vls_map; + + if (cpu->sve_vls_map & ~ovls) { + error_report("sve-vls-map=0x%lx is not valid on this host " + "which supports 0x%lx", cpu->sve_vls_map, ovls); + return -EINVAL; + } + + i = arm_cpu_fls64(cpu->sve_vls_map); + if (cpu->sve_max_vq && cpu->sve_max_vq != -1 && + cpu->sve_max_vq != i) { + error_report("sve-vls-map and sve-max-vq are inconsistent"); + return -EINVAL; + } + cpu->sve_max_vq = i; + + /* + * sve-vls-map must have all the same vector lengths up to its + * max vq that the host supports. + */ + if (cpu->sve_vls_map != (ovls & (BIT_MASK(cpu->sve_max_vq) - 1))) { + error_report("sve-vls-map=0x%lx is not valid on this host " + "which supports 0x%lx", cpu->sve_vls_map, ovls); + error_printf("All host vector lengths up to %d must also " + "be selected.\n", cpu->sve_max_vq); + return -EINVAL; + } + } if (cpu->sve_max_vq == -1) { cpu->sve_max_vq = ret; } else if (cpu->sve_max_vq > ret) { From patchwork Sun May 12 08:36:24 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Jones X-Patchwork-Id: 10939935 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B7FEE13AD for ; Sun, 12 May 2019 08:43:02 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9F8EF20564 for ; Sun, 12 May 2019 08:43:02 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 9167D232A7; Sun, 12 May 2019 08:43:02 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 3164C20564 for ; Sun, 12 May 2019 08:43:02 +0000 (UTC) Received: from localhost ([127.0.0.1]:40555 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hPk4T-0000Nn-B9 for patchwork-qemu-devel@patchwork.kernel.org; Sun, 12 May 2019 04:43:01 -0400 Received: from eggs.gnu.org ([209.51.188.92]:38430) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hPjzX-0004Jj-GJ for qemu-devel@nongnu.org; Sun, 12 May 2019 04:37:57 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hPjzV-0003lU-PB for qemu-devel@nongnu.org; Sun, 12 May 2019 04:37:55 -0400 Received: from mx1.redhat.com ([209.132.183.28]:48346) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hPjzE-0003Z4-U4; Sun, 12 May 2019 04:37:49 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 3C29781F13; Sun, 12 May 2019 08:37:36 +0000 (UTC) Received: from kamzik.brq.redhat.com (ovpn-116-87.ams2.redhat.com [10.36.116.87]) by smtp.corp.redhat.com (Postfix) with ESMTP id 571015D706; Sun, 12 May 2019 08:37:30 +0000 (UTC) From: Andrew Jones To: qemu-devel@nongnu.org, qemu-arm@nongnu.org Date: Sun, 12 May 2019 10:36:24 +0200 Message-Id: <20190512083624.8916-14-drjones@redhat.com> In-Reply-To: <20190512083624.8916-1-drjones@redhat.com> References: <20190512083624.8916-1-drjones@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.27]); Sun, 12 May 2019 08:37:36 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH 13/13] target/arm/kvm: host cpu: Add support for sve-vls-map X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, richard.henderson@linaro.org, armbru@redhat.com, abologna@redhat.com, alex.bennee@linaro.org, Dave.Martin@arm.com Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP Allow the host cpu type to enable SVE in guests with the sve-vls-map cpu property. Signed-off-by: Andrew Jones --- target/arm/cpu.c | 1 + target/arm/cpu.h | 2 ++ target/arm/cpu64.c | 12 +++++++++--- 3 files changed, 12 insertions(+), 3 deletions(-) diff --git a/target/arm/cpu.c b/target/arm/cpu.c index ea0e24bba8b6..a5c01ff42c78 100644 --- a/target/arm/cpu.c +++ b/target/arm/cpu.c @@ -2222,6 +2222,7 @@ static void arm_host_initfn(Object *obj) ARMCPU *cpu = ARM_CPU(obj); kvm_arm_set_cpu_features_from_host(cpu); + aarch64_add_sve_vls_map_property(obj); arm_cpu_post_init(obj); } diff --git a/target/arm/cpu.h b/target/arm/cpu.h index f0d0ce759ba8..13731ccb39f3 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -976,11 +976,13 @@ int aarch64_cpu_gdb_write_register(CPUState *cpu, uint8_t *buf, int reg); void aarch64_sve_narrow_vq(CPUARMState *env, unsigned vq); void aarch64_sve_change_el(CPUARMState *env, int old_el, int new_el, bool el0_a64); +void aarch64_add_sve_vls_map_property(Object *obj); #else static inline void aarch64_sve_narrow_vq(CPUARMState *env, unsigned vq) { } static inline void aarch64_sve_change_el(CPUARMState *env, int o, int n, bool a) { } +void aarch64_add_sve_vls_map_property(Object *obj) { } #endif target_ulong do_arm_semihosting(CPUARMState *env); diff --git a/target/arm/cpu64.c b/target/arm/cpu64.c index 94f3dd5b51e5..3b0b900a4d97 100644 --- a/target/arm/cpu64.c +++ b/target/arm/cpu64.c @@ -310,7 +310,8 @@ static void cpu_set_sve_vls_map(Object *obj, Visitor *v, const char *name, error_setg(&err, "SVE vector length map has unsupported lengths"); error_append_hint(&err, "Valid vector lengths in range [1-%d]\n", ARM_MAX_VQ); - } else if (cpu->sve_max_vq != ARM_MAX_VQ && cpu->sve_max_vq != -1 && + } else if (cpu->sve_max_vq && cpu->sve_max_vq != ARM_MAX_VQ && + cpu->sve_max_vq != -1 && cpu->sve_max_vq != arm_cpu_fls64(cpu->sve_vls_map)) { /* * If the user provides both sve-max-vq and sve-vls-map, with @@ -340,6 +341,12 @@ static void cpu_set_sve_vls_map(Object *obj, Visitor *v, const char *name, error_propagate(errp, err); } +void aarch64_add_sve_vls_map_property(Object *obj) +{ + object_property_add(obj, "sve-vls-map", "uint64", cpu_get_sve_vls_map, + cpu_set_sve_vls_map, NULL, NULL, &error_fatal); +} + /* -cpu max: if KVM is enabled, like -cpu host (best possible with this host); * otherwise, a CPU with as many features enabled as our emulation supports. * The version of '-cpu max' for qemu-system-arm is defined in cpu.c; @@ -437,8 +444,7 @@ static void aarch64_max_initfn(Object *obj) object_property_add(obj, "sve-max-vq", "uint32", cpu_max_get_sve_vq, cpu_max_set_sve_vq, NULL, NULL, &error_fatal); - object_property_add(obj, "sve-vls-map", "uint64", cpu_get_sve_vls_map, - cpu_set_sve_vls_map, NULL, NULL, &error_fatal); + aarch64_add_sve_vls_map_property(obj); } struct ARMCPUInfo {