From patchwork Thu Jun 6 16:44:54 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Martin X-Patchwork-Id: 10980079 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id ABE0414C0 for ; Thu, 6 Jun 2019 16:45:53 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 95F0D28998 for ; Thu, 6 Jun 2019 16:45:53 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 86301289C5; Thu, 6 Jun 2019 16:45:53 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 2299C28998 for ; Thu, 6 Jun 2019 16:45:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=Oq32T1HayDBrjkoqyQIxBWw3cJShco4c6ssF/fcMmWk=; b=Sk5qAIVjM6GmH4kwXsVkhIEtss /ZLruTpa4XOHstnNalVO5hp6/LpqtyQ48Q9NMIGwITVDLvaba7iIu+KCSGO8u80NeKZG+4SrIPrbI NGc4OovCmdqWV6f6bLHWX8xO+qk0/gMrrXzVVpHqGPyMlSFKvY1zHulX4YzbYoRVdo7xTJOQCNkuW qKXMsaYzC3vuOT2Wh680AmR4cQklwDfFgiLwe2vZppx5mlFxbkH5+TvRyScgT71Uja1ki0WFTUdAI fQPSkGX04BvmQtV/sYc38L2bE5hdOt+DexFsrNCw+mku9Yoe2v12lvfYTSVw5Y4WzGw8mK/1VmKC3 xdgMAjwg==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1hYvWN-000886-Q7; Thu, 06 Jun 2019 16:45:47 +0000 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70] helo=foss.arm.com) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1hYvWL-00087V-2U for linux-arm-kernel@lists.infradead.org; Thu, 06 Jun 2019 16:45:46 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 9B1D9374; Thu, 6 Jun 2019 09:45:44 -0700 (PDT) Received: from e103592.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 446B03F690; Thu, 6 Jun 2019 09:45:43 -0700 (PDT) From: Dave Martin To: linux-arm-kernel@lists.infradead.org Subject: [PATCH 1/2] arm64/sve: Factor out FPSIMD to SVE state conversion Date: Thu, 6 Jun 2019 17:44:54 +0100 Message-Id: <1559839495-22315-2-git-send-email-Dave.Martin@arm.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1559839495-22315-1-git-send-email-Dave.Martin@arm.com> References: <1559839495-22315-1-git-send-email-Dave.Martin@arm.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20190606_094545_114398_EC31BE4B X-CRM114-Status: GOOD ( 14.31 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Peter Maydell , gdb@sourceware.org, Will Deacon , Zhang Lei , Julien Grall , Catalin Marinas , =?utf-8?q?Alex_Benn=C3=A9e?= MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP Currently we convert from FPSIMD to SVE register state in memory in two places. We are about to amend the way this works, so factor this operation out so that subsequent changes only have to be made in one place. Signed-off-by: Dave Martin --- arch/arm64/kernel/fpsimd.c | 21 ++++++++++++--------- 1 file changed, 12 insertions(+), 9 deletions(-) diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c index a38bf74..61ceeb9 100644 --- a/arch/arm64/kernel/fpsimd.c +++ b/arch/arm64/kernel/fpsimd.c @@ -352,6 +352,16 @@ static int __init sve_sysctl_init(void) { return 0; } #define ZREG(sve_state, vq, n) ((char *)(sve_state) + \ (SVE_SIG_ZREG_OFFSET(vq, n) - SVE_SIG_REGS_OFFSET)) +static void __fpsimd_to_sve(void *sst, struct user_fpsimd_state const *fst, + unsigned int vq) +{ + unsigned int i; + + for (i = 0; i < 32; ++i) + memcpy(ZREG(sst, vq, i), &fst->vregs[i], + sizeof(fst->vregs[i])); +} + /* * Transfer the FPSIMD state in task->thread.uw.fpsimd_state to * task->thread.sve_state. @@ -368,15 +378,12 @@ static void fpsimd_to_sve(struct task_struct *task) unsigned int vq; void *sst = task->thread.sve_state; struct user_fpsimd_state const *fst = &task->thread.uw.fpsimd_state; - unsigned int i; if (!system_supports_sve()) return; vq = sve_vq_from_vl(task->thread.sve_vl); - for (i = 0; i < 32; ++i) - memcpy(ZREG(sst, vq, i), &fst->vregs[i], - sizeof(fst->vregs[i])); + __fpsimd_to_sve(sst, fst, vq); } /* @@ -490,7 +497,6 @@ void sve_sync_from_fpsimd_zeropad(struct task_struct *task) unsigned int vq; void *sst = task->thread.sve_state; struct user_fpsimd_state const *fst = &task->thread.uw.fpsimd_state; - unsigned int i; if (!test_tsk_thread_flag(task, TIF_SVE)) return; @@ -498,10 +504,7 @@ void sve_sync_from_fpsimd_zeropad(struct task_struct *task) vq = sve_vq_from_vl(task->thread.sve_vl); memset(sst, 0, SVE_SIG_REGS_SIZE(vq)); - - for (i = 0; i < 32; ++i) - memcpy(ZREG(sst, vq, i), &fst->vregs[i], - sizeof(fst->vregs[i])); + __fpsimd_to_sve(sst, fst, vq); } int sve_set_vector_length(struct task_struct *task, From patchwork Thu Jun 6 16:44:55 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Martin X-Patchwork-Id: 10980081 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7D5DF924 for ; Thu, 6 Jun 2019 16:45:59 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 660E028998 for ; Thu, 6 Jun 2019 16:45:59 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 57806289C5; Thu, 6 Jun 2019 16:45:59 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 98EA728998 for ; Thu, 6 Jun 2019 16:45:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=1R45fw6Cv7jQ6sVXQDtxbRZG5fl1oFeTRelbDPRAbyg=; b=n7U+x1GPE0SYqw4CuXko9Uu3/S TOMcPUdaxFzbRgK3YKSJwRW6YTAJifXZLB7qp+v+GkkAMIe+iT1WOcrAozA8Hynx+X8zg1WCGH31a MCCfAH8wz9TdacNNMkI+ZK3//qUrqlV6OjnOfyOT3pS6BWSngeiqT2Ol7jaAokyIyq6r3WFkUUaf2 EzcBpSeyA7mGwKinHD4qKhEwgCNZiP2fkghydoWjs+NGZszFlq4dqm3+bUJoLSwSB68WHGf6c8OuY XwC+go+qqA6ITABZyMHxr1ME3ReuraNLa/6M1o+L5XqS4LLM2gHvGwwqhdpr7CoKHAE5IPjTaCvTv GdNE5igQ==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1hYvWT-0008Ev-BX; Thu, 06 Jun 2019 16:45:53 +0000 Received: from foss.arm.com ([217.140.101.70]) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1hYvWM-00087n-Bv for linux-arm-kernel@lists.infradead.org; Thu, 06 Jun 2019 16:45:48 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 4129E15A2; Thu, 6 Jun 2019 09:45:46 -0700 (PDT) Received: from e103592.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id D8F163F690; Thu, 6 Jun 2019 09:45:44 -0700 (PDT) From: Dave Martin To: linux-arm-kernel@lists.infradead.org Subject: [PATCH 2/2] arm64/sve: Fix missing SVE/FPSIMD endianness conversions Date: Thu, 6 Jun 2019 17:44:55 +0100 Message-Id: <1559839495-22315-3-git-send-email-Dave.Martin@arm.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1559839495-22315-1-git-send-email-Dave.Martin@arm.com> References: <1559839495-22315-1-git-send-email-Dave.Martin@arm.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20190606_094546_411927_92725629 X-CRM114-Status: GOOD ( 23.43 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Peter Maydell , gdb@sourceware.org, Will Deacon , Zhang Lei , Julien Grall , Catalin Marinas , =?utf-8?q?Alex_Benn=C3=A9e?= MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP The in-memory representation of SVE and FPSIMD registers is different: the FPSIMD V-registers are stored as single 128-bit host-endian values, whereas SVE registers are stored in an endianness-invariant byte order. This means that the two representations differ when running on a big-endian host. But we blindly copy data from one representation to another when converting between the two, resulting in the register contents being unintentionally byteswapped in certain situations. Currently this can be triggered by the first SVE instruction after a syscall, for example (though the potential trigger points may vary in future). So, fix the conversion functions __fpsimd_to_sve() and sve_to_fpsimd() to swab where appropriate. There is no common swahl128() or swab128() that we could use here. Maybe it would be worth making this generic, but for now add a simple local hack. Since the byte order differences are exposed in ABI, also clarify the docuentation. Fixes: bc0ee4760364 ("arm64/sve: Core task context handling") Fixes: 8cd969d28fd2 ("arm64/sve: Signal handling support") Fixes: 43d4da2c45b2 ("arm64/sve: ptrace and ELF coredump support") Signed-off-by: Dave Martin --- The ptrace change is theoretically an ABI break, but since the current behaviour is obviously wrong, I consider this a fix. Tested on the Arm Fast Model, using: * asm code that mixes SVE instructions and syscalls; * ptrace interactions that mix the SVE_PT_REGS_FPSIMD and SVE_PT_REGS_SVE views of NT_ARM_SVE. Signal frame behaviour not directly tested (since the underlying conversion functions are the same in all cases). Demonstrator code, on a big-endian platform: .arch_extension sve index z0.b, #0, #1 str v0, [x0] // x0 -> 0f 0e 0d 0c 0b 0a 09 08 07 06 05 04 03 02 01 00 mov w8, #__NR_getpid svc #0 // any noop syscall, reverts to non-SVE regs str v0, [x0] // x0 -> 0f 0e 0d 0c 0b 0a 09 08 07 06 05 04 03 02 01 00 mov z0.d, z0.d // triggers an SVE trap and buggy FPSIMD->SVE conversion str v0, [x0] // x0 -> 00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f While a "cool feature", this is not what was intended. --- Documentation/arm64/sve.txt | 16 +++++++++++++++ arch/arm64/include/uapi/asm/kvm.h | 7 +++++++ arch/arm64/include/uapi/asm/ptrace.h | 4 ++++ arch/arm64/include/uapi/asm/sigcontext.h | 14 +++++++++++++ arch/arm64/kernel/fpsimd.c | 34 ++++++++++++++++++++++++++------ 5 files changed, 69 insertions(+), 6 deletions(-) diff --git a/Documentation/arm64/sve.txt b/Documentation/arm64/sve.txt index 9940e92..6c0bed3 100644 --- a/Documentation/arm64/sve.txt +++ b/Documentation/arm64/sve.txt @@ -56,6 +56,18 @@ model features for SVE is included in Appendix A. is to connect to a target process first and then attempt a ptrace(PTRACE_GETREGSET, pid, NT_ARM_SVE, &iov). +* Whenever SVE scalable register values (Zn, Pn, FFR) are exchanged in memory + between userspace and the kernel, the register value is encoded in memory in + an endianness-invariant layout, with bits [(8 * i + 7) : (8 * i)] encoded at + byte offset i in from the start of the memory representation. This affects + for example the signal frame (struct sve_context) and ptrace interface + (struct user_sve_header) and associated data. + + Beware that on big-endian systems this results in a different byte order than + for the FPSIMD V-registers, which are stored as single host-endian 128-bit + values, with bits [(127 - 8 * i) : (120 - 8 * i)] of the register encoded at + byte offset i. (struct fpsimd_context, struct user_fpsimd_state). + 2. Vector length terminology ----------------------------- @@ -124,6 +136,10 @@ the SVE instruction set architecture. size and layout. Macros SVE_SIG_* are defined [1] to facilitate access to the members. +* Each scalable register (Zn, Pn, FFR) is stored in an endianness-invariant + layout, with bits [(8 * i + 7) : (8 * i)] stored at byte offset i from the + start of the register's representation in memory. + * If the SVE context is too big to fit in sigcontext.__reserved[], then extra space is allocated on the stack, an extra_context record is written in __reserved[] referencing this space. sve_context is then written in the diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h index 7b7ac0f..072ea1e 100644 --- a/arch/arm64/include/uapi/asm/kvm.h +++ b/arch/arm64/include/uapi/asm/kvm.h @@ -260,6 +260,13 @@ struct kvm_vcpu_events { KVM_REG_SIZE_U256 | \ ((i) & (KVM_ARM64_SVE_MAX_SLICES - 1))) +/* + * Register values for KVM_REG_ARM64_SVE_ZREG(), KVM_REG_ARM64_SVE_PREG() and + * KVM_REG_ARM64_SVE_FFR() and represented in memory in an endianness- + * invariant layout which differs from the layout used for the FPSIMD + * V-registers on big-endian systems: see sigcontext.h for more explanaion. + */ + #define KVM_ARM64_SVE_VQ_MIN __SVE_VQ_MIN #define KVM_ARM64_SVE_VQ_MAX __SVE_VQ_MAX diff --git a/arch/arm64/include/uapi/asm/ptrace.h b/arch/arm64/include/uapi/asm/ptrace.h index d78623a..018bf7d 100644 --- a/arch/arm64/include/uapi/asm/ptrace.h +++ b/arch/arm64/include/uapi/asm/ptrace.h @@ -176,6 +176,10 @@ struct user_sve_header { * FPCR uint32_t FPCR * * Additional data might be appended in the future. + * + * The Z-, P- and FFR registers and represented in memory in an endianness- + * invariant layout which differs from the layout used for the FPSIMD + * V-registers on big-endian systems: see sigcontext.h for more explanaion. */ #define SVE_PT_SVE_ZREG_SIZE(vq) __SVE_ZREG_SIZE(vq) diff --git a/arch/arm64/include/uapi/asm/sigcontext.h b/arch/arm64/include/uapi/asm/sigcontext.h index 5f3c0ce..5231569 100644 --- a/arch/arm64/include/uapi/asm/sigcontext.h +++ b/arch/arm64/include/uapi/asm/sigcontext.h @@ -77,6 +77,15 @@ struct fpsimd_context { __uint128_t vregs[32]; }; +/* + * Note: similarly to all other integer fields, each V-register is stored in an + * endianness-dependent format, with the byte at offset i from the start of the + * in-memory representation of the register value containing + * + * bits [(7 + 8 * i) : (8 * i)] of the register on little-endian hosts; or + * bits [(127 - 8 * i) : (120 - 8 * i)] on big-endian hosts. + */ + /* ESR_EL1 context */ #define ESR_MAGIC 0x45535201 @@ -204,6 +213,11 @@ struct sve_context { * FFR uint16_t[vq] first-fault status register * * Additional data might be appended in the future. + * + * Unlike vregs[] in fpsimd_context, each SVE scalable register (Z-, P- or FFR) + * is encoded in memory an endianness-invariant format, with the byte at offset + * i from the start of the in-memory representation containing bits + * [(7 + 8 * i) : (8 * i)] of the register value. */ #define SVE_SIG_ZREG_SIZE(vq) __SVE_ZREG_SIZE(vq) diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c index 61ceeb9..d2f7544 100644 --- a/arch/arm64/kernel/fpsimd.c +++ b/arch/arm64/kernel/fpsimd.c @@ -39,6 +39,7 @@ #include #include #include +#include #include #include @@ -352,14 +353,33 @@ static int __init sve_sysctl_init(void) { return 0; } #define ZREG(sve_state, vq, n) ((char *)(sve_state) + \ (SVE_SIG_ZREG_OFFSET(vq, n) - SVE_SIG_REGS_OFFSET)) +#ifdef CONFIG_CPU_BIG_ENDIAN +static __uint128_t arm64_cpu_to_le128(__uint128_t x) +{ + u64 a = swab64(x); + u64 b = swab64(x >> 64); + + return ((__uint128_t)a << 64) | b; +} +#else +static __uint128_t arm64_cpu_to_le128(__uint128_t x) +{ + return x; +} +#endif + +#define arm64_le128_to_cpu(x) arm64_cpu_to_le128(x) + static void __fpsimd_to_sve(void *sst, struct user_fpsimd_state const *fst, unsigned int vq) { unsigned int i; + __uint128_t *p; - for (i = 0; i < 32; ++i) - memcpy(ZREG(sst, vq, i), &fst->vregs[i], - sizeof(fst->vregs[i])); + for (i = 0; i < 32; ++i) { + p = (__uint128_t *)ZREG(sst, vq, i); + *p = arm64_cpu_to_le128(fst->vregs[i]); + } } /* @@ -402,14 +422,16 @@ static void sve_to_fpsimd(struct task_struct *task) void const *sst = task->thread.sve_state; struct user_fpsimd_state *fst = &task->thread.uw.fpsimd_state; unsigned int i; + __uint128_t const *p; if (!system_supports_sve()) return; vq = sve_vq_from_vl(task->thread.sve_vl); - for (i = 0; i < 32; ++i) - memcpy(&fst->vregs[i], ZREG(sst, vq, i), - sizeof(fst->vregs[i])); + for (i = 0; i < 32; ++i) { + p = (__uint128_t const *)ZREG(sst, vq, i); + fst->vregs[i] = arm64_le128_to_cpu(*p); + } } #ifdef CONFIG_ARM64_SVE