From patchwork Thu Mar 16 08:20:04 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 13177171 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 26A13C6FD19 for ; Thu, 16 Mar 2023 08:21:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=UbcArv01dMNyziROop+WehQtsSnBXE9bPU5JcBdoE/k=; b=w8Q3rXdVOu4f8X Kz0GMpdj6BWYv5S4/KkHBU3Xx/pIVOsVlrOnrPbszT+gabZnfut1d6Z1zITVb90sWkMc5G6+u6Yz8 +sqIjd+KXHS4t8WdCYZbSGMpvbXLxwyNHOh3RUwVKFo5wRivcWb2CN1YoUfnVYZlfWX1rJIbRoMCd ux9E+2VEIjFjMtBSheUkMosQ/pZ2mv9F6rYkxu0c7QUFgG9J+XgQyA2oxmQC5DtVgdiQSLp/gxwoE uLjw+O3oTJGX3z8xESYM/b6iazSnqzpFNE0zDRjMGR4fZeSwf5KPZcEmHNej4aL3vaFc5JZ3RJx/z jG8aocB+olIapDH4x66A==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1pciqd-00Fdu9-0h; Thu, 16 Mar 2023 08:20:31 +0000 Received: from ams.source.kernel.org ([2604:1380:4601:e00::1]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1pciqV-00FdpK-33 for linux-arm-kernel@lists.infradead.org; Thu, 16 Mar 2023 08:20:25 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 96FD6B82042; Thu, 16 Mar 2023 08:20:22 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9E40FC433A0; Thu, 16 Mar 2023 08:20:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1678954821; bh=ApwpSuQ/0LynuxzoaJbNQOuNWmQBjXXMoHzayRWE/TI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=opCHFJVO4EZqeomRSDS24Tk4pwQVg8s9HMZZsqx+ilxQjk6eN2EQWRw56Px5SVpDm zYn8V3RcO0FqWt6a8BpVK9Iowds/y4FrfJ4abCtp9MXa7zMueOJuz09N5IyUsQNr1j fwePeJGOvQtb0m5Ugi3QmWdftnm0fbdnEvnw7sD4L3QbJWYgK2+uaG7VmGWRHQrWA5 FNfF/F2MT42cuOCCGXJx4FW7Wy883q3rwsqpAjEihueZ11OkOstF7f/v5g8GWwE6nI gGFIbz1sPTOlVAr2qMopyBVlp7NYKju84155tEzn47quYAOfdCr3pceO8rK4yUyRdY aH1cdhzqOtodw== From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org, linux@armlinux.org.uk Cc: Ard Biesheuvel , Frederic Weisbecker , Guenter Roeck , Peter Zijlstra , Linus Walleij , Arnd Bergmann Subject: [PATCH v3 1/4] ARM: vfp: Pass thread_info pointer to vfp_support_entry Date: Thu, 16 Mar 2023 09:20:04 +0100 Message-Id: <20230316082007.652669-2-ardb@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230316082007.652669-1-ardb@kernel.org> References: <20230316082007.652669-1-ardb@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=2469; i=ardb@kernel.org; h=from:subject; bh=ApwpSuQ/0LynuxzoaJbNQOuNWmQBjXXMoHzayRWE/TI=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIUXoou7bJWKK6zP0VUKy1vNZSBpPiWC4I2Vr4WZtsjj4Y //222IdpSwMYhwMsmKKLAKz/77beXqiVK3zLFmYOaxMIEMYuDgFYCKvLzEyfOQxN3uSfuDen7nv i79a311Qejd6vdjDvrD1sr9LhCRbYxgZFnP+Oh1XfKT2T9ee811SaxzDfxu7P/lZt3+NhpR+yL8 TTAA= X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230316_012024_280875_C50F264C X-CRM114-Status: GOOD ( 15.41 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Instead of dereferencing thread_info in do_vfp, pass the thread_info pointer to vfp_support_entry via R1. That way, we only use a single caller save register, which makes it easier to convert do_vfp to C code in a subsequent patch. Note that, unlike the CPU number, which can change due to preemption, passing the thread_info pointer can safely be done with preemption enabled. Signed-off-by: Ard Biesheuvel Reviewed-by: Linus Walleij Tested-by: Guenter Roeck --- arch/arm/vfp/entry.S | 5 +---- arch/arm/vfp/vfphw.S | 10 +++++++--- 2 files changed, 8 insertions(+), 7 deletions(-) diff --git a/arch/arm/vfp/entry.S b/arch/arm/vfp/entry.S index 9a89264cdcc0b46e..cfedc2a3dbd68f1c 100644 --- a/arch/arm/vfp/entry.S +++ b/arch/arm/vfp/entry.S @@ -22,15 +22,12 @@ @ IRQs enabled. @ ENTRY(do_vfp) - local_bh_disable r10, r4 + mov r1, r10 ldr r4, .LCvfp - ldr r11, [r10, #TI_CPU] @ CPU number - add r10, r10, #TI_VFPSTATE @ r10 = workspace ldr pc, [r4] @ call VFP entry point ENDPROC(do_vfp) ENTRY(vfp_null_entry) - local_bh_enable_ti r10, r4 ret lr ENDPROC(vfp_null_entry) diff --git a/arch/arm/vfp/vfphw.S b/arch/arm/vfp/vfphw.S index 26c4f61ecfa39638..6d056d810e4868c2 100644 --- a/arch/arm/vfp/vfphw.S +++ b/arch/arm/vfp/vfphw.S @@ -6,9 +6,9 @@ * Written by Deep Blue Solutions Limited. * * This code is called from the kernel's undefined instruction trap. + * r1 holds the thread_info pointer * r9 holds the return address for successful handling. * lr holds the return address for unrecognised instructions. - * r10 points at the start of the private FP workspace in the thread structure * sp points to a struct pt_regs (as defined in include/asm/proc/ptrace.h) */ #include @@ -69,13 +69,17 @@ @ VFP hardware support entry point. @ @ r0 = instruction opcode (32-bit ARM or two 16-bit Thumb) +@ r1 = thread_info pointer @ r2 = PC value to resume execution after successful emulation @ r9 = normal "successful" return address -@ r10 = vfp_state union -@ r11 = CPU number @ lr = unrecognised instruction return address @ IRQs enabled. ENTRY(vfp_support_entry) + local_bh_disable r1, r4 + + ldr r11, [r1, #TI_CPU] @ CPU number + add r10, r1, #TI_VFPSTATE @ r10 = workspace + DBGSTR3 "instr %08x pc %08x state %p", r0, r2, r10 .fpu vfpv2 From patchwork Thu Mar 16 08:20:05 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 13177174 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 56C08C6FD19 for ; Thu, 16 Mar 2023 08:21:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=wmmhy5GpdnljoGuUHoYeYcZlfvmCzw8n7cc15UYvXXI=; b=Lv8RfD8ynEnHRf f9VzUQCX9mQL3IpmH2oYX5gqdGlIJYjJGoQye0NagpozSD5DudD3EKjF0oM+pfQ6hujTRQE1b2l70 UtVlPN2YbB92Q386PxFCbxEfVHJihliOz4nylruW5ZT8J8sX+2hApFF3egFc806/M72rsvOeUwOS0 b021J6tbhyXxqmsLK7IEYMrx+Qvgp1Xx/jlFUI6AiaXOf7Es4fRWaNfT4ewRYvcUfOM+M5tIWylfJ U7cGGoe2MiueQc/ufGafOkPQaZRE/beE6h4o9moy6tW4GdfGiPXfyx1DSmmaF0TwVFWIDftEomjFi xqFka5gZbg+IlxfyHpJw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1pciqe-00Fdur-1M; Thu, 16 Mar 2023 08:20:32 +0000 Received: from dfw.source.kernel.org ([139.178.84.217]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1pciqW-00Fdpc-1U for linux-arm-kernel@lists.infradead.org; Thu, 16 Mar 2023 08:20:27 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id EA5C161F8B; Thu, 16 Mar 2023 08:20:23 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id AE42FC433A7; Thu, 16 Mar 2023 08:20:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1678954823; bh=cvW3K10cwPJIDxWFtbNkAbwAJgpniljdex/6gIjaLzQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Sme4xhwIhuddyqTUM6H9z5Z9xVUmn3ZW30iYaBUmr+Hfz3PL6snbxziiMuS1Kgea0 91FMoJ7bdsx7TVn/6cIpl4k3gCqtfza0emrifdAm2WBA/DHiOMf6lhiJNEGLJTslup o3t6Uw3rVFDpoLZWCop1yql6KnnmZh31+QZ+qpqXvjlmiWc7PhMsW0YwzIguflWf4y 5rWl18qD6Ryw5TWlLDqi6JLSMcjzPAOwjtkQylcqTeOnvak5ZYDHCWfs51ekYi5zWB muAZ6MI9whruiKDLTJGht4z/TO/qEHuqAsVMvFEylh+Xbm7fNmxY4LG7643XTjLnIT ITU3RSplBoQXw== From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org, linux@armlinux.org.uk Cc: Ard Biesheuvel , Frederic Weisbecker , Guenter Roeck , Peter Zijlstra , Linus Walleij , Arnd Bergmann Subject: [PATCH v3 2/4] ARM: vfp: Pass successful return address via register R3 Date: Thu, 16 Mar 2023 09:20:05 +0100 Message-Id: <20230316082007.652669-3-ardb@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230316082007.652669-1-ardb@kernel.org> References: <20230316082007.652669-1-ardb@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=3246; i=ardb@kernel.org; h=from:subject; bh=cvW3K10cwPJIDxWFtbNkAbwAJgpniljdex/6gIjaLzQ=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIUXoor6Zdvb/Dzk6xppLNwYsfWwnprdVf8Oq73KZifLXH TuNnDI7SlkYxDgYZMUUWQRm/3238/REqVrnWbIwc1iZQIYwcHEKwEScoxkZNj+Xijvxw75Xr/jP ZP135tPcfnR5pbw5oeF/NUY3KDnGhJHhxYHpnGoO8QzP2k6INF3ZsrN0m9Ep30sXFl8+cjmN8WY SOwA= X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230316_012024_581416_CD566484 X-CRM114-Status: GOOD ( 17.54 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org In preparation for reimplementing the do_vfp()->vfp_support_entry() handover in C code, switch to using R3 to pass the 'success' return address, rather than R9, as it cannot be used for parameter passing. Signed-off-by: Ard Biesheuvel Reviewed-by: Linus Walleij Tested-by: Guenter Roeck --- arch/arm/vfp/entry.S | 1 + arch/arm/vfp/vfphw.S | 14 +++++++------- 2 files changed, 8 insertions(+), 7 deletions(-) diff --git a/arch/arm/vfp/entry.S b/arch/arm/vfp/entry.S index cfedc2a3dbd68f1c..6dabb47617781a5f 100644 --- a/arch/arm/vfp/entry.S +++ b/arch/arm/vfp/entry.S @@ -23,6 +23,7 @@ @ ENTRY(do_vfp) mov r1, r10 + mov r3, r9 ldr r4, .LCvfp ldr pc, [r4] @ call VFP entry point ENDPROC(do_vfp) diff --git a/arch/arm/vfp/vfphw.S b/arch/arm/vfp/vfphw.S index 6d056d810e4868c2..60acd42e05786e95 100644 --- a/arch/arm/vfp/vfphw.S +++ b/arch/arm/vfp/vfphw.S @@ -7,7 +7,7 @@ * * This code is called from the kernel's undefined instruction trap. * r1 holds the thread_info pointer - * r9 holds the return address for successful handling. + * r3 holds the return address for successful handling. * lr holds the return address for unrecognised instructions. * sp points to a struct pt_regs (as defined in include/asm/proc/ptrace.h) */ @@ -71,7 +71,7 @@ @ r0 = instruction opcode (32-bit ARM or two 16-bit Thumb) @ r1 = thread_info pointer @ r2 = PC value to resume execution after successful emulation -@ r9 = normal "successful" return address +@ r3 = normal "successful" return address @ lr = unrecognised instruction return address @ IRQs enabled. ENTRY(vfp_support_entry) @@ -89,9 +89,9 @@ ENTRY(vfp_support_entry) bne look_for_VFP_exceptions @ VFP is already enabled DBGSTR1 "enable %x", r10 - ldr r3, vfp_current_hw_state_address + ldr r9, vfp_current_hw_state_address orr r1, r1, #FPEXC_EN @ user FPEXC has the enable bit set - ldr r4, [r3, r11, lsl #2] @ vfp_current_hw_state pointer + ldr r4, [r9, r11, lsl #2] @ vfp_current_hw_state pointer bic r5, r1, #FPEXC_EX @ make sure exceptions are disabled cmp r4, r10 @ this thread owns the hw context? #ifndef CONFIG_SMP @@ -150,7 +150,7 @@ vfp_reload_hw: #endif DBGSTR1 "load state %p", r10 - str r10, [r3, r11, lsl #2] @ update the vfp_current_hw_state pointer + str r10, [r9, r11, lsl #2] @ update the vfp_current_hw_state pointer @ Load the saved state back into the VFP VFPFLDMIA r10, r5 @ reload the working registers while @ FPEXC is in a safe state @@ -180,7 +180,7 @@ vfp_hw_state_valid: @ always subtract 4 from the following @ instruction address. local_bh_enable_ti r10, r4 - ret r9 @ we think we have handled things + ret r3 @ we think we have handled things look_for_VFP_exceptions: @@ -210,7 +210,7 @@ skip: process_exception: DBGSTR "bounce" mov r2, sp @ nothing stacked - regdump is at TOS - mov lr, r9 @ setup for a return to the user code. + mov lr, r3 @ setup for a return to the user code. @ Now call the C code to package up the bounce to the support code @ r0 holds the trigger instruction From patchwork Thu Mar 16 08:20:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 13177173 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 2B277C6FD19 for ; Thu, 16 Mar 2023 08:21:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=hVMKph7mNYYy8y2QpYOVGif/N+bvNk+ICQZiDK2+ylc=; b=dGFySItvhw4tUw OLN4TEqOOVe5YpH6bZ1JXTJt1tO9nMKFZmc95xxZ4KVL6CbOIoUN9sWVINVw0FqHifSR6MZ5E/0MR d3HbbHGiHxKwyFxqIoiiKVvOVmyZE2UfNsyQ83A1IBIpvq6UZ+yeehl4/oiNG2hw7L4BLv4f22eYW ltngmVtDeGODRuDT5ZZVXtSiVP0PxWIf0mUaClsX+2sRpRlhjcFTSaC1bJEMikmo/wojk2eTKrfMm 8z8D+QwVTB6hT3ndr9zRKVK9/Wl1RS4uXhc0i8rSvVFguBsOStyHCb1+H7hOGEUre+BZa6DRqyBRD kwek590Yi0GqiDMOJ3dg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1pciqf-00FdvU-1s; Thu, 16 Mar 2023 08:20:33 +0000 Received: from dfw.source.kernel.org ([139.178.84.217]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1pciqY-00FdrE-1c for linux-arm-kernel@lists.infradead.org; Thu, 16 Mar 2023 08:20:29 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 02EAD61F57; Thu, 16 Mar 2023 08:20:26 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id BCE5FC4339C; Thu, 16 Mar 2023 08:20:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1678954825; bh=XLUkRu1FO02a8Cpw0ONEzkZtQNXoswbOq5+Dq9VRjWw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=RbI0KF0Uf/+cGcSg0c47PzvIiZFnllk+rQ8gsZjvxxqo69iDGuEXcmQ70yRMMyFJT hM1s19w98u0f/+CrDz9PhtoRsoLCZtyn23PRNPGcr2vbT9TAWCjWDOBNdBN4LMAZ6l BOtXy/inNO4isWxaqG8JH5X7lwt73CiUONLrJBfyOXHIk6njO7E3OO+iOBNf1uWG05 pLEX9eAOBJzRme9IZGsCdFAZVsZXVu8yvfxN4sHQuOmx1F5OywL+Ng5FKrL5VMoNgX DfMPGrPHv3TcdKtIHo5PHhZJ+ti0UVr2+U5DSnU8wL7BJGJJKv3GGCjdpZ4m6iS9CQ 9DbxJDXRd4Maw== From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org, linux@armlinux.org.uk Cc: Ard Biesheuvel , Frederic Weisbecker , Guenter Roeck , Peter Zijlstra , Linus Walleij , Arnd Bergmann Subject: [PATCH v3 3/4] ARM: vfp: Fix broken softirq handling with instrumentation enabled Date: Thu, 16 Mar 2023 09:20:06 +0100 Message-Id: <20230316082007.652669-4-ardb@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230316082007.652669-1-ardb@kernel.org> References: <20230316082007.652669-1-ardb@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=5298; i=ardb@kernel.org; h=from:subject; bh=XLUkRu1FO02a8Cpw0ONEzkZtQNXoswbOq5+Dq9VRjWw=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIUXootH3mQLK8yazHuP/MdtT0OS3jqOnuMIq+fvTmJbcW b2y19uso5SFQYyDQVZMkUVg9t93O09PlKp1niULM4eVCWQIAxenAExkmRAjw9lznL6HJ7HWnGO7 oBZ7LXZ3V9erXOMqZd13HObmzNd60hkZXji//XFDvfD91CUfVP+scXNNmr/S/RFblKjCqZN6lbX anAA= X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230316_012026_656979_3B358F76 X-CRM114-Status: GOOD ( 17.18 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Commit 62b95a7b44d1 ("ARM: 9282/1: vfp: Manipulate task VFP state with softirqs disabled") replaced the en/disable preemption calls inside the VFP state handling code with en/disabling of soft IRQs, which is necessary to allow kernel use of the VFP/SIMD unit when handling a soft IRQ. Unfortunately, when lockdep is enabled (or other instrumentation that enables TRACE_IRQFLAGS), the disable path implemented in asm fails to perform the lockdep and RCU related bookkeeping, resulting in spurious warnings and other badness. Set let's rework the VFP entry code a little bit so we can make the local_bh_disable() call from C, with all the instrumentations that happen to have been configured. Calling local_bh_enable() can be done from asm, as it is a simple wrapper around __local_bh_enable_ip(), which is always a callable function. Link: https://lore.kernel.org/all/ZBBYCSZUJOWBg1s8@localhost.localdomain/ Fixes: 62b95a7b44d1 ("ARM: 9282/1: vfp: Manipulate task VFP state with softirqs disabled") Signed-off-by: Ard Biesheuvel Reviewed-by: Linus Walleij Tested-by: Guenter Roeck --- arch/arm/include/asm/assembler.h | 13 ---------- arch/arm/vfp/entry.S | 11 +------- arch/arm/vfp/vfphw.S | 12 ++++----- arch/arm/vfp/vfpmodule.c | 27 ++++++++++++++++---- 4 files changed, 29 insertions(+), 34 deletions(-) diff --git a/arch/arm/include/asm/assembler.h b/arch/arm/include/asm/assembler.h index 06b48ce23e1ca245..505a306e0271a9c4 100644 --- a/arch/arm/include/asm/assembler.h +++ b/arch/arm/include/asm/assembler.h @@ -244,19 +244,6 @@ THUMB( fpreg .req r7 ) .endm #endif - .macro local_bh_disable, ti, tmp - ldr \tmp, [\ti, #TI_PREEMPT] - add \tmp, \tmp, #SOFTIRQ_DISABLE_OFFSET - str \tmp, [\ti, #TI_PREEMPT] - .endm - - .macro local_bh_enable_ti, ti, tmp - get_thread_info \ti - ldr \tmp, [\ti, #TI_PREEMPT] - sub \tmp, \tmp, #SOFTIRQ_DISABLE_OFFSET - str \tmp, [\ti, #TI_PREEMPT] - .endm - #define USERL(l, x...) \ 9999: x; \ .pushsection __ex_table,"a"; \ diff --git a/arch/arm/vfp/entry.S b/arch/arm/vfp/entry.S index 6dabb47617781a5f..7483ef8bccda394c 100644 --- a/arch/arm/vfp/entry.S +++ b/arch/arm/vfp/entry.S @@ -24,14 +24,5 @@ ENTRY(do_vfp) mov r1, r10 mov r3, r9 - ldr r4, .LCvfp - ldr pc, [r4] @ call VFP entry point + b vfp_entry ENDPROC(do_vfp) - -ENTRY(vfp_null_entry) - ret lr -ENDPROC(vfp_null_entry) - - .align 2 -.LCvfp: - .word vfp_vector diff --git a/arch/arm/vfp/vfphw.S b/arch/arm/vfp/vfphw.S index 60acd42e05786e95..4d8478264d82b3d2 100644 --- a/arch/arm/vfp/vfphw.S +++ b/arch/arm/vfp/vfphw.S @@ -75,8 +75,6 @@ @ lr = unrecognised instruction return address @ IRQs enabled. ENTRY(vfp_support_entry) - local_bh_disable r1, r4 - ldr r11, [r1, #TI_CPU] @ CPU number add r10, r1, #TI_VFPSTATE @ r10 = workspace @@ -179,9 +177,12 @@ vfp_hw_state_valid: @ else it's one 32-bit instruction, so @ always subtract 4 from the following @ instruction address. - local_bh_enable_ti r10, r4 - ret r3 @ we think we have handled things + mov lr, r3 @ we think we have handled things +local_bh_enable_and_ret: + adr r0, . + mov r1, #SOFTIRQ_DISABLE_OFFSET + b __local_bh_enable_ip @ tail call look_for_VFP_exceptions: @ Check for synchronous or asynchronous exception @@ -204,8 +205,7 @@ skip: @ not recognised by VFP DBGSTR "not VFP" - local_bh_enable_ti r10, r4 - ret lr + b local_bh_enable_and_ret process_exception: DBGSTR "bounce" diff --git a/arch/arm/vfp/vfpmodule.c b/arch/arm/vfp/vfpmodule.c index 86fb0be41ae14b6c..3f88599c285a5d4b 100644 --- a/arch/arm/vfp/vfpmodule.c +++ b/arch/arm/vfp/vfpmodule.c @@ -32,10 +32,9 @@ /* * Our undef handlers (in entry.S) */ -asmlinkage void vfp_support_entry(void); -asmlinkage void vfp_null_entry(void); +asmlinkage void vfp_support_entry(u32, void *, u32, u32); -asmlinkage void (*vfp_vector)(void) = vfp_null_entry; +static bool have_vfp __ro_after_init; /* * Dual-use variable. @@ -799,7 +798,6 @@ static int __init vfp_init(void) vfpsid = fmrx(FPSID); barrier(); unregister_undef_hook(&vfp_detect_hook); - vfp_vector = vfp_null_entry; pr_info("VFP support v0.3: "); if (VFP_arch) { @@ -884,7 +882,7 @@ static int __init vfp_init(void) "arm/vfp:starting", vfp_starting_cpu, vfp_dying_cpu); - vfp_vector = vfp_support_entry; + have_vfp = true; thread_register_notifier(&vfp_notifier_block); vfp_pm_init(); @@ -906,3 +904,22 @@ static int __init vfp_init(void) } core_initcall(vfp_init); + +/* + * Entered with: + * + * r0 = instruction opcode (32-bit ARM or two 16-bit Thumb) + * r1 = thread_info pointer + * r2 = PC value to resume execution after successful emulation + * r3 = normal "successful" return address + * lr = unrecognised instruction return address + */ +asmlinkage void vfp_entry(u32 trigger, struct thread_info *ti, u32 resume_pc, + u32 resume_return_address) +{ + if (unlikely(!have_vfp)) + return; + + local_bh_disable(); + vfp_support_entry(opcode, ti, resume_pc, resume_return_address); +} From patchwork Thu Mar 16 08:20:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 13177175 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 590D1C6FD19 for ; Thu, 16 Mar 2023 08:21:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=CoiNEKefTE4m/KQc5Qq4GOAsCZzarV4rIPFTzgS5O1I=; b=zYrzdHq5K1NZZr il/0/jJV8pSZsr/3pgE3gTcW517ChtmhORb8SdNjxwA6qMtv7Kg0dClBlOgFK6Pm21U2xcBermUBP uy723Mgd1/hAH1uT6uG5XMY2CPubXkWlei/XsvTyCxXQMoKg9ufFHnSMWZDYqPrjiPlUX9kcRIKMs wQ3MTBpq7rP/4PHbxblbWsSLoCAZ9BaxM6LWn/U0DdFVkHP6AN+1Zwoieo2MCil3+sZQNIAZso2x7 X4Xpyyjh/w/rhoYi8zRtH/nJTr3EEP+iX/K56MQMOKE2HdFFOdSlyXaGoWOwlXCDj2ZtueFoCXbpF dYrG6F7WIQRzm6kpoBHA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1pciqt-00Fe11-2P; Thu, 16 Mar 2023 08:20:47 +0000 Received: from ams.source.kernel.org ([2604:1380:4601:e00::1]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1pciqc-00Fdsu-0Y for linux-arm-kernel@lists.infradead.org; Thu, 16 Mar 2023 08:20:33 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id ABD93B82032; Thu, 16 Mar 2023 08:20:28 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id CBD74C4339E; Thu, 16 Mar 2023 08:20:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1678954827; bh=gTpCnMuklaEbXseha6fdkIHtrm5UZnocH2O9AHq4hQc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=X/EJcg/2cLCquS5yNLI35eMXqjkuV3s2JD38/KTS00hAV82i8761V+FEiUsAwwXgc E73T0NVVl5j5D0n54JkEkqYn6TM2c7o+PB1OU+zR+sQnfivCnXOCEft4VoriQuk7l1 MR841Q2znuRD/15i0zEBJQLdUDA2KUbidDo3d15GqS1qlvmV/6vg8mn224bQ1ZeZcg 4UnUNiUFNBxEn9Hd2QNvSy+v9LWcisX3pYhbu/72Q1s0C2GMkgSWdoGH6nwcHa1zkk FkLjer8FYpauCzSvexwm3dCWvXB2Yzp5AtIaJ/OBj0wXgM8nmgkKqQ6+GxcZK3ZKJ9 dCx6LyjKknIWA== From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org, linux@armlinux.org.uk Cc: Ard Biesheuvel , Frederic Weisbecker , Guenter Roeck , Peter Zijlstra , Linus Walleij , Arnd Bergmann Subject: [PATCH v3 4/4] ARM: vfp: Reimplement VFP exception entry in C code Date: Thu, 16 Mar 2023 09:20:07 +0100 Message-Id: <20230316082007.652669-5-ardb@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230316082007.652669-1-ardb@kernel.org> References: <20230316082007.652669-1-ardb@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=16539; i=ardb@kernel.org; h=from:subject; bh=gTpCnMuklaEbXseha6fdkIHtrm5UZnocH2O9AHq4hQc=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIUXoomnZTPH32TP91DJ3amzZm2P5M27GlNNZviu/50R/f CuXPdGzo5SFQYyDQVZMkUVg9t93O09PlKp1niULM4eVCWQIAxenAEyE/Q/DP+tPEbvnfL7m9d3F l/NtVoGbnMo6PuHXE2b+O3Fs2yWTgABGhgMrPoo9imTmLQmV77q3Pm3uTLmel/cafWLaVX47/ss TZwUA X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230316_012030_518641_6DCDDCE7 X-CRM114-Status: GOOD ( 41.31 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org En/disabling softirqs from asm code turned out to be trickier than expected, so vfp_support_entry now returns by tail calling __local_enable_bh_ip() and passing the same arguments that a C call to local_bh_enable() would pass. However, this is slightly hacky, as we don't want to carry our own implementation of local_bh_enable(). So let's bite the bullet, and get rid of the asm logic in vfp_support_entry that reasons about whether or not to save and/or reload the VFP state, and about whether or not an FP exception is pending, and only keep the VFP loading logic as a function that is callable from C. Replicate the removed logic in vfp_entry(), and use the exact same reasoning as in the asm code. To emphasize the correspondance, retain some of the asm comments in the C version as well. Signed-off-by: Ard Biesheuvel Acked-by: Linus Walleij --- arch/arm/kernel/entry-armv.S | 12 ++ arch/arm/vfp/Makefile | 2 +- arch/arm/vfp/entry.S | 28 --- arch/arm/vfp/vfp.h | 1 + arch/arm/vfp/vfphw.S | 204 ++------------------ arch/arm/vfp/vfpmodule.c | 118 +++++++++-- 6 files changed, 131 insertions(+), 234 deletions(-) diff --git a/arch/arm/kernel/entry-armv.S b/arch/arm/kernel/entry-armv.S index c39303e5c23470e6..759404e72c63287c 100644 --- a/arch/arm/kernel/entry-armv.S +++ b/arch/arm/kernel/entry-armv.S @@ -662,6 +662,18 @@ call_fpe: .word 0x00000000 @ opcode #endif +@ VFP entry point. +@ +@ r0 = instruction opcode (32-bit ARM or two 16-bit Thumb) +@ r9 = normal "successful" return address +@ lr = unrecognised instruction return address +do_vfp: + mov r1, sp @ pass struct pt_regs via R1 + bl vfp_entry @ dispatch the VFP exception + cmp r0, #0 @ handled successfully? + reteq r9 @ then use R9 as return address + ret lr + do_fpe: add r10, r10, #TI_FPSTATE @ r10 = workspace ldr_va pc, fp_enter, tmp=r4 @ Call FP module USR entry point diff --git a/arch/arm/vfp/Makefile b/arch/arm/vfp/Makefile index 749901a72d6dc6c4..dfd64bc2b2fbdd06 100644 --- a/arch/arm/vfp/Makefile +++ b/arch/arm/vfp/Makefile @@ -8,4 +8,4 @@ # ccflags-y := -DDEBUG # asflags-y := -DDEBUG -obj-y += vfpmodule.o entry.o vfphw.o vfpsingle.o vfpdouble.o +obj-y += vfpmodule.o vfphw.o vfpsingle.o vfpdouble.o diff --git a/arch/arm/vfp/entry.S b/arch/arm/vfp/entry.S deleted file mode 100644 index 7483ef8bccda394c..0000000000000000 --- a/arch/arm/vfp/entry.S +++ /dev/null @@ -1,28 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0-only */ -/* - * linux/arch/arm/vfp/entry.S - * - * Copyright (C) 2004 ARM Limited. - * Written by Deep Blue Solutions Limited. - */ -#include -#include -#include -#include -#include -#include - -@ VFP entry point. -@ -@ r0 = instruction opcode (32-bit ARM or two 16-bit Thumb) -@ r2 = PC value to resume execution after successful emulation -@ r9 = normal "successful" return address -@ r10 = this threads thread_info structure -@ lr = unrecognised instruction return address -@ IRQs enabled. -@ -ENTRY(do_vfp) - mov r1, r10 - mov r3, r9 - b vfp_entry -ENDPROC(do_vfp) diff --git a/arch/arm/vfp/vfp.h b/arch/arm/vfp/vfp.h index 5cd6d5053271760e..e43a630f8a164f9d 100644 --- a/arch/arm/vfp/vfp.h +++ b/arch/arm/vfp/vfp.h @@ -375,3 +375,4 @@ struct op { }; asmlinkage void vfp_save_state(void *location, u32 fpexc); +asmlinkage u32 vfp_load_state(const void *location); diff --git a/arch/arm/vfp/vfphw.S b/arch/arm/vfp/vfphw.S index 4d8478264d82b3d2..e82a1577afb83666 100644 --- a/arch/arm/vfp/vfphw.S +++ b/arch/arm/vfp/vfphw.S @@ -4,12 +4,6 @@ * * Copyright (C) 2004 ARM Limited. * Written by Deep Blue Solutions Limited. - * - * This code is called from the kernel's undefined instruction trap. - * r1 holds the thread_info pointer - * r3 holds the return address for successful handling. - * lr holds the return address for unrecognised instructions. - * sp points to a struct pt_regs (as defined in include/asm/proc/ptrace.h) */ #include #include @@ -19,20 +13,6 @@ #include #include - .macro DBGSTR, str -#ifdef DEBUG - stmfd sp!, {r0-r3, ip, lr} - ldr r0, =1f - bl _printk - ldmfd sp!, {r0-r3, ip, lr} - - .pushsection .rodata, "a" -1: .ascii KERN_DEBUG "VFP: \str\n" - .byte 0 - .previous -#endif - .endm - .macro DBGSTR1, str, arg #ifdef DEBUG stmfd sp!, {r0-r3, ip, lr} @@ -48,179 +28,27 @@ #endif .endm - .macro DBGSTR3, str, arg1, arg2, arg3 -#ifdef DEBUG - stmfd sp!, {r0-r3, ip, lr} - mov r3, \arg3 - mov r2, \arg2 - mov r1, \arg1 - ldr r0, =1f - bl _printk - ldmfd sp!, {r0-r3, ip, lr} - - .pushsection .rodata, "a" -1: .ascii KERN_DEBUG "VFP: \str\n" - .byte 0 - .previous -#endif - .endm - - -@ VFP hardware support entry point. -@ -@ r0 = instruction opcode (32-bit ARM or two 16-bit Thumb) -@ r1 = thread_info pointer -@ r2 = PC value to resume execution after successful emulation -@ r3 = normal "successful" return address -@ lr = unrecognised instruction return address -@ IRQs enabled. -ENTRY(vfp_support_entry) - ldr r11, [r1, #TI_CPU] @ CPU number - add r10, r1, #TI_VFPSTATE @ r10 = workspace - - DBGSTR3 "instr %08x pc %08x state %p", r0, r2, r10 - - .fpu vfpv2 - VFPFMRX r1, FPEXC @ Is the VFP enabled? - DBGSTR1 "fpexc %08x", r1 - tst r1, #FPEXC_EN - bne look_for_VFP_exceptions @ VFP is already enabled - - DBGSTR1 "enable %x", r10 - ldr r9, vfp_current_hw_state_address - orr r1, r1, #FPEXC_EN @ user FPEXC has the enable bit set - ldr r4, [r9, r11, lsl #2] @ vfp_current_hw_state pointer - bic r5, r1, #FPEXC_EX @ make sure exceptions are disabled - cmp r4, r10 @ this thread owns the hw context? -#ifndef CONFIG_SMP - @ For UP, checking that this thread owns the hw context is - @ sufficient to determine that the hardware state is valid. - beq vfp_hw_state_valid - - @ On UP, we lazily save the VFP context. As a different - @ thread wants ownership of the VFP hardware, save the old - @ state if there was a previous (valid) owner. - - VFPFMXR FPEXC, r5 @ enable VFP, disable any pending - @ exceptions, so we can get at the - @ rest of it - - DBGSTR1 "save old state %p", r4 - cmp r4, #0 @ if the vfp_current_hw_state is NULL - beq vfp_reload_hw @ then the hw state needs reloading - VFPFSTMIA r4, r5 @ save the working registers - VFPFMRX r5, FPSCR @ current status -#ifndef CONFIG_CPU_FEROCEON - tst r1, #FPEXC_EX @ is there additional state to save? - beq 1f - VFPFMRX r6, FPINST @ FPINST (only if FPEXC.EX is set) - tst r1, #FPEXC_FP2V @ is there an FPINST2 to read? - beq 1f - VFPFMRX r8, FPINST2 @ FPINST2 if needed (and present) -1: -#endif - stmia r4, {r1, r5, r6, r8} @ save FPEXC, FPSCR, FPINST, FPINST2 -vfp_reload_hw: - -#else - @ For SMP, if this thread does not own the hw context, then we - @ need to reload it. No need to save the old state as on SMP, - @ we always save the state when we switch away from a thread. - bne vfp_reload_hw - - @ This thread has ownership of the current hardware context. - @ However, it may have been migrated to another CPU, in which - @ case the saved state is newer than the hardware context. - @ Check this by looking at the CPU number which the state was - @ last loaded onto. - ldr ip, [r10, #VFP_CPU] - teq ip, r11 - beq vfp_hw_state_valid - -vfp_reload_hw: - @ We're loading this threads state into the VFP hardware. Update - @ the CPU number which contains the most up to date VFP context. - str r11, [r10, #VFP_CPU] - - VFPFMXR FPEXC, r5 @ enable VFP, disable any pending - @ exceptions, so we can get at the - @ rest of it -#endif - - DBGSTR1 "load state %p", r10 - str r10, [r9, r11, lsl #2] @ update the vfp_current_hw_state pointer +ENTRY(vfp_load_state) + @ Load the current VFP state + @ r0 - load location + @ returns FPEXC + DBGSTR1 "load VFP state %p", r0 @ Load the saved state back into the VFP - VFPFLDMIA r10, r5 @ reload the working registers while + VFPFLDMIA r0, r1 @ reload the working registers while @ FPEXC is in a safe state - ldmia r10, {r1, r5, r6, r8} @ load FPEXC, FPSCR, FPINST, FPINST2 + ldmia r0, {r0-r3} @ load FPEXC, FPSCR, FPINST, FPINST2 #ifndef CONFIG_CPU_FEROCEON - tst r1, #FPEXC_EX @ is there additional state to restore? + tst r0, #FPEXC_EX @ is there additional state to restore? beq 1f - VFPFMXR FPINST, r6 @ restore FPINST (only if FPEXC.EX is set) - tst r1, #FPEXC_FP2V @ is there an FPINST2 to write? + VFPFMXR FPINST, r2 @ restore FPINST (only if FPEXC.EX is set) + tst r0, #FPEXC_FP2V @ is there an FPINST2 to write? beq 1f - VFPFMXR FPINST2, r8 @ FPINST2 if needed (and present) + VFPFMXR FPINST2, r3 @ FPINST2 if needed (and present) 1: #endif - VFPFMXR FPSCR, r5 @ restore status - -@ The context stored in the VFP hardware is up to date with this thread -vfp_hw_state_valid: - tst r1, #FPEXC_EX - bne process_exception @ might as well handle the pending - @ exception before retrying branch - @ out before setting an FPEXC that - @ stops us reading stuff - VFPFMXR FPEXC, r1 @ Restore FPEXC last - sub r2, r2, #4 @ Retry current instruction - if Thumb - str r2, [sp, #S_PC] @ mode it's two 16-bit instructions, - @ else it's one 32-bit instruction, so - @ always subtract 4 from the following - @ instruction address. - - mov lr, r3 @ we think we have handled things -local_bh_enable_and_ret: - adr r0, . - mov r1, #SOFTIRQ_DISABLE_OFFSET - b __local_bh_enable_ip @ tail call - -look_for_VFP_exceptions: - @ Check for synchronous or asynchronous exception - tst r1, #FPEXC_EX | FPEXC_DEX - bne process_exception - @ On some implementations of the VFP subarch 1, setting FPSCR.IXE - @ causes all the CDP instructions to be bounced synchronously without - @ setting the FPEXC.EX bit - VFPFMRX r5, FPSCR - tst r5, #FPSCR_IXE - bne process_exception - - tst r5, #FPSCR_LENGTH_MASK - beq skip - orr r1, r1, #FPEXC_DEX - b process_exception -skip: - - @ Fall into hand on to next handler - appropriate coproc instr - @ not recognised by VFP - - DBGSTR "not VFP" - b local_bh_enable_and_ret - -process_exception: - DBGSTR "bounce" - mov r2, sp @ nothing stacked - regdump is at TOS - mov lr, r3 @ setup for a return to the user code. - - @ Now call the C code to package up the bounce to the support code - @ r0 holds the trigger instruction - @ r1 holds the FPEXC value - @ r2 pointer to register dump - b VFP_bounce @ we have handled this - the support - @ code will raise an exception if - @ required. If not, the user code will - @ retry the faulted instruction -ENDPROC(vfp_support_entry) + VFPFMXR FPSCR, r1 @ restore status + ret lr +ENDPROC(vfp_load_state) ENTRY(vfp_save_state) @ Save the current VFP state @@ -240,10 +68,6 @@ ENTRY(vfp_save_state) ret lr ENDPROC(vfp_save_state) - .align -vfp_current_hw_state_address: - .word vfp_current_hw_state - .macro tbl_branch, base, tmp, shift #ifdef CONFIG_THUMB2_KERNEL adr \tmp, 1f diff --git a/arch/arm/vfp/vfpmodule.c b/arch/arm/vfp/vfpmodule.c index 3f88599c285a5d4b..d8e490f8fb536861 100644 --- a/arch/arm/vfp/vfpmodule.c +++ b/arch/arm/vfp/vfpmodule.c @@ -319,7 +319,7 @@ static u32 vfp_emulate_instruction(u32 inst, u32 fpscr, struct pt_regs *regs) /* * Package up a bounce condition. */ -void VFP_bounce(u32 trigger, u32 fpexc, struct pt_regs *regs) +static void VFP_bounce(u32 trigger, u32 fpexc, struct pt_regs *regs) { u32 fpscr, orig_fpscr, fpsid, exceptions; @@ -370,7 +370,7 @@ void VFP_bounce(u32 trigger, u32 fpexc, struct pt_regs *regs) * on VFP subarch 1. */ vfp_raise_exceptions(VFP_EXCEPTION_ERROR, trigger, fpscr, regs); - goto exit; + return; } /* @@ -401,7 +401,7 @@ void VFP_bounce(u32 trigger, u32 fpexc, struct pt_regs *regs) * the FPEXC.FP2V bit is valid only if FPEXC.EX is 1. */ if ((fpexc & (FPEXC_EX | FPEXC_FP2V)) != (FPEXC_EX | FPEXC_FP2V)) - goto exit; + return; /* * The barrier() here prevents fpinst2 being read @@ -414,8 +414,6 @@ void VFP_bounce(u32 trigger, u32 fpexc, struct pt_regs *regs) exceptions = vfp_emulate_instruction(trigger, orig_fpscr, regs); if (exceptions) vfp_raise_exceptions(exceptions, trigger, orig_fpscr, regs); - exit: - local_bh_enable(); } static void vfp_enable(void *unused) @@ -906,20 +904,110 @@ static int __init vfp_init(void) core_initcall(vfp_init); /* - * Entered with: + * vfp_entry - Handle VFP exception from user mode * - * r0 = instruction opcode (32-bit ARM or two 16-bit Thumb) - * r1 = thread_info pointer - * r2 = PC value to resume execution after successful emulation - * r3 = normal "successful" return address - * lr = unrecognised instruction return address + * @trigger: The opcode of the instruction that triggered the exception + * @regs: pt_regs structure holding the register state at exception entry + * + * Returns 0 if the exception was handled, or an error code otherwise. */ -asmlinkage void vfp_entry(u32 trigger, struct thread_info *ti, u32 resume_pc, - u32 resume_return_address) +asmlinkage int vfp_entry(u32 trigger, struct pt_regs *regs) { + struct thread_info *ti = current_thread_info(); + u32 fpexc; + if (unlikely(!have_vfp)) - return; + return -ENODEV; local_bh_disable(); - vfp_support_entry(opcode, ti, resume_pc, resume_return_address); + fpexc = fmrx(FPEXC); + + /* + * If the VFP unit was not enabled yet, we have to check whether the + * VFP state in the CPU's registers is the most recent VFP state + * associated with the process. On UP systems, we don't save the VFP + * state eagerly on a context switch, so we may need to save the + * VFP state to memory first, as it may belong to another process. + */ + if (!(fpexc & FPEXC_EN)) { + /* + * Enable the VFP unit but mask the FP exception flag for the + * time being, so we can access all the registers. + */ + fpexc |= FPEXC_EN; + fmxr(FPEXC, fpexc & ~FPEXC_EX); + + /* + * Check whether or not the VFP state in the CPU's registers is + * the most recent VFP state associated with this task. On SMP, + * migration may result in multiple CPUs holding VFP states + * that belong to the same task, but only the most recent one + * is valid. + */ + if (!vfp_state_in_hw(ti->cpu, ti)) { + if (!IS_ENABLED(CONFIG_SMP) && + vfp_current_hw_state[ti->cpu] != NULL) { + /* + * This CPU is currently holding the most + * recent VFP state associated with another + * task, and we must save that to memory first. + */ + vfp_save_state(vfp_current_hw_state[ti->cpu], + fpexc); + } + + /* + * We can now proceed with loading the task's VFP state + * from memory into the CPU registers. + */ + fpexc = vfp_load_state(&ti->vfpstate); + vfp_current_hw_state[ti->cpu] = &ti->vfpstate; +#ifdef CONFIG_SMP + /* + * Record that this CPU is now the one holding the most + * recent VFP state of the task. + */ + ti->vfpstate.hard.cpu = ti->cpu; +#endif + } + + if (fpexc & FPEXC_EX) + /* + * Might as well handle the pending exception before + * retrying branch out before setting an FPEXC that + * stops us reading stuff. + */ + goto bounce; + + /* + * No FP exception is pending: just enable the VFP and + * replay the instruction that trapped. + */ + fmxr(FPEXC, fpexc); + regs->ARM_pc -= 4; + } else { + /* Check for synchronous or asynchronous exceptions */ + if (!(fpexc & (FPEXC_EX | FPEXC_DEX))) { + u32 fpscr = fmrx(FPSCR); + + /* + * On some implementations of the VFP subarch 1, + * setting FPSCR.IXE causes all the CDP instructions to + * be bounced synchronously without setting the + * FPEXC.EX bit + */ + if (!(fpscr & FPSCR_IXE)) { + if (!(fpscr & FPSCR_LENGTH_MASK)) { + pr_debug("not VFP\n"); + local_bh_enable(); + return -ENOEXEC; + } + fpexc |= FPEXC_DEX; + } + } +bounce: VFP_bounce(trigger, fpexc, regs); + } + + local_bh_enable(); + return 0; }