From patchwork Fri Apr 4 17:44:22 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Rutland X-Patchwork-Id: 14038705 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 853F2C36010 for ; Fri, 4 Apr 2025 17:50:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=aSEZBbvEe3qpG73LK4o8HjwaIhiytyeGBDtmBgYScYQ=; b=fHr5ee7IyC2y2SnrjeGuCR6pKz xuri8N0hRrjgpUpT7qNhBRNNmauWUPd6UPQXsfN9yBNpCXFHzkNyFZfVaLe0O4enflvk3corpbU0l IO0I4kExAXX2V3Hub8KcIraEEq3vh0/C1m06ZVKnoXKylE2RcCT1EYwJVUE1LbRb7ywmLvxQLo215 NYqyUYNW4K6C3AWkR4bStHZrp/rzV8S2KZKQfDsrd1xh/W0mXUzpkgp6l+L/QEdMqs9JV9a17STmN mRrt/PLNMuC7U0zc0YDhmZ2vlnFi4+cLhPN9P9MnsktbuLc0gNMfQYqM2Y3tt0yIbiCUlBPJ7e2od v6Ld9a3A==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98.1 #2 (Red Hat Linux)) id 1u0lBP-0000000CSCh-0UiR; Fri, 04 Apr 2025 17:50:23 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.98.1 #2 (Red Hat Linux)) id 1u0l5x-0000000CRMt-49A0 for linux-arm-kernel@lists.infradead.org; Fri, 04 Apr 2025 17:44:47 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id AC37C1516; Fri, 4 Apr 2025 10:44:45 -0700 (PDT) Received: from lakrids.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 36EE23F63F; Fri, 4 Apr 2025 10:44:42 -0700 (PDT) From: Mark Rutland To: linux-arm-kernel@lists.infradead.org Cc: ardb@kernel.org, broonie@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, maz@kernel.org, will@kernel.org Subject: [PATCH 01/14] arm64/fpsimd: Avoid RES0 bits in the SME trap handler Date: Fri, 4 Apr 2025 18:44:22 +0100 Message-Id: <20250404174435.3288106-2-mark.rutland@arm.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20250404174435.3288106-1-mark.rutland@arm.com> References: <20250404174435.3288106-1-mark.rutland@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250404_104446_080564_D628185C X-CRM114-Status: GOOD ( 11.81 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The SME trap handler consumes RES0 bits from the ESR when determining the reason for the trap, and depends upon those bits reading as zero. This may break in future when those RES0 bits are allocated a meaning and stop reading as zero. For SME traps taken with ESR_ELx.EC == 0b011101, the specific reason for the trap is indicated by ESR_ELx.ISS.SMTC ("SME Trap Code"). This field occupies bits [2:0] of ESR_ELx.ISS, and as of ARM DDI 0487 L.a, bits [24:3] of ESR_ELx.ISS are RES0. ESR_ELx.ISS itself occupies bits [24:0] of ESR_ELx. Extract the SMTC field specifically, matching the way we handle ESR_ELx fields elsewhere, and ensuring that the handler is future-proof. Fixes: 8bd7f91c03d886f4 ("arm64/sme: Implement traps and syscall handling for SME") Signed-off-by: Mark Rutland Cc: Catalin Marinas Cc: Marc Zyngier Cc: Mark Brown Cc: Will Deacon Reviewed-by: Mark Brown --- arch/arm64/include/asm/esr.h | 14 ++++++++------ arch/arm64/kernel/fpsimd.c | 2 +- 2 files changed, 9 insertions(+), 7 deletions(-) diff --git a/arch/arm64/include/asm/esr.h b/arch/arm64/include/asm/esr.h index d1b1a33f9a8b0..63aed18a933fe 100644 --- a/arch/arm64/include/asm/esr.h +++ b/arch/arm64/include/asm/esr.h @@ -371,12 +371,14 @@ /* * ISS values for SME traps */ - -#define ESR_ELx_SME_ISS_SME_DISABLED 0 -#define ESR_ELx_SME_ISS_ILL 1 -#define ESR_ELx_SME_ISS_SM_DISABLED 2 -#define ESR_ELx_SME_ISS_ZA_DISABLED 3 -#define ESR_ELx_SME_ISS_ZT_DISABLED 4 +#define ESR_ELx_SME_ISS_SMTC_MASK GENMASK(2, 0) +#define ESR_ELx_SME_ISS_SMTC(esr) ((esr) & ESR_ELx_SME_ISS_SMTC_MASK) + +#define ESR_ELx_SME_ISS_SMTC_SME_DISABLED 0 +#define ESR_ELx_SME_ISS_SMTC_ILL 1 +#define ESR_ELx_SME_ISS_SMTC_SM_DISABLED 2 +#define ESR_ELx_SME_ISS_SMTC_ZA_DISABLED 3 +#define ESR_ELx_SME_ISS_SMTC_ZT_DISABLED 4 /* ISS field definitions for MOPS exceptions */ #define ESR_ELx_MOPS_ISS_MEM_INST (UL(1) << 24) diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c index 8370d55f03533..1ee5f330b8ed3 100644 --- a/arch/arm64/kernel/fpsimd.c +++ b/arch/arm64/kernel/fpsimd.c @@ -1436,7 +1436,7 @@ void do_sme_acc(unsigned long esr, struct pt_regs *regs) * If this not a trap due to SME being disabled then something * is being used in the wrong mode, report as SIGILL. */ - if (ESR_ELx_ISS(esr) != ESR_ELx_SME_ISS_SME_DISABLED) { + if (ESR_ELx_SME_ISS_SMTC(esr) != ESR_ELx_SME_ISS_SMTC_SME_DISABLED) { force_signal_inject(SIGILL, ILL_ILLOPC, regs->pc, 0); return; } From patchwork Fri Apr 4 17:44:23 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Rutland X-Patchwork-Id: 14038704 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 29CC1C36010 for ; Fri, 4 Apr 2025 17:48:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=ZvW0FZ81QZGgPkPnffGLt2x6zapzarvRDL3O6zHo+uA=; b=mKGyK5LRwM+2+Uj05rsfIf7VYv UjaCaewl2QH3RFcjM748EgvGaVQiv9ntSaMZc3MeRqaExdqXeaa4km9Yo2FeUsH2Y+4aFtgeFxsuf SfsYPazObwYHQqgBOrHbiGsf8/j4A+oHa619xvt881nX2glI7NT8pJeGD0GxqDx6iUe0DQuTed5VY ahANLAhvr3pxdWCOUGTMsa6J9VPmlawVhAlMmjtATjlcA4GMnDGfyBHif69IgUiffgmQ163r9xwET Lu2fR/B/WpgTWU2cByTGsZQvIMZkIGafwj8dOUv3Z6kh+7e5u0R+qo5bpTl/3iYGVBvE+YXBFkwHZ cvvN+AoA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98.1 #2 (Red Hat Linux)) id 1u0l9b-0000000CRup-1Dd3; Fri, 04 Apr 2025 17:48:31 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.98.1 #2 (Red Hat Linux)) id 1u0l5x-0000000CRMs-3whv for linux-arm-kernel@lists.infradead.org; Fri, 04 Apr 2025 17:44:47 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id CACA21515; Fri, 4 Apr 2025 10:44:47 -0700 (PDT) Received: from lakrids.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 55BC53F63F; Fri, 4 Apr 2025 10:44:44 -0700 (PDT) From: Mark Rutland To: linux-arm-kernel@lists.infradead.org Cc: ardb@kernel.org, broonie@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, maz@kernel.org, will@kernel.org Subject: [PATCH 02/14] arm64/fpsimd: Remove unused fpsimd_force_sync_to_sve() Date: Fri, 4 Apr 2025 18:44:23 +0100 Message-Id: <20250404174435.3288106-3-mark.rutland@arm.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20250404174435.3288106-1-mark.rutland@arm.com> References: <20250404174435.3288106-1-mark.rutland@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250404_104446_032684_E58C3F20 X-CRM114-Status: GOOD ( 10.30 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org There have been no users of fpsimd_force_sync_to_sve() since commit: bbc6172eefdb276b ("arm64/fpsimd: SME no longer requires SVE register state") Remove fpsimd_force_sync_to_sve(). Signed-off-by: Mark Rutland Cc: Catalin Marinas Cc: Marc Zyngier Cc: Mark Brown Cc: Will Deacon --- arch/arm64/kernel/fpsimd.c | 14 -------------- 1 file changed, 14 deletions(-) diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c index 1ee5f330b8ed3..1391a491f2226 100644 --- a/arch/arm64/kernel/fpsimd.c +++ b/arch/arm64/kernel/fpsimd.c @@ -758,20 +758,6 @@ void sve_alloc(struct task_struct *task, bool flush) kzalloc(sve_state_size(task), GFP_KERNEL); } - -/* - * Force the FPSIMD state shared with SVE to be updated in the SVE state - * even if the SVE state is the current active state. - * - * This should only be called by ptrace. task must be non-runnable. - * task->thread.sve_state must point to at least sve_state_size(task) - * bytes of allocated kernel memory. - */ -void fpsimd_force_sync_to_sve(struct task_struct *task) -{ - fpsimd_to_sve(task); -} - /* * Ensure that task->thread.sve_state is up to date with respect to * the user task, irrespective of when SVE is in use or not. From patchwork Fri Apr 4 17:44:24 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Rutland X-Patchwork-Id: 14038710 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3A5E2C36010 for ; Fri, 4 Apr 2025 17:52:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=urcjPGO/ldVu/mWI8JTzBSN6SKwOtRGa7u2Kt8myzPo=; b=CmJKV2jUNUemzK4HotsKOJo4RP lipR398hz4rUqhVU3M+blDS2icEZqmTWshQ7QTt6oIxY4oEYqCQAa3Hh2O7sJMOxrWNRnw8ucwzg/ gcXXJTMkBZhyuim9wEinJDMfT9U7JOqcddFQmz0oGjRhmR84guqB5tGPLELdhpEmDooxWW2NaD842 PyxTKQ6sC/0hXOMZEZthIHxeZIjCNGfkDmVAZn1ehVCGssF4aBjWRV32ojN2ALBCER8mXN+35ibGN 1kK72W9mtd51L8CwZ8h5g+/cuWqhIAIxb0NqAiOI/4959YZJj2scT1pYnbxplx3Ec5YjarVXWYUF7 guP8ksFQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98.1 #2 (Red Hat Linux)) id 1u0lDD-0000000CSS0-3Psv; Fri, 04 Apr 2025 17:52:15 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.98.1 #2 (Red Hat Linux)) id 1u0l60-0000000CROD-23qO for linux-arm-kernel@lists.infradead.org; Fri, 04 Apr 2025 17:44:49 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id E43991516; Fri, 4 Apr 2025 10:44:49 -0700 (PDT) Received: from lakrids.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 6F0B33F63F; Fri, 4 Apr 2025 10:44:46 -0700 (PDT) From: Mark Rutland To: linux-arm-kernel@lists.infradead.org Cc: ardb@kernel.org, broonie@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, maz@kernel.org, will@kernel.org Subject: [PATCH 03/14] arm64/fpsimd: Remove redundant clearing of TIF_SVE Date: Fri, 4 Apr 2025 18:44:24 +0100 Message-Id: <20250404174435.3288106-4-mark.rutland@arm.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20250404174435.3288106-1-mark.rutland@arm.com> References: <20250404174435.3288106-1-mark.rutland@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250404_104448_591186_CD70C388 X-CRM114-Status: GOOD ( 13.50 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org TIF_SVE implies that sve_state has been allocated. Barring specific transient periods (e.g. during changes to SVE/SME vector lengths), it is not valid for a task to have TIF_SVE set while sve_state is NULL. This was not taken into account in commit: 7559b7d7d651d397 ("arm64/sve: Better handle failure to allocate SVE register storage") As of that commit, sve_set_common() and restore_sve_fpsimd_context() clear TIF_SVE if sve_alloc() fails to allocate memory. In these cases TIF_SVE cannot legitimately have been set to begin with, and clearing TIF_SVE only serves to complicate the code. No other code paths clear TIF_SVE if sve_alloc() fails to allocate memory. The unnecessary clearing of TIF_SVE lead to further unnecessary changes in commit: baa8515281b30861 ("arm64/fpsimd: Track the saved FPSIMD state type separately to TIF_SVE") As of that commit, sve_set_common() also sets the task's saved fp_type to FP_STATE_FPSIMD if sve_alloc() fails to allocate memory. This is unnecessary as when sve_state is NULL, the saved fp_type can only legitimately be FP_STATE_FPSIMD. That commit did not update restore_sve_fpsimd_context() similarly, and so restore_sve_fpsimd_context() does not alter the task's saved fp_type if sve_alloc() fails to allocate memory. Remove the unnecessary and confusing clearing of TIF_SVE, and remove the similarly unnecessary and confusing update of the saved fp_type. Signed-off-by: Mark Rutland Cc: Catalin Marinas Cc: Marc Zyngier Cc: Mark Brown Cc: Will Deacon --- arch/arm64/kernel/ptrace.c | 2 -- arch/arm64/kernel/signal.c | 4 +--- 2 files changed, 1 insertion(+), 5 deletions(-) diff --git a/arch/arm64/kernel/ptrace.c b/arch/arm64/kernel/ptrace.c index f79b0d5f71ac9..575ececb8b720 100644 --- a/arch/arm64/kernel/ptrace.c +++ b/arch/arm64/kernel/ptrace.c @@ -969,8 +969,6 @@ static int sve_set_common(struct task_struct *target, sve_alloc(target, true); if (!target->thread.sve_state) { ret = -ENOMEM; - clear_tsk_thread_flag(target, TIF_SVE); - target->thread.fp_type = FP_STATE_FPSIMD; goto out; } diff --git a/arch/arm64/kernel/signal.c b/arch/arm64/kernel/signal.c index 99ea26d400ffe..40c572869325b 100644 --- a/arch/arm64/kernel/signal.c +++ b/arch/arm64/kernel/signal.c @@ -443,10 +443,8 @@ static int restore_sve_fpsimd_context(struct user_ctxs *user) /* From now, fpsimd_thread_switch() won't touch thread.sve_state */ sve_alloc(current, true); - if (!current->thread.sve_state) { - clear_thread_flag(TIF_SVE); + if (!current->thread.sve_state) return -ENOMEM; - } err = __copy_from_user(current->thread.sve_state, (char __user const *)user->sve + From patchwork Fri Apr 4 17:44:25 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Rutland X-Patchwork-Id: 14038711 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5EBEEC36010 for ; Fri, 4 Apr 2025 17:54:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=XvEJXAwiIYvDdL2Da1ynSRl36Og8jx7xSRRU2gHkMLs=; b=d5D81KyNiqYdVEHKdjH3spApex 84yxd7lfial74HfS5ZyJX+pVSMvOQI0oU7xChN9w6TjPX/3hVfQ0cinSNmTlngD7HdoJdA8nX5hFi RrBRBZn7QgK+tQPfZSJy+bCsmRindu6B/fjR+qvJfEESwUj9/74yMOKAv3D+XYPn2dIzGxCT0AFR2 8ozye/l6j8zobGF2n7PAw/tNADPhu4rnDjQmybVlvnQ23mcm2ZkRR3YXrbESij5voJni6ZKRQrZzg mv7z6vXpPOS5BjMtt61D+v/yGEGW3JY6FAs4Obs+5a7ochfDc9BtiyBosSLt/sxBAWGnEus2Z2VDH fsfAoUhQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98.1 #2 (Red Hat Linux)) id 1u0lF1-0000000CSeQ-2GnZ; Fri, 04 Apr 2025 17:54:07 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.98.1 #2 (Red Hat Linux)) id 1u0l62-0000000CROf-0mRe for linux-arm-kernel@lists.infradead.org; Fri, 04 Apr 2025 17:44:51 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 1ABCD1515; Fri, 4 Apr 2025 10:44:52 -0700 (PDT) Received: from lakrids.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 99A803F63F; Fri, 4 Apr 2025 10:44:48 -0700 (PDT) From: Mark Rutland To: linux-arm-kernel@lists.infradead.org Cc: ardb@kernel.org, broonie@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, maz@kernel.org, will@kernel.org Subject: [PATCH 04/14] arm64/fpsimd: Remove redundant SVE trap manipulation Date: Fri, 4 Apr 2025 18:44:25 +0100 Message-Id: <20250404174435.3288106-5-mark.rutland@arm.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20250404174435.3288106-1-mark.rutland@arm.com> References: <20250404174435.3288106-1-mark.rutland@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250404_104450_268275_92CC7432 X-CRM114-Status: GOOD ( 11.76 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org When task_fpsimd_load() loads the saved FPSIMD/SVE/SME state, it configures EL0 SVE traps by calling sve_user_{enable,disable}(). This is unnecessary, and this is suspicious/confusing as task_fpsimd_load() does not configure EL0 SME traps. All calls to task_fpsimd_load() are followed by a call to fpsimd_bind_task_to_cpu(), where the latter configures traps for SVE and SME dependent upon the current values of TIF_SVE and TIF_SME, overriding any trap configuration performed by task_fpsimd_load(). The calls to sve_user_{enable,disable}() calls in task_fpsimd_load() have been redundant (though benign) since they were introduced in commit: a0136be443d51803 ("arm64/fpsimd: Load FP state based on recorded data type") Remove the unnecessary and confusing SVE trap manipulation. Signed-off-by: Mark Rutland Cc: Catalin Marinas Cc: Marc Zyngier Cc: Mark Brown Cc: Will Deacon Reviewed-by: Mark Brown --- arch/arm64/kernel/fpsimd.c | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c index 1391a491f2226..757445d72e5b7 100644 --- a/arch/arm64/kernel/fpsimd.c +++ b/arch/arm64/kernel/fpsimd.c @@ -366,13 +366,11 @@ static void task_fpsimd_load(void) switch (current->thread.fp_type) { case FP_STATE_FPSIMD: /* Stop tracking SVE for this task until next use. */ - if (test_and_clear_thread_flag(TIF_SVE)) - sve_user_disable(); + clear_thread_flag(TIF_SVE); break; case FP_STATE_SVE: - if (!thread_sm_enabled(¤t->thread) && - !WARN_ON_ONCE(!test_and_set_thread_flag(TIF_SVE))) - sve_user_enable(); + if (!thread_sm_enabled(¤t->thread)) + WARN_ON_ONCE(!test_and_set_thread_flag(TIF_SVE)); if (test_thread_flag(TIF_SVE)) sve_set_vq(sve_vq_from_vl(task_get_sve_vl(current)) - 1); From patchwork Fri Apr 4 17:44:26 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Rutland X-Patchwork-Id: 14038712 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 903EAC36010 for ; Fri, 4 Apr 2025 17:56:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=aA/RKwHC7mS+0+Bt1pgJ1xABFDPHflt6LFD6Rkl82dw=; b=PLk6Nh78Fwp7QIqiJc0KFmBo1m mCL47kGJQMP2vchaK0wuskszbZ5mI11j+FkXWZIuG1ZNr+EN0ZPNOlrZsw2OIRIP6NuhCkUootxh4 hfpn7a4zqKCV8Sw0AyVVIKxtWdjF1qdOKaoXv3CyHOWfCXxZZ7ur2QqN9Ub9HrC75AMwfiliw1eCk 4BTW16uvU6UZknnRKGWj7xw4CaKFWPsqGCgjbG1mKedHshfqsyQRQqm9ja3mLzWVqVH4Z0ifnLApB 6n4blD/EaGj2oEliCUI39gBFrKkJiJed7B2ASbci1cKO9GbCIvWy+R/oSYhxqBMIdjJ995wpgi3wg ulP+mqig==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98.1 #2 (Red Hat Linux)) id 1u0lGq-0000000CT8T-1VsH; Fri, 04 Apr 2025 17:56:00 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.98.1 #2 (Red Hat Linux)) id 1u0l64-0000000CRPH-2xrn for linux-arm-kernel@lists.infradead.org; Fri, 04 Apr 2025 17:44:53 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 2CB0B1515; Fri, 4 Apr 2025 10:44:54 -0700 (PDT) Received: from lakrids.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id ABF1C3F63F; Fri, 4 Apr 2025 10:44:50 -0700 (PDT) From: Mark Rutland To: linux-arm-kernel@lists.infradead.org Cc: ardb@kernel.org, broonie@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, maz@kernel.org, will@kernel.org Subject: [PATCH 05/14] arm64/fpsimd: Remove opportunistic freeing of SME state Date: Fri, 4 Apr 2025 18:44:26 +0100 Message-Id: <20250404174435.3288106-6-mark.rutland@arm.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20250404174435.3288106-1-mark.rutland@arm.com> References: <20250404174435.3288106-1-mark.rutland@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250404_104452_789971_885D1DAA X-CRM114-Status: GOOD ( 10.49 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org When a task's SVE vector length (NSVL) is changed, and the task happens to have SVCR.{SM,ZA}=={0,0}, vec_set_vector_length() opportunistically frees the task's sme_state and clears TIF_SME. The opportunistic freeing was added with no rationale in commit: d4d5be94a8787242 ("arm64/fpsimd: Ensure SME storage is allocated after SVE VL changes") That commit fixed an unrelated problem where the task's sve_state was freed while it could be used to store streaming mode register state, where the fix was to re-allocate the task's sve_state. There is no need to free and/or reallocate the task's sme_state when the SVE vector length changes, and there is no need to clear TIF_SME. Given the SME vector length (SVL) doesn't change, the task's sme_state remains correctly sized. Remove the unnecessary opportunistic freeing of the task's sme_state. Signed-off-by: Mark Rutland Cc: Catalin Marinas Cc: Marc Zyngier Cc: Mark Brown Cc: Will Deacon Reviewed-by: Mark Brown --- arch/arm64/kernel/fpsimd.c | 17 ++++------------- 1 file changed, 4 insertions(+), 13 deletions(-) diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c index 757445d72e5b7..128774015772a 100644 --- a/arch/arm64/kernel/fpsimd.c +++ b/arch/arm64/kernel/fpsimd.c @@ -868,19 +868,10 @@ int vec_set_vector_length(struct task_struct *task, enum vec_type type, task->thread.fp_type = FP_STATE_FPSIMD; } - if (system_supports_sme()) { - if (type == ARM64_VEC_SME || - !(task->thread.svcr & (SVCR_SM_MASK | SVCR_ZA_MASK))) { - /* - * We are changing the SME VL or weren't using - * SME anyway, discard the state and force a - * reallocation. - */ - task->thread.svcr &= ~(SVCR_SM_MASK | - SVCR_ZA_MASK); - clear_tsk_thread_flag(task, TIF_SME); - free_sme = true; - } + if (system_supports_sme() && type == ARM64_VEC_SME) { + task->thread.svcr &= ~(SVCR_SM_MASK | SVCR_ZA_MASK); + clear_tsk_thread_flag(task, TIF_SME); + free_sme = true; } if (task == current) From patchwork Fri Apr 4 17:44:27 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Rutland X-Patchwork-Id: 14038713 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 58333C36010 for ; Fri, 4 Apr 2025 17:58:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=JHA8jKZbxyVmnYMLzHVWyd6qL64Gcvo0HCQVPnx3SDs=; b=L/ubLmYnSjwvuIFFLKEgtQbvwA OHhW9svFy++wRy5wxqAz8iqURHz25qWBOO/gcTesLoaCg/aowlgFmpp9ZloaofYCQ0hVQYxd0iVGm XtJcUKsXYznQP+F3Q6rUsQh969yBeOSDqmgQH3UgkG07tUJKsSAHOj2e5gr3VxFa9tS+W507aHqbC le6PgkrilRPs9KuwkjR0FhIMRm2kwZygESSo9bAkAwvG1EUynDR2TXDx10yrS20Xfa0gCILteA99I J/OXL6RtI08uYt9Ki0F1pgS9rAAoeFVfgVcVvw0QUQzkzAtEhxBQZtuxgQjK70HNjnzRyDMCAFuHY y+4kEkyA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98.1 #2 (Red Hat Linux)) id 1u0lIe-0000000CTT1-0PMd; Fri, 04 Apr 2025 17:57:52 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.98.1 #2 (Red Hat Linux)) id 1u0l66-0000000CRPh-1q2M for linux-arm-kernel@lists.infradead.org; Fri, 04 Apr 2025 17:44:55 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 3C4731516; Fri, 4 Apr 2025 10:44:56 -0700 (PDT) Received: from lakrids.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id BB38E3F63F; Fri, 4 Apr 2025 10:44:52 -0700 (PDT) From: Mark Rutland To: linux-arm-kernel@lists.infradead.org Cc: ardb@kernel.org, broonie@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, maz@kernel.org, will@kernel.org Subject: [PATCH 06/14] arm64/fpsimd: Discard stale CPU state when handling SME traps Date: Fri, 4 Apr 2025 18:44:27 +0100 Message-Id: <20250404174435.3288106-7-mark.rutland@arm.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20250404174435.3288106-1-mark.rutland@arm.com> References: <20250404174435.3288106-1-mark.rutland@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250404_104454_610416_000E190E X-CRM114-Status: GOOD ( 16.03 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Mark Brown The logic for handling SME traps manipulates saved FPSIMD/SVE/SME state incorrectly, and a race with preemption can result in a task having TIF_SME set and TIF_FOREIGN_FPSTATE clear even though the live CPU state is stale (e.g. with SME traps enabled). This can result in warnings from do_sme_acc() where SME traps are not expected while TIF_SME is set: | /* With TIF_SME userspace shouldn't generate any traps */ | if (test_and_set_thread_flag(TIF_SME)) | WARN_ON(1); This is very similar to the SVE issue we fixed in commit: 751ecf6afd6568ad ("arm64/sve: Discard stale CPU state when handling SVE traps") The race can occur when the SME trap handler is preempted before and after manipulating the saved FPSIMD/SVE/SME state, starting and ending on the same CPU, e.g. | void do_sme_acc(unsigned long esr, struct pt_regs *regs) | { | // Trap on CPU 0 with TIF_SME clear, SME traps enabled | // task->fpsimd_cpu is 0. | // per_cpu_ptr(&fpsimd_last_state, 0) is task. | | ... | | // Preempted; migrated from CPU 0 to CPU 1. | // TIF_FOREIGN_FPSTATE is set. | | get_cpu_fpsimd_context(); | | /* With TIF_SME userspace shouldn't generate any traps */ | if (test_and_set_thread_flag(TIF_SME)) | WARN_ON(1); | | if (!test_thread_flag(TIF_FOREIGN_FPSTATE)) { | unsigned long vq_minus_one = | sve_vq_from_vl(task_get_sme_vl(current)) - 1; | sme_set_vq(vq_minus_one); | | fpsimd_bind_task_to_cpu(); | } | | put_cpu_fpsimd_context(); | | // Preempted; migrated from CPU 1 to CPU 0. | // task->fpsimd_cpu is still 0 | // If per_cpu_ptr(&fpsimd_last_state, 0) is still task then: | // - Stale HW state is reused (with SME traps enabled) | // - TIF_FOREIGN_FPSTATE is cleared | // - A return to userspace skips HW state restore | } Fix the case where the state is not live and TIF_FOREIGN_FPSTATE is set by calling fpsimd_flush_task_state() to detach from the saved CPU state. This ensures that a subsequent context switch will not reuse the stale CPU state, and will instead set TIF_FOREIGN_FPSTATE, forcing the new state to be reloaded from memory prior to a return to userspace. Note: this was originallly posted as [1]. Fixes: 8bd7f91c03d8 ("arm64/sme: Implement traps and syscall handling for SME") Reported-by: Mark Rutland Signed-off-by: Mark Brown Link: https://lore.kernel.org/linux-arm-kernel/20241204-arm64-sme-reenable-v2-1-bae87728251d@kernel.org/ # [1] [ Rutland: rewrite commit message ] Signed-off-by: Mark Rutland Cc: Catalin Marinas Cc: Marc Zyngier Cc: Will Deacon --- arch/arm64/kernel/fpsimd.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c index 128774015772a..b4858d85292b4 100644 --- a/arch/arm64/kernel/fpsimd.c +++ b/arch/arm64/kernel/fpsimd.c @@ -1435,6 +1435,8 @@ void do_sme_acc(unsigned long esr, struct pt_regs *regs) sme_set_vq(vq_minus_one); fpsimd_bind_task_to_cpu(); + } else { + fpsimd_flush_task_state(current); } put_cpu_fpsimd_context(); From patchwork Fri Apr 4 17:44:28 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Rutland X-Patchwork-Id: 14038714 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5C388C36010 for ; Fri, 4 Apr 2025 17:59:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=fwdaJv1p1f03pvmV73v7i63rIbWdamxaEk7BesKB8Qw=; b=ud4a3fqv+JqzRnhWoh95nUNUwf +Cy5FWhkmfGPJIggVTU1p9Qn6lWLn2t8lzLO3+YBlmiEVZnaTDboT0uTc6HtRTeHUzxIIGUyrXQHo HsJ9iy8dwnnR2z0KSL5uhNexPgqXoSIpdpMaFOEpM+SkcNQDFHGJH0gcHr4NoaFIyOtx5zHtIZN1F wPKS5g7Z05yFSsyQo4L8dYTV+pavtY91dpwnFWHdYicb36YTqrhASjkbbWyz39NgmE6etiTgVu8PH Z4crCzpM3xaLC335XHvxFA2c04Q83sXen+Xm0q2MR9FAb3AXPMJXEaPQ35aUPngCUvYFIWYEi1OVW LmY5sr6A==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98.1 #2 (Red Hat Linux)) id 1u0lKS-0000000CTig-3EDd; Fri, 04 Apr 2025 17:59:44 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.98.1 #2 (Red Hat Linux)) id 1u0l68-0000000CRQH-1svU for linux-arm-kernel@lists.infradead.org; Fri, 04 Apr 2025 17:44:57 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 57EBA1515; Fri, 4 Apr 2025 10:44:58 -0700 (PDT) Received: from lakrids.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id D6BBF3F63F; Fri, 4 Apr 2025 10:44:54 -0700 (PDT) From: Mark Rutland To: linux-arm-kernel@lists.infradead.org Cc: ardb@kernel.org, broonie@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, maz@kernel.org, will@kernel.org Subject: [PATCH 07/14] arm64/fpsimd: Don't corrupt FPMR when streaming mode changes Date: Fri, 4 Apr 2025 18:44:28 +0100 Message-Id: <20250404174435.3288106-8-mark.rutland@arm.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20250404174435.3288106-1-mark.rutland@arm.com> References: <20250404174435.3288106-1-mark.rutland@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250404_104456_545042_B29A4B7F X-CRM114-Status: GOOD ( 13.41 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Mark Brown When the effective value of PSTATE.SM is changed from 0 to 1 or from 1 to 0 by any method, an entry or exit to/from streaming SVE mode is performed, and hardware automatically resets a number of registers. As of ARM DDI 0487 L.a, this means: * All implemented bits of the SVE vector registers are set to zero. * All implemented bits of the SVE predicate registers are set to zero. * All implemented bits of FFR are set to zero, if FFR is implemented in the new mode. * FPSR is set to 0x0000_0000_0800_009f. * FPMR is set to 0, if FPMR is implemented. Currently task_fpsimd_load() restores FPMR before restoring SVCR (which is an accessor for PSTATE.{SM,ZA}), and so the restored value of FPMR may be clobbered if the restored value of PSTATE.SM happens to differ from the initial value of PSTATE.SM. Fix this by moving the restore of FPMR later. Note: this was originally posted as [1]. Fixes: 203f2b95a882 ("arm64/fpsimd: Support FEAT_FPMR") Signed-off-by: Mark Brown Link: https://lore.kernel.org/linux-arm-kernel/20241204-arm64-sme-reenable-v2-2-bae87728251d@kernel.org/ # [1] [ Rutland: rewrite commit message ] Signed-off-by: Mark Rutland --- arch/arm64/kernel/fpsimd.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c index b4858d85292b4..b19736f354a71 100644 --- a/arch/arm64/kernel/fpsimd.c +++ b/arch/arm64/kernel/fpsimd.c @@ -359,9 +359,6 @@ static void task_fpsimd_load(void) WARN_ON(preemptible()); WARN_ON(test_thread_flag(TIF_KERNEL_FPSTATE)); - if (system_supports_fpmr()) - write_sysreg_s(current->thread.uw.fpmr, SYS_FPMR); - if (system_supports_sve() || system_supports_sme()) { switch (current->thread.fp_type) { case FP_STATE_FPSIMD: @@ -411,6 +408,9 @@ static void task_fpsimd_load(void) restore_ffr = system_supports_fa64(); } + if (system_supports_fpmr()) + write_sysreg_s(current->thread.uw.fpmr, SYS_FPMR); + if (restore_sve_regs) { WARN_ON_ONCE(current->thread.fp_type != FP_STATE_SVE); sve_load_state(sve_pffr(¤t->thread), From patchwork Fri Apr 4 17:44:29 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Rutland X-Patchwork-Id: 14038715 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0D1D2C36010 for ; Fri, 4 Apr 2025 18:01:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=kaqsx1T3+z+wsKZMt5BYMEFxX9Jh3Fvf/2DcN3lcP+o=; b=3Hk+0M3/j2731Q2/l8UtZz+2ey V5fGj9ey+5abZ/wl1MkHSU2lO214qGdKWg/ejftqA64Edq5XJw+J68fpBkhri/UxKodb1IatirfSp oBZOaAlUYyU5GP/pdYcc5buHUr6RcnaJ9e0nq7zuiGXggHCvLUGtwMGt7WbmeqXHG1s5IGJ2YSeTz 4v+UmEGJ7nFO1Zw2Ov1apN6cIlug9ciJzIMO05Eizs0FoN3MUfaASkv6OTH70txxiGKMsBPp9MzgL Fm//8RAdguTrAvE22Qzg5NFHCDqDogwPpatyHyq8bEl2mtGJyGyi9dKi3rxf+sYTax0VFNCp3VG+p IdIa4uAA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98.1 #2 (Red Hat Linux)) id 1u0lMG-0000000CTxE-2iKf; Fri, 04 Apr 2025 18:01:36 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.98.1 #2 (Red Hat Linux)) id 1u0l6A-0000000CRQe-26yM for linux-arm-kernel@lists.infradead.org; Fri, 04 Apr 2025 17:44:59 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 65DCD1515; Fri, 4 Apr 2025 10:45:00 -0700 (PDT) Received: from lakrids.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id E40B13F63F; Fri, 4 Apr 2025 10:44:56 -0700 (PDT) From: Mark Rutland To: linux-arm-kernel@lists.infradead.org Cc: ardb@kernel.org, broonie@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, maz@kernel.org, will@kernel.org Subject: [PATCH 08/14] arm64/fpsimd: Avoid clobbering kernel FPSIMD state with SMSTOP Date: Fri, 4 Apr 2025 18:44:29 +0100 Message-Id: <20250404174435.3288106-9-mark.rutland@arm.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20250404174435.3288106-1-mark.rutland@arm.com> References: <20250404174435.3288106-1-mark.rutland@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250404_104458_589007_323ECA45 X-CRM114-Status: GOOD ( 12.70 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On system with SME, a thread's kernel FPSIMD state may be erroneously clobbered during a context switch immediately after that state is restored. Systems without SME are unaffected. If the CPU happens to be in streaming SVE mode before a context switch to a thread with kernel FPSIMD state, fpsimd_thread_switch() will restore the kernel FPSIMD state using fpsimd_load_kernel_state() while the CPU is still in streaming SVE mode. When fpsimd_thread_switch() subsequently calls fpsimd_flush_cpu_state(), this will execute an SMSTOP, causing an exit from streaming SVE mode. The exit from streaming SVE mode will cause the hardware to reset a number of FPSIMD/SVE/SME registers, clobbering the FPSIMD state. Fix this by calling fpsimd_flush_cpu_state() before restoring the kernel FPSIMD state. Fixes: e92bee9f861b466c ("arm64/fpsimd: Avoid erroneous elide of user state reload") Signed-off-by: Mark Rutland Cc: Ard Biesheuvel Cc: Catalin Marinas Cc: Mark Brown Cc: Will Deacon Reviewed-by: Mark Brown --- arch/arm64/kernel/fpsimd.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c index b19736f354a71..4a0b0bb3a3fad 100644 --- a/arch/arm64/kernel/fpsimd.c +++ b/arch/arm64/kernel/fpsimd.c @@ -1550,8 +1550,8 @@ void fpsimd_thread_switch(struct task_struct *next) fpsimd_save_user_state(); if (test_tsk_thread_flag(next, TIF_KERNEL_FPSTATE)) { - fpsimd_load_kernel_state(next); fpsimd_flush_cpu_state(); + fpsimd_load_kernel_state(next); } else { /* * Fix up TIF_FOREIGN_FPSTATE to correctly describe next's From patchwork Fri Apr 4 17:44:30 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Rutland X-Patchwork-Id: 14038718 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6E615C36010 for ; Fri, 4 Apr 2025 18:03:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=tAhGbtmQmHw/NZLI92StPB8qVxht4n/61+8HsGthBRM=; b=Q8oJlS5TpBTqcoZumtQm9gBphx 5ysHAI54Xa/WiLWGPWtINLULp5SXJUcsIhsRB5HsAoJDqY9/DIfBthNbifMDANOpx0wIj+4GMff3w k8YSv1M2ZnanTt/W6j5p/UtP95kWk0u4uZfK6m2Ta2nzLPekfWx+LpPuvvkwsv+sK8fvvvE//ZWnB XGkWnl3ccm61n2fFAhtRM1/TZUIrIMAXVN3QZ/SIjzFi63BDwwDq/Ukl4dPUJgqt0EuIyAW6Fv5bK VYvd8NuQrCV6xjLIF2iFnZMXafwOQiw9ga4zOUn+wI1bBEqHnjAenwCt70J9m4oXWB4Fw1HrcpJAf rMtwDXGA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98.1 #2 (Red Hat Linux)) id 1u0lO4-0000000CUB8-1PF1; Fri, 04 Apr 2025 18:03:28 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.98.1 #2 (Red Hat Linux)) id 1u0l6C-0000000CRRJ-2tbQ for linux-arm-kernel@lists.infradead.org; Fri, 04 Apr 2025 17:45:01 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 711C31515; Fri, 4 Apr 2025 10:45:02 -0700 (PDT) Received: from lakrids.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id E6FA93F63F; Fri, 4 Apr 2025 10:44:58 -0700 (PDT) From: Mark Rutland To: linux-arm-kernel@lists.infradead.org Cc: ardb@kernel.org, broonie@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, maz@kernel.org, will@kernel.org Subject: [PATCH 09/14] arm64/fpsimd: Reset FPMR upon exec() Date: Fri, 4 Apr 2025 18:44:30 +0100 Message-Id: <20250404174435.3288106-10-mark.rutland@arm.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20250404174435.3288106-1-mark.rutland@arm.com> References: <20250404174435.3288106-1-mark.rutland@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250404_104500_783186_677A96C6 X-CRM114-Status: GOOD ( 10.25 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org An exec() is expected to reset all FPSIMD/SVE/SME state, and barring special handling of the vector lengths, the state is expected to reset to zero. This reset is handled in fpsimd_flush_thread(), which the core exec() code calls via flush_thread(). When support was added for FPMR, no logic was added to fpsimd_flush_thread() to reset the FPMR value, and thus it is erroneously inherited across an exec(). Add the missing reset of FPMR. Fixes: 203f2b95a882dc46 ("arm64/fpsimd: Support FEAT_FPMR") Signed-off-by: Mark Rutland Cc: Catalin Marinas Cc: Marc Zyngier Cc: Mark Brown Cc: Will Deacon Reviewed-by: Mark Brown --- arch/arm64/kernel/fpsimd.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c index 4a0b0bb3a3fad..0b6fda5b7bad5 100644 --- a/arch/arm64/kernel/fpsimd.c +++ b/arch/arm64/kernel/fpsimd.c @@ -1638,6 +1638,9 @@ void fpsimd_flush_thread(void) current->thread.svcr = 0; } + if (system_supports_fpmr()) + current->thread.uw.fpmr = 0; + current->thread.fp_type = FP_STATE_FPSIMD; put_cpu_fpsimd_context(); From patchwork Fri Apr 4 17:44:31 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Rutland X-Patchwork-Id: 14038719 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 31AD0C36010 for ; Fri, 4 Apr 2025 18:05:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=puWYV2cYc6F9+qW9ayvR+1RoNKwykAZgctnTwTUn+Lg=; b=wbmXgh3zu28YjF3qeg2LBQpaQ/ Cx361pu1MoJ9wDEgltOKmgYVKO+zxSG4S4OCsRLz5DAHU7RoOz+gFDb72T17IRD+tlhkHTVFicXUz QQD53Fw2Midnq0D96PAzFeCqseah/5laPmoI0Za3MgYJ3ZXH/ZOIfpenSB2mF8DxUDLr51RFBd468 iSagB13XhjY+Sgk0gqkuey9vRKdFPXByPk3HxmqR2Fb1yP5Upe11K3/Nxbq4hoN9WI6MfTKXhHbWY WWLx/KGRpXUenf089JeFP+l/Sbu7ujA7niqmcMf0rI9QYwwxs/tglnAXLvIGiQq4lguNnu2aoLIpS pm2JreeQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98.1 #2 (Red Hat Linux)) id 1u0lPt-0000000CUOJ-0fv5; Fri, 04 Apr 2025 18:05:21 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.98.1 #2 (Red Hat Linux)) id 1u0l6E-0000000CRRp-2Jk1 for linux-arm-kernel@lists.infradead.org; Fri, 04 Apr 2025 17:45:03 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 650E41515; Fri, 4 Apr 2025 10:45:04 -0700 (PDT) Received: from lakrids.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id E388C3F63F; Fri, 4 Apr 2025 10:45:00 -0700 (PDT) From: Mark Rutland To: linux-arm-kernel@lists.infradead.org Cc: ardb@kernel.org, broonie@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, maz@kernel.org, will@kernel.org Subject: [PATCH 10/14] arm64/fpsimd: Fix merging of FPSIMD state during signal return Date: Fri, 4 Apr 2025 18:44:31 +0100 Message-Id: <20250404174435.3288106-11-mark.rutland@arm.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20250404174435.3288106-1-mark.rutland@arm.com> References: <20250404174435.3288106-1-mark.rutland@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250404_104502_730583_E3DE450A X-CRM114-Status: GOOD ( 17.18 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org For backwards compatibility reasons, when a signal return occurs which restores SVE state, the effective lower 128 bits of each of the SVE vector registers are restored from the corresponding FPSIMD vector register in the FPSIMD signal frame, overriding the values in the SVE signal frame. This is intended to be the case regardless of streaming mode. To make this happen, restore_sve_fpsimd_context() uses fpsimd_update_current_state() to merge the lower 128 bits from the FPSIMD signal frame into the SVE register state. Unfortunately, fpsimd_update_current_state() performs this merging dependent upon TIF_SVE, which is not always correct for streaming SVE register state: * When restoring non-streaming SVE register state there is no observable problem, as the signal return code configures TIF_SVE and the saved fp_type to match before calling fpsimd_update_current_state(), which observes either: - TIF_SVE set AND fp_type == FP_STATE_SVE - TIF_SVE clear AND fp_type == FP_STATE_FPSIMD * On systems which have SME but not SVE, TIF_SVE cannot be set. Thus the merging will never happen for the streaming SVE register state. * On systems which have SVE and SME, TIF_SVE can be set and cleared independently of PSTATE.SM. Thus the merging may or may not happen for streaming SVE register state. As TIF_SVE can be cleared non-deterministically during syscalls (including at the start of sigreturn()), the merging may occur non-deterministically from the perspective of userspace. This logic has been broken since its introduction in commit: 85ed24dad2904f7c ("arm64/sme: Implement streaming SVE signal handling") ... at which point both fpsimd_signal_preserve_current_state() and fpsimd_update_current_state() only checked TIF SVE. When PSTATE.SM==1 and TIF_SVE was clear, signal delivery would place stale FPSIMD state into the FPSIMD signal frame, and signal return would not merge this into the restored register state. Subsequently, signal delivery was fixed as part of commit: 61da7c8e2a602f66 ("arm64/signal: Don't assume that TIF_SVE means we saved SVE state") ... but signal restore was not given a corresponding fix, and when TIF_SVE was clear, signal restore would still fail to merge the FPSIMD state into the restored SVE register state. The 'Fixes' tag did not indicate that this had been broken since its introduction. Fix this by merging the FPSIMD state dependent upon the saved fp_type, matching what we (currently) do during signal delivery. As described above, when backporting this commit, it will also be necessary to backport commit: 61da7c8e2a602f66 ("arm64/signal: Don't assume that TIF_SVE means we saved SVE state") ... and prior to commit: baa8515281b30861 ("arm64/fpsimd: Track the saved FPSIMD state type separately to TIF_SVE") ... it will be necessary for fpsimd_signal_preserve_current_state() and fpsimd_update_current_state() to consider both TIF_SVE and thread_sm_enabled(¤t->thread), in place of the saved fp_type. Fixes: 85ed24dad2904f7c ("arm64/sme: Implement streaming SVE signal handling") Signed-off-by: Mark Rutland Cc: Catalin Marinas Cc: Marc Zyngier Cc: Mark Brown Cc: Will Deacon --- arch/arm64/kernel/fpsimd.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c index 0b6fda5b7bad5..11f21809d3b7d 100644 --- a/arch/arm64/kernel/fpsimd.c +++ b/arch/arm64/kernel/fpsimd.c @@ -1781,7 +1781,7 @@ void fpsimd_update_current_state(struct user_fpsimd_state const *state) get_cpu_fpsimd_context(); current->thread.uw.fpsimd_state = *state; - if (test_thread_flag(TIF_SVE)) + if (current->thread.fp_type == FP_STATE_SVE) fpsimd_to_sve(current); task_fpsimd_load(); From patchwork Fri Apr 4 17:44:32 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Rutland X-Patchwork-Id: 14038720 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 716E7C36010 for ; Fri, 4 Apr 2025 18:07:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=63IAw1jLZgb86/bQzlqbOKnYntiakT87A/FbR/kbNHE=; b=1FhzdjEaPGj5vEX88S4stCsGYD S8MN/anhVsm0GYV9VyHqzpgSecgUcSgRKq8Ktsh6Nv7HoaRujYji1FSr+/n2WtYgQPN7dIpkCpFyp BEPQ6W02bPJjyCvV7wNlHsfQsjzhTnUp7H0PM0iQCz3wRt1AYUhtLoQIYIuZk5Ts/LoRch6L2rz0J Vqwecte/J7XwWg82JlyFHS6GIMCDspZ1V1LZGq+ynSUELE3PRiV6CETi5Xdxqo9+UDdjZrgytlkQb H1Hyg/YVsoI0opOoujBovx1AwXv62hMoc2i+brSSkukgQZL3pF9A2+lLRRDDYh0OqaseYT5m5pREh /gWfHQtw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98.1 #2 (Red Hat Linux)) id 1u0lRg-0000000CUbV-3r5P; Fri, 04 Apr 2025 18:07:12 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.98.1 #2 (Red Hat Linux)) id 1u0l6G-0000000CRSg-2qkM for linux-arm-kernel@lists.infradead.org; Fri, 04 Apr 2025 17:45:05 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 6B1161515; Fri, 4 Apr 2025 10:45:06 -0700 (PDT) Received: from lakrids.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id E843B3F63F; Fri, 4 Apr 2025 10:45:02 -0700 (PDT) From: Mark Rutland To: linux-arm-kernel@lists.infradead.org Cc: ardb@kernel.org, broonie@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, maz@kernel.org, will@kernel.org Subject: [PATCH 11/14] arm64/fpsimd: Add fpsimd_save_and_flush_current_state() Date: Fri, 4 Apr 2025 18:44:32 +0100 Message-Id: <20250404174435.3288106-12-mark.rutland@arm.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20250404174435.3288106-1-mark.rutland@arm.com> References: <20250404174435.3288106-1-mark.rutland@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250404_104504_764752_4C543F97 X-CRM114-Status: GOOD ( 13.41 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org When the current task's FPSIMD/SVE/SME state may be live on *any* CPU in the system, special care must be taken when manipulating that state, as this manipulation can race with preemption and/or asynchronous usage of FPSIMD/SVE/SME (e.g. kernel-mode NEON in softirq handlers). Even when manipulation is is protected with get_cpu_fpsimd_context() and get_cpu_fpsimd_context(), the logic necessary when the state is live on the current CPU can be wildly different from the logic necessary when the state is not live on the current CPU. A number of historical and extant issues result from failing to handle these cases consistetntly and/or correctly. To make it easier to get such manipulation correct, add a new fpsimd_save_and_flush_current_state() helper function, which ensures that the current task's state has been saved to memory and any stale state on any CPU has been "flushed" such that is not live on any CPU in the system. This will allow code to safely manipulate the saved state without risk of races. Subsequent patches will use the new function. Signed-off-by: Mark Rutland Cc: Catalin Marinas Cc: Marc Zyngier Cc: Mark Brown Cc: Will Deacon --- arch/arm64/include/asm/fpsimd.h | 1 + arch/arm64/kernel/fpsimd.c | 11 +++++++++++ 2 files changed, 12 insertions(+) diff --git a/arch/arm64/include/asm/fpsimd.h b/arch/arm64/include/asm/fpsimd.h index f2a84efc36185..3f2f8e7b2b7e5 100644 --- a/arch/arm64/include/asm/fpsimd.h +++ b/arch/arm64/include/asm/fpsimd.h @@ -97,6 +97,7 @@ struct cpu_fp_state { extern void fpsimd_bind_state_to_cpu(struct cpu_fp_state *fp_state); extern void fpsimd_flush_task_state(struct task_struct *target); +extern void fpsimd_save_and_flush_current_state(void); extern void fpsimd_save_and_flush_cpu_state(void); static inline bool thread_sm_enabled(struct thread_struct *thread) diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c index 11f21809d3b7d..ea07c4577f17e 100644 --- a/arch/arm64/kernel/fpsimd.c +++ b/arch/arm64/kernel/fpsimd.c @@ -1819,6 +1819,17 @@ void fpsimd_flush_task_state(struct task_struct *t) barrier(); } +void fpsimd_save_and_flush_current_state(void) +{ + if (!system_supports_fpsimd()) + return; + + get_cpu_fpsimd_context(); + fpsimd_save_user_state(); + fpsimd_flush_task_state(current); + put_cpu_fpsimd_context(); +} + /* * Save the FPSIMD state to memory and invalidate cpu view. * This function must be called with preemption disabled. From patchwork Fri Apr 4 17:44:33 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Rutland X-Patchwork-Id: 14038721 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id BA67FC3601A for ; Fri, 4 Apr 2025 18:09:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=E/G9uDnqvO/SNte/vpWdizBSe8394GiUY5FDFE6yDmU=; b=PLKOKM73gVjFjVRodSxmH1FVkh 0qcgC7KLXgdACDvTjMNaiFuWNeITMNllKyFnIAEGAbZAAlvymhPqgnWEfJZsnDt7WCnCfZofq7XKw 77pzKBBKvbnfpWKlZBO/aey8Kwv7vZ6A9k6/lKX8iIzKIeQKEB3VHKqRYJpyaG3SvGAUR6W8lKt9l He92bakMCmAeAzrUZn/oKeViq6d4T8HpES9C9cCY23ekMyetpdkGglkGUkeOOUXryD9+cUAJMJyMh E5gBuTchdtSvkjbSX8/tL1ynEMTj+afRUThZFcYKGsLE4ZMv/BjLmCrDvP4LVP7bticRJ4e6uHNNZ ZjcjYJ9Q==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98.1 #2 (Red Hat Linux)) id 1u0lTT-0000000CUqy-2r35; Fri, 04 Apr 2025 18:09:03 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.98.1 #2 (Red Hat Linux)) id 1u0l6I-0000000CRTL-2FbB for linux-arm-kernel@lists.infradead.org; Fri, 04 Apr 2025 17:45:07 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 6DF5D1515; Fri, 4 Apr 2025 10:45:08 -0700 (PDT) Received: from lakrids.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id EC4093F63F; Fri, 4 Apr 2025 10:45:04 -0700 (PDT) From: Mark Rutland To: linux-arm-kernel@lists.infradead.org Cc: ardb@kernel.org, broonie@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, maz@kernel.org, will@kernel.org Subject: [PATCH 12/14] arm64/fpsimd: signal32: Always save+flush state early Date: Fri, 4 Apr 2025 18:44:33 +0100 Message-Id: <20250404174435.3288106-13-mark.rutland@arm.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20250404174435.3288106-1-mark.rutland@arm.com> References: <20250404174435.3288106-1-mark.rutland@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250404_104506_621018_9DE6D988 X-CRM114-Status: GOOD ( 16.00 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org There are several issues with the way the native signal handling code manipulates FPSIMD/SVE/SME state. To fix those issues, subsequent patches will rework the native signal handling code to always save+flush the current task's FPSIMD/SVE/SME state before manipulating that state. In preparation for those changes, rework the compat signal handling code to save+flush the current task's FPSIMD state before manipulating it. Subsequent patches will remove fpsimd_signal_preserve_current_state() and fpsimd_update_current_state(). Compat tasks can only have FPSIMD state, and cannot have any SVE or SME state. Thus, the SVE state manipulation present in fpsimd_signal_preserve_current_state() and fpsimd_update_current_state() is not necessary, and it is safe to directly manipulate current->thread.uw.fpsimd_state once it has been saved+flushed. Use fpsimd_save_and_flush_current_state() to save+flush the state for both signal delivery and signal return, before the state is manipulated in any way. While it would be safe for compat_restore_vfp_context() to use fpsimd_flush_task_state(current), there are several extant issues in the native signal code resulting from incorrect use of fpsimd_flush_task_state(), and for consistency it is preferable to use fpsimd_save_and_flush_current_state(). Signed-off-by: Mark Rutland Cc: Catalin Marinas Cc: Marc Zyngier Cc: Mark Brown Cc: Will Deacon --- arch/arm64/kernel/signal32.c | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/arch/arm64/kernel/signal32.c b/arch/arm64/kernel/signal32.c index 81e798b6dadaf..bb3b526ff43f7 100644 --- a/arch/arm64/kernel/signal32.c +++ b/arch/arm64/kernel/signal32.c @@ -103,7 +103,7 @@ static int compat_preserve_vfp_context(struct compat_vfp_sigframe __user *frame) * Note that this also saves V16-31, which aren't visible * in AArch32. */ - fpsimd_signal_preserve_current_state(); + fpsimd_save_and_flush_current_state(); /* Place structure header on the stack */ __put_user_error(magic, &frame->magic, err); @@ -169,14 +169,17 @@ static int compat_restore_vfp_context(struct compat_vfp_sigframe __user *frame) fpsimd.fpsr = fpscr & VFP_FPSCR_STAT_MASK; fpsimd.fpcr = fpscr & VFP_FPSCR_CTRL_MASK; + if (err) + return -EFAULT; + /* * We don't need to touch the exception register, so * reload the hardware state. */ - if (!err) - fpsimd_update_current_state(&fpsimd); + fpsimd_save_and_flush_current_state(); + current->thread.uw.fpsimd_state = fpsimd; - return err ? -EFAULT : 0; + return 0; } static int compat_restore_sigframe(struct pt_regs *regs, From patchwork Fri Apr 4 17:44:34 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Rutland X-Patchwork-Id: 14038722 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 71B59C36010 for ; Fri, 4 Apr 2025 18:11:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=OUBzWpHZMGMbqRNBBK6RP2xMbtJyzuBev2UAPFJCWVo=; b=yNVjch4iRiRHAUnACih2FoB68i L5jTK0DCgdlhU4zgoVSbEqh38S0YNE4MHa3Hv30yoqUfN3psSZr8yirG/i5ESj6UfyqB75BCBKTUq oT96xVG5AHviZgjUjvhc6JkqgDEjEwyhNJgLi+jKvoYQfhQGSO6xcXukiSmNzh20KE42gIe0eFiPq xiTZkJfr14CjjZ6vs3SNoCglbHuMOKon7mQ151/3NMvYc4tdCpvrlw/HO4UMxNndcW7VlzfPNl0iO dVCRyBPusA1lB+ReX13is6DSA+9TRcWWzDDA0lyTJ0WPhLmZyj3UKzzOaGw7ZX2Xy0NRXr0QcX2Kl oXLVd0ow==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98.1 #2 (Red Hat Linux)) id 1u0lVG-0000000CV0r-1QFY; Fri, 04 Apr 2025 18:10:54 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.98.1 #2 (Red Hat Linux)) id 1u0l6M-0000000CRTn-0Cp4 for linux-arm-kernel@lists.infradead.org; Fri, 04 Apr 2025 17:45:11 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id EB1531515; Fri, 4 Apr 2025 10:45:11 -0700 (PDT) Received: from lakrids.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 762AE3F63F; Fri, 4 Apr 2025 10:45:08 -0700 (PDT) From: Mark Rutland To: linux-arm-kernel@lists.infradead.org Cc: ardb@kernel.org, broonie@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, maz@kernel.org, will@kernel.org Subject: [PATCH 13/14] arm64/fpsimd: signal: Always save+flush state early Date: Fri, 4 Apr 2025 18:44:34 +0100 Message-Id: <20250404174435.3288106-14-mark.rutland@arm.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20250404174435.3288106-1-mark.rutland@arm.com> References: <20250404174435.3288106-1-mark.rutland@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250404_104510_187210_074F7C6C X-CRM114-Status: GOOD ( 31.98 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org There are several issues with the way the native signal handling code manipulates FPSIMD/SVE/SME state, described in detail below. These issues largely result from races with preemption and inconsistent handling of live state vs saved state. Known issues with native FPSIMD/SVE/SME state management include: * On systems with FPMR, the code to save/restore the FPMR accesses the register while it is not owned by the current task. Consequently, this may corrupt the FPMR of the current task and/or may corrupt the FPMR of an unrelated task. The FPMR save/restore has been broken since it was introduced in commit: 8c46def44409fc91 ("arm64/signal: Add FPMR signal handling") * On systems with SME, setup_return() modifies both the live register state and the saved state register state regardless of whether the task's state is live, and without holding the cpu fpsimd context. Consequently: - This may corrupt the state an unrelated task which has PSTATE.SM set and/or PSTATE.ZA set. - The task may enter the signal handler in streaming mode, and or with ZA storage enabled unexpectedly. - The task may enter the signal handler in non-streaming SVE mode with stale SVE register state, which may have been inherited from streaming SVE mode unexpectedly. Where the streaming and non-streaming vector lengths differ, this may be packed into registers arbitrarily. This logic has been broken since it was introduced in commit: 40a8e87bb32855b3 ("arm64/sme: Disable ZA and streaming mode when handling signals") Further incorrect manipulation of state was added in commits: ea64baacbc36a0d5 ("arm64/signal: Flush FPSIMD register state when disabling streaming mode") baa8515281b30861 ("arm64/fpsimd: Track the saved FPSIMD state type separately to TIF_SVE") * Several restoration functions use fpsimd_flush_task_state() to discard the live FPSIMD/SVE/SME while the in-memory copy is stale. When a subset of the FPSIMD/SVE/SME state is restored, the remainder may be non-deterministically reset to a stale snapshot from some arbitrary point in the past. This non-deterministic discarding was introduced in commit: 8cd969d28fd2848d ("arm64/sve: Signal handling support") As of that commit, when TIF_SVE was initially clear, failure to restore the SVE signal frame could reset the FPSIMD registers to a stale snapshot. The pattern of discarding unsaved state was subsequently copied into restoration functions for some new state in commits: 39782210eb7e8763 ("arm64/sme: Implement ZA signal handling") ee072cf708048c0d ("arm64/sme: Implement signal handling for ZT") * On systems with SME/SME2, the entire FPSIMD/SVE/SME state may be loaded onto the CPU redundantly. Either restore_fpsimd_context() or restore_sve_fpsimd_context() will load the entire FPSIMD/SVE/SME state via fpsimd_update_current_state() before restore_za_context() and restore_zt_context() each discard the state via fpsimd_flush_task_state(). This is purely redundant work, and not a functional bug. To fix these issues, rework the native signal handling code to always save+flush the current task's FPSIMD/SVE/SME state before manipulating that state. This avoids races with preemption and ensures that state is manipulated consistently regardless of whether it happened to be live prior to manipulation. This largely involes: * Using fpsimd_save_and_flush_current_state() to save+flush the state for both signal delivery and signal return, before the state is manipulated in any way. * Removing fpsimd_signal_preserve_current_state() and updating preserve_fpsimd_context() to explicitly ensure that the FPSIMD state is up-to-date, as preserve_fpsimd_context() is the only consumer of the FPSIMD state during signal delivery. * Modifying fpsimd_update_current_state() to not reload the FPSIMD state onto the CPU. Ideally we'd remove fpsimd_update_current_state() entirely, but I've left that for subsequent patches as there are a number of of other problems with the FPSIMD<->SVE conversion helpers that should be addressed at the same time. For now I've removed the misleading comment. For setup_return(), we need to decide (for ABI reasons) whether signal delivery should have all the side-effects of an SMSTOP. For now I've left a TODO comment, as there are other questions in this area that I'll address with subsequent patches. Fixes: 8c46def44409fc91 ("arm64/signal: Add FPMR signal handling") Fixes: 40a8e87bb32855b3 ("arm64/sme: Disable ZA and streaming mode when handling signals") Fixes: ea64baacbc36a0d5 ("arm64/signal: Flush FPSIMD register state when disabling streaming mode") Fixes: baa8515281b30861 ("arm64/fpsimd: Track the saved FPSIMD state type separately to TIF_SVE") Fixes: 8cd969d28fd2848d ("arm64/sve: Signal handling support") Fixes: 39782210eb7e8763 ("arm64/sme: Implement ZA signal handling") Fixes: ee072cf708048c0d ("arm64/sme: Implement signal handling for ZT") Signed-off-by: Mark Rutland Cc: Catalin Marinas Cc: Marc Zyngier Cc: Mark Brown Cc: Will Deacon --- arch/arm64/include/asm/fpsimd.h | 1 - arch/arm64/kernel/fpsimd.c | 28 -------------- arch/arm64/kernel/signal.c | 66 ++++++--------------------------- 3 files changed, 12 insertions(+), 83 deletions(-) diff --git a/arch/arm64/include/asm/fpsimd.h b/arch/arm64/include/asm/fpsimd.h index 3f2f8e7b2b7e5..ef2d921181f67 100644 --- a/arch/arm64/include/asm/fpsimd.h +++ b/arch/arm64/include/asm/fpsimd.h @@ -76,7 +76,6 @@ extern void fpsimd_load_state(struct user_fpsimd_state *state); extern void fpsimd_thread_switch(struct task_struct *next); extern void fpsimd_flush_thread(void); -extern void fpsimd_signal_preserve_current_state(void); extern void fpsimd_preserve_current_state(void); extern void fpsimd_restore_current_state(void); extern void fpsimd_update_current_state(struct user_fpsimd_state const *state); diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c index ea07c4577f17e..b0874402f7ecc 100644 --- a/arch/arm64/kernel/fpsimd.c +++ b/arch/arm64/kernel/fpsimd.c @@ -1662,18 +1662,6 @@ void fpsimd_preserve_current_state(void) put_cpu_fpsimd_context(); } -/* - * Like fpsimd_preserve_current_state(), but ensure that - * current->thread.uw.fpsimd_state is updated so that it can be copied to - * the signal frame. - */ -void fpsimd_signal_preserve_current_state(void) -{ - fpsimd_preserve_current_state(); - if (current->thread.fp_type == FP_STATE_SVE) - sve_to_fpsimd(current); -} - /* * Associate current's FPSIMD context with this cpu * The caller must have ownership of the cpu FPSIMD context before calling @@ -1766,30 +1754,14 @@ void fpsimd_restore_current_state(void) put_cpu_fpsimd_context(); } -/* - * Load an updated userland FPSIMD state for 'current' from memory and set the - * flag that indicates that the FPSIMD register contents are the most recent - * FPSIMD state of 'current'. This is used by the signal code to restore the - * register state when returning from a signal handler in FPSIMD only cases, - * any SVE context will be discarded. - */ void fpsimd_update_current_state(struct user_fpsimd_state const *state) { if (WARN_ON(!system_supports_fpsimd())) return; - get_cpu_fpsimd_context(); - current->thread.uw.fpsimd_state = *state; if (current->thread.fp_type == FP_STATE_SVE) fpsimd_to_sve(current); - - task_fpsimd_load(); - fpsimd_bind_task_to_cpu(); - - clear_thread_flag(TIF_FOREIGN_FPSTATE); - - put_cpu_fpsimd_context(); } /* diff --git a/arch/arm64/kernel/signal.c b/arch/arm64/kernel/signal.c index 40c572869325b..b27357aeba065 100644 --- a/arch/arm64/kernel/signal.c +++ b/arch/arm64/kernel/signal.c @@ -250,6 +250,8 @@ static int preserve_fpsimd_context(struct fpsimd_context __user *ctx) ¤t->thread.uw.fpsimd_state; int err; + sve_sync_to_fpsimd(current); + /* copy the FP and status/control registers */ err = __copy_to_user(ctx->vregs, fpsimd->vregs, sizeof(fpsimd->vregs)); __put_user_error(fpsimd->fpsr, &ctx->fpsr, err); @@ -291,8 +293,6 @@ static int preserve_fpmr_context(struct fpmr_context __user *ctx) { int err = 0; - current->thread.uw.fpmr = read_sysreg_s(SYS_FPMR); - __put_user_error(FPMR_MAGIC, &ctx->head.magic, err); __put_user_error(sizeof(*ctx), &ctx->head.size, err); __put_user_error(current->thread.uw.fpmr, &ctx->fpmr, err); @@ -310,7 +310,7 @@ static int restore_fpmr_context(struct user_ctxs *user) __get_user_error(fpmr, &user->fpmr->fpmr, err); if (!err) - write_sysreg_s(fpmr, SYS_FPMR); + current->thread.uw.fpmr = fpmr; return err; } @@ -372,11 +372,6 @@ static int preserve_sve_context(struct sve_context __user *ctx) err |= __copy_to_user(&ctx->__reserved, reserved, sizeof(reserved)); if (vq) { - /* - * This assumes that the SVE state has already been saved to - * the task struct by calling the function - * fpsimd_signal_preserve_current_state(). - */ err |= __copy_to_user((char __user *)ctx + SVE_SIG_REGS_OFFSET, current->thread.sve_state, SVE_SIG_REGS_SIZE(vq)); @@ -432,16 +427,6 @@ static int restore_sve_fpsimd_context(struct user_ctxs *user) if (user->sve_size < SVE_SIG_CONTEXT_SIZE(vq)) return -EINVAL; - /* - * Careful: we are about __copy_from_user() directly into - * thread.sve_state with preemption enabled, so protection is - * needed to prevent a racing context switch from writing stale - * registers back over the new data. - */ - - fpsimd_flush_task_state(current); - /* From now, fpsimd_thread_switch() won't touch thread.sve_state */ - sve_alloc(current, true); if (!current->thread.sve_state) return -ENOMEM; @@ -539,11 +524,6 @@ static int preserve_za_context(struct za_context __user *ctx) err |= __copy_to_user(&ctx->__reserved, reserved, sizeof(reserved)); if (vq) { - /* - * This assumes that the ZA state has already been saved to - * the task struct by calling the function - * fpsimd_signal_preserve_current_state(). - */ err |= __copy_to_user((char __user *)ctx + ZA_SIG_REGS_OFFSET, current->thread.sme_state, ZA_SIG_REGS_SIZE(vq)); @@ -578,16 +558,6 @@ static int restore_za_context(struct user_ctxs *user) if (user->za_size < ZA_SIG_CONTEXT_SIZE(vq)) return -EINVAL; - /* - * Careful: we are about __copy_from_user() directly into - * thread.sme_state with preemption enabled, so protection is - * needed to prevent a racing context switch from writing stale - * registers back over the new data. - */ - - fpsimd_flush_task_state(current); - /* From now, fpsimd_thread_switch() won't touch thread.sve_state */ - sme_alloc(current, true); if (!current->thread.sme_state) { current->thread.svcr &= ~SVCR_ZA_MASK; @@ -625,11 +595,6 @@ static int preserve_zt_context(struct zt_context __user *ctx) BUILD_BUG_ON(sizeof(ctx->__reserved) != sizeof(reserved)); err |= __copy_to_user(&ctx->__reserved, reserved, sizeof(reserved)); - /* - * This assumes that the ZT state has already been saved to - * the task struct by calling the function - * fpsimd_signal_preserve_current_state(). - */ err |= __copy_to_user((char __user *)ctx + ZT_SIG_REGS_OFFSET, thread_zt_state(¤t->thread), ZT_SIG_REGS_SIZE(1)); @@ -655,16 +620,6 @@ static int restore_zt_context(struct user_ctxs *user) if (nregs != 1) return -EINVAL; - /* - * Careful: we are about __copy_from_user() directly into - * thread.zt_state with preemption enabled, so protection is - * needed to prevent a racing context switch from writing stale - * registers back over the new data. - */ - - fpsimd_flush_task_state(current); - /* From now, fpsimd_thread_switch() won't touch ZT in thread state */ - err = __copy_from_user(thread_zt_state(¤t->thread), (char __user const *)user->zt + ZT_SIG_REGS_OFFSET, @@ -1015,6 +970,8 @@ static int restore_sigframe(struct pt_regs *regs, */ forget_syscall(regs); + fpsimd_save_and_flush_current_state(); + err |= !valid_user_regs(®s->user_regs, current); if (err == 0) err = parse_user_sigframe(&user, sf); @@ -1508,8 +1465,11 @@ static int setup_return(struct pt_regs *regs, struct ksignal *ksig, /* * If we were in streaming mode the saved register * state was SVE but we will exit SM and use the - * FPSIMD register state - flush the saved FPSIMD - * register state in case it gets loaded. + * FPSIMD register state. + * + * TODO: decide if this should behave as SMSTOP (e.g. reset + * FPSR + FPMR), or whether this should only clear the scalable + * registers + ZA state. */ if (current->thread.svcr & SVCR_SM_MASK) { memset(¤t->thread.uw.fpsimd_state, 0, @@ -1517,9 +1477,7 @@ static int setup_return(struct pt_regs *regs, struct ksignal *ksig, current->thread.fp_type = FP_STATE_FPSIMD; } - current->thread.svcr &= ~(SVCR_ZA_MASK | - SVCR_SM_MASK); - sme_smstop(); + current->thread.svcr &= ~(SVCR_ZA_MASK | SVCR_SM_MASK); } return 0; @@ -1533,7 +1491,7 @@ static int setup_rt_frame(int usig, struct ksignal *ksig, sigset_t *set, struct user_access_state ua_state; int err = 0; - fpsimd_signal_preserve_current_state(); + fpsimd_save_and_flush_current_state(); if (get_sigframe(&user, ksig, regs)) return 1; From patchwork Fri Apr 4 17:44:35 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Rutland X-Patchwork-Id: 14038731 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 83911C369A1 for ; Fri, 4 Apr 2025 18:12:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=o/Cs8lYUGOH5SnjCmGX9zMXwzw+sPq2MDvbljJYlujY=; b=jaZntEwbrst2kux1RR7QfdFcVR +wg3FIvOt/Id8erEnHRqfuhFQCStLCEAcoXlZqIfWl/cBKZG1qrhnvkna0mcBf2I6wbq8kT7WRTrz jc+PaMFyvDV+GPTBfhv/laRc6mHnVMyixSTgXl59D6YPakgAHzKLCC8LnbbzBz6ASQIC0FE4w+0xz H9DkOgVzcMk9mUAes4mLwOn+JZ+xVO4i1dqxzw5OZRMuNsBZRGgvn7Q8D5mSSC3VjK6GuIqHQVEj3 V5WjXJc9WFwLbEGipjOfTci3WL3RxA3HNLnXvMHHPZhaA8hIGKQ+k0f6aXcozl0Oq5cbnPdjSf/MV pVEXnXKg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98.1 #2 (Red Hat Linux)) id 1u0lX3-0000000CV8C-0Vg8; Fri, 04 Apr 2025 18:12:45 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.98.1 #2 (Red Hat Linux)) id 1u0l6O-0000000CRUR-1kCt for linux-arm-kernel@lists.infradead.org; Fri, 04 Apr 2025 17:45:13 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 2BD661515; Fri, 4 Apr 2025 10:45:14 -0700 (PDT) Received: from lakrids.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id AB3EA3F63F; Fri, 4 Apr 2025 10:45:10 -0700 (PDT) From: Mark Rutland To: linux-arm-kernel@lists.infradead.org Cc: ardb@kernel.org, broonie@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, maz@kernel.org, will@kernel.org Subject: [PATCH 14/14] arm64/fpsimd: signal: Simplify preserve_tpidr2_context() Date: Fri, 4 Apr 2025 18:44:35 +0100 Message-Id: <20250404174435.3288106-15-mark.rutland@arm.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20250404174435.3288106-1-mark.rutland@arm.com> References: <20250404174435.3288106-1-mark.rutland@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250404_104512_503290_49721E22 X-CRM114-Status: UNSURE ( 9.65 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org During a context-switch, tls_thread_switch() reads and writes a task's thread_struct::tpidr2_el0 field. Other code shouldn't access this field for an active task, as such accesses would form a data-race with a concurrent context-switch. The usage in preserve_tpidr2_context() is suspicious, but benign as any race with a context switch will write the same value back to current->thread.tpidr2_el0. Make this clearer and match restore_tpidr2_context() by using a temporary variable instead, avoiding the (benign) data-race. Signed-off-by: Mark Rutland Cc: Catalin Marinas Cc: Marc Zyngier Cc: Mark Brown Cc: Will Deacon Reviewed-by: Mark Brown --- arch/arm64/kernel/signal.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/arch/arm64/kernel/signal.c b/arch/arm64/kernel/signal.c index b27357aeba065..a4e3d4273a360 100644 --- a/arch/arm64/kernel/signal.c +++ b/arch/arm64/kernel/signal.c @@ -476,13 +476,12 @@ extern int preserve_sve_context(void __user *ctx); static int preserve_tpidr2_context(struct tpidr2_context __user *ctx) { + u64 tpidr2_el0 = read_sysreg_s(SYS_TPIDR2_EL0); int err = 0; - current->thread.tpidr2_el0 = read_sysreg_s(SYS_TPIDR2_EL0); - __put_user_error(TPIDR2_MAGIC, &ctx->head.magic, err); __put_user_error(sizeof(*ctx), &ctx->head.size, err); - __put_user_error(current->thread.tpidr2_el0, &ctx->tpidr2, err); + __put_user_error(tpidr2_el0, &ctx->tpidr2, err); return err; }