From patchwork Thu Oct 21 18:07:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 12576207 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C3E3FC433EF for ; Thu, 21 Oct 2021 18:36:43 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 8B62C60E54 for ; Thu, 21 Oct 2021 18:36:43 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 8B62C60E54 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=1DoW3jXgb3SZa31CP//q0idHAfW+0G8kcJ1+MvrILiY=; b=dAcxQFdzMql/8W yoTSt0LLc9Diw+qSxKycXsY9+PqqZJyT/T+nNHRg/+ACPcZmUY7vE0sjtnYKDjtRz/dNeaNhOdXcs QllMAG2ku4FEYnDa6ZKMIUenhBahXRhJHsNSf/5UMkQe8+HgV+GEthPbCmJfqX47/Bak0YtgjTkmU I/NXmwoM5IZdDjEP67amjCwLzOqErp3/bV/gRyYNZzOxea09XAv3eLn8aLLheiUhbk3STENyhUHhk CnAMJZonzKl8WUHah2MRKMS58ObHC7kiCPrY7ViPEC3H5gxVI26H5ubgLVHvvfAFRTjE2Z4esJNEG KitzO4Qzn+FLGgHmFgOw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mdcu9-008njZ-Dw; Thu, 21 Oct 2021 18:35:06 +0000 Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mdcV9-008cfv-Bb for linux-arm-kernel@lists.infradead.org; Thu, 21 Oct 2021 18:09:16 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id 6F09F61B02; Thu, 21 Oct 2021 18:09:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1634839755; bh=9qDWgiu5vJ3aJ/SkyuYiO231kDNUJDdP3sBILOMUncg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=eL8/+el/unUzhurn9PyCJI4hRuiOgcLy5EsvUgpcLu1bZb9tgP2Mr0KYY+dcmLu3a 6lYP4TkLvJlYpxfox6NRvq0JJfY4DSZBZSNo35eYwljv/Evn1eTPHep0HoFTE19vzw tAa+vFplKjuMKrDVO2clrzd4eHx8g9f9bH7EhUnqUTx3BcGf9s7yLUnFYo1WQI34mH npRSQ3bTeS/q7V7d3DbbM7PDsZerjkMCFXrps6glDz+DWwpCIyizlfNxurKx5xdgrf dCI2gGxHH7Y/NfLL/Ah49WYK9jHQViEuQhRm2GRJITRbd/ahNEL3ja1UfJb9ByCwhd 5Y+TwVmV5re+g== From: Mark Brown To: Catalin Marinas , Will Deacon , Shuah Khan , Shuah Khan Cc: Alan Hayward , Luis Machado , Salil Akerkar , Basant Kumar Dwivedi , Szabolcs Nagy , linux-arm-kernel@lists.infradead.org, linux-kselftest@vger.kernel.org, Mark Brown Subject: [PATCH v4 24/33] arm64/sme: Save and restore streaming mode over EFI runtime calls Date: Thu, 21 Oct 2021 19:07:13 +0100 Message-Id: <20211021180722.3699248-25-broonie@kernel.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20211021180722.3699248-1-broonie@kernel.org> References: <20211021180722.3699248-1-broonie@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=3623; h=from:subject; bh=9qDWgiu5vJ3aJ/SkyuYiO231kDNUJDdP3sBILOMUncg=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBhcaxR6X08PDr9+9Kwl7X6H191zd1p/UaU+SrmnSQ5 hWnHV6mJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCYXGsUQAKCRAk1otyXVSH0GydB/ 9ZZYMV6/74FyhHoOz9UQUKS2TEFCaoOWAVHDS45mUwMs//ctc7/l2cvgON2aczhTRfijlezEN+zUy7 JDwtYFm8klfjc0WbZLpVHI3Swh7tRJuLhDQtdqCqTp271MET2FSxI6FFJg58bRKCzChO2QaAlk76wi yjxpwtvLonoghlP2dq4ARiqhi+33CzzIXpkux4B5uIeyz6+/Ym92l8K01adknQfxPLtUdNXYg/hn8N 92/ge7/OXCvfBj3ubHX6fcNMD0J+EGaTTXk26UKaKvttz+zwxhTfCV6ad4zqeSBM8/k0yl69M20zVN I/zkY2wa6FeCh+UcTovYRNAeRVhNem X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211021_110915_502297_5EE9DBD1 X-CRM114-Status: GOOD ( 17.24 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org When saving and restoring the floating point state over an EFI runtime call ensure that we handle streaming mode, only handling FFR if we are not in streaming mode and ensuring that we are in normal mode over the call into runtime services. We currently assume that ZA will not be modified by runtime services, the specification is not yet finalised so this may need updating if that changes. Signed-off-by: Mark Brown --- arch/arm64/kernel/fpsimd.c | 47 +++++++++++++++++++++++++++++++++----- 1 file changed, 41 insertions(+), 6 deletions(-) diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c index cdbc1fd8fde2..242afb78b000 100644 --- a/arch/arm64/kernel/fpsimd.c +++ b/arch/arm64/kernel/fpsimd.c @@ -1046,21 +1046,25 @@ int vec_verify_vq_map(enum vec_type type) static void __init sve_efi_setup(void) { - struct vl_info *info = &vl_info[ARM64_VEC_SVE]; + int max_vl = 0; + int i; if (!IS_ENABLED(CONFIG_EFI)) return; + for (i = 0; i < ARRAY_SIZE(vl_info); i++) + max_vl = max(vl_info[i].max_vl, max_vl); + /* * alloc_percpu() warns and prints a backtrace if this goes wrong. * This is evidence of a crippled system and we are returning void, * so no attempt is made to handle this situation here. */ - if (!sve_vl_valid(info->max_vl)) + if (!sve_vl_valid(max_vl)) goto fail; efi_sve_state = __alloc_percpu( - SVE_SIG_REGS_SIZE(sve_vq_from_vl(info->max_vl)), SVE_VQ_BYTES); + SVE_SIG_REGS_SIZE(sve_vq_from_vl(max_vl)), SVE_VQ_BYTES); if (!efi_sve_state) goto fail; @@ -1830,6 +1834,7 @@ EXPORT_SYMBOL(kernel_neon_end); static DEFINE_PER_CPU(struct user_fpsimd_state, efi_fpsimd_state); static DEFINE_PER_CPU(bool, efi_fpsimd_state_used); static DEFINE_PER_CPU(bool, efi_sve_state_used); +static DEFINE_PER_CPU(bool, efi_sm_state); /* * EFI runtime services support functions @@ -1864,12 +1869,28 @@ void __efi_fpsimd_begin(void) */ if (system_supports_sve() && likely(efi_sve_state)) { char *sve_state = this_cpu_ptr(efi_sve_state); + bool ffr = true; + u64 svcr; __this_cpu_write(efi_sve_state_used, true); + /* If we are in streaming mode don't touch FFR */ + if (system_supports_sme()) { + svcr = read_sysreg_s(SYS_SVCR_EL0); + + ffr = svcr & SYS_SVCR_EL0_SM_MASK; + + __this_cpu_write(efi_sm_state, ffr); + } + sve_save_state(sve_state + sve_ffr_offset(sve_max_vl()), &this_cpu_ptr(&efi_fpsimd_state)->fpsr, - true); + ffr); + + if (system_supports_sme()) + sysreg_clear_set_s(SYS_SVCR_EL0, + SYS_SVCR_EL0_SM_MASK, 0); + } else { fpsimd_save_state(this_cpu_ptr(&efi_fpsimd_state)); } @@ -1892,11 +1913,25 @@ void __efi_fpsimd_end(void) if (system_supports_sve() && likely(__this_cpu_read(efi_sve_state_used))) { char const *sve_state = this_cpu_ptr(efi_sve_state); + bool ffr = true; + + /* + * Restore streaming mode; EFI calls are + * normal function calls so should not return in + * streaming mode. + */ + if (system_supports_sme()) { + if (__this_cpu_read(efi_sm_state)) { + sysreg_clear_set_s(SYS_SVCR_EL0, + 0, + SYS_SVCR_EL0_SM_MASK); + ffr = false; + } + } - sve_set_vq(sve_vq_from_vl(sve_get_vl()) - 1); sve_load_state(sve_state + sve_ffr_offset(sve_max_vl()), &this_cpu_ptr(&efi_fpsimd_state)->fpsr, - true); + ffr); __this_cpu_write(efi_sve_state_used, false); } else {