From patchwork Sun Jul 31 11:32:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 12933073 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 2FA81C00140 for ; Sun, 31 Jul 2022 11:34:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-Id:Date:Subject:Cc :To:From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References: List-Owner; bh=ULd8wcYp6awristfAaevwqT1CZUIKFGzxfpOBEXixcY=; b=CPwW51mseIQ3jD 8ujqAeAMVGG7BDRR4hldQjvK6nyqx2OGPo+O+xu4fSvaN/wJCUFNzChW2pFFC63fCOM953kUI96EO AQvlk8QOQq7KnJWHlh9+3enoRkd6NLjtCBA1aHVsa5Qdk6DG3kfB8owZmDsCEYYclCLU/yZNVvRWi tWaMVLj6OgAN5wiRwqx8XitsYcRQVdbuThr76ETi+oupRy1TnlhTc6KXb/qkZcTlxXZaWFQN4kmdF RMEFqRvxGA3vU3MWQg5nUtFT182b994rKQthIEhYShObupuwfhQfWJmgHlMiDC9fz86B83ly+XdDq Cg3LPca2m+8WuxqDRQ/g==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1oI7Bl-00Bhjf-3U; Sun, 31 Jul 2022 11:32:53 +0000 Received: from dfw.source.kernel.org ([139.178.84.217]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1oI7Bh-00Bhgk-Ef for linux-arm-kernel@lists.infradead.org; Sun, 31 Jul 2022 11:32:51 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 8041F60C85; Sun, 31 Jul 2022 11:32:47 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0DC8EC433C1; Sun, 31 Jul 2022 11:32:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1659267166; bh=fJz0OdVxH3TEbDpmHWJ300zAFqdMg+041dzG1XIn6e8=; h=From:To:Cc:Subject:Date:From; b=UXuX+JZFnYgR0yc+27KvRGIQicdscMSyL2M2vctsAudoF/aWUHUrwS2sEHYKD8ur+ r541k9+I8DMNJXg4TE7EyRuV59vewMGuuZG2YLw+8qTIPJnb5hPVnuFdo/lkvAzd8J KJMJRwueEF0kQDC9ezsfU0IeIJDvgp1izKpoFxtJnbEBjPnQXUmCAvZMiP8JgbN9yU PNbPHRmsaEBd5ljJ1iX71LJqHLF6KykhBwfUOG1nc5Nxd9IZYOwuiUao1cw5cRbtX+ X+oVGR6VcXYaCMxAjMP2GRIDVxiuGTMDNtoL3QM7zu2atlvUePawtNgf3WOkUNhoso aB1ky8s+yj1XQ== From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: linux@armlinux.org.uk, Ard Biesheuvel Subject: [PATCH] ARM: permit non-nested kernel mode NEON in softirq context Date: Sun, 31 Jul 2022 13:32:33 +0200 Message-Id: <20220731113233.883259-1-ardb@kernel.org> X-Mailer: git-send-email 2.35.1 MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=2344; h=from:subject; bh=fJz0OdVxH3TEbDpmHWJ300zAFqdMg+041dzG1XIn6e8=; b=owEB7QES/pANAwAKAcNPIjmS2Y8kAcsmYgBi5mhQGYm4MNcZpjA53sVATbTuAAEnh+PWiXb4e+zQ 6HgP9OGJAbMEAAEKAB0WIQT72WJ8QGnJQhU3VynDTyI5ktmPJAUCYuZoUAAKCRDDTyI5ktmPJLLpDA CmG7NIiQOzD8oz2D2sSAp8pdk2Vr1FsWH9aLYCT5fSEtkH/huLkXSVB9zcrB9Vvqah/dWThbFhX2xF SBV3GwGMhPryOMaJhTX5kK5yf5a1zNea2w3ojMJqOxjAhX5VRUqZmyMCliMFfl/OT8EcUmdizp7GIC MP+EIU98O7YmySIgY6O22ejZ3nDnSSTdZfy55wA74eXwxnIqgDSiLBXeL6J5FodutjQeOzShkrpDCt K0PdTSsa5d+I84YdE3o57ejy0F3rNU/bQYu5l92h7WuzOnF8U7Lka5kGeRJBlc4j78Jnw3U7TwBkn4 Qy6w6Wp9QsSxSh4MdEGLDbgffz4m/PAf7xHmFQ4z5VKBLamUrcM1tuMHZo78tWmJF1u1Fmfu+9Mkry NMT/C5Kofk7nBGLUsR7FbgtXYtt09WYS84NjwOitXBc2jGStzi1WEujU9Gb6PmTWKZv3Y/q5iZH5BD Gucuz8w3MI4QYHpZcJptuRJXW40k4glzxVEkJYl+sWG3w= X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220731_043249_607765_9E1C8BEF X-CRM114-Status: GOOD ( 18.68 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org We currently only permit kernel mode NEON in process context, to avoid the need to preserve/restore the NEON register file when taking an exception while running in the kernel. Like we did on arm64, we can relax this restriction substantially, by permitting kernel mode NEON from softirq context, while ensuring that softirq processing is disabled when the NEON is being used in kernel mode. This guarantees that only NEON context belonging to user space needs to be preserved and restored, which is already taken care of. This is especially relevant for network encryption, where incoming frames are typically handled in softirq context, and deferring software decryption to a kernel thread or falling back to C code are both undesirable from a performance PoV. Signed-off-by: Ard Biesheuvel --- arch/arm/include/asm/simd.h | 8 ++++++++ arch/arm/vfp/vfpmodule.c | 9 +++++---- 2 files changed, 13 insertions(+), 4 deletions(-) create mode 100644 arch/arm/include/asm/simd.h diff --git a/arch/arm/include/asm/simd.h b/arch/arm/include/asm/simd.h new file mode 100644 index 000000000000..82191dbd7e78 --- /dev/null +++ b/arch/arm/include/asm/simd.h @@ -0,0 +1,8 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +#include + +static __must_check inline bool may_use_simd(void) +{ + return IS_ENABLED(CONFIG_KERNEL_MODE_NEON) && !in_hardirq(); +} diff --git a/arch/arm/vfp/vfpmodule.c b/arch/arm/vfp/vfpmodule.c index 2cb355c1b5b7..753df7a70468 100644 --- a/arch/arm/vfp/vfpmodule.c +++ b/arch/arm/vfp/vfpmodule.c @@ -718,12 +718,13 @@ void kernel_neon_begin(void) u32 fpexc; /* - * Kernel mode NEON is only allowed outside of interrupt context + * Kernel mode NEON is only allowed outside of hardirq context * with preemption disabled. This will make sure that the kernel * mode NEON register contents never need to be preserved. */ - BUG_ON(in_interrupt()); - cpu = get_cpu(); + BUG_ON(in_hardirq()); + local_bh_disable(); + cpu = __smp_processor_id(); fpexc = fmrx(FPEXC) | FPEXC_EN; fmxr(FPEXC, fpexc); @@ -746,7 +747,7 @@ void kernel_neon_end(void) { /* Disable the NEON/VFP unit. */ fmxr(FPEXC, fmrx(FPEXC) & ~FPEXC_EN); - put_cpu(); + local_bh_enable(); } EXPORT_SYMBOL(kernel_neon_end);