From patchwork Sat Mar 10 15:21:56 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 10273659 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 5DA3A602BD for ; Sat, 10 Mar 2018 15:27:48 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4CCAD28BA5 for ; Sat, 10 Mar 2018 15:27:48 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 403C428BDE; Sat, 10 Mar 2018 15:27:48 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID autolearn=unavailable version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id A6B2728BA5 for ; Sat, 10 Mar 2018 15:27:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=wrjGv4ckUOkgn6EWd8n1hyqCTkfXPY+T/3j/k2W5thE=; b=TDzq0fjggkTKIAhecSRJKPbkg5 RNV1ogjBJjW015GmDp3IY/Njrc3YQ6T2xS1xGYmfB74L6WFq9k82jZZ42qTfLk3p0atUjPUzNd2zs pI9kHCUG0/RGw9wVYYsIuPLOlP9dr2dRhawEQ3yuIxhi051W/IAmrFpIwxbRj2LOxscS+o/2ai3jC Lsxt2p6CqdrW7+d0gLiDrBaDXwv9okwsDsC3lAhkSC7HXwQnmQzt/Hu4AHMUQhXk7kbXFs8wlRJjq V22DR6xCw+R1BgZFs6Fq7KfexmA6zgbhYtmflzviUMcshsXe9rzbrKI3hvzCLDfS23z3ip2G7q/QG 4qFylK7Q==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.89 #1 (Red Hat Linux)) id 1eugPK-0003Uy-VZ; Sat, 10 Mar 2018 15:27:39 +0000 Received: from mail-wm0-x241.google.com ([2a00:1450:400c:c09::241]) by bombadil.infradead.org with esmtps (Exim 4.89 #1 (Red Hat Linux)) id 1eugL3-0008Ci-At for linux-arm-kernel@lists.infradead.org; Sat, 10 Mar 2018 15:23:24 +0000 Received: by mail-wm0-x241.google.com with SMTP id a20so7234170wmd.1 for ; Sat, 10 Mar 2018 07:23:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=igH9KImZTLC7b3FEy2vhduXKBfEy6hhH3sJVJ6VEP8Q=; b=FVsvPWHitHJSBOohv0LlBjibo2VrwyfVHxbYh4IhXPj6AMPo41FiXMR2a8MlyBIYyU xKJgA5UQzEAHOAeVWumMh/oPI3S8vy+QeSuxNYdzc7rgXz9AXQfR5a/W9ZUNYTwnD/Qa NBTMgEPCUbHkANZHWDzeqJql6L01j2f+pOWPk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=igH9KImZTLC7b3FEy2vhduXKBfEy6hhH3sJVJ6VEP8Q=; b=ag7GkPYuwM2cPV1P9fVfr6XXBmeXf7Jr1fm/kZzlKfO0MbYcdlhKNzYvFYRUg/M4is g8DNG2Qi5k078pQFL18I5IP9QpqShrpeoeaOihZ7Uj3Se5YbQ4fcu7aHwRT7ytG9nfrw qBOwf1afxSrVmXcnjAE0p2wZDxpR5vX2W8XzNdeIaX1TrnEzRRVnA1HhemMogeFLeJBC MHZzy/5ab2HQLK0d2OGj1q2giaPzB9djCy7FTa2C4NG9JMUbtf8BfUO8dGl5KdfD2PwT e1F0T+dhvDrqEIpTFBalZADkV3dc2TBPVCw1GsvmGablwSS91jEaw9gTX82HGyPfHbYk F3Tw== X-Gm-Message-State: AElRT7HGif9ncCwsnl28uikqlpJsWP6PnXAXnjfiFNV65Fm3xCPBsVCB +LFSO7V2ay2Vmvf74ZId/9qE9Q== X-Google-Smtp-Source: AG47ELtzELQ4cKetyIfFr0GGdNeUsKqF14vBzJe5E08bRgRR9TmrDyyliG+qbMdpKRGj8LZ7lknbpw== X-Received: by 10.28.191.138 with SMTP id o10mr1328691wmi.26.1520695381839; Sat, 10 Mar 2018 07:23:01 -0800 (PST) Received: from localhost.localdomain ([105.148.128.186]) by smtp.gmail.com with ESMTPSA id m9sm7027531wrf.13.2018.03.10.07.22.58 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 10 Mar 2018 07:23:01 -0800 (PST) From: Ard Biesheuvel To: linux-crypto@vger.kernel.org Subject: [PATCH v5 11/23] arm64: assembler: add macros to conditionally yield the NEON under PREEMPT Date: Sat, 10 Mar 2018 15:21:56 +0000 Message-Id: <20180310152208.10369-12-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.15.1 In-Reply-To: <20180310152208.10369-1-ard.biesheuvel@linaro.org> References: <20180310152208.10369-1-ard.biesheuvel@linaro.org> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20180310_072313_712128_39DB357A X-CRM114-Status: GOOD ( 18.14 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , herbert@gondor.apana.org.au, Ard Biesheuvel , Peter Zijlstra , Catalin Marinas , Sebastian Andrzej Siewior , Will Deacon , Russell King - ARM Linux , Steven Rostedt , Thomas Gleixner , Dave Martin , linux-arm-kernel@lists.infradead.org, linux-rt-users@vger.kernel.org MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP Add support macros to conditionally yield the NEON (and thus the CPU) that may be called from the assembler code. In some cases, yielding the NEON involves saving and restoring a non trivial amount of context (especially in the CRC folding algorithms), and so the macro is split into three, and the code in between is only executed when the yield path is taken, allowing the context to be preserved. The third macro takes an optional label argument that marks the resume path after a yield has been performed. Signed-off-by: Ard Biesheuvel --- arch/arm64/include/asm/assembler.h | 64 ++++++++++++++++++++ arch/arm64/kernel/asm-offsets.c | 2 + 2 files changed, 66 insertions(+) diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h index eef1fd2c1c0b..61168cbe9781 100644 --- a/arch/arm64/include/asm/assembler.h +++ b/arch/arm64/include/asm/assembler.h @@ -635,4 +635,68 @@ alternative_else_nop_endif .endif .endm +/* + * Check whether to yield to another runnable task from kernel mode NEON code + * (which runs with preemption disabled). + * + * if_will_cond_yield_neon + * // pre-yield patchup code + * do_cond_yield_neon + * // post-yield patchup code + * endif_yield_neon