From patchwork Wed Feb 17 14:26:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 12091713 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6E735C433DB for ; Wed, 17 Feb 2021 14:28:41 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 21E9564DE0 for ; Wed, 17 Feb 2021 14:28:41 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 21E9564DE0 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:Message-Id:Date:Subject:To:From: Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender :Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References:List-Owner; bh=e3aEQGXy8GbCid5i5/6JmvODVuexhGorBCLdZKVfHBc=; b=oufEASOnk9aiQ353Ff3zFjrFwS PZ4A1E9tAGALcuxDZmDZ9bwQ4oEnb/NZpDaYYf84QUzV5nUmaB6C0/rc1OpjERbhyj6iLxxbQ2XIC 6G1G8Zl9PlXSxD/f3h2Nk17g/3YsdOW8+bev7FHM5nzOJX23V6y/wd+5XmzyFr0bnymCxljlcwE/j r7uHblmIMwxwcyq1vMwfXscf6YoFO9YgjcjdpX97yKk12KXDiitkwVMGxZKBJiEuB65coxP6Nd1CO yw2RdMmFglbZ28/knQXgEJttqeSrRsuEbSG+8V6sgmvlcpTu2u1xt70hHqRoaUiV9wcpsXyLWEseU OpElPH6w==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1lCNmd-0001ZS-Te; Wed, 17 Feb 2021 14:26:27 +0000 Received: from mail.kernel.org ([198.145.29.99]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1lCNma-0001Y5-OC for linux-arm-kernel@lists.infradead.org; Wed, 17 Feb 2021 14:26:25 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id D5F3A64DE0; Wed, 17 Feb 2021 14:26:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1613571983; bh=Eh2Y38B6qR35/KfSec5haSygRfK852nfnK/XqKsasKY=; h=From:To:Cc:Subject:Date:From; b=PzC5iMGWuiTd+yf7rLcwGWyI43tys5576kutM0GgBRgEIw48VczcOVrkUYaBZwL1T gN+gjPXVfVVGRatd6bVnyf2VdoocXngSs1SRclvIbAH37Mfr2y2vwk2ekjca0B3YLc 5vkboqZwcG5iXBStLkQW/43IwY9+H/QKWNMSNSh3aab9TyPlxsK5GSAo4xQPpEwhb/ 2TRitZFVXG7VpaiuOi47LSH0ajLolOh+VVK0EWj+kkQymdTmeqTbp2qKZHmICEq4ZF F1tOYVS7oelrcWSDuM7o+2itOZ05tRNZq2XX+MAyZBm362FAZZQW0cpoYxd/h23Irb A0kbKsn6MG/7g== From: Will Deacon To: linux-arm-kernel@lists.infradead.org Subject: [PATCH] arm64: spectre: Avoid locking on v4 mitigation enable path Date: Wed, 17 Feb 2021 14:26:17 +0000 Message-Id: <20210217142617.4039-1-will@kernel.org> X-Mailer: git-send-email 2.20.1 MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210217_092624_940273_EFA9056E X-CRM114-Status: GOOD ( 16.18 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Lorenzo Pieralisi , Saravana Kannan , Peter Zijlstra , Marc Zyngier , Boqun Feng , Sami Tolvanen , Will Deacon Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The Spectre-v4 workaround is re-configured when resuming from suspend, as the firmware may have re-enabled the mitigation despite the user previously asking for it to be disabled. Enabling or disabling the workaround can result in an undefined instruction exception on CPUs which implement PSTATE.SSBS but only allow it to be configured by adjusting the SPSR on exception return. We handle this by installing an 'undef hook' which effectively emulates the access. Installing this hook requires us to take a couple of spinlocks both to avoid corrupting the internal list of hooks but also to ensure that we don't run into an unhandled exception. Unfortunately, when resuming from suspend, we haven't yet called rcu_idle_exit() and so lockdep gets angry about "suspicious RCU usage". In doing so, it tries to print a warning, which leads it to get even more suspicious, this time about itself: | rcu_scheduler_active = 2, debug_locks = 1 | RCU used illegally from extended quiescent state! | 1 lock held by swapper/0: | #0: (logbuf_lock){-.-.}-{2:2}, at: vprintk_emit+0x88/0x198 | | Call trace: | dump_backtrace+0x0/0x1d8 | show_stack+0x18/0x24 | dump_stack+0xe0/0x17c | lockdep_rcu_suspicious+0x11c/0x134 | trace_lock_release+0xa0/0x160 | lock_release+0x3c/0x290 | _raw_spin_unlock+0x44/0x80 | vprintk_emit+0xbc/0x198 | vprintk_default+0x44/0x6c | vprintk_func+0x1f4/0x1fc | printk+0x54/0x7c | lockdep_rcu_suspicious+0x30/0x134 | trace_lock_acquire+0xa0/0x188 | lock_acquire+0x50/0x2fc | _raw_spin_lock+0x68/0x80 | spectre_v4_enable_mitigation+0xa8/0x30c | __cpu_suspend_exit+0xd4/0x1a8 | cpu_suspend+0xa0/0x104 | psci_cpu_suspend_enter+0x3c/0x5c | psci_enter_idle_state+0x44/0x74 | cpuidle_enter_state+0x148/0x2f8 | cpuidle_enter+0x38/0x50 | do_idle+0x1f0/0x2b4 Prevent these splats by avoiding locking entirely if we know that a hook has already been registered for the mitigation. Cc: Mark Rutland Cc: Lorenzo Pieralisi Cc: Peter Zijlstra Cc: Boqun Feng Cc: Marc Zyngier Cc: Saravana Kannan Reported-by: Sami Tolvanen Fixes: c28762070ca6 ("arm64: Rewrite Spectre-v4 mitigation code") Signed-off-by: Will Deacon Tested-by: Sami Tolvanen --- I'm not hugely pleased with this patch, as it adds significant complexity to something which is a slow-path and should be straightforward. I also notice that __cpu_suspend_exit() is marked 'notrace' but it makes plenty of calls to functions which are not marked inline or notrace. arch/arm64/kernel/proton-pack.c | 27 ++++++++++++++++++++------- 1 file changed, 20 insertions(+), 7 deletions(-) diff --git a/arch/arm64/kernel/proton-pack.c b/arch/arm64/kernel/proton-pack.c index 902e4084c477..cb5b430b2dac 100644 --- a/arch/arm64/kernel/proton-pack.c +++ b/arch/arm64/kernel/proton-pack.c @@ -498,10 +498,24 @@ static struct undef_hook ssbs_emulation_hook = { .fn = ssbs_emulation_handler, }; -static enum mitigation_state spectre_v4_enable_hw_mitigation(void) +static void spectre_v4_register_undef_hook(void) { static bool undef_hook_registered = false; static DEFINE_RAW_SPINLOCK(hook_lock); + + if (READ_ONCE(undef_hook_registered)) + return; + + raw_spin_lock(&hook_lock); + if (!undef_hook_registered) { + register_undef_hook(&ssbs_emulation_hook); + smp_store_release(&undef_hook_registered, true); + } + raw_spin_unlock(&hook_lock); +} + +static enum mitigation_state spectre_v4_enable_hw_mitigation(void) +{ enum mitigation_state state; /* @@ -512,12 +526,11 @@ static enum mitigation_state spectre_v4_enable_hw_mitigation(void) if (state != SPECTRE_MITIGATED || !this_cpu_has_cap(ARM64_SSBS)) return state; - raw_spin_lock(&hook_lock); - if (!undef_hook_registered) { - register_undef_hook(&ssbs_emulation_hook); - undef_hook_registered = true; - } - raw_spin_unlock(&hook_lock); + /* + * Toggling PSTATE.SSBS directly may trigger an undefined instruction + * exception, so ensure we're ready to deal with the emulation. + */ + spectre_v4_register_undef_hook(); if (spectre_v4_mitigations_off()) { sysreg_clear_set(sctlr_el1, 0, SCTLR_ELx_DSSBS);