From patchwork Thu Nov 2 12:12:34 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Morse X-Patchwork-Id: 10038613 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 3A9D56032D for ; Thu, 2 Nov 2017 12:15:11 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 26AE728F25 for ; Thu, 2 Nov 2017 12:15:11 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1B25D28F27; Thu, 2 Nov 2017 12:15:11 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [65.50.211.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 3C3A128BAB for ; Thu, 2 Nov 2017 12:15:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=9p5i42VzO6oKKDYIIOhFtgVHJK+GSLHhMwCf9b5TKNs=; b=fCx7FZZlq5/1FwUJq9MPPOlYvb HfGwEQK5I0nOVI+qDYjz2LV1pX+c4rE4//xIx5CzE68zMaBvJ72hwxwVpEpaGj7qPe1nlm6m1sIqq AMcCrLlNP6uP0Y+L9oykOm6gCjU/Yhkm4KyMBLK1OWJ5qQ10Gi+S4XFbf8mNz7b++tlNCAcy7jm10 a/HQiog/4mJT2CLlEHlxOomix1+d+XcvQkyv6sM42sAPcgNMdg9q+Q62X7GwG3WtpONLJEQkSEz01 /hZHvalIXZbM0nA/zGFepjhyxn2JqAkSxagD50i8j3ojTgUxXmgn13DIpibmDwnsn4AgM0YSgYJ+x JT+p/aaA==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux)) id 1eAEOj-0001dI-Rk; Thu, 02 Nov 2017 12:15:01 +0000 Received: from foss.arm.com ([217.140.101.70]) by bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux)) id 1eAEOc-0001N0-Dn for linux-arm-kernel@lists.infradead.org; Thu, 02 Nov 2017 12:14:59 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id BEBCA1596; Thu, 2 Nov 2017 05:14:36 -0700 (PDT) Received: from melchizedek.cambridge.arm.com (melchizedek.cambridge.arm.com [10.1.207.55]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 467543F3E1; Thu, 2 Nov 2017 05:14:35 -0700 (PDT) From: James Morse To: linux-arm-kernel@lists.infradead.org Subject: [RESEND PATCH v4 1/9] arm64: explicitly mask all exceptions Date: Thu, 2 Nov 2017 12:12:34 +0000 Message-Id: <20171102121242.8925-2-james.morse@arm.com> X-Mailer: git-send-email 2.15.0.rc2 In-Reply-To: <20171102121242.8925-1-james.morse@arm.com> References: <20171102121242.8925-1-james.morse@arm.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20171102_051454_483928_9A367C0F X-CRM114-Status: GOOD ( 21.45 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Julien Thierry , Catalin Marinas , Xie XiuQi , Will Deacon , wangxiongfeng2@huawei.com, James Morse , kvmarm@lists.cs.columbia.edu MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP There are a few places where we want to mask all exceptions. Today we do this in a piecemeal fashion, typically we expect the caller to have masked irqs and the arch code masks debug exceptions, ignoring serror which is probably masked. Make it clear that 'mask all exceptions' is the intention by adding helpers to do exactly that. This will let us unmask SError without having to add 'oh and SError' to these paths. Signed-off-by: James Morse Reviewed-by: Julien Thierry Reviewed-by: Catalin Marinas == Remove the disable IRQs comment above cpu_die(): we return from idle via cpu_resume. This comment is confusing once the local_irq_disable() call is changed. Changes since v3: * Split local_mask_daif into a save and mask versions, * swapped verb/daif word-order. * Removed {asm,linux} includes - one will always include the other --- arch/arm64/include/asm/assembler.h | 17 ++++++++++ arch/arm64/include/asm/daifflags.h | 69 ++++++++++++++++++++++++++++++++++++++ arch/arm64/kernel/hibernate.c | 5 +-- arch/arm64/kernel/machine_kexec.c | 4 +-- arch/arm64/kernel/smp.c | 9 ++--- arch/arm64/kernel/suspend.c | 7 ++-- arch/arm64/kernel/traps.c | 3 +- arch/arm64/mm/proc.S | 9 +++-- 8 files changed, 104 insertions(+), 19 deletions(-) create mode 100644 arch/arm64/include/asm/daifflags.h diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h index d58a6253c6ab..114e741ca873 100644 --- a/arch/arm64/include/asm/assembler.h +++ b/arch/arm64/include/asm/assembler.h @@ -31,6 +31,23 @@ #include #include + .macro save_and_disable_daif, flags + mrs \flags, daif + msr daifset, #0xf + .endm + + .macro disable_daif + msr daifset, #0xf + .endm + + .macro enable_daif + msr daifclr, #0xf + .endm + + .macro restore_daif, flags:req + msr daif, \flags + .endm + /* * Enable and disable interrupts. */ diff --git a/arch/arm64/include/asm/daifflags.h b/arch/arm64/include/asm/daifflags.h new file mode 100644 index 000000000000..55e2598a8c4c --- /dev/null +++ b/arch/arm64/include/asm/daifflags.h @@ -0,0 +1,69 @@ +/* + * Copyright (C) 2017 ARM Ltd. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ +#ifndef __ASM_DAIFFLAGS_H +#define __ASM_DAIFFLAGS_H + +#include + +/* mask/save/unmask/restore all exceptions, including interrupts. */ +static inline void local_daif_mask(void) +{ + asm volatile( + "msr daifset, #0xf // local_daif_mask\n" + : + : + : "memory"); + trace_hardirqs_off(); +} + +static inline unsigned long local_daif_save(void) +{ + unsigned long flags; + + asm volatile( + "mrs %0, daif // local_daif_save\n" + : "=r" (flags) + : + : "memory"); + local_daif_mask(); + + return flags; +} + +static inline void local_daif_unmask(void) +{ + trace_hardirqs_on(); + asm volatile( + "msr daifclr, #0xf // local_daif_unmask" + : + : + : "memory"); +} + +static inline void local_daif_restore(unsigned long flags) +{ + if (!arch_irqs_disabled_flags(flags)) + trace_hardirqs_on(); + asm volatile( + "msr daif, %0 // local_daif_restore" + : + : "r" (flags) + : "memory"); + if (arch_irqs_disabled_flags(flags)) + trace_hardirqs_off(); +} + +#endif diff --git a/arch/arm64/kernel/hibernate.c b/arch/arm64/kernel/hibernate.c index 095d3c170f5d..3009b8b80f08 100644 --- a/arch/arm64/kernel/hibernate.c +++ b/arch/arm64/kernel/hibernate.c @@ -27,6 +27,7 @@ #include #include #include +#include #include #include #include @@ -285,7 +286,7 @@ int swsusp_arch_suspend(void) return -EBUSY; } - local_dbg_save(flags); + flags = local_daif_save(); if (__cpu_suspend_enter(&state)) { /* make the crash dump kernel image visible/saveable */ @@ -315,7 +316,7 @@ int swsusp_arch_suspend(void) __cpu_suspend_exit(); } - local_dbg_restore(flags); + local_daif_restore(flags); return ret; } diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index 11121f608eb5..f76ea92dff91 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -18,6 +18,7 @@ #include #include +#include #include #include #include @@ -195,8 +196,7 @@ void machine_kexec(struct kimage *kimage) pr_info("Bye!\n"); - /* Disable all DAIF exceptions. */ - asm volatile ("msr daifset, #0xf" : : : "memory"); + local_daif_mask(); /* * cpu_soft_restart will shutdown the MMU, disable data caches, then diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c index 9f7195a5773e..5a407eba01f7 100644 --- a/arch/arm64/kernel/smp.c +++ b/arch/arm64/kernel/smp.c @@ -47,6 +47,7 @@ #include #include #include +#include #include #include #include @@ -368,10 +369,6 @@ void __cpu_die(unsigned int cpu) /* * Called from the idle thread for the CPU which has been shutdown. * - * Note that we disable IRQs here, but do not re-enable them - * before returning to the caller. This is also the behaviour - * of the other hotplug-cpu capable cores, so presumably coming - * out of idle fixes this. */ void cpu_die(void) { @@ -379,7 +376,7 @@ void cpu_die(void) idle_task_exit(); - local_irq_disable(); + local_daif_mask(); /* Tell __cpu_die() that this CPU is now safe to dispose of */ (void)cpu_report_death(); @@ -837,7 +834,7 @@ static void ipi_cpu_stop(unsigned int cpu) { set_cpu_online(cpu, false); - local_irq_disable(); + local_daif_mask(); while (1) cpu_relax(); diff --git a/arch/arm64/kernel/suspend.c b/arch/arm64/kernel/suspend.c index 1e3be9064cfa..d2d2edac78f2 100644 --- a/arch/arm64/kernel/suspend.c +++ b/arch/arm64/kernel/suspend.c @@ -4,6 +4,7 @@ #include #include #include +#include #include #include #include @@ -57,7 +58,7 @@ void notrace __cpu_suspend_exit(void) /* * Restore HW breakpoint registers to sane values * before debug exceptions are possibly reenabled - * through local_dbg_restore. + * by cpu_suspend()s local_daif_restore() call. */ if (hw_breakpoint_restore) hw_breakpoint_restore(cpu); @@ -81,7 +82,7 @@ int cpu_suspend(unsigned long arg, int (*fn)(unsigned long)) * updates to mdscr register (saved and restored along with * general purpose registers) from kernel debuggers. */ - local_dbg_save(flags); + flags = local_daif_save(); /* * Function graph tracer state gets incosistent when the kernel @@ -114,7 +115,7 @@ int cpu_suspend(unsigned long arg, int (*fn)(unsigned long)) * restored, so from this point onwards, debugging is fully * renabled if it was enabled when core started shutdown. */ - local_dbg_restore(flags); + local_daif_restore(flags); return ret; } diff --git a/arch/arm64/kernel/traps.c b/arch/arm64/kernel/traps.c index 5ea4b85aee0e..1808be65d22f 100644 --- a/arch/arm64/kernel/traps.c +++ b/arch/arm64/kernel/traps.c @@ -38,6 +38,7 @@ #include #include +#include #include #include #include @@ -642,7 +643,7 @@ asmlinkage void bad_mode(struct pt_regs *regs, int reason, unsigned int esr) esr_get_class_string(esr)); die("Oops - bad mode", regs, 0); - local_irq_disable(); + local_daif_mask(); panic("bad mode"); } diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S index 877d42fb0df6..95233dfc4c39 100644 --- a/arch/arm64/mm/proc.S +++ b/arch/arm64/mm/proc.S @@ -109,10 +109,10 @@ ENTRY(cpu_do_resume) /* * __cpu_setup() cleared MDSCR_EL1.MDE and friends, before unmasking * debug exceptions. By restoring MDSCR_EL1 here, we may take a debug - * exception. Mask them until local_dbg_restore() in cpu_suspend() + * exception. Mask them until local_daif_restore() in cpu_suspend() * resets them. */ - disable_dbg + disable_daif msr mdscr_el1, x10 msr sctlr_el1, x12 @@ -155,8 +155,7 @@ ENDPROC(cpu_do_switch_mm) * called by anything else. It can only be executed from a TTBR0 mapping. */ ENTRY(idmap_cpu_replace_ttbr1) - mrs x2, daif - msr daifset, #0xf + save_and_disable_daif flags=x2 adrp x1, empty_zero_page msr ttbr1_el1, x1 @@ -169,7 +168,7 @@ ENTRY(idmap_cpu_replace_ttbr1) msr ttbr1_el1, x0 isb - msr daif, x2 + restore_daif x2 ret ENDPROC(idmap_cpu_replace_ttbr1)