From patchwork Fri May 25 09:49:30 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Thierry X-Patchwork-Id: 10426919 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id D6DEF6025B for ; Fri, 25 May 2018 10:07:11 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CC46429644 for ; Fri, 25 May 2018 10:07:11 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id BE09129647; Fri, 25 May 2018 10:07:11 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI autolearn=unavailable version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 912B029642 for ; Fri, 25 May 2018 10:07:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=5JLGmL7h8+ZUApeHXC9yYsi7f848g+jrExOIBN2lWVg=; b=iaOxqkkHbaUK3XcyFu56nOCoVN g2t+ewPWBdr2w0fZyrP2tsrWLJ7GpdcdLRPk9adhec468ppMOeWJ9h/lu492/XZzjoRtLSUpnP4Mi PhHiWZUswOmkBGrPTorEQEfHWQJxN2YyN5icli5SDuhX8fy35pL/FQvSoh3BuT0fH0/CAS7rOIb2B IP6s3dzUqsbGzdlTG5VXVMNBCbtM98QqB2Tz+6W4giKD5ADy7bede2kkDD6KdS2bnGvx51vwXdQmQ XtD7JYLpraFUHAFsMPUVIP302sBKkaUt/oy38cx96AnlFRjb5zdTuEBDHYECtKsre0kIQlyQ1Ejnx gAie57Zg==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1fM9ci-0004KL-0t; Fri, 25 May 2018 10:07:00 +0000 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70] helo=foss.arm.com) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1fM9N8-0007uF-Tu for linux-arm-kernel@lists.infradead.org; Fri, 25 May 2018 09:50:58 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id D62101688; Fri, 25 May 2018 02:50:54 -0700 (PDT) Received: from e112298-lin.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 8B17D3F25D; Fri, 25 May 2018 02:50:52 -0700 (PDT) From: Julien Thierry To: linux-arm-kernel@lists.infradead.org Subject: [PATCH v4 24/26] irqchip/gic-v3: Add base support for pseudo-NMI Date: Fri, 25 May 2018 10:49:30 +0100 Message-Id: <1527241772-48007-25-git-send-email-julien.thierry@arm.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1527241772-48007-1-git-send-email-julien.thierry@arm.com> References: <1527241772-48007-1-git-send-email-julien.thierry@arm.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20180525_025055_115501_8796A5B2 X-CRM114-Status: GOOD ( 17.06 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mark.rutland@arm.com, daniel.thompson@linaro.org, Jason Cooper , Julien Thierry , marc.zyngier@arm.com, catalin.marinas@arm.com, will.deacon@arm.com, linux-kernel@vger.kernel.org, christoffer.dall@arm.com, james.morse@arm.com, joel@joelfernandes.org, Russell King , Thomas Gleixner MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP Provide a higher priority to be used for pseudo-NMIs. When such an interrupt is received, enter the NMI state and prevent other NMIs to be raised. When returning from a pseudo-NMI, skip preemption and tracing if the interrupted context has interrupts disabled. Signed-off-by: Julien Thierry Cc: Russell King Cc: Catalin Marinas Cc: Will Deacon Cc: Thomas Gleixner Cc: Jason Cooper Cc: Marc Zyngier --- arch/arm/include/asm/arch_gicv3.h | 6 ++++++ arch/arm64/include/asm/arch_gicv3.h | 6 ++++++ arch/arm64/kernel/entry.S | 43 +++++++++++++++++++++++++++++++++++++ drivers/irqchip/irq-gic-v3.c | 41 +++++++++++++++++++++++++++++++++++ 4 files changed, 96 insertions(+) diff --git a/arch/arm/include/asm/arch_gicv3.h b/arch/arm/include/asm/arch_gicv3.h index b39d620..1ed0476 100644 --- a/arch/arm/include/asm/arch_gicv3.h +++ b/arch/arm/include/asm/arch_gicv3.h @@ -374,5 +374,11 @@ static inline void gic_start_pmr_masking(void) WARN_ON(true); } +static inline void gic_set_nmi_active(void) +{ + /* Should not get called */ + WARN_ON(true); +} + #endif /* !__ASSEMBLY__ */ #endif /* !__ASM_ARCH_GICV3_H */ diff --git a/arch/arm64/include/asm/arch_gicv3.h b/arch/arm64/include/asm/arch_gicv3.h index 23c88ac0..3196cf1 100644 --- a/arch/arm64/include/asm/arch_gicv3.h +++ b/arch/arm64/include/asm/arch_gicv3.h @@ -166,5 +166,11 @@ static inline void gic_start_pmr_masking(void) asm volatile ("msr daifclr, #2" : : : "memory"); } +/* Notify an NMI is active, blocking other NMIs */ +static inline void gic_set_nmi_active(void) +{ + asm volatile ("msr daifset, #2" : : : "memory"); +} + #endif /* __ASSEMBLY__ */ #endif /* __ASM_ARCH_GICV3_H */ diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index f56f27e..0d0c829 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -391,6 +391,16 @@ alternative_insn eret, nop, ARM64_UNMAP_KERNEL_AT_EL0 mov sp, x19 .endm + /* Should be checked on return from irq handlers */ + .macro branch_if_was_nmi, tmp, target + alternative_if ARM64_HAS_IRQ_PRIO_MASKING + mrs \tmp, daif + alternative_else + mov \tmp, #0 + alternative_endif + tbnz \tmp, #7, \target // Exiting an NMI + .endm + /* * These are the registers used in the syscall handler, and allow us to * have in theory up to 7 arguments to a function - x0 to x6. @@ -611,12 +621,30 @@ ENDPROC(el1_sync) el1_irq: kernel_entry 1 enable_da_f + #ifdef CONFIG_TRACE_IRQFLAGS +#ifdef CONFIG_USE_ICC_SYSREGS_FOR_IRQFLAGS + ldr x20, [sp, #S_PMR_SAVE] + /* Irqs were disabled, don't trace */ + tbz x20, ICC_PMR_EL1_EN_SHIFT, 1f +#endif bl trace_hardirqs_off +1: #endif irq_handler +#ifdef CONFIG_USE_ICC_SYSREGS_FOR_IRQFLAGS + /* + * Irqs were disabled, we have an nmi. + * We might have interrupted a context with interrupt disabled that set + * NEED_RESCHED flag. + * Skip preemption and irq tracing if needed. + */ + tbz x20, ICC_PMR_EL1_EN_SHIFT, untraced_irq_exit + branch_if_was_nmi x0, skip_preempt +#endif + #ifdef CONFIG_PREEMPT ldr w24, [tsk, #TSK_TI_PREEMPT] // get preempt count cbnz w24, 1f // preempt count != 0 @@ -625,9 +653,13 @@ el1_irq: bl el1_preempt 1: #endif + +skip_preempt: #ifdef CONFIG_TRACE_IRQFLAGS bl trace_hardirqs_on #endif + +untraced_irq_exit: kernel_exit 1 ENDPROC(el1_irq) @@ -858,6 +890,9 @@ el0_irq_naked: #ifdef CONFIG_TRACE_IRQFLAGS bl trace_hardirqs_on #endif + + branch_if_was_nmi x2, nmi_ret_to_user + b ret_to_user ENDPROC(el0_irq) @@ -1353,3 +1388,11 @@ alternative_else_nop_endif ENDPROC(__sdei_asm_handler) NOKPROBE(__sdei_asm_handler) #endif /* CONFIG_ARM_SDE_INTERFACE */ + +/* + * NMI return path to EL0 + */ +nmi_ret_to_user: + ldr x1, [tsk, #TSK_TI_FLAGS] + b finish_ret_to_user +ENDPROC(nmi_ret_to_user) diff --git a/drivers/irqchip/irq-gic-v3.c b/drivers/irqchip/irq-gic-v3.c index b144f73..4be5996 100644 --- a/drivers/irqchip/irq-gic-v3.c +++ b/drivers/irqchip/irq-gic-v3.c @@ -41,6 +41,8 @@ #include "irq-gic-common.h" +#define GICD_INT_NMI_PRI 0xa0 + struct redist_region { void __iomem *redist_base; phys_addr_t phys_base; @@ -253,6 +255,12 @@ static inline bool arch_uses_gic_prios(void) return IS_ENABLED(CONFIG_USE_ICC_SYSREGS_FOR_IRQFLAGS); } +static inline bool gic_supports_nmi(void) +{ + return arch_uses_gic_prios() + && static_branch_likely(&have_non_secure_prio_view); +} + static int gic_irq_set_irqchip_state(struct irq_data *d, enum irqchip_irq_state which, bool val) { @@ -371,6 +379,20 @@ static u64 gic_mpidr_to_affinity(unsigned long mpidr) return aff; } +static void do_handle_nmi(unsigned int hwirq, struct pt_regs *regs) +{ + struct pt_regs *old_regs = set_irq_regs(regs); + unsigned int irq; + + nmi_enter(); + + irq = irq_find_mapping(gic_data.domain, hwirq); + generic_handle_irq(irq); + + nmi_exit(); + set_irq_regs(old_regs); +} + static asmlinkage void __exception_irq_entry gic_handle_irq(struct pt_regs *regs) { u32 irqnr; @@ -386,6 +408,23 @@ static asmlinkage void __exception_irq_entry gic_handle_irq(struct pt_regs *regs if (likely(irqnr > 15 && irqnr < 1020) || irqnr >= 8192) { int err; + if (gic_supports_nmi() + && unlikely(gic_read_rpr() == GICD_INT_NMI_PRI)) { + /* + * We need to prevent other NMIs to occur even after a + * priority drop. + * We keep I flag set until cpsr is restored from + * kernel_exit. + */ + gic_set_nmi_active(); + + if (static_branch_likely(&supports_deactivate_key)) + gic_write_eoir(irqnr); + + do_handle_nmi(irqnr, regs); + return; + } + if (static_branch_likely(&supports_deactivate_key)) gic_write_eoir(irqnr); else if (!arch_uses_gic_prios()) @@ -1183,6 +1222,8 @@ static int __init gic_init_bases(void __iomem *dist_base, if (arch_uses_gic_prios()) { if (!gic_has_group0() || gic_dist_security_disabled()) static_branch_enable(&have_non_secure_prio_view); + else + pr_warn("SCR_EL3.FIQ set, cannot enable use of pseudo-NMIs\n"); } return 0;