From patchwork Tue Aug 28 15:51:35 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Thierry X-Patchwork-Id: 10578829 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 66EDB920 for ; Tue, 28 Aug 2018 16:09:11 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 53FE72A68B for ; Tue, 28 Aug 2018 16:09:11 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 4856D2A68D; Tue, 28 Aug 2018 16:09:11 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 08F0C2A68B for ; Tue, 28 Aug 2018 16:09:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=4py2SewDCvKrzO+2gwxjiSI8NbdeuasGrNxF+RSgeis=; b=ohUIN+C/i9efQJch9pgmH5oQn8 1mkQUumEiWg6U788HyJQCwM+tsIHrQieaXJram7DVXWyGz0IvSeOp/5erQWNyB5YFfs23IcD+z8Qe Ff+Lm/agtMYWSqvSPh7TUldeDb/VYaItrh/5mw3f01AHq/7kIemZteNHB79ji5HHrD1OSYMzWvNXK 5Z8SLHMUh1MfKNuQebdiNKw7fswxXjKAoMhifL9XvQn/z3SPXUdnBWoOID1RGu3tE9/x+m9CNS7X2 yl0nUsR0+H0dlXtOz0kbqPSe8iez1fK8A0y2ia0hdlEgjy5Y78vOX9mo8qu9t+6HN/ZCQZ+0DJpYU wcqO3rlA==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1fugY8-0007dS-7o; Tue, 28 Aug 2018 16:09:00 +0000 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70] helo=foss.arm.com) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1fugIY-0005T3-Sb for linux-arm-kernel@lists.infradead.org; Tue, 28 Aug 2018 15:53:01 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 893731993; Tue, 28 Aug 2018 08:52:47 -0700 (PDT) Received: from e112298-lin.Emea.Arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 53CDF3F557; Tue, 28 Aug 2018 08:52:45 -0700 (PDT) From: Julien Thierry To: linux-arm-kernel@lists.infradead.org Subject: [PATCH v5 25/27] irqchip/gic-v3: Add base support for pseudo-NMI Date: Tue, 28 Aug 2018 16:51:35 +0100 Message-Id: <1535471497-38854-26-git-send-email-julien.thierry@arm.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1535471497-38854-1-git-send-email-julien.thierry@arm.com> References: <1535471497-38854-1-git-send-email-julien.thierry@arm.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20180828_085255_049785_9CCB244B X-CRM114-Status: GOOD ( 17.67 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mark.rutland@arm.com, daniel.thompson@linaro.org, Jason Cooper , Julien Thierry , marc.zyngier@arm.com, catalin.marinas@arm.com, will.deacon@arm.com, linux-kernel@vger.kernel.org, christoffer.dall@arm.com, james.morse@arm.com, joel@joelfernandes.org, Russell King , Thomas Gleixner MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP Provide a higher priority to be used for pseudo-NMIs. When such an interrupt is received, enter the NMI state and prevent other NMIs to be raised. When returning from a pseudo-NMI, skip preemption and tracing if the interrupted context has interrupts disabled. Signed-off-by: Julien Thierry Cc: Russell King Cc: Catalin Marinas Cc: Will Deacon Cc: Thomas Gleixner Cc: Jason Cooper Cc: Marc Zyngier --- arch/arm/include/asm/arch_gicv3.h | 6 ++++++ arch/arm64/include/asm/arch_gicv3.h | 6 ++++++ arch/arm64/kernel/entry.S | 43 +++++++++++++++++++++++++++++++++++++ drivers/irqchip/irq-gic-v3.c | 34 ++++++++++++++++++++++++++++- 4 files changed, 88 insertions(+), 1 deletion(-) diff --git a/arch/arm/include/asm/arch_gicv3.h b/arch/arm/include/asm/arch_gicv3.h index b39d620..1ed0476 100644 --- a/arch/arm/include/asm/arch_gicv3.h +++ b/arch/arm/include/asm/arch_gicv3.h @@ -374,5 +374,11 @@ static inline void gic_start_pmr_masking(void) WARN_ON(true); } +static inline void gic_set_nmi_active(void) +{ + /* Should not get called */ + WARN_ON(true); +} + #endif /* !__ASSEMBLY__ */ #endif /* !__ASM_ARCH_GICV3_H */ diff --git a/arch/arm64/include/asm/arch_gicv3.h b/arch/arm64/include/asm/arch_gicv3.h index eb55da8..6213d6f 100644 --- a/arch/arm64/include/asm/arch_gicv3.h +++ b/arch/arm64/include/asm/arch_gicv3.h @@ -167,5 +167,11 @@ static inline void gic_start_pmr_masking(void) asm volatile ("msr daifclr, #2" : : : "memory"); } +/* Notify an NMI is active, blocking other NMIs */ +static inline void gic_set_nmi_active(void) +{ + asm volatile ("msr daifset, #2" : : : "memory"); +} + #endif /* __ASSEMBLY__ */ #endif /* __ASM_ARCH_GICV3_H */ diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index 91e1e3d..39d6ef0 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -411,6 +411,16 @@ alternative_insn eret, nop, ARM64_UNMAP_KERNEL_AT_EL0 mov sp, x19 .endm + /* Should be checked on return from irq handlers */ + .macro branch_if_was_nmi, tmp, target + alternative_if ARM64_HAS_IRQ_PRIO_MASKING + mrs \tmp, daif + alternative_else + mov \tmp, #0 + alternative_endif + tbnz \tmp, #7, \target // Exiting an NMI + .endm + /* * These are the registers used in the syscall handler, and allow us to * have in theory up to 7 arguments to a function - x0 to x6. @@ -631,12 +641,30 @@ ENDPROC(el1_sync) el1_irq: kernel_entry 1 enable_da_f + #ifdef CONFIG_TRACE_IRQFLAGS +#ifdef CONFIG_USE_ICC_SYSREGS_FOR_IRQFLAGS + ldr x20, [sp, #S_PMR_SAVE] + /* Irqs were disabled, don't trace */ + tbz x20, ICC_PMR_EL1_EN_SHIFT, 1f +#endif bl trace_hardirqs_off +1: #endif irq_handler +#ifdef CONFIG_USE_ICC_SYSREGS_FOR_IRQFLAGS + /* + * Irqs were disabled, we have an nmi. + * We might have interrupted a context with interrupt disabled that set + * NEED_RESCHED flag. + * Skip preemption and irq tracing if needed. + */ + tbz x20, ICC_PMR_EL1_EN_SHIFT, untraced_irq_exit + branch_if_was_nmi x0, skip_preempt +#endif + #ifdef CONFIG_PREEMPT ldr w24, [tsk, #TSK_TI_PREEMPT] // get preempt count cbnz w24, 1f // preempt count != 0 @@ -645,9 +673,13 @@ el1_irq: bl el1_preempt 1: #endif + +skip_preempt: #ifdef CONFIG_TRACE_IRQFLAGS bl trace_hardirqs_on #endif + +untraced_irq_exit: kernel_exit 1 ENDPROC(el1_irq) @@ -873,6 +905,9 @@ el0_irq_naked: #ifdef CONFIG_TRACE_IRQFLAGS bl trace_hardirqs_on #endif + + branch_if_was_nmi x2, nmi_ret_to_user + b ret_to_user ENDPROC(el0_irq) @@ -1269,3 +1304,11 @@ alternative_else_nop_endif ENDPROC(__sdei_asm_handler) NOKPROBE(__sdei_asm_handler) #endif /* CONFIG_ARM_SDE_INTERFACE */ + +/* + * NMI return path to EL0 + */ +nmi_ret_to_user: + ldr x1, [tsk, #TSK_TI_FLAGS] + b finish_ret_to_user +ENDPROC(nmi_ret_to_user) diff --git a/drivers/irqchip/irq-gic-v3.c b/drivers/irqchip/irq-gic-v3.c index ffe98e8..1af2fcc 100644 --- a/drivers/irqchip/irq-gic-v3.c +++ b/drivers/irqchip/irq-gic-v3.c @@ -41,6 +41,8 @@ #include "irq-gic-common.h" +#define GICD_INT_NMI_PRI 0xa0 + struct redist_region { void __iomem *redist_base; phys_addr_t phys_base; @@ -248,6 +250,12 @@ static void gic_unmask_irq(struct irq_data *d) gic_poke_irq(d, GICD_ISENABLER); } +static inline bool gic_supports_nmi(void) +{ + return gic_prio_masking_enabled() + && static_branch_likely(&have_non_secure_prio_view); +} + static int gic_irq_set_irqchip_state(struct irq_data *d, enum irqchip_irq_state which, bool val) { @@ -381,6 +389,23 @@ static asmlinkage void __exception_irq_entry gic_handle_irq(struct pt_regs *regs if (likely(irqnr > 15 && irqnr < 1020) || irqnr >= 8192) { int err; + if (gic_supports_nmi() + && unlikely(gic_read_rpr() == GICD_INT_NMI_PRI)) { + /* + * We need to prevent other NMIs to occur even after a + * priority drop. + * We keep I flag set until cpsr is restored from + * kernel_exit. + */ + gic_set_nmi_active(); + + if (static_branch_likely(&supports_deactivate_key)) + gic_write_eoir(irqnr); + + err = handle_domain_nmi(gic_data.domain, irqnr, regs); + return; + } + if (static_branch_likely(&supports_deactivate_key)) gic_write_eoir(irqnr); else if (!gic_prio_masking_enabled()) @@ -1119,6 +1144,11 @@ static int partition_domain_translate(struct irq_domain *d, .select = gic_irq_domain_select, }; +static void gic_enable_nmi_support(void) +{ + static_branch_enable(&have_non_secure_prio_view); +} + static int __init gic_init_bases(void __iomem *dist_base, struct redist_region *rdist_regs, u32 nr_redist_regions, @@ -1188,7 +1218,9 @@ static int __init gic_init_bases(void __iomem *dist_base, if (gic_prio_masking_enabled()) { if (!gic_has_group0() || gic_dist_security_disabled()) - static_branch_enable(&have_non_secure_prio_view); + gic_enable_nmi_support(); + else + pr_warn("SCR_EL3.FIQ set, cannot enable use of pseudo-NMIs\n"); } return 0;