From patchwork Mon Aug 18 13:40:51 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Thompson X-Patchwork-Id: 4736731 Return-Path: X-Original-To: patchwork-linux-arm@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.19.201]) by patchwork1.web.kernel.org (Postfix) with ESMTP id 9BCBA9F344 for ; Mon, 18 Aug 2014 13:45:56 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id C16342010F for ; Mon, 18 Aug 2014 13:45:54 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.9]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5BD7F2010C for ; Mon, 18 Aug 2014 13:45:53 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.80.1 #2 (Red Hat Linux)) id 1XJNCF-00010z-2B; Mon, 18 Aug 2014 13:42:03 +0000 Received: from mail-wi0-f175.google.com ([209.85.212.175]) by bombadil.infradead.org with esmtps (Exim 4.80.1 #2 (Red Hat Linux)) id 1XJNBx-0000k2-Mf for linux-arm-kernel@lists.infradead.org; Mon, 18 Aug 2014 13:41:47 +0000 Received: by mail-wi0-f175.google.com with SMTP id ho1so3735602wib.2 for ; Mon, 18 Aug 2014 06:41:23 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=3fyGG+7O1fdgOfAdlQuo/TirTVsRdYYFBkNRL5WCatw=; b=SiLbds5pSrYGjURvhCOugDf290fX5bo0U5PAza+Ck3mDrzWsXG6peLbf22OdU1LvU8 M1f2KjEb7LAUNVpZk4jsJqhXwU/e/JdCDy5d+J6go/zRCnn6Ul42vGvGMNwvR4EAZ6Bv 41kcVUYvOVDEhSBgGYm3qYV7oJrhzJyCwSg39pg++F7CmQuS1U9okF8BsryWlefjkZwZ NSCDYsTY67bDbgJdCKGplsc6MYyhe2KsBjOku5k1mxI4+iOydZdGVxDCwsaU4BW88t7P dLY4PlJYjQw8bwDMxFzPp/U9py41IJxilLyi/Z+DB3kpPr+l6YNYsCMN39dJyj3i+sFg sNuw== X-Gm-Message-State: ALoCoQmjUjUwbzhm7ALyt4EdSpV2OBmYyV94iE1JQkx9+aWam4sb01dS+Kvz14VVJQ6DELZ1H5Wq X-Received: by 10.180.107.170 with SMTP id hd10mr71146791wib.77.1408369283079; Mon, 18 Aug 2014 06:41:23 -0700 (PDT) Received: from sundance.lan (cpc4-aztw19-0-0-cust157.18-1.cable.virginm.net. [82.33.25.158]) by mx.google.com with ESMTPSA id y6sm6351438wia.0.2014.08.18.06.41.20 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 18 Aug 2014 06:41:22 -0700 (PDT) From: Daniel Thompson To: Russell King Subject: [PATCH v9 03/16] arm: fiq: Replace default FIQ handler Date: Mon, 18 Aug 2014 14:40:51 +0100 Message-Id: <1408369264-14242-4-git-send-email-daniel.thompson@linaro.org> X-Mailer: git-send-email 1.9.3 In-Reply-To: <1408369264-14242-1-git-send-email-daniel.thompson@linaro.org> References: <1404979427-12943-1-git-send-email-daniel.thompson@linaro.org> <1408369264-14242-1-git-send-email-daniel.thompson@linaro.org> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20140818_064146_055119_6706DC07 X-CRM114-Status: GOOD ( 22.64 ) X-Spam-Score: -0.7 (/) Cc: Daniel Thompson , linaro-kernel@lists.linaro.org, Catalin Marinas , patches@linaro.org, kgdb-bugreport@lists.sourceforge.net, Linus Walleij , Nicolas Pitre , linux-kernel@vger.kernel.org, Frederic Weisbecker , Anton Vorontsov , Ben Dooks , John Stultz , Fabio Estevam , Colin Cross , kernel-team@android.com, Dave Martin , linux-arm-kernel@lists.infradead.org X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_NONE, RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP This patch introduces a new default FIQ handler that is structured in a similar way to the existing ARM exception handler and result in the FIQ being handled by C code running on the SVC stack (despite this code run in the FIQ handler is subject to severe limitations with respect to locking making normal interaction with the kernel impossible). This default handler allows concepts that on x86 would be handled using NMIs to be realized on ARM. Credit: This patch is a near complete re-write of a patch originally provided by Anton Vorontsov. Today only a couple of small fragments survive, however without Anton's work to build from this patch would not exist. Signed-off-by: Daniel Thompson Cc: Russell King Cc: Nicolas Pitre Cc: Catalin Marinas --- arch/arm/include/asm/fiq.h | 1 + arch/arm/kernel/entry-armv.S | 125 +++++++++++++++++++++++++++++++++++++++---- arch/arm/kernel/fiq.c | 17 ++++++ arch/arm/kernel/setup.c | 8 ++- 4 files changed, 140 insertions(+), 11 deletions(-) diff --git a/arch/arm/include/asm/fiq.h b/arch/arm/include/asm/fiq.h index a25c952..175bfed 100644 --- a/arch/arm/include/asm/fiq.h +++ b/arch/arm/include/asm/fiq.h @@ -54,6 +54,7 @@ extern void disable_fiq(int fiq); extern int ack_fiq(int fiq); extern void eoi_fiq(int fiq); extern bool has_fiq(int fiq); +extern int register_fiq_nmi_notifier(struct notifier_block *nb); extern void fiq_register_mapping(int irq, struct fiq_chip *chip); /* helpers defined in fiqasm.S: */ diff --git a/arch/arm/kernel/entry-armv.S b/arch/arm/kernel/entry-armv.S index 36276cd..ba0234b 100644 --- a/arch/arm/kernel/entry-armv.S +++ b/arch/arm/kernel/entry-armv.S @@ -79,6 +79,15 @@ #endif .endm + .macro fiq_handler + ldr r1, =.LChandle_fiq + mov r0, sp + adr lr, BSYM(9998f) + ldr pc, [r1] +9998: + .endm + + #ifdef CONFIG_KPROBES .section .kprobes.text,"ax",%progbits #else @@ -146,7 +155,7 @@ ENDPROC(__und_invalid) #define SPFIX(code...) #endif - .macro svc_entry, stack_hole=0 + .macro svc_entry, stack_hole=0, call_trace=1 UNWIND(.fnstart ) UNWIND(.save {r0 - pc} ) sub sp, sp, #(S_FRAME_SIZE + \stack_hole - 4) @@ -183,10 +192,35 @@ ENDPROC(__und_invalid) stmia r7, {r2 - r6} #ifdef CONFIG_TRACE_IRQFLAGS + .if \call_trace bl trace_hardirqs_off + .endif #endif .endm +@ +@ svc_exit_via_fiq - similar to svc_exit but switches to FIQ mode before exit +@ +@ This macro acts in a similar manner to svc_exit but switches to FIQ +@ mode to restore the final part of the register state. +@ +@ We cannot use the normal svc_exit procedure because that would +@ clobber spsr_svc (FIQ could be delivered during the first few instructions +@ of vector_swi meaning its contents have not been saved anywhere). +@ + .macro svc_exit_via_fiq, rpsr + + mov r0, sp + ldmib r0, {r1 - r14} @ abort is deadly from here onward (it will + @ clobber state restored below) + msr cpsr_c, #FIQ_MODE | PSR_I_BIT | PSR_F_BIT + add r8, r0, #S_PC + ldr r9, [r0, #S_PSR] + msr spsr_cxsf, r9 + ldr r0, [r0, #S_R0] + ldmia r8, {pc}^ + .endm + .align 5 __dabt_svc: svc_entry @@ -295,6 +329,14 @@ __pabt_svc: ENDPROC(__pabt_svc) .align 5 +__fiq_svc: + svc_entry 0, 0 + fiq_handler + svc_exit_via_fiq r5 + UNWIND(.fnend ) +ENDPROC(__fiq_svc) + + .align 5 .LCcralign: .word cr_alignment #ifdef MULTI_DABORT @@ -303,6 +345,39 @@ ENDPROC(__pabt_svc) #endif .LCfp: .word fp_enter +.LChandle_fiq: + .word fiq_nmi_handler + +/* + * Abort mode handlers + */ + +@ +@ Taking a FIQ in abort mode is similar to taking a FIQ in SVC mode +@ and reuses the same macros. However in abort mode we must also +@ save/restore lr_abt and spsr_abt to make nested aborts safe. +@ + .align 5 +__fiq_abt: + svc_entry 0, 0 + + msr cpsr_c, #ABT_MODE | PSR_I_BIT | PSR_F_BIT + mov r0, lr @ Save lr_abt + mrs r1, spsr @ Save spsr_abt, abort is now safe + msr cpsr_c, #SVC_MODE | PSR_I_BIT | PSR_F_BIT + push {r0 - r1} + + fiq_handler + + pop {r0 - r1} + msr cpsr_c, #ABT_MODE | PSR_I_BIT | PSR_F_BIT + mov lr, r0 @ Restore lr_abt, abort is unsafe + msr spsr_cxsf, r1 @ Restore spsr_abt + msr cpsr_c, #SVC_MODE | PSR_I_BIT | PSR_F_BIT + + svc_exit_via_fiq r5 + UNWIND(.fnend ) +ENDPROC(__fiq_svc) /* * User mode handlers @@ -683,6 +758,17 @@ ENTRY(ret_from_exception) ENDPROC(__pabt_usr) ENDPROC(ret_from_exception) + .align 5 +__fiq_usr: + usr_entry + kuser_cmpxchg_check + fiq_handler + get_thread_info tsk + mov why, #0 + b ret_to_user_from_irq + UNWIND(.fnend ) +ENDPROC(__fiq_usr) + /* * Register switch for ARMv3 and ARMv4 processors * r0 = previous task_struct, r1 = previous thread_info, r2 = next thread_info @@ -1118,17 +1204,36 @@ vector_addrexcptn: b vector_addrexcptn /*============================================================================= - * Undefined FIQs + * FIQ "NMI" handler *----------------------------------------------------------------------------- - * Enter in FIQ mode, spsr = ANY CPSR, lr = ANY PC - * MUST PRESERVE SVC SPSR, but need to switch to SVC mode to show our msg. - * Basically to switch modes, we *HAVE* to clobber one register... brain - * damage alert! I don't think that we can execute any code in here in any - * other mode than FIQ... Ok you can switch to another mode, but you can't - * get out of that mode without clobbering one register. + * Handle a FIQ using the SVC stack allowing FIQ act like NMI on x86 + * systems. The runtime environment for NMIs is extremely restrictive + * (NMIs can pre-empt critical sections meaning almost all locking is + * forbidden) meaning this default FIQ handling must only be used in + * circumstances where non-maskability improves robustness, such as + * watchdog or debug logic. + * + * This handler is inappropriate for high performance (fast) interrupt + * servicing and can be overridden using set_fiq_handler. */ -vector_fiq: - subs pc, lr, #4 + vector_stub fiq, FIQ_MODE, 4 + + .long __fiq_usr @ 0 (USR_26 / USR_32) + .long __fiq_svc @ 1 (FIQ_26 / FIQ_32) + .long __fiq_svc @ 2 (IRQ_26 / IRQ_32) + .long __fiq_svc @ 3 (SVC_26 / SVC_32) + .long __fiq_svc @ 4 + .long __fiq_svc @ 5 + .long __fiq_svc @ 6 + .long __fiq_abt @ 7 + .long __fiq_svc @ 8 + .long __fiq_svc @ 9 + .long __fiq_svc @ a + .long __fiq_svc @ b + .long __fiq_svc @ c + .long __fiq_svc @ d + .long __fiq_svc @ e + .long __fiq_svc @ f .globl vector_fiq_offset .equ vector_fiq_offset, vector_fiq diff --git a/arch/arm/kernel/fiq.c b/arch/arm/kernel/fiq.c index 3ccaa8c..b2bd1c7 100644 --- a/arch/arm/kernel/fiq.c +++ b/arch/arm/kernel/fiq.c @@ -46,6 +46,7 @@ #include #include +#include #include #include #include @@ -64,6 +65,7 @@ static unsigned long no_fiq_insn; static int fiq_start = -1; static RADIX_TREE(fiq_data_tree, GFP_KERNEL); static DEFINE_MUTEX(fiq_data_mutex); +static ATOMIC_NOTIFIER_HEAD(fiq_nmi_chain); /* Default reacquire function * - we always relinquish FIQ control @@ -216,6 +218,21 @@ bool has_fiq(int fiq) } EXPORT_SYMBOL(has_fiq); +int register_fiq_nmi_notifier(struct notifier_block *nb) +{ + return atomic_notifier_chain_register(&fiq_nmi_chain, nb); +} + +asmlinkage void __exception_irq_entry fiq_nmi_handler(struct pt_regs *regs) +{ + struct pt_regs *old_regs = set_irq_regs(regs); + + nmi_enter(); + atomic_notifier_call_chain(&fiq_nmi_chain, (unsigned long)regs, NULL); + nmi_exit(); + set_irq_regs(old_regs); +} + EXPORT_SYMBOL(set_fiq_handler); EXPORT_SYMBOL(__set_fiq_regs); /* defined in fiqasm.S */ EXPORT_SYMBOL(__get_fiq_regs); /* defined in fiqasm.S */ diff --git a/arch/arm/kernel/setup.c b/arch/arm/kernel/setup.c index 84db893d..c031063 100644 --- a/arch/arm/kernel/setup.c +++ b/arch/arm/kernel/setup.c @@ -133,6 +133,7 @@ struct stack { u32 irq[3]; u32 abt[3]; u32 und[3]; + u32 fiq[3]; } ____cacheline_aligned; #ifndef CONFIG_CPU_V7M @@ -470,7 +471,10 @@ void notrace cpu_init(void) "msr cpsr_c, %5\n\t" "add r14, %0, %6\n\t" "mov sp, r14\n\t" - "msr cpsr_c, %7" + "msr cpsr_c, %7\n\t" + "add r14, %0, %8\n\t" + "mov sp, r14\n\t" + "msr cpsr_c, %9" : : "r" (stk), PLC (PSR_F_BIT | PSR_I_BIT | IRQ_MODE), @@ -479,6 +483,8 @@ void notrace cpu_init(void) "I" (offsetof(struct stack, abt[0])), PLC (PSR_F_BIT | PSR_I_BIT | UND_MODE), "I" (offsetof(struct stack, und[0])), + PLC (PSR_F_BIT | PSR_I_BIT | FIQ_MODE), + "I" (offsetof(struct stack, fiq[0])), PLC (PSR_F_BIT | PSR_I_BIT | SVC_MODE) : "r14"); #endif