From patchwork Wed Jan 20 11:36:47 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mohamed Mediouni X-Patchwork-Id: 12032137 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2CC9BC433E0 for ; Wed, 20 Jan 2021 11:39:04 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D4A6522AAA for ; Wed, 20 Jan 2021 11:39:03 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D4A6522AAA Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=caramail.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:To:Date:Message-Id:Subject:Mime-Version:From: Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender :Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References:List-Owner; bh=kavF6/QrvDRjzZQGIBuWWU2tu6us1fzQkiXT5pr7xDs=; b=SUQxs9jVDCvjiCtQsuoPuFuz7T roFxdQodTU3j52lgF977hTmgfOXBauJ7L8+VNmieXsq9ED3sMafXkoBs7hvpMkKm5tJHxONx6Z3LV fgv+lsCuYwuiRzLh80gfeeEVFi7PvytLjQargF/28wveRlqVO4hYsB5YaKnAxVfkjcpgGrpEXjV4b RhbyjMcRzf9vUJoT+tBLfDjiYP3rbeloDrlD6dqOPn+GBButjuQdihW9f5M8QJDeCigfPR/bJbNiu a+jSLEhoDEXJImCB9+QrzEf3gCn3KqBtnYGZA7tTdZB3naUtP+/Pw19DW4tqhAUr9DNlpe4CZvfHb a7PX+VbQ==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1l2Bnj-0003DL-Rz; Wed, 20 Jan 2021 11:37:27 +0000 Received: from casper.infradead.org ([2001:8b0:10b:1236::1]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1l2Bna-0003B4-U8 for linux-arm-kernel@merlin.infradead.org; Wed, 20 Jan 2021 11:37:20 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=To:Cc:Date:Message-Id:Subject: Mime-Version:Content-Transfer-Encoding:Content-Type:From:Sender:Reply-To: Content-ID:Content-Description:In-Reply-To:References; bh=RQRq81uk6SA0smI3TLZA0pzLO4tzfMbTVwbwb6XqWQs=; b=UpYQfCZ/FSfLIjSsz7tXitC3ao g/awrUBYmnwe78Jukx/leSGvtoBtN0WX0qWwn5+KMrd/0ZjMiXfG2145aXPSGecAfPrjgnlLubMvT lLz52GFbAqVz4rLIe1UUOUzobwma40DZgSUNCrIXOskfCuB4N54hi+vfjT2qIa9gCKCE2K/W1LfMr VWadPzv7tOwJVflyLh/kU1Bj+ycbUbKqxuINJx4YDKZCjWTOqMJ3GPSkpnbynONFavCu0rLiZGgZo Mckoo9foOy3mUak/rgefSZn4J8fKSM5m2SMA9Txv99zfVvgioPylOOPJmk4RQ22tuwSMhmRL1vS2n I+db90aw==; Received: from mout.gmx.net ([212.227.17.22]) by casper.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1l2BnM-00FbUY-Ay for linux-arm-kernel@lists.infradead.org; Wed, 20 Jan 2021 11:37:06 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=badeba3b8450; t=1611142608; bh=84eP5VOW0S5T7+C+G7fOeLTqAiaH7xYaSSXNl3ohDHE=; h=X-UI-Sender-Class:From:Subject:Date:Cc:To; b=iWIoXiLQq0IRz944xmOYzpxZvSnkM8RlSL3quBNkPtjlymlhL6zTc8MJx+04jw2mY q7v2mUVJy68jfW8VGJBZiukaHhE8ZZDKjUXGftT2gakcIrFQ3nPKHETP7zA8tiKlJP z1c1H/39nzjGkfbTWb7MdvpboQF5rRKzdnT3oibk= X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c Received: from [10.42.0.78] ([83.204.192.78]) by mail.gmx.net (mrgmx104 [212.227.17.174]) with ESMTPSA (Nemesis) id 1MUGiJ-1lTW5L12Yd-00RES7; Wed, 20 Jan 2021 12:36:48 +0100 From: Mohamed Mediouni Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.60.0.2.5\)) Subject: [PATCH 1/3] arm64/kernel: FIQ support Message-Id: <9C00B3A8-FDE3-4F5F-B23B-6296A7993C9F@caramail.com> Date: Wed, 20 Jan 2021 12:36:47 +0100 To: linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org X-Mailer: Apple Mail (2.3654.60.0.2.5) X-Provags-ID: V03:K1:s9EzbGnke5cLQiejSGfkvJDnuLlral9u9HPC6a7FstSt49bgHF6 OPrUNCWbeSytXZlZ4KA9NbEd4G+Y/y61WM15g4FROL0HAgIVqF4KD2R4B2fhsekaAeOqAkh IuVDRNFi7aswab9iuuxzeimzZTBKZ9XMQvSrgSd2pQlVhTWpgZUn+tPMsaVNPSaA5VxFW4h 0WMyjHCtmhX7MNJ5N84vA== X-UI-Out-Filterresults: notjunk:1;V03:K0:USBG8NL5v1s=:Px50PhVP95U3L62hK+3MCb vhIx/EEdHYS5IZO/dg9UjM3ON65godE2+Rw17PYdTcuC9NUnfuxWGk66wKZ3muyrBWljeopTf KVJ7RODGE+SmHHIaoJ9tF8Pfjj5IT2fZW7DLtPCS0BpIkgJ6oOTlnRbTdv7t6VGPGJUsx2G19 h76buL6UyyxIUZYaoyizgy5gto5WQfY8zSvLBrbQFxzigOWdumQ0Zxtdb/JYRkCIBwMaqcON5 4pc5CpnPkSoKo238ydArLaMHHBcxk4v4yQKKXO3zS/hDtqL1K7Uko47U8p+ztAc6W7z5sifWy wlEq8ZIp6CG0mpFJXXwowUnGdMoQlYuih7Si+jGyE1j9qlMJwLa2dUSG7gIcqV+lRPrb8RfIw 6/6nEmktpOX8FwuU6H8eiKe4DSVYIq4hyWwInghavtAESBnOLO7+r6fqTIJnjtf29Q6gELQ/m H65eoTariHlKvXt8RGVA4uTKhXQQNqvX2wal/ahrm8Cp68lvK99pDKOSRBNJXy65PT3Jk3NAm u1iCKdgvcq9OkUfNY/1xA1/fuNZh8t67L3Ttwo943iW6jcs44mgVPTzZ1EYCzHqqP5Rj1rbq2 XNHjPJ/uUKrn/qmgaee6JDnoDIRHLb+7hvx1cDuMy3KltDt29odqPB6aC9/Mo/tGi/qtMnNWl 7E8gzsw5NDHQ38NLIE0KM0LsK5iTfinoJuQ41Ux54J9EYX5RBdmUdSjrf9n3dNdaL6OlnmxYn iXxiqiDtJeH7Yk/fzCHkdmV5dBoBvkt+YUPD8BY1hBoO4BfreUwCN2crdkHN/5gCycXVHQBGq QURGW62CMakBMwIab5i0koN9iUcN1Tbj/WVUX3lsHgeOD5Vkjzwku+IA8L6HBXHb1rzIVMird l2348jgka6Gx3LxgfLKin6ezxFXevlActBHi52wgY= X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210120_113706_238295_8804371B X-CRM114-Status: GOOD ( 20.36 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Catalin Marinas , Will Deacon , Stan Skowronek Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Stan Skowronek On Apple processors, the timer is wired through FIQ. As such, add FIQ support to the kernel. Signed-off-by: Stan Skowronek --- arch/arm64/include/asm/arch_gicv3.h | 2 +- arch/arm64/include/asm/assembler.h | 8 ++-- arch/arm64/include/asm/daifflags.h | 4 +- arch/arm64/include/asm/irq.h | 4 ++ arch/arm64/include/asm/irqflags.h | 6 +-- arch/arm64/kernel/entry.S | 74 ++++++++++++++++++++++++++--- arch/arm64/kernel/irq.c | 14 ++++++ arch/arm64/kernel/process.c | 2 +- 8 files changed, 97 insertions(+), 17 deletions(-) diff --git a/arch/arm64/include/asm/arch_gicv3.h b/arch/arm64/include/asm/arch_gicv3.h index 880b9054d75c..934b9be582d2 100644 --- a/arch/arm64/include/asm/arch_gicv3.h +++ b/arch/arm64/include/asm/arch_gicv3.h @@ -173,7 +173,7 @@ static inline void gic_pmr_mask_irqs(void) static inline void gic_arch_enable_irqs(void) { - asm volatile ("msr daifclr, #2" : : : "memory"); + asm volatile ("msr daifclr, #3" : : : "memory"); } #endif /* __ASSEMBLY__ */ diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h index bf125c591116..6fe55713dfe0 100644 --- a/arch/arm64/include/asm/assembler.h +++ b/arch/arm64/include/asm/assembler.h @@ -40,9 +40,9 @@ msr daif, \flags .endm - /* IRQ is the lowest priority flag, unconditionally unmask the rest. */ - .macro enable_da_f - msr daifclr, #(8 | 4 | 1) + /* IRQ/FIQ is the lowest priority flag, unconditionally unmask the rest. */ + .macro enable_da + msr daifclr, #(8 | 4) .endm /* @@ -50,7 +50,7 @@ */ .macro save_and_disable_irq, flags mrs \flags, daif - msr daifset, #2 + msr daifset, #3 .endm .macro restore_irq, flags diff --git a/arch/arm64/include/asm/daifflags.h b/arch/arm64/include/asm/daifflags.h index 1c26d7baa67f..44de96c7fb1a 100644 --- a/arch/arm64/include/asm/daifflags.h +++ b/arch/arm64/include/asm/daifflags.h @@ -13,8 +13,8 @@ #include #define DAIF_PROCCTX 0 -#define DAIF_PROCCTX_NOIRQ PSR_I_BIT -#define DAIF_ERRCTX (PSR_I_BIT | PSR_A_BIT) +#define DAIF_PROCCTX_NOIRQ (PSR_I_BIT | PSR_F_BIT) +#define DAIF_ERRCTX (PSR_I_BIT | PSR_F_BIT | PSR_A_BIT) #define DAIF_MASK (PSR_D_BIT | PSR_A_BIT | PSR_I_BIT | PSR_F_BIT) diff --git a/arch/arm64/include/asm/irq.h b/arch/arm64/include/asm/irq.h index b2b0c6405eb0..2d1537d3a245 100644 --- a/arch/arm64/include/asm/irq.h +++ b/arch/arm64/include/asm/irq.h @@ -13,5 +13,9 @@ static inline int nr_legacy_irqs(void) return 0; } +int set_handle_fiq(void (*handle_fiq)(struct pt_regs *)); + +extern void (*handle_arch_fiq)(struct pt_regs *) __ro_after_init; + #endif /* !__ASSEMBLER__ */ #endif diff --git a/arch/arm64/include/asm/irqflags.h b/arch/arm64/include/asm/irqflags.h index ff328e5bbb75..26d7f378113e 100644 --- a/arch/arm64/include/asm/irqflags.h +++ b/arch/arm64/include/asm/irqflags.h @@ -35,7 +35,7 @@ static inline void arch_local_irq_enable(void) } asm volatile(ALTERNATIVE( - "msr daifclr, #2 // arch_local_irq_enable", + "msr daifclr, #3 // arch_local_irq_enable", __msr_s(SYS_ICC_PMR_EL1, "%0"), ARM64_HAS_IRQ_PRIO_MASKING) : @@ -54,7 +54,7 @@ static inline void arch_local_irq_disable(void) } asm volatile(ALTERNATIVE( - "msr daifset, #2 // arch_local_irq_disable", + "msr daifset, #3 // arch_local_irq_disable", __msr_s(SYS_ICC_PMR_EL1, "%0"), ARM64_HAS_IRQ_PRIO_MASKING) : @@ -85,7 +85,7 @@ static inline int arch_irqs_disabled_flags(unsigned long flags) int res; asm volatile(ALTERNATIVE( - "and %w0, %w1, #" __stringify(PSR_I_BIT), + "and %w0, %w1, #" __stringify(PSR_I_BIT | PSR_F_BIT), "eor %w0, %w1, #" __stringify(GIC_PRIO_IRQON), ARM64_HAS_IRQ_PRIO_MASKING) : "=&r" (res) diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index c9bae73f2621..abcca0db0736 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -499,6 +499,14 @@ tsk .req x28 // current thread_info irq_stack_exit .endm + .macro fiq_handler + ldr_l x1, handle_arch_fiq + mov x0, sp + irq_stack_entry + blr x1 + irq_stack_exit + .endm + #ifdef CONFIG_ARM64_PSEUDO_NMI /* * Set res to 0 if irqs were unmasked in interrupted context. @@ -547,18 +555,18 @@ SYM_CODE_START(vectors) kernel_ventry 1, sync // Synchronous EL1h kernel_ventry 1, irq // IRQ EL1h - kernel_ventry 1, fiq_invalid // FIQ EL1h + kernel_ventry 1, fiq // FIQ EL1h kernel_ventry 1, error // Error EL1h kernel_ventry 0, sync // Synchronous 64-bit EL0 kernel_ventry 0, irq // IRQ 64-bit EL0 - kernel_ventry 0, fiq_invalid // FIQ 64-bit EL0 + kernel_ventry 0, fiq // FIQ 64-bit EL0 kernel_ventry 0, error // Error 64-bit EL0 #ifdef CONFIG_COMPAT kernel_ventry 0, sync_compat, 32 // Synchronous 32-bit EL0 kernel_ventry 0, irq_compat, 32 // IRQ 32-bit EL0 - kernel_ventry 0, fiq_invalid_compat, 32 // FIQ 32-bit EL0 + kernel_ventry 0, fiq_compat, 32 // FIQ 32-bit EL0 kernel_ventry 0, error_compat, 32 // Error 32-bit EL0 #else kernel_ventry 0, sync_invalid, 32 // Synchronous 32-bit EL0 @@ -661,7 +669,7 @@ SYM_CODE_END(el1_sync) SYM_CODE_START_LOCAL_NOALIGN(el1_irq) kernel_entry 1 gic_prio_irq_setup pmr=x20, tmp=x1 - enable_da_f + enable_da mov x0, sp bl enter_el1_irq_or_nmi @@ -689,6 +697,38 @@ alternative_else_nop_endif kernel_exit 1 SYM_CODE_END(el1_irq) + .align 6 +SYM_CODE_START_LOCAL_NOALIGN(el1_fiq) + kernel_entry 1 + gic_prio_irq_setup pmr=x20, tmp=x1 + enable_da + + mov x0, sp + bl enter_el1_irq_or_nmi + + fiq_handler + +#ifdef CONFIG_PREEMPTION + ldr x24, [tsk, #TSK_TI_PREEMPT] // get preempt count +alternative_if ARM64_HAS_IRQ_PRIO_MASKING + /* + * DA_F were cleared at start of handling. If anything is set in DAIF, + * we come back from an NMI, so skip preemption + */ + mrs x0, daif + orr x24, x24, x0 +alternative_else_nop_endif + cbnz x24, 1f // preempt count != 0 || NMI return path + bl arm64_preempt_schedule_irq // irq en/disable is done inside +1: +#endif + + mov x0, sp + bl exit_el1_irq_or_nmi + + kernel_exit 1 +SYM_CODE_END(el1_fiq) + /* * EL0 mode handlers. */ @@ -715,6 +755,12 @@ SYM_CODE_START_LOCAL_NOALIGN(el0_irq_compat) b el0_irq_naked SYM_CODE_END(el0_irq_compat) + .align 6 +SYM_CODE_START_LOCAL_NOALIGN(el0_fiq_compat) + kernel_entry 0, 32 + b el0_fiq_naked +SYM_CODE_END(el0_fiq_compat) + SYM_CODE_START_LOCAL_NOALIGN(el0_error_compat) kernel_entry 0, 32 b el0_error_naked @@ -727,7 +773,7 @@ SYM_CODE_START_LOCAL_NOALIGN(el0_irq) el0_irq_naked: gic_prio_irq_setup pmr=x20, tmp=x0 user_exit_irqoff - enable_da_f + enable_da tbz x22, #55, 1f bl do_el0_irq_bp_hardening @@ -737,6 +783,22 @@ el0_irq_naked: b ret_to_user SYM_CODE_END(el0_irq) + .align 6 +SYM_CODE_START_LOCAL_NOALIGN(el0_fiq) + kernel_entry 0 +el0_fiq_naked: + gic_prio_irq_setup pmr=x20, tmp=x0 + user_exit_irqoff + enable_da + + tbz x22, #55, 1f + bl do_el0_irq_bp_hardening +1: + fiq_handler + + b ret_to_user +SYM_CODE_END(el0_fiq) + SYM_CODE_START_LOCAL(el1_error) kernel_entry 1 mrs x1, esr_el1 @@ -757,7 +819,7 @@ el0_error_naked: mov x0, sp mov x1, x25 bl do_serror - enable_da_f + enable_da b ret_to_user SYM_CODE_END(el0_error) diff --git a/arch/arm64/kernel/irq.c b/arch/arm64/kernel/irq.c index dfb1feab867d..4d7a9fb41d93 100644 --- a/arch/arm64/kernel/irq.c +++ b/arch/arm64/kernel/irq.c @@ -88,3 +88,17 @@ void __init init_IRQ(void) local_daif_restore(DAIF_PROCCTX_NOIRQ); } } + +/* + * Analogous to the generic handle_arch_irq / set_handle_irq + */ +void (*handle_arch_fiq)(struct pt_regs *) __ro_after_init; + +int __init set_handle_fiq(void (*handle_fiq)(struct pt_regs *)) +{ + if (handle_arch_fiq) + return -EBUSY; + + handle_arch_fiq = handle_fiq; + return 0; +} diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c index 6616486a58fe..34ec400288d0 100644 --- a/arch/arm64/kernel/process.c +++ b/arch/arm64/kernel/process.c @@ -84,7 +84,7 @@ static void noinstr __cpu_do_idle_irqprio(void) unsigned long daif_bits; daif_bits = read_sysreg(daif); - write_sysreg(daif_bits | PSR_I_BIT, daif); + write_sysreg(daif_bits | PSR_I_BIT | PSR_F_BIT, daif); /* * Unmask PMR before going idle to make sure interrupts can From patchwork Wed Jan 20 11:36:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mohamed Mediouni X-Patchwork-Id: 12032133 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7D10EC433DB for ; Wed, 20 Jan 2021 11:38:57 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2A35422AAA for ; Wed, 20 Jan 2021 11:38:57 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2A35422AAA Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=caramail.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:To:Date:Message-Id:Subject:Mime-Version:From: Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender :Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References:List-Owner; bh=B5DSHQxjY/4S7m+IvWHy30K83+4afd9jaU7cEjvyqEY=; b=kl8DuAPL56AwawHpfg/I7TuCoo ALtKPdSq8ITrPTKTJ3vTBvJyZe7sqjxkEJpDoVHxYlSXBqlVsQgTE9pUBllszgXilKR9Bh332UB0c SHMsCkGeePNYvimp308e3bTLjDSAGGosuXlLgPfSws7DBJlvrKD/tqbUbpHI5yhbeyP1l1IA1OdGM 6B6033B43wbMr3WMmdi5BSgwFnjfuokciuOt2NQjk84YFdafqtWXxdemxA7UOon14XQJ1aV92o7sW 515aU52eQkQ+oGBujr0r6WXOnzX+geDhV+8qR5COWXNhn0aym1koosT0MHySN0Ro6ZJ9wA5xMl2Cl sGTnWuow==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1l2BnU-0003AJ-Jl; Wed, 20 Jan 2021 11:37:12 +0000 Received: from mout.gmx.net ([212.227.17.22]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1l2BnR-00038Q-1r for linux-arm-kernel@lists.infradead.org; Wed, 20 Jan 2021 11:37:10 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=badeba3b8450; t=1611142616; bh=TfsKMmfCS6hUsi+ELxrDF9ONPAZSrITaCLKQesH2eds=; h=X-UI-Sender-Class:From:Subject:Date:Cc:To; b=cy5YCZ+V0M+BC90L8VUKPmy3qshIUbcD/T9rlNmPGxTDfARwcQrI9k+MFMUq36tQ/ nBRGdGLnmEDyW+ssPBXOTeT4ssTL2D551VitoOZkug9hzG5zwAr5VD8PZ6AX7wWvlD jU9Im6ZTTGueQsCMxEZwY8hwwkaj5qw5el3Mcixk= X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c Received: from [10.42.0.78] ([83.204.192.78]) by mail.gmx.net (mrgmx104 [212.227.17.174]) with ESMTPSA (Nemesis) id 1M2O2Q-1l1e5x3pxG-003xMQ; Wed, 20 Jan 2021 12:36:56 +0100 From: Mohamed Mediouni Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.60.0.2.5\)) Subject: [PATCH 2/3] arm64: kernel: Add a WFI hook. Message-Id: Date: Wed, 20 Jan 2021 12:36:55 +0100 To: linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org X-Mailer: Apple Mail (2.3654.60.0.2.5) X-Provags-ID: V03:K1:YQwZ4VjCCi1L8rJGcbkX5VUuw9Y3iNAMG/BESg3Vepb5mGSS09y mV7hnZOxFc0efByqcMHtfuT3KtSqHBEv9+gCEbQcuCIzHletK5KG7dqTbeM4mDGv/vL26F8 9yzTk3ZeBzyfc2JTQFziYTXv8vikoCKHeYyMB/sgM/CDqXaS1+Ek/FGAtAXbY+T2V/Pb9dr N1Y5W7MDp266ICLpQfjrw== X-UI-Out-Filterresults: notjunk:1;V03:K0:AsF8zO/Sv94=:Euoys7ZJzixTLxmHmmGkjZ T7f0QvzMwlejIimQIQqMHVqfGjNMMn2qmGlvQQedvfXTMkyzhXXJ0uJBKZg38M0ejC0h9tG0i Mr6qAwqdu/ykEXbPuDHweE92JSArl3P11BX8fpcO/vP/I78jf9ks+KQzg9+fdel/dBSNpt/mj QqDtZ4o+zfL3NUWlKezGSQ96ItMFOOJtFP+XzqOWXt99mpFyDzGk2Wc+SSS2lfAsDoVU8KEES 0T9WZQg7JswhhNgf9L+cKCRWTgRZ7C5Pb455iX5xGfWhlAbegbZBNK4JEBWWKK5cAjmbR/7Oq Br9RyympkrwcqLOjhWtmoQBH3VFlQruuD80fnDuPfXw1+n7jclnOZc/UhXPpwuKj9wwxrz0YF axxpgOATlYYn8u3Y1bDd/g/q9wxm0KVGcs0m21kbhaLUzoov6goaFDJC/MwMc8WEgwqJhsbRO 0WyjWmh0kPU7oDbuY1stOw0DVE4F8inH9oIQF+i0x2P2DqCQpRXE7cs7Zx9TYiPvpPTuYzzsn 9nNDjmwVVVemDwdNO6TUth+0uApTQy97CjCEuGPg7O1v/UXAUgMyIEftfR09jRuhqSVoaePMF pdMJdHEjP8Iih6klLM7iPfV19yXZyDTHodEbqjOr4X9u/bNq3qUcGnwsMiz1zEfzM6mfgoHzI yqPUTSU/ZSC1Vfzj26Yewm+xt0YnSypa/MLDUNbxB7s64DQ50fSox6pKeVakEtSYTUjzRbNBA UG/bex0v4P8otx2iwh8Ux4klPkWR9IgBMXjAiC6QDeeIpo+NRX/d653jluD7YU8z2ZbUJP2P4 LVj/2riV3fXJYu8FyFjKhw3j+afH+J4TN/cmGWiDQeu2p8DbV/AYs1XW68orGK7/IXtQP3HA9 50+rkNXnnhyngcaYdSHJm4s8UhHqq6ZnLUMXRvTpc= X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210120_063709_374051_E765BCBD X-CRM114-Status: GOOD ( 16.02 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Catalin Marinas , Will Deacon , Stan Skowronek Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Stan Skowronek WFI drops register state on Apple Silicon for SMP systems. This hook will be used for a hardware workaround in the Apple CPU start driver. Signed-off-by: Stan Skowronek --- arch/arm64/include/asm/cpu_ops.h | 2 ++ arch/arm64/kernel/cpu_ops.c | 6 ++++++ arch/arm64/kernel/process.c | 11 +++++++++-- 3 files changed, 17 insertions(+), 2 deletions(-) diff --git a/arch/arm64/include/asm/cpu_ops.h b/arch/arm64/include/asm/cpu_ops.h index e95c4df83911..4be0fc5bcaf9 100644 --- a/arch/arm64/include/asm/cpu_ops.h +++ b/arch/arm64/include/asm/cpu_ops.h @@ -23,6 +23,7 @@ * @cpu_boot: Boots a cpu into the kernel. * @cpu_postboot: Optionally, perform any post-boot cleanup or necessary * synchronisation. Called from the cpu being booted. + * @cpu_wfi: Optionally, replace calls to WFI in default idle with this. * @cpu_can_disable: Determines whether a CPU can be disabled based on * mechanism-specific information. * @cpu_disable: Prepares a cpu to die. May fail for some mechanism-specific @@ -43,6 +44,7 @@ struct cpu_operations { int (*cpu_prepare)(unsigned int); int (*cpu_boot)(unsigned int); void (*cpu_postboot)(void); + void (*cpu_wfi)(void); #ifdef CONFIG_HOTPLUG_CPU bool (*cpu_can_disable)(unsigned int cpu); int (*cpu_disable)(unsigned int cpu); diff --git a/arch/arm64/kernel/cpu_ops.c b/arch/arm64/kernel/cpu_ops.c index e133011f64b5..6979fc4490b2 100644 --- a/arch/arm64/kernel/cpu_ops.c +++ b/arch/arm64/kernel/cpu_ops.c @@ -19,12 +19,18 @@ extern const struct cpu_operations smp_spin_table_ops; extern const struct cpu_operations acpi_parking_protocol_ops; #endif extern const struct cpu_operations cpu_psci_ops; +#ifdef CONFIG_ARCH_APPLE +extern const struct cpu_operations cpu_apple_start_ops; +#endif static const struct cpu_operations *cpu_ops[NR_CPUS] __ro_after_init; static const struct cpu_operations *const dt_supported_cpu_ops[] __initconst = { &smp_spin_table_ops, &cpu_psci_ops, +#ifdef CONFIG_ARCH_APPLE + &cpu_apple_start_ops, +#endif NULL, }; diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c index 34ec400288d0..611c639e20be 100644 --- a/arch/arm64/kernel/process.c +++ b/arch/arm64/kernel/process.c @@ -57,6 +57,7 @@ #include #include #include +#include #if defined(CONFIG_STACKPROTECTOR) && !defined(CONFIG_STACKPROTECTOR_PER_TASK) #include @@ -74,8 +75,14 @@ void (*arm_pm_restart)(enum reboot_mode reboot_mode, const char *cmd); static void noinstr __cpu_do_idle(void) { - dsb(sy); - wfi(); + const struct cpu_operations *ops = get_cpu_ops(task_cpu(current)); + + if (ops->cpu_wfi) { + ops->cpu_wfi(); + } else { + dsb(sy); + wfi(); + } } static void noinstr __cpu_do_idle_irqprio(void) From patchwork Wed Jan 20 11:37:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mohamed Mediouni X-Patchwork-Id: 12032135 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 224B0C433E0 for ; Wed, 20 Jan 2021 11:39:01 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C4E9022AAA for ; Wed, 20 Jan 2021 11:39:00 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C4E9022AAA Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=caramail.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:To:Date:Message-Id:Subject:Mime-Version:From: Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender :Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References:List-Owner; bh=XNVGbY4ZWovjWFigI/+yGaXjYgWXGC+0tVIH6SNElR4=; b=1rdDJkGWJ2lved6C5yvoAi54Xt M4W87QSTEDJyVV1S7npy6WHsAEwnZe92tjcuap5sSG3nS8kJHWRiyMpjjE8PZZ1SlqzDU0DazmAUF F8A8m400D3776KHebG136M/LbKNKoUSP+xN0AA0V+beOANRyqeRNDzyp7ZMersd3Ximqjloap2G1m IVJs3nN4yLZ8UZ8S82hcxyrYqzUykduhddQ7mrja0kISi2cyCOaqUPZSD6ToMPW9wticpqU5S7MCY M7eyoU9xjBFKs2W69pUM7biiYcG46Uljj6ZghbwHJjcsjc4FAZizXANhg9rKBdgNkZ6tdNTSA9cly uauHpWUg==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1l2Bnc-0003B6-HZ; Wed, 20 Jan 2021 11:37:20 +0000 Received: from mout.gmx.net ([212.227.17.20]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1l2BnT-00039O-2H for linux-arm-kernel@lists.infradead.org; Wed, 20 Jan 2021 11:37:12 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=badeba3b8450; t=1611142623; bh=yP+0ZpEyODQg4IQgrFLqLHkUEaxLQnyVarhdKMvY+Ew=; h=X-UI-Sender-Class:From:Subject:Date:Cc:To; b=NidgcRekJDfzb+7vs8u2A5qpqDVYwChkU6hZx25xb87lgh9e1MOepMBqBd76mZmI4 IipsRmJChZ5/nJwb5YNaBT+JwnATdKESpc9dH66Hx7sMgA2QUr7ArVn70U93HdgVQA Vx0HJGrtEAVvXqL4VBfPcc58GjHFYdLCnLXzATa0= X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c Received: from [10.42.0.78] ([83.204.192.78]) by mail.gmx.net (mrgmx104 [212.227.17.174]) with ESMTPSA (Nemesis) id 1N2mBQ-1m0vh43aA8-0132VU; Wed, 20 Jan 2021 12:37:03 +0100 From: Mohamed Mediouni Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.60.0.2.5\)) Subject: [PATCH 3/3] arm64: mm: use nGnRnE instead of nGnRE on Apple processors Message-Id: Date: Wed, 20 Jan 2021 12:37:02 +0100 To: linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org X-Mailer: Apple Mail (2.3654.60.0.2.5) X-Provags-ID: V03:K1:LqipaoXg1JiKrBNorCn8+FwCfrzoRZrJ9EZuGtcEElXHOzz6vPB 5h9Axeed+jmKMBiQzSOm4DnrO6Qxsy38Pn5BCZ7DjgmpKvALx6JxHjTUXXdO853F1wobyon LFjV1yo+vkNQn8AF2mwPL2pPH31q0Mdx8HTWx055vLJggoB2Chsi90CZlOoiWIUwo6otIMn 7lI08uR3Nb9w1hDK3qz0w== X-UI-Out-Filterresults: notjunk:1;V03:K0:YWTH2k09Pjc=:ekdBfAdYjnPaVM/vkUpGmN 5QogswEZLhOsx3vsb30ktDLV5aVF7PDTVctRPm3XyWQrKH8kTkf6hJC1YUJTQ07F4NpvsTHRV nNxDLm2bw0Xn2znW/Cx6FQc/yiyd7veYwJxH2hiEpIpwlDlHlPSxhPlmllqLJFnHV8XrDD6PY l5d7QTU35Azliwid1IbClWUSgdXNU6+zZuG7jK+a9oYhLdO6On0OhR4KKthKx+Qdn3RjOcjgS A6SaC5LZtMfXADc1ali2y5+r+IJY+oqOQrNu1Qdd54NYBd7KtKjfQbSf9okjo3DtDRSauQdJM gc/Z52KYc4idMORGG/TZdyC0iNXVUpz+cdBfzFyY8MlUAevEsTY7Saq4nBl9l/jNasc4IFRgf R5cUAn8xRxdPCp65caEG9ZIeRXEUW3GZT+uLYACVX+1d4IddkE73I+xWuQCLLLchbceXfQ8Hw 3yqkpCATYdMekyhuHcQNR/W3jILPGq6pCNZtDw0inufklUV2vAa9Ez44avQarqDATQKjtBzM7 vqrQZWDryCWrw3P8irtPn7CGCa5X0U1zI3L3simRNdh3hkO+o1ppkEMAKBI+//MZ4s91dnzp6 zCfJMpGRgeeB9tCEbVXtQRqA7wk5j+0O6LYaTXR7oTDHHnyvxw94Nl9znT2N9yVdQ6CakgIq2 a/m6UaTmIlQ7M0aDHdnjQCQZGBE9FA/wh5C8ddRK1Whb5cO73cI2lbAh+Z2amhHkrKl+svDay VrTiHJvx8aKyiRxOC6F3tl16KzT9JQC9+dMxE/rWFsXx2UVMUvnTLnBCP/kWbweysmsOTonZi BfLEKWPW5BXVcf7gqFnmt/hMDeUXEqi7XB81aRVQzDxR79PZZdTlj0BC1bVhsyLEN41TfuDHj k7hcpYXW1leFbkL+lbNBW6QIYl6l8p4+dHW8wQ+no= X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210120_063711_394072_71DDB8A5 X-CRM114-Status: GOOD ( 12.03 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Catalin Marinas , Will Deacon , Stan Skowronek Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Mohamed Mediouni Use nGnRnE instead of nGnRE on Apple SoCs to workaround a serious hardware quirk. On Apple processors, writes using the nGnRE device memory type get dropped in flight, getting to nowhere. Signed-off-by: Mohamed Mediouni Signed-off-by: Stan Skowronek --- arch/arm64/mm/proc.S | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S index 1f7ee8c8b7b8..06436916f137 100644 --- a/arch/arm64/mm/proc.S +++ b/arch/arm64/mm/proc.S @@ -51,6 +51,25 @@ #define TCR_KASAN_HW_FLAGS 0 #endif +#ifdef CONFIG_ARCH_APPLE + +/* + * Apple cores appear to black-hole writes done with nGnRE. + * We settled on a work-around that uses MAIR vs changing every single user of + * nGnRE across the arm64 code. + */ + +#define MAIR_EL1_SET_APPLE \ + (MAIR_ATTRIDX(MAIR_ATTR_DEVICE_nGnRnE, MT_DEVICE_nGnRnE) | \ + MAIR_ATTRIDX(MAIR_ATTR_DEVICE_nGnRnE, MT_DEVICE_nGnRE) | \ + MAIR_ATTRIDX(MAIR_ATTR_DEVICE_GRE, MT_DEVICE_GRE) | \ + MAIR_ATTRIDX(MAIR_ATTR_NORMAL_NC, MT_NORMAL_NC) | \ + MAIR_ATTRIDX(MAIR_ATTR_NORMAL, MT_NORMAL) | \ + MAIR_ATTRIDX(MAIR_ATTR_NORMAL_WT, MT_NORMAL_WT) | \ + MAIR_ATTRIDX(MAIR_ATTR_NORMAL, MT_NORMAL_TAGGED)) + +#endif + /* * Default MAIR_EL1. MT_NORMAL_TAGGED is initially mapped as Normal memory and * changed during __cpu_setup to Normal Tagged if the system supports MTE. @@ -432,6 +451,13 @@ SYM_FUNC_START(__cpu_setup) * Memory region attributes */ mov_q x5, MAIR_EL1_SET +#ifdef CONFIG_ARCH_APPLE + mrs x0, MIDR_EL1 + lsr w0, w0, #24 + mov_q x1, MAIR_EL1_SET_APPLE + cmp x0, #0x61 // 0x61 = Implementer: Apple + csel x5, x1, x5, eq +#endif #ifdef CONFIG_ARM64_MTE mte_tcr .req x20