From patchwork Sun Oct 17 13:17:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 12564303 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2761AC433F5 for ; Sun, 17 Oct 2021 13:19:09 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D2E9B60F9E for ; Sun, 17 Oct 2021 13:19:08 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org D2E9B60F9E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=3uRhM5kiQHesHEsktosGYiXtgVnZEu0xYtUPYU+QL8E=; b=Uet0oP3YvhIHF7 CZl+iaNuyV7rKaSpEQw/D3Rq5mTHtCR1h473nhEGRGXt23SeJa4FUpemh+k1Bd48A2uVkYksDsMrO G8Uku4eqmqMIWzP/sZx1Q0O5wzK+saH05z0jVSUZUgQlNIiVzBICZgPvD5hiFPy30Jdj9BMjx2dYG RUo0WdtOWd91IydTIvvLpNhJ4bju+ZPhciH+HoXOmi07h9X155fH/IoPUA17xdaW11GYMYGYYyYS3 h/2q4Oq1ZjaW6dZXOtdtsXph7Xnbxr4phoXEvihapSpSQIZdGUjApJlPQVRUjl95z1nlvD/7DoZSX FzrGaRI/HecljuufMhrQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mc62u-00Caca-LB; Sun, 17 Oct 2021 13:17:48 +0000 Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mc62g-00CaZi-Gx for linux-arm-kernel@lists.infradead.org; Sun, 17 Oct 2021 13:17:35 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id 1743761038; Sun, 17 Oct 2021 13:17:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1634476654; bh=BEGWQcwgHbPDt9tLz36F9hY/R8tBmJ/ZFHNBbwCWD/s=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=FCrE5OAHp8vlrT9zAqD37EX//JCBrzACB/3Hy/DS/o9rT+7qnqTqD97YUjEjZbGgn qWERZ/gUWulOCg2yOsRdMu0YXWSQNlFwo9h5nERdpi7T6Sdk5s5+CenWLAodVuQytC GDB70zN983SJUjDyG2NyNn0ldw7EG4S+sCVIY84o6S3FoMJc/FIUNH8lHdjctVO33m LiAiLkUum6xLpUGEoIXkCeh2jRGV/19xyfRGzsFEBddmQdYQJyq6kRLVAxrS+xgbbN bQcmLGYH8UX5Y/TeSVRjzIgbkWY+20XEeUyiXY4pIX3sHx1W1rvgi92P3+S/voreMv zc9Ceso9v0NdQ== From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: Ard Biesheuvel , Russell King , Arnd Bergmann , Kees Cook , Keith Packard , Linus Walleij , Nick Desaulniers , Nicolas Pitre Subject: [PATCH v3 01/10] ARM: remove some dead code Date: Sun, 17 Oct 2021 15:17:14 +0200 Message-Id: <20211017131723.4034662-2-ardb@kernel.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20211017131723.4034662-1-ardb@kernel.org> References: <20211017131723.4034662-1-ardb@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=2180; h=from:subject; bh=BEGWQcwgHbPDt9tLz36F9hY/R8tBmJ/ZFHNBbwCWD/s=; b=owEB7QES/pANAwAKAcNPIjmS2Y8kAcsmYgBhbCJQ1nhSFOO91Foe3wCPsA6zTCOv4pxqiG+NgvO8 3JiR4caJAbMEAAEKAB0WIQT72WJ8QGnJQhU3VynDTyI5ktmPJAUCYWwiUAAKCRDDTyI5ktmPJA4EC/ wIeCERmnxXa/V/d/D2T6lt/VFtSrEi9YrffPVTyzfvKfAIqqwQPBfJPwi0b7xURFDn/X69SpKaEJiK C41Iw8EMD5imxWuNgx8VSZZzLgQtjtmFSD/wINGxDS72guCIx9DNfRwAX4fMiVEPlBKyW3293piy5O RNkD22Wksbh6VKSFCO0pQT2JOyk8xCoIOSwu9V+nIAbbYyEWxEWIuLAjB1osKPUS49Q/LsCr/bCDfe bWsYvwod3ePSsuI0NzFRyQ3Ru4aNeTA2oDjeDVcXU1BVc2SYXUlm5gSl0Ax+iwQz5f4jlvqgrLNK++ yYfqP43G1I9d4yJ997X7HQmIlUR1of5+Nq2aKUcPum++GQJiQx7e37lBHbaltLf0FX89HPZlagZoGg waAvoGU+LGBSL38gRi82tJ/Q+g+avLJHstmAjRrl5dwn5SCkh8Y9cQw3GbSOskd8+z07G/TVN2TNeQ fNMfQLKYvoaqDEQL1RT/Nkx6JExW1Akbexb21jyRQyDjg= X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211017_061734_623934_859C8661 X-CRM114-Status: GOOD ( 12.96 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org This code appears to be no longer used so let's get rid of it. Signed-off-by: Ard Biesheuvel Reviewed-by: Arnd Bergmann Acked-by: Linus Walleij Tested-by: Keith Packard --- arch/arm/include/asm/entry-macro-multi.S | 24 -------------------- arch/arm/include/asm/smp.h | 5 ---- arch/arm/kernel/smp.c | 5 ---- 3 files changed, 34 deletions(-) diff --git a/arch/arm/include/asm/entry-macro-multi.S b/arch/arm/include/asm/entry-macro-multi.S index dfc6bfa43012..24486dad9e19 100644 --- a/arch/arm/include/asm/entry-macro-multi.S +++ b/arch/arm/include/asm/entry-macro-multi.S @@ -13,28 +13,4 @@ @ badrne lr, 1b bne asm_do_IRQ - -#ifdef CONFIG_SMP - /* - * XXX - * - * this macro assumes that irqstat (r2) and base (r6) are - * preserved from get_irqnr_and_base above - */ - ALT_SMP(test_for_ipi r0, r2, r6, lr) - ALT_UP_B(9997f) - movne r1, sp - badrne lr, 1b - bne do_IPI -#endif -9997: - .endm - - .macro arch_irq_handler, symbol_name - .align 5 - .global \symbol_name -\symbol_name: - mov r8, lr - arch_irq_handler_default - ret r8 .endm diff --git a/arch/arm/include/asm/smp.h b/arch/arm/include/asm/smp.h index f16cbbd5cda4..7c1c90d9f582 100644 --- a/arch/arm/include/asm/smp.h +++ b/arch/arm/include/asm/smp.h @@ -24,11 +24,6 @@ struct seq_file; */ extern void show_ipi_list(struct seq_file *, int); -/* - * Called from assembly code, this handles an IPI. - */ -asmlinkage void do_IPI(int ipinr, struct pt_regs *regs); - /* * Called from C code, this handles an IPI. */ diff --git a/arch/arm/kernel/smp.c b/arch/arm/kernel/smp.c index cde5b6d8bac5..9c55ca915ba4 100644 --- a/arch/arm/kernel/smp.c +++ b/arch/arm/kernel/smp.c @@ -631,11 +631,6 @@ static void ipi_complete(unsigned int cpu) /* * Main handler for inter-processor interrupts */ -asmlinkage void __exception_irq_entry do_IPI(int ipinr, struct pt_regs *regs) -{ - handle_IPI(ipinr, regs); -} - static void do_handle_IPI(int ipinr) { unsigned int cpu = smp_processor_id(); From patchwork Sun Oct 17 13:17:15 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 12564307 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5255AC433F5 for ; Sun, 17 Oct 2021 13:19:13 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 25F9E60F9E for ; Sun, 17 Oct 2021 13:19:13 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 25F9E60F9E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=gsaFkdOqtVQMScZY2o+2PQSYgQlCfNwVgfZSn3g3iA0=; b=s3U+Ng528b/Oe3 AQDV4jkA/CWjV+E6bsEnEewFa+oOZqA1s9tAkn/mMJmUyMgiEucJnqQRjrQ3BZHdUrms4OWKzhUYK V3gXXZoCRE+a3D6PW51xs974tLD4VREWcjWVFCfzu6wA1Xe4F6tKvXqtZkVADOmfNbD/2ZX0Ny8wP gX5y22rEg+77D3my4mOZAxVWziXVNfqSbGxoEkijtChTPwyjjMIiVnHiiR5cQyuIexJQ+KfFkRDgM kt4WNHCZuTw7gOyOCrSI3OmJIfQqxdy42DC2bMtG+SABM5/94lSjP7+mm2wbXeYaX8x/X4HcNHXE+ 619Q3bcraRLiAZzI+nfw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mc634-00CafF-L4; Sun, 17 Oct 2021 13:17:58 +0000 Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mc62j-00CaaZ-6M for linux-arm-kernel@lists.infradead.org; Sun, 17 Oct 2021 13:17:38 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id B34FB60FF2; Sun, 17 Oct 2021 13:17:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1634476656; bh=zhPGaTjr4mYt5ZVj+I3r4jcIFtNN14alZ9CqE96rhlA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=qEBAqiWXhwuL7HROcvRv8GjvUxgtuC71AbABExcD7gdwP/Z+lhDKdJ3tssx5frpD6 p3FuIdPVSUJ9ZggfMjWG0edXW3J0i2ZT2UBYJ6fwM7OqiE3ZhjvNJZ9Q/fYbWuMU6/ /sOv3+74+sM41JFx2Erny1iHT8K2sypceR2IGnMK1lc5baQUddwU/pu8evVjtRfsiW W2fTMNPxJmeeYxZH8gnzCoQGmw6BWf1KCJHkLsYPuVscupnhrpQePV/JHwOIHkefGl lx20PITelMlNcoOR7ktmCyYURNo2pXb+7I93W5foootKU8aGlYTt/4/yBzo1T2CaWn uVddC0DIFSV0w== From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: Ard Biesheuvel , Russell King , Arnd Bergmann , Kees Cook , Keith Packard , Linus Walleij , Nick Desaulniers , Nicolas Pitre Subject: [PATCH v3 02/10] ARM: assembler: introduce bl_r and bl_m macros Date: Sun, 17 Oct 2021 15:17:15 +0200 Message-Id: <20211017131723.4034662-3-ardb@kernel.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20211017131723.4034662-1-ardb@kernel.org> References: <20211017131723.4034662-1-ardb@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=1797; h=from:subject; bh=zhPGaTjr4mYt5ZVj+I3r4jcIFtNN14alZ9CqE96rhlA=; b=owEB7QES/pANAwAKAcNPIjmS2Y8kAcsmYgBhbCJSvAHE3GyB8YcT2CYIx0jSPcZ+N5P9ht8vVJsb tmiQabOJAbMEAAEKAB0WIQT72WJ8QGnJQhU3VynDTyI5ktmPJAUCYWwiUgAKCRDDTyI5ktmPJA/hC/ 9L9Jn3jVc6XUpF+Ikz9Me5SmKqeUrWepfM7oanpDAyMNRG9y9GOdvBY9CWAJY6wv7O2LVhfoIwmIod u9U4mr8NvC5Dln/eyLu7pPkGQ/6tARGPoAfg1AGJYoJIuE7bjC2XSuzPg3RFmIUbDowcL4lggezdSn ZNT+TpXp1cO5RxFAzIHNSnhLubdsDPFopFX9BhJz+WSnZwvDeB8/Um2bOp2sbZFgnMp+KV/NVQBqbq eBLrx2CW1oFeW+TEXuM2LxZP3FVUcX0g42GFaLIowZh5Kv5J7nOvIP9qNjMOpM9e5a/L43o6wNpcTL cPR0JlzRHkCwKpupaP8gDr6S0tXc55/EMwCbZxWVJMhz1xFHxqCk0MP/UXTy6U8+BOBB0IakIX8xCv BysJnxeWOETweF90gCd7VcmZ5EX/6NM/42nzUncC6fj1MNmNc3KFPmgiYKd6lYijzB+8nHuCETpBPX PNyqtzimOV3y8chK990fyN1ALEn8Dh8akGFOy7e2WhroA= X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211017_061737_297192_E1132D08 X-CRM114-Status: GOOD ( 13.51 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Add some macros that abstract the difference between the ways indirect calls are performed on older and newer ARM architecture revisions. The main difference is to prefer blx instructions over explicit LR assignments when possible, as these tend to confuse the prediction logic in out-of-order cores when speculating across a function return. Signed-off-by: Ard Biesheuvel Reviewed-by: Arnd Bergmann Acked-by: Linus Walleij Tested-by: Keith Packard --- arch/arm/include/asm/assembler.h | 37 ++++++++++++++++++++ 1 file changed, 37 insertions(+) diff --git a/arch/arm/include/asm/assembler.h b/arch/arm/include/asm/assembler.h index 7d23d4bb2168..abb8202ef0da 100644 --- a/arch/arm/include/asm/assembler.h +++ b/arch/arm/include/asm/assembler.h @@ -624,4 +624,41 @@ THUMB( orr \reg , \reg , #PSR_T_BIT ) .endif .endm + /* + * bl_r - branch and link to register + * + * @dst: target to branch to + * @c: conditional opcode suffix + */ + .macro bl_r, dst:req, c + .if __LINUX_ARM_ARCH__ < 6 + mov\c lr, pc + mov\c pc, \dst + .else + blx\c \dst + .endif + .endm + + /* + * bl_m - branch and link to address held in memory + * + * @dst: memory reference to load the branch target from + * + * This uses the same syntax as ldr instructions, including things like + * register writeback (where it makes sense). For example, + * + * bl_m [ip, #] + * + * Note that @dst should not reference lr as it may be clobbered early. + */ + .macro bl_m, dst:vararg + .if __LINUX_ARM_ARCH__ < 6 + mov lr, pc + ldr pc, \dst + .else + ldr lr, \dst + blx lr + .endif + .endm + #endif /* __ASM_ASSEMBLER_H__ */ From patchwork Sun Oct 17 13:17:16 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 12564309 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E9636C433F5 for ; Sun, 17 Oct 2021 13:19:27 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B363D61260 for ; Sun, 17 Oct 2021 13:19:27 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org B363D61260 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=m5karYvYsLlEZvpCPI4jJXZyg4bgOmt0l4ISN/CcNRo=; b=u/oDQvKw8c9nk4 6a23DNgPlLwVidNDAue9kjCjP+XEYG0N+k7UPpoyp8CxuAU0rL0bsgZdZxLPYguUEUljd/AMbFHAs MlhJSBSozrJO+jISwuz9LS0cqh0Q3aeuumCJUVIMFRBBNYZgm1dWxAm1STLo8OIpDiPV5PgbPj1u6 a6AT4+d1+QfH37t7kCSiGi23klTRfKditsiaMb7iUSqOx0L7Mn33W0nX1RyEPwP5cevkxMhxp79ix IDGgEe75bGP5D3fj/rIn5BvX8WOdbcWswQcihKMC8U+mkBQysCeqjFm5y3z0K4lceG7RwmEfdxfPO Q6Fl7/KsPH9+1gdVdv4w==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mc63H-00CaiZ-2u; Sun, 17 Oct 2021 13:18:11 +0000 Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mc62l-00Cab5-PQ for linux-arm-kernel@lists.infradead.org; Sun, 17 Oct 2021 13:17:41 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id 587F061248; Sun, 17 Oct 2021 13:17:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1634476659; bh=G99G9KD+pR2ry5P+S6YGVbG/WqxJXWwyustgbvojX7E=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=hSOrnDWdg8HJfJeS4An+uXFmw3nqVJjGG3uWIzXAHTjCjKL8ziPucxHsErIMW8/1o 4O5acDHPXhBrQITQOyVbOoe0WW198sk4shRDXFVjwQIBs7JQ3wqzrujtUY90h5Cqel 4GtD7YO1iq0gbZQpVJbqrRUD9mYalg3GzpGMi8hLBftW7AtPRnmtHiAVvaN+gPAmKA CvbApI8BP3irSBIWsalbOtMoQfnGGdzzRi/czigLFRUQ6dIBUfFzAq+JEoxzkVd6v6 MtWm0Z0H7qYBceW+MWVwPAkBzndwR4UwEl190urn1PsJt1wuKAemZiDdUW/54vUno8 a2t99eQ3559tw== From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: Ard Biesheuvel , Russell King , Arnd Bergmann , Kees Cook , Keith Packard , Linus Walleij , Nick Desaulniers , Nicolas Pitre Subject: [PATCH v3 03/10] ARM: optimize indirect call to handle_arch_irq for v7 cores Date: Sun, 17 Oct 2021 15:17:16 +0200 Message-Id: <20211017131723.4034662-4-ardb@kernel.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20211017131723.4034662-1-ardb@kernel.org> References: <20211017131723.4034662-1-ardb@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=1384; h=from:subject; bh=G99G9KD+pR2ry5P+S6YGVbG/WqxJXWwyustgbvojX7E=; b=owEB7QES/pANAwAKAcNPIjmS2Y8kAcsmYgBhbCJUiArqG4jnO54xjKLbnURKMLuDr6yiDr6r0wh2 1So/OOmJAbMEAAEKAB0WIQT72WJ8QGnJQhU3VynDTyI5ktmPJAUCYWwiVAAKCRDDTyI5ktmPJFVSC/ 9GtDgtSzwwAhSzG7EaF/aP3hGHvzE34MjEmfwU6UiUtDGi5olBPCtpE6mFwh3nY3aIvU/T71I3HdE/ DMx00yVmt9htvVxYhpn+vqAi+3Av1S5BWxw7GNVqqRGaZHgpiVVGZ5E1wNqXtw8pUgm8tYJX2mnlGt 7KqvwRLNAn4EU8Up+20RbN1hXXtga9mpdFrFLxRRkNfL3Q3LOxPa2tHOohDsd/GKDFzVQiVr5uGxl+ /XEjTcBJyZO4dqEVq6nAFdWFSUbCivoCW1NR0SrTsWaRexfQM0wk7DOlQM3xp4tr+eaCUw3DkVPoV+ 7AhkwSq4d0b3eUhg42JsoTfLhBZj2sRqj2ndoJYQBl6fhq0TT201DFamyt7pi2ffm+T/zlMW//Uhwr dHQimn0c5jTtVpXgw41I2Rd5KRMx4jcp5UcVXs5RclJGX9UtWdmaEv+7GJyVl7PSXT1wQrVSeMDC7T TtXqv+vwP2tHfVvOSVrjTgn8/LPHtBGIEwwI8qAkhoAY4= X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211017_061739_868691_52D56821 X-CRM114-Status: GOOD ( 12.12 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The current irq_handler macro uses a literal load followed by an explicit assignment of lr before an indirect call, both of which are sub-optimal on recent ARM cores. Replace it by a mov_l call, which will evaluate to a movw/movt pair on v7 cores, avoiding one of the two loads, followed by a call to a newly introduced 'bl_m' macro, which will evaluate to a blx instruction on cores that support it. (Setting lr explicitly rather than via a bl or blx instruction confuses the return address prediction that modern out-of-order cores use to speculatively perform the function return.) Signed-off-by: Ard Biesheuvel Reviewed-by: Arnd Bergmann Acked-by: Linus Walleij Tested-by: Keith Packard --- arch/arm/kernel/entry-armv.S | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/arch/arm/kernel/entry-armv.S b/arch/arm/kernel/entry-armv.S index a54b5044d406..21896f702447 100644 --- a/arch/arm/kernel/entry-armv.S +++ b/arch/arm/kernel/entry-armv.S @@ -38,14 +38,12 @@ */ .macro irq_handler #ifdef CONFIG_GENERIC_IRQ_MULTI_HANDLER - ldr r1, =handle_arch_irq + mov_l r1, handle_arch_irq mov r0, sp - badr lr, 9997f - ldr pc, [r1] + bl_m [r1] #else arch_irq_handler_default #endif -9997: .endm .macro pabt_helper From patchwork Sun Oct 17 13:17:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 12564311 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E63E0C433EF for ; Sun, 17 Oct 2021 13:19:34 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id AF00361260 for ; Sun, 17 Oct 2021 13:19:34 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org AF00361260 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=By6uoqQ9oPlGu9eD6CCI2M7gMhHbSXKicZ7ZfUZXgEQ=; b=JWsCHU84BiyrQt g+yWn3U6gzGbybpcfToJVHyQSCAtfGs2AAzwnmnhWIUsOXmT6fPbefRwqyiJjg6lcsW06PTfTsCV+ PsMtWoJNwKDDBe5eX5bx6U8Ic1KEinVg1MmznKv+/dvo1rLm9CuUkMKJj3ZuDnh6evMSsrLbfEgPM 09ne+vF5jtJzB4S6DdR9O0xAKlUpwOv6hiiOJWwCpgyd3ppVFgCVCiICszwYEWRPKIZ7bMACKf+Uh /w2AhymxMxJ5x2bc5t2PO0JtCOQMM9YpgDulq1M5vrDYnYIreoMEx02goxfoIEWFRg5ppGNuUxmRs ajBjjfOl9GmOqoogpuDA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mc63S-00Calx-EN; Sun, 17 Oct 2021 13:18:22 +0000 Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mc62p-00CabS-EF for linux-arm-kernel@lists.infradead.org; Sun, 17 Oct 2021 13:17:44 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id F1FCC61260; Sun, 17 Oct 2021 13:17:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1634476662; bh=UNhsqFF6lWxejQQEY2CUNtek34KtXIXbhy6vPpJObyg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=fT/LE4XZrcqadEgenzJ4ggtWviRh0/zyFluRaffGDPgYfXsdigkmiGPG7bw43b+Cp 2S2Wt7LAmn67ghXOOCLQa7o5mRbRy/q6+Ou0ZkKJO3asc/0R4+8fqZ/mQ7SLd/0DgE OnBZwaDAy1z9sUG7JUNnhRg0Qk5RL1mW881zOrVN/GzcisOZTpTYc2cuCV+oOtYDx4 V+8so7JOeTma44BBiwsqm58qLwS/viPpxGs//3/twy0tcPh/G9iAnMR+VGTluFWZkI rA3rpdBXNP6Tpm7ScKGeX3bQCKP40hK12BqOIBrm5gTczuwnU/4nlnTCz6oqFjR8a6 gPI3PYjGBrt6A== From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: Ard Biesheuvel , Russell King , Arnd Bergmann , Kees Cook , Keith Packard , Linus Walleij , Nick Desaulniers , Nicolas Pitre Subject: [PATCH v3 04/10] ARM: unwind: support unwinding across multiple stacks Date: Sun, 17 Oct 2021 15:17:17 +0200 Message-Id: <20211017131723.4034662-5-ardb@kernel.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20211017131723.4034662-1-ardb@kernel.org> References: <20211017131723.4034662-1-ardb@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=3497; h=from:subject; bh=UNhsqFF6lWxejQQEY2CUNtek34KtXIXbhy6vPpJObyg=; b=owEB7QES/pANAwAKAcNPIjmS2Y8kAcsmYgBhbCJWTRKtkpo7eMVQFzOjsKyA+STe3jLh999ziPTo 7ayb8YqJAbMEAAEKAB0WIQT72WJ8QGnJQhU3VynDTyI5ktmPJAUCYWwiVgAKCRDDTyI5ktmPJM4RC/ 0dA0ryKZREjyXsg1alQHsfFLbdSis+/gYonnDVQd1WkUS4BQbFe9D6SgPE8cUcZwhnIgbCNl6bcJgs n/2jfQ18H2/yTKuQSd+FlJ0H9RHPsLsmk7xJyPlEyKiH6wHx+oTk7Ouw5xaToXOrkxe0Zro2vgshVW +0JtvTtFWAeYf+2htvqVUBIHUic8HK7RcIDMpI1dfb5A0Ae4ILa0zPtOURUIF5SIoBIiBpAupHaqO6 x5+YgIRtCJSy5FIm/oesAIT1Yi1+GQnyIOmS6xghDk1EsjTp+ZOwkapvBLvnT61jV1jq8FSe9mxw8R UOI4pJUAoqmPxgl9qY2g28yblx2RBWNY6XD6TiTq9XMj0K3dbwHK36Zn/G++vYbKgz/0E2jGVH/Dtr 1OgttNdLnni2r2lVfKx5WubcQJMWlangP9+9z0pLdEU9GOEYtjkuM2/JgBOZb3A5ZVqt0uiUEMqZL9 84Q2JgUdhu30YzomVthrBal3RRaPmZsaRFXqTfJlkqDQY= X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211017_061743_548859_E5F9D927 X-CRM114-Status: GOOD ( 19.29 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Implement support in the unwinder for dealing with multiple stacks. This will be needed once we add support for IRQ stacks, or for the overflow stack used by the vmap'ed stacks code. This involves tracking the unwind opcodes that either update the virtual stack pointer from another virtual register, or perform an explicit subtract on the virtual stack pointer, and updating the low and high bounds that we use to sanitize the stack pointer accordingly. Signed-off-by: Ard Biesheuvel Reviewed-by: Arnd Bergmann Acked-by: Linus Walleij Tested-by: Keith Packard --- arch/arm/kernel/unwind.c | 25 +++++++++++++------- 1 file changed, 16 insertions(+), 9 deletions(-) diff --git a/arch/arm/kernel/unwind.c b/arch/arm/kernel/unwind.c index 59fdf257bf8b..9cb9af3fc433 100644 --- a/arch/arm/kernel/unwind.c +++ b/arch/arm/kernel/unwind.c @@ -52,6 +52,7 @@ EXPORT_SYMBOL(__aeabi_unwind_cpp_pr2); struct unwind_ctrl_block { unsigned long vrs[16]; /* virtual register set */ const unsigned long *insn; /* pointer to the current instructions word */ + unsigned long sp_low; /* lowest value of sp allowed */ unsigned long sp_high; /* highest value of sp allowed */ /* * 1 : check for stack overflow for each register pop. @@ -256,8 +257,12 @@ static int unwind_exec_pop_subset_r4_to_r13(struct unwind_ctrl_block *ctrl, mask >>= 1; reg++; } - if (!load_sp) + if (!load_sp) { ctrl->vrs[SP] = (unsigned long)vsp; + } else { + ctrl->sp_low = ctrl->vrs[SP]; + ctrl->sp_high = ALIGN(ctrl->sp_low, THREAD_SIZE); + } return URC_OK; } @@ -313,9 +318,10 @@ static int unwind_exec_insn(struct unwind_ctrl_block *ctrl) if ((insn & 0xc0) == 0x00) ctrl->vrs[SP] += ((insn & 0x3f) << 2) + 4; - else if ((insn & 0xc0) == 0x40) + else if ((insn & 0xc0) == 0x40) { ctrl->vrs[SP] -= ((insn & 0x3f) << 2) + 4; - else if ((insn & 0xf0) == 0x80) { + ctrl->sp_low = ctrl->vrs[SP]; + } else if ((insn & 0xf0) == 0x80) { unsigned long mask; insn = (insn << 8) | unwind_get_byte(ctrl); @@ -330,9 +336,11 @@ static int unwind_exec_insn(struct unwind_ctrl_block *ctrl) if (ret) goto error; } else if ((insn & 0xf0) == 0x90 && - (insn & 0x0d) != 0x0d) + (insn & 0x0d) != 0x0d) { ctrl->vrs[SP] = ctrl->vrs[insn & 0x0f]; - else if ((insn & 0xf0) == 0xa0) { + ctrl->sp_low = ctrl->vrs[SP]; + ctrl->sp_high = ALIGN(ctrl->sp_low, THREAD_SIZE); + } else if ((insn & 0xf0) == 0xa0) { ret = unwind_exec_pop_r4_to_rN(ctrl, insn); if (ret) goto error; @@ -375,13 +383,12 @@ static int unwind_exec_insn(struct unwind_ctrl_block *ctrl) */ int unwind_frame(struct stackframe *frame) { - unsigned long low; const struct unwind_idx *idx; struct unwind_ctrl_block ctrl; /* store the highest address on the stack to avoid crossing it*/ - low = frame->sp; - ctrl.sp_high = ALIGN(low, THREAD_SIZE); + ctrl.sp_low = frame->sp; + ctrl.sp_high = ALIGN(ctrl.sp_low, THREAD_SIZE); pr_debug("%s(pc = %08lx lr = %08lx sp = %08lx)\n", __func__, frame->pc, frame->lr, frame->sp); @@ -437,7 +444,7 @@ int unwind_frame(struct stackframe *frame) urc = unwind_exec_insn(&ctrl); if (urc < 0) return urc; - if (ctrl.vrs[SP] < low || ctrl.vrs[SP] >= ctrl.sp_high) + if (ctrl.vrs[SP] < ctrl.sp_low || ctrl.vrs[SP] > ctrl.sp_high) return -URC_FAILURE; } From patchwork Sun Oct 17 13:17:18 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 12564313 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0490AC433F5 for ; Sun, 17 Oct 2021 13:19:54 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C01B161260 for ; Sun, 17 Oct 2021 13:19:53 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org C01B161260 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=EXKhozKaJKaIpg0Ft3asqRJZqnfrutFv05ppMxVzbNM=; b=GXD4+MgQM2Iiz5 yx/H/8uf1jwOwIkG6wMm9ZD2hNG0IgIR0whsHt6sjgbqukf2sKpHCREHMwloOmfPi3ziKhYR+vOWo CfFrlmVrSJRuXGEDNXD47XHZLA9paGirieV9f9MangpIjxi5mVTKy1cKxMJ2gX5vCHFOYbDntOrmA yRs1GTw470gFFYeuK6c/nlkPpsGrf6kq6jC0nO5eNWB+izTb/ubnCWH06iW97Q1Ez/c9phLToMcHi hVsQgfgu7Lz1iOisHHxiI+MnVWlEpeKQ9AHtJo2deIANc5oiR75/jvh5NCww/2jP4mRKVSDrODyyT tebGW2aV3R4Q9BNgl5+g==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mc63f-00CaqH-Mw; Sun, 17 Oct 2021 13:18:35 +0000 Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mc62r-00Cabs-27 for linux-arm-kernel@lists.infradead.org; Sun, 17 Oct 2021 13:17:46 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id 9897C60FF2; Sun, 17 Oct 2021 13:17:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1634476664; bh=Q4V4waG2Y64DsC7IEGrfB3ehvrACB8oh1KcBym0q/ek=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Iv2625urm2Ye4FtsF1/PYMl1A/H2bGg9uHYmFYYabAiI2gyD5NHAf3+N4TG2HVtck NTMGW4p6I4HrMKF82FaRZE3AumaiT6Axv4Bfzo0XHgNZK+CJ57gos0YRZ5MCcoT/Bx eu/lfggkXj4srJWjqksULoWKIyH7W1zrZygzN+m+oel4Q5ZbYhp+hLtgUmes9OQVWD WAj2R4b1xfRALpQ4Bi82FyuxToec6pOwO9pq9QEanzDAZh80rs+KEptPVdYyB1Q31G I/0VEM0DD0lp8a+/+2ZIyce26PSFjF+e90ZEYymdsASkARbs0NGOX41i/iuQ0SM6tG Eyg2RdxL1QGPw== From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: Ard Biesheuvel , Russell King , Arnd Bergmann , Kees Cook , Keith Packard , Linus Walleij , Nick Desaulniers , Nicolas Pitre Subject: [PATCH v3 05/10] ARM: export dump_mem() to other objects Date: Sun, 17 Oct 2021 15:17:18 +0200 Message-Id: <20211017131723.4034662-6-ardb@kernel.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20211017131723.4034662-1-ardb@kernel.org> References: <20211017131723.4034662-1-ardb@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=2163; h=from:subject; bh=Q4V4waG2Y64DsC7IEGrfB3ehvrACB8oh1KcBym0q/ek=; b=owEB7QES/pANAwAKAcNPIjmS2Y8kAcsmYgBhbCJYYMvRJEm0/6ZcTxjoP5aiPYzSt9tJP0oSBxYf CFQh+iKJAbMEAAEKAB0WIQT72WJ8QGnJQhU3VynDTyI5ktmPJAUCYWwiWAAKCRDDTyI5ktmPJNQEC/ 4lUYofzzbq7FMnUMKfcpIkorXK0UhbJ38Tm96DDdaHXEbu4HXIIS5z/gG7uryvmZxK2YlTBa9bKNVh O638RPfh8sHS7IRr5r8rg/QqON8+IMlUId653H/e5HBcfDcItfhjBjNuxmLIOmEN0J/DZS1fCBJ32p 528ccR2MGe3oEYQkMLkbNBMFusV+VutOwVCHoXd0YVPuqKXIoro2u9ShKzxAo7c1hm9MkBQNoA1Q4a 8gcywPlStsUq6AmWLRBE+dQVxSwTjx2sZLz+3uaSqNE/1C4dyU2qSMUYMuW2G1n+0IrqiAa79p48dt 4YwjQMfrV2hIeT+7f1KX2JQE62M57feQKNvIxKzRD1H2q/rQDwpH2YUp2vt5qPmBNRJimldRyQhtP6 deskHB/Tm8RS7LS09t+cyAOaU1vkVGBIEwtwWWlT6tp35kgdfBLu3yB39ifeAS9+F+rXUYMiCmSDMk qFtzMOFJULw+8KovvqKihH5yHCcy4BgwRvtW2pUfBmDlw= X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211017_061745_171197_FB6D224D X-CRM114-Status: GOOD ( 13.37 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The unwind info based stack unwinder will make its own call to dump_mem() to dump the exception stack, so give it external linkage. Signed-off-by: Ard Biesheuvel Reviewed-by: Arnd Bergmann Acked-by: Linus Walleij Tested-by: Keith Packard --- arch/arm/include/asm/stacktrace.h | 2 ++ arch/arm/kernel/traps.c | 7 +++---- 2 files changed, 5 insertions(+), 4 deletions(-) diff --git a/arch/arm/include/asm/stacktrace.h b/arch/arm/include/asm/stacktrace.h index 2d76a2e29f05..e86f55155500 100644 --- a/arch/arm/include/asm/stacktrace.h +++ b/arch/arm/include/asm/stacktrace.h @@ -27,5 +27,7 @@ void arm_get_current_stackframe(struct pt_regs *regs, struct stackframe *frame) extern int unwind_frame(struct stackframe *frame); extern void walk_stackframe(struct stackframe *frame, int (*fn)(struct stackframe *, void *), void *data); +extern void dump_mem(const char *lvl, const char *str, unsigned long bottom, + unsigned long top); #endif /* __ASM_STACKTRACE_H */ diff --git a/arch/arm/kernel/traps.c b/arch/arm/kernel/traps.c index 4a7edc6e848f..b1ea2e4870ce 100644 --- a/arch/arm/kernel/traps.c +++ b/arch/arm/kernel/traps.c @@ -35,6 +35,7 @@ #include #include #include +#include #include #include @@ -60,8 +61,6 @@ static int __init user_debug_setup(char *str) __setup("user_debug=", user_debug_setup); #endif -static void dump_mem(const char *, const char *, unsigned long, unsigned long); - void dump_backtrace_entry(unsigned long where, unsigned long from, unsigned long frame, const char *loglvl) { @@ -118,8 +117,8 @@ static int verify_stack(unsigned long sp) /* * Dump out the contents of some memory nicely... */ -static void dump_mem(const char *lvl, const char *str, unsigned long bottom, - unsigned long top) +void dump_mem(const char *lvl, const char *str, unsigned long bottom, + unsigned long top) { unsigned long first; int i; From patchwork Sun Oct 17 13:17:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 12564315 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 09CBFC433F5 for ; Sun, 17 Oct 2021 13:20:19 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C329360F9E for ; Sun, 17 Oct 2021 13:20:18 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org C329360F9E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=LOrnmSl2DGFto+Lv76PTa+gg7eZHc6xNV2GXt4ohUsU=; b=uXd5zvRxGN5COX L76bh3u2yMctqT0cebTS//eQkbCgUVZWJCQhtRVo6X4ZfdtYQHK2pM2IHBgEnFCoFa5XViw/9u49L AfOR8QFZuaMmFQ5ppx+T+DKDIieGpHFuAL2FTfYCECu//5aj8e8I9hmika7W6vPD/I63H6RbQBf5T vThuQ2+kgH30k/d/wirjapjVHFVh9fA/a6K4KpXvJltrymjRstyYBd7CQKrrX+yvUDHdePIT+LR6c 6jpXyT9SiQ91Do6+qCn+ybpn5SuKWVe1rF0Ceyxy9Xckbv8E4iXWGjH2CjTFV82wdnMV2vf+FYFSU Yk/kQeBjIg9muCnPjaNw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mc63w-00Cays-MU; Sun, 17 Oct 2021 13:18:52 +0000 Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mc62t-00Cacb-Nz for linux-arm-kernel@lists.infradead.org; Sun, 17 Oct 2021 13:17:49 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id 3D65A61038; Sun, 17 Oct 2021 13:17:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1634476667; bh=WCtbL8eWm9oMJI6f5UjA9WazSfc2NFC7dxpRMWhxVx8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Kn3QSZnpn+8GKfNGDahmFSW4fzRgZvLBaOSI6zEqT9ZWCIOgkLuOfkoHVXQAuJQ7O CUw/BaGvd9OaJ0QGLfdU4MleLEfpzSi80K5oIW+KBglajNNP6jfeBkYQ/XoQ5kE7Vf oCcBhqx1NkOX/5n7xxEWJrxGfkcYxR8vpa4XpGsoyRgYKzJA5jYGeEGFkmxty5ivia gK9+hdxtvvCjrbMwc0N46IEab8XBKmkfMYQKWgQPU/I237h1dilc13NbLdt3nnTLHq QMDChT7tJ2CyJxht2pBArKxVV7EO8Wgh8fh1+P8puJ4fL71Euj48rQ+Ud2kjwbjcnW PE0vlTqHzf3ug== From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: Ard Biesheuvel , Russell King , Arnd Bergmann , Kees Cook , Keith Packard , Linus Walleij , Nick Desaulniers , Nicolas Pitre Subject: [PATCH v3 06/10] ARM: unwind: dump exception stack from calling frame Date: Sun, 17 Oct 2021 15:17:19 +0200 Message-Id: <20211017131723.4034662-7-ardb@kernel.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20211017131723.4034662-1-ardb@kernel.org> References: <20211017131723.4034662-1-ardb@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=3329; h=from:subject; bh=WCtbL8eWm9oMJI6f5UjA9WazSfc2NFC7dxpRMWhxVx8=; b=owEB7QES/pANAwAKAcNPIjmS2Y8kAcsmYgBhbCJaofeU3TJ2gvYRaVAFybndjO9ZVY8sudSAWpjk IoKJqfCJAbMEAAEKAB0WIQT72WJ8QGnJQhU3VynDTyI5ktmPJAUCYWwiWgAKCRDDTyI5ktmPJHGLC/ 9jVFHnDtSzCHjFE16Vw3wVFHjADCBkDsPvtLYRm8DFNJtMx5Z44rNZCAm5lhF4f/UhKRp9GlWsIpUu 1hTk96JQ9rP5IHdcK/ojW9eRIpMv/daZNW2Tr6s5AMwekDa7e+CceX6nWInMCxg+IN4OB46xZCqv4S vcLGK/3Lb9b5HYH6BmkpUyTB1Nr8ov9AX4TxJpQ9i5ql2DusRJdR295ilylyIsR8QnOK36VKz0B0ca wgJTnv7kSDnosb3HF++0f1WyM2khopDX0CXSYXRz6sw52sV638bi3CdmNIK6a+cYWsGg45JijOLoWZ EIHnOZWih2MFD5IoeO9qOsuzVLPXazY48qfKv8GdJ0vfTY07xbbYjgAumaPzf8LdOL/+XKZJCH5a59 j6eyjDpolKw1SLzGsmTRXeKUCvDkHPR5OVsVOeFeabEO6Jw801hy2oolzJrcsC1f2zWlswu8Tj8JIc 22aqSzryveFUvt2FeNZOrnUFIC1/Z/Z57HF+Aj6fQYPeE= X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211017_061747_853894_07E08FFB X-CRM114-Status: GOOD ( 18.78 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The existing code that dumps the contents of the pt_regs structure passed to __entry routines does so while unwinding the callee frame, and dereferences the stack pointer as a struct pt_regs*. This will no longer work when we enable support for IRQ or overflow stacks, because the struct pt_regs may live on the task stack, while we are executing from another stack. The unwinder has access to this information, but only while unwinding the calling frame. So let's combine the exception stack dumping code with the handling of the calling frame as well. By printing it before dumping the caller/callee addresses, the output order is preserved. Signed-off-by: Ard Biesheuvel Reviewed-by: Arnd Bergmann Acked-by: Linus Walleij Tested-by: Keith Packard --- arch/arm/include/asm/stacktrace.h | 11 +++++++++++ arch/arm/kernel/traps.c | 3 ++- arch/arm/kernel/unwind.c | 8 +++++++- 3 files changed, 20 insertions(+), 2 deletions(-) diff --git a/arch/arm/include/asm/stacktrace.h b/arch/arm/include/asm/stacktrace.h index e86f55155500..8dcad8156561 100644 --- a/arch/arm/include/asm/stacktrace.h +++ b/arch/arm/include/asm/stacktrace.h @@ -13,6 +13,17 @@ struct stackframe { unsigned long sp; unsigned long lr; unsigned long pc; + +#ifdef CONFIG_ARM_UNWIND + /* + * This field is used to track the stack pointer value when calling + * __entry routines. This is needed when IRQ stacks and overflow stacks + * are used, because in that case, the struct pt_regs passed to these + * __entry routines may be at the top of the task stack, while we are + * executing from another stack. + */ + unsigned long sp_low; +#endif }; static __always_inline diff --git a/arch/arm/kernel/traps.c b/arch/arm/kernel/traps.c index b1ea2e4870ce..cda32385552b 100644 --- a/arch/arm/kernel/traps.c +++ b/arch/arm/kernel/traps.c @@ -74,7 +74,8 @@ void dump_backtrace_entry(unsigned long where, unsigned long from, loglvl, where, from); #endif - if (in_entry_text(from) && end <= ALIGN(frame, THREAD_SIZE)) + if (!IS_ENABLED(CONFIG_UNWINDER_ARM) && + in_entry_text(from) && end <= ALIGN(frame, THREAD_SIZE)) dump_mem(loglvl, "Exception stack", frame + 4, end); } diff --git a/arch/arm/kernel/unwind.c b/arch/arm/kernel/unwind.c index 9cb9af3fc433..b7a6141c342f 100644 --- a/arch/arm/kernel/unwind.c +++ b/arch/arm/kernel/unwind.c @@ -29,6 +29,7 @@ #include #include +#include #include #include #include @@ -459,6 +460,7 @@ int unwind_frame(struct stackframe *frame) frame->sp = ctrl.vrs[SP]; frame->lr = ctrl.vrs[LR]; frame->pc = ctrl.vrs[PC]; + frame->sp_low = ctrl.sp_low; return URC_OK; } @@ -502,7 +504,11 @@ void unwind_backtrace(struct pt_regs *regs, struct task_struct *tsk, urc = unwind_frame(&frame); if (urc < 0) break; - dump_backtrace_entry(where, frame.pc, frame.sp - 4, loglvl); + if (in_entry_text(where)) + dump_mem(loglvl, "Exception stack", frame.sp_low, + frame.sp_low + sizeof(struct pt_regs)); + + dump_backtrace_entry(where, frame.pc, 0, loglvl); } } From patchwork Sun Oct 17 13:17:20 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 12564317 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BFB58C433F5 for ; Sun, 17 Oct 2021 13:20:30 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 852B361260 for ; Sun, 17 Oct 2021 13:20:30 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 852B361260 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=KmXq6Cjyq7y5Zna29dxI7H/BJGkNf/zgc6J2FWa4gjA=; b=TTxzvvcNhNGBut 8g/3hzIeuZMLFe7Kb7AGROxTSRufNxtf4jeNJAnnqHD74PcIGrI0viMvX6HDtengBT9CgnhG3TLZb BRicMsY2rSVzZinQnXYq2JffdDPnX/MB9S+BdkeARgJOfdLagM3wXDCyXWpOziwuyojG8ndnUwKx8 I9Acen+/dYNIF58X5iiDp7OU+/VAhLVzXtOhLBc8XfqYFPjIeTPE7gmJkAHe5hB87pG1K/tc8PQ7X kEdILm+V1mNuaBwEsTlb+Y4UkCVorxSN54fzeVakYhwrl4pzRZRd9EmWl7oBKdXWiAGXeKYLLA/Af i7MCF5stsc5huTP6CNJQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mc64E-00Cb7I-6b; Sun, 17 Oct 2021 13:19:10 +0000 Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mc62w-00CadX-9y for linux-arm-kernel@lists.infradead.org; Sun, 17 Oct 2021 13:17:51 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id D596361248; Sun, 17 Oct 2021 13:17:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1634476670; bh=n6ZLz4R2AQLlOKG2q9QP07c9sYrfSKTHO3Dw4AMpi+A=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ROUJ3zhORSjgRDjtV3EssySuMwFgQncws8rj1NlqtMFSjvuRCA2RG5+iYy9U/k5iw ScuYqWbujmh6yHhyv+ZgfQwiLBTLnkJVM7Hog3nY1L1SWX/M/meo7CBPTRQLD/yiDn ync/IJLSvPTfjSOUoeTQTLrVAqtSzzjr0OhpUIeozOf1uFphIZ5LKUwK3VgYdZwGyc 3GpcuHKiXmiNHohMqHUloweZLwyisui378Wcq3f2iAyPFIs8NHBpGXdai48hOpMqZs koY9DfbkbOKV6KON6BI2RdjRHsugbxtgDnW+RsgsUuFc/DJg9oWITE/UJPb17WKe90 my6iVKx7hAc0g== From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: Ard Biesheuvel , Russell King , Arnd Bergmann , Kees Cook , Keith Packard , Linus Walleij , Nick Desaulniers , Nicolas Pitre Subject: [PATCH v3 07/10] ARM: backtrace-clang: avoid crash on bogus frame pointer Date: Sun, 17 Oct 2021 15:17:20 +0200 Message-Id: <20211017131723.4034662-8-ardb@kernel.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20211017131723.4034662-1-ardb@kernel.org> References: <20211017131723.4034662-1-ardb@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=1682; h=from:subject; bh=n6ZLz4R2AQLlOKG2q9QP07c9sYrfSKTHO3Dw4AMpi+A=; b=owEB7QES/pANAwAKAcNPIjmS2Y8kAcsmYgBhbCJc6EbBIeO1q9gX8PB+hWVbtSDkELzzSC8RdlZF Vva74f+JAbMEAAEKAB0WIQT72WJ8QGnJQhU3VynDTyI5ktmPJAUCYWwiXAAKCRDDTyI5ktmPJN7UC/ 9CDSqQ5PfCg0Z1oPKtQwzG4TSKMmlZ+QeeuLk1UbvpjAbjCevbXMuqF1X3Vw9gNdxcEKcliSG/yLMU xBIR4BZmXoXNi4o5Fq8ip+17bkLmhWnPKCapvlUQl5zFixal2cJmBjq4rra7pwbspv6m596N1ceTyp Glb3IOvleFLmrwXdbaU8mMTGry5wG06sblEaGDB4t3NoRMoniPHGGWPMQuvG2BaFMoV+bqkGqLkKE+ YCwMAuFwqqgtDnHb6C/7UuRLAiHY5gndDtOE56P068D+wmuFH0iBN8coHxOGwcXvv2zB7knc7+0Iq/ O5MECICXQ8XI9WKJiIwz/kaWuh+WQwpKyglsMYQeKue67ixfMD9FN2Llm41So/WeNqdQEE60DKngYU TZtHq6t6fbAnX6zE2YxxFoKw4amtAckgtDYGmzcIixY686omsx+q5tn9hQ12fzd8zpYPhYUKLUCJsW kcqBqbKZxKkObnhZf1iKJ2DyKQU9gWvoqZ9jmCqHpe57w= X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211017_061750_400332_12E5E97A X-CRM114-Status: GOOD ( 13.42 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The Clang backtrace code dereferences the link register value pulled from the stack to decide whether the caller was a branch-and-link instruction, in order to subsequently decode the offset to find the start of the calling function. Unlike other loads in this routine, this one is not protected by a fixup, and may therefore cause a crash if the address in question is bogus. So let's fix this, by treating the fault as a failure to decode the 'bl' instruction. To avoid a label renum, reuse a fixup label that guards an instruction that cannot fault to begin with. Signed-off-by: Ard Biesheuvel Reviewed-by: Nick Desaulniers --- arch/arm/lib/backtrace-clang.S | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/arch/arm/lib/backtrace-clang.S b/arch/arm/lib/backtrace-clang.S index 5b2cdb1003e3..5b4bca85d06d 100644 --- a/arch/arm/lib/backtrace-clang.S +++ b/arch/arm/lib/backtrace-clang.S @@ -144,7 +144,7 @@ for_each_frame: tst frame, mask @ Check for address exceptions */ 1003: ldr sv_lr, [sv_fp, #4] @ get saved lr from next frame - ldr r0, [sv_lr, #-4] @ get call instruction +1004: ldr r0, [sv_lr, #-4] @ get call instruction ldr r3, .Lopcode+4 and r2, r3, r0 @ is this a bl call teq r2, r3 @@ -164,7 +164,7 @@ finished_setup: /* * Print the function (sv_pc) and where it was called from (sv_lr). */ -1004: mov r0, sv_pc + mov r0, sv_pc mov r1, sv_lr mov r2, frame @@ -210,7 +210,7 @@ ENDPROC(c_backtrace) .long 1001b, 1006b .long 1002b, 1006b .long 1003b, 1006b - .long 1004b, 1006b + .long 1004b, finished_setup .long 1005b, 1006b .popsection From patchwork Sun Oct 17 13:17:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 12564319 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5D571C433F5 for ; Sun, 17 Oct 2021 13:21:15 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1E97861260 for ; Sun, 17 Oct 2021 13:21:15 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 1E97861260 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=ZPYZikuy1hfHlaNpxqyEZiWsdwF/vrfyJktFbv3FFFo=; b=DZWTggIRQO8+iB N0badOPlUvcOCBYVSPiplYaLkfloF9fcCjmjrU5M0ybgz+SoETg2erlFhUBtLsVVLVppT+UXf5R/w CIO70l/jxy2kv6qIrM3DmGbbaYclc658fS0Q1T7mW1dDM3dmeetkZ0gHg9YR2r5Qqog1ooHEyZaSP O1/8BqJ8TU98Dp5kT2oFAIPOt/t0vqQwrQU3XjNDQMANA1Lt+5fBayNKFHZWlp7OzSu8Ji4d/a0mL Y9YWH7byoxbAZDZ0vz/ENsKceqKhDUq1vRj+z6RltZKt7jQgqLb4p0zipW8zBLsAgYu5VKNwuleid 5xB3GkouRG4XTjZA8mJw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mc64j-00CbP8-SI; Sun, 17 Oct 2021 13:19:42 +0000 Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mc62y-00Cae3-Uq for linux-arm-kernel@lists.infradead.org; Sun, 17 Oct 2021 13:17:54 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id 7A69A60F9E; Sun, 17 Oct 2021 13:17:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1634476672; bh=Jh31lop36U9UoBRVUxxn06wC8IwH6k1aGBTjDOR+3bs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=IIGL097Wx3hgcfXKUrf7gdbSWwy+7Ml9DQ4Ycw1m99Sn3tey4NclpjeDFUz3XcDB9 BGYDaSArZ5mqhyA+HkYGJHuiizKjlAn8UuqDFN8102ADi53b7f5oYVoXezjKDgpyof 2IQgtsMjX9IL+spGHQjaeEftEU+SkdZ9E2PtMK4hqV0CcY3/JWiq/HpDNtXj6y2vGn Ob3ZZ14cro624EEl7mr50XAdd7YoJgGOZNbcrzI5o1+j05iFM4qQHQ0BeUxwD4DW4w 3rX/GKsxEbJobg2EQ75rJHpwOx40a1KQ/VDv16hnFc/LKC5Yxj/pccc6PLFs5bqX4Q VKtbLGfOj1RNA== From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: Ard Biesheuvel , Russell King , Arnd Bergmann , Kees Cook , Keith Packard , Linus Walleij , Nick Desaulniers , Nicolas Pitre Subject: [PATCH v3 08/10] ARM: implement IRQ stacks Date: Sun, 17 Oct 2021 15:17:21 +0200 Message-Id: <20211017131723.4034662-9-ardb@kernel.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20211017131723.4034662-1-ardb@kernel.org> References: <20211017131723.4034662-1-ardb@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=7684; h=from:subject; bh=Jh31lop36U9UoBRVUxxn06wC8IwH6k1aGBTjDOR+3bs=; b=owEB7QES/pANAwAKAcNPIjmS2Y8kAcsmYgBhbCJe0RiC9X+C/WWF9cKRjFuQV3vOt0qYPNR93J1E kdbYM6eJAbMEAAEKAB0WIQT72WJ8QGnJQhU3VynDTyI5ktmPJAUCYWwiXgAKCRDDTyI5ktmPJFTIDA CqCEJZJaPBJ49T16Gh1LXzU38sbQ8/2jTxgy1D1R3zBb54mgwWsa3rPPPIJVTCa9zwBYnW2Fq/t0co JQliJ7l2GpC6QrBWTR1uhZOFiywSJbRoiyNfgGegPNXNVvUelxiJ0w1koY0Nk4U5tQjCbiF21AqrzE l8QhmMk/xwFYE7VAugt1OJahF+Rt5J7UPbg9iLhmNqppJBzEpFoR73pPsVGj9JZPo6+DyH0bv2fkDB fesWtqlMWhYF/R1Udj10EiB5YWq7iGvg8sJkj3yXh6e/r8bkYzDZzFWOuvCaczq0shImpvU3co/eRX gpH0izQc0sqsj4j9Na2VUBEgZKaDbIP5dU3yeOGJrBsYMcW6QksFwIKE1jNFC3cmnoU2MIegqnaeMg yeylMJnXBAawRmjlesAaUVVKudLVzbAxKyuUD/3amFLXWUw3XLhMS/EXq3rUUz+Ctbh8oydDqXlWx8 Q4fWB85g3lJvNoDOOfrUBeFWeX65c3VQ1BeaO6MeSwpYU= X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211017_061753_063427_127FCD30 X-CRM114-Status: GOOD ( 23.97 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Now that we no longer rely on the stack pointer to access the current task struct or thread info, we can implement support for IRQ stacks cleanly as well. Define a per-CPU IRQ stack and switch to this stack when taking an IRQ, provided that we were not already using that stack in the interrupted context. This is never the case for IRQs taken from user space, but ones taken while running in the kernel could fire while one taken from user space has not completed yet. Signed-off-by: Ard Biesheuvel Acked-by: Linus Walleij Tested-by: Keith Packard Acked-by: Nick Desaulniers --- arch/arm/Kconfig | 4 ++ arch/arm/include/asm/assembler.h | 4 ++ arch/arm/kernel/entry-armv.S | 54 ++++++++++++++++++-- arch/arm/kernel/irq.c | 23 +++++++++ arch/arm/kernel/traps.c | 15 +++++- arch/arm/lib/backtrace-clang.S | 8 +++ arch/arm/lib/backtrace.S | 8 +++ 7 files changed, 112 insertions(+), 4 deletions(-) diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig index 4f61c9789e7f..a8c1db0736f3 100644 --- a/arch/arm/Kconfig +++ b/arch/arm/Kconfig @@ -1163,6 +1163,10 @@ config CURRENT_POINTER_IN_TPIDRURO def_bool y depends on SMP && CPU_32v6K && !CPU_V6 +config IRQSTACKS + def_bool y + depends on GENERIC_IRQ_MULTI_HANDLER && THREAD_INFO_IN_TASK + config ARM_CPU_TOPOLOGY bool "Support cpu topology definition" depends on SMP && CPU_V7 diff --git a/arch/arm/include/asm/assembler.h b/arch/arm/include/asm/assembler.h index abb8202ef0da..405950494208 100644 --- a/arch/arm/include/asm/assembler.h +++ b/arch/arm/include/asm/assembler.h @@ -86,6 +86,10 @@ #define IMM12_MASK 0xfff +/* the frame pointer used for stack unwinding */ +ARM( fpreg .req r11 ) +THUMB( fpreg .req r7 ) + /* * Enable and disable interrupts */ diff --git a/arch/arm/kernel/entry-armv.S b/arch/arm/kernel/entry-armv.S index 21896f702447..2a426bd27a69 100644 --- a/arch/arm/kernel/entry-armv.S +++ b/arch/arm/kernel/entry-armv.S @@ -36,11 +36,59 @@ /* * Interrupt handling. */ - .macro irq_handler + .macro irq_handler, from_user:req #ifdef CONFIG_GENERIC_IRQ_MULTI_HANDLER mov_l r1, handle_arch_irq mov r0, sp +#ifdef CONFIG_IRQSTACKS + mov_l r2, irq_stack_ptr @ Take base address + mrc p15, 0, r3, c13, c0, 4 @ Get CPU offset +#ifdef CONFIG_UNWINDER_ARM + mov fpreg, sp @ Preserve original SP +#else + mov r8, fp @ Preserve original FP + mov r9, sp @ Preserve original SP +#endif + ldr sp, [r2, r3] @ Load SP from per-CPU var + .if \from_user == 0 +UNWIND( .setfp fpreg, sp ) + @ + @ If we took the interrupt while running in the kernel, we may already + @ be using the IRQ stack, so revert to the original value in that case. + @ + subs r2, sp, r0 @ SP above bottom of IRQ stack? + rsbscs r2, r2, #THREAD_SIZE @ ... and below the top? + movcs sp, r0 @ If so, revert to incoming SP + +#ifndef CONFIG_UNWINDER_ARM + @ + @ Inform the frame pointer unwinder where the next frame lives + @ + movcc lr, pc @ Make LR point into .entry.text so + @ that we will get a dump of the + @ exception stack for this frame. +#ifdef CONFIG_CC_IS_GCC + movcc ip, r0 @ Store the old SP in the frame record. + stmdbcc sp!, {fp, ip, lr, pc} @ Push frame record + addcc fp, sp, #12 +#else + stmdbcc sp!, {fp, lr} @ Push frame record + movcc fp, sp +#endif // CONFIG_CC_IS_GCC +#endif // CONFIG_UNWINDER_ARM + .endif +#endif // CONFIG_IRQSTACKS + bl_m [r1] + +#ifdef CONFIG_IRQSTACKS +#ifdef CONFIG_UNWINDER_ARM + mov sp, fpreg @ Restore original SP +#else + mov fp, r8 @ Restore original FP + mov sp, r9 @ Restore original SP +#endif // CONFIG_UNWINDER_ARM +#endif // CONFIG_IRQSTACKS #else arch_irq_handler_default #endif @@ -200,7 +248,7 @@ ENDPROC(__dabt_svc) .align 5 __irq_svc: svc_entry - irq_handler + irq_handler from_user=0 #ifdef CONFIG_PREEMPTION ldr r8, [tsk, #TI_PREEMPT] @ get preempt count @@ -427,7 +475,7 @@ ENDPROC(__dabt_usr) __irq_usr: usr_entry kuser_cmpxchg_check - irq_handler + irq_handler from_user=1 get_thread_info tsk mov why, #0 b ret_to_user_from_irq diff --git a/arch/arm/kernel/irq.c b/arch/arm/kernel/irq.c index 20ab1e607522..58af2adb1583 100644 --- a/arch/arm/kernel/irq.c +++ b/arch/arm/kernel/irq.c @@ -43,6 +43,25 @@ unsigned long irq_err_count; +#ifdef CONFIG_IRQSTACKS + +asmlinkage DEFINE_PER_CPU_READ_MOSTLY(u8 *, irq_stack_ptr); + +static void __init init_irq_stacks(void) +{ + u8 *stack; + int cpu; + + for_each_possible_cpu(cpu) { + stack = (u8 *)__get_free_pages(GFP_KERNEL, THREAD_SIZE_ORDER); + if (WARN_ON(!stack)) + break; + per_cpu(irq_stack_ptr, cpu) = &stack[THREAD_SIZE]; + } +} + +#endif + int arch_show_interrupts(struct seq_file *p, int prec) { #ifdef CONFIG_FIQ @@ -99,6 +118,10 @@ void __init init_IRQ(void) { int ret; +#ifdef CONFIG_IRQSTACKS + init_irq_stacks(); +#endif + if (IS_ENABLED(CONFIG_OF) && !machine_desc->init_irq) irqchip_init(); else diff --git a/arch/arm/kernel/traps.c b/arch/arm/kernel/traps.c index cda32385552b..1ba9fc2eb564 100644 --- a/arch/arm/kernel/traps.c +++ b/arch/arm/kernel/traps.c @@ -66,6 +66,19 @@ void dump_backtrace_entry(unsigned long where, unsigned long from, { unsigned long end = frame + 4 + sizeof(struct pt_regs); + if (IS_ENABLED(CONFIG_UNWINDER_FRAME_POINTER) && + IS_ENABLED(CONFIG_CC_IS_GCC) && + end > ALIGN(frame, THREAD_SIZE)) { + /* + * If we are walking past the end of the stack, it may be due + * to the fact that we are on an IRQ or overflow stack. In this + * case, we can load the address of the other stack from the + * frame record. + */ + frame = ((unsigned long *)frame)[-2] - 4; + end = frame + 4 + sizeof(struct pt_regs); + } + #ifdef CONFIG_KALLSYMS printk("%s[<%08lx>] (%ps) from [<%08lx>] (%pS)\n", loglvl, where, (void *)where, from, (void *)from); @@ -278,7 +291,7 @@ static int __die(const char *str, int err, struct pt_regs *regs) if (!user_mode(regs) || in_interrupt()) { dump_mem(KERN_EMERG, "Stack: ", regs->ARM_sp, - THREAD_SIZE + (unsigned long)task_stack_page(tsk)); + ALIGN(regs->ARM_sp, THREAD_SIZE)); dump_backtrace(regs, tsk, KERN_EMERG); dump_instr(KERN_EMERG, regs); } diff --git a/arch/arm/lib/backtrace-clang.S b/arch/arm/lib/backtrace-clang.S index 5b4bca85d06d..290c52a60fc6 100644 --- a/arch/arm/lib/backtrace-clang.S +++ b/arch/arm/lib/backtrace-clang.S @@ -197,6 +197,14 @@ finished_setup: cmp sv_fp, frame @ next frame must be mov frame, sv_fp @ above the current frame +#ifdef CONFIG_IRQSTACKS + @ + @ Kernel stacks may be discontiguous in memory. If the next + @ frame is below the previous frame, accept it as long as it + @ lives in kernel memory. + @ + cmpls sv_fp, #PAGE_OFFSET +#endif bhi for_each_frame 1006: adr r0, .Lbad diff --git a/arch/arm/lib/backtrace.S b/arch/arm/lib/backtrace.S index e8408f22d4dc..293a2716bd20 100644 --- a/arch/arm/lib/backtrace.S +++ b/arch/arm/lib/backtrace.S @@ -98,6 +98,14 @@ for_each_frame: tst frame, mask @ Check for address exceptions cmp sv_fp, frame @ next frame must be mov frame, sv_fp @ above the current frame +#ifdef CONFIG_IRQSTACKS + @ + @ Kernel stacks may be discontiguous in memory. If the next + @ frame is below the previous frame, accept it as long as it + @ lives in kernel memory. + @ + cmpls sv_fp, #PAGE_OFFSET +#endif bhi for_each_frame 1006: adr r0, .Lbad From patchwork Sun Oct 17 13:17:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 12564321 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E9FFBC433EF for ; Sun, 17 Oct 2021 13:21:46 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B02FB61261 for ; Sun, 17 Oct 2021 13:21:46 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org B02FB61261 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=bTAcFp3i/MBU5ObVa7UjUy6BS3x+QYFVlzJLsm8ebfM=; b=Olw30VviSEgOe0 9nxurwKZCqP3G+uK8K2/azg1JAHbh5G+CtSfnjcS4E/PryMO+wEBsYBeUyRg2lQceUtT/B4WdOezr jz70yCE1MNENsQWxqZ6T2M5M5oM6YrPN4ev60PIQB4/XDSaCpkRq4VkMjs1nXytlbOQhPq7e7Fs30 lqtdxbY7tWn3+e5qixzK39Jg6lq4sC2kNqB4AOUeJjMQ15E9ehO9jP6/rdnFqpE7AimVgMkZBdGkQ zkvp2BtZDeDTGypHMi9rGy+nJJ7QidT2anOxZ8/NTlgQlUpQtp+yh/Coc6CyoAaLSYMGdrFCSPcxD JiJl9NUNGZKbNtBuL4nA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mc65H-00Cbi6-KU; Sun, 17 Oct 2021 13:20:16 +0000 Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mc631-00Caed-IO for linux-arm-kernel@lists.infradead.org; Sun, 17 Oct 2021 13:17:56 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id 1F02661260; Sun, 17 Oct 2021 13:17:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1634476675; bh=LFNmcmbv8AoeZVVEZ/A7okfwBcaUiFosuYCBLoolcgs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=IN2fJxGkvcuZCXZvBoi+GFH85HlDhgUm1TzmeIwIvp8tKc7u0FfKIVZkEKpny05Jz UG9gQKi2Npa6FpMpQbcZ48Un3DuaC3q4mKdROCvqEc9BkTlZVUu5hpnZhgJeru055G oZe9keCRAZKlKLJYAaTshhBBI2qkMpOVtQXC5zHZ7hOSahPNIP251uKYfVL7ynCCRw aVuldsOvw9bvku3ym7zd8tZPQriH8PrC+pxgpVMQZCic+UsWrm7mQK7YVKSNA7ByeS oqbf0pwcKH4MVW671WO4mWxm+DL9mdz01n/2025cQM9/gmSU1ClKwDig7JFddrz4RH 5Cou79wjKXHEw== From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: Ard Biesheuvel , Russell King , Arnd Bergmann , Kees Cook , Keith Packard , Linus Walleij , Nick Desaulniers , Nicolas Pitre Subject: [PATCH v3 09/10] ARM: call_with_stack: add unwind support Date: Sun, 17 Oct 2021 15:17:22 +0200 Message-Id: <20211017131723.4034662-10-ardb@kernel.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20211017131723.4034662-1-ardb@kernel.org> References: <20211017131723.4034662-1-ardb@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=2139; h=from:subject; bh=LFNmcmbv8AoeZVVEZ/A7okfwBcaUiFosuYCBLoolcgs=; b=owEB7QES/pANAwAKAcNPIjmS2Y8kAcsmYgBhbCJgoXCFc4tegZoYCByVzjMsEuVgnkzmO1opDNAD Aes/lwyJAbMEAAEKAB0WIQT72WJ8QGnJQhU3VynDTyI5ktmPJAUCYWwiYAAKCRDDTyI5ktmPJNW/C/ 9hImfV2oAOcboNYnUn1/eFs4BnpZEL83KJ2iS1hS7JaIcWuZfwT73rzcXRLlGCbEQ7u2QU1w7z7ui3 UfmXjQ1EhE0HOe2IBimdf2cctmaBqiJMZfZASbn4ZC+LRXhmdSgeoNO1gp4kms8U98nvLHt/0Oza5D zW+IVMCys6heQHhs5HtUkbakluhsh0zZELFqlsf5H51o9PYsJmfK5AurV0zw34Rg/6B1sQNwy0WTyP cpUVuA2H1bK/p4tTibylIy2fAIKsMHk5SC6PCcfc2MjV9Mk6+Z4KWzjX6VTcDCx+hcVGMPrlHin9VB MZ/NjDdQgit0O/Hi/v4uPGL+dj6+IiJ6DuNyXPCli6tJ03yA7Nn+hqG7noyuegTTi1xZxV25tCylYl EFCKxTXpPlqj7nXReiBMVVM8j00iQX/JsRLjBF0vIlqTMT6HECHpeKvTLdFjCWuH2txiY5cRSXddD4 P953A/5+akVGyg2A93BOhDMcZ8/No4U0h3VTviluJ6+KU= X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211017_061755_679165_C627C506 X-CRM114-Status: GOOD ( 13.38 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Restructure the code and add the unwind annotations so that both the frame pointer unwinder as well as the EHABI unwind info based unwinder will be able to follow the call stack through call_with_stack(). Since GCC and Clang use different formats for the stack frame, two methods are implemented: a GCC version that pushes fp, sp, lr and pc for compatibility with the frame pointer unwinder, and a second version that works with Clang, as well as with the EHABI unwinder both in ARM and Thumb2 modes. Signed-off-by: Ard Biesheuvel Acked-by: Linus Walleij Tested-by: Keith Packard Reviewed-by: Nick Desaulniers --- arch/arm/lib/call_with_stack.S | 33 +++++++++++++++----- 1 file changed, 25 insertions(+), 8 deletions(-) diff --git a/arch/arm/lib/call_with_stack.S b/arch/arm/lib/call_with_stack.S index 28b0341ae786..0a268a6c513c 100644 --- a/arch/arm/lib/call_with_stack.S +++ b/arch/arm/lib/call_with_stack.S @@ -8,25 +8,42 @@ #include #include +#include /* * void call_with_stack(void (*fn)(void *), void *arg, void *sp) * * Change the stack to that pointed at by sp, then invoke fn(arg) with * the new stack. + * + * The sequence below follows the APCS frame convention for frame pointer + * unwinding, and implements the unwinder annotations needed by the EABI + * unwinder. */ -ENTRY(call_with_stack) - str sp, [r2, #-4]! - str lr, [r2, #-4]! +ENTRY(call_with_stack) +#if defined(CONFIG_UNWINDER_FRAME_POINTER) && defined(CONFIG_CC_IS_GCC) + mov ip, sp + push {fp, ip, lr, pc} + sub fp, ip, #4 +#else +UNWIND( .fnstart ) +UNWIND( .save {fpreg, lr} ) + push {fpreg, lr} +UNWIND( .setfp fpreg, sp ) + mov fpreg, sp +#endif mov sp, r2 mov r2, r0 mov r0, r1 - badr lr, 1f - ret r2 + bl_r r2 -1: ldr lr, [sp] - ldr sp, [sp, #4] - ret lr +#if defined(CONFIG_UNWINDER_FRAME_POINTER) && defined(CONFIG_CC_IS_GCC) + ldmdb fp, {fp, sp, pc} +#else + mov sp, fpreg + pop {fpreg, pc} +UNWIND( .fnend ) +#endif ENDPROC(call_with_stack) From patchwork Sun Oct 17 13:17:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 12564323 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 81B6EC433F5 for ; Sun, 17 Oct 2021 13:22:14 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 39F1E611C1 for ; Sun, 17 Oct 2021 13:22:14 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 39F1E611C1 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=11Bm9sEnO0V2NGs3CTz4cqRYy2w65yFaagX8O+BM1vk=; b=SIGr1aObgCr3JM ZIULpKThNB12QkRaEdWvx0gvV0w4+KCIKAwR99mcoRdsK+xFDxfrxlPZx5lyEh0wGabR95KiqqvbA IxVAc/aSUoNKoXtnGdSzdClS/xUcnQKwKTto8a4Yt6CeOiAux95dk/S6bpYn/4Tin0MvaOsnHBYay TX6fXXrZ26VfmN9PdlkDlH10hfeya2HyiXjiatGGxuBBxdYufXG3jCk5cJ7jHDIafISYPtseNV4fb McYAefNLh6Yc8XPgd+eopT1LWcC7wXkG5igqmdpSLnYgrNum0BuiO6BuLNDBjmGd4KdnbEuvRry6m 4Y/HmWLYnU7TNNO6/YBw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mc65q-00Cc2H-6M; Sun, 17 Oct 2021 13:20:50 +0000 Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mc634-00CafN-84 for linux-arm-kernel@lists.infradead.org; Sun, 17 Oct 2021 13:17:59 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id B82E861038; Sun, 17 Oct 2021 13:17:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1634476677; bh=WEnM3Q+1Lfdtyu6dum6ZekjVRce8rDM9sREPoIw6q6I=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=TBUTo8xSW+Gjr25B4dqk59Q2NkrfRaunQiL2cBSUVqDl1Kb7SEmTEF/K0fdFfLPj9 zIXKX3Q4zfcqsnoqBEolt1zIU5g5rAAt83NBRjGvMZR5CMcW8hqek8N5ALhW+wGx+T 10Zi0ajEwSrQbefNwN7as497L12BovpyScpoohn+OVTJ6N7tqRJdaxcmquOYkS0gO7 75XgoF/s6FuCtGrFKwZtWMpdAxUu4hQx1X75wK/CYPnU7UW672MxJDG2q5S546Jy1r 4unsb8mCOWI/HSXs3/yvjEgV5BOBJ+MFLbfsuRbuA2R6BNeOqlHHWbw0GzFpQ5uf1Z UxI1Z1FqiQ6GQ== From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: Ard Biesheuvel , Russell King , Arnd Bergmann , Kees Cook , Keith Packard , Linus Walleij , Nick Desaulniers , Nicolas Pitre Subject: [PATCH v3 10/10] ARM: run softirqs on the per-CPU IRQ stack Date: Sun, 17 Oct 2021 15:17:23 +0200 Message-Id: <20211017131723.4034662-11-ardb@kernel.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20211017131723.4034662-1-ardb@kernel.org> References: <20211017131723.4034662-1-ardb@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=2217; h=from:subject; bh=WEnM3Q+1Lfdtyu6dum6ZekjVRce8rDM9sREPoIw6q6I=; b=owEB7QES/pANAwAKAcNPIjmS2Y8kAcsmYgBhbCJivX1+um/2Ygzc/jqLyD0JTv6B3bpJYf8MMmQv ulantUKJAbMEAAEKAB0WIQT72WJ8QGnJQhU3VynDTyI5ktmPJAUCYWwiYgAKCRDDTyI5ktmPJDfmC/ 9IGcTZxST7PvWSqZQQ/n53zxC7k72AqhR9yMsrL6UrHbSD3pb0D0PSWZ5vJzhckcLapY9zugsd/FI3 SvKogfpMlZtUN6vTELf8hMrJBxQnT2UNBhanJKB/txJesDgjeVF9mlKgb4pDonYZlx6t19GbjhEC+x gu+Y9uJVvWHzICNsbkWESjAgOtPCtmSc3JjNt8ysZGoioT8AjhtAD6CEdDz8y8dZKwkDwviKsd6WJV soeIHM3IDAs85r3XkqY7aezVE6ggVv67yCtZ8wJVONNNCSq7dv7bS4Zl0C1P63rmOEdtbJD5RwySym z2IJH/sCiKT3oWeDsymT/TkPrUoD/HLXYA8CQKryIzA4quQ9aaECZhF1zbBeB9Vzidpb0bwKoHcsko sLRwwE/wx7IZIaGhaTqF88HhCJ6uhfEJV8rBxzhZRAI5mV9KIhCNgqDI6wQqc4YMenJRAUT5bZPb+O j4uWgx2WUbs5JN8ELqF5WORwplTRsApZhVNopUJOQ0VLs= X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211017_061758_376090_A1BC1CA7 X-CRM114-Status: GOOD ( 13.41 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Now that we have enabled IRQ stacks, any softIRQs that are handled over the back of a hard IRQ will run from the IRQ stack as well. However, any synchronous softirq processing that happens when re-enabling softIRQs from task context will still execute on that task's stack. Since any call to local_bh_enable() at any level in the task's call stack may trigger a softIRQ processing run, which could potentially cause a task stack overflow if the combined stack footprints exceed the stack's size, let's run these synchronous invocations of do_softirq() on the IRQ stack as well. Signed-off-by: Ard Biesheuvel Reviewed-by: Arnd Bergmann Acked-by: Linus Walleij Tested-by: Keith Packard --- arch/arm/Kconfig | 2 ++ arch/arm/kernel/irq.c | 14 ++++++++++++++ 2 files changed, 16 insertions(+) diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig index a8c1db0736f3..d46b243e1b26 100644 --- a/arch/arm/Kconfig +++ b/arch/arm/Kconfig @@ -1166,6 +1166,8 @@ config CURRENT_POINTER_IN_TPIDRURO config IRQSTACKS def_bool y depends on GENERIC_IRQ_MULTI_HANDLER && THREAD_INFO_IN_TASK + select HAVE_IRQ_EXIT_ON_IRQ_STACK + select HAVE_SOFTIRQ_ON_OWN_STACK config ARM_CPU_TOPOLOGY bool "Support cpu topology definition" diff --git a/arch/arm/kernel/irq.c b/arch/arm/kernel/irq.c index 58af2adb1583..bab3159b9074 100644 --- a/arch/arm/kernel/irq.c +++ b/arch/arm/kernel/irq.c @@ -36,11 +36,14 @@ #include #include #include +#include #include #include #include #include +#include "reboot.h" + unsigned long irq_err_count; #ifdef CONFIG_IRQSTACKS @@ -60,6 +63,17 @@ static void __init init_irq_stacks(void) } } +static void ____do_softirq(void *arg) +{ + __do_softirq(); +} + +void do_softirq_own_stack(void) +{ + call_with_stack(____do_softirq, NULL, + __this_cpu_read(irq_stack_ptr)); +} + #endif int arch_show_interrupts(struct seq_file *p, int prec)