From patchwork Fri Sep 24 13:28:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pingfan Liu X-Patchwork-Id: 12515511 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.8 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B399EC433F5 for ; Fri, 24 Sep 2021 13:32:06 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 88B0A61039 for ; Fri, 24 Sep 2021 13:32:06 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 88B0A61039 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=jqS5RSWe79CYsAjSCEqORGdYSIj3dIp6q9MhwIC5BxQ=; b=2rrcxrsxqirdsg LUwxk3XAgZZKsSzz7faD+fdAfFHYtj14S41BX3Ut8UesIfVSVjxhRz6ruNzts3jJoS9KXpUi/E3We tqKEMJQygXjeWa83zVhgnoacPQsseMG2hc0EXFaQdEwCvjORpBTtkRx6vLTKJ8ciClNKqsqI/I8Ee RRU+PixTUZ4AjmfXkncZs5Gx2rKLQLNhLrFMyHggwxoNGHuZMm/FhodZh01fPJpOOwCuVBO74EpHj JLP0UZJPdFV/A6dRakSMtUEDqveqvumkS2tloq2n3emmjLKzSEyg2EKOPfd5zIsTmB3Qr5NvK9Br1 Hbb6Yj15X7LL5tNXShwg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mTlHC-00EaxW-Bq; Fri, 24 Sep 2021 13:30:07 +0000 Received: from mail-pg1-x531.google.com ([2607:f8b0:4864:20::531]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mTlG8-00EacN-1s for linux-arm-kernel@lists.infradead.org; Fri, 24 Sep 2021 13:29:02 +0000 Received: by mail-pg1-x531.google.com with SMTP id 17so9868027pgp.4 for ; Fri, 24 Sep 2021 06:28:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=KM6OpafNDbs5gC7AhwUIhQlXXaIlZMI4KerHz+Wd/II=; b=W8dKLKjMyGZdP4ZKFty5vBxQDsRnRPYKTyDI3rKgGPBBIcDH+d3mwjYE5zUc2tIRBD cNN33F4x4VFcdRJz/H6V7SPtKnlFjfSND0dj5e1ia0E4QFNvkQ2BoJvwm/1P2hFLO7no C9rZjjbuxNfwojHLvsjiOzUMiE4kzb+ySCW02XPvAgVoezjyuVg02/TDJ1AX/4qOfOuD Gf6r3gwD2q5R4GJOKs5R1tYuc1wh7nWQ1YA/HZKYUUzqtGkY8uiUe9iF/b1C7HbQHADo FhMLwDtVb5gnPPV4K/jG7YhfMNJ/UPemcqBk5yd8qWbLZhKT0kOYzUgcWtcVWbAOt0rW DJog== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=KM6OpafNDbs5gC7AhwUIhQlXXaIlZMI4KerHz+Wd/II=; b=bfpAYyZ/v6tA87xnC1Ew8tK+Run7ZGr9bY5eK8IyIar1ikktdt3abWNoAdOOBbD6U3 4cqvqhpBVspWaM6T1Wo1C24W/iJTZSyhm4wgkQcuvoYMyemKgxXjGtytEFZhrMda/ZFX l8zHrOHuRjlG4ywkwYEW94PCl+3nGb8TrbzgP1rO0jdeqzsAPFdgGMfXPmb+E/isVoOb rO0vTFOpmE/5wzmu6Deu8HQ+pQYXA/SNh9SdwjPLc+ARPHDCa6jlOYELvOJdElHUKOfh X6E9yntIJlxyasGSFSu3CfhVF8EHcyM9ZFfbY3NgKcDooRsIl8TKMJY6KVuWmQ52MwWG Vofw== X-Gm-Message-State: AOAM532OugTWOBo3PDMWazw0cEvtLsk8C4rhhZzuLz0YQbeJDQ5r9DXj Khj6W6WGf80kohFeBLCToDJXK1ZF4w== X-Google-Smtp-Source: ABdhPJwMVXMC/ZxnaF+ueP2XXMs36pJoqLmeGnmtD8aWkjTW+JtZpZTjYn8ik4EF9vAznSNty9fE3g== X-Received: by 2002:a63:5902:: with SMTP id n2mr3787246pgb.305.1632490138549; Fri, 24 Sep 2021 06:28:58 -0700 (PDT) Received: from piliu.users.ipa.redhat.com ([209.132.188.80]) by smtp.gmail.com with ESMTPSA id 26sm11756573pgx.72.2021.09.24.06.28.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 24 Sep 2021 06:28:58 -0700 (PDT) From: Pingfan Liu To: linux-arm-kernel@lists.infradead.org Cc: Pingfan Liu , Catalin Marinas , Will Deacon , Mark Rutland , Marc Zyngier , Joey Gouly , Sami Tolvanen , Julien Thierry , Thomas Gleixner , Yuichi Ito , linux-kernel@vger.kernel.org Subject: [PATCHv2 1/5] arm64/entry-common: push the judgement of nmi ahead Date: Fri, 24 Sep 2021 21:28:33 +0800 Message-Id: <20210924132837.45994-2-kernelfans@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210924132837.45994-1-kernelfans@gmail.com> References: <20210924132837.45994-1-kernelfans@gmail.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210924_062900_178054_8344FBE7 X-CRM114-Status: GOOD ( 16.70 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org In enter_el1_irq_or_nmi(), it can be the case which NMI interrupts an irq, which makes the condition !interrupts_enabled(regs) fail to detect the NMI. This will cause a mistaken account for irq. Introducing two interfaces: handle_arch_nmi and interrupt_is_nmi to judge NMI at this stage. Signed-off-by: Pingfan Liu Cc: Catalin Marinas Cc: Will Deacon Cc: Mark Rutland Cc: Marc Zyngier Cc: Joey Gouly Cc: Sami Tolvanen Cc: Julien Thierry Cc: Thomas Gleixner Cc: Yuichi Ito Cc: linux-kernel@vger.kernel.org To: linux-arm-kernel@lists.infradead.org --- arch/arm64/include/asm/irq.h | 5 ++++ arch/arm64/kernel/entry-common.c | 45 ++++++++++++++++++++++---------- arch/arm64/kernel/irq.c | 29 ++++++++++++++++++++ 3 files changed, 65 insertions(+), 14 deletions(-) diff --git a/arch/arm64/include/asm/irq.h b/arch/arm64/include/asm/irq.h index fac08e18bcd5..a59b1745f458 100644 --- a/arch/arm64/include/asm/irq.h +++ b/arch/arm64/include/asm/irq.h @@ -12,6 +12,11 @@ int set_handle_irq(void (*handle_irq)(struct pt_regs *)); #define set_handle_irq set_handle_irq int set_handle_fiq(void (*handle_fiq)(struct pt_regs *)); +extern void (*handle_arch_irq)(struct pt_regs *regs); +extern void (*handle_arch_fiq)(struct pt_regs *regs); +extern void (*handle_arch_nmi)(struct pt_regs *regs); +extern bool (*interrupt_is_nmi)(void); + static inline int nr_legacy_irqs(void) { return 0; diff --git a/arch/arm64/kernel/entry-common.c b/arch/arm64/kernel/entry-common.c index 32f9796c4ffe..69a8cc082712 100644 --- a/arch/arm64/kernel/entry-common.c +++ b/arch/arm64/kernel/entry-common.c @@ -24,6 +24,7 @@ #include #include #include +#include /* * Handle IRQ/context state management when entering from kernel mode. @@ -219,17 +220,28 @@ static void noinstr arm64_exit_el1_dbg(struct pt_regs *regs) lockdep_hardirqs_on(CALLER_ADDR0); } -static void noinstr enter_el1_irq_or_nmi(struct pt_regs *regs) +static inline bool arm64_in_nmi(struct pt_regs *regs) { - if (IS_ENABLED(CONFIG_ARM64_PSEUDO_NMI) && !interrupts_enabled(regs)) + if (!interrupts_enabled(regs) || (*interrupt_is_nmi)()) + return true; + return false; +} + +/* return true if in irq, otherwise in nmi */ +static bool noinstr enter_el1_irq_or_nmi(struct pt_regs *regs) +{ + if (IS_ENABLED(CONFIG_ARM64_PSEUDO_NMI) && arm64_in_nmi(regs)) { arm64_enter_nmi(regs); - else + return false; + } else { enter_from_kernel_mode(regs); + return true; + } } -static void noinstr exit_el1_irq_or_nmi(struct pt_regs *regs) +static void noinstr exit_el1_irq_or_nmi(struct pt_regs *regs, bool in_irq) { - if (IS_ENABLED(CONFIG_ARM64_PSEUDO_NMI) && !interrupts_enabled(regs)) + if (IS_ENABLED(CONFIG_ARM64_PSEUDO_NMI) && !in_irq) arm64_exit_nmi(regs); else exit_to_kernel_mode(regs); @@ -269,9 +281,6 @@ static void do_interrupt_handler(struct pt_regs *regs, handler(regs); } -extern void (*handle_arch_irq)(struct pt_regs *); -extern void (*handle_arch_fiq)(struct pt_regs *); - static void noinstr __panic_unhandled(struct pt_regs *regs, const char *vector, unsigned int esr) { @@ -433,12 +442,20 @@ asmlinkage void noinstr el1h_64_sync_handler(struct pt_regs *regs) } static void noinstr el1_interrupt(struct pt_regs *regs, - void (*handler)(struct pt_regs *)) + void (*handler)(struct pt_regs *), + void (*nmi_handler)(struct pt_regs *)) { + bool in_irq; + void (*h)(struct pt_regs *regs); + write_sysreg(DAIF_PROCCTX_NOIRQ, daif); - enter_el1_irq_or_nmi(regs); - do_interrupt_handler(regs, handler); + in_irq = enter_el1_irq_or_nmi(regs); + if (in_irq) + h = handler; + else + h = nmi_handler; + do_interrupt_handler(regs, h); /* * Note: thread_info::preempt_count includes both thread_info::count @@ -449,17 +466,17 @@ static void noinstr el1_interrupt(struct pt_regs *regs, READ_ONCE(current_thread_info()->preempt_count) == 0) arm64_preempt_schedule_irq(); - exit_el1_irq_or_nmi(regs); + exit_el1_irq_or_nmi(regs, in_irq); } asmlinkage void noinstr el1h_64_irq_handler(struct pt_regs *regs) { - el1_interrupt(regs, handle_arch_irq); + el1_interrupt(regs, handle_arch_irq, handle_arch_nmi); } asmlinkage void noinstr el1h_64_fiq_handler(struct pt_regs *regs) { - el1_interrupt(regs, handle_arch_fiq); + el1_interrupt(regs, handle_arch_fiq, handle_arch_nmi); } asmlinkage void noinstr el1h_64_error_handler(struct pt_regs *regs) diff --git a/arch/arm64/kernel/irq.c b/arch/arm64/kernel/irq.c index bda49430c9ea..e67435eb4cba 100644 --- a/arch/arm64/kernel/irq.c +++ b/arch/arm64/kernel/irq.c @@ -81,8 +81,19 @@ static void default_handle_fiq(struct pt_regs *regs) panic("FIQ taken without a root FIQ handler\n"); } +static void default_handle_nmi(struct pt_regs *unused) +{ +} + +static bool default_nmi_discriminator(void) +{ + return false; +} + void (*handle_arch_irq)(struct pt_regs *) __ro_after_init = default_handle_irq; void (*handle_arch_fiq)(struct pt_regs *) __ro_after_init = default_handle_fiq; +void (*handle_arch_nmi)(struct pt_regs *) __ro_after_init = default_handle_nmi; +bool (*interrupt_is_nmi)(void) __ro_after_init = default_nmi_discriminator; int __init set_handle_irq(void (*handle_irq)(struct pt_regs *)) { @@ -104,6 +115,24 @@ int __init set_handle_fiq(void (*handle_fiq)(struct pt_regs *)) return 0; } +int __init set_handle_nmi(void (*handle_nmi)(struct pt_regs *)) +{ + if (handle_arch_nmi != default_handle_nmi) + return -EBUSY; + + handle_arch_nmi = handle_nmi; + return 0; +} + +int __init set_nmi_discriminator(bool (*discriminator)(void)) +{ + if (interrupt_is_nmi != default_nmi_discriminator) + return -EBUSY; + + interrupt_is_nmi = discriminator; + return 0; +} + void __init init_IRQ(void) { init_irq_stacks();