From patchwork Mon Oct 23 08:29:00 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xu Lu X-Patchwork-Id: 13432509 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 1A500C25B41 for ; Mon, 23 Oct 2023 08:29:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=oroy76SzILL1mjoKXPntLhNApgHV8DDDJSfFQRFjAso=; b=iP2E6pJQXgrYnS UCJhefTEEeqNYx2iY5lZplbWh9hTyCmX8YRfrzySShWAER9Z9MEdvX+pCnk3txNYoeuXC/lc11/As 242GtVoSvpP6yzDvctnatPp//GHe1mexLInSV9H2owCwqNAOm+0R9rlmiQYqr6AG3cbSHVKm4S9od TwOStFiINIMlVBeFNT9ndktAVdXdIWrpRsRaRvAIHPIyXSW7suxBMIzpa2hHOx62jTCBAf1oIAkwX c4Bbsd9blYM9P1ggSYNM4PhDSNlsY7LrgV/jRK0GzpIcdwN/JhuDgdG2kCKBXe5TKnzAmkvHxnEte zT09YI3tjMm4t/XNIOpw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1quqJa-006kvE-22; Mon, 23 Oct 2023 08:29:34 +0000 Received: from mail-pf1-x431.google.com ([2607:f8b0:4864:20::431]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1quqJX-006kt2-11 for linux-riscv@lists.infradead.org; Mon, 23 Oct 2023 08:29:32 +0000 Received: by mail-pf1-x431.google.com with SMTP id d2e1a72fcca58-6b89ab5ddb7so3021987b3a.0 for ; Mon, 23 Oct 2023 01:29:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1698049767; x=1698654567; darn=lists.infradead.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=yC7lpeLF6zJ2U1K9d3gKHK6RO2c15uLNrRvH89Y0oYM=; b=aTH14A3Bc+c+5Nt8Uq5fxGhO7qV4N1qJl9UJo85Jq/Wd/Wq6Z2qbLxTkJIXiQLHhYF Cowh76SyFeFkV9nqCUgZLHpHk9crD/j75eIByRCR1RtDcH8OTPp+Hz8gMHa5cVDQfwtz fgEQcZeS8JWDNlTwqA03PMkgYC0sj0+HOm213GgmPVuvnyB1EnT9uBIyl5eMFqUD54Dd TadhOGv5AszYCX5s9Ic7prTkyk2kcZKLMO9v5b76Xrk5p6g6+p/zegtAJrPdWOLO1ysY 8/2fsLgKTqEBRSAdjyMrzugtYgLY3XwioFHzX9q9PC4+jtSG1LwWsmKegF7vDEo6PK31 FL4w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1698049767; x=1698654567; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=yC7lpeLF6zJ2U1K9d3gKHK6RO2c15uLNrRvH89Y0oYM=; b=ATij3HM1kjvZ5RzWUjwna5Z+DNreEn6q4LtF8MlYIhm379VY0O6So9B4dKdNcAPXq8 4MGKRQfycpKzOPd6s2sw/pyKf8S//aPm+UaBdLq30hIORb1rxGmavSKu4OyTahKsS4bq AjnjrTvib5RZQyW1IJdDCbfIIVapkCk6FXdP/+nfdJAmtacfQDjcqkLtdTeCjiWFKhQE dfWGgZZenzt8ziZAZnxeGou33CzJKWBm/lrO8L6M6SI+TIf6pCqfsgUlUnQ2S4ZG46aP M8ntUAfqzZiMRoduuc5gT+WXt9pcc07fMj3nlUNOGzY/0yOQJRllRb+OyaDjfUAAHg6j HEXg== X-Gm-Message-State: AOJu0YxJpuvfS2pdAyykd6Cdil1ogEyXsU53+GjpjXXLlOpIPY5bS8rV CPY/7lKuy4cSphLbuXAJ8md1Eg== X-Google-Smtp-Source: AGHT+IHCH1nMZJNeGDEKv3u1kEAVWCuStY+cy0vOx97pHNcDM4nqv54AkAVkcpOIZIr2dvR9THZ/jQ== X-Received: by 2002:a05:6a20:a108:b0:14c:daa9:5e22 with SMTP id q8-20020a056a20a10800b0014cdaa95e22mr10514808pzk.45.1698049766796; Mon, 23 Oct 2023 01:29:26 -0700 (PDT) Received: from J9GPGXL7NT.bytedance.net ([203.208.167.147]) by smtp.gmail.com with ESMTPSA id d15-20020a170903230f00b001b8b07bc600sm5415805plh.186.2023.10.23.01.29.21 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Mon, 23 Oct 2023 01:29:26 -0700 (PDT) From: Xu Lu To: paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, tglx@linutronix.de, maz@kernel.org, anup@brainfault.org, atishp@atishpatra.org Cc: dengliang.1214@bytedance.com, liyu.yukiteru@bytedance.com, sunjiadong.lff@bytedance.com, xieyongji@bytedance.com, lihangjing@bytedance.com, chaiwen.cc@bytedance.com, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Xu Lu Subject: [RFC 01/12] riscv: Introduce CONFIG_RISCV_PSEUDO_NMI Date: Mon, 23 Oct 2023 16:29:00 +0800 Message-Id: <20231023082911.23242-2-luxu.kernel@bytedance.com> X-Mailer: git-send-email 2.39.3 (Apple Git-145) In-Reply-To: <20231023082911.23242-1-luxu.kernel@bytedance.com> References: <20231023082911.23242-1-luxu.kernel@bytedance.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20231023_012931_357161_899A4484 X-CRM114-Status: UNSURE ( 7.13 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org This commit introduces a new config RISCV_PSEUDO_NMI to control whether enabling the pseudo NMI feature on RISC-V. Signed-off-by: Xu Lu --- arch/riscv/Kconfig | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index d607ab0f7c6d..487e4293f31e 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -669,6 +669,16 @@ config RISCV_BOOT_SPINWAIT If unsure what to do here, say N. +config RISCV_PSEUDO_NMI + bool "Support for NMI-like interrupts" + depends on !RISCV_M_MODE + default n + help + Adds support for mimicking Non-Maskable Interrupts through the use of + CSR_IE register. + + If unsure, say N. + config ARCH_SUPPORTS_KEXEC def_bool MMU From patchwork Mon Oct 23 08:29:01 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xu Lu X-Patchwork-Id: 13432510 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id EB23CC25B41 for ; Mon, 23 Oct 2023 08:29:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=Isdg7AwgBxR1gsHypGmU3cFcid2k3HYVErS2w3JYoxA=; b=rQa3V9yUV7J2V0 BthkkKm+1YyDIUfQ0PROAr80UVgcxJhQt8mxZ6gjxsAC7DX+Nk4F/HFagpx/wryul5telBNG4gCZd qVPWqdkxpcT7sVYK0Bcc73ZSDLc0z7rVv5u9VyDEeKJlmOtTuMMNKcBbq8Z7R1PLjYtA8/7ZgKlnN J7grtBfPpWwEGBNuMIBfeG32FDXmebap7HQI2gv7Yt/xk3m0n7Q7MQi2w+5DmZ7cvyD4nyYJFB51Y EmlEaU0tWCjTwL8KK/1Tc7Y4o/LP7VG6mQx6p1cyl/3KG8jZMhDkNhibFNuauZPBBx7y4q1L9m27C zX/T5BCywaBIUJXSxJEQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1quqJe-006kwg-1F; Mon, 23 Oct 2023 08:29:38 +0000 Received: from mail-pl1-x62c.google.com ([2607:f8b0:4864:20::62c]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1quqJb-006kuq-1U for linux-riscv@lists.infradead.org; Mon, 23 Oct 2023 08:29:37 +0000 Received: by mail-pl1-x62c.google.com with SMTP id d9443c01a7336-1c9c5a1b87bso17468645ad.3 for ; Mon, 23 Oct 2023 01:29:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1698049772; x=1698654572; darn=lists.infradead.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=RLEuNYLzmNdpMekkYxMIqBfr7ettCjIz89PT5TQjtwM=; b=QVyJZMlmJU3Un8YiEIaJ72XRc8r1oOCDHrj0zTE3XOBoImKSv7nETu7IfATaoKOLP3 X6Ka2Op3PHb7ZEO3x9l/xtiuCME+UvW5YOn5WV+n2W08VO2i4LigCKLUIsMYAk/3V4xt UnzhU4othCkkf/4appHl6bmEOHyYe0tUAaJ5nunBtHpOzmm7jd92O3jFUxHS9j9s9IIa SDDVbDzzFm8GKji+x7RDtLCz9vvTNyVwcp/6hkfP9qYsR7WnIp20mAY6cCUkd9Fpz9XP 8CH9YwP/28eDl7jrOs6HXVerTQrzokuk39TbzoIztW8b+noAM9+WFLV8CCrs9z2Sctay YRKg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1698049772; x=1698654572; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=RLEuNYLzmNdpMekkYxMIqBfr7ettCjIz89PT5TQjtwM=; b=TEhOlw0bbQNp2jkofpnB7Vg68oJ6990UC6Dki95g+4qIQ7ilcx4nrvtwCJcd6K9CGB vw9tofh04lqZGAMEvmpEpB2BzIap/YQ/IrKT63VKWRGN04yB0ssnZXqkB7MJT4XBa6Em E1svxpHZVVo+mlJxrR+MZNvFlKOvz92AOARBzS6pYDAFZfoUdXwlDxX3iDPzc2ZOf7SN F/IIK/aJWzhJnA3jqnrjcRSwUvGSQvw1FApA0KIDEXmEBedDYKwSIgJt0cM21Sr+aToL jwu3dUbF0NoSQZWvwi3Ytpb0pIchM5XGNCZnKUc5gctkC/2P3aggQSuxvsnVYO0N10Dc jr9g== X-Gm-Message-State: AOJu0YwgvykYP/wI0on/UC15N4/y/dLn33ouhk3wgHGks29JJVVoFF0e 3qOdQYEIdOKFC7ISiURoehr9mA== X-Google-Smtp-Source: AGHT+IFwiuZzN1VxYpGUrbYGS2JDDA5Axr6zrj45LnHXNTd5WQzI3Ka79FvuRD+C40wTzv0jOoJZUg== X-Received: by 2002:a17:902:dad1:b0:1c5:befa:d81d with SMTP id q17-20020a170902dad100b001c5befad81dmr8117421plx.10.1698049772541; Mon, 23 Oct 2023 01:29:32 -0700 (PDT) Received: from J9GPGXL7NT.bytedance.net ([203.208.167.147]) by smtp.gmail.com with ESMTPSA id d15-20020a170903230f00b001b8b07bc600sm5415805plh.186.2023.10.23.01.29.27 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Mon, 23 Oct 2023 01:29:32 -0700 (PDT) From: Xu Lu To: paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, tglx@linutronix.de, maz@kernel.org, anup@brainfault.org, atishp@atishpatra.org Cc: dengliang.1214@bytedance.com, liyu.yukiteru@bytedance.com, sunjiadong.lff@bytedance.com, xieyongji@bytedance.com, lihangjing@bytedance.com, chaiwen.cc@bytedance.com, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Xu Lu Subject: [RFC 02/12] riscv: Make CSR_IE register part of context Date: Mon, 23 Oct 2023 16:29:01 +0800 Message-Id: <20231023082911.23242-3-luxu.kernel@bytedance.com> X-Mailer: git-send-email 2.39.3 (Apple Git-145) In-Reply-To: <20231023082911.23242-1-luxu.kernel@bytedance.com> References: <20231023082911.23242-1-luxu.kernel@bytedance.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20231023_012935_509237_05E77E8B X-CRM114-Status: GOOD ( 18.06 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org This commit makes CSR_IE register part of thread context. Kernel nowadays saves and restores irq status of each thread via CSR_STATUS register. When a thread traps into kernel, irq status of it is automatically stored in SR_PIE field of CSR_STATUS by hardware. And when kernel returns back, irq status will be automatically restored from CSR_STATUS. Things get different when we switch to CSR_IE masking for irq disabling. Hardware won't save or restore CSR_IE value during traps. In this case, when trapped into kernel, we should save CSR_IE value for previous thread manually and then clear all CSR_IE bits to disable irqs during traps. Also, we should manually restore SIE field of CSR_STATUS as we do not depends on it to disable irqs. When kernel returns back, we manually restore CSR_IE value from previous saved value. Signed-off-by: Xu Lu Signed-off-by: Hangjing Li Reviewed-by: Liang Deng Reviewed-by: Yu Li --- arch/riscv/include/asm/csr.h | 17 +++++++++++++++++ arch/riscv/include/asm/ptrace.h | 3 +++ arch/riscv/kernel/asm-offsets.c | 3 +++ arch/riscv/kernel/entry.S | 13 +++++++++++++ arch/riscv/kernel/process.c | 6 ++++++ arch/riscv/kernel/suspend_entry.S | 1 + drivers/clocksource/timer-clint.c | 4 ++++ drivers/clocksource/timer-riscv.c | 4 ++++ drivers/irqchip/irq-riscv-intc.c | 4 ++++ drivers/perf/riscv_pmu_sbi.c | 4 ++++ 10 files changed, 59 insertions(+) diff --git a/arch/riscv/include/asm/csr.h b/arch/riscv/include/asm/csr.h index 777cb8299551..6520bd826d52 100644 --- a/arch/riscv/include/asm/csr.h +++ b/arch/riscv/include/asm/csr.h @@ -7,6 +7,7 @@ #define _ASM_RISCV_CSR_H #include +#include #include /* Status register flags */ @@ -451,6 +452,22 @@ #define IE_TIE (_AC(0x1, UL) << RV_IRQ_TIMER) #define IE_EIE (_AC(0x1, UL) << RV_IRQ_EXT) +#ifdef CONFIG_RISCV_PSEUDO_NMI +#define IRQS_ENABLED_IE (IE_SIE | IE_TIE | IE_EIE) +#define irqs_enabled_ie \ +({ \ + unsigned long __v; \ + asm (ALTERNATIVE( \ + "li %0, " __stringify(IRQS_ENABLED_IE) "\n\t" \ + "nop", \ + "li %0, " __stringify(IRQS_ENABLED_IE | SIP_LCOFIP),\ + 0, RISCV_ISA_EXT_SSCOFPMF, \ + CONFIG_RISCV_PSEUDO_NMI) \ + : "=r"(__v) : : ); \ + __v; \ +}) +#endif /* CONFIG_RISCV_PSEUDO_NMI */ + #ifndef __ASSEMBLY__ #define csr_swap(csr, val) \ diff --git a/arch/riscv/include/asm/ptrace.h b/arch/riscv/include/asm/ptrace.h index b5b0adcc85c1..b57d3a6b232f 100644 --- a/arch/riscv/include/asm/ptrace.h +++ b/arch/riscv/include/asm/ptrace.h @@ -47,6 +47,9 @@ struct pt_regs { unsigned long t6; /* Supervisor/Machine CSRs */ unsigned long status; +#ifdef CONFIG_RISCV_PSEUDO_NMI + unsigned long ie; +#endif unsigned long badaddr; unsigned long cause; /* a0 value before the syscall */ diff --git a/arch/riscv/kernel/asm-offsets.c b/arch/riscv/kernel/asm-offsets.c index d6a75aac1d27..165f6f9fc458 100644 --- a/arch/riscv/kernel/asm-offsets.c +++ b/arch/riscv/kernel/asm-offsets.c @@ -112,6 +112,9 @@ void asm_offsets(void) OFFSET(PT_GP, pt_regs, gp); OFFSET(PT_ORIG_A0, pt_regs, orig_a0); OFFSET(PT_STATUS, pt_regs, status); +#ifdef CONFIG_RISCV_PSEUDO_NMI + OFFSET(PT_IE, pt_regs, ie); +#endif OFFSET(PT_BADADDR, pt_regs, badaddr); OFFSET(PT_CAUSE, pt_regs, cause); diff --git a/arch/riscv/kernel/entry.S b/arch/riscv/kernel/entry.S index 143a2bb3e697..19ba7c4520b9 100644 --- a/arch/riscv/kernel/entry.S +++ b/arch/riscv/kernel/entry.S @@ -65,6 +65,10 @@ _save_context: REG_S s3, PT_BADADDR(sp) REG_S s4, PT_CAUSE(sp) REG_S s5, PT_TP(sp) +#ifdef CONFIG_RISCV_PSEUDO_NMI + csrr s0, CSR_IE + REG_S s0, PT_IE(sp) +#endif /* CONFIG_RISCV_PSEUDO_NMI */ /* * Set the scratch register to 0, so that if a recursive exception @@ -153,6 +157,11 @@ SYM_CODE_START_NOALIGN(ret_from_exception) csrw CSR_STATUS, a0 csrw CSR_EPC, a2 +#ifdef CONFIG_RISCV_PSEUDO_NMI + REG_L s0, PT_IE(sp) + csrw CSR_IE, s0 +#endif /* CONFIG_RISCV_PSEUDO_NMI */ + REG_L x1, PT_RA(sp) REG_L x3, PT_GP(sp) REG_L x4, PT_TP(sp) @@ -251,6 +260,10 @@ restore_caller_reg: REG_S s3, PT_BADADDR(sp) REG_S s4, PT_CAUSE(sp) REG_S s5, PT_TP(sp) +#ifdef CONFIG_RISCV_PSEUDO_NMI + csrr s0, CSR_IE + REG_S s0, PT_IE(sp) +#endif /* CONFIG_RISCV_PSEUDO_NMI */ move a0, sp tail handle_bad_stack SYM_CODE_END(handle_kernel_stack_overflow) diff --git a/arch/riscv/kernel/process.c b/arch/riscv/kernel/process.c index e32d737e039f..9663bae23c57 100644 --- a/arch/riscv/kernel/process.c +++ b/arch/riscv/kernel/process.c @@ -115,6 +115,9 @@ void start_thread(struct pt_regs *regs, unsigned long pc, unsigned long sp) { regs->status = SR_PIE; +#ifdef CONFIG_RISCV_PSEUDO_NMI + regs->ie = irqs_enabled_ie; +#endif if (has_fpu()) { regs->status |= SR_FS_INITIAL; /* @@ -189,6 +192,9 @@ int copy_thread(struct task_struct *p, const struct kernel_clone_args *args) childregs->gp = gp_in_global; /* Supervisor/Machine, irqs on: */ childregs->status = SR_PP | SR_PIE; +#ifdef CONFIG_RISCV_PSEUDO_NMI + childregs->ie = irqs_enabled_ie; +#endif p->thread.s[0] = (unsigned long)args->fn; p->thread.s[1] = (unsigned long)args->fn_arg; diff --git a/arch/riscv/kernel/suspend_entry.S b/arch/riscv/kernel/suspend_entry.S index f7960c7c5f9e..6825f4836be4 100644 --- a/arch/riscv/kernel/suspend_entry.S +++ b/arch/riscv/kernel/suspend_entry.S @@ -47,6 +47,7 @@ ENTRY(__cpu_suspend_enter) REG_S t0, (SUSPEND_CONTEXT_REGS + PT_EPC)(a0) csrr t0, CSR_STATUS REG_S t0, (SUSPEND_CONTEXT_REGS + PT_STATUS)(a0) + /* There is no need to save CSR_IE as it is maintained in memory */ csrr t0, CSR_TVAL REG_S t0, (SUSPEND_CONTEXT_REGS + PT_BADADDR)(a0) csrr t0, CSR_CAUSE diff --git a/drivers/clocksource/timer-clint.c b/drivers/clocksource/timer-clint.c index 9a55e733ae99..bdc10be9d3b4 100644 --- a/drivers/clocksource/timer-clint.c +++ b/drivers/clocksource/timer-clint.c @@ -114,7 +114,9 @@ static int clint_clock_next_event(unsigned long delta, void __iomem *r = clint_timer_cmp + cpuid_to_hartid_map(smp_processor_id()); +#ifndef CONFIG_RISCV_PSEUDO_NMI csr_set(CSR_IE, IE_TIE); +#endif writeq_relaxed(clint_get_cycles64() + delta, r); return 0; } @@ -155,7 +157,9 @@ static irqreturn_t clint_timer_interrupt(int irq, void *dev_id) { struct clock_event_device *evdev = this_cpu_ptr(&clint_clock_event); +#ifndef CONFIG_RISCV_PSEUDO_NMI csr_clear(CSR_IE, IE_TIE); +#endif evdev->event_handler(evdev); return IRQ_HANDLED; diff --git a/drivers/clocksource/timer-riscv.c b/drivers/clocksource/timer-riscv.c index da3071b387eb..b730e01a7f02 100644 --- a/drivers/clocksource/timer-riscv.c +++ b/drivers/clocksource/timer-riscv.c @@ -36,7 +36,9 @@ static int riscv_clock_next_event(unsigned long delta, { u64 next_tval = get_cycles64() + delta; +#ifndef CONFIG_RISCV_PSEUDO_NMI csr_set(CSR_IE, IE_TIE); +#endif if (static_branch_likely(&riscv_sstc_available)) { #if defined(CONFIG_32BIT) csr_write(CSR_STIMECMP, next_tval & 0xFFFFFFFF); @@ -119,7 +121,9 @@ static irqreturn_t riscv_timer_interrupt(int irq, void *dev_id) { struct clock_event_device *evdev = this_cpu_ptr(&riscv_clock_event); +#ifndef CONFIG_RISCV_PSEUDO_NMI csr_clear(CSR_IE, IE_TIE); +#endif evdev->event_handler(evdev); return IRQ_HANDLED; diff --git a/drivers/irqchip/irq-riscv-intc.c b/drivers/irqchip/irq-riscv-intc.c index e8d01b14ccdd..7fad1ba37e5c 100644 --- a/drivers/irqchip/irq-riscv-intc.c +++ b/drivers/irqchip/irq-riscv-intc.c @@ -39,12 +39,16 @@ static asmlinkage void riscv_intc_irq(struct pt_regs *regs) static void riscv_intc_irq_mask(struct irq_data *d) { +#ifndef CONFIG_RISCV_PSEUDO_NMI csr_clear(CSR_IE, BIT(d->hwirq)); +#endif } static void riscv_intc_irq_unmask(struct irq_data *d) { +#ifndef CONFIG_RISCV_PSEUDO_NMI csr_set(CSR_IE, BIT(d->hwirq)); +#endif } static void riscv_intc_irq_eoi(struct irq_data *d) diff --git a/drivers/perf/riscv_pmu_sbi.c b/drivers/perf/riscv_pmu_sbi.c index 96c7f670c8f0..995b501ec721 100644 --- a/drivers/perf/riscv_pmu_sbi.c +++ b/drivers/perf/riscv_pmu_sbi.c @@ -778,7 +778,9 @@ static int pmu_sbi_starting_cpu(unsigned int cpu, struct hlist_node *node) if (riscv_pmu_use_irq) { cpu_hw_evt->irq = riscv_pmu_irq; csr_clear(CSR_IP, BIT(riscv_pmu_irq_num)); +#ifndef CONFIG_RISCV_PSEUDO_NMI csr_set(CSR_IE, BIT(riscv_pmu_irq_num)); +#endif enable_percpu_irq(riscv_pmu_irq, IRQ_TYPE_NONE); } @@ -789,7 +791,9 @@ static int pmu_sbi_dying_cpu(unsigned int cpu, struct hlist_node *node) { if (riscv_pmu_use_irq) { disable_percpu_irq(riscv_pmu_irq); +#ifndef CONFIG_RISCV_PSEUDO_NMI csr_clear(CSR_IE, BIT(riscv_pmu_irq_num)); +#endif } /* Disable all counters access for user mode now */ From patchwork Mon Oct 23 08:29:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xu Lu X-Patchwork-Id: 13432511 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 7DA3CCDB474 for ; Mon, 23 Oct 2023 08:29:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=cIJRQBmNoxCV2YIzBXNsEv7v+S9Tt/J0qgdb1Orwl54=; b=ocZXJVy/VnPZAO 632JI0JZLIYj29mEnmuOpqQ3/aal+IhzmBKq4FrdmzKgePkw7dYDgus0QBQc0cWyorcIzwccymi94 4+C4lXCBqbIxhPBjhXigXagA4wWaAFWhw2FJqarI+GuLeEtPDYQuffBKflXh18yWoYV/9MD3UQGyb QXv2v8SuFzVqWtwGRRYzOhHH5e896znn18bjW2tvSnd6sc4GT7f1gUaVopgUHKnXKcXQFNlfKGedK UEiFjt6VnF2+AurOBKzNPo5JqFFQinSVjN+7NP/K6gsnjvNNBUWOoje2ZiFbJIpqiRE+2LJuNk0yV vPLDnPpOqTQWBMA3J1Eg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1quqJj-006kz0-0M; Mon, 23 Oct 2023 08:29:43 +0000 Received: from mail-pf1-x42a.google.com ([2607:f8b0:4864:20::42a]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1quqJg-006kwu-0o for linux-riscv@lists.infradead.org; Mon, 23 Oct 2023 08:29:41 +0000 Received: by mail-pf1-x42a.google.com with SMTP id d2e1a72fcca58-6b89ab5ddb7so3022093b3a.0 for ; Mon, 23 Oct 2023 01:29:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1698049778; x=1698654578; darn=lists.infradead.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=at7wi3me+Z61wYFd7J2d3j74V70AFyRlLZ+mJARIjAE=; b=Yf+qwgTeC86msv+fzS41+u6C/NPzqYa1vGzBRdQBAEZVolHDLZuQjGQrR+WnooCMwi 5tfmulygWXlwOu+Wue/AuN+0wA8EaTBZXkBiEinM9TBTAVAyrfTfDO/qEH9/GxYOe4+K 5Q8lzzEtuPpy2Yg37qyuxYbWzkccIyOqRvmVXsXLKSH+jzpBqnqEfqCRmS7EAwKy9wiM nvjrKKfzPWDm/cx88QG3I7vS/y89cLMdB9CJFY3gQPYDp7VsQEmuywe0FBtmg65qr7nR YTNW0/nNuxww/nkLJAk6TlwoLPsmSPntQZYmU9EE2/OK9iDqSodQbc9u7jAdoGmFy9O7 5dug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1698049778; x=1698654578; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=at7wi3me+Z61wYFd7J2d3j74V70AFyRlLZ+mJARIjAE=; b=p9sosB1fkfmYxD/d63BahYoSLxUMLE5DpXeezFPJQHWxMMPPX0Ap0jsmyuXGk3ZEWZ 34vwl+Qsiz1xYLGTHI3ZD/hV+LJz9JyjkktVXfCIEJnk1NVVQxfhIYYBZD82Z2qvEaDj y79H12ZYpupxgfCv0MvNCD0749nJ4nItvdB6/YpCWmhQNm7TfPfyPW9QL7AY4laeuXmT ES+eeSheoDtFXalIjHRQ0W4si62ryA1LX4hzFPykkULk8506nmqSDC5PLvpSnvAFS4uz ZYDcqrTUOKgLwPz2UucLKxlC9iWP+Nn1SlYwS8Qc6NkHfzaPaF25tIohh9lpV5gEFu2b Mv7A== X-Gm-Message-State: AOJu0Yw22qjcIjZUfhuWJ7I4/UjZxbsczUpKauVJ4gYZXLSd/fTDsars B2kFN43Px2dHo/ukTIYyYfMQXA== X-Google-Smtp-Source: AGHT+IHAxAXPREhEH68MgYTJlOqTtsbSB0gFuNM7voWNCv0htKF7/ukJX/oxNlGp1Ca/X+qqi24MCg== X-Received: by 2002:a05:6a20:7489:b0:16b:afc2:3b68 with SMTP id p9-20020a056a20748900b0016bafc23b68mr10853205pzd.25.1698049778199; Mon, 23 Oct 2023 01:29:38 -0700 (PDT) Received: from J9GPGXL7NT.bytedance.net ([203.208.167.147]) by smtp.gmail.com with ESMTPSA id d15-20020a170903230f00b001b8b07bc600sm5415805plh.186.2023.10.23.01.29.32 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Mon, 23 Oct 2023 01:29:37 -0700 (PDT) From: Xu Lu To: paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, tglx@linutronix.de, maz@kernel.org, anup@brainfault.org, atishp@atishpatra.org Cc: dengliang.1214@bytedance.com, liyu.yukiteru@bytedance.com, sunjiadong.lff@bytedance.com, xieyongji@bytedance.com, lihangjing@bytedance.com, chaiwen.cc@bytedance.com, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Xu Lu Subject: [RFC 03/12] riscv: Switch to CSR_IE masking when disabling irqs Date: Mon, 23 Oct 2023 16:29:02 +0800 Message-Id: <20231023082911.23242-4-luxu.kernel@bytedance.com> X-Mailer: git-send-email 2.39.3 (Apple Git-145) In-Reply-To: <20231023082911.23242-1-luxu.kernel@bytedance.com> References: <20231023082911.23242-1-luxu.kernel@bytedance.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20231023_012940_291184_BC89ABE5 X-CRM114-Status: GOOD ( 10.40 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org This commit switch the way of disabling irqs to CSR_IE masking. After CSR_IE has been made a part of context, now we can safely switch to CSR_IE masking when disabling irqs. Signed-off-by: Xu Lu Signed-off-by: Hangjing Li Reviewed-by: Liang Deng Reviewed-by: Yu Li --- arch/riscv/include/asm/irqflags.h | 58 +++++++++++++++++++++++++++++++ arch/riscv/include/asm/ptrace.h | 4 +++ arch/riscv/kernel/entry.S | 7 +++- arch/riscv/kernel/head.S | 10 ++++++ 4 files changed, 78 insertions(+), 1 deletion(-) diff --git a/arch/riscv/include/asm/irqflags.h b/arch/riscv/include/asm/irqflags.h index 08d4d6a5b7e9..e0ff37315178 100644 --- a/arch/riscv/include/asm/irqflags.h +++ b/arch/riscv/include/asm/irqflags.h @@ -10,6 +10,62 @@ #include #include +#ifdef CONFIG_RISCV_PSEUDO_NMI + +static inline void local_irq_switch_on(void) +{ + csr_set(CSR_STATUS, SR_IE); +} + +static inline void local_irq_switch_off(void) +{ + csr_clear(CSR_STATUS, SR_IE); +} + +/* read interrupt enabled status */ +static inline unsigned long arch_local_save_flags(void) +{ + return csr_read(CSR_IE); +} + +/* unconditionally enable interrupts */ +static inline void arch_local_irq_enable(void) +{ + csr_set(CSR_IE, irqs_enabled_ie); +} + +/* unconditionally disable interrupts */ +static inline void arch_local_irq_disable(void) +{ + csr_clear(CSR_IE, irqs_enabled_ie); +} + +/* get status and disable interrupts */ +static inline unsigned long arch_local_irq_save(void) +{ + return csr_read_clear(CSR_IE, irqs_enabled_ie); +} + +/* test flags */ +static inline int arch_irqs_disabled_flags(unsigned long flags) +{ + return (flags != irqs_enabled_ie); +} + +/* test hardware interrupt enable bit */ +static inline int arch_irqs_disabled(void) +{ + return arch_irqs_disabled_flags(arch_local_save_flags()); +} + +/* set interrupt enabled status */ +static inline void arch_local_irq_restore(unsigned long flags) +{ + csr_write(CSR_IE, flags); +} + +#else /* CONFIG_RISCV_PSEUDO_NMI */ + /* read interrupt enabled status */ static inline unsigned long arch_local_save_flags(void) { @@ -52,4 +108,6 @@ static inline void arch_local_irq_restore(unsigned long flags) csr_set(CSR_STATUS, flags & SR_IE); } +#endif /* !CONFIG_RISCV_PSEUDO_NMI */ + #endif /* _ASM_RISCV_IRQFLAGS_H */ diff --git a/arch/riscv/include/asm/ptrace.h b/arch/riscv/include/asm/ptrace.h index b57d3a6b232f..e552e7fb46f3 100644 --- a/arch/riscv/include/asm/ptrace.h +++ b/arch/riscv/include/asm/ptrace.h @@ -178,7 +178,11 @@ static inline unsigned long regs_get_kernel_argument(struct pt_regs *regs, static inline int regs_irqs_disabled(struct pt_regs *regs) { +#ifdef CONFIG_RISCV_PSEUDO_NMI + return (regs->ie != irqs_enabled_ie); +#else return !(regs->status & SR_PIE); +#endif } #endif /* __ASSEMBLY__ */ diff --git a/arch/riscv/kernel/entry.S b/arch/riscv/kernel/entry.S index 19ba7c4520b9..d1f28dab02f7 100644 --- a/arch/riscv/kernel/entry.S +++ b/arch/riscv/kernel/entry.S @@ -66,8 +66,13 @@ _save_context: REG_S s4, PT_CAUSE(sp) REG_S s5, PT_TP(sp) #ifdef CONFIG_RISCV_PSEUDO_NMI - csrr s0, CSR_IE + csrrw s0, CSR_IE, x0 REG_S s0, PT_IE(sp) + andi s1, s1, SR_PIE + beqz s1, 1f + li s1, SR_IE + csrs CSR_STATUS, s1 +1: #endif /* CONFIG_RISCV_PSEUDO_NMI */ /* diff --git a/arch/riscv/kernel/head.S b/arch/riscv/kernel/head.S index 3710ea5d160f..4f9446defacd 100644 --- a/arch/riscv/kernel/head.S +++ b/arch/riscv/kernel/head.S @@ -169,6 +169,10 @@ secondary_start_sbi: call relocate_enable_mmu #endif call setup_trap_vector +#ifdef CONFIG_RISCV_PSEUDO_NMI + li t0, SR_IE + csrs CSR_STATUS, t0 +#endif tail smp_callin #endif /* CONFIG_SMP */ @@ -320,6 +324,12 @@ clear_bss_done: #ifdef CONFIG_KASAN call kasan_early_init #endif + +#ifdef CONFIG_RISCV_PSEUDO_NMI + li t0, SR_IE + csrs CSR_STATUS, t0 +#endif + /* Start the kernel */ call soc_early_init tail start_kernel From patchwork Mon Oct 23 08:29:03 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xu Lu X-Patchwork-Id: 13432512 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 46A14CDB474 for ; Mon, 23 Oct 2023 08:29:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=DeTqQw0EXrF7zhKlwCPeG79pEvtehafR2dPo824c12k=; b=iZjiz9Xx/CD537 51xTv60RJlXkrsPixsbB91hw5DGnb8L9qKTbLiwllQkidmRKx98ABgCCpvg9dsOJJt5/JA0JCZ6ZC IgGouUMqSh+9Lw2VzLpeXOj1yZKwF8yrB7p2s+c0X0+mfFsOp+ojV/tSSAb/+aQd4SfsWz7Kk6q6G I3ramdw21n+Eul9ywEEDf2y4RuTqHl4Ieqjare0l2YrDXz9FxrO0fpSfCmqU/IRvMWF6Kgt8Cg+AF DY/ljHzROqu45yB7AapekgnYmO91O77v6lU9SpqaJfJfQFEptGCbPG964CxtYdFPW7q2AesYLHcRQ /xBf1uJy4EK2C8CZXTYg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1quqJn-006l0t-2d; Mon, 23 Oct 2023 08:29:47 +0000 Received: from mail-pf1-x42e.google.com ([2607:f8b0:4864:20::42e]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1quqJl-006kzX-1v for linux-riscv@lists.infradead.org; Mon, 23 Oct 2023 08:29:46 +0000 Received: by mail-pf1-x42e.google.com with SMTP id d2e1a72fcca58-6b5cac99cfdso2348215b3a.2 for ; Mon, 23 Oct 2023 01:29:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1698049784; x=1698654584; darn=lists.infradead.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=bTPdJw2kGsvWjCGGMUBG2xz20Ji8Kpf17R3X8M8D7Aw=; b=d9SNTciTsZYsbbZCBH/t4VLIjxlOv+n3roQksbB7v0unjOq1aNkrIoU7wdON/bXHBK EJeEmL40lsgjuB2+EBfgk6K7saL6qHDIM4M58dXMfmVn6E/cquJEaneFMc+bBBuuc5SI 5eOBei9fpDVCd0fqC7NJ/7tN+Oe4JL8WLUfvu1WUiP+z+Fy6XQ+PbM7CBL65jdkyP27V G5WOaqmMoA2XYhoanJTDh4dJZTw5N5mzzK8xbCpsKPMXdwcE6zQuQBUP3ML3q+6cl75D MOE2yOPwCrI9EjYcMqF+7bZTUHn65I6B9O/yK3AYsMOo4TLG1LHPnALutWEW+evM9khr FP+w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1698049784; x=1698654584; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=bTPdJw2kGsvWjCGGMUBG2xz20Ji8Kpf17R3X8M8D7Aw=; b=suy/ZMsr24PNNYGP7czBHGbM2E3aQjJ8H3lK07iaGXMgIp4IYxIyEP24leXbzy584t 3yaddLc0RsEaUbcECtey3zCmoR/O8L0dsvmTLwAU6gE6PzoCzPRUtpUD4JAaWysSqa23 5gAs9DnMTtveJOLbIbHSeR0jAHLVney4s7p8c/41XZtwrzWpRpMF8ZZdHyd2e6qRC/Lg 2XgXiK/lQTqVxA9dgjTw4nmdfmYypEuD2UuJIKiPRJBgsVdtiqiWq/bAI0LWqtuoPJDD XoI/YBUZqBX02Q9jobIiDdyvJ9lGnK+BquZSnEOlSTAj1rl+DiYoJ3mpa1RsitgP25zw v/gQ== X-Gm-Message-State: AOJu0YwOIG07uVMITCDb3l5l2XJ/c0eO6kvk35C/Tzva9hDi6W1WkW3u F47LMnf6AN6F3GsWkOZ+DgudkA== X-Google-Smtp-Source: AGHT+IHy0ee6CNfNaI0e1G2cSIdQi3bfs3nWqXWt1dZTbNcZKi88wQbICOA1lqr6IbE3K2mo4LQppQ== X-Received: by 2002:a05:6a21:3398:b0:174:1466:359c with SMTP id yy24-20020a056a21339800b001741466359cmr6820626pzb.61.1698049783818; Mon, 23 Oct 2023 01:29:43 -0700 (PDT) Received: from J9GPGXL7NT.bytedance.net ([203.208.167.147]) by smtp.gmail.com with ESMTPSA id d15-20020a170903230f00b001b8b07bc600sm5415805plh.186.2023.10.23.01.29.38 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Mon, 23 Oct 2023 01:29:43 -0700 (PDT) From: Xu Lu To: paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, tglx@linutronix.de, maz@kernel.org, anup@brainfault.org, atishp@atishpatra.org Cc: dengliang.1214@bytedance.com, liyu.yukiteru@bytedance.com, sunjiadong.lff@bytedance.com, xieyongji@bytedance.com, lihangjing@bytedance.com, chaiwen.cc@bytedance.com, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Xu Lu Subject: [RFC 04/12] riscv: Switch back to CSR_STATUS masking when going idle Date: Mon, 23 Oct 2023 16:29:03 +0800 Message-Id: <20231023082911.23242-5-luxu.kernel@bytedance.com> X-Mailer: git-send-email 2.39.3 (Apple Git-145) In-Reply-To: <20231023082911.23242-1-luxu.kernel@bytedance.com> References: <20231023082911.23242-1-luxu.kernel@bytedance.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20231023_012945_638879_F61D7BA0 X-CRM114-Status: UNSURE ( 9.36 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org The WFI instruction makes current core stall until interrupt happens. In WFI's implementation, core can only be waken up from interrupt which is both pending in CSR_IP and enabled in CSR_IE. After we switch to CSR_IE masking for irq disabling, WFI instruction can never resume execution if CSR_IE is masked. This commit handles this special case. When WFI instruction is called with CSR_IE masked, we unmask CSR_IE first and disable irqs in traditional CSR_STATUS way instead. Signed-off-by: Xu Lu --- arch/riscv/include/asm/processor.h | 4 ++++ arch/riscv/kernel/irq.c | 17 +++++++++++++++++ 2 files changed, 21 insertions(+) diff --git a/arch/riscv/include/asm/processor.h b/arch/riscv/include/asm/processor.h index 3e23e1786d05..ab9b2b974979 100644 --- a/arch/riscv/include/asm/processor.h +++ b/arch/riscv/include/asm/processor.h @@ -111,10 +111,14 @@ extern void start_thread(struct pt_regs *regs, extern unsigned long __get_wchan(struct task_struct *p); +#ifndef CONFIG_RISCV_PSEUDO_NMI static inline void wait_for_interrupt(void) { __asm__ __volatile__ ("wfi"); } +#else +void wait_for_interrupt(void); +#endif struct device_node; int riscv_of_processor_hartid(struct device_node *node, unsigned long *hartid); diff --git a/arch/riscv/kernel/irq.c b/arch/riscv/kernel/irq.c index 9cc0a7669271..e7dfd68e9ca3 100644 --- a/arch/riscv/kernel/irq.c +++ b/arch/riscv/kernel/irq.c @@ -15,6 +15,23 @@ #include #include +#ifdef CONFIG_RISCV_PSEUDO_NMI + +void wait_for_interrupt(void) +{ + if (irqs_disabled()) { + local_irq_switch_off(); + local_irq_enable(); + __asm__ __volatile__ ("wfi"); + local_irq_disable(); + local_irq_switch_on(); + } else { + __asm__ __volatile__ ("wfi"); + } +} + +#endif /* CONFIG_RISCV_PSEUDO_NMI */ + static struct fwnode_handle *(*__get_intc_node)(void); void riscv_set_intc_hwnode_fn(struct fwnode_handle *(*fn)(void)) From patchwork Mon Oct 23 08:29:04 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xu Lu X-Patchwork-Id: 13432513 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 491B5C25B41 for ; Mon, 23 Oct 2023 08:30:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=/MoE5DawG/oz1HpT0S8jQR+4EIXK6eC4KV/wdNxp8ZY=; b=FRriEvLQ6DSNML 82XodiPM+YsA/NE4TwJAvgS8sf00r+RJMyxyIZIUzo0gx5mDmVzNyRLOniEhGuXiIwNh+po1sM8Ja jT6kp1aEO00Pa7eXjGAz9rult/aSuuIPX9DOOkmMX5jwV8krf2aBpqN9TWHgxAlXxVnAbPdBVKZj9 f8VnUTYrfxVYrbW5+ZeiZwlJkAt0g6JqpQytNWIN419zZbiiOnJ9QD+KFjQKWVlt7HtCHxFQtfRss K+Pq6V4XUV7tjqVggRA6K9AwuEzEprUdQFyz4SFOtpY+xHtu4QPmjOUqbWpdficgazLdyUOu5ChWV EyIW8k949kKEgqGQQHvA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1quqJv-006l4i-1z; Mon, 23 Oct 2023 08:29:55 +0000 Received: from mail-pl1-x62e.google.com ([2607:f8b0:4864:20::62e]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1quqJs-006l1t-1F for linux-riscv@lists.infradead.org; Mon, 23 Oct 2023 08:29:53 +0000 Received: by mail-pl1-x62e.google.com with SMTP id d9443c01a7336-1bdf4752c3cso15992655ad.2 for ; Mon, 23 Oct 2023 01:29:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1698049789; x=1698654589; darn=lists.infradead.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=4WLOSQOA5Tm0Adw6Ut8ujQkn4XXYrNjbdXHKCGaB3LM=; b=MzCZPuLXMvd+nGO85zng0yKUML8RkiunMimT2sA0QZGYs3LL2SMlR73FZm3MAT+Cb4 yY67oXLZoWnc4hVf5wCmmuDqhqBh+nKowl+jZnTx2cUZrlqL7DqPvPKEc5J9S+8wPftD J59dZRxsaRX/MvVb8kG/AepMuftaWRxui6+GfsUdp4gJ6HrQfa9aJTFfKVGsX+I1UYwt 1f6cz3FlcWo6gu+4rocIa9qFdHXxrZbNWnu+CM7ShH6Y38JcYvmLfva7deECp0o/y64z txs55bIA+ni0AKVzAfqiDdj5HEBqH/eKBSk/yUxIPiRlHSH0qHcfIO/DfKwQH+37inKW rvOA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1698049789; x=1698654589; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=4WLOSQOA5Tm0Adw6Ut8ujQkn4XXYrNjbdXHKCGaB3LM=; b=V53zWPrgxVperzdYvroWe8rIbyJK6ljB/zuhCkYCmRF+XcB+BPULO00btPcsW8/+0t IliJV1sRNkYlMrIo1/N6kX07Ry6GessqJ+j9eGKAtSJWGhe+88AH1nrYlOho6Ge3YuiP 0AQ4yn8MVY2CNiQmp+QjlyB3idB94QmXKG5UAfCZhpN8jyRkHQHT+8nEEpDwPL2dJJcg B+U+DjHExQdyPduoNVDmyVl726QEA4gjOqUTeEzXixiwlo9CZXGnINz1lJYzjOuQgUQq 5C/31D1pekj1Pi86jdUh6my9sBMDk+dIkJA8CBkjV7daTNWeRGeDIzOTlNDyVV1ZzO3K La8Q== X-Gm-Message-State: AOJu0Yws4R4Mm3MjZYVRzbuiaBsBTlPcRqc0GWiHf2MOsLFS/Qc//tW9 HHkb5H8efhZ9eiJEVv3mW8ozUA== X-Google-Smtp-Source: AGHT+IHnm6GiYKkee1BBGlEg9/2GVy9OXofYG3BGHLaj+6z7TyuT5to3Ev1GjwdsT/XgsL2yDJJgGg== X-Received: by 2002:a17:902:e548:b0:1c9:b187:4d84 with SMTP id n8-20020a170902e54800b001c9b1874d84mr8674687plf.14.1698049789506; Mon, 23 Oct 2023 01:29:49 -0700 (PDT) Received: from J9GPGXL7NT.bytedance.net ([203.208.167.147]) by smtp.gmail.com with ESMTPSA id d15-20020a170903230f00b001b8b07bc600sm5415805plh.186.2023.10.23.01.29.44 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Mon, 23 Oct 2023 01:29:49 -0700 (PDT) From: Xu Lu To: paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, tglx@linutronix.de, maz@kernel.org, anup@brainfault.org, atishp@atishpatra.org Cc: dengliang.1214@bytedance.com, liyu.yukiteru@bytedance.com, sunjiadong.lff@bytedance.com, xieyongji@bytedance.com, lihangjing@bytedance.com, chaiwen.cc@bytedance.com, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Xu Lu Subject: [RFC 05/12] riscv: kvm: Switch back to CSR_STATUS masking when entering guest Date: Mon, 23 Oct 2023 16:29:04 +0800 Message-Id: <20231023082911.23242-6-luxu.kernel@bytedance.com> X-Mailer: git-send-email 2.39.3 (Apple Git-145) In-Reply-To: <20231023082911.23242-1-luxu.kernel@bytedance.com> References: <20231023082911.23242-1-luxu.kernel@bytedance.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20231023_012952_424950_382D4C35 X-CRM114-Status: GOOD ( 10.27 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org When kvm enters vcpu, it first disables local irqs before preparing vcpu context and uses SRET instruction to enter guest mode after vcpu context is ready, which automatically restores guest's irq status. However, after we switch to CSR_IE masking for interrupt disabling, the SRET instruction itself can not restore guest's irq status correctly as interrupts are still masked by CSR_IE. This commit handles this special case by switching to traditional CSR_STATUS way to disable irqs before entering guest mode. Signed-off-by: Xu Lu --- arch/riscv/include/asm/irqflags.h | 3 +++ arch/riscv/kvm/vcpu.c | 18 +++++++++++++----- 2 files changed, 16 insertions(+), 5 deletions(-) diff --git a/arch/riscv/include/asm/irqflags.h b/arch/riscv/include/asm/irqflags.h index e0ff37315178..60c19f8b57f0 100644 --- a/arch/riscv/include/asm/irqflags.h +++ b/arch/riscv/include/asm/irqflags.h @@ -64,6 +64,9 @@ static inline void arch_local_irq_restore(unsigned long flags) csr_write(CSR_IE, flags); } +#define local_irq_enable_vcpu_run local_irq_switch_on +#define local_irq_disable_vcpu_run local_irq_switch_off + #else /* CONFIG_RISCV_PSEUDO_NMI */ /* read interrupt enabled status */ diff --git a/arch/riscv/kvm/vcpu.c b/arch/riscv/kvm/vcpu.c index 82229db1ce73..233408247da7 100644 --- a/arch/riscv/kvm/vcpu.c +++ b/arch/riscv/kvm/vcpu.c @@ -621,6 +621,14 @@ static void noinstr kvm_riscv_vcpu_enter_exit(struct kvm_vcpu *vcpu) guest_state_exit_irqoff(); } +#ifndef local_irq_enable_vcpu_run +#define local_irq_enable_vcpu_run local_irq_enable +#endif + +#ifndef local_irq_disable_vcpu_run +#define local_irq_disable_vcpu_run local_irq_disable +#endif + int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu) { int ret; @@ -685,7 +693,7 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu) continue; } - local_irq_disable(); + local_irq_disable_vcpu_run(); /* * Ensure we set mode to IN_GUEST_MODE after we disable @@ -712,7 +720,7 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu) kvm_request_pending(vcpu) || xfer_to_guest_mode_work_pending()) { vcpu->mode = OUTSIDE_GUEST_MODE; - local_irq_enable(); + local_irq_enable_vcpu_run(); preempt_enable(); kvm_vcpu_srcu_read_lock(vcpu); continue; @@ -757,12 +765,12 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu) * recognised, so we just hope that the CPU takes any pending * interrupts between the enable and disable. */ - local_irq_enable(); - local_irq_disable(); + local_irq_enable_vcpu_run(); + local_irq_disable_vcpu_run(); guest_timing_exit_irqoff(); - local_irq_enable(); + local_irq_enable_vcpu_run(); preempt_enable(); From patchwork Mon Oct 23 08:29:05 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xu Lu X-Patchwork-Id: 13432514 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D2D23C001E0 for ; Mon, 23 Oct 2023 08:30:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=F/YNTRnHw0yOQaL0ss/iiCJGRfZCwUYRWDFd0ewvW7g=; b=Rf13uCR1i//nk7 Zijj/lG2Dd52fW97dmBj9Gu9Q/EP8r6zb3tFMcvPasMGSiVg3cEfoCPAdzh9xV2ObDKBKDCWPcaUo L4n9TjxUW/7ywHG4Ot5VDydmlRo8b5svyLQA7qwbCNGSp1bpqTIsWgOLPdDcGNXOiD+8yvweuK8hR aTrbFGTHthh3W2guVv0o+hglYBVhgRcgisVov85IrGDGEbD85xU3HTx8I3/BvCHgRMJfp3iGXqzlu duVrTYRpF8Nk9MG38XLKAzkoa3Nu2qJf9H03Sy7RmAJqDtyI/RpzG34jAnxVeRqVgH1y3mVwm56H0 TrtZHr1dPL1IDpzTsQtQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1quqJy-006l6O-1E; Mon, 23 Oct 2023 08:29:58 +0000 Received: from mail-pf1-x42a.google.com ([2607:f8b0:4864:20::42a]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1quqJv-006l4e-20 for linux-riscv@lists.infradead.org; Mon, 23 Oct 2023 08:29:57 +0000 Received: by mail-pf1-x42a.google.com with SMTP id d2e1a72fcca58-68fb85afef4so2396866b3a.1 for ; Mon, 23 Oct 2023 01:29:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1698049795; x=1698654595; darn=lists.infradead.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=AUO1/BVdvM4H5C6LBNItdP0IIJipRGNxmDV7eD7nopg=; b=YOEkYWaJX4J6nQH/j/ix0gyM0HNFxdvdxB3PPUu483b2ogm/0WrxAWTpoLuY0mY6ui yyXAz1xLRTNuALQfkeQDpj51w2dtws8vQOT5S4Tx+xSYqoOhnjGpiBWUh/Cwic7QIbyr HS07uV2uJZsMW6r8TfD204CKZRsHjkbCxVN34Gr2fEasxw7I7QSDKklXjq0ixfaXRsO6 HnT2N6CYAeQsjDWeZM9tNdMTrrVMpXQHdE0YY+U9GKKsTDsoq1XlWlb7LfrriTLoQhLg ow8WuC4PGQDtwgTkWf3iEeQwVqtEMBBh2cVEcQ+CGr/svbCWMg53saZIiJqIlyodW21C XIpw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1698049795; x=1698654595; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=AUO1/BVdvM4H5C6LBNItdP0IIJipRGNxmDV7eD7nopg=; b=X9Pq3tG8tICGhFOKBkKEnvMnzMIFZ5VsNDNyu/duvrsAzigzp/+JM2G7wlcEzCoDRn ZvyHU8uBuJNebN6sV06CA+r1cUVT4yQGsEULCkFYcrLbvQ7QFSslwDAIqg6WDlrGi2qo +MMFAvrmqNAzUwJteZF/tJjCNg3RLPGGQKNnH9T4FzfmCPwQOKk/aoBJSyPnnovq5s7z 092YRCRp9+q/IeY16rW4kbjHPAtzCM4CfvyzGfwevNkslZaSQYTxdDtd4oZZVqq8X6iC /M4pvvm0i3Ywq/lcCMKX9a08p7xOMZh0aSis2N90WC/nhk6Cnkiecr/9bYZzCRWtEQWF if9Q== X-Gm-Message-State: AOJu0YxaPapKIR5YIPWemdQk1oUNjWGOxYoUt6iVgme+AMxBl8/3O8WR cgJnAU0UPYRJFIlUfTB7KFSywA== X-Google-Smtp-Source: AGHT+IEuewyI9+tw5QJUm2oN6guHecDPjWsmm8bO+1FZNslxYqHTr+PcfG3+59e4kYZAI8n0lGmkLw== X-Received: by 2002:a05:6a20:2615:b0:151:7d4c:899c with SMTP id i21-20020a056a20261500b001517d4c899cmr4935811pze.25.1698049795184; Mon, 23 Oct 2023 01:29:55 -0700 (PDT) Received: from J9GPGXL7NT.bytedance.net ([203.208.167.147]) by smtp.gmail.com with ESMTPSA id d15-20020a170903230f00b001b8b07bc600sm5415805plh.186.2023.10.23.01.29.49 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Mon, 23 Oct 2023 01:29:54 -0700 (PDT) From: Xu Lu To: paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, tglx@linutronix.de, maz@kernel.org, anup@brainfault.org, atishp@atishpatra.org Cc: dengliang.1214@bytedance.com, liyu.yukiteru@bytedance.com, sunjiadong.lff@bytedance.com, xieyongji@bytedance.com, lihangjing@bytedance.com, chaiwen.cc@bytedance.com, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Xu Lu Subject: [RFC 06/12] riscv: Allow requesting irq as pseudo NMI Date: Mon, 23 Oct 2023 16:29:05 +0800 Message-Id: <20231023082911.23242-7-luxu.kernel@bytedance.com> X-Mailer: git-send-email 2.39.3 (Apple Git-145) In-Reply-To: <20231023082911.23242-1-luxu.kernel@bytedance.com> References: <20231023082911.23242-1-luxu.kernel@bytedance.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20231023_012955_666345_2A554591 X-CRM114-Status: GOOD ( 10.81 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org This commit implements pseudo NMI callbacks for riscv_intc_irq chip. We use an immediate macro to denote NMIs of each cpu. Each bit of it represents an irq. Bit 1 means corresponding irq is registered as NMI while bit 0 means not. Signed-off-by: Xu Lu Signed-off-by: Hangjing Li Reviewed-by: Liang Deng Reviewed-by: Yu Li --- arch/riscv/include/asm/irqflags.h | 17 ++++++++++++++ drivers/irqchip/irq-riscv-intc.c | 38 +++++++++++++++++++++++++++++++ 2 files changed, 55 insertions(+) diff --git a/arch/riscv/include/asm/irqflags.h b/arch/riscv/include/asm/irqflags.h index 60c19f8b57f0..9700a17a003a 100644 --- a/arch/riscv/include/asm/irqflags.h +++ b/arch/riscv/include/asm/irqflags.h @@ -12,6 +12,23 @@ #ifdef CONFIG_RISCV_PSEUDO_NMI +#define __ALLOWED_NMI_MASK 0 +#define ALLOWED_NMI_MASK (__ALLOWED_NMI_MASK & irqs_enabled_ie) + +static inline bool nmi_allowed(int irq) +{ + return (BIT(irq) & ALLOWED_NMI_MASK); +} + +static inline bool is_nmi(int irq) +{ + return (BIT(irq) & ALLOWED_NMI_MASK); +} + +static inline void set_nmi(int irq) {} + +static inline void unset_nmi(int irq) {} + static inline void local_irq_switch_on(void) { csr_set(CSR_STATUS, SR_IE); diff --git a/drivers/irqchip/irq-riscv-intc.c b/drivers/irqchip/irq-riscv-intc.c index 7fad1ba37e5c..83a0a744fce6 100644 --- a/drivers/irqchip/irq-riscv-intc.c +++ b/drivers/irqchip/irq-riscv-intc.c @@ -67,11 +67,49 @@ static void riscv_intc_irq_eoi(struct irq_data *d) */ } +#ifdef CONFIG_RISCV_PSEUDO_NMI + +static int riscv_intc_irq_nmi_setup(struct irq_data *d) +{ + unsigned int hwirq = d->hwirq; + struct irq_desc *desc = irq_to_desc(d->irq); + + if (WARN_ON((hwirq >= BITS_PER_LONG) || !nmi_allowed(hwirq))) + return -EINVAL; + + desc->handle_irq = handle_percpu_devid_fasteoi_nmi; + set_nmi(hwirq); + + return 0; +} + +static void riscv_intc_irq_nmi_teardown(struct irq_data *d) +{ + unsigned int hwirq = d->hwirq; + struct irq_desc *desc = irq_to_desc(d->irq); + + if (WARN_ON(hwirq >= BITS_PER_LONG)) + return; + + if (WARN_ON(!is_nmi(hwirq))) + return; + + desc->handle_irq = handle_percpu_devid_irq; + unset_nmi(hwirq); +} + +#endif /* CONFIG_RISCV_PSEUDO_NMI */ + static struct irq_chip riscv_intc_chip = { .name = "RISC-V INTC", .irq_mask = riscv_intc_irq_mask, .irq_unmask = riscv_intc_irq_unmask, .irq_eoi = riscv_intc_irq_eoi, +#ifdef CONFIG_RISCV_PSEUDO_NMI + .irq_nmi_setup = riscv_intc_irq_nmi_setup, + .irq_nmi_teardown = riscv_intc_irq_nmi_teardown, + .flags = IRQCHIP_SUPPORTS_NMI, +#endif }; static int riscv_intc_domain_map(struct irq_domain *d, unsigned int irq, From patchwork Mon Oct 23 08:29:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xu Lu X-Patchwork-Id: 13432515 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id F1AA3C001E0 for ; Mon, 23 Oct 2023 08:30:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=F9DfPsIlwDCxOl76QqHN+lA/yhsMlVPrKXbIldSb5Ws=; b=Sjj7Pza3C8CX5F puL2SzJD9BHtfGk+Jch296giA50GXdbCkev0dpSnHhsbK/an6f4ZU6ATMmzv4I1ISTtzpyU02nEVJ ESYmY85ZTjKm7183BLTbbL2wbHUZjHwzsgPVP5h3EP2eglTiYTCX3ZQWaeeSxbBRdz9gPfZJZ8CfK xGvWFydMK7tC1ths1TKT7sGstJyGQqRfEAD7hKn1s8B8yXz0s/ofAr2Lxk+rshSEfZGzarM6nDoKx A3QnzRMj8U0IdslS43ezTrv8k0VEKYI6fKV3BLkpvn8d/g5cn+oytYHZHeiAVDYMYtE5FnW1QDTTO le0ZGGi6NlITq8hHLNRA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1quqK7-006lAq-0f; Mon, 23 Oct 2023 08:30:07 +0000 Received: from mail-pl1-x629.google.com ([2607:f8b0:4864:20::629]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1quqK4-006l89-1P for linux-riscv@lists.infradead.org; Mon, 23 Oct 2023 08:30:05 +0000 Received: by mail-pl1-x629.google.com with SMTP id d9443c01a7336-1c87a85332bso25201535ad.2 for ; Mon, 23 Oct 2023 01:30:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1698049801; x=1698654601; darn=lists.infradead.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=KDj8p8VlOUWmt6ePZtJj52Ju7p2Y1636xaXynBRmH4M=; b=ODJdGxmReoD7hWYIRmt0lJdTnjfOkyC2a32HqyIcehNhtpHKnaDr+dt0r9YiHTvAiA JBVu7ujaycbLs28tmD0edu72bODe26eefvVViQXdw9LIJkJgLp/G8hDUmA1EZaUrt6Hi 8uHFq42WiUBCIQWIY2n+iiduun7FHBG5LCcd3F1nEU33nqtiARnmiFVg7KFBhjW5rasJ 7Cp/jJtTWSncGo2rrSB1i0JlZJ4Vzw6V3DgtSNejGQz6sIwN5mlb8MSIRJsEJopFtuKv 77I8TWxmLZk9Gq3wnivb+8fWwjYoVSczp+pdA2KNJ+7Yh9MFR8ybBulLKIVv2l7d4H7E R8oA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1698049801; x=1698654601; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=KDj8p8VlOUWmt6ePZtJj52Ju7p2Y1636xaXynBRmH4M=; b=RPcomi1STB2mgBDC/16zAcF8CKc6GU3T6m1tjQqo26m8kPoNAzFLJdZ5/7TPSa3iVa epGvsoGa0nXQva4yO2WEtrU9gU1WW/mN2ehVNw4u3aRayFCcSPOsq5tTz8Mka0XBVf9q PrgVeb9TCXWZgUUpYsyMdLa38i5s2L3v9/QyuctVDRmKvG7v3FmYE6DQEDyi6d/HbrrT b8wak+OfkYBAkV69fdFTOSeim+4u2nyIosIlB6hUuy4JYvTn4fQRKyBTKGEug1U1EsQY VsuizDx5tQAeVVAFPKqo3PutKMInygtAhFxqj4c/VWEjoDoQQITif9LuvqwNmdkij6QR IGFQ== X-Gm-Message-State: AOJu0Yx7e78eqrGtGvZAE3K3i47+pzNtrQLa7rdta4vpGhdVLr9baZ6j aQpHgJWi0OfLCCbnFifyGcIy5g== X-Google-Smtp-Source: AGHT+IHcCdTfYDcXkZ0Qt5csB3IUm6TeiHVMhTQKrBuA3FQKpUEaJPQvyodQ8j5twuKfm3ICNLk/lg== X-Received: by 2002:a17:902:c602:b0:1c8:7822:e5c3 with SMTP id r2-20020a170902c60200b001c87822e5c3mr7531739plr.47.1698049800869; Mon, 23 Oct 2023 01:30:00 -0700 (PDT) Received: from J9GPGXL7NT.bytedance.net ([203.208.167.147]) by smtp.gmail.com with ESMTPSA id d15-20020a170903230f00b001b8b07bc600sm5415805plh.186.2023.10.23.01.29.55 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Mon, 23 Oct 2023 01:30:00 -0700 (PDT) From: Xu Lu To: paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, tglx@linutronix.de, maz@kernel.org, anup@brainfault.org, atishp@atishpatra.org Cc: dengliang.1214@bytedance.com, liyu.yukiteru@bytedance.com, sunjiadong.lff@bytedance.com, xieyongji@bytedance.com, lihangjing@bytedance.com, chaiwen.cc@bytedance.com, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Xu Lu Subject: [RFC 07/12] riscv: Handle pseudo NMI in arch irq handler Date: Mon, 23 Oct 2023 16:29:06 +0800 Message-Id: <20231023082911.23242-8-luxu.kernel@bytedance.com> X-Mailer: git-send-email 2.39.3 (Apple Git-145) In-Reply-To: <20231023082911.23242-1-luxu.kernel@bytedance.com> References: <20231023082911.23242-1-luxu.kernel@bytedance.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20231023_013004_480208_3FE01922 X-CRM114-Status: UNSURE ( 8.91 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org This commit handles pseudo NMI in arch irq handler. We enter NMI context before handling NMI and keeps all interrupts disabled during NMI handling to avoid interrupt nesting. Signed-off-by: Xu Lu Signed-off-by: Hangjing Li --- drivers/irqchip/irq-riscv-intc.c | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/drivers/irqchip/irq-riscv-intc.c b/drivers/irqchip/irq-riscv-intc.c index 83a0a744fce6..c672c0c64d5d 100644 --- a/drivers/irqchip/irq-riscv-intc.c +++ b/drivers/irqchip/irq-riscv-intc.c @@ -20,6 +20,26 @@ static struct irq_domain *intc_domain; +#ifdef CONFIG_RISCV_PSEUDO_NMI + +static asmlinkage void riscv_intc_irq(struct pt_regs *regs) +{ + unsigned long cause = regs->cause & ~CAUSE_IRQ_FLAG; + + if (unlikely(cause >= BITS_PER_LONG)) + panic("unexpected interrupt cause"); + + if (is_nmi(cause)) { + nmi_enter(); + generic_handle_domain_nmi(intc_domain, cause); + nmi_exit(); + } else { + generic_handle_domain_irq(intc_domain, cause); + } +} + +#else /* CONFIG_RISCV_PSEUDO_NMI */ + static asmlinkage void riscv_intc_irq(struct pt_regs *regs) { unsigned long cause = regs->cause & ~CAUSE_IRQ_FLAG; @@ -30,6 +50,8 @@ static asmlinkage void riscv_intc_irq(struct pt_regs *regs) generic_handle_domain_irq(intc_domain, cause); } +#endif /* CONFIG_RISCV_PSEUDO_NMI */ + /* * On RISC-V systems local interrupts are masked or unmasked by writing * the SIE (Supervisor Interrupt Enable) CSR. As CSRs can only be written From patchwork Mon Oct 23 08:29:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xu Lu X-Patchwork-Id: 13432516 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 47AA5C25B42 for ; Mon, 23 Oct 2023 08:30:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=2lHLv2X50W9820+wOaqWCiF5aFI91WthJUa7cSvA6SI=; b=yLqX/xjeBsRA6p MZlQW8bUPy9RAFeUaeWXNpiaUkfKWK4mWZsS6pXDTUWFQhHCigiP5DX9KLfVwPLKjH7WwVCG7a/gj zJvJSjjDx27TQDvoWp9pTh3kqPSlS4d5Om7BPCbKYnBL58SzbRzMsmUcjPhV9IbATDIIiL7xqV8We yRiOdXQJaoLRtdUlwPjIs84myuZr4kpHjnsyTp4yn91UKnC1s+O5a3RiGsL/k6D9jf+U3CqzmY+Rs fBMU9gHzR8XCS1ifYt1iWFN43HAKSlBsFZJ/qtwNEcEgZ9RTQ5cXJHRqbZu9rotP3PoMaOBHwjen4 0cdhGkm63wnmG+tNWoEQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1quqK9-006lCH-31; Mon, 23 Oct 2023 08:30:09 +0000 Received: from mail-pl1-x62c.google.com ([2607:f8b0:4864:20::62c]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1quqK7-006lAO-0C for linux-riscv@lists.infradead.org; Mon, 23 Oct 2023 08:30:08 +0000 Received: by mail-pl1-x62c.google.com with SMTP id d9443c01a7336-1ca6809fb8aso18195255ad.1 for ; Mon, 23 Oct 2023 01:30:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1698049806; x=1698654606; darn=lists.infradead.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=tFjJUDD1OgrURVFCNfyqhpsKd5+0kFPMkXoyM2g55XE=; b=Cd0OUPbSSlrJ/ShnnfPDf9dyvEkC77LPwZ9oeqWQFf7LLk7DpIMz2+Q54JMUka9SDz b3j4PYpcO/Oy4E0jlSSG8T2e0EC80CZyMnor8HhJrJwIoEzp9YBGI29tnmCq4e7cQMSc PFuDX96sDlUJy13KPLfPwOvXSCmEvB4mAMEYoLmxIUjbYDrwsLUHlithh9tmOyxARW0G YoQNOV70RSi4dwlrOC02MWO25rHJHkOKtTlBunEgKgMF1rQeJWuMmkOVHKiI4q8MO3uv tzoin1Z7dN/NSUdDWCkcCWYh9u4D14YFCETULpyuo6WecgKNP3PccLyncAorbb7geu/l 4iuQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1698049806; x=1698654606; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=tFjJUDD1OgrURVFCNfyqhpsKd5+0kFPMkXoyM2g55XE=; b=arejs7y9P+emU8UUYwEjNQcAPP38hlDLx67KRpxa8ErKBqxeFKjGTaELTc7nACVZ56 9vE8pu34rWkDcwJiRCu3HZLZQnBodgkhZbkN0uHLpY1LN7OStU0Rs5wZ0BqRPpwXHP8z L4hEkL4OHk1mNmM8mRJIN6to9hyp5EnCWrx8bjLbAKw7zUFyk2FrhQacbrG1h8Or57Hd 23iUQMRFIZjZjgxjDkcmQBk51mSJxpv1RPt5MuOes8Knf5Q+kIzxahs7U7DdZFcDpm2w YTWYOy3mlUHX6VZk9IDflHCeMyw2MhRck5eOACOP5NOQXJ6ghb6+WHJQeDh2552MMw4/ Zmgw== X-Gm-Message-State: AOJu0YwZkPNjoA+T4G2qOC2tchAt3jxk0TpYfFAOtQoz018gKFo4xCRD a2VjkZDeDszcoJdbAxIGDRkDDw== X-Google-Smtp-Source: AGHT+IHxUbIFNlkoPGxo4yFGNIhvxArkkFh19bAzzG/qz3kZJnf4Ua56mvZKJnXAyEFXPQjbUdh3rQ== X-Received: by 2002:a17:902:ecc6:b0:1c9:cc41:76e4 with SMTP id a6-20020a170902ecc600b001c9cc4176e4mr5988168plh.10.1698049806521; Mon, 23 Oct 2023 01:30:06 -0700 (PDT) Received: from J9GPGXL7NT.bytedance.net ([203.208.167.147]) by smtp.gmail.com with ESMTPSA id d15-20020a170903230f00b001b8b07bc600sm5415805plh.186.2023.10.23.01.30.01 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Mon, 23 Oct 2023 01:30:06 -0700 (PDT) From: Xu Lu To: paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, tglx@linutronix.de, maz@kernel.org, anup@brainfault.org, atishp@atishpatra.org Cc: dengliang.1214@bytedance.com, liyu.yukiteru@bytedance.com, sunjiadong.lff@bytedance.com, xieyongji@bytedance.com, lihangjing@bytedance.com, chaiwen.cc@bytedance.com, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Xu Lu Subject: [RFC 08/12] riscv: Enable NMIs during irqs disabled context Date: Mon, 23 Oct 2023 16:29:07 +0800 Message-Id: <20231023082911.23242-9-luxu.kernel@bytedance.com> X-Mailer: git-send-email 2.39.3 (Apple Git-145) In-Reply-To: <20231023082911.23242-1-luxu.kernel@bytedance.com> References: <20231023082911.23242-1-luxu.kernel@bytedance.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20231023_013007_099700_495029ED X-CRM114-Status: UNSURE ( 9.20 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org This commit allows NMIs to happen even when irqs are disabled. When disabling irqs, we mask all normal irqs via clearing corresponding bits in CSR_IE while leaving NMI bits alone. Signed-off-by: Xu Lu --- arch/riscv/include/asm/irqflags.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/arch/riscv/include/asm/irqflags.h b/arch/riscv/include/asm/irqflags.h index 9700a17a003a..42f7803582df 100644 --- a/arch/riscv/include/asm/irqflags.h +++ b/arch/riscv/include/asm/irqflags.h @@ -54,13 +54,13 @@ static inline void arch_local_irq_enable(void) /* unconditionally disable interrupts */ static inline void arch_local_irq_disable(void) { - csr_clear(CSR_IE, irqs_enabled_ie); + csr_clear(CSR_IE, ~ALLOWED_NMI_MASK); } /* get status and disable interrupts */ static inline unsigned long arch_local_irq_save(void) { - return csr_read_clear(CSR_IE, irqs_enabled_ie); + return csr_read_clear(CSR_IE, ~ALLOWED_NMI_MASK); } /* test flags */ From patchwork Mon Oct 23 08:29:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xu Lu X-Patchwork-Id: 13432517 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 82CEFCDB474 for ; Mon, 23 Oct 2023 08:30:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=DvhMJRAkdtLX/lu8MU60bYPRPfBD9b7wzFvQG7buroQ=; b=Atr1IU8xN3nUJ2 ZSgp6bLH+AbfEHi716UFKGzcptvyR24Cy3j/hJ3RVRyoV/AD0SWOtNfC65IBmKZ6y7P5Ggsob2/Dm 8sonWodTJR/1zIx5cPI0MnhXodmvN+glgHpkPAe6YSlhoL2yIwI2z0aNE/JwJkxkKx7WcwLzpvWHg mnEoIsXGBOYrMZbbnQrUMm5OFNDiaZ6EsaRkSuohDDKBxAtaAIc55xBEw3bzpWCaxKCJZSFxgtHsz 7J6UyT8Fh2jNMYSL3Tra+DhyfCAwu8QwVn1LV6CjMOMaGD5K2OhuyHK+VtgO9/FjuIW4Z7fKcICCu i+6MUmBADK31I2VNXHZw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1quqKI-006lGW-2k; Mon, 23 Oct 2023 08:30:18 +0000 Received: from mail-pf1-x436.google.com ([2607:f8b0:4864:20::436]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1quqKF-006lDy-2V for linux-riscv@lists.infradead.org; Mon, 23 Oct 2023 08:30:17 +0000 Received: by mail-pf1-x436.google.com with SMTP id d2e1a72fcca58-6ba54c3ed97so2910293b3a.2 for ; Mon, 23 Oct 2023 01:30:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1698049812; x=1698654612; darn=lists.infradead.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=CJ95gw/OAzQTy7Pqa/IxTLGW5cdMj2QHsOf1G6j+k+o=; b=VqN51kCiEKhqVD3ruj0WFW8AhO35SlNSGprhK7YGvMvIA+fxbnifYJws41AYU+poKp GAm5tccyOSdHqusRyKE1tjqmE4nNiOEG1cuOvEJDVtJBkyztFNRE0AGcWA9SMrM/ZqTD iof8H1PASd6GHhy0c+Z+NmYqqkVy+gVO1QBL8ONSQ0VwJidR0qxwtK1wRFgFc6F9E1VY xbvhUbpG/w22JGmTMI1ATdbHfjWSFWqsF5LqYxUSP6HlDZWgrHc4nKwGJmCZptbOEkJW +gwkze6IY2df7xH+dlzBT9W7jNkjB+QstsUkhQEzMnNcRb823iUrM8ZmmfE5KzCgcdZt BOkQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1698049812; x=1698654612; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=CJ95gw/OAzQTy7Pqa/IxTLGW5cdMj2QHsOf1G6j+k+o=; b=t2uUZVLDVg/3cpNDIFMGriGa7WBX91ASVgC5wRw2TBKAJu3M2JTZ85lf9SAaBQhDIr d6QtTT0l0QY+giFT71LuZ9G/5tjFYdNHYZaa0850BF22If+oeqGdWo62Fb/1alzXp3+C 7zdNehbg6vQbIndbuLMIaOXzuWazEqSlD6LmJboJbjOpMA0JgNyEWg+ryahMjQuiQDQC 0nFlIViuajXha5//Qz76DET/YRR8gR2cs8+nNmq0i2PIeCoWCPvdt2x6WKyohS64pGYK rB0bM+ijlLkVKuIA3H3wIfOgmDaW0cWnKRsVzXg+ok8D6mrV7lwbUHzVWisk94AN19xn nYgA== X-Gm-Message-State: AOJu0YwsFUjhRyOr5ZMkuHmYR+GLwOVTrZiBrUHTYo0dpfzwh2El2acs Hb0LUfhKdHQEEUuBSNt0ypezmw== X-Google-Smtp-Source: AGHT+IGiT6MKYuJi2ZVpq5CAD0Z+TUVQ/Ug4skMI+tZ5HzA4krihsDz81Ez71goYp3lfV7OVtvYOHA== X-Received: by 2002:a05:6a20:1611:b0:17b:8016:6f79 with SMTP id l17-20020a056a20161100b0017b80166f79mr10196547pzj.3.1698049812218; Mon, 23 Oct 2023 01:30:12 -0700 (PDT) Received: from J9GPGXL7NT.bytedance.net ([203.208.167.147]) by smtp.gmail.com with ESMTPSA id d15-20020a170903230f00b001b8b07bc600sm5415805plh.186.2023.10.23.01.30.06 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Mon, 23 Oct 2023 01:30:11 -0700 (PDT) From: Xu Lu To: paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, tglx@linutronix.de, maz@kernel.org, anup@brainfault.org, atishp@atishpatra.org Cc: dengliang.1214@bytedance.com, liyu.yukiteru@bytedance.com, sunjiadong.lff@bytedance.com, xieyongji@bytedance.com, lihangjing@bytedance.com, chaiwen.cc@bytedance.com, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Xu Lu Subject: [RFC 09/12] riscv: Enable NMIs during exceptions Date: Mon, 23 Oct 2023 16:29:08 +0800 Message-Id: <20231023082911.23242-10-luxu.kernel@bytedance.com> X-Mailer: git-send-email 2.39.3 (Apple Git-145) In-Reply-To: <20231023082911.23242-1-luxu.kernel@bytedance.com> References: <20231023082911.23242-1-luxu.kernel@bytedance.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20231023_013015_817159_750DA83C X-CRM114-Status: UNSURE ( 9.20 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org We have switched the way of disabling irqs to CSR_IE masking. But hardware still automatically clearing SIE field of CSR_STATUS whenever thread traps into kernel, which disabling all irqs including NMIs. This commit re-enables NMIs and normal irqs during exceptions by setting the SIE field in CSR_STATUS and restoring NMI and irq bits in CSR_IE. Signed-off-by: Xu Lu --- arch/riscv/include/asm/irqflags.h | 13 +++++++++++++ arch/riscv/include/asm/switch_to.h | 7 +++++++ arch/riscv/kernel/traps.c | 10 ++++++++++ 3 files changed, 30 insertions(+) diff --git a/arch/riscv/include/asm/irqflags.h b/arch/riscv/include/asm/irqflags.h index 42f7803582df..6a709e9c69ca 100644 --- a/arch/riscv/include/asm/irqflags.h +++ b/arch/riscv/include/asm/irqflags.h @@ -29,6 +29,16 @@ static inline void set_nmi(int irq) {} static inline void unset_nmi(int irq) {} +static inline void enable_nmis(void) +{ + csr_set(CSR_IE, ALLOWED_NMI_MASK); +} + +static inline void disable_nmis(void) +{ + csr_clear(CSR_IE, ALLOWED_NMI_MASK); +} + static inline void local_irq_switch_on(void) { csr_set(CSR_STATUS, SR_IE); @@ -128,6 +138,9 @@ static inline void arch_local_irq_restore(unsigned long flags) csr_set(CSR_STATUS, flags & SR_IE); } +static inline void enable_nmis(void) {} +static inline void disable_nmis(void) {} + #endif /* !CONFIG_RISCV_PSEUDO_NMI */ #endif /* _ASM_RISCV_IRQFLAGS_H */ diff --git a/arch/riscv/include/asm/switch_to.h b/arch/riscv/include/asm/switch_to.h index a727be723c56..116cffeaa6bf 100644 --- a/arch/riscv/include/asm/switch_to.h +++ b/arch/riscv/include/asm/switch_to.h @@ -84,4 +84,11 @@ do { \ ((last) = __switch_to(__prev, __next)); \ } while (0) +#ifdef CONFIG_RISCV_PSEUDO_NMI + +#define prepare_arch_switch(next) disable_nmis() +#define finish_arch_post_lock_switch() enable_nmis() + +#endif /* CONFIG_RISCV_PSEUDO_NMI */ + #endif /* _ASM_RISCV_SWITCH_TO_H */ diff --git a/arch/riscv/kernel/traps.c b/arch/riscv/kernel/traps.c index fae8f610d867..63d3c1417563 100644 --- a/arch/riscv/kernel/traps.c +++ b/arch/riscv/kernel/traps.c @@ -135,7 +135,9 @@ asmlinkage __visible __trap_section void name(struct pt_regs *regs) \ { \ if (user_mode(regs)) { \ irqentry_enter_from_user_mode(regs); \ + enable_nmis(); \ do_trap_error(regs, signo, code, regs->epc, "Oops - " str); \ + disable_nmis(); \ irqentry_exit_to_user_mode(regs); \ } else { \ irqentry_state_t state = irqentry_nmi_enter(regs); \ @@ -292,8 +294,12 @@ asmlinkage __visible __trap_section void do_trap_break(struct pt_regs *regs) if (user_mode(regs)) { irqentry_enter_from_user_mode(regs); + enable_nmis(); + handle_break(regs); + disable_nmis(); + irqentry_exit_to_user_mode(regs); } else { irqentry_state_t state = irqentry_nmi_enter(regs); @@ -338,10 +344,14 @@ asmlinkage __visible noinstr void do_page_fault(struct pt_regs *regs) { irqentry_state_t state = irqentry_enter(regs); + enable_nmis(); + handle_page_fault(regs); local_irq_disable(); + disable_nmis(); + irqentry_exit(regs, state); } #endif From patchwork Mon Oct 23 08:29:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xu Lu X-Patchwork-Id: 13432518 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E4A60C001E0 for ; Mon, 23 Oct 2023 08:30:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=/a94+DSaU6nfMZiefWNu9CAuFzObDWFUeUAzI8lzroM=; b=3Y2klCN3Q8ynGX BH535Mpd0qJKrzo+ebgStVv46ixnxOXR79Sl7juj1WuHZKkNrgC4Zh+4+NtNH4QvFbTho6hA52laJ DOQ3jf2BBOb9huka6u2Yww27jVl5eDIjDLRtey7nVPoNiZl2M39MZRFJAvpZVoxw/lNdMCftZom17 ZAYFVvTWxTN+PuWfREZxay/chf7iU8l2fEiJo2aVDKMkS6oFsXjx9aUTvFgJWebICeo14DUxz6MMm 7t/lFAjga0ymt2Hj0WIHfUkUL3PgCxWy7busH1kVCZL51Gx+OWz7HA6g9gKbfXO8EOw0moVXNuRPr scFR07xaOGm4EkN8qbzA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1quqKN-006lJz-2f; Mon, 23 Oct 2023 08:30:23 +0000 Received: from mail-pj1-x102c.google.com ([2607:f8b0:4864:20::102c]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1quqKK-006lGD-2L for linux-riscv@lists.infradead.org; Mon, 23 Oct 2023 08:30:22 +0000 Received: by mail-pj1-x102c.google.com with SMTP id 98e67ed59e1d1-27d45f5658fso2379400a91.3 for ; Mon, 23 Oct 2023 01:30:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1698049818; x=1698654618; darn=lists.infradead.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=+zaYgIQbfD9rezIYpqI9laPzzTdWx+ZAuyZYYGD6lvk=; b=LdkLoUiHBqLxuy3x49n0+CdlkGBGaLjIKG7tGdywIEDuxNpDPzP5I5Ql1BVI9NnHH6 WZFRlnP6jMCvdyoZUS6s5+IUyl5T0D5YbsTM/CQLlJqGbGI+rLKNmnRwFQ2pkNxVe5dZ xCWCu2OI+skAafKPF1CL8KdYqVZw1smjFG331r/XqetVCxBlhLOu27AhJ9+F2n3K21hn MipU9V0VV+yJqFI8ebdIY3uLYZaVTi+xl/qfk45eG02fAvF7QO4OjAv47vGf/kwmzW/N 7iP1SBzyxmfyzYg0wFYpqpaL8k7R8LgcFy+s+TOjN9mvo3kcXtk71h0rpNGFzKlhrWXH Yx3Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1698049818; x=1698654618; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=+zaYgIQbfD9rezIYpqI9laPzzTdWx+ZAuyZYYGD6lvk=; b=XHPZ9bcR00JWEVJqXlMf4HyTDlJzZffG6NaLCWrVacpNofU9A+2cKuPJSLHcHyqhQ4 pcqQ5Ow61j/Bq3vLTREqvXYnz8UsaRkbkMy4QYNz3zYwwajlUIywgzEtWnCBgoUPziOk o6tG/C8L513aXX9iVOTA5HUtA3WbcD8yD20WRhKEChCw7ekt122uqccslyWqRyqg2Bn5 jPDPcL+vosucRXzXQf33tm76GU76z/nWkmbvN896WAV3fg4hwpYiBdrd/tgKwFc8H6AQ Q0U8/EQ6U8opnCMRgbbYfSqHG1fAoIIAknE/xhwUJlhjvtzVzTXjEsEcqNvWbr3LVOzU sRyQ== X-Gm-Message-State: AOJu0YwCpcFwOxY0zbXMfYo+gmDcHiRHLdG8SBq6vYkicDhAFpRr4RYl 48oT4CGPvIBX4nqbahnyBNCl3g== X-Google-Smtp-Source: AGHT+IFpt0TBJ9j577UmWU9zDZbpQtF3JzZZq4hl33QCJ42B75FqE8SvRamj8B8r3i2TgYxF2UyAzw== X-Received: by 2002:a17:90b:3e85:b0:27d:1567:f51d with SMTP id rj5-20020a17090b3e8500b0027d1567f51dmr9084887pjb.47.1698049817843; Mon, 23 Oct 2023 01:30:17 -0700 (PDT) Received: from J9GPGXL7NT.bytedance.net ([203.208.167.147]) by smtp.gmail.com with ESMTPSA id d15-20020a170903230f00b001b8b07bc600sm5415805plh.186.2023.10.23.01.30.12 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Mon, 23 Oct 2023 01:30:17 -0700 (PDT) From: Xu Lu To: paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, tglx@linutronix.de, maz@kernel.org, anup@brainfault.org, atishp@atishpatra.org Cc: dengliang.1214@bytedance.com, liyu.yukiteru@bytedance.com, sunjiadong.lff@bytedance.com, xieyongji@bytedance.com, lihangjing@bytedance.com, chaiwen.cc@bytedance.com, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Xu Lu Subject: [RFC 10/12] riscv: Enable NMIs during interrupt handling Date: Mon, 23 Oct 2023 16:29:09 +0800 Message-Id: <20231023082911.23242-11-luxu.kernel@bytedance.com> X-Mailer: git-send-email 2.39.3 (Apple Git-145) In-Reply-To: <20231023082911.23242-1-luxu.kernel@bytedance.com> References: <20231023082911.23242-1-luxu.kernel@bytedance.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20231023_013020_767217_AE4BEF69 X-CRM114-Status: UNSURE ( 9.98 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Hardware automatically clearing SIE field of CSR_STATUS whenever thread traps into kernel by interrupt, disabling all irqs including NMIs during interrupt handling. This commit re-enable NMIs during interrupt handling by setting the SIE field in CSR_STATUS and restoring NMIs bits in CSR_IE. Normal interrupts are still disabled during interrupt handling and NMIs are also disabled during NMIs handling to avoid nesting. Signed-off-by: Xu Lu Signed-off-by: Hangjing Li Reviewed-by: Liang Deng Reviewed-by: Yu Li --- arch/riscv/kernel/traps.c | 44 +++++++++++++++++++++++--------- drivers/irqchip/irq-riscv-intc.c | 2 ++ 2 files changed, 34 insertions(+), 12 deletions(-) diff --git a/arch/riscv/kernel/traps.c b/arch/riscv/kernel/traps.c index 63d3c1417563..185743edfa09 100644 --- a/arch/riscv/kernel/traps.c +++ b/arch/riscv/kernel/traps.c @@ -356,20 +356,11 @@ asmlinkage __visible noinstr void do_page_fault(struct pt_regs *regs) } #endif -static void noinstr handle_riscv_irq(struct pt_regs *regs) +static void noinstr do_interrupt(struct pt_regs *regs) { struct pt_regs *old_regs; - irq_enter_rcu(); old_regs = set_irq_regs(regs); - handle_arch_irq(regs); - set_irq_regs(old_regs); - irq_exit_rcu(); -} - -asmlinkage void noinstr do_irq(struct pt_regs *regs) -{ - irqentry_state_t state = irqentry_enter(regs); #ifdef CONFIG_IRQ_STACKS if (on_thread_stack()) { ulong *sp = per_cpu(irq_stack_ptr, smp_processor_id()) @@ -382,7 +373,9 @@ asmlinkage void noinstr do_irq(struct pt_regs *regs) "addi s0, sp, 2*"RISCV_SZPTR "\n" "move sp, %[sp] \n" "move a0, %[regs] \n" - "call handle_riscv_irq \n" + "la t0, handle_arch_irq \n" + REG_L" t1, (t0) \n" + "jalr t1 \n" "addi sp, s0, -2*"RISCV_SZPTR"\n" REG_L" s0, (sp) \n" "addi sp, sp, "RISCV_SZPTR "\n" @@ -398,11 +391,38 @@ asmlinkage void noinstr do_irq(struct pt_regs *regs) "memory"); } else #endif - handle_riscv_irq(regs); + handle_arch_irq(regs); + set_irq_regs(old_regs); +} + +static __always_inline void __do_nmi(struct pt_regs *regs) +{ + irqentry_state_t state = irqentry_nmi_enter(regs); + + do_interrupt(regs); + + irqentry_nmi_exit(regs, state); +} + +static __always_inline void __do_irq(struct pt_regs *regs) +{ + irqentry_state_t state = irqentry_enter(regs); + + irq_enter_rcu(); + do_interrupt(regs); + irq_exit_rcu(); irqentry_exit(regs, state); } +asmlinkage void noinstr do_irq(struct pt_regs *regs) +{ + if (IS_ENABLED(CONFIG_RISCV_PSEUDO_NMI) && regs_irqs_disabled(regs)) + __do_nmi(regs); + else + __do_irq(regs); +} + #ifdef CONFIG_GENERIC_BUG int is_valid_bugaddr(unsigned long pc) { diff --git a/drivers/irqchip/irq-riscv-intc.c b/drivers/irqchip/irq-riscv-intc.c index c672c0c64d5d..80ed8606e04d 100644 --- a/drivers/irqchip/irq-riscv-intc.c +++ b/drivers/irqchip/irq-riscv-intc.c @@ -34,7 +34,9 @@ static asmlinkage void riscv_intc_irq(struct pt_regs *regs) generic_handle_domain_nmi(intc_domain, cause); nmi_exit(); } else { + enable_nmis(); generic_handle_domain_irq(intc_domain, cause); + disable_nmis(); } } From patchwork Mon Oct 23 08:29:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xu Lu X-Patchwork-Id: 13432519 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 7198DCDB474 for ; Mon, 23 Oct 2023 08:30:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=JysFqVdfENDyPf5ltNOKFgDFqnkufuI3AvrQvp7XI+k=; b=VFpmhmf4OOF3TG /lkduIfJs9nnDNn4NNx+TA9r5WHBGsgnQqMuqufPuWEIGnZb8KHkmgGU1nLR0rn2a8MEboOknYePm hPZiWx6aDFVHQY85r5eQPu1cCLLduEfNaTxy4F3sHY1gV13clIG/T+8TlbRqa7VRa0T/i3L/dh4NM NwlMlfwv7Ge2GbTqpM+eKOJ0ORGkhtAI2LekR6Sv4wo/d3tW+SfceyzSx+o6vE+/VLWRudNemWmNN G35cgT776UAVvg6drC0gODoMIgvHIt746+pvwZxvXUvVN+3CO1c2f/CpOh605TSeY1SVaS7IeG9PV GQa0AkpWdCHTNPNAV+zw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1quqKR-006lLl-1d; Mon, 23 Oct 2023 08:30:27 +0000 Received: from mail-pl1-x634.google.com ([2607:f8b0:4864:20::634]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1quqKO-006lK3-3C for linux-riscv@lists.infradead.org; Mon, 23 Oct 2023 08:30:26 +0000 Received: by mail-pl1-x634.google.com with SMTP id d9443c01a7336-1c9e06f058bso29330325ad.0 for ; Mon, 23 Oct 2023 01:30:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1698049823; x=1698654623; darn=lists.infradead.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=0DBuRxeBwydw4BwBQS/PBMNuUJKwvOuK50YBTu/VQqs=; b=fSJk8zMYngAD5EGJ4K0FP25izTLvroaccnT5B6kysHgklXot5Jd74NCsRtc89bveEO +z8zH88Ib0SsbxvF2QXOFLgpRU0l1v/iENSdpJV6XJvil8xqYnwB+patPlkGGZCVxLLg nHiBu1jh4tf8/maing9yKR+oAVzbXngjCjCgRgX6nhUqrnzKFfG3ET9R3WGO0PQBgxW+ HvtxZvzdgWny2QENtTfzZHjQl4/oIerWRl/33c5tdV9C1hxSHuUi403+XOowoMn0u7+u 0xTzWvYVd/FW/v8yFoGM04Y2mrb7uiHQMQL20DFqYA9+sXAm1wuqmmUvAd/UVJFmrMcf jfSg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1698049823; x=1698654623; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=0DBuRxeBwydw4BwBQS/PBMNuUJKwvOuK50YBTu/VQqs=; b=YutzziCu3djhoXlFyX0xbyn5Lj8XDWm/TzI5wjaKzQWpprA6qdKVFXx3ygqUWiZntH Sxb4q/vKGdVpQ4JXj/Hnw3CGQCgDiAMnVJeAzO+uUqZtdRFFDbLXWuX6r7bvt6eT0g1u sfJpRXk80Kv21yi+whCiq1lXCWHyXMHYRBQVR3CitnrijLAiw2dMtefqSu6aTA1UNZSw QlBQHV7v5UCThq6uSd4rgmocSjXMojarDJsRSkLT7LZ1a5JFue4YuFXz89Qldv8yH9op W/dynylYL0fVFvC19NwyTi7W+1wYK56NmlLn93biwhKNbELBAUMAptA3NEMAp7fuVC7N 4D3w== X-Gm-Message-State: AOJu0YwsaF8ERpaz8cq4q9TQuUn1YahIyh2ICwViYyRIHr56A17i/qaJ aWaBbMOfZ1CDcow0kzIV8XFlfA== X-Google-Smtp-Source: AGHT+IFbYL9LzR1f7qPWMFMhpIw/jq2fW6XuDR85z/dEj8js9eHh6BWdpBWgonHNNRMuACn/U0IMZw== X-Received: by 2002:a17:903:32d2:b0:1c6:d0a:cf01 with SMTP id i18-20020a17090332d200b001c60d0acf01mr16447416plr.11.1698049823515; Mon, 23 Oct 2023 01:30:23 -0700 (PDT) Received: from J9GPGXL7NT.bytedance.net ([203.208.167.147]) by smtp.gmail.com with ESMTPSA id d15-20020a170903230f00b001b8b07bc600sm5415805plh.186.2023.10.23.01.30.18 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Mon, 23 Oct 2023 01:30:23 -0700 (PDT) From: Xu Lu To: paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, tglx@linutronix.de, maz@kernel.org, anup@brainfault.org, atishp@atishpatra.org Cc: dengliang.1214@bytedance.com, liyu.yukiteru@bytedance.com, sunjiadong.lff@bytedance.com, xieyongji@bytedance.com, lihangjing@bytedance.com, chaiwen.cc@bytedance.com, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Xu Lu Subject: [RFC 11/12] riscv: Request pmu overflow interrupt as NMI Date: Mon, 23 Oct 2023 16:29:10 +0800 Message-Id: <20231023082911.23242-12-luxu.kernel@bytedance.com> X-Mailer: git-send-email 2.39.3 (Apple Git-145) In-Reply-To: <20231023082911.23242-1-luxu.kernel@bytedance.com> References: <20231023082911.23242-1-luxu.kernel@bytedance.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20231023_013025_031297_77F893B5 X-CRM114-Status: GOOD ( 11.94 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org This commit registers pmu overflow interrupt as NMI to improve the accuracy of perf sampling. Signed-off-by: Xu Lu --- arch/riscv/include/asm/irqflags.h | 2 +- drivers/perf/riscv_pmu_sbi.c | 23 +++++++++++++++++++---- 2 files changed, 20 insertions(+), 5 deletions(-) diff --git a/arch/riscv/include/asm/irqflags.h b/arch/riscv/include/asm/irqflags.h index 6a709e9c69ca..be840e297559 100644 --- a/arch/riscv/include/asm/irqflags.h +++ b/arch/riscv/include/asm/irqflags.h @@ -12,7 +12,7 @@ #ifdef CONFIG_RISCV_PSEUDO_NMI -#define __ALLOWED_NMI_MASK 0 +#define __ALLOWED_NMI_MASK BIT(IRQ_PMU_OVF) #define ALLOWED_NMI_MASK (__ALLOWED_NMI_MASK & irqs_enabled_ie) static inline bool nmi_allowed(int irq) diff --git a/drivers/perf/riscv_pmu_sbi.c b/drivers/perf/riscv_pmu_sbi.c index 995b501ec721..85abb7dd43b9 100644 --- a/drivers/perf/riscv_pmu_sbi.c +++ b/drivers/perf/riscv_pmu_sbi.c @@ -760,6 +760,7 @@ static irqreturn_t pmu_sbi_ovf_handler(int irq, void *dev) static int pmu_sbi_starting_cpu(unsigned int cpu, struct hlist_node *node) { + int ret = 0; struct riscv_pmu *pmu = hlist_entry_safe(node, struct riscv_pmu, node); struct cpu_hw_events *cpu_hw_evt = this_cpu_ptr(pmu->hw_events); @@ -778,20 +779,30 @@ static int pmu_sbi_starting_cpu(unsigned int cpu, struct hlist_node *node) if (riscv_pmu_use_irq) { cpu_hw_evt->irq = riscv_pmu_irq; csr_clear(CSR_IP, BIT(riscv_pmu_irq_num)); -#ifndef CONFIG_RISCV_PSEUDO_NMI +#ifdef CONFIG_RISCV_PSEUDO_NMI + ret = prepare_percpu_nmi(riscv_pmu_irq); + if (ret != 0) { + pr_err("Failed to prepare percpu nmi:%d\n", ret); + return ret; + } + enable_percpu_nmi(riscv_pmu_irq, IRQ_TYPE_NONE); +#else csr_set(CSR_IE, BIT(riscv_pmu_irq_num)); -#endif enable_percpu_irq(riscv_pmu_irq, IRQ_TYPE_NONE); +#endif } - return 0; + return ret; } static int pmu_sbi_dying_cpu(unsigned int cpu, struct hlist_node *node) { if (riscv_pmu_use_irq) { +#ifdef CONFIG_RISCV_PSEUDO_NMI + disable_percpu_nmi(riscv_pmu_irq); + teardown_percpu_nmi(riscv_pmu_irq); +#else disable_percpu_irq(riscv_pmu_irq); -#ifndef CONFIG_RISCV_PSEUDO_NMI csr_clear(CSR_IE, BIT(riscv_pmu_irq_num)); #endif } @@ -835,7 +846,11 @@ static int pmu_sbi_setup_irqs(struct riscv_pmu *pmu, struct platform_device *pde return -ENODEV; } +#ifdef CONFIG_RISCV_PSEUDO_NMI + ret = request_percpu_nmi(riscv_pmu_irq, pmu_sbi_ovf_handler, "riscv-pmu", hw_events); +#else ret = request_percpu_irq(riscv_pmu_irq, pmu_sbi_ovf_handler, "riscv-pmu", hw_events); +#endif if (ret) { pr_err("registering percpu irq failed [%d]\n", ret); return ret; From patchwork Mon Oct 23 08:29:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xu Lu X-Patchwork-Id: 13432520 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C66E7C001E0 for ; Mon, 23 Oct 2023 08:30:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=lE5jjWbA+kBHJmnkajQZYgjRQISpPEXsYteJhDNbIj4=; b=0xKQD0zx1P2zeL pEilhEMUP5G5zwRse9iiJJnN9sA2ptr1wF8LC09tIb5LRwyW+vtWFlKwFUz0P2Ik5T7e1zH0JPrOA vva+KyFH/eizs10UXB6p0jGKMrqW7wHskmxNXW7uSrcV5Oy5TPE1X/jYglwSAp+pcBIA8Un04uqRn ebRAYAZxjC3jOvAzSn3vIMWrfv7u+M0YZee435pba7KGRMYXXQyShmLZPGKrtIW7tATwcpLrriKGC GzpSJE5M1DfmHPzr5I8v/HVaQ0HYrjGCDs2H9/L7ia2w0z9npXAdfw190SUpRH6UKflg3JlvaUpMB ui5lgrz1lb+UG4MFjw3w==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1quqKX-006lOl-1J; Mon, 23 Oct 2023 08:30:33 +0000 Received: from mail-pf1-x430.google.com ([2607:f8b0:4864:20::430]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1quqKV-006lMo-1H for linux-riscv@lists.infradead.org; Mon, 23 Oct 2023 08:30:32 +0000 Received: by mail-pf1-x430.google.com with SMTP id d2e1a72fcca58-6b87c1edfd5so2169689b3a.1 for ; Mon, 23 Oct 2023 01:30:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1698049829; x=1698654629; darn=lists.infradead.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=qwqRYcNnh3bL2bnUHbsBXwm7mmQqzZo0pVLQzAgbeF0=; b=Qu2p3HQSi7SIzmTcWU49W2O6uPKqT3+OmWOzzQrK6f79s7heXjBgYp35v7elqlldGR Ay/uSh96T5mpK5MGhYLGKQvln0tc/twnag18mpGI6w8P5C9HQNJ/vl1eSiZ557Sbzhll YACXL5BEFSFYlnvYnOLDquicEceB3ikflneGK2m1rLGJeDT5Li3HP7rT4iV9PfPbz5az Bza1bvNTapGU2v8Ec/71wWzc7Y2Q+WNM3Z8niP8aqDYEqRAVVNKWUcbzV4HH9LgkgrNM aSDU4x8c5o01WYua8la28p5hTSqUau2w2G0rRy8PKRQjTmtqYPY4ZDeGWMhIdj6+fdmV B3Xw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1698049829; x=1698654629; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=qwqRYcNnh3bL2bnUHbsBXwm7mmQqzZo0pVLQzAgbeF0=; b=UCB91yC1AeE6+pwx8813oeXraoY/3p0lLi23NLc8AQV0+xJrbQP58aDBn032zYKF9M KG7avVmdwRh/quoevdvo+W5Tx8Li4Ewyy19tzI5PFfz1MDQcefl5qD9rs8u20s+uzetP FUQsNXyt+19PHXVdUBwHA/ljEC1RwsodgfMQJryd0Iv0Pkl8aBsYeXTyB3RsZ8BuiLZC PLBoJ5lRdn/aYPtoGTzUKwiGqLyBctOjc+h1vj0QWhIgtd+tbqudtSFnSA/eO/DBgrBt 39zaPVdZfGlPoZHlNP4XeKGtfJZuc3NaDfXkw3xXxiB/pep6z5GPIBxS0LBJqGmxnbe7 P1pQ== X-Gm-Message-State: AOJu0YzW8NNY5Evk6HsVQupJ30ywBzbhvIVFHdNJ3zQyaR77SJiiuubA qI8or2+JyRFC0M3n6yqdMpBUJQ== X-Google-Smtp-Source: AGHT+IGyFbUXBRXdZQh2eZcjKZcfVc6v/Ld3/FBhlNzBXs9NtpZ6v0cDf/3miDn1ZPth5cbBQ//TYA== X-Received: by 2002:a05:6a21:498b:b0:17b:d39c:1d6 with SMTP id ax11-20020a056a21498b00b0017bd39c01d6mr5328684pzc.30.1698049829431; Mon, 23 Oct 2023 01:30:29 -0700 (PDT) Received: from J9GPGXL7NT.bytedance.net ([203.208.167.147]) by smtp.gmail.com with ESMTPSA id d15-20020a170903230f00b001b8b07bc600sm5415805plh.186.2023.10.23.01.30.23 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Mon, 23 Oct 2023 01:30:28 -0700 (PDT) From: Xu Lu To: paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, tglx@linutronix.de, maz@kernel.org, anup@brainfault.org, atishp@atishpatra.org Cc: dengliang.1214@bytedance.com, liyu.yukiteru@bytedance.com, sunjiadong.lff@bytedance.com, xieyongji@bytedance.com, lihangjing@bytedance.com, chaiwen.cc@bytedance.com, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Xu Lu Subject: [RFC 12/12] riscv: Enable CONFIG_RISCV_PSEUDO_NMI in default Date: Mon, 23 Oct 2023 16:29:11 +0800 Message-Id: <20231023082911.23242-13-luxu.kernel@bytedance.com> X-Mailer: git-send-email 2.39.3 (Apple Git-145) In-Reply-To: <20231023082911.23242-1-luxu.kernel@bytedance.com> References: <20231023082911.23242-1-luxu.kernel@bytedance.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20231023_013031_436360_73706B75 X-CRM114-Status: UNSURE ( 8.86 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org This commit enables CONFIG_RISCV_PSEUDO_NMI in default. Now pseudo NMI feature is defaultly enabled on RISC-V. Signed-off-by: Xu Lu --- arch/riscv/Kconfig | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index 487e4293f31e..ecccdc91563f 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -672,7 +672,7 @@ config RISCV_BOOT_SPINWAIT config RISCV_PSEUDO_NMI bool "Support for NMI-like interrupts" depends on !RISCV_M_MODE - default n + default y help Adds support for mimicking Non-Maskable Interrupts through the use of CSR_IE register.