From patchwork Tue Nov 16 08:24:48 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pingfan Liu X-Patchwork-Id: 12692711 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 92CCAC433EF for ; Tue, 16 Nov 2021 08:27:29 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5B99861B49 for ; Tue, 16 Nov 2021 08:27:29 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 5B99861B49 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=3oOHl3JKBixrH1GFhBlypsSeMDz0FJhlsrJbm6Rp0KE=; b=sXm5QNK/5sUjjv PyJn3UavPUpmSKYMPKhqL8r7F9EJwh51DVQ1PmSo61pedFy0il5fJSVi0dcIUqb9yQ0jep8kCxtdd VIhD/V5hL69Y4X08nhGZfT53UnnjsRBxKOlQNcaopER5G3kCkiI1ud7Y6tQ8/OoxsSlVlXmFFSrGy H5RsSQHEGL0rn4kDQ9HnEggPhVaRYBzzYsUd/qyhZ5q1fNnlFd6uzHYXnc1Hr7Zz2EhOEgdDBNzV0 fQ6KVGLGQDUbNBkPIkghwbehotrPPNCznzoxFhr68Mqx28gM/xXcj+TjZka5qB0npJXvnvAcM9ZSe 9kIsembNrSH4KPx1kTOg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mmtmu-000hMd-6r; Tue, 16 Nov 2021 08:25:56 +0000 Received: from mail-pj1-x1032.google.com ([2607:f8b0:4864:20::1032]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mmtmS-000hFG-Bp for linux-arm-kernel@lists.infradead.org; Tue, 16 Nov 2021 08:25:29 +0000 Received: by mail-pj1-x1032.google.com with SMTP id h24so15149532pjq.2 for ; Tue, 16 Nov 2021 00:25:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Wk9eWki6Vbr1g31d7LZ5+7G/iNYTD2zJddi+RKAgoB0=; b=qbQgqz/6Q4V2kYlnnIy4Cm0ACd+yZ6NY9vHiRejN/0QP3DdiFmCNfBsvlwNNiAcAQR evQve1SXuMG27TeaKq9uqwfbMziTpF0Ono9iNzytndm5pxxNEynJYkYc5XS1sqvbzlJA Qk7rHY51WRAWfnsW53QfEWaTAj+ygL+hbZRMrJaVlSOZnboNjmP+K5wXSbA9QP6eTjWt L9KzIruwbF2d0tFIKUleW8iQye3lK7UckJthsUCirZgKW/0dNb/pyRv/JLtIo7+RmDAM uPir8R09bq01bEgE05LPkD95Bo83PelkxEtR2kVqJ+FA4bcHmt2nCoUEf29TFw5HkTGC 7UPA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Wk9eWki6Vbr1g31d7LZ5+7G/iNYTD2zJddi+RKAgoB0=; b=EAfMPvy98GXVaE8qI+84+Q19WE4nx+9fK59lxnge2/1JwGlzejnHMj+g6IfFoyQdwt fmvQpqEj9iFMCtB112zXm7VYGAk7+WndR/JAMXtROA1BLU41eqWug6E8DOfpj8P93yUD JgqGqoFocb6bBKd8KpDMAUUdn/Pj5ExWcdqL7fUxudqXzryS8+v5VkMSlkYdk3PU/IH3 wuxz5NDQtPxLuC4g2J/n/eSVyQpERRQZ7U0iwy+wbhWnpD2drUwxfbFobrLTKbFDRJwa IVOPZbCxqy0fQ5y5rLZSpnLmxb4qB7SiRnnhtA9AiF4y4EESsUCenAY4WHXaq/JE4Os7 juEA== X-Gm-Message-State: AOAM533i4uR2uQ8MiNuU7Zm5Pw5VF8VvoCULQ9pzjYPYLdeN1h4BzvTp B9gn2f/Wgbc+T87S2ik4qRJcmzO/ng== X-Google-Smtp-Source: ABdhPJy/K+gtd8EEbXE7wMYw1LqoOMvVTVeuoRYSX4WAXNtZ2IJko0ykR48BE85DAsBHQeQcm+FwAA== X-Received: by 2002:a17:902:ce85:b0:141:de7d:514e with SMTP id f5-20020a170902ce8500b00141de7d514emr42952552plg.0.1637051126766; Tue, 16 Nov 2021 00:25:26 -0800 (PST) Received: from piliu.users.ipa.redhat.com ([209.132.188.80]) by smtp.gmail.com with ESMTPSA id 11sm17104011pfl.41.2021.11.16.00.25.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 Nov 2021 00:25:26 -0800 (PST) From: Pingfan Liu To: linux-arm-kernel@lists.infradead.org Cc: Pingfan Liu , Mark Rutland , "Paul E . McKenney" , Catalin Marinas , Will Deacon , Marc Zyngier , Thomas Gleixner , Joey Gouly , Sami Tolvanen , Julien Thierry , Yuichi Ito , rcu@vger.kernel.org Subject: [PATCHv3 2/4] arm64: entry: distinguish pNMI earlier in el0 interrupt Date: Tue, 16 Nov 2021 16:24:48 +0800 Message-Id: <20211116082450.10357-3-kernelfans@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20211116082450.10357-1-kernelfans@gmail.com> References: <20211116082450.10357-1-kernelfans@gmail.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211116_002528_482327_7B81DB11 X-CRM114-Status: GOOD ( 15.19 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org For ease of unifying code, it is helpful to lift nmi_{enter,exit}() housekeeping from gic_handle_nmi() to el0_interrupt(). Because gic_handle_nmi() is called by either el1 interrupt or el0, and the housekeeping has already been done in arch level code when el1 interrupt. Note about the original code, which calls enter_from_user_mode() in pNMI context. Although it is weird to call rcu_eqs_exit() in the pseudo NMI context, it has no problem. This is due to the essentiality of pNMI, a higher priority interrupt but not akin to NMI, which allows a break-in at any time. Signed-off-by: Pingfan Liu Cc: Mark Rutland Cc: Paul E. McKenney Cc: Catalin Marinas Cc: Will Deacon Cc: Marc Zyngier Cc: Thomas Gleixner Cc: Joey Gouly Cc: Sami Tolvanen Cc: Julien Thierry Cc: Yuichi Ito Cc: rcu@vger.kernel.org To: linux-arm-kernel@lists.infradead.org --- arch/arm64/kernel/entry-common.c | 35 ++++++++++++++++++++++++-------- 1 file changed, 27 insertions(+), 8 deletions(-) diff --git a/arch/arm64/kernel/entry-common.c b/arch/arm64/kernel/entry-common.c index 5a1a5dd66d04..afcde43f1b73 100644 --- a/arch/arm64/kernel/entry-common.c +++ b/arch/arm64/kernel/entry-common.c @@ -429,7 +429,7 @@ asmlinkage void noinstr el1h_64_sync_handler(struct pt_regs *regs) } } -static __always_inline void __el1_pnmi(struct pt_regs *regs, +static __always_inline void __pnmi_handler_common(struct pt_regs *regs, void (*handler)(struct pt_regs *)) { arm64_enter_nmi(regs); @@ -437,6 +437,12 @@ static __always_inline void __el1_pnmi(struct pt_regs *regs, arm64_exit_nmi(regs); } +static __always_inline void __el1_pnmi(struct pt_regs *regs, + void (*handler)(struct pt_regs *)) +{ + __pnmi_handler_common(regs, handler); +} + static __always_inline void __el1_irq(struct pt_regs *regs, void (*handler)(struct pt_regs *)) { @@ -673,21 +679,34 @@ asmlinkage void noinstr el0t_64_sync_handler(struct pt_regs *regs) } } -static void noinstr el0_interrupt(struct pt_regs *regs, - void (*handler)(struct pt_regs *)) +static __always_inline void __el0_pnmi(struct pt_regs *regs, + void (*handler)(struct pt_regs *)) +{ + __pnmi_handler_common(regs, handler); +} + +static __always_inline void __el0_irq(struct pt_regs *regs, + void (*handler)(struct pt_regs *)) { enter_from_user_mode(regs); + irq_enter_rcu(); + do_interrupt_handler(regs, handler); + irq_exit_rcu(); + exit_to_user_mode(regs); +} +static void noinstr el0_interrupt(struct pt_regs *regs, + void (*handler)(struct pt_regs *)) +{ write_sysreg(DAIF_PROCCTX_NOIRQ, daif); if (regs->pc & BIT(55)) arm64_apply_bp_hardening(); - irq_enter_rcu(); - do_interrupt_handler(regs, handler); - irq_exit_rcu(); - - exit_to_user_mode(regs); + if (IS_ENABLED(CONFIG_ARM64_PSEUDO_NMI) && is_in_pnmi(regs)) + __el0_pnmi(regs, handler); + else + __el0_irq(regs, handler); } static void noinstr __el0_irq_handler_common(struct pt_regs *regs)