From patchwork Tue Feb 25 09:54:48 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13989716 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 792F3C021B2 for ; Tue, 25 Feb 2025 10:10:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=2IJUdaO+5dTLVXPMwfcKz8isZKL74F9HfpoDdIYxpPA=; b=I8dIfHhzP4qZ9cKZBNC1v+U3/v OxNy/q5mesXVnHJEjQVwhBOV1qhIxq3H5KuE8d9cAUs39rTF9uq/QDyjmG9TUiCwoEzWu6yga50rd iGPSbfEfFstiMDFeV3RQJzI3t9vSTpZDk92Taagrm1lynsoPQPoB2Mgl0GGpFGJ33HewoMYAkpe7q YstCp3SCCycuHJwgFwj75VbQnHEXL8Zw49dCLqZ/OgBiBLPs6uhWrsCwBuNhTqkY8R7JIJUNXRZAT 7zCy8OJ5cziZmMCItchCf9PjW/v1/uBdWjbdjlegmWLwnRMzJ1KOmaUdN4WYhvRuPvWpulIca0WLG HfuP+qqg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tmrsv-0000000GlQ1-3kDA; Tue, 25 Feb 2025 10:09:53 +0000 Received: from mail-lf1-x132.google.com ([2a00:1450:4864:20::132]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tmreY-0000000Ghad-2MQL for linux-arm-kernel@lists.infradead.org; Tue, 25 Feb 2025 09:55:03 +0000 Received: by mail-lf1-x132.google.com with SMTP id 2adb3069b0e04-54622829175so5107609e87.0 for ; Tue, 25 Feb 2025 01:55:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1740477301; x=1741082101; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=2IJUdaO+5dTLVXPMwfcKz8isZKL74F9HfpoDdIYxpPA=; b=L3athRDjhL8TJ51SJe8+Y0X48MIIbC76RaWQfYELB7JWDZZjX1nCYkYMW6Cg5ORK2J HusBht0MmZG4G+8X13GZhh+irm4icJ/Ox5HvIuk1EuzaXkHjzkoMXgOLRdjnh9t2dKSv RhIXcKxOel/rYVNRChEFTJy8h1jyy+JtAtBZbezQlGKCH94SZUgJGJdS4LYOWr1shm42 5sXiST7sODdzfTaeqgbEG3/ZhkNOMA5fQVVOCy9OD0pW+tWXadUhVZr1hPpqAa1gnmDo 2bxOTbjZLGwmPYHqoCOADtUxpwDTF8F9Z9pSKl/LahyMMm7o1oc/lpt9EHZhydXqfUCu oWrw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740477301; x=1741082101; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=2IJUdaO+5dTLVXPMwfcKz8isZKL74F9HfpoDdIYxpPA=; b=RxDECzf3gFvkT7gM343klin7KNPyQ50mrApFX60CIX4EKs3y5kUuDW1zXUCX42lNUn 1rAvaDirmXfGSM0spk7iXQi16onVGYna5JJxtj0fUWG1b25+ZwuWeMg5msSjku7eiVF0 zr4mpnkwMta6sredk+Q/nyIY5/b0fBfYdNVqOseegWxOqpz3rM3niBJnUVP3Jf+I1k4J pcMEgtxPEH9q1L9cj8WubDaovnI0tyTWdTxig1qpiw0AqegJ5h9lLx8O8aLlvtbSUDFv cogVCHR0StEN+j2Qr/xD+mBBDCSoNzKgK/ERaU0FzVJraV9Vwnngw0HlBBWXxemzHSRc yeWQ== X-Gm-Message-State: AOJu0Yxizeak97Wnx+hF3FWrCi9ixunua0UWdMtlOEtIfNjwSwvDTL3F ZJ1coIz3FgnAlSXNCCt25KnkWslZmvtsT8nH6D/dm8PwezsM9QEkJR1jShgf83M= X-Gm-Gg: ASbGncvqodm1vO6uJ1X7BbNFZT6Kxpn+hVczCiDA1FQK8c6DT5uMbk/kIvpBuNjeLzL sqksCpN4b7MYojAPcfdjv2lCdxYx0ZjmwR5+N6hXVSYJINVmDlq1JKZnvuLwKc89HIzNCsbtvHc e8N3gTsUASt+yPpVFUif+K7qmOMoBVAI6DLo4ksGcaze3mgGSxsabvOIiqmT4wOWlNAyrXCBkGX fDNsb/ab8RXc2pheXF6znVqwyAbkM3ZBtShJDd6RjgY7T8muLeMsWJ+t6bibBZSGZePzJvzm3ne SXWuaQ2ZCJHp2VpdDPC1hmY9C4Z56OLRdIF3 X-Google-Smtp-Source: AGHT+IHd+JITPLxMXULU7ZpD1f9xac7PGEst2Tk9nAmNTghxAif9Rd8qWrav88C8IW+IaffyeOes+A== X-Received: by 2002:a05:6512:2310:b0:545:16a5:10f5 with SMTP id 2adb3069b0e04-548391452b5mr5483522e87.30.1740477300601; Tue, 25 Feb 2025 01:55:00 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id 2adb3069b0e04-548514b261esm132867e87.24.2025.02.25.01.54.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 25 Feb 2025 01:55:00 -0800 (PST) From: Linus Walleij Date: Tue, 25 Feb 2025 10:54:48 +0100 Subject: [PATCH v5 01/31] ARM: Prepare includes for generic entry MIME-Version: 1.0 Message-Id: <20250225-arm-generic-entry-v5-1-2f02313653e5@linaro.org> References: <20250225-arm-generic-entry-v5-0-2f02313653e5@linaro.org> In-Reply-To: <20250225-arm-generic-entry-v5-0-2f02313653e5@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250225_015502_604908_1B10E7D3 X-CRM114-Status: GOOD ( 14.05 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The generic entry code needs a static inline function for regs_irq_disabled() so add this, in further requires the signature of on_thread_stack() to be __always_inline so add this as well. Signed-off-by: Linus Walleij --- arch/arm/include/asm/ptrace.h | 5 +++++ arch/arm/include/asm/stacktrace.h | 2 +- 2 files changed, 6 insertions(+), 1 deletion(-) diff --git a/arch/arm/include/asm/ptrace.h b/arch/arm/include/asm/ptrace.h index 6eb311fb2da06fa393f8be7caec8d997637a88b2..a01c66f0ad907882af2a383e53de8aca68c551e5 100644 --- a/arch/arm/include/asm/ptrace.h +++ b/arch/arm/include/asm/ptrace.h @@ -52,6 +52,11 @@ struct svc_pt_regs { #define fast_interrupts_enabled(regs) \ (!((regs)->ARM_cpsr & PSR_F_BIT)) +static inline int regs_irqs_disabled(struct pt_regs *regs) +{ + return !interrupts_enabled(regs); +} + /* Are the current registers suitable for user mode? * (used to maintain security in signal handlers) */ diff --git a/arch/arm/include/asm/stacktrace.h b/arch/arm/include/asm/stacktrace.h index f80a85b091d6c4ff365e15ae7100af1c3aed597f..815b5f256af164daa053f3bfd6cb2f54042a8d2b 100644 --- a/arch/arm/include/asm/stacktrace.h +++ b/arch/arm/include/asm/stacktrace.h @@ -26,7 +26,7 @@ struct stackframe { #endif }; -static inline bool on_thread_stack(void) +static __always_inline bool on_thread_stack(void) { unsigned long delta = current_stack_pointer ^ (unsigned long)current->stack; From patchwork Tue Feb 25 09:54:49 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13989722 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E04CAC021B2 for ; Tue, 25 Feb 2025 10:11:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=LeNYI4Yc1OrceUVt/GJDCyO7YwVHZ9lpFFe5ekDh0x0=; b=4g/uBNABkCEhlEskldOyI+P61n uJDcWHbU+1+rFEfrbebnZk8WolDC5PeodR86EtOMACgQNGP6XyjDB3qlnVT5uJAxYFg87R89I3/2g iA/ymaVx7hy7X1tTZL50hCoSR33MOwebyvZ7Y2fqUyhU9ovhkGaBh3NmmvprT0B1th3saqWNd+hL2 myAmN6CKRFEEwM0bYjTjvz0Tw3e9PRE3RuoWe/WQvfw69q6VACPDdRCB+wM9zFlQP8xLMsA0wfIwF Ua2BZJE1eEWL5XPDUrCk4Pk6Ul6CAkCy/hzApTPCXuTzPexIbHXJNG3hdOsBNB6hkG2A/jzG+Um8H vOuTm8DA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tmruS-0000000Glgo-2WPE; Tue, 25 Feb 2025 10:11:28 +0000 Received: from mail-lf1-x12e.google.com ([2a00:1450:4864:20::12e]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tmreZ-0000000Ghav-2XgU for linux-arm-kernel@lists.infradead.org; Tue, 25 Feb 2025 09:55:04 +0000 Received: by mail-lf1-x12e.google.com with SMTP id 2adb3069b0e04-5452e6f2999so5746973e87.3 for ; Tue, 25 Feb 2025 01:55:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1740477302; x=1741082102; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=LeNYI4Yc1OrceUVt/GJDCyO7YwVHZ9lpFFe5ekDh0x0=; b=Kop6qwSZvdVkroZ8GZOYV5kKwR5LfxmdyykF4XMei3TmNBDdei74kYKwp74rUFZNCK twfH3TDIiUqyBM4zQasbxyVjGL8Qj4UTcb8z+rtbh2gKo2lVqla4LnjsQXGAHjhHoQoj gR0pQ+1Swm/fyOXsB0VxXa6xVTFN7MR6w9zt/mnchmMR6Ih3H+bC/JBmKHb/cCI+KZcI lA7q/TkflHsMQ2m8+cPUKoRcAU8qzUG/uHFanmCE2fyatjb4xMHHkZCo1zGKGHbtWXHC rd9hJIoynqGqyJe2XCBT8y8tIS+nR3OeR++rhnT/VzSJoBJg24r8pp8wObsts20SP/J+ stEA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740477302; x=1741082102; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=LeNYI4Yc1OrceUVt/GJDCyO7YwVHZ9lpFFe5ekDh0x0=; b=VU2oaacm8ODIJ5yC1o0mc58tmv5cHGFAEiPKy0Njhq8913rJH/nnLlFv8/xgNZlgXc onENGh61K5N4tpJFhxgYZIoo6A5YVMB5D489zpJyhj0ZTe+UadswedAtwQgd+yvVcZqg Cnj2BJoPO05BqnPeqnawIXgCqGdSBe9StVdTL6eRHuBxfXu8F7Zmh41LtI/lfoDozqlJ CEyBF9o6q5jABtyI6eCBQO9xM5a7lkHAkVCeDDwp4WqmiXI2q7cw58aZxRpp53Fpknf4 VsqgoPZ74gPHRbtHYG6IWubE0HVTbBnEXoHUxQ0yqqEqMPuyvTtiBo2IvJJ6CgjuENEM FNHg== X-Gm-Message-State: AOJu0Yxgg3rH1EDHytHqVD63OnWew3fPlUn4jbAAcfk9pd1JTzroHw37 LghKiZuq1DCF4j89opqPa/7wv/D4zVPxWNf+RH3VoX4Fao68fDjmPqYOBxTYVC8= X-Gm-Gg: ASbGncsv4lKIdV2No35A2ku00FVuXeKutNaOFgj3j1MNLymIQke3P6/Du6glZrcci3m qYjmt8Aj2fc0wbUx7fQT6SVJkUdxg6sfhrVGaHK125EsJlsR3Z3yMZV2OOIzed0z6Q+hEv/EcZ/ VJxQb0G633sNLvTGspRVupIs3k8AggNXZzVcKR/GKYP3cjiwijoHuK43CgCNBbcG+/BV2w1CfwN ofO/WaR/ogsnjzEsPkq7PU743MODGJu0Z8+DkcoiKYkPxvaezdl8eey10u8J4eyA9F7OWgNDJJW C6bzxIZeox9EkpcWq21ZqPNLKuTh8NZLWP9s X-Google-Smtp-Source: AGHT+IGq0OxvT7/cZlFfZbbX0H/MkEPa6FnOUXsHEgn9uErHuYCGdCabGyfw1Gq2rovjVGQpSBjALQ== X-Received: by 2002:a05:6512:130b:b0:545:3032:91fd with SMTP id 2adb3069b0e04-548510cf943mr985679e87.16.1740477301678; Tue, 25 Feb 2025 01:55:01 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id 2adb3069b0e04-548514b261esm132867e87.24.2025.02.25.01.55.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 25 Feb 2025 01:55:01 -0800 (PST) From: Linus Walleij Date: Tue, 25 Feb 2025 10:54:49 +0100 Subject: [PATCH v5 02/31] ARM: ptrace: Split report_syscall() MIME-Version: 1.0 Message-Id: <20250225-arm-generic-entry-v5-2-2f02313653e5@linaro.org> References: <20250225-arm-generic-entry-v5-0-2f02313653e5@linaro.org> In-Reply-To: <20250225-arm-generic-entry-v5-0-2f02313653e5@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250225_015503_656566_4B417441 X-CRM114-Status: GOOD ( 14.26 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The generic entry code requires that report_syscall() be provided in two parts: report_syscall_enter() and report_syscall_exit() so split the combined function in two. Signed-off-by: Linus Walleij --- arch/arm/kernel/ptrace.c | 30 ++++++++++++++++++++++-------- 1 file changed, 22 insertions(+), 8 deletions(-) diff --git a/arch/arm/kernel/ptrace.c b/arch/arm/kernel/ptrace.c index c421a899fc84c476a672cbfe1c8ece8355d8512d..07b0daf47441f1f76a8af416acc74fa5ed770403 100644 --- a/arch/arm/kernel/ptrace.c +++ b/arch/arm/kernel/ptrace.c @@ -827,31 +827,45 @@ enum ptrace_syscall_dir { PTRACE_SYSCALL_EXIT, }; -static void report_syscall(struct pt_regs *regs, enum ptrace_syscall_dir dir) +static void report_syscall_enter(struct pt_regs *regs) { unsigned long ip; /* * IP is used to denote syscall entry/exit: - * IP = 0 -> entry, =1 -> exit + * IP = 0 -> entry */ ip = regs->ARM_ip; - regs->ARM_ip = dir; + regs->ARM_ip = PTRACE_SYSCALL_ENTER; - if (dir == PTRACE_SYSCALL_EXIT) - ptrace_report_syscall_exit(regs, 0); - else if (ptrace_report_syscall_entry(regs)) + if (ptrace_report_syscall_entry(regs)) current_thread_info()->abi_syscall = -1; regs->ARM_ip = ip; } +static void report_syscall_exit(struct pt_regs *regs) +{ + unsigned long ip; + + /* + * IP is used to denote syscall entry/exit: + * IP = 1 -> exit + */ + ip = regs->ARM_ip; + regs->ARM_ip = PTRACE_SYSCALL_EXIT; + + ptrace_report_syscall_exit(regs, 0); + + regs->ARM_ip = ip; +} + asmlinkage int syscall_trace_enter(struct pt_regs *regs) { int scno; if (test_thread_flag(TIF_SYSCALL_TRACE)) - report_syscall(regs, PTRACE_SYSCALL_ENTER); + report_syscall_enter(regs); /* Do seccomp after ptrace; syscall may have changed. */ #ifdef CONFIG_HAVE_ARCH_SECCOMP_FILTER @@ -892,5 +906,5 @@ asmlinkage void syscall_trace_exit(struct pt_regs *regs) trace_sys_exit(regs, regs_return_value(regs)); if (test_thread_flag(TIF_SYSCALL_TRACE)) - report_syscall(regs, PTRACE_SYSCALL_EXIT); + report_syscall_exit(regs); } From patchwork Tue Feb 25 09:54:50 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13989723 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0D308C021B2 for ; Tue, 25 Feb 2025 10:13:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=e5mhd6DszF79mgTOhWslIzEoODnT/4x94Ab5tm+ZGKw=; b=YQilIfj1YyKsLjILJLRmoi8nsI bJnv6KmQ5Ws2RMNxo+ranXUqPMck5PfxykgO/DGw0ceWZK4TalVeJ2d4j2edUf5a31BgvidwEgVWk 9Dw4RYjJ76NCezdowJsoPdl/Kc8bgU4MQMHBWgniMowTsoKX1AvTkk4z0/yhN31yslUFOCijzWz3k uej191lFFSJ+j8yrmiwYwXZD1InAXV1B05KioTaFD70ENd3AJxpquH9GepwVyGvmBPDN9RkKqBCap Hv2NSNw8BQMUQkkaay2a1zUSHZSDQR7yIxuWJw3ALU6hs1OzoUbHGtWCMtqcpgA+Iws5qIgD7smJ8 cUF/qziA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tmrvz-0000000Gm3g-1a73; Tue, 25 Feb 2025 10:13:03 +0000 Received: from mail-lf1-x12d.google.com ([2a00:1450:4864:20::12d]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tmrea-0000000GhbK-2WRF for linux-arm-kernel@lists.infradead.org; Tue, 25 Feb 2025 09:55:05 +0000 Received: by mail-lf1-x12d.google.com with SMTP id 2adb3069b0e04-54529e15643so5866981e87.1 for ; Tue, 25 Feb 2025 01:55:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1740477303; x=1741082103; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=e5mhd6DszF79mgTOhWslIzEoODnT/4x94Ab5tm+ZGKw=; b=TCxKM0GECgptn6pj1ir1RnfoAGqfKbc6lkjG7euLKFC4rbxG21tH8kUnkA2AuCN7+V rg9+ELSONBrqhumFn5mMBB8Y+Q+QC5TcmulSZ85RA3qX2YpvRxVo4eEBJHmeTxoc+E+1 hv8ijvaGP0lvW4Mn+ckcviW6H52uN27XSO7RihaGRjKlAyyc5OFi93i7U3g2FZ2XlPMU oiSLmU2tkrSD8gT0U0E5kLRONrELoiuVXucga4p0pM5If2wsylZxpgP05rdQDQNmiQ3z ttJD6y/1+NvXLZKW2iF7+oQTkK2kpWypQ9oFYHq5sEOKhs8i6BmioG2maqL9sTLCgiyd GcgA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740477303; x=1741082103; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=e5mhd6DszF79mgTOhWslIzEoODnT/4x94Ab5tm+ZGKw=; b=KZpq/yOChTf/Ju4WAwYAyvgT61cD+y0v6e05ueSC8dzOKbPTQde1iK9MfysdN//TaC z0fvBstFAYzfAhQNao4Rbx05avzeHHGMttItTXyPOgpmZ+YIhx5U0wYYJDXDLNAnG0YF 1NvvBQxUAtX5eKT5a7Fzsb9g5ZeD862Jk6gydObOjRS2fo57To9+Q2hPplWHAdK/FgRc h85KZQy1LRLowhOSkgeNosSBpqiIvmAQo6lRb+aKSowMPw0siMFok6cgg5eWdiLMJ25s uHCCtFXAvlutDvOVaHvWXGYFDXetnYcyGXzoNk2+9IUY7/Nbvr88bWOJEZvSpWC1nhi+ mA3g== X-Gm-Message-State: AOJu0YxoFiHk17e9hZ96JTrg8ZoSK1S7lR5wPjXR6G8o+yA+V09mSn8u 7/2JJSiUzHBJqYDvmIDP08Xr+U8UL1F2cyolQjsbbj0J0NJWu0mDSFzQQ4ubC34= X-Gm-Gg: ASbGncvf8XmCaQdTTOQq7NA9EiJESRM79pIj8B67DCcGlgRtyN+3n1S5qqssfHqYqTw wjpNLV7PakXYnZky8UxokjjIERJxUsFO7wb+7Jmf1zSCnn0tqayVIQ0MhFJ4pTje8LtlJScgmm+ UjoQTOffCKKLQmDn3GYO79eijHyTas3/1WH9ptxRDi319o4Qs41p+qeh1WwC0getljvwJGdR/Kr 75B9icD8tckvZ541SxH/aznZUhZlKQ7SS46qAt+bn68BCItC78614otma0LBNzqT7m929MwWq85 a5aXpQgHVLphiDuMFVFyHw2quRIo1z6eoxaP X-Google-Smtp-Source: AGHT+IHxMI3/M9aEsI7YosRrRsIEAAGbkbNTPpvA+15Mry08Go3W+i3rmuxeIPJbQpwMTvgT3L7elQ== X-Received: by 2002:a05:6512:308f:b0:546:1d8c:60f1 with SMTP id 2adb3069b0e04-546e4662ce7mr8313980e87.15.1740477302668; Tue, 25 Feb 2025 01:55:02 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id 2adb3069b0e04-548514b261esm132867e87.24.2025.02.25.01.55.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 25 Feb 2025 01:55:02 -0800 (PST) From: Linus Walleij Date: Tue, 25 Feb 2025 10:54:50 +0100 Subject: [PATCH v5 03/31] ARM: entry: Skip ret_slow_syscall label MIME-Version: 1.0 Message-Id: <20250225-arm-generic-entry-v5-3-2f02313653e5@linaro.org> References: <20250225-arm-generic-entry-v5-0-2f02313653e5@linaro.org> In-Reply-To: <20250225-arm-generic-entry-v5-0-2f02313653e5@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250225_015504_660368_3D13BD1E X-CRM114-Status: GOOD ( 11.09 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Don't introduce this extra label anymore now that we have a clearly defined entry point for ret_to_user. Signed-off-by: Linus Walleij --- arch/arm/kernel/entry-common.S | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/arch/arm/kernel/entry-common.S b/arch/arm/kernel/entry-common.S index f379c852dcb72fa5e960bba5621cc2152f0df0d3..2f93c2f6d5b8fdf6aaf3dda0192bde144fcf238d 100644 --- a/arch/arm/kernel/entry-common.S +++ b/arch/arm/kernel/entry-common.S @@ -102,7 +102,6 @@ ENDPROC(ret_fast_syscall) * do_work_pending() will update this state if necessary. */ ENTRY(ret_to_user) -ret_slow_syscall: #if IS_ENABLED(CONFIG_DEBUG_RSEQ) /* do_rseq_syscall needs interrupts enabled. */ enable_irq_notrace @ enable interrupts @@ -136,7 +135,7 @@ ENTRY(ret_from_fork) badrne lr, 1f retne r5 1: get_thread_info tsk - b ret_slow_syscall + b ret_to_user ENDPROC(ret_from_fork) /*============================================================================= @@ -310,13 +309,13 @@ __sys_trace_return_nosave: enable_irq_notrace mov r0, sp bl syscall_trace_exit - b ret_slow_syscall + b ret_to_user __sys_trace_return: str r0, [sp, #S_R0 + S_OFF]! @ save returned r0 mov r0, sp bl syscall_trace_exit - b ret_slow_syscall + b ret_to_user .macro syscall_table_start, sym .equ __sys_nr, 0 From patchwork Tue Feb 25 09:54:51 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13989724 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id BEAA7C021B6 for ; Tue, 25 Feb 2025 10:14:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=7LU1Mx0M1JJLQ6X7nOndn1GNJ7UBsOKiHS90jS7YqHQ=; b=K+31ufQo/uDlhKSZfA8giFLjCv 7z7eSkKgSG+IeeKM/ZNZgfu6yDJyscEz+vcwXFmQa7PEEWdRw2Rf5M9S4J85lc7Afv8Wt8VNa66E/ 05yT5os3KAb7PfMIkgWzCX1DDf4k09umDIsQh8JMJSAGcAsfH7gYV5ONNpeunu8B986N963q5ngWF S11/viHmEq3J6l/qLEnTJ+6eahwK2O6/xGjHR3tncK8IN4EWsjE9e5F2tLTXB0y4OYPd580lRzE5g 3PlsQLHl3OKBl9A4xgXwggprbcAX+ob3gwS8kDAfrPHHAQGSKfw5GcDpvdDarvwQhrm4JGs814tgc ol/lYamg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tmrxV-0000000GmQG-0DvR; Tue, 25 Feb 2025 10:14:37 +0000 Received: from mail-lf1-x12d.google.com ([2a00:1450:4864:20::12d]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tmreb-0000000Ghc6-3dmh for linux-arm-kernel@lists.infradead.org; Tue, 25 Feb 2025 09:55:07 +0000 Received: by mail-lf1-x12d.google.com with SMTP id 2adb3069b0e04-54524740032so5581490e87.3 for ; Tue, 25 Feb 2025 01:55:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1740477304; x=1741082104; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=7LU1Mx0M1JJLQ6X7nOndn1GNJ7UBsOKiHS90jS7YqHQ=; b=KtlNRyHq5d1xLgm80kUk9UNutJCYczAM6bS7MG02bJlcoLAqq1WhOr3aUsU05BW1FN Hg1ys9/rHfpjmTjJR4RzlhxSwdTvAJ7zPQBKvjq3e1+KEvlcSUZlVqpna6fbB4WJ0K/l EKjto9OXawVGfyFaikMHQMD5EoSgdF/r+RwhgqDXa9ISwo3HUV9BAld9cSYjxiPyQF+L xvVZI9NsQv/V56Zdt56iWMsg0Zxxy8yqwUCoipUyOk4+Q4N1LEvkbkeIiYK+xrAmCyQp mg2x7mSvRBecrWEqvxHAn5b9KTNyvDCn2S+nHBfuAkQCawFrywo3NdZKCQGTGHeZYKDU lSAw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740477304; x=1741082104; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=7LU1Mx0M1JJLQ6X7nOndn1GNJ7UBsOKiHS90jS7YqHQ=; b=bo3YL8/r6M5TAtkpByKWwYoCjpv1JzZi9XJVYrqpnJ0xbopIns5dEC5G2t2OL0m4od 7avUn4JAIF+jIEd26aewaF1Q7BRgxcLAtuqxOymUfVcIoLe5qN4ntDN/2n2wnp1qslno 9y9WOG3TF57RBBrZ9Hyd8eOk2uwvzYkwZd0YP7plk1QGMQNe5B/pqCG0wPPqRrOkjVqO 5bDCf9z83Wct5BAVNZuK8NXgN+7XwjAeQB16NponvzP2MK+k/5t3JvYlDF7jw7Psez2I MGCCeM2LuF8MNSAV0+bqNmXN4tK4Jl4/iLTGkKJJz6CWW3i2FbBfW6SSU90yaRUEh1eb vjrA== X-Gm-Message-State: AOJu0YxBWcZF2RqzNELaXtPrievzD+TNrfsfX1i6/yGw0y1c7sfCojNg mEeCqZYp3UTmWBe8QcWyU8CN/nfGd9UKP4F3WRfhv+bqCkzQjtL2EMpyMUFuSIQ= X-Gm-Gg: ASbGnctp5GHLyqy6wrhukrurjEG/whXmgVUy5Ec75lbvLQbPhw54KXN1j2DlEJLgxUM 3Z/5jV3cgw/kVUY2xO/3Lpb4ACxuNBoqgvoCJzExpbYT78gfBN4poYsOZP6rUnmE/N7KABG2hVG EXNaD3iyPeyWpLPefLUX2nr1FZ0mEJ7gKNXC8Jra/Kyfowz9XOA6PV5sRJV9zDWS6FvFU3ZIYNm j08DOLdisk2rRpvT3QOPNVMxguHgWNU5GkNLl/ABnDmkwLnbzaY1Wa1Z58Mm5VuEVIiExbg5c5I mAfshpOblW90sk5dLSsaK2u6Mu+k1LQq7Kpj X-Google-Smtp-Source: AGHT+IFtzOWDVP2t0fNNEuMAk2DHjCVRvoPLmi06C6RBehYosOgRfNKzND5MSNgjwgco26/R488VhA== X-Received: by 2002:a05:6512:104e:b0:546:2ff9:1542 with SMTP id 2adb3069b0e04-548510ef882mr775478e87.53.1740477304323; Tue, 25 Feb 2025 01:55:04 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id 2adb3069b0e04-548514b261esm132867e87.24.2025.02.25.01.55.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 25 Feb 2025 01:55:03 -0800 (PST) From: Linus Walleij Date: Tue, 25 Feb 2025 10:54:51 +0100 Subject: [PATCH v5 04/31] ARM: process: Rewrite ret_from_fork i C MIME-Version: 1.0 Message-Id: <20250225-arm-generic-entry-v5-4-2f02313653e5@linaro.org> References: <20250225-arm-generic-entry-v5-0-2f02313653e5@linaro.org> In-Reply-To: <20250225-arm-generic-entry-v5-0-2f02313653e5@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250225_015505_928508_CB7419F2 X-CRM114-Status: GOOD ( 19.11 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Move the direct assembly call to schedule_tail() and the call starting a kernel thread into a C version of ret_from_fork() in process.c and rename the old ret_from_fork to ret_from_fork_asm following the pattern in x86. Leave a comment on what we will do when we get to switching to generic entry with this as a base. Signed-off-by: Linus Walleij --- arch/arm/include/asm/switch_to.h | 4 ++++ arch/arm/kernel/entry-common.S | 22 +++++++++++++++------- arch/arm/kernel/process.c | 24 ++++++++++++++++++++++-- 3 files changed, 41 insertions(+), 9 deletions(-) diff --git a/arch/arm/include/asm/switch_to.h b/arch/arm/include/asm/switch_to.h index 9372348516ce9b80fa713966943d0bc622e86066..18d084aff5a4faa7cc5e40d48fbc4aa97a89b0c6 100644 --- a/arch/arm/include/asm/switch_to.h +++ b/arch/arm/include/asm/switch_to.h @@ -17,6 +17,10 @@ #define __complete_pending_tlbi() #endif +asmlinkage void ret_from_fork_asm(void); +__visible void ret_from_fork(struct task_struct *prev, struct pt_regs *regs, + int (*fn)(void *), void *fn_arg); + /* * switch_to(prev, next) should switch from task `prev' to `next' * `prev' will never be the same as `next'. schedule() itself diff --git a/arch/arm/kernel/entry-common.S b/arch/arm/kernel/entry-common.S index 2f93c2f6d5b8fdf6aaf3dda0192bde144fcf238d..df6961a1006b7a0f3522728812bf52ccb849e511 100644 --- a/arch/arm/kernel/entry-common.S +++ b/arch/arm/kernel/entry-common.S @@ -5,6 +5,7 @@ * Copyright (C) 2000 Russell King */ +#include #include #include #include @@ -128,15 +129,22 @@ ENDPROC(ret_to_user) /* * This is how we return from a fork. */ -ENTRY(ret_from_fork) - bl schedule_tail - cmp r5, #0 - movne r0, r4 - badrne lr, 1f - retne r5 +SYM_TYPED_FUNC_START(ret_from_fork_asm) + /* + * ret_from_fork() expects: + * r0: prev + * r1: regs + * r2: fn() for kernel process or NULL + * r3: fn_arg for kernel process or N/A + */ + mov r1, sp + mov r2, r5 + mov r3, r4 + bl ret_from_fork + mov r0, #0 1: get_thread_info tsk b ret_to_user -ENDPROC(ret_from_fork) +SYM_FUNC_END(ret_from_fork_asm) /*============================================================================= * SWI handler diff --git a/arch/arm/kernel/process.c b/arch/arm/kernel/process.c index e16ed102960cb01d625e2ccc484f238b824b03b3..81c1c3f988344185917cf4e53d0d8ee47ae912f4 100644 --- a/arch/arm/kernel/process.c +++ b/arch/arm/kernel/process.c @@ -29,6 +29,7 @@ #include #include #include +#include #include #include #include @@ -230,7 +231,26 @@ void flush_thread(void) thread_notify(THREAD_NOTIFY_FLUSH, thread); } -asmlinkage void ret_from_fork(void) __asm__("ret_from_fork"); +/* The assembly stub sets up the parameters and calls ret_from_fork() */ +asmlinkage void ret_from_fork_asm(void) __asm__("ret_from_fork_asm"); + +__visible void ret_from_fork(struct task_struct *prev, struct pt_regs *regs, + int (*fn)(void *), void *fn_arg) +{ + schedule_tail(prev); + + /* Is this a kernel thread? */ + if (unlikely(fn)) { + fn(fn_arg); + /* + * A kernel thread is allowed to return here after successfully + * calling kernel_execve(). Exit to userspace to complete the + * execve() syscall. + */ + } + + /* syscall_exit_to_user_mode(regs); here once we switch to generic entry */ +} int copy_thread(struct task_struct *p, const struct kernel_clone_args *args) { @@ -263,7 +283,7 @@ int copy_thread(struct task_struct *p, const struct kernel_clone_args *args) thread->cpu_context.r5 = (unsigned long)args->fn; childregs->ARM_cpsr = SVC_MODE; } - thread->cpu_context.pc = (unsigned long)ret_from_fork; + thread->cpu_context.pc = (unsigned long)ret_from_fork_asm; thread->cpu_context.sp = (unsigned long)childregs; clear_ptrace_hw_breakpoint(p); From patchwork Tue Feb 25 09:54:52 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13989739 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6DC06C021B6 for ; Tue, 25 Feb 2025 10:16:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=tokJWXMpPS52pGnN85+C7ykU4Uc3aikcglEb93YpSqg=; b=G0CWpN1eLt+lgsl7Vplj6isxhG EMtgpNYOEGD1LhCeA5Q6L1Kvj1FUOeP5z/SBRBxWxqO2XB2YH0Y7PHadPln70/YljvLRn51tn/W5v 7TpoudRycC6UwFZ2GgKqxVNZTvpsGT5vu+u7xcPdB7gJUv1opER9Ihcuvlk1nwKszTX8sxRG4UN2D IYTj5klYL+uq2MZrm78XKCA+Hz9v2/1NIc/cHRFm9WyY+AuAdMBpXzWCv2r3DmUN/tdAK0vVD9JG/ OJAmETJbGdZ/at8zlPGxSP0eTPvoxpjVWckUdsPT+6HGiofrtJK4hZ3FIkzW2ODq/VoZLfQdezvKd OPPa/jww==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tmrz2-0000000Gmmc-3Oyh; Tue, 25 Feb 2025 10:16:12 +0000 Received: from mail-lf1-x130.google.com ([2a00:1450:4864:20::130]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tmred-0000000GhcQ-1BmV for linux-arm-kernel@lists.infradead.org; Tue, 25 Feb 2025 09:55:08 +0000 Received: by mail-lf1-x130.google.com with SMTP id 2adb3069b0e04-54843052bcdso2642080e87.1 for ; Tue, 25 Feb 2025 01:55:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1740477305; x=1741082105; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=tokJWXMpPS52pGnN85+C7ykU4Uc3aikcglEb93YpSqg=; b=egCfBPJ5V15MjtHSddryBUV0pmm+w4OgZ9z8+T1lXpe+hWz7n20mVkIdo83UXNinPk sU7Xm25E5LDMTRLbOGw2m4GuLAWHxlZhJCOmQTLAKPg1zP1yBBqoL65Zp4la0wqmiu8S QFeBHUL7hsPmEjfi9taKxqRiVAzz7JIJXHOZV5GRvH6QmqofD3K4uLesmMNqyy7/WouJ 80yM92glOb0CGd9rD4b7rJQTkOcDspLqIg0z/R4YQoBfz1Y7Th0Nh+gjrt/0McdjyHjF a8t6MxKBGSJClv2TIQeAV96k3gDPjUpZavyRX5h2pCnf/i1F6bL/9GlFRuVHQlTo928n lrmQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740477305; x=1741082105; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=tokJWXMpPS52pGnN85+C7ykU4Uc3aikcglEb93YpSqg=; b=gXOGi7IKstGXmTuCYJTFTp97ynZDj7quFb61a7HN01kxBvEUqwPN04yBtlN/sdy5EP aUdpb5xL94su7j12Ri9EG8hwPKwa/sgfFgo+luhVNSGT1B98ECdgz9jTKMFTrHssthOZ PScQsR6CaWvRZVyo0K6OLi0tVZXFZq+OzVvBjKSvFZ8U7/sZW9Y1rOwSsNnbbgt5qmCQ n4IMCThLwwu9sCPyRex4jdazczdFbK0NBSvUvIbpO/C8GPagkJnFDTYctMisYBbbJjo/ 0abfo56HtWA0fYwxVW8ZrSLO5Y5mgh4C2u9I/ykdGn2WQyrdV+CjY8wQo4u/L3JmhkYr XO5A== X-Gm-Message-State: AOJu0YxwJdBve5kjUBeVlWjmjPihdlyHXRBhjRJxms/AmAB6iHUHTQxV yvMf6jykOUAnqCAit4MlSpYlaSfTo2KHlYMUzA2TDkhw9uZWh0f0znKJnAgd2Fs= X-Gm-Gg: ASbGncvQb1eU/B/7h+Vp1y71r1peP1GBwa6Bfz3hu/najPoRsN8p/ZmEvT+g6fW/yb9 mj0BQyubUXT1WpmxB3ave9B7ZkhIl9wsAj7m1DtCS3hdtOIKfnf9k/VQG5IV9lV6GyhZApVhW61 72rKIwE3aEwpq90DUnq2GftSNbVjcM597dOrsXL/wL26bZzXy1iOMxnlzc5CpxcDhuNaEIDmfDb iTG061nrTNc8Dxudubxr5NGuQm5xSFDxL6NmG8jCuGgwwC9LXr451M75BKIE9+VIA3NilneHr7g vcNPjwnraKW1F1LulXr3c+l31teYvgkXZ9C3 X-Google-Smtp-Source: AGHT+IGAmJN5oP6/0azJ8UrklFwsjFNaH/XCFoS/k5MzdQEera2+lJew2uD4+v03oYKKl/xsUE50dA== X-Received: by 2002:a05:6512:1305:b0:545:2b20:5b21 with SMTP id 2adb3069b0e04-54851109ba6mr924438e87.50.1740477305355; Tue, 25 Feb 2025 01:55:05 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id 2adb3069b0e04-548514b261esm132867e87.24.2025.02.25.01.55.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 25 Feb 2025 01:55:04 -0800 (PST) From: Linus Walleij Date: Tue, 25 Feb 2025 10:54:52 +0100 Subject: [PATCH v5 05/31] ARM: process: Remove local restart MIME-Version: 1.0 Message-Id: <20250225-arm-generic-entry-v5-5-2f02313653e5@linaro.org> References: <20250225-arm-generic-entry-v5-0-2f02313653e5@linaro.org> In-Reply-To: <20250225-arm-generic-entry-v5-0-2f02313653e5@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250225_015507_332554_44AAB6E0 X-CRM114-Status: GOOD ( 22.02 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The ARM kernel contains a quirk to handle syscall restarts inside the kernel without exiting to userspace. The generic entry cannot handle this. Rename do_signal() to arch_do_signal_or_restart() to fit with the upcoming generic entry conversion. This is essentially a revert of commit 81783786d5cf "ARM: 7473/1: deal with handlerless restarts without leaving the kernel" from 2012. After the conversion to generic entry later in this series, the local restart will be reimplemented again, but in C. Link: http://lists.infradead.org/pipermail/linux-arm-kernel/2012-June/104733.html Link: https://lore.kernel.org/all/1340377626-17075-1-git-send-email-will.deacon@arm.com/ Signed-off-by: Linus Walleij --- arch/arm/include/asm/signal.h | 3 +-- arch/arm/kernel/entry-common.S | 9 +-------- arch/arm/kernel/signal.c | 28 +++++++++------------------- 3 files changed, 11 insertions(+), 29 deletions(-) diff --git a/arch/arm/include/asm/signal.h b/arch/arm/include/asm/signal.h index 8b84092d151800cb4076672d1c31f0a83777d0d8..7acccc96840c8a17744cc3f2894f19d142aba4fd 100644 --- a/arch/arm/include/asm/signal.h +++ b/arch/arm/include/asm/signal.h @@ -24,7 +24,6 @@ typedef struct { #include void do_rseq_syscall(struct pt_regs *regs); -int do_work_pending(struct pt_regs *regs, unsigned int thread_flags, - int syscall); +void do_work_pending(struct pt_regs *regs, unsigned int thread_flags); #endif diff --git a/arch/arm/kernel/entry-common.S b/arch/arm/kernel/entry-common.S index df6961a1006b7a0f3522728812bf52ccb849e511..da5c2d4b62e56cce644b15e329ab4149d69c5b79 100644 --- a/arch/arm/kernel/entry-common.S +++ b/arch/arm/kernel/entry-common.S @@ -86,14 +86,8 @@ ENDPROC(ret_fast_syscall) bne __sys_trace_return_nosave slow_work_pending: mov r0, sp @ 'regs' - mov r2, why @ 'syscall' bl do_work_pending - cmp r0, #0 - beq no_work_pending - movlt scno, #(__NR_restart_syscall - __NR_SYSCALL_BASE) - str scno, [tsk, #TI_ABI_SYSCALL] @ make sure tracers see update - ldmia sp, {r0 - r6} @ have to reload r0 - r6 - b local_restart @ ... and off we go + b no_work_pending ENDPROC(ret_fast_syscall) /* @@ -266,7 +260,6 @@ ENTRY(vector_swi) */ TRACE( ldmia sp, {r0 - r3} ) -local_restart: ldr r10, [tsk, #TI_FLAGS] @ check for syscall tracing stmdb sp!, {r4, r5} @ push fifth and sixth args diff --git a/arch/arm/kernel/signal.c b/arch/arm/kernel/signal.c index 79a6730fa0eb7d6581343bdbbeb6834dcaabb9f8..7b1a16e86b236575efdc29cda9b751e8e2a3f64e 100644 --- a/arch/arm/kernel/signal.c +++ b/arch/arm/kernel/signal.c @@ -18,6 +18,7 @@ #include #include #include +#include #include #include "signal.h" @@ -534,9 +535,10 @@ static void handle_signal(struct ksignal *ksig, struct pt_regs *regs) * the kernel can handle, and then we build all the user-level signal handling * stack-frames in one go after that. */ -static int do_signal(struct pt_regs *regs, int syscall) +static void arch_do_signal_or_restart(struct pt_regs *regs) { unsigned int retval = 0, continue_addr = 0, restart_addr = 0; + bool syscall = (syscall_get_nr(current, regs) != -1); struct ksignal ksig; int restart = 0; @@ -590,16 +592,14 @@ static int do_signal(struct pt_regs *regs, int syscall) } else { /* no handler */ restore_saved_sigmask(); - if (unlikely(restart) && regs->ARM_pc == restart_addr) { + if (unlikely(restart) && regs->ARM_pc == restart_addr) regs->ARM_pc = continue_addr; - return restart; - } } - return 0; + return; } -asmlinkage int -do_work_pending(struct pt_regs *regs, unsigned int thread_flags, int syscall) +asmlinkage void +do_work_pending(struct pt_regs *regs, unsigned int thread_flags) { /* * The assembly code enters us with IRQs off, but it hasn't @@ -612,19 +612,10 @@ do_work_pending(struct pt_regs *regs, unsigned int thread_flags, int syscall) schedule(); } else { if (unlikely(!user_mode(regs))) - return 0; + return; local_irq_enable(); if (thread_flags & (_TIF_SIGPENDING | _TIF_NOTIFY_SIGNAL)) { - int restart = do_signal(regs, syscall); - if (unlikely(restart)) { - /* - * Restart without handlers. - * Deal with it without leaving - * the kernel space. - */ - return restart; - } - syscall = 0; + arch_do_signal_or_restart(regs); } else if (thread_flags & _TIF_UPROBE) { uprobe_notify_resume(regs); } else { @@ -634,7 +625,6 @@ do_work_pending(struct pt_regs *regs, unsigned int thread_flags, int syscall) local_irq_disable(); thread_flags = read_thread_flags(); } while (thread_flags & _TIF_WORK_MASK); - return 0; } struct page *get_signal_page(void) From patchwork Tue Feb 25 09:54:53 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13989740 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4F3EFC021B6 for ; Tue, 25 Feb 2025 10:17:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=H2oUY/WKIEq8Evp3ZyFIu+cgZC04yoqr/o6eiYlPpGs=; b=d8obHRqU4g+ceHvx6XD8kruaVk X1wraZd8GqDyMQnoS2w45wNxeLBt02ZGBBX4hK15aHR17FpFfDBkzFL6fprBlvL/KZgC1PPrWXdlG BYyfvVMSo8/1yLGe+KhPIr3Z+XnTy6NJDq+GIPNSTugT6EwFb+ZTzB2clFBEHJlMKtOWzeWPiXyEH xdKJO9geQzAsULJ0xRaoa4xQc9y+R7snqk+LugkYHx+3euE894cgXADOJgcMFtYk9jTHIvJwJTGDY 6InIMs7PwBUtzxYi4tqfri6E+OPJ8NHRL7dikJBGfsl6GIrr6sFrEQjYOK4GvydKsnGEp1BcotpcR CnmG/QoA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tms0Y-0000000Gn3F-23l2; Tue, 25 Feb 2025 10:17:46 +0000 Received: from mail-lf1-x132.google.com ([2a00:1450:4864:20::132]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tmred-0000000GhcY-3vQV for linux-arm-kernel@lists.infradead.org; Tue, 25 Feb 2025 09:55:09 +0000 Received: by mail-lf1-x132.google.com with SMTP id 2adb3069b0e04-54527a7270eso5408025e87.0 for ; Tue, 25 Feb 2025 01:55:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1740477306; x=1741082106; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=H2oUY/WKIEq8Evp3ZyFIu+cgZC04yoqr/o6eiYlPpGs=; b=HUKRLtUuZ97COIrNgn1OjRGqi7CS0wAHODcFTiUj74OOR3vd/FZFnwnePajA2zG0mh plqP+wNa83WiTSlXWvJvPEgBFQoUaRf9Xofh/TYYyXVBqkS5tZXGg54zlra7f625Mru/ LUkdorQMqgcgKYItevpTaxrKTwaWxaMngSJB+bsa6opWQOr09d/sgJBQVWzjTsIGCOHx TPbt1gAmuqg5lQzmq+X5RY0rGFWsbb5BjvOXRuCHADk897lVax1b18AObQ/NpKPB50d9 vCvwx8nnRPnD9+WuEF9SxH1hj5SqgKGocB5/Sx8u9DdMszcsoJ35fRMxPcVXmrZ8agwO B7Ow== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740477306; x=1741082106; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=H2oUY/WKIEq8Evp3ZyFIu+cgZC04yoqr/o6eiYlPpGs=; b=v0sM2RMNKm1tRRnMp1ySo96YINm0KWvgR3G7J2bT/vxWVIvBDD706KPdTLqZMQQZbK PNg5qeodL7fhLWOsvbVwGXLTHS97MOq1apNQZZWt9Nsx8XPS4/xAUyfGi68u+T/P5Ngx viCeYkz5ts2Z5iG4ZV0yVTgnWy32jk9Y0HqsTaJgw6zR6ffrNd9T9B/GxW/PzFOuf+u9 1BBQb6/9kXlnlsC4pVpL3Acb98OYoN4D+8jQIi3Ajj6+uM19a7TJSuWuT9RUlC0pTj1s zxuWgu+5044oUUe+Ze15Zmo/7GY+DfRN8IhNFZL45v3ktmyyf9e4uimY7Nj+YA0Cu14S navw== X-Gm-Message-State: AOJu0YypAJrsU06v4RbfVYGbsfoCKJo+lsnf3gCDzWGUkYkSDHGQsaUh sCbfJ5aIo4MR1Z3CltShKww6+S9I00XWafxi8/v0jOVpHQGV8Enrqq5g730BgVk= X-Gm-Gg: ASbGncu2NZZh3M7mlRFGqqsccg68uLasRqphd2a1wyLjtSEUWC/dTKSfJ2gPUsGYE+0 98pzHEXGGcu61bRYD5Cf1kaUKWMZPHCuS7NS3EUuvhiV/4p1U2XjXBTubO1RXWH0IrTtPghljLF IgBrLlqxig1g3VhvDXgdIeF5SSaZ6XahYGaijaFV6JCmLyR6ZBdnYHKOmSHoCzEpL0QzQwIIXXx G2wHWEknoTrJlwmzhEE1POYfP9kknJpQOV4Z7IBjKoFHOGM92Rczol1UaHgxjD/33UofWLf4uxu QGuUGUP7hQnwUWwH4pxQE3Jm1+g+ay6GeVxf X-Google-Smtp-Source: AGHT+IHAFWSNeFOCqj7itWqejwjbFtkz5zB14hTY/lexRvyjjC7gm0PhxyvvhicInIiEhLg5asuYew== X-Received: by 2002:a05:6512:b05:b0:545:2efc:7458 with SMTP id 2adb3069b0e04-54839259c52mr6699501e87.39.1740477306361; Tue, 25 Feb 2025 01:55:06 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id 2adb3069b0e04-548514b261esm132867e87.24.2025.02.25.01.55.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 25 Feb 2025 01:55:05 -0800 (PST) From: Linus Walleij Date: Tue, 25 Feb 2025 10:54:53 +0100 Subject: [PATCH v5 06/31] ARM: entry: Invoke syscalls using C MIME-Version: 1.0 Message-Id: <20250225-arm-generic-entry-v5-6-2f02313653e5@linaro.org> References: <20250225-arm-generic-entry-v5-0-2f02313653e5@linaro.org> In-Reply-To: <20250225-arm-generic-entry-v5-0-2f02313653e5@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250225_015508_009201_A442A908 X-CRM114-Status: GOOD ( 28.54 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org This makes the assembly invoking syscalls switch over to doing this from C with a small assembly stub to actually jump into the syscall. Split the syscall invocation into two paths: one for plain invocation and one for tracing (also known as "reload" as it was reloading the registers from regs). We rename this path with the infix "trace" as that code will be trace-specific as we move code over to C. Some registers such as r1 and lr get cobbled during the C calls and need to be restored when we return. Right now the part in C doesn't do much more than check the syscall number to be valid (a test previously done with a cmp r #NR_syscalls inside the invoke_syscall macro) but we will gradually factor over more assembly to C that can then be switched to the generic entry code so the exercise gets a point. Since both functions are called from a C file that is in turn only called from assembly, the functions have to be tagged with __ADDRESSABLE() in order not to upset CFI. Tested with a full system boot and by issuing some command line tools with strace to make sure the tracing path still works. Signed-off-by: Linus Walleij --- arch/arm/include/asm/syscall.h | 3 ++ arch/arm/kernel/Makefile | 3 +- arch/arm/kernel/entry-common.S | 65 ++++++++++++++++++++++++++++++++++++------ arch/arm/kernel/entry-header.S | 25 ---------------- arch/arm/kernel/syscall.c | 28 ++++++++++++++++++ 5 files changed, 90 insertions(+), 34 deletions(-) diff --git a/arch/arm/include/asm/syscall.h b/arch/arm/include/asm/syscall.h index fe4326d938c18efb54c96930e6b52ab7da9b2942..1d21f26ecf510ac00a878b4d51a5753ee49475f5 100644 --- a/arch/arm/include/asm/syscall.h +++ b/arch/arm/include/asm/syscall.h @@ -19,6 +19,9 @@ extern const unsigned long sys_call_table[]; +int invoke_syscall(void *table, struct pt_regs *regs, int scno, void *retp); +int invoke_syscall_trace(void *table, struct pt_regs *regs, int scno, void *retp); + static inline int syscall_get_nr(struct task_struct *task, struct pt_regs *regs) { diff --git a/arch/arm/kernel/Makefile b/arch/arm/kernel/Makefile index b3333d070390a8ff366a306931614b1b260647bd..69ddf51081f4791982518d37df60c11211b8955a 100644 --- a/arch/arm/kernel/Makefile +++ b/arch/arm/kernel/Makefile @@ -20,7 +20,8 @@ CFLAGS_REMOVE_return_address.o = -pg obj-y := elf.o entry-common.o irq.o opcodes.o \ process.o ptrace.o reboot.o io.o \ setup.o signal.o sigreturn_codes.o \ - stacktrace.o sys_arm.o time.o traps.o + stacktrace.o sys_arm.o time.o traps.o \ + syscall.o KASAN_SANITIZE_stacktrace.o := n KASAN_SANITIZE_traps.o := n diff --git a/arch/arm/kernel/entry-common.S b/arch/arm/kernel/entry-common.S index da5c2d4b62e56cce644b15e329ab4149d69c5b79..57aa1084a0476e9c218cd100ce4fdf4aaa748234 100644 --- a/arch/arm/kernel/entry-common.S +++ b/arch/arm/kernel/entry-common.S @@ -254,21 +254,23 @@ ENTRY(vector_swi) #else str scno, [tsk, #TI_ABI_SYSCALL] #endif - /* - * Reload the registers that may have been corrupted on entry to - * the syscall assembly (by tracing or context tracking.) - */ - TRACE( ldmia sp, {r0 - r3} ) - + mov r1, sp @ put regs into r1 ldr r10, [tsk, #TI_FLAGS] @ check for syscall tracing stmdb sp!, {r4, r5} @ push fifth and sixth args tst r10, #_TIF_SYSCALL_WORK @ are we tracing syscalls? bne __sys_trace - invoke_syscall tbl, scno, r10, __ret_fast_syscall + mov r0, tbl + /* r1 already contains regs */ + mov r2, scno @ syscall number from r7 + badr r3, __ret_fast_syscall + bl invoke_syscall + /* Restore regs into r1 and lr after C call */ + badr lr, __ret_fast_syscall add r1, sp, #S_OFF + 2: cmp scno, #(__ARM_NR_BASE - __NR_SYSCALL_BASE) eor r0, scno, #__NR_SYSCALL_BASE @ put OS number back bcs arm_syscall @@ -301,7 +303,16 @@ __sys_trace: add r0, sp, #S_OFF bl syscall_trace_enter mov scno, r0 - invoke_syscall tbl, scno, r10, __sys_trace_return, reload=1 + mov r2, r0 @ scno into r2 + add r1, sp, #S_R0 + S_OFF @ pointer to regs + mov r0, tbl + badr r3, __sys_trace_return + bl invoke_syscall_trace + + /* Restore regs into r1 and lr after C call */ + add r1, sp, #S_R0 + S_OFF @ pointer to regs + badr lr, __sys_trace_return + cmp scno, #-1 @ skip the syscall? bne 2b add sp, sp, #S_OFF @ restore stack @@ -415,6 +426,44 @@ sys_mmap2: b sys_mmap_pgoff ENDPROC(sys_mmap2) +/* + * This call wants: + * r0: syscall table + * r1: regs + * r2: syscall number + * r3: pointer to return function + */ +SYM_TYPED_FUNC_START(invoke_syscall_asm) +#ifdef CONFIG_CPU_SPECTRE + csdb +#endif + mov tbl, r0 + mov scno, r2 + mov lr, r3 @ return address + ldmia r1, {r0 - r3} @ reload r0-r3 + /* Arguments 5 and 6 are (hopefully) on the stack */ + ldr pc, [tbl, scno, lsl #2] @ call sys_* routine +SYM_FUNC_END(invoke_syscall_asm) + +/* + * This call wants: + * r0: syscall table + * r1: regs + * r2: syscall number + * r3: pointer to return function + */ +SYM_TYPED_FUNC_START(invoke_syscall_trace_asm) +#ifdef CONFIG_CPU_SPECTRE + csdb +#endif + mov tbl, r0 + mov scno, r2 + mov lr, r3 @ return address + ldmia r1, {r0 - r6} @ reload r0-r6 + stmia sp, {r4, r5} @ update stack arguments + ldr pc, [tbl, scno, lsl #2] @ call sys_* routine +SYM_FUNC_END(invoke_syscall_trace_asm) + #ifdef CONFIG_OABI_COMPAT /* diff --git a/arch/arm/kernel/entry-header.S b/arch/arm/kernel/entry-header.S index 99411fa913501cad8b55ef04a2b8ab3d44f3e39e..52b4fa97226dbfa4b55aca8eaf74ae24e1e220f8 100644 --- a/arch/arm/kernel/entry-header.S +++ b/arch/arm/kernel/entry-header.S @@ -389,31 +389,6 @@ ALT_UP_B(.L1_\@) #endif .endm - .macro invoke_syscall, table, nr, tmp, ret, reload=0 -#ifdef CONFIG_CPU_SPECTRE - mov \tmp, \nr - cmp \tmp, #NR_syscalls @ check upper syscall limit - movcs \tmp, #0 - csdb - badr lr, \ret @ return address - .if \reload - add r1, sp, #S_R0 + S_OFF @ pointer to regs - ldmiacc r1, {r0 - r6} @ reload r0-r6 - stmiacc sp, {r4, r5} @ update stack arguments - .endif - ldrcc pc, [\table, \tmp, lsl #2] @ call sys_* routine -#else - cmp \nr, #NR_syscalls @ check upper syscall limit - badr lr, \ret @ return address - .if \reload - add r1, sp, #S_R0 + S_OFF @ pointer to regs - ldmiacc r1, {r0 - r6} @ reload r0-r6 - stmiacc sp, {r4, r5} @ update stack arguments - .endif - ldrcc pc, [\table, \nr, lsl #2] @ call sys_* routine -#endif - .endm - /* * These are the registers used in the syscall handler, and allow us to * have in theory up to 7 arguments to a function - r0 to r6. diff --git a/arch/arm/kernel/syscall.c b/arch/arm/kernel/syscall.c new file mode 100644 index 0000000000000000000000000000000000000000..edc0ac88ec60ce3f23149a526de7dc1205906552 --- /dev/null +++ b/arch/arm/kernel/syscall.c @@ -0,0 +1,28 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include +#include + +int invoke_syscall_asm(void *table, struct pt_regs *regs, int scno, void *retp); +__ADDRESSABLE(invoke_syscall_asm); + +__visible int invoke_syscall(void *table, struct pt_regs *regs, int scno, void *retp) +{ + if (scno < NR_syscalls) + /* Doing this with return makes sure the stack gets pop:ed */ + return invoke_syscall_asm(table, regs, scno, retp); + + return 0; +} + +int invoke_syscall_trace_asm(void *table, struct pt_regs *regs, int scno, void *retp); +__ADDRESSABLE(invoke_syscall_trace_asm); + +__visible int invoke_syscall_trace(void *table, struct pt_regs *regs, int scno, void *retp) +{ + if (scno < NR_syscalls) + /* Doing this with return makes sure the stack gets pop:ed */ + return invoke_syscall_trace_asm(table, regs, scno, retp); + + return 0; +} From patchwork Tue Feb 25 09:54:54 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13989741 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 172E1C021B6 for ; Tue, 25 Feb 2025 10:19:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=yplTGeg8q5Jb1IHX+lZzUxdHnSw+Ki6MseFOXEsgMb4=; b=tfvbc4kJAT7BF/FwwgRsvlCArX lklyMuACGCmBhsLdc/a7QaeN05VgbiA+byesZoB14hyL2N2bI0X8EGSTuGCmP+hcQIE6kWfaVKC7Q lAo2ToTxOn9zjV5U+IL1o9SSEpK1Br22Yo7CjhLl4szXWL+JtgxnaGkHSoWrE8/Oup1zhEgM68Fn0 gMaxOu7fL8BNN6fFz02okVMcUTPI9Tp6myoz7UFQ13KUWg1aaruMt1tjIw57LOKim6dkUTThQ1BHl piBEhQa5FPqcUq5E40+KO789FbnFxBHZmNmLk+RX2nmGgNd15Umb9SLdwsgUxDtt/0Mn4MGwiJWj0 3xH2zd5A==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tms25-0000000GnPk-1Uo4; Tue, 25 Feb 2025 10:19:21 +0000 Received: from mail-lf1-x12e.google.com ([2a00:1450:4864:20::12e]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tmree-0000000Ghd4-3vpz for linux-arm-kernel@lists.infradead.org; Tue, 25 Feb 2025 09:55:10 +0000 Received: by mail-lf1-x12e.google.com with SMTP id 2adb3069b0e04-5452e6f2999so5747078e87.3 for ; Tue, 25 Feb 2025 01:55:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1740477307; x=1741082107; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=yplTGeg8q5Jb1IHX+lZzUxdHnSw+Ki6MseFOXEsgMb4=; b=XVKL3hkNlLU4F2XxrUuciZlkuJSt8Axm6vV7zlz93raptb2gfeA5iDlxD3T/dtCZ+A qZU9CSzIviPF0u6/qixkvhAm2qVECiYE6kj/JE+ik9BBdhSh4s7Z4EgiJyszHmR1dOc+ ipsEHCQSX2P93/nkImoCdcQ0Ha8cx76nwJnn+EGHn52puttu5ta5BCqzulnMcLW0qSbo 7ASPM6BrM6GQo7Q0RUHO6IoJew0tCjUpnk/gf/LuN1W+W4RE3q5JjqKfu4+a2oR4lbUV jAk7e62ufs7AnmZXl2NNsgGvNgOg+iXZEmCOICjdAsKZ/Gd1VbL5gk6Sq/D/2IWOPnOM 8FfA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740477307; x=1741082107; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=yplTGeg8q5Jb1IHX+lZzUxdHnSw+Ki6MseFOXEsgMb4=; b=cag0VJpbAMtgVrgXL15+H5eV26zmoRTH/porMHzZzXwGkJ8rOWj78oQIyHzfNtTOAL sO01qOtCJa8XMaFhRAwGLY/Ij1HlLClo8IlcqZhqNhupWiTtuV7XOS2zGpSLRDEh//Dc qxx09iI3lq1NfWEkUeRZJwnHBKqauFjKlu4gMAQFq4mApPvMP6uogANNbwpe+vdUxB6g /R4tm6BFthl3NgPqC8lqaMFLl2KdynLC3hOOgsanxQTJjqzmqGtXTmS4nJRJkxRFNB9i pT2bBqQw6SDbbqiQrqGTqrvUmxRlo4CMkK+hjjFliro7MbR1JcX1fj9FLOfxbKF3wq7o sZWw== X-Gm-Message-State: AOJu0Yy1MuPliKwCKhqnRVM2v2IR2Ouq2osSDfUgcPsMtuQiS0tcGtJI JmWxzQTFzUqSbnjm/uNXXotpCJ2h4jCR8e3uhtAtgWKF4fvLArzW0fEzeSJJir0= X-Gm-Gg: ASbGnctG7fg0S7RTj0bD+76wiLNW9FyRovZGfa10uwnJqXkSZndsTw9O0dozC7pgOi/ 1EAIZ0+AsDVaSMmf2AOxx4+a6UNc7HbJBvGrDD+vGM8gqZrfdIavD4hv6hSNFX5jOE3vBQH1XNn ShrTS3dN/N4JVlIvMWSaqqfaG9sXPZjfpTwvFG0jX6v/MtF6Xl0KzpoJHog3VNGrHUvI8IBcYIW edD/JqUWP0HIK5A1Hhg3ySjuNKBe4DFr7ppeNURu1ZYABk1CWmnWz6YkmmlJb/cAPauQ8k5yCyL aycwXaOBpfPE9/hkO6UG/n/y8VMVgKigQ1QD X-Google-Smtp-Source: AGHT+IGoTxLJRfhLsWrHbxJYsDg42lv8XbjjipYTWunjh9XthEIABhj8ikVXP3Y+TyRAMRgkg54cow== X-Received: by 2002:a05:6512:3f10:b0:545:27ee:79f3 with SMTP id 2adb3069b0e04-54850e8790bmr925786e87.0.1740477307404; Tue, 25 Feb 2025 01:55:07 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id 2adb3069b0e04-548514b261esm132867e87.24.2025.02.25.01.55.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 25 Feb 2025 01:55:06 -0800 (PST) From: Linus Walleij Date: Tue, 25 Feb 2025 10:54:54 +0100 Subject: [PATCH v5 07/31] ARM: entry: Rewrite two asm calls in C MIME-Version: 1.0 Message-Id: <20250225-arm-generic-entry-v5-7-2f02313653e5@linaro.org> References: <20250225-arm-generic-entry-v5-0-2f02313653e5@linaro.org> In-Reply-To: <20250225-arm-generic-entry-v5-0-2f02313653e5@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250225_015509_005064_E2DD8351 X-CRM114-Status: GOOD ( 17.46 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The normal and trace entry code calls out to arm_syscall() and sys_ni_syscall() from assembly, but these calls can be moved over to the new C implementation. Signed-off-by: Linus Walleij --- arch/arm/include/asm/traps.h | 2 +- arch/arm/kernel/entry-common.S | 23 +++++------------------ arch/arm/kernel/syscall.c | 13 +++++++++++-- arch/arm/kernel/traps.c | 2 +- 4 files changed, 18 insertions(+), 22 deletions(-) diff --git a/arch/arm/include/asm/traps.h b/arch/arm/include/asm/traps.h index 2621b9fb9b19b064aa40a8ab4a3a89067b98a010..b888912c2450e0e8eec8139166c7292f283545c7 100644 --- a/arch/arm/include/asm/traps.h +++ b/arch/arm/include/asm/traps.h @@ -40,7 +40,7 @@ asmlinkage void dump_backtrace_stm(u32 *stack, u32 instruction, const char *logl asmlinkage void do_undefinstr(struct pt_regs *regs); asmlinkage void handle_fiq_as_nmi(struct pt_regs *regs); asmlinkage void bad_mode(struct pt_regs *regs, int reason); -asmlinkage int arm_syscall(int no, struct pt_regs *regs); +int arm_syscall(int no, struct pt_regs *regs); asmlinkage void baddataabort(int code, unsigned long instr, struct pt_regs *regs); asmlinkage void __div0(void); asmlinkage void handle_bad_stack(struct pt_regs *regs); diff --git a/arch/arm/kernel/entry-common.S b/arch/arm/kernel/entry-common.S index 57aa1084a0476e9c218cd100ce4fdf4aaa748234..77801d039b5f6562b231a6fd4979e318c7e736a6 100644 --- a/arch/arm/kernel/entry-common.S +++ b/arch/arm/kernel/entry-common.S @@ -264,18 +264,10 @@ ENTRY(vector_swi) mov r0, tbl /* r1 already contains regs */ mov r2, scno @ syscall number from r7 - badr r3, __ret_fast_syscall - bl invoke_syscall - - /* Restore regs into r1 and lr after C call */ + /* We return here no matter what, also pass this as an argument */ badr lr, __ret_fast_syscall - add r1, sp, #S_OFF - -2: cmp scno, #(__ARM_NR_BASE - __NR_SYSCALL_BASE) - eor r0, scno, #__NR_SYSCALL_BASE @ put OS number back - bcs arm_syscall - mov why, #0 @ no longer a real syscall - b sys_ni_syscall @ not private func + mov r3, lr + b invoke_syscall #if defined(CONFIG_OABI_COMPAT) || !defined(CONFIG_AEABI) /* @@ -308,13 +300,8 @@ __sys_trace: mov r0, tbl badr r3, __sys_trace_return bl invoke_syscall_trace - - /* Restore regs into r1 and lr after C call */ - add r1, sp, #S_R0 + S_OFF @ pointer to regs - badr lr, __sys_trace_return - - cmp scno, #-1 @ skip the syscall? - bne 2b + cmp r0, #-1 + bne __sys_trace_return add sp, sp, #S_OFF @ restore stack __sys_trace_return_nosave: diff --git a/arch/arm/kernel/syscall.c b/arch/arm/kernel/syscall.c index edc0ac88ec60ce3f23149a526de7dc1205906552..d637dc74b1c70415933898fbcadbedc71df5f654 100644 --- a/arch/arm/kernel/syscall.c +++ b/arch/arm/kernel/syscall.c @@ -12,7 +12,10 @@ __visible int invoke_syscall(void *table, struct pt_regs *regs, int scno, void * /* Doing this with return makes sure the stack gets pop:ed */ return invoke_syscall_asm(table, regs, scno, retp); - return 0; + if (scno >= __ARM_NR_BASE) + return arm_syscall(scno, regs); + + return sys_ni_syscall(); } int invoke_syscall_trace_asm(void *table, struct pt_regs *regs, int scno, void *retp); @@ -20,9 +23,15 @@ __ADDRESSABLE(invoke_syscall_trace_asm); __visible int invoke_syscall_trace(void *table, struct pt_regs *regs, int scno, void *retp) { + if (scno == -1) + return -1; + if (scno < NR_syscalls) /* Doing this with return makes sure the stack gets pop:ed */ return invoke_syscall_trace_asm(table, regs, scno, retp); - return 0; + if (scno >= __ARM_NR_BASE) + return arm_syscall(scno, regs); + + return sys_ni_syscall(); } diff --git a/arch/arm/kernel/traps.c b/arch/arm/kernel/traps.c index 6ea645939573fb65ed36f5435a9c94e98828f45b..942c77767919466d5b89d83d8eb77032b119473e 100644 --- a/arch/arm/kernel/traps.c +++ b/arch/arm/kernel/traps.c @@ -609,7 +609,7 @@ do_cache_op(unsigned long start, unsigned long end, int flags) * 0x9f0000 - 0x9fffff are some more esoteric system calls */ #define NR(x) ((__ARM_NR_##x) - __ARM_NR_BASE) -asmlinkage int arm_syscall(int no, struct pt_regs *regs) +int arm_syscall(int no, struct pt_regs *regs) { if ((no >> 16) != (__ARM_NR_BASE>> 16)) return bad_syscall(no, regs); From patchwork Tue Feb 25 09:54:55 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13989742 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 79C65C021B2 for ; Tue, 25 Feb 2025 10:21:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=gCv9qyhC7Wn0x41/PmaNXNh/X9H9ffqS+9z2MGX0UwA=; b=qkVT5XQBoapNW0d6MJKHqWYN44 nNc7+NXh6p0jDEtJKQ8in8aEDU7TYIOFaBwEXQrmQYBvDyM1TIq4x1rZPT0+4TEV79LbSR01wlHIn 9RnB/rkmEt/5SNsWL3/3X9YUEJNNvKnJho/NNiKnSYqOKsDHfaRlWlbz7Pe0iv8/f2HcMJybmc+SL 9B805P/HkywifacLHUgp+R6CC9uFI1VFjrtdW7ftK2Ax/KCCsNMI+2RSOETfCataDc+kTH5odRuEM nlTEW6qbOGLDE0lkxmZwBVHAqu9hvBgB1DZau40Gwr0+kSdUfI4jcoDwplQhOj3IcS24l8UBLyNDe jISilypw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tms3b-0000000GnjY-0Cd0; Tue, 25 Feb 2025 10:20:55 +0000 Received: from mail-lf1-x12d.google.com ([2a00:1450:4864:20::12d]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tmref-0000000GhdQ-3rCy for linux-arm-kernel@lists.infradead.org; Tue, 25 Feb 2025 09:55:11 +0000 Received: by mail-lf1-x12d.google.com with SMTP id 2adb3069b0e04-5461cb12e39so5193492e87.2 for ; Tue, 25 Feb 2025 01:55:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1740477308; x=1741082108; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=gCv9qyhC7Wn0x41/PmaNXNh/X9H9ffqS+9z2MGX0UwA=; b=NQHKDL5vEo21vXoO4Db/YPX09CjA2vAzyBfydrevx8XqXH2upgH76kPtlGJdHhHbmm mPfhVDFzUdX8tlO+P9pLkvgie8OZO//2g3/APrMSJbq1Ys+6HeHzTR8IsaNqLKsl0co3 UPBR1l/Go5CIqMxNiyCVClE0sWWPx5Onv2nrV5lmUQx0x5W92bn0UQlbDEVAi0Agp72S AAhtAZcQdSYY2FBQn66W/oPpNbE5PeXN210dsG5L9SqZ8/x2+oTJAF0Uk09NFnNGUt2B 48YuTNMH9fBKnGZkappI4xfTes9pnz9LfiBdYyPAHOLZ98CXBC7aLdP3PBohHdZ2J7hc YBmw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740477308; x=1741082108; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=gCv9qyhC7Wn0x41/PmaNXNh/X9H9ffqS+9z2MGX0UwA=; b=fAX+3ngZMDUEatLoX+n0j1mnyCDzvBaIADjiRRI0pmBZidaWvhLbz+Xz6iroHnuAqq FzoH8KrEWwwxl/th4bw1HBU3B3VFx5lRpm4OsN8b8wT/oCQBAvUmx7AD1ajVVgdh59/b b0jkB0glCQxSVFfrxsHJSrwzJ302YmLPLZwEB9FR1wf1GG0tQywoVZoefZMDUaC9Ga+8 F8+0WRtUcsJvTk6SN0R4rhshMwBCKbn5jXyxgY5sKMLS9eOnI7v5ClH2caNUD5eqXP78 F7MKFUkTKgYKRj+VesEG6+DXZ0S7LU2UU6m+nZ+FPwXlST+cXLepCoXeY/N0FY0xFnF4 t0dw== X-Gm-Message-State: AOJu0Yw6lJ6tDAlkTbDkJNpeQi8XdtBkuQCva35fPBROQIPIGCPvRqtV 48jsdOgpDHC1rGNVWUWGcMF7kGX0426qUDIXA+LLlMJwsYgHfNLuF0B4ltTzgAg= X-Gm-Gg: ASbGncvlwQGBMi8zgvgELYS2yKW+8ZRufRryA6NC6sYp5l9mM4KlQC8zSEMFoaQHoVl SH0QH6xVo3BHPcIj0lVMbIerrEl1nHr+iYsuJ8RGjJnWWQbhISDsVoL2fJy6V3urp0KBjJ8YBb6 Ys8kLvA8mWkak+hQhE/EwEibrmDhRghkEw5NR4eUZAAqf87BwAF+VEnac12mmvNvBT6GWi1D3v9 jfPH4410TfbzfPfNr9kRxVqLQI625GVXk4IWWpJP/Puxr9WPjCvqgeSFeCPBRDoW61wX6ZXqKbL KeYuQxVtoIHouyMdRNxcALyebLpJ9jxHvN/T X-Google-Smtp-Source: AGHT+IH/i15/+svDVlzRmeCj49N+7fLYS0nSEMx6zNrdg8fMuOZRZIVmoIl+m7ZLFVdJmllqsHEC6Q== X-Received: by 2002:a05:6512:3049:b0:545:3031:b4d7 with SMTP id 2adb3069b0e04-54838edda6fmr5696766e87.6.1740477308397; Tue, 25 Feb 2025 01:55:08 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id 2adb3069b0e04-548514b261esm132867e87.24.2025.02.25.01.55.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 25 Feb 2025 01:55:07 -0800 (PST) From: Linus Walleij Date: Tue, 25 Feb 2025 10:54:55 +0100 Subject: [PATCH v5 08/31] ARM: entry: Move trace entry to C function MIME-Version: 1.0 Message-Id: <20250225-arm-generic-entry-v5-8-2f02313653e5@linaro.org> References: <20250225-arm-generic-entry-v5-0-2f02313653e5@linaro.org> In-Reply-To: <20250225-arm-generic-entry-v5-0-2f02313653e5@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250225_015509_967381_F6ED29A9 X-CRM114-Status: GOOD ( 15.33 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org This moves over the code entering into tracing into the C syscall invocation code. As syscall_trace_enter() will provide the current syscall number from the regs we need not provide the syscall number to the invocation function any more. Signed-off-by: Linus Walleij --- arch/arm/include/asm/syscall.h | 2 +- arch/arm/kernel/entry-common.S | 6 +----- arch/arm/kernel/syscall.c | 5 ++++- 3 files changed, 6 insertions(+), 7 deletions(-) diff --git a/arch/arm/include/asm/syscall.h b/arch/arm/include/asm/syscall.h index 1d21f26ecf510ac00a878b4d51a5753ee49475f5..66067d165ba3fbbe0f840a89ae396eb1a311bacc 100644 --- a/arch/arm/include/asm/syscall.h +++ b/arch/arm/include/asm/syscall.h @@ -20,7 +20,7 @@ extern const unsigned long sys_call_table[]; int invoke_syscall(void *table, struct pt_regs *regs, int scno, void *retp); -int invoke_syscall_trace(void *table, struct pt_regs *regs, int scno, void *retp); +int invoke_syscall_trace(void *table, struct pt_regs *regs, void *retp); static inline int syscall_get_nr(struct task_struct *task, struct pt_regs *regs) diff --git a/arch/arm/kernel/entry-common.S b/arch/arm/kernel/entry-common.S index 77801d039b5f6562b231a6fd4979e318c7e736a6..3cfc6d952ff99be9c4c1be4481ac3039260e3e57 100644 --- a/arch/arm/kernel/entry-common.S +++ b/arch/arm/kernel/entry-common.S @@ -292,13 +292,9 @@ ENDPROC(vector_swi) * context switches, and waiting for our parent to respond. */ __sys_trace: - add r0, sp, #S_OFF - bl syscall_trace_enter - mov scno, r0 - mov r2, r0 @ scno into r2 add r1, sp, #S_R0 + S_OFF @ pointer to regs mov r0, tbl - badr r3, __sys_trace_return + badr r2, __sys_trace_return bl invoke_syscall_trace cmp r0, #-1 bne __sys_trace_return diff --git a/arch/arm/kernel/syscall.c b/arch/arm/kernel/syscall.c index d637dc74b1c70415933898fbcadbedc71df5f654..377e16c8c53c8e809e3f8b157a889ef115d5c043 100644 --- a/arch/arm/kernel/syscall.c +++ b/arch/arm/kernel/syscall.c @@ -21,8 +21,11 @@ __visible int invoke_syscall(void *table, struct pt_regs *regs, int scno, void * int invoke_syscall_trace_asm(void *table, struct pt_regs *regs, int scno, void *retp); __ADDRESSABLE(invoke_syscall_trace_asm); -__visible int invoke_syscall_trace(void *table, struct pt_regs *regs, int scno, void *retp) +__visible int invoke_syscall_trace(void *table, struct pt_regs *regs, void *retp) { + int scno; + + scno = syscall_trace_enter(regs); if (scno == -1) return -1; From patchwork Tue Feb 25 09:54:56 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13989752 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B1A41C021B6 for ; Tue, 25 Feb 2025 10:22:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=aUy2L+59XSB2dO0W6jnqgQpUaRtZ0LvR6wu2Q9qABCw=; b=LbEuBwPhCG4AKti26Mvobyms7u LBiUaT6iPpxLFIHWl+h0vbGzIg6Gr5IkXYW7z2ncaIw3q9309Q7kFpJOhi+R8+clVYB4MGsjDdSay +dijP4HrVlyiYxrvn9QIknuYx8hhXUIoDMtAr8MWHqS+bCrnWUv7NNSJSDtkxThy/wUrlhdWoRrwD vDyOoWpjgQuZrk0znu+4DcHKo80x5D8bnp756WwJkFsOWo/yPVI/4n6vZV/TeJL4zHTJO9EQNfNOh 1W2J8/a702HDaQmXsdIRNNMFpu5PQFKbDIgKu0XNyRN6+MEW6LJyV5rgTH53yn2h4nUpdKgQr/lIP y7dS+QYQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tms56-0000000Go4A-35Dm; Tue, 25 Feb 2025 10:22:28 +0000 Received: from mail-lj1-x22b.google.com ([2a00:1450:4864:20::22b]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tmreh-0000000Ghe9-3dte for linux-arm-kernel@lists.infradead.org; Tue, 25 Feb 2025 09:55:12 +0000 Received: by mail-lj1-x22b.google.com with SMTP id 38308e7fff4ca-30a2f240156so55037031fa.3 for ; Tue, 25 Feb 2025 01:55:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1740477310; x=1741082110; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=aUy2L+59XSB2dO0W6jnqgQpUaRtZ0LvR6wu2Q9qABCw=; b=LKsmTpONUoieVquq/zW3GpukA3VluXvHBQZO+qPZFjX/Ol7wty3vmyu3Gie6IyaA/e 3XFTO2bnU2c+2XnYgQx1kDlPzAgILeNIRVbPtbezVyQEesdN4Y+EizLnG9aN9iVdxPEd DXZMUZLt1/LU/q3/XErgBbPrMWvc74PbwlQX2DoHghMof3D3/seuwrIyY9jCQWmEFRk9 tHnUrvsCB0AIKh26nP2EbhyywfvaspczUfF9xq4Lno8GPcUFWeFJGNJTFLPgeWJIvKuP f+1WRYXQTq6U25CVYYtWa4rFGd0NA7OPDAr9qFruHejpukLlw7jdmpnjaERSgDasWtTm GzZw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740477310; x=1741082110; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=aUy2L+59XSB2dO0W6jnqgQpUaRtZ0LvR6wu2Q9qABCw=; b=J+Ys1DHauhWmhySULh3jjJWr2wulif0TgCTBeuUop0vIFQi0aXFy6uuJj4oOVCHfAQ XMGKQW1tRRWu8m1Ley2+IvzyRsbryW3H/wId44wkIL8bOcxLe81o/bjr7GtJyuGIYwZr 3GC9ObY2Mp9QsEiEK//sx2UaeAUb4aVYcEqEmpeEeHNx3isUWqAaPiL5HDUJ15E1k58L QzRr3tr3yJ499wjl5qxuqKjxDjMiz6icfRiMN1xvQNLsHMG+ZnF6XQnTQw4eiDTmtQnP qM9BPoFP+igQv1dSEGa39RwEQHCH51ph9GstWhj990SJLzboRogqWB9NOPtzhIs3DkXn 5xcA== X-Gm-Message-State: AOJu0Yx0I3XW5d6KYju40FwruWyiB9dXxO9e/WxAYB68l51HJMKB8Tt9 zheL0x2QpX/6OKc7FH4X5PkLAp6RxyQAIabcnj5rMk4xpFtuuJ0xgVEjxHELOhI= X-Gm-Gg: ASbGncvC2Qjpgg2CUGE3pq7jUr4yGUCtLR8Aol7rGv5YWBmNPhb1hHimjTG8u6vkrT5 Hbj8LD1vMc/XVOYjzRPl28pRv/8895vNO2+T4jZa9Z+H49KJFR6p2WsQGy3QEGjFupIbXkYtzLC mqXr4ZZSYQ3Ul7hjRhkOob/BqRJUOmY/D6T7OooyBDxOoWFAfZudW3w1ALcBk3q2vM7YYB/xtSz IDPr98Rhvbl3LzljataeY3Si994Be/SHFegHaPWHCe67rlA6epa7Re3WJCiP7nZ9EZeH7QmLqtt GuKFr3E+HOcFFrEjc+rCc8S7wEi5yGrCe8Ci X-Google-Smtp-Source: AGHT+IF8cIBO3SlK9WGiwosYyqGfJqHooI+lszN8vN5QA7gNtCgUPpe2l1NosqypYf0ggr1xuNTUZw== X-Received: by 2002:a05:6512:110d:b0:53e:383a:639a with SMTP id 2adb3069b0e04-54838f4e402mr7330345e87.37.1740477309978; Tue, 25 Feb 2025 01:55:09 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id 2adb3069b0e04-548514b261esm132867e87.24.2025.02.25.01.55.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 25 Feb 2025 01:55:08 -0800 (PST) From: Linus Walleij Date: Tue, 25 Feb 2025 10:54:56 +0100 Subject: [PATCH v5 09/31] ARM: entry: save the syscall sp in thread_info MIME-Version: 1.0 Message-Id: <20250225-arm-generic-entry-v5-9-2f02313653e5@linaro.org> References: <20250225-arm-generic-entry-v5-0-2f02313653e5@linaro.org> In-Reply-To: <20250225-arm-generic-entry-v5-0-2f02313653e5@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250225_015511_917347_7A0C2A7F X-CRM114-Status: GOOD ( 18.90 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org We are going to rewrite the syscall handling in C, which means that the stack used by the call code is no longer predicatably 8 bytes (for syscall arguments r4 and r5) but a varying number of bytes depending on how nested the C code is. However the current code is just assuming it can rewind the stack by adding 8 to sp if a syscall is interrupted by a sigreturn call. Solve this by storing the entry sp in the per-task struct thread_info and use that in the sigreturn wrapper instead. We already have the thread info available in the SWI entry and sigreturn is probably not so common that retrieveing a pointer to thread_info should affect anything very much. Storing this per-task in thread_info makes the solution SMP robust. Signed-off-by: Linus Walleij --- arch/arm/include/asm/thread_info.h | 1 + arch/arm/kernel/asm-offsets.c | 1 + arch/arm/kernel/entry-common.S | 8 ++++++-- 3 files changed, 8 insertions(+), 2 deletions(-) diff --git a/arch/arm/include/asm/thread_info.h b/arch/arm/include/asm/thread_info.h index 943ffcf069d29cf4a035964d20d56f7ebdd6d602..d8a45c5a10496aaf806bfeaa0353d5e8985bd6f5 100644 --- a/arch/arm/include/asm/thread_info.h +++ b/arch/arm/include/asm/thread_info.h @@ -67,6 +67,7 @@ struct thread_info { __u32 cpu_domain; /* cpu domain */ struct cpu_context_save cpu_context; /* cpu context */ __u32 abi_syscall; /* ABI type and syscall nr */ + __u32 sp_syscall; /* SP when entering syscall */ unsigned long tp_value[2]; /* TLS registers */ union fp_state fpstate __attribute__((aligned(8))); union vfp_state vfpstate; diff --git a/arch/arm/kernel/asm-offsets.c b/arch/arm/kernel/asm-offsets.c index 4853875740d0fe61c6bbc32ddd9a16fa8d1fb530..c9525cbb26b73827821aa746030e56b037f49556 100644 --- a/arch/arm/kernel/asm-offsets.c +++ b/arch/arm/kernel/asm-offsets.c @@ -49,6 +49,7 @@ int main(void) DEFINE(TI_CPU_DOMAIN, offsetof(struct thread_info, cpu_domain)); DEFINE(TI_CPU_SAVE, offsetof(struct thread_info, cpu_context)); DEFINE(TI_ABI_SYSCALL, offsetof(struct thread_info, abi_syscall)); + DEFINE(TI_SP_SYSCALL, offsetof(struct thread_info, sp_syscall)); DEFINE(TI_TP_VALUE, offsetof(struct thread_info, tp_value)); DEFINE(TI_FPSTATE, offsetof(struct thread_info, fpstate)); #ifdef CONFIG_VFP diff --git a/arch/arm/kernel/entry-common.S b/arch/arm/kernel/entry-common.S index 3cfc6d952ff99be9c4c1be4481ac3039260e3e57..8baab7f97f59c434396f30b08ddd3029c5f9c0e5 100644 --- a/arch/arm/kernel/entry-common.S +++ b/arch/arm/kernel/entry-common.S @@ -232,6 +232,8 @@ ENTRY(vector_swi) uaccess_disable tbl get_thread_info tsk + /* Save a per-task copy of SP for sigreturn */ + str sp, [tsk, #TI_SP_SYSCALL] adr tbl, sys_call_table @ load syscall table pointer @@ -377,13 +379,15 @@ sys_syscall: ENDPROC(sys_syscall) sys_sigreturn_wrapper: - add r0, sp, #S_OFF + get_thread_info tsk + ldr r0, [tsk, #TI_SP_SYSCALL] @ read back SP mov why, #0 @ prevent syscall restart handling b sys_sigreturn ENDPROC(sys_sigreturn_wrapper) sys_rt_sigreturn_wrapper: - add r0, sp, #S_OFF + get_thread_info tsk + ldr r0, [tsk, #TI_SP_SYSCALL] @ read back SP mov why, #0 @ prevent syscall restart handling b sys_rt_sigreturn ENDPROC(sys_rt_sigreturn_wrapper) From patchwork Tue Feb 25 09:54:57 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13989753 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6C3DEC021B2 for ; Tue, 25 Feb 2025 10:24:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=QZ+1V9qTPsDzOzqvLQMw/vPgzhrwZ5TSLeEMjw/aHNM=; b=tiI5parfeBdLU1dO4FWt+rgOeB 1diSE7CzZG7mTV8elDMmPzpycmrLjBDpDV+tz0rBO/McnN/DctQizqN90Gcm3FJbisEebSUwzp2R7 m4Cw29NBce6NrrilD8wPna0pMrWHsHM42dSPXJFq1n2qa6Pgd3w1OBDUZhFjrA2lpG20BBnjsBXXg IgOkERWnX0P62F82xtRqnyh1Hoq5R782BNS96PhNFAxxsXyoPOAWbTl4VSpYa5lu6dBd7JS16mpdT Rlnv8TAvAjQYbwVtLNI8aYoL0MewXQwWJxOXsDun4YMw9SgYA3L28dJY16Vioq6Rt1gtEKRIVrdrx AVMy9+Pw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tms6d-0000000GoQG-1eX1; Tue, 25 Feb 2025 10:24:03 +0000 Received: from mail-lf1-x12d.google.com ([2a00:1450:4864:20::12d]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tmrei-0000000GheR-2eIR for linux-arm-kernel@lists.infradead.org; Tue, 25 Feb 2025 09:55:14 +0000 Received: by mail-lf1-x12d.google.com with SMTP id 2adb3069b0e04-5461f2ca386so4626911e87.1 for ; Tue, 25 Feb 2025 01:55:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1740477311; x=1741082111; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=QZ+1V9qTPsDzOzqvLQMw/vPgzhrwZ5TSLeEMjw/aHNM=; b=En34w7+jh5gfhgC/tnIfutzXWAh3jHf+aA3YHb0Ck8Q65mVygbvGs2PT3rTbrcjw21 04U72dsAr1rACS/K+s7f+EXUk9Ll9gV0MNm24LCRX6fne9XrVDxEK1IaaqgE7ltnovAX +Pch4md6W+y3ovt59Gu56oW6qmKYyMvs1tUrYoh8yI61+uP/HkSFi4FyWeDfbtlR1FHn Fkf9Cqus6G3hjnyDDxJxf5Mx8X1p+BG3jsGQCiTrhhelNXxDfwbUpHEY7830DtUP75ik JN4FzzrdaKWan/8fc1U2Z/q79d/8cm4jc2alNlFQ21EbeCy3nYSTV8IFqA3J/z1qTGdu evWg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740477311; x=1741082111; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=QZ+1V9qTPsDzOzqvLQMw/vPgzhrwZ5TSLeEMjw/aHNM=; b=fXpWfnKlq2vaE70Cq/+knuH0a+dj0iatnPfOM4fNue5JlrXLl0heZbGPqMk04bSX8b 12D6B2yvJoEb9Uz0G4KAKCmP4/aYyanc4SzEj0AvOaBSECekebJ82ztJtu/r/LSJ/RYA 8GNGwryTrSUSHiYoQZHS48ZAmTWwnzps/HM6WIqV6atN2KAtjZT4H6qBlsPl+c0ilLFL NmGATjLnSwq+Obwz2fQaNDfjmh6+j6LmygdGezGFykbI82wStujEg/q3EijZTJCQp0hA 5mh4IDDE3tpgZ6tqI/+QfM+9TQdC4Zbui6ZwJz7SpI3MIjnDcYWAlXJD1Wn7btJ2gbi7 4TyA== X-Gm-Message-State: AOJu0Yw16z0hPkaPc859uHML+EZpPkUnudooiZxtskjiJnDV9+B8QnYw HFqf+mVxKMYblEPkxnod1f2Y8EFVvOSGBQRUnXJ3qp+uFvRhvwzbUUPrwXDn4qg= X-Gm-Gg: ASbGnctTVRzLdTA0jHUsQOYxSUIvWjbIw1fUASS+U45VoqVZ4yNlj/jJ9wr7fkb32Qo VEIR73SLouoAYrtM4Gw4odqHf9MW3YAQ3/C4qaPilZhParsrYXxgJn268YWMsxCW9wmc0FH1l5z bf+UAKjESal0O4iLA29RbhwKSBA+i47o0+frxCJmjR1T8YByEo3regnP+meAZ1t5qSve7+Mwovu mdb/uOfyr+k+EQEjMjyi0bz39D7o0C0yQ6/5lH+4oKRsYklP+m7xD6cZAQNBlvQzbVe6dlXqNR+ NFfJWlOP/7J2Wbwd4s93IHFYcUVFN39PF3WJ X-Google-Smtp-Source: AGHT+IFDeQ/M9ZOCtIWpnUI5fVFHuPS9quPNejIMQ+9GozsbWKhR8saTISbNhtUaShSnwrshkm2CXg== X-Received: by 2002:a19:ca14:0:b0:549:39b1:65c5 with SMTP id 2adb3069b0e04-54939b16f10mr3456e87.47.1740477311051; Tue, 25 Feb 2025 01:55:11 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id 2adb3069b0e04-548514b261esm132867e87.24.2025.02.25.01.55.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 25 Feb 2025 01:55:10 -0800 (PST) From: Linus Walleij Date: Tue, 25 Feb 2025 10:54:57 +0100 Subject: [PATCH v5 10/31] ARM: entry: move all tracing invocation to C MIME-Version: 1.0 Message-Id: <20250225-arm-generic-entry-v5-10-2f02313653e5@linaro.org> References: <20250225-arm-generic-entry-v5-0-2f02313653e5@linaro.org> In-Reply-To: <20250225-arm-generic-entry-v5-0-2f02313653e5@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250225_015512_693563_E4E8E3CC X-CRM114-Status: GOOD ( 21.32 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The traced invocation of syscall is rewritten in C, moving over also the call to syscall_trace_exit() to C. To do this we cannot have invoke_syscall_trace_asm() be a leaf call, which is the mechanism we hithereto relied on to make sure the stack is pulled back to the state we were at before we called out to the code written in C. Push the registers potentially used by C on the stack, and make a copy of the two stack-pushed syscall arguments on the top of the stack before invoking the syscall and then drop these copied arguments and pop back the registers before returning from invoke_syscall_trace_asm(). As we are now calling from and returning to C, we no longer need to pass the regs pointer around so drop it. Signed-off-by: Linus Walleij --- arch/arm/include/asm/syscall.h | 2 +- arch/arm/kernel/entry-common.S | 32 +++++++++++++++----------------- arch/arm/kernel/ptrace.c | 2 +- arch/arm/kernel/syscall.c | 30 +++++++++++++++++++++--------- 4 files changed, 38 insertions(+), 28 deletions(-) diff --git a/arch/arm/include/asm/syscall.h b/arch/arm/include/asm/syscall.h index 66067d165ba3fbbe0f840a89ae396eb1a311bacc..cb0073c4151bf70a82d15e17a95b6b6f48b245d6 100644 --- a/arch/arm/include/asm/syscall.h +++ b/arch/arm/include/asm/syscall.h @@ -20,7 +20,7 @@ extern const unsigned long sys_call_table[]; int invoke_syscall(void *table, struct pt_regs *regs, int scno, void *retp); -int invoke_syscall_trace(void *table, struct pt_regs *regs, void *retp); +void invoke_syscall_trace(void *table, struct pt_regs *regs); static inline int syscall_get_nr(struct task_struct *task, struct pt_regs *regs) diff --git a/arch/arm/kernel/entry-common.S b/arch/arm/kernel/entry-common.S index 8baab7f97f59c434396f30b08ddd3029c5f9c0e5..dbc947d301ec11bb13007cb4bb161c035ede5c10 100644 --- a/arch/arm/kernel/entry-common.S +++ b/arch/arm/kernel/entry-common.S @@ -296,22 +296,11 @@ ENDPROC(vector_swi) __sys_trace: add r1, sp, #S_R0 + S_OFF @ pointer to regs mov r0, tbl - badr r2, __sys_trace_return bl invoke_syscall_trace - cmp r0, #-1 - bne __sys_trace_return - add sp, sp, #S_OFF @ restore stack - -__sys_trace_return_nosave: - enable_irq_notrace - mov r0, sp - bl syscall_trace_exit + add sp, sp, #S_OFF @ restore stack pointer b ret_to_user -__sys_trace_return: - str r0, [sp, #S_R0 + S_OFF]! @ save returned r0 - mov r0, sp - bl syscall_trace_exit +__sys_trace_return_nosave: b ret_to_user .macro syscall_table_start, sym @@ -437,18 +426,27 @@ SYM_FUNC_END(invoke_syscall_asm) * r0: syscall table * r1: regs * r2: syscall number - * r3: pointer to return function */ SYM_TYPED_FUNC_START(invoke_syscall_trace_asm) #ifdef CONFIG_CPU_SPECTRE csdb #endif + /* Save registers because we are being called from C */ + push {r4 - r10, lr} mov tbl, r0 + /* Make space to copy the two syscall stack arguments */ + sub sp, sp, #S_OFF mov scno, r2 - mov lr, r3 @ return address - ldmia r1, {r0 - r6} @ reload r0-r6 - stmia sp, {r4, r5} @ update stack arguments + badr lr, __invoke_syscall_trace_ret @ return right here + ldmia r1, {r0 - r6} @ reload r0-r6 from regs + stmia sp, {r4, r5} @ copy stack arguments ldr pc, [tbl, scno, lsl #2] @ call sys_* routine +__invoke_syscall_trace_ret: + /* Drop the copied stack arguments */ + add sp, sp, #S_OFF + pop {r4 - r10, lr} + ARM( mov pc, lr ) + THUMB( bx lr ) SYM_FUNC_END(invoke_syscall_trace_asm) #ifdef CONFIG_OABI_COMPAT diff --git a/arch/arm/kernel/ptrace.c b/arch/arm/kernel/ptrace.c index 07b0daf47441f1f76a8af416acc74fa5ed770403..ac7b98ae47249b4a00fc1cb871bb2d309bfd8c88 100644 --- a/arch/arm/kernel/ptrace.c +++ b/arch/arm/kernel/ptrace.c @@ -888,7 +888,7 @@ asmlinkage int syscall_trace_enter(struct pt_regs *regs) return scno; } -asmlinkage void syscall_trace_exit(struct pt_regs *regs) +void syscall_trace_exit(struct pt_regs *regs) { /* * Audit the syscall before anything else, as a debugger may diff --git a/arch/arm/kernel/syscall.c b/arch/arm/kernel/syscall.c index 377e16c8c53c8e809e3f8b157a889ef115d5c043..e90f04f35b3485968a1e9046cf0889f56cca92e5 100644 --- a/arch/arm/kernel/syscall.c +++ b/arch/arm/kernel/syscall.c @@ -18,23 +18,35 @@ __visible int invoke_syscall(void *table, struct pt_regs *regs, int scno, void * return sys_ni_syscall(); } -int invoke_syscall_trace_asm(void *table, struct pt_regs *regs, int scno, void *retp); +int invoke_syscall_trace_asm(void *table, struct pt_regs *regs, int scno); __ADDRESSABLE(invoke_syscall_trace_asm); -__visible int invoke_syscall_trace(void *table, struct pt_regs *regs, void *retp) +__visible void invoke_syscall_trace(void *table, struct pt_regs *regs) { int scno; + int ret; scno = syscall_trace_enter(regs); if (scno == -1) - return -1; + goto trace_exit_nosave; - if (scno < NR_syscalls) - /* Doing this with return makes sure the stack gets pop:ed */ - return invoke_syscall_trace_asm(table, regs, scno, retp); + if (scno < NR_syscalls) { + ret = invoke_syscall_trace_asm(table, regs, scno); + goto trace_exit_save; + } - if (scno >= __ARM_NR_BASE) - return arm_syscall(scno, regs); + if (scno >= __ARM_NR_BASE) { + ret = arm_syscall(scno, regs); + goto trace_exit_save; + } - return sys_ni_syscall(); + ret = sys_ni_syscall(); + +trace_exit_save: + /* Save return value from syscall */ + regs->ARM_r0 = ret; + +trace_exit_nosave: + local_irq_enable(); + syscall_trace_exit(regs); } From patchwork Tue Feb 25 09:54:58 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13989754 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 21294C021B2 for ; Tue, 25 Feb 2025 10:25:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=fjCfgKpzfePVa0TyVy/JvnFpB7AUCXqKC35hzfV3hLA=; b=RY6sxaatw6BF7a7J7fAOSDQA8b GuYIZmU0GM6F47INQp1MeCPzpwy4XRKDpRZPzSTY4fLbbO+H7K3hZ/qoHo3dPdsbQ6AOlhCKOrUpI gK73hkhEB+Wv7bwc19HV570cIhsj+gXEAYpKn+QUqY8XhaoDlzFJy+GfN+P3xFQ7c0oAkt/gOAXiE J9Xjy/2LT5NdMK/NkELIJA2LQCHiAhNzK4y/9ORrwOLE4Z9/Mg77KuOYXvHGrtNqtlgWY8G80KtDQ w+q9tB35Ao8AGAs64Ou6nu08S9W9r03rUMIV2GaDu/NIUhG2KvWWvTN15hSp3juJwHIjSD/V7IXAD zJUTDIMw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tms8A-0000000Goly-0f7z; Tue, 25 Feb 2025 10:25:38 +0000 Received: from mail-lf1-x136.google.com ([2a00:1450:4864:20::136]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tmrek-0000000GhfA-0h5h for linux-arm-kernel@lists.infradead.org; Tue, 25 Feb 2025 09:55:15 +0000 Received: by mail-lf1-x136.google.com with SMTP id 2adb3069b0e04-54527a7270eso5408121e87.0 for ; Tue, 25 Feb 2025 01:55:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1740477312; x=1741082112; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=fjCfgKpzfePVa0TyVy/JvnFpB7AUCXqKC35hzfV3hLA=; b=fgf+BrJA0HBwA3JQe1rNpZB+KDJt19OJWjyBUu8+x1xy7K1VRVdebwsRwnhHmp5Q9X pfOjG/MgFNfsrLMwpnaFnwqv92qGtjEtdjhtXLzMfxIgSfpSz5KRQjbS4DFtsemDjlpe 2cORcLpAWs6LG7kjwliHs9mmVnHscefMeTsmy66UlPoo/DqJRoH02BUsZGjOpQR6d2hy ZFOydkZtKxcmAUMQXX+l3ceb/5O55l7gJk1/JpA5SJzkl4rjtfukIOb1J3kvG1eBM7IP NHAoqUPWqVHZpgoJHQrHzfCF8FeWB6XNzI+VXokQ+uIfO+08ihaHsQ1QGSuUkFr+7iPR 2QuA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740477312; x=1741082112; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=fjCfgKpzfePVa0TyVy/JvnFpB7AUCXqKC35hzfV3hLA=; b=ffLo1jMT4VIWxJAYzIPArv4H2XDd6l5RUPeZuUVyGeiNadj4Uup0/7U7yOHy1sCIpJ aXKxjMjJoBel5yxEYDdPk2GFRHpAbSOQAbKO8B7V11FM958cI7iuy4q/L5BTnLeg++1E um3gTrYeEiMwgXjZOcP53JGB0WP7NgpJlJ6uUuj094TGLyhPRTClFn76wdftutKqHpFr zQNM0IcJot5biqoHFlQ0fk9khlHJaUFkeBW8RIhrr6k1rnxHThvfm1QCv+90mONPsXvV j/3JXUIcggV/Czs0jnyBkX64HD/2aGviZ1S8gIQIl3YmRsQzlZUjRwamBzeTEGRa65HS 6rBg== X-Gm-Message-State: AOJu0YwhVJWjKaLIWXx+0AurBUFZWupfo8unGUXQbiqjSG1yVoAMLCQF sGD42UaL+EzpjjasgCErWEBPJ7i2oA+cFl8gk+6QQ9WZ/0nr9mogywhjMqWaC8s= X-Gm-Gg: ASbGncumHlXwRImJU62EJmCkCvfuuuoMG/PwyWcHH957gG3TIuYzBqQnJpXRQ5LPCMp e7+ajdK7BnG32+v2miWivXgh8EXwVnzklHjKkTCRrOHYCvQrC/fNq8sEKwd+j8rE40UdHxzcQxh 9NzrkeabDGl3JA6bYOONzDwuuMFUHo9IXNTnHA9xurL9SReexbk5ctjsA6rt9c2L1jM+DLHq4VN SaGSH28HYlhTHQU6NbepHPvqLF0+wVXFxqOqGHfNDmlh/aqYbfaT+dsIQkOVdea1OmWpGZ+7LL8 hQtF1s/bDadQw/XHn1sJhb6uDXtoFu7bm7qJ X-Google-Smtp-Source: AGHT+IGwYLcZB4R2TB4QO0BnMrSnOzNlEWNGxZkIDWtOs5zUcIXQhO/Jl40QyYsPgythtdK2Cm9SRw== X-Received: by 2002:a05:6512:3d88:b0:545:1dea:7b2f with SMTP id 2adb3069b0e04-548392633bcmr7441543e87.45.1740477312152; Tue, 25 Feb 2025 01:55:12 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id 2adb3069b0e04-548514b261esm132867e87.24.2025.02.25.01.55.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 25 Feb 2025 01:55:11 -0800 (PST) From: Linus Walleij Date: Tue, 25 Feb 2025 10:54:58 +0100 Subject: [PATCH v5 11/31] ARM: entry: Merge the common and trace entry code MIME-Version: 1.0 Message-Id: <20250225-arm-generic-entry-v5-11-2f02313653e5@linaro.org> References: <20250225-arm-generic-entry-v5-0-2f02313653e5@linaro.org> In-Reply-To: <20250225-arm-generic-entry-v5-0-2f02313653e5@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250225_015514_216042_04E02CFC X-CRM114-Status: GOOD ( 24.41 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The trace entry code now can handle recursive and complex calls in C using stack. Move the common code over to using that approach. We now use the ret_fast_syscall return path also when tracing, which appears to work just fine. Signed-off-by: Linus Walleij --- arch/arm/include/asm/syscall.h | 3 +-- arch/arm/kernel/entry-common.S | 61 +++++++++--------------------------------- arch/arm/kernel/syscall.c | 35 +++++++++++------------- 3 files changed, 28 insertions(+), 71 deletions(-) diff --git a/arch/arm/include/asm/syscall.h b/arch/arm/include/asm/syscall.h index cb0073c4151bf70a82d15e17a95b6b6f48b245d6..9c664d8c5718f58034f442c20a030b59faf65dd9 100644 --- a/arch/arm/include/asm/syscall.h +++ b/arch/arm/include/asm/syscall.h @@ -19,8 +19,7 @@ extern const unsigned long sys_call_table[]; -int invoke_syscall(void *table, struct pt_regs *regs, int scno, void *retp); -void invoke_syscall_trace(void *table, struct pt_regs *regs); +int invoke_syscall_trace(void *table, struct pt_regs *regs, int scno); static inline int syscall_get_nr(struct task_struct *task, struct pt_regs *regs) diff --git a/arch/arm/kernel/entry-common.S b/arch/arm/kernel/entry-common.S index dbc947d301ec11bb13007cb4bb161c035ede5c10..f0f1f8723965313b287158fb2a238908db72fcd2 100644 --- a/arch/arm/kernel/entry-common.S +++ b/arch/arm/kernel/entry-common.S @@ -39,7 +39,6 @@ saved_pc .req lr * from those features make this path too inefficient. */ ret_fast_syscall: -__ret_fast_syscall: UNWIND(.fnstart ) UNWIND(.cantunwind ) disable_irq_notrace @ disable interrupts @@ -47,13 +46,13 @@ __ret_fast_syscall: movs r1, r1, lsl #16 bne fast_work_pending - restore_user_regs fast = 1, offset = S_OFF + restore_user_regs fast = 0, offset = S_OFF UNWIND(.fnend ) ENDPROC(ret_fast_syscall) /* Ok, we need to do extra processing, enter the slow path. */ fast_work_pending: - str r0, [sp, #S_R0+S_OFF]! @ returned r0 + add sp, sp, #(S_R0 + S_OFF) /* fall through to work_pending */ #else /* @@ -63,10 +62,9 @@ fast_work_pending: * call. */ ret_fast_syscall: -__ret_fast_syscall: UNWIND(.fnstart ) UNWIND(.cantunwind ) - str r0, [sp, #S_R0 + S_OFF]! @ save returned r0 + add sp, sp, #(S_R0 + S_OFF) #if IS_ENABLED(CONFIG_DEBUG_RSEQ) /* do_rseq_syscall needs interrupts enabled. */ mov r0, sp @ 'regs' @@ -83,7 +81,9 @@ ENDPROC(ret_fast_syscall) #endif tst r1, #_TIF_SYSCALL_WORK - bne __sys_trace_return_nosave + beq slow_work_pending + b ret_to_user + slow_work_pending: mov r0, sp @ 'regs' bl do_work_pending @@ -257,19 +257,15 @@ ENTRY(vector_swi) str scno, [tsk, #TI_ABI_SYSCALL] #endif mov r1, sp @ put regs into r1 - ldr r10, [tsk, #TI_FLAGS] @ check for syscall tracing stmdb sp!, {r4, r5} @ push fifth and sixth args - - tst r10, #_TIF_SYSCALL_WORK @ are we tracing syscalls? - bne __sys_trace - mov r0, tbl - /* r1 already contains regs */ mov r2, scno @ syscall number from r7 - /* We return here no matter what, also pass this as an argument */ - badr lr, __ret_fast_syscall - mov r3, lr - b invoke_syscall + bl invoke_syscall_trace + cmp r0, #0 + beq ret_fast_syscall + /* This path taken when tracing */ + add sp, sp, #(S_R0 + S_OFF) + b ret_to_user #if defined(CONFIG_OABI_COMPAT) || !defined(CONFIG_AEABI) /* @@ -289,20 +285,6 @@ ENTRY(vector_swi) ENDPROC(vector_swi) .ltorg - /* - * This is the really slow path. We're going to be doing - * context switches, and waiting for our parent to respond. - */ -__sys_trace: - add r1, sp, #S_R0 + S_OFF @ pointer to regs - mov r0, tbl - bl invoke_syscall_trace - add sp, sp, #S_OFF @ restore stack pointer - b ret_to_user - -__sys_trace_return_nosave: - b ret_to_user - .macro syscall_table_start, sym .equ __sys_nr, 0 .type \sym, #object @@ -402,25 +384,6 @@ sys_mmap2: b sys_mmap_pgoff ENDPROC(sys_mmap2) -/* - * This call wants: - * r0: syscall table - * r1: regs - * r2: syscall number - * r3: pointer to return function - */ -SYM_TYPED_FUNC_START(invoke_syscall_asm) -#ifdef CONFIG_CPU_SPECTRE - csdb -#endif - mov tbl, r0 - mov scno, r2 - mov lr, r3 @ return address - ldmia r1, {r0 - r3} @ reload r0-r3 - /* Arguments 5 and 6 are (hopefully) on the stack */ - ldr pc, [tbl, scno, lsl #2] @ call sys_* routine -SYM_FUNC_END(invoke_syscall_asm) - /* * This call wants: * r0: syscall table diff --git a/arch/arm/kernel/syscall.c b/arch/arm/kernel/syscall.c index e90f04f35b3485968a1e9046cf0889f56cca92e5..36330023b5484399a4ca2fdb5727b410e2c74b6f 100644 --- a/arch/arm/kernel/syscall.c +++ b/arch/arm/kernel/syscall.c @@ -3,50 +3,45 @@ #include #include -int invoke_syscall_asm(void *table, struct pt_regs *regs, int scno, void *retp); -__ADDRESSABLE(invoke_syscall_asm); - -__visible int invoke_syscall(void *table, struct pt_regs *regs, int scno, void *retp) +static inline bool has_syscall_work(unsigned long flags) { - if (scno < NR_syscalls) - /* Doing this with return makes sure the stack gets pop:ed */ - return invoke_syscall_asm(table, regs, scno, retp); - - if (scno >= __ARM_NR_BASE) - return arm_syscall(scno, regs); - - return sys_ni_syscall(); + return unlikely(flags & _TIF_SYSCALL_WORK); } int invoke_syscall_trace_asm(void *table, struct pt_regs *regs, int scno); __ADDRESSABLE(invoke_syscall_trace_asm); -__visible void invoke_syscall_trace(void *table, struct pt_regs *regs) +__visible int invoke_syscall_trace(void *table, struct pt_regs *regs, int scno) { - int scno; + unsigned long flags = read_thread_flags(); int ret; - scno = syscall_trace_enter(regs); - if (scno == -1) - goto trace_exit_nosave; + if (has_syscall_work(flags)) { + scno = syscall_trace_enter(regs); + if (scno == -1) + goto trace_exit_nosave; + } if (scno < NR_syscalls) { ret = invoke_syscall_trace_asm(table, regs, scno); - goto trace_exit_save; + goto exit_save; } if (scno >= __ARM_NR_BASE) { ret = arm_syscall(scno, regs); - goto trace_exit_save; + goto exit_save; } ret = sys_ni_syscall(); -trace_exit_save: +exit_save: /* Save return value from syscall */ regs->ARM_r0 = ret; + if (!has_syscall_work(flags)) + return 0; trace_exit_nosave: local_irq_enable(); syscall_trace_exit(regs); + return 1; } From patchwork Tue Feb 25 09:54:59 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13989764 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 783F1C021B2 for ; Tue, 25 Feb 2025 10:27:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=pr2oSZFQs/khvnnGeJohJHSvChXi9NfDOygi+WZQm/M=; b=rmsnQoWD/GXOKPvz1sLIUAhtXA zCAqGDbn8ERzwKfEq7sUkzrvu+nvS44sTVWDjMMMxWD8f5UUhcCUCG+i2b598GMoFNtpIQd0D/lNT BKStV4AhBCz+pYTmGJLnM5dnCta/f9IpCoZg5LJGkMbIXZKuxAHu4/Nq718NaShRdjIWBmZzhc9kk 1zjc0Dje9fVln04Sp5JWvGw5G9yDSFGecPCCbb+x31mljN2cy7VNaCp9+pv4lPC5iUxNKMu+6kKk9 DU1+57qhQ/9mGW/ltvgDa8BvDlswkbKMQED2rACR/RFhD1a2ZiJQv1W5QUW6k+e02ojsq1VHitIxz yRi5i5vg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tms9g-0000000Gp6N-3WSO; Tue, 25 Feb 2025 10:27:12 +0000 Received: from mail-lj1-x235.google.com ([2a00:1450:4864:20::235]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tmrel-0000000Ghfu-3Dxn for linux-arm-kernel@lists.infradead.org; Tue, 25 Feb 2025 09:55:16 +0000 Received: by mail-lj1-x235.google.com with SMTP id 38308e7fff4ca-30a2dfcfd83so50428181fa.1 for ; Tue, 25 Feb 2025 01:55:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1740477314; x=1741082114; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=pr2oSZFQs/khvnnGeJohJHSvChXi9NfDOygi+WZQm/M=; b=y8JYFAsFnib4ooTd1nduSieQMzZ0+T9Dnwdo6yIFHu+xDAltKk8l4SZ7aT5iSY3CdH DdRmftgEjBU62+UnM0bmV866tRwarD8/PYJz7T3ZycvbKexrkCzM+HEagKRPV8Z1yFoQ upD5y9S11vMy7wW63AqlmRYhKzQkAoZif95jMadFUALL7Nl5MeURVVzXhLX0mijtmhqn L2O53+twM2Goiva2iBrjNsSK/GR0VjXeqWKSkQ8IcIZ72aqcRMA98qaXuz3oXPFbEjke Z8NjTS8SnOHoPsYw7/dnxEdY0Gm3FGiRt5Msxm6j9O1YrnvHEU7iwzi7iv8InE2HgQPX x68Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740477314; x=1741082114; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=pr2oSZFQs/khvnnGeJohJHSvChXi9NfDOygi+WZQm/M=; b=sdr/LoNS6FNGwY9dh3l70PAsqReItnhICRx/WN0EaZYU9vIQwAQtNgM8uq7m441C4G p0yvkEaHeZuJvBovwQBKYeI6rUP9L5nld+qqEblt2Ej9SMvpaSB7T2TZJibYDTkJdGEm QruuHbxarhd3J5LxFxUApuJjvU/gFZqgjfb10Q7A0gUbT9Y3ap9bEBQLsbAruovDcPQg gBgCv3wBXn4pL5+ixH+X+S8HqCnan8WdnrEyzzYjJuVOEPARfq8sWA8OQCBvcAhtuS0M 7FRn7j9zSx962UzDLL2xBkI35koFgm3qcn+a9P/01Zqyr+fLf/F3lznO03tv81wBSd+9 ln9Q== X-Gm-Message-State: AOJu0Yz/q6fQ5JrHafgs+XZEu81biqAHFQ++VeA4r4dGWeVNf9r4jSVZ NA9SoE0+t2r7QnVz5vjZeXJRwy71voeqvugG9d5adzSP5Me10alCGHfwtpUGO8g= X-Gm-Gg: ASbGncugYvRt/nqzDCMJbe1bWZPOcg3auJka5AOhymneJ5uuaPmuA2dk6p98V8irLyD ihZgkiqbdq1fMjlM0h7cbNlUBMJ5RLSCQDa+qkBTEAAGOVBINEKVqQoqsxibZeP+6nBeo+gsYGS 7cRzvnJVGre19phPGKHEw1aNGouh85OW46wQkA97U82rSpTYsgQ1fei8shOYJXnq7Dv40xysPuW CD7d5I/gy3aJRWY5RKnbL4HN23mIorQqXmDRS2DouQQ3etgcPUGpQ90KudL5OYNsBx2Wews3Zzl 0cmLVwSl+Zy+wxTIiA304yhMvdIXDQGbrAZf X-Google-Smtp-Source: AGHT+IFeLR30oGpW5vklNxUSsMhMgUVaKjzRtV+OjULqK8ylumNd1C7a3hr2K2xQCsmZr/q/cwPOIA== X-Received: by 2002:a05:6512:3297:b0:549:2ae5:99d8 with SMTP id 2adb3069b0e04-5492ae59d04mr117932e87.3.1740477313811; Tue, 25 Feb 2025 01:55:13 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id 2adb3069b0e04-548514b261esm132867e87.24.2025.02.25.01.55.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 25 Feb 2025 01:55:12 -0800 (PST) From: Linus Walleij Date: Tue, 25 Feb 2025 10:54:59 +0100 Subject: [PATCH v5 12/31] ARM: entry: Rename syscall invocation MIME-Version: 1.0 Message-Id: <20250225-arm-generic-entry-v5-12-2f02313653e5@linaro.org> References: <20250225-arm-generic-entry-v5-0-2f02313653e5@linaro.org> In-Reply-To: <20250225-arm-generic-entry-v5-0-2f02313653e5@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250225_015515_814664_E4F0BD41 X-CRM114-Status: GOOD ( 15.91 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Now that we do not have a separate trace syscall invocation path, rename the invocation functions to be generic. This can be squashed into the previous patch, it is just done here for the previous patch to be easier to get a cleaner diff and be easier to read. Signed-off-by: Linus Walleij --- arch/arm/include/asm/syscall.h | 2 +- arch/arm/kernel/entry-common.S | 10 +++++----- arch/arm/kernel/syscall.c | 8 ++++---- 3 files changed, 10 insertions(+), 10 deletions(-) diff --git a/arch/arm/include/asm/syscall.h b/arch/arm/include/asm/syscall.h index 9c664d8c5718f58034f442c20a030b59faf65dd9..b94fd7ce17f82f0baf06b74800245112e1f13002 100644 --- a/arch/arm/include/asm/syscall.h +++ b/arch/arm/include/asm/syscall.h @@ -19,7 +19,7 @@ extern const unsigned long sys_call_table[]; -int invoke_syscall_trace(void *table, struct pt_regs *regs, int scno); +int invoke_syscall(void *table, struct pt_regs *regs, int scno); static inline int syscall_get_nr(struct task_struct *task, struct pt_regs *regs) diff --git a/arch/arm/kernel/entry-common.S b/arch/arm/kernel/entry-common.S index f0f1f8723965313b287158fb2a238908db72fcd2..f1e48002bd3075ea67b5883178583127fa0055c6 100644 --- a/arch/arm/kernel/entry-common.S +++ b/arch/arm/kernel/entry-common.S @@ -260,7 +260,7 @@ ENTRY(vector_swi) stmdb sp!, {r4, r5} @ push fifth and sixth args mov r0, tbl mov r2, scno @ syscall number from r7 - bl invoke_syscall_trace + bl invoke_syscall cmp r0, #0 beq ret_fast_syscall /* This path taken when tracing */ @@ -390,7 +390,7 @@ ENDPROC(sys_mmap2) * r1: regs * r2: syscall number */ -SYM_TYPED_FUNC_START(invoke_syscall_trace_asm) +SYM_TYPED_FUNC_START(invoke_syscall_asm) #ifdef CONFIG_CPU_SPECTRE csdb #endif @@ -400,17 +400,17 @@ SYM_TYPED_FUNC_START(invoke_syscall_trace_asm) /* Make space to copy the two syscall stack arguments */ sub sp, sp, #S_OFF mov scno, r2 - badr lr, __invoke_syscall_trace_ret @ return right here + badr lr, __invoke_syscall_ret @ return right here ldmia r1, {r0 - r6} @ reload r0-r6 from regs stmia sp, {r4, r5} @ copy stack arguments ldr pc, [tbl, scno, lsl #2] @ call sys_* routine -__invoke_syscall_trace_ret: +__invoke_syscall_ret: /* Drop the copied stack arguments */ add sp, sp, #S_OFF pop {r4 - r10, lr} ARM( mov pc, lr ) THUMB( bx lr ) -SYM_FUNC_END(invoke_syscall_trace_asm) +SYM_FUNC_END(invoke_syscall_asm) #ifdef CONFIG_OABI_COMPAT diff --git a/arch/arm/kernel/syscall.c b/arch/arm/kernel/syscall.c index 36330023b5484399a4ca2fdb5727b410e2c74b6f..6f518ec3870fb36a64d1b671b20f96d63a52bf78 100644 --- a/arch/arm/kernel/syscall.c +++ b/arch/arm/kernel/syscall.c @@ -8,10 +8,10 @@ static inline bool has_syscall_work(unsigned long flags) return unlikely(flags & _TIF_SYSCALL_WORK); } -int invoke_syscall_trace_asm(void *table, struct pt_regs *regs, int scno); -__ADDRESSABLE(invoke_syscall_trace_asm); +int invoke_syscall_asm(void *table, struct pt_regs *regs, int scno); +__ADDRESSABLE(invoke_syscall_asm); -__visible int invoke_syscall_trace(void *table, struct pt_regs *regs, int scno) +__visible int invoke_syscall(void *table, struct pt_regs *regs, int scno) { unsigned long flags = read_thread_flags(); int ret; @@ -23,7 +23,7 @@ __visible int invoke_syscall_trace(void *table, struct pt_regs *regs, int scno) } if (scno < NR_syscalls) { - ret = invoke_syscall_trace_asm(table, regs, scno); + ret = invoke_syscall_asm(table, regs, scno); goto exit_save; } From patchwork Tue Feb 25 09:55:00 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13989765 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9D2E5C021B2 for ; Tue, 25 Feb 2025 10:28:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=Ugjh5wDawxRKPKlBJt/tVpltkHg0eR7Bu8LyZ1i/cWo=; b=2gcKHJrWw3u7Cj9k7doPhfEm75 flT5xd7z0aoICxH8X1aThJ6hD7XJvdzaH8yeME8iB2334nTSqMdfCPqpS9aNYnI9VSYH1bUy+1nWm Vcxgj4ayd7efFwMUbtRuSH/AUB8u+WACrEbbvzCkB8dzWroQ+L8he4NMcMaK11ebCS3GnyERGpHz9 cog36eF6WpstdN25U4b5lkFS9mKKIRd3vhTMV5FweW0FttgpgD/RFYkIOX1+ZcxIWUlsfa8HiSXmi IpcPPGYdPurt5s5BRpdMLHiB+Dez+aUjijVLLhE2Pogm+JG/4ds+WObZOFpdgOl2knH3olMXFPaXC gn6BrI8w==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tmsBC-0000000GpPj-2dyR; Tue, 25 Feb 2025 10:28:46 +0000 Received: from mail-lj1-x22c.google.com ([2a00:1450:4864:20::22c]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tmrem-0000000Ghg6-3mHi for linux-arm-kernel@lists.infradead.org; Tue, 25 Feb 2025 09:55:18 +0000 Received: by mail-lj1-x22c.google.com with SMTP id 38308e7fff4ca-30613802a6bso56328831fa.1 for ; Tue, 25 Feb 2025 01:55:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1740477315; x=1741082115; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=Ugjh5wDawxRKPKlBJt/tVpltkHg0eR7Bu8LyZ1i/cWo=; b=i/XrIB6RaMuKeomijbAoCcxvAOJg4LZnjHfXNuCWxybgF2tuNtjYZKKDJaRdDT9pf8 S3RmjZPVolneb09C3KvmdG0g1J5AHtxVUug17k+JqT6kqzINQDVRDdnrVx60aEZ9OId7 eHEvJ8CP+DxSjBkLEOmvhPiiBL0yREzcT0LPcOVRUAY1tbSE7ZHHgpF7jPQMuI/0uZck T/EEuBKKa5PcgVzXyTWWD5rOQKasjH4Ql4/wQo8DjB96QbwEM3FPX3kJg7qvfrQu/oDF zo3Rv+yJf/P5t2vzSxkrwToHtepijgmC4EjEhJdoit3CLwY2RKNTHMHsZ19+1O1+Pr7k fz3w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740477315; x=1741082115; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Ugjh5wDawxRKPKlBJt/tVpltkHg0eR7Bu8LyZ1i/cWo=; b=ty5wt40mhn3VcD8ciXUW5Pll2cH94hB1QPaQp+rynEdhb2Gb9UR+1+AcjOzjUF9N2C YOXWW2vZdRGKYJ3izNV3jREbaX7WI3YuTwSyFd/jNaa3dSs4iLU95NgNga4PtsZZYuB3 y6y6FL/7uBks5T5/QWmFpebG2IcqvLNPtf8Bej7UZ3Mrh0tcR+uFzHPW8Rv/oNj3kGBq bgkKhWANE9dN+bSvmwAgtXK3hBjQyY4/QQUZOcsozy107wgWBsIhq/Z44dnyBcjc2YuQ ZBbZKtoUjzj3mhqGxwdgjFHA1BH5bS+HAZpL5oBMnaIbA9ElKiyQpQTnS/OnEL6J2pRa CmWQ== X-Gm-Message-State: AOJu0YzI3sHTigFycscxcNQziApspKds0PjHBbgzV9kEqyFxMBC45Bu6 GeOpOLrjsVWndijnln3+5FctgDbdSUX2ieyS+nY763yKMSAz3VAa+RtlveD07XQ= X-Gm-Gg: ASbGncuFkrW1k4wVmRUoTBBOurjkhzD/V0dsALkKEimp+x8mALjROL4M2bYh6Ihed2W qIx+C7hGpIcLFpVIjpZ0w/ZBPJQn/K9MzwgIo5a6pgsSlIz6r0n0WFzSSv5KOWWk/k0KsBbN0bA JJ2YOLKLLl0eSsha54U/dfmw6180HjyZUSDayXiTcGPUfAOvmFUoEnu8Ju9Jfvr9f0GSbmhLpgd UOSq/zGvV5qofAqwMyGK3ufjf9OHpiUepVUfs9+9dFQjuneKFSWQUwdTQdSZNd5d6lqwh4w5vIW D1al9SFTFf2Cnkr6BOz5HMQ1yhn6z4N1TRCr X-Google-Smtp-Source: AGHT+IGqkcF8DuM2/ZHDNxMpAtKrMC3XCNpt2RI/Sl+BYKODnNKv6ouUbi026+ezhu+8diXjeteQjw== X-Received: by 2002:a05:6512:10c8:b0:545:17b:3cf9 with SMTP id 2adb3069b0e04-548510ed5f5mr1084027e87.48.1740477314909; Tue, 25 Feb 2025 01:55:14 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id 2adb3069b0e04-548514b261esm132867e87.24.2025.02.25.01.55.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 25 Feb 2025 01:55:14 -0800 (PST) From: Linus Walleij Date: Tue, 25 Feb 2025 10:55:00 +0100 Subject: [PATCH v5 13/31] ARM: entry: Create user_mode_enter/exit MIME-Version: 1.0 Message-Id: <20250225-arm-generic-entry-v5-13-2f02313653e5@linaro.org> References: <20250225-arm-generic-entry-v5-0-2f02313653e5@linaro.org> In-Reply-To: <20250225-arm-generic-entry-v5-0-2f02313653e5@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250225_015516_953198_3B704FB7 X-CRM114-Status: GOOD ( 19.20 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The callbacks to the context tracking will be repurposed for several uses that are needed on the IRQ transition to/from userspace. Rename the macro, establish call sites in C calling into the context tracking following the corresponding generic entry function prototypes, despite the assembly macro names become a bit long this makes it clear to readers exactly what is going on and where this call will go. Drop the ifdefs pertaining to context tracking from the macro. The C calls we will use have stubs that will compile these out anyway. The inversion of the signature of the context tracking calls are especially confusing since the generic entry uses the reverse semantics: *enter from* user mode (to kernel mode) and *exit to* user mode (from kernel mode) instead of the other way around as the old context tracker code user_exit_callable() and user_enter_callable() which have inverted semantics. Signed-off-by: Linus Walleij --- arch/arm/include/asm/entry.h | 14 ++++++++++++++ arch/arm/kernel/Makefile | 2 +- arch/arm/kernel/entry-armv.S | 2 +- arch/arm/kernel/entry-common.S | 4 ++-- arch/arm/kernel/entry-header.S | 24 ++++++++++++------------ arch/arm/kernel/entry.c | 15 +++++++++++++++ 6 files changed, 45 insertions(+), 16 deletions(-) diff --git a/arch/arm/include/asm/entry.h b/arch/arm/include/asm/entry.h new file mode 100644 index 0000000000000000000000000000000000000000..e26f369375ca3cf762f92fb499657a666b223ca2 --- /dev/null +++ b/arch/arm/include/asm/entry.h @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +#ifndef __ASM_ENTRY_H__ +#define __ASM_ENTRY_H__ + +struct pt_regs; + +/* + * These are copies of generic entry headers so we can transition + * to generic entry once they are semantically equivalent. + */ +void irqentry_enter_from_user_mode(struct pt_regs *regs); +void irqentry_exit_to_user_mode(struct pt_regs *regs); + +#endif /* __ASM_ENTRY_H__ */ diff --git a/arch/arm/kernel/Makefile b/arch/arm/kernel/Makefile index 69ddf51081f4791982518d37df60c11211b8955a..dd45c6eb8cbdd2595edc4f26026c12b25dd49717 100644 --- a/arch/arm/kernel/Makefile +++ b/arch/arm/kernel/Makefile @@ -17,7 +17,7 @@ CFLAGS_REMOVE_return_address.o = -pg # Object file lists. -obj-y := elf.o entry-common.o irq.o opcodes.o \ +obj-y := elf.o entry.o entry-common.o irq.o opcodes.o \ process.o ptrace.o reboot.o io.o \ setup.o signal.o sigreturn_codes.o \ stacktrace.o sys_arm.o time.o traps.o \ diff --git a/arch/arm/kernel/entry-armv.S b/arch/arm/kernel/entry-armv.S index ef6a657c8d130663e68acd07226815ce8a3fff7d..93a37437a9520118eff411f009e8f6bd729412fc 100644 --- a/arch/arm/kernel/entry-armv.S +++ b/arch/arm/kernel/entry-armv.S @@ -406,7 +406,7 @@ ENDPROC(__fiq_abt) #ifdef CONFIG_TRACE_IRQFLAGS bl trace_hardirqs_off #endif - ct_user_exit save = 0 + asm_irqentry_enter_from_user_mode save = 0 .endif .endm diff --git a/arch/arm/kernel/entry-common.S b/arch/arm/kernel/entry-common.S index f1e48002bd3075ea67b5883178583127fa0055c6..ff1dd3169346f3770cad6b7e218f5d74ffc646fe 100644 --- a/arch/arm/kernel/entry-common.S +++ b/arch/arm/kernel/entry-common.S @@ -111,7 +111,7 @@ ENTRY(ret_to_user_from_irq) no_work_pending: asm_trace_hardirqs_on save = 0 - ct_user_enter save = 0 + asm_irqentry_exit_to_user_mode save = 0 #ifdef CONFIG_GCC_PLUGIN_STACKLEAK bl stackleak_erase_on_task_stack @@ -191,7 +191,7 @@ ENTRY(vector_swi) alignment_trap r10, ip, cr_alignment asm_trace_hardirqs_on save=0 enable_irq_notrace - ct_user_exit save=0 + asm_irqentry_enter_from_user_mode save = 0 /* * Get the system call number. diff --git a/arch/arm/kernel/entry-header.S b/arch/arm/kernel/entry-header.S index 52b4fa97226dbfa4b55aca8eaf74ae24e1e220f8..fb5bb019199b2871e29e306a29bea8fdf47dd7f3 100644 --- a/arch/arm/kernel/entry-header.S +++ b/arch/arm/kernel/entry-header.S @@ -362,31 +362,31 @@ ALT_UP_B(.L1_\@) .endm /* - * Context tracking subsystem. Used to instrument transitions + * Context tracking and other mode transitions. Used to instrument transitions * between user and kernel mode. - */ - .macro ct_user_exit, save = 1 -#ifdef CONFIG_CONTEXT_TRACKING_USER +*/ + .macro asm_irqentry_enter_from_user_mode, save = 1 .if \save stmdb sp!, {r0-r3, ip, lr} - bl user_exit_callable + mov r0, sp @ regs + bl irqentry_enter_from_user_mode ldmia sp!, {r0-r3, ip, lr} .else - bl user_exit_callable + mov r0, sp @ regs + bl irqentry_enter_from_user_mode .endif -#endif .endm - .macro ct_user_enter, save = 1 -#ifdef CONFIG_CONTEXT_TRACKING_USER + .macro asm_irqentry_exit_to_user_mode, save = 1 .if \save stmdb sp!, {r0-r3, ip, lr} - bl user_enter_callable + mov r0, sp @ regs + bl irqentry_exit_to_user_mode ldmia sp!, {r0-r3, ip, lr} .else - bl user_enter_callable + mov r0, sp @ regs + bl irqentry_exit_to_user_mode .endif -#endif .endm /* diff --git a/arch/arm/kernel/entry.c b/arch/arm/kernel/entry.c new file mode 100644 index 0000000000000000000000000000000000000000..8b2e8ea66c1376759d6c0c14aad8728895b3ff1e --- /dev/null +++ b/arch/arm/kernel/entry.c @@ -0,0 +1,15 @@ +// SPDX-License-Identifier: GPL-2.0 +#include +#include + +noinstr void irqentry_enter_from_user_mode(struct pt_regs *regs) +{ + /* This context tracking call has inverse naming */ + user_exit_callable(); +} + +noinstr void irqentry_exit_to_user_mode(struct pt_regs *regs) +{ + /* This context tracking call has inverse naming */ + user_enter_callable(); +} From patchwork Tue Feb 25 09:55:01 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13989766 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id CDFA4C021B2 for ; Tue, 25 Feb 2025 10:30:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=MgvrpQqwFQ0R0umdPRsAplUpCRMZEe+n8Q2dyj0aMbc=; b=XP8kN4MVQSxHNokbAADIzMm+kT PpBKTfvdtjgPC8r4I8vQXl3UmxcYYw1gZ6sgd0Gal7kyOAbv4yXPIA7sshi5b0biISMMSq7vCaYMQ DfTR0lqREc2Rr0pfiWnGcB81ZvWwqzjJXZ/Tm5uludkfymPAjTzZufB9zD/tRW/p5EvdFgTzB2I1e C+S38xnUkBhJIA0j795aayTfh+pLvEPpjATzCT2CaHwaaVj77d4NTCdw3HsbyvSBbSHbUYVYZK+Q1 X/h5xi+Vq8NyYZ2ne7Dr5wx0CuQtONbWv5Q7qz81a/Cv3mlLiWnhw0/p5FB1xBF3L1Ug7nOJp7qtE mkhGRpjw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tmsCj-0000000GppX-1Fiy; Tue, 25 Feb 2025 10:30:21 +0000 Received: from mail-lf1-x136.google.com ([2a00:1450:4864:20::136]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tmren-0000000GhgQ-2Dzm for linux-arm-kernel@lists.infradead.org; Tue, 25 Feb 2025 09:55:19 +0000 Received: by mail-lf1-x136.google.com with SMTP id 2adb3069b0e04-5452efeb87aso5417649e87.3 for ; Tue, 25 Feb 2025 01:55:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1740477316; x=1741082116; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=MgvrpQqwFQ0R0umdPRsAplUpCRMZEe+n8Q2dyj0aMbc=; b=Tx5gAie6/0aXgeQW/z/S9dcfCd0fn9f6MFUh/9xMv0iH4XiDR9hH7sBc/6S7EvWVCo F7NmyWV8o93EO/4sM0KpaQZ2i2ig7Rga2qyxoqhYeic2oHpOUJyFMTWk09PMwUEaqvQ9 NeQxstThVxM8zEekSZ/X32I0jGvkIaLBJP5ntx7/bkQcm6zfeXQOJai3VkvBiqqsleSX dQyE8iipzAzWOhs9YbbJCaKuzdl1Tn1FHP5ABkT/HvGgCetYryv2k0bCk75XfhBvS54m hUB1mSMxGJQv3XIwrHANszPzz5SZWRIf7+PJ1fXyARTfwWXldSAVHamQ/U63b2xt0wSd tq9g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740477316; x=1741082116; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=MgvrpQqwFQ0R0umdPRsAplUpCRMZEe+n8Q2dyj0aMbc=; b=LYIH+SA5Tq/P1PNtO2CwDorbIu9nTWH3xKr9Oa70Spk56YABtQwKIYn/F8vpEvboSh SynGG8eGG0Ls1JRlHaatuIS8MQ7PiAMDxFjRcsndbPFwlCpLqgry6Dx6hClGe9SqZP/8 DE3Vdr6+YZx2Cf48wdqxWFiyydBZQsKOOzMw72zOWL6xKDXPPJwYn0UyV5+LjgVrgICn 447IMRHrClqztf0qB0Y9fjxVRH514dr5EHPIF3ggmYTcTapG7Pv6ZmPIl7vLtgYhKzQF vNsiCGrx5YJFuczmfbKcfFTUUf4EYjnG44u9+DbkS2qn6qdt+oy5xwlVPWJP9MDvrA6O 5+jA== X-Gm-Message-State: AOJu0YziFdDTCdI+XsrJEbXBVdQZfAy5F26Bk2TdwUSSorhrMRqRp/WQ zZZ60BbRoDAGs6OUozXLkgn84yG6cLSQmIw016o67cDmSIPME9jX0fyvpcfRN/U= X-Gm-Gg: ASbGncvJwMTieIPdwfq+4gJE5UmmTpobwMiQk6xq7SpO0L82GPSGPBUB8fHjlEwn7lA 7AYoutTz7FOJ9I8VFgCjmERzIMN4dyTywPITMekQmB62X50/PWM2TEKp2HFfYrLmHTKHcFXStA+ NAnVJQJcBHYm3C3xU1AveJb8TYefNKcLKjLG3T+MCsy+HMPS0U5UfYbGb8M/QWtc4RTEWAi31y2 cpugHcEl/OsTufxefIBP0qQmA3dh7a1rya2ejiJ60jN+UBPa0ginkO7a/Lsbvc7etxfYLmzQPGC OUfKTM3aCfc5UGgseYHYeCDCwoZOjUDxRnEM X-Google-Smtp-Source: AGHT+IFzWjwa7+x+DsN9qo/60H31u5WVx0JVKoFSYgag5yjlwdX8Fst2CGM/hjw802aY7bipwjamsg== X-Received: by 2002:a05:6512:ba6:b0:545:2f5d:813d with SMTP id 2adb3069b0e04-54838f5a9bcmr6922156e87.37.1740477315950; Tue, 25 Feb 2025 01:55:15 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id 2adb3069b0e04-548514b261esm132867e87.24.2025.02.25.01.55.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 25 Feb 2025 01:55:15 -0800 (PST) From: Linus Walleij Date: Tue, 25 Feb 2025 10:55:01 +0100 Subject: [PATCH v5 14/31] ARM: entry: Drop trace argument from usr_entry macro MIME-Version: 1.0 Message-Id: <20250225-arm-generic-entry-v5-14-2f02313653e5@linaro.org> References: <20250225-arm-generic-entry-v5-0-2f02313653e5@linaro.org> In-Reply-To: <20250225-arm-generic-entry-v5-0-2f02313653e5@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250225_015517_576294_84232D0D X-CRM114-Status: GOOD ( 12.76 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The trace argument select whether to look into tracing of the hardirqs for the exceptions from userspace, but also selects whether to call further down to establish context for RCU. Split off a second macro that is called on everything except the FIQ since this is the only outlier. Signed-off-by: Linus Walleij --- arch/arm/kernel/entry-armv.S | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/arch/arm/kernel/entry-armv.S b/arch/arm/kernel/entry-armv.S index 93a37437a9520118eff411f009e8f6bd729412fc..c71110126fc105fc6ac2d6cb0f5f399b4c8b1548 100644 --- a/arch/arm/kernel/entry-armv.S +++ b/arch/arm/kernel/entry-armv.S @@ -357,7 +357,7 @@ ENDPROC(__fiq_abt) #error "sizeof(struct pt_regs) must be a multiple of 8" #endif - .macro usr_entry, trace=1, uaccess=1 + .macro usr_entry, uaccess=1 UNWIND(.fnstart ) UNWIND(.cantunwind ) @ don't unwind the user space sub sp, sp, #PT_REGS_SIZE @@ -401,13 +401,14 @@ ENDPROC(__fiq_abt) @ Clear FP to mark the first stack frame @ zero_fp + .endm - .if \trace + /* Called after usr_entry for everything except FIQ */ + .macro usr_entry_enter #ifdef CONFIG_TRACE_IRQFLAGS bl trace_hardirqs_off #endif asm_irqentry_enter_from_user_mode save = 0 - .endif .endm .macro kuser_cmpxchg_check @@ -429,6 +430,7 @@ ENDPROC(__fiq_abt) .align 5 __dabt_usr: usr_entry uaccess=0 + usr_entry_enter kuser_cmpxchg_check mov r2, sp dabt_helper @@ -439,6 +441,7 @@ ENDPROC(__dabt_usr) .align 5 __irq_usr: usr_entry + usr_entry_enter kuser_cmpxchg_check irq_handler from_user=1 get_thread_info tsk @@ -452,6 +455,7 @@ ENDPROC(__irq_usr) .align 5 __und_usr: usr_entry uaccess=0 + usr_entry_enter @ IRQs must be enabled before attempting to read the instruction from @ user space since that could cause a page/translation fault if the @@ -476,6 +480,7 @@ ENDPROC(__und_usr) .align 5 __pabt_usr: usr_entry + usr_entry_enter mov r2, sp @ regs pabt_helper UNWIND(.fnend ) @@ -495,7 +500,7 @@ ENDPROC(ret_from_exception) .align 5 __fiq_usr: - usr_entry trace=0 + usr_entry kuser_cmpxchg_check mov r0, sp @ struct pt_regs *regs bl handle_fiq_as_nmi From patchwork Tue Feb 25 09:55:02 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13989767 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id CEC4EC021B2 for ; Tue, 25 Feb 2025 10:32:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=0l4jcdIT0+K6QMNd0VK8RPhMVALn2TkHG1rRCOsPTks=; b=VLj7X+BPlxFLRklPYVZM8x9+Sb YnrKuJWMKRdcur6kKg0ebAOK4xypvqYte1jjJI7BhWkQ8tz2ts5jkf8Ln24rnyJdGpWXqHtHMeK/0 i6LXhhuYPOmYSVFcJXHwnFD6mlJq+qSaCij/v4Ga/Dl5/cWjn7lQHoPq/8JlKp7ZY25IQlnVcDQfY uGaHM8H47OFN9NJjNLHRD+IKhSCI/DBK11NGyaC8oorMFZ34bFbGhKZkU1vb/+82oCkRPzQ+c8ziV 7mtc5Gs+V3zS2ZkMGw/e6be0VIdn+jjrG58v6gXv2i2bwm+nPKTRurNeJ3sTfGNY8g1x39YvzaTVq xbZCoiog==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tmsEF-0000000Gq7p-0Jqd; Tue, 25 Feb 2025 10:31:55 +0000 Received: from mail-lf1-x136.google.com ([2a00:1450:4864:20::136]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tmreo-0000000GhhS-2bEz for linux-arm-kernel@lists.infradead.org; Tue, 25 Feb 2025 09:55:19 +0000 Received: by mail-lf1-x136.google.com with SMTP id 2adb3069b0e04-5462a2b9dedso6455959e87.1 for ; Tue, 25 Feb 2025 01:55:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1740477317; x=1741082117; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=0l4jcdIT0+K6QMNd0VK8RPhMVALn2TkHG1rRCOsPTks=; b=qMk87P8xVbfYbOKaRg8U0iu3WsAN+w6kqIQhDLeLPc7zeONcSrXEw1ChYGxAGn2mkS lugkXbEAVCnWMzGKmGgCGFnHDeH132+HUktZSKPmpIdyiQ+Gg1LMwzFAYhANjpJohTRx lknM0iNSkACQy1jtKZiStbXNCdmkxG6TqFq7MuUFpUcJ+qThAbglE5wTQFRAWemwHkvW 5CaWmz1S/4hnIdlUxCgyz4q5fR/DQ4UAIgzvH4dnfn0ZVs7aN9nTL5y+LldsszmltGqx AzmmQpGRuOWPO1+0vPezSfB4yTbjkj3UAIlfnN+npGL5vmF4gjgcm4Sglsz/MVddY2LS J3yg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740477317; x=1741082117; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=0l4jcdIT0+K6QMNd0VK8RPhMVALn2TkHG1rRCOsPTks=; b=aFrjKlaj3hNtwaDHFxWw2eLtwwelOuWrrBHhUi6IB4DlaxL95HgCJBhvpHbGQx+fSN QxJIo1KFO+ArLJG6UF6vVTMyZjUV4IpvQxvmqsfT7qKcTYswwlzO7x7DfQguUKkX6eU0 atKFio918n9/NpbDB+WA/C0JD9ncOgFp63C2yhCTzhPnRpFa1k0QXqi2mr+s8QiDI1vP R+87CGKu8K6blOzWYEKPY2OmEWg2cpU6Ckn7j0L0otKTPw3umxaTHV/6wf/CF1oPTr6n sTlIu65yJXr8B+b74KFYTuaiEmfUZEx3SWDr7iDABscuLAfV+Awuuood8Jc4yyxeuVaL QtcA== X-Gm-Message-State: AOJu0YyGI8EMMZHjkLY1fZhwjIzcCn0vE4+sXKyWRETl9Le0B+3idxy4 NiCPoi3y7OhsMz/jTPkllpvNIEbiAlJegBpgGdvJLuhMTAktfz1S2/D+HAKb2Ok= X-Gm-Gg: ASbGncuqKNCBW7KxbwnxU+zmYJjwZuK00gX4K6GBOJH7XAyMiy41ioPDjPuDUIHojos JOR9H20IDMZyyZNTUFnvAgIs9cGVv0Wiv4CQCGAKFQm7dR9HeCI5yANhgfVH3NLZwzn16eKruIz wSkoPXqQSlusrZp+RKpGFCxLpDKkplBhqZEO+DQEx/Dayfc4vS/0UUnYdk6kWvTEdB1XQJgdwNl XcpnBEyc+vqqxRDtpFZQ/6cKBAUyJOxOvNXjPQe7vn/Yzoklzk8gJMv+7ryApr2MOHVtEW/C0/+ 6Lkh/tM6822yLpDTIYnX3iHoPCDqt8EE9yoe X-Google-Smtp-Source: AGHT+IHtmonsQI8mjfkxDPF9rkkwlOMD+FnaR5UIKDlAt4SXZ1OQl3Nrl9JuRZs/P07o5ZESbealNQ== X-Received: by 2002:a05:6512:b98:b0:545:16fe:536a with SMTP id 2adb3069b0e04-548391452a9mr7078517e87.31.1740477317024; Tue, 25 Feb 2025 01:55:17 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id 2adb3069b0e04-548514b261esm132867e87.24.2025.02.25.01.55.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 25 Feb 2025 01:55:16 -0800 (PST) From: Linus Walleij Date: Tue, 25 Feb 2025 10:55:02 +0100 Subject: [PATCH v5 15/31] ARM: entry: Separate call path for syscall SWI entry MIME-Version: 1.0 Message-Id: <20250225-arm-generic-entry-v5-15-2f02313653e5@linaro.org> References: <20250225-arm-generic-entry-v5-0-2f02313653e5@linaro.org> In-Reply-To: <20250225-arm-generic-entry-v5-0-2f02313653e5@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250225_015518_694523_B9AD9A5B X-CRM114-Status: GOOD ( 18.88 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The syscalls (SWIs, software interrupts) are deviating from how any other interrupts are handled as they enable the IRQs again while processing the syscall, while "hard" IRQs disable all interrupts until they are handled. Break out syscall_enter_from_user_mode() into its own function and call it instead of irqentry_enter_from_user_mode(). As we are moving toward generic entry, we use the signature from the generic function. As the generic function requires the syscall number to be determined, we move the call down below the code that figures out the syscall number, the only practical effect should be that interrupts are re-enabled a few instructions later. As we move the trace_hardirqs_on/off() calls into C, we can just get rid of the helper macro usr_entry_enter again and call asm_irqentry_enter_from_user_mode directly. Signed-off-by: Linus Walleij --- arch/arm/include/asm/entry.h | 1 + arch/arm/kernel/entry-armv.S | 16 ++++------------ arch/arm/kernel/entry-common.S | 18 +++++++++++++----- arch/arm/kernel/entry.c | 14 ++++++++++++++ 4 files changed, 32 insertions(+), 17 deletions(-) diff --git a/arch/arm/include/asm/entry.h b/arch/arm/include/asm/entry.h index e26f369375ca3cf762f92fb499657a666b223ca2..e259b074caef75c7f777b18199623f07bebee5b4 100644 --- a/arch/arm/include/asm/entry.h +++ b/arch/arm/include/asm/entry.h @@ -8,6 +8,7 @@ struct pt_regs; * These are copies of generic entry headers so we can transition * to generic entry once they are semantically equivalent. */ +long syscall_enter_from_user_mode(struct pt_regs *regs, long); void irqentry_enter_from_user_mode(struct pt_regs *regs); void irqentry_exit_to_user_mode(struct pt_regs *regs); diff --git a/arch/arm/kernel/entry-armv.S b/arch/arm/kernel/entry-armv.S index c71110126fc105fc6ac2d6cb0f5f399b4c8b1548..6edf362ab1e1035dafebf6fb7c55db71462c1eae 100644 --- a/arch/arm/kernel/entry-armv.S +++ b/arch/arm/kernel/entry-armv.S @@ -403,14 +403,6 @@ ENDPROC(__fiq_abt) zero_fp .endm - /* Called after usr_entry for everything except FIQ */ - .macro usr_entry_enter -#ifdef CONFIG_TRACE_IRQFLAGS - bl trace_hardirqs_off -#endif - asm_irqentry_enter_from_user_mode save = 0 - .endm - .macro kuser_cmpxchg_check #if !defined(CONFIG_CPU_32v6K) && defined(CONFIG_KUSER_HELPERS) #ifndef CONFIG_MMU @@ -430,7 +422,7 @@ ENDPROC(__fiq_abt) .align 5 __dabt_usr: usr_entry uaccess=0 - usr_entry_enter + asm_irqentry_enter_from_user_mode save = 0 kuser_cmpxchg_check mov r2, sp dabt_helper @@ -441,7 +433,7 @@ ENDPROC(__dabt_usr) .align 5 __irq_usr: usr_entry - usr_entry_enter + asm_irqentry_enter_from_user_mode save = 0 kuser_cmpxchg_check irq_handler from_user=1 get_thread_info tsk @@ -455,7 +447,7 @@ ENDPROC(__irq_usr) .align 5 __und_usr: usr_entry uaccess=0 - usr_entry_enter + asm_irqentry_enter_from_user_mode save = 0 @ IRQs must be enabled before attempting to read the instruction from @ user space since that could cause a page/translation fault if the @@ -480,7 +472,7 @@ ENDPROC(__und_usr) .align 5 __pabt_usr: usr_entry - usr_entry_enter + asm_irqentry_enter_from_user_mode save = 0 mov r2, sp @ regs pabt_helper UNWIND(.fnend ) diff --git a/arch/arm/kernel/entry-common.S b/arch/arm/kernel/entry-common.S index ff1dd3169346f3770cad6b7e218f5d74ffc646fe..14b2495cae3c2f95b0dfecd849b4e16ec143dbe9 100644 --- a/arch/arm/kernel/entry-common.S +++ b/arch/arm/kernel/entry-common.S @@ -109,8 +109,6 @@ ENTRY(ret_to_user_from_irq) movs r1, r1, lsl #16 bne slow_work_pending no_work_pending: - asm_trace_hardirqs_on save = 0 - asm_irqentry_exit_to_user_mode save = 0 #ifdef CONFIG_GCC_PLUGIN_STACKLEAK @@ -189,9 +187,6 @@ ENTRY(vector_swi) reload_current r10, ip zero_fp alignment_trap r10, ip, cr_alignment - asm_trace_hardirqs_on save=0 - enable_irq_notrace - asm_irqentry_enter_from_user_mode save = 0 /* * Get the system call number. @@ -256,6 +251,19 @@ ENTRY(vector_swi) #else str scno, [tsk, #TI_ABI_SYSCALL] #endif + + /* + * Calling out to C to be careful to save and restore registers. + * This call could modify the syscall number. scno is r7 so we + * do not save and restore r7. + */ + mov r0, sp @ regs + mov r1, scno + push {r4 - r6, r8 - r10, lr} + bl syscall_enter_from_user_mode + pop {r4 - r6, r8 - r10, lr} + mov scno, r0 + mov r1, sp @ put regs into r1 stmdb sp!, {r4, r5} @ push fifth and sixth args mov r0, tbl diff --git a/arch/arm/kernel/entry.c b/arch/arm/kernel/entry.c index 8b2e8ea66c1376759d6c0c14aad8728895b3ff1e..1973947c7ad753fccd694b3ef334fba1326f58b6 100644 --- a/arch/arm/kernel/entry.c +++ b/arch/arm/kernel/entry.c @@ -1,15 +1,29 @@ // SPDX-License-Identifier: GPL-2.0 #include #include +#include + +long syscall_enter_from_user_mode(struct pt_regs *regs, long syscall) +{ + trace_hardirqs_on(); + local_irq_enable(); + /* This context tracking call has inverse naming */ + user_exit_callable(); + + /* This will optionally be modified later */ + return syscall; +} noinstr void irqentry_enter_from_user_mode(struct pt_regs *regs) { + trace_hardirqs_off(); /* This context tracking call has inverse naming */ user_exit_callable(); } noinstr void irqentry_exit_to_user_mode(struct pt_regs *regs) { + trace_hardirqs_on(); /* This context tracking call has inverse naming */ user_enter_callable(); } From patchwork Tue Feb 25 09:55:03 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13989768 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 7CFA5C021B2 for ; Tue, 25 Feb 2025 10:33:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=ggLp2N6nIh1N25l9qqVFGy2pp6+E0t6/i4rH9zHeAXo=; b=mHB8zRS3j/KYfDFGZJk4PQow25 KbjNrG0UBBSqcSxe+xmEoMtBA3wLM7ZDJZ2+aQJ2Nm0GOXh02kDgrcX4lNtHf6+V0pAV6xispfKN2 kJ/txto8N6o4kp2Y+bLmOQauFSJaGrXKPb3a4Ht2VKgE/r1YZushBbp6xP516Vk8bMukNQWIHbcPU 2rlEqrfo+ibDF4ckQcC88T4+bPoM2uysvhmhtyFVISuImMTGPzwF9nsgDf1t6be110hj/jzHLEmM/ f4US328LUp0c+XbkryXzQLvw2Bg0IMxvA28/p0/CzhzCppR7yGZqSnRi3O1jiAsKG3rIhfDhGYCKD W7bWO/tg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tmsFm-0000000GqSO-37f4; Tue, 25 Feb 2025 10:33:30 +0000 Received: from mail-lf1-x130.google.com ([2a00:1450:4864:20::130]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tmreq-0000000GhiQ-15bF for linux-arm-kernel@lists.infradead.org; Tue, 25 Feb 2025 09:55:21 +0000 Received: by mail-lf1-x130.google.com with SMTP id 2adb3069b0e04-5462ea9691cso6000961e87.2 for ; Tue, 25 Feb 2025 01:55:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1740477319; x=1741082119; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=ggLp2N6nIh1N25l9qqVFGy2pp6+E0t6/i4rH9zHeAXo=; b=qCNQ1PFquHvQfpVDyvqYn+3UB4Hmg0rHCiQsAP7uLjUZC70TMFkglhoQTEtXAtrdoa yWWZwxnPfuzcSFIn79/ZLA4JcWjEZT4Kq+1v2R1NY0fHz58Ydhr+XEcV/6DRGo9a4KIc VH4UXvmG6a39b7fIzJTJGSzuvTxblJOldawEUrzkM6aE6L3gOvARl/b3a28p7XzhZUUr XZrpP+0b9pwEs67W0dp9Tsvk58olrhlhCguEL5nexY1kpYuW0MSI/+QtXspMVHMxQm1E ysc5yn5miGrCd1zeyn/4fZ0N3TpXqruyO7+N/KHWNtJ2tvx6LEF95nrOV+DS09prJLcV DSLg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740477319; x=1741082119; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ggLp2N6nIh1N25l9qqVFGy2pp6+E0t6/i4rH9zHeAXo=; b=KfqPB8d1wNDG76qfz+U3tc6fwmvy82RL5F8uqFZCGrrSlq2d6IrjnklFT9ZF3LWPB8 kP6UVT4Xkiaxi1KmqvAAUvvSVRueiYalL/DA6bzHRjiXbYGdww3RdSb9KKfvXRVaUpn1 7ixh1vMwLQFWIJR2hDICtXY+uNPGssSPXCyudwcF1SQr4prMBf13fIpSqOIM+6C2qUXH MpWNs+N0bsISmzvBRxQAHgz3n58rQvGxQ/D4u1BjWHKh9n/OxI5SBiIPK8xqTrgKXbpu b3iGHo/CviJTJwudWp5gBVWICetV85wz2r1O6YuBAyj8hZ0PMcWC+tjOr5GXwrEoSlUA pW1g== X-Gm-Message-State: AOJu0YzByhDCfDmiiLRF2dtpYJouugFMEqiROZ3NVIEiliwMyS1waD5H 6WI6akLRxaWZ72mMb6ibiFmGZSeG3UFx6IWxKN70P6BlJIjMsY/Z7fSrA731UZo= X-Gm-Gg: ASbGncsG2PTEQ7y50vk67hb+FYgorTZSAXSrku7Uai7THeiJTEfAy0ivOPydOGhORMD 6Fg+GAJsgy9/7NWkMlus6WaUBYV6JOZvH68B78snK70z4Lp+v521+YZGxerHo3Y/EIqRu25nMHK 3aXJluSsSeWlEZntjfLKctHwIeHVqmHAl4aqxd7xdM6922veUywQaON2eSANBmPS98sutb5/g27 wJ4LqEOz8/NZiMjBMugo2Jb/6ZQdJmTzfs1fQeDZwecdwDPN8Y/XkiPhX7CuR3WgDEynJLJ1DeR DuqlIcj4nNghsFodahLAoPZPeOfVJ3eKJ/x7 X-Google-Smtp-Source: AGHT+IHX3+a18efpxx2qbHXAhs4U0wbk1ul7gmrkFn8LxyQ+grE1C90J+pJbcS1vktIQA6he1PGJqw== X-Received: by 2002:a05:6512:3f19:b0:545:646:7519 with SMTP id 2adb3069b0e04-54838d3d9d5mr6616063e87.0.1740477318630; Tue, 25 Feb 2025 01:55:18 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id 2adb3069b0e04-548514b261esm132867e87.24.2025.02.25.01.55.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 25 Feb 2025 01:55:17 -0800 (PST) From: Linus Walleij Date: Tue, 25 Feb 2025 10:55:03 +0100 Subject: [PATCH v5 16/31] ARM: entry: Drop argument to asm_irqentry macros MIME-Version: 1.0 Message-Id: <20250225-arm-generic-entry-v5-16-2f02313653e5@linaro.org> References: <20250225-arm-generic-entry-v5-0-2f02313653e5@linaro.org> In-Reply-To: <20250225-arm-generic-entry-v5-0-2f02313653e5@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250225_015520_304900_146151B5 X-CRM114-Status: GOOD ( 13.59 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org asm_irqentry_enter_from_user_mode and asm_irqentry_exit_to_user_mode have a "save" argument that will save and restore registers before the call to the C function. Now all invocations set this argument to 0 so drop the surplus code. Signed-off-by: Linus Walleij --- arch/arm/kernel/entry-armv.S | 8 ++++---- arch/arm/kernel/entry-common.S | 2 +- arch/arm/kernel/entry-header.S | 18 ++---------------- 3 files changed, 7 insertions(+), 21 deletions(-) diff --git a/arch/arm/kernel/entry-armv.S b/arch/arm/kernel/entry-armv.S index 6edf362ab1e1035dafebf6fb7c55db71462c1eae..460aa92c3c1f50de905550acf363c58f509bfe0a 100644 --- a/arch/arm/kernel/entry-armv.S +++ b/arch/arm/kernel/entry-armv.S @@ -422,7 +422,7 @@ ENDPROC(__fiq_abt) .align 5 __dabt_usr: usr_entry uaccess=0 - asm_irqentry_enter_from_user_mode save = 0 + asm_irqentry_enter_from_user_mode kuser_cmpxchg_check mov r2, sp dabt_helper @@ -433,7 +433,7 @@ ENDPROC(__dabt_usr) .align 5 __irq_usr: usr_entry - asm_irqentry_enter_from_user_mode save = 0 + asm_irqentry_enter_from_user_mode kuser_cmpxchg_check irq_handler from_user=1 get_thread_info tsk @@ -447,7 +447,7 @@ ENDPROC(__irq_usr) .align 5 __und_usr: usr_entry uaccess=0 - asm_irqentry_enter_from_user_mode save = 0 + asm_irqentry_enter_from_user_mode @ IRQs must be enabled before attempting to read the instruction from @ user space since that could cause a page/translation fault if the @@ -472,7 +472,7 @@ ENDPROC(__und_usr) .align 5 __pabt_usr: usr_entry - asm_irqentry_enter_from_user_mode save = 0 + asm_irqentry_enter_from_user_mode mov r2, sp @ regs pabt_helper UNWIND(.fnend ) diff --git a/arch/arm/kernel/entry-common.S b/arch/arm/kernel/entry-common.S index 14b2495cae3c2f95b0dfecd849b4e16ec143dbe9..df564388905ee019cd5553f8b37e678da59e3222 100644 --- a/arch/arm/kernel/entry-common.S +++ b/arch/arm/kernel/entry-common.S @@ -109,7 +109,7 @@ ENTRY(ret_to_user_from_irq) movs r1, r1, lsl #16 bne slow_work_pending no_work_pending: - asm_irqentry_exit_to_user_mode save = 0 + asm_irqentry_exit_to_user_mode #ifdef CONFIG_GCC_PLUGIN_STACKLEAK bl stackleak_erase_on_task_stack diff --git a/arch/arm/kernel/entry-header.S b/arch/arm/kernel/entry-header.S index fb5bb019199b2871e29e306a29bea8fdf47dd7f3..50c0b55adc7421e7be123c9d00f94b1ebb93ff9e 100644 --- a/arch/arm/kernel/entry-header.S +++ b/arch/arm/kernel/entry-header.S @@ -365,28 +365,14 @@ ALT_UP_B(.L1_\@) * Context tracking and other mode transitions. Used to instrument transitions * between user and kernel mode. */ - .macro asm_irqentry_enter_from_user_mode, save = 1 - .if \save - stmdb sp!, {r0-r3, ip, lr} + .macro asm_irqentry_enter_from_user_mode mov r0, sp @ regs bl irqentry_enter_from_user_mode - ldmia sp!, {r0-r3, ip, lr} - .else - mov r0, sp @ regs - bl irqentry_enter_from_user_mode - .endif .endm - .macro asm_irqentry_exit_to_user_mode, save = 1 - .if \save - stmdb sp!, {r0-r3, ip, lr} + .macro asm_irqentry_exit_to_user_mode mov r0, sp @ regs bl irqentry_exit_to_user_mode - ldmia sp!, {r0-r3, ip, lr} - .else - mov r0, sp @ regs - bl irqentry_exit_to_user_mode - .endif .endm /* From patchwork Tue Feb 25 09:55:04 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13989769 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E1054C021B2 for ; Tue, 25 Feb 2025 10:35:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=0KMIzffvEsBK43v+gCyDeKdrTwv9xJqaffLfxp2Y9ec=; b=qDdjyQ+siZuCDHANKppKclUjvV WKxQorbzV3Rd1pnweyWSjqt3l/bl4RKPdnt1T8m4MJuhZWIKbh7avqrnIH+8NA0Rhtn/rG2X8QIho i1BayIPOpRzsiga/TjesfYhxon2E2Bq6Lt5LxkJq8POpnIFir9EzLpJ9LaDvDQy1Rj2VKiEqPUdxi +YQ6sekzreVyIuFHV9XE6jlOHlGTOCTpzZ08HouAvlmzp49OM2E73vAi6om/8oikhk0sE7mmBrm4k UpP7afrPcCAepmVUYmKPoVuYrCEtvd3naNAcs3EPz05ywP2wyLjp8/ncW6eDbN147TQa6ygWQoplj 9dKYdOWA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tmsHJ-0000000Gqt8-1vWw; Tue, 25 Feb 2025 10:35:05 +0000 Received: from mail-lf1-x12d.google.com ([2a00:1450:4864:20::12d]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tmrer-0000000Ghiv-1GOM for linux-arm-kernel@lists.infradead.org; Tue, 25 Feb 2025 09:55:22 +0000 Received: by mail-lf1-x12d.google.com with SMTP id 2adb3069b0e04-545316f80beso4810684e87.1 for ; Tue, 25 Feb 2025 01:55:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1740477320; x=1741082120; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=0KMIzffvEsBK43v+gCyDeKdrTwv9xJqaffLfxp2Y9ec=; b=GnI4E65fZFyKnp/lpWwHxj3YsuAeKEZ55SeBoz+L43GgpsrlOlRi5zyKVsn1L+C00f 07eaukAIMjp55PVLvVyxJ+sqhQ70r71WSwGfVwzvpYQXBeocPudFzMYdEstniU4omBPL OXkCca8W1ohMSLSLyoSiW5PEKWpGmiXSrQvJpm0g7kBPta3PpS64GPgPkLLJmf82Bnar +TG/6ygEshSV8O/UOpRCQpiZciSELnBi1kd9Iz/PBQu6Q332NbxGayEJXp7TaZo+uM1l AVDcuNQGIxaTjFUKfR+HpPXGcGuofgF94GJU7qOE4YU85ZmVXfDZOarA8JVWduakChgY mmiA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740477320; x=1741082120; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=0KMIzffvEsBK43v+gCyDeKdrTwv9xJqaffLfxp2Y9ec=; b=CuzLTyf/5YTzNhsytT20GhNcTGaGvz7s0+x+ncGmTMP3OhRQabRFY56Czp5u3LR0su lQ3zR7mZbt58pG/1QopVvACGezbWyg9a9yhfl0YQyvqwWAWJMlFR3jSIQVveTuz6tY8H D8oDJSuEAWvcuWzfqluJ92pZdWCuQ8Jd+S7pgv/IGerzhtqn7cGxyTrTveNDg4AghGsq Swz0377ptkzA7yEAYsFaferyYhl2JxLdMC3htER7nc56QHJ3PEIicPqQWcITVBDnxITW yU5RdbuG8ymlQgHOWHOLNKQLCTHN9aYVvJyLABMZRqg/NqOwgF2sOdlLdQDMLIearudh q0zA== X-Gm-Message-State: AOJu0YwtBJghx4ukKWZGNrXNTwHuMaFOxyKITKvuNriXviySdTuq9/yZ u0goXYleQJwahsmPJGm3K/2wCW3EUX/aCsuyItDkXCbgGxRCs6FIZaxbAzHvQ1M= X-Gm-Gg: ASbGncueaJJ6R9ekGzQqSXTBBZBL5GauqG2uDlqxgYTED2W0SwYjbm1wmVPONW7VMS7 a3rzTFbKwgcmYTXzDnfvgMObXpduyAMipXAGrW/tZgZTP9iZtoL/MhLPKZff9mxLave+OxXMSNL r0krcU+rG7Ml4L349/Y70zkExqSpeOl8LiB6WLIQPatudZv9objmoZ8wGoer+K6qygZjI11XBgM Wb7NtR69gfb67mW+JbsVMSueNj9m9rVWViZPh6QVoBoAZBd1nrGJuk9iFwzroYRKGOTInWwmEkt rNzP6RR6PIurcTmPmF1gjHiZLRdLi24v8hGj X-Google-Smtp-Source: AGHT+IFQbbLnCCyj6FgNCwGxulDwvPP6jZb6g7kcFOt1Jz/gH3uLa0VdebmCRshYEfTm6zK7LqNLZQ== X-Received: by 2002:a05:6512:6d0:b0:545:93b:30c3 with SMTP id 2adb3069b0e04-548510cf94emr1053860e87.18.1740477319703; Tue, 25 Feb 2025 01:55:19 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id 2adb3069b0e04-548514b261esm132867e87.24.2025.02.25.01.55.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 25 Feb 2025 01:55:19 -0800 (PST) From: Linus Walleij Date: Tue, 25 Feb 2025 10:55:04 +0100 Subject: [PATCH v5 17/31] ARM: entry: Implement syscall_exit_to_user_mode() MIME-Version: 1.0 Message-Id: <20250225-arm-generic-entry-v5-17-2f02313653e5@linaro.org> References: <20250225-arm-generic-entry-v5-0-2f02313653e5@linaro.org> In-Reply-To: <20250225-arm-generic-entry-v5-0-2f02313653e5@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250225_015521_346609_4EC24777 X-CRM114-Status: GOOD ( 15.60 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Mirroring syscall_enter_from_user_mode() we create the sibling call syscall_exit_to_user_mode() and start to move some stuff over from assembly, first the rseq call and the interrupt disablement. Signed-off-by: Linus Walleij --- arch/arm/include/asm/entry.h | 1 + arch/arm/include/asm/signal.h | 1 - arch/arm/kernel/entry-common.S | 10 ++-------- arch/arm/kernel/entry.c | 8 ++++++++ arch/arm/kernel/signal.c | 7 ------- 5 files changed, 11 insertions(+), 16 deletions(-) diff --git a/arch/arm/include/asm/entry.h b/arch/arm/include/asm/entry.h index e259b074caef75c7f777b18199623f07bebee5b4..8b8cc9a1eefbd00dd75d4e344576fc184a413768 100644 --- a/arch/arm/include/asm/entry.h +++ b/arch/arm/include/asm/entry.h @@ -9,6 +9,7 @@ struct pt_regs; * to generic entry once they are semantically equivalent. */ long syscall_enter_from_user_mode(struct pt_regs *regs, long); +void syscall_exit_to_user_mode(struct pt_regs *regs); void irqentry_enter_from_user_mode(struct pt_regs *regs); void irqentry_exit_to_user_mode(struct pt_regs *regs); diff --git a/arch/arm/include/asm/signal.h b/arch/arm/include/asm/signal.h index 7acccc96840c8a17744cc3f2894f19d142aba4fd..bb753cd3e29fc54186543c48b2aee7b221301a37 100644 --- a/arch/arm/include/asm/signal.h +++ b/arch/arm/include/asm/signal.h @@ -23,7 +23,6 @@ typedef struct { #include -void do_rseq_syscall(struct pt_regs *regs); void do_work_pending(struct pt_regs *regs, unsigned int thread_flags); #endif diff --git a/arch/arm/kernel/entry-common.S b/arch/arm/kernel/entry-common.S index df564388905ee019cd5553f8b37e678da59e3222..3559e0a9f5e079f1af91392bf000d39f9b45906c 100644 --- a/arch/arm/kernel/entry-common.S +++ b/arch/arm/kernel/entry-common.S @@ -65,12 +65,9 @@ ret_fast_syscall: UNWIND(.fnstart ) UNWIND(.cantunwind ) add sp, sp, #(S_R0 + S_OFF) -#if IS_ENABLED(CONFIG_DEBUG_RSEQ) /* do_rseq_syscall needs interrupts enabled. */ mov r0, sp @ 'regs' - bl do_rseq_syscall -#endif - disable_irq_notrace @ disable interrupts + bl syscall_exit_to_user_mode ldr r1, [tsk, #TI_FLAGS] @ re-check for syscall tracing movs r1, r1, lsl #16 beq no_work_pending @@ -97,13 +94,10 @@ ENDPROC(ret_fast_syscall) * do_work_pending() will update this state if necessary. */ ENTRY(ret_to_user) -#if IS_ENABLED(CONFIG_DEBUG_RSEQ) /* do_rseq_syscall needs interrupts enabled. */ enable_irq_notrace @ enable interrupts mov r0, sp @ 'regs' - bl do_rseq_syscall -#endif - disable_irq_notrace @ disable interrupts + bl syscall_exit_to_user_mode ENTRY(ret_to_user_from_irq) ldr r1, [tsk, #TI_FLAGS] movs r1, r1, lsl #16 diff --git a/arch/arm/kernel/entry.c b/arch/arm/kernel/entry.c index 1973947c7ad753fccd694b3ef334fba1326f58b6..47233804733752aef0f2fec6f54badeab18e98d0 100644 --- a/arch/arm/kernel/entry.c +++ b/arch/arm/kernel/entry.c @@ -1,7 +1,9 @@ // SPDX-License-Identifier: GPL-2.0 #include +#include #include #include +#include long syscall_enter_from_user_mode(struct pt_regs *regs, long syscall) { @@ -14,6 +16,12 @@ long syscall_enter_from_user_mode(struct pt_regs *regs, long syscall) return syscall; } +void syscall_exit_to_user_mode(struct pt_regs *regs) +{ + rseq_syscall(regs); + local_irq_disable(); +} + noinstr void irqentry_enter_from_user_mode(struct pt_regs *regs) { trace_hardirqs_off(); diff --git a/arch/arm/kernel/signal.c b/arch/arm/kernel/signal.c index 7b1a16e86b236575efdc29cda9b751e8e2a3f64e..03b20637a2e1ca6f327223c9132915a97a9c4697 100644 --- a/arch/arm/kernel/signal.c +++ b/arch/arm/kernel/signal.c @@ -659,13 +659,6 @@ struct page *get_signal_page(void) return page; } -#ifdef CONFIG_DEBUG_RSEQ -asmlinkage void do_rseq_syscall(struct pt_regs *regs) -{ - rseq_syscall(regs); -} -#endif - /* * Compile-time assertions for siginfo_t offsets. Check NSIG* as well, as * changes likely come with new fields that should be added below. From patchwork Tue Feb 25 09:55:05 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13989770 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 63536C021B6 for ; Tue, 25 Feb 2025 10:36:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=kElQgdyrfkPgzICs9KEd5Is5KMXf1IbQ9ph46r/iv/o=; b=kKfQldru6gI/hDlzbmrHdn+f+r Ax4MOGk0ueWM09q9Df7TW2vGvZL7UqV9pnvwkwdqniO7f7QnWlsqGOWWzQqMv+hTkH03wOEMYDYCR IaqjjnoLPPf7YkIneBZiMu9+gi2UpC8OFmSLm741zTuaFvfmkOyVMlkisNthYK5T3vXTVc2oJr5ZX p7aC7FnT3N0dILuS75k7H3LyiqJdPAMdMO0/eWja7i2LWygb2CK1QaQvwT1mS4BV7AKDI+ZJZ9Ceg fA42B39opacx90zXuIFlp2vtiFUVDondwPtCcxI+AZ2XC8oNKgc3SO/bZ5wCdDC0uTpiR4afgBPTK x2J7wHTw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tmsIp-0000000Gr7h-0Rsv; Tue, 25 Feb 2025 10:36:39 +0000 Received: from mail-lf1-x132.google.com ([2a00:1450:4864:20::132]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tmres-0000000Ghjq-1e6A for linux-arm-kernel@lists.infradead.org; Tue, 25 Feb 2025 09:55:23 +0000 Received: by mail-lf1-x132.google.com with SMTP id 2adb3069b0e04-5461a485a72so5229579e87.0 for ; Tue, 25 Feb 2025 01:55:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1740477321; x=1741082121; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=kElQgdyrfkPgzICs9KEd5Is5KMXf1IbQ9ph46r/iv/o=; b=nGoSM2+aRxdbfDH472hF5AQfLPU/6YlO27v5LW0vFR410XiNug09TNAEFLfAZHGwrO Go6szEtwXe2cwdB0kk2TXBxpuUFFHu1EbR6CzBF+4mRHSEBxVhgkMnyY5/LVVWRCc00/ aEktKOOE8dLqmq6xDbOf+Ennh87dgdeqrmH57pgLPy034/Bksy/P6tt1SNlVOh22VbIk 0Gg7w7ncN4Sx97+fJOStlqzH+Brcz92o8D7H1OFlpmT75THHW5D6Py7npbhh4n1g+eCS OyWDdw+2TfogSxUKsfr+RXw9qtvc8OFMDrVi9SwcCcNZlnOhHEGwoqQ24n5zdGh82W/2 aFPQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740477321; x=1741082121; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=kElQgdyrfkPgzICs9KEd5Is5KMXf1IbQ9ph46r/iv/o=; b=Myq81uFLrbz9m3WyVCF3fWbLfwZkdGx1lTCYrb9yTq5yvRgm7Hwr1vNNxXfOTmGs6c vO/FkTSmenipciOU5KoLrYxE/ZABu7ccyqeyTBqo3tL4F5RjC1mXNFTiDDQkvH6I1Nms RhglfvVwUn3KUP2J7My3GmZeJMb5/kjqVxXmlLWsbt8NIQPJD5haf4p1HbC9UwSruXuW MX3MciHe8BOblQYo3fEKwjkap6tMb94MK83blbF6Jn5hwI3Y3Z4oQkrqoLAR88TwcuEq saIsVONKu38HzS6pFqRJYEU5ebXaNjaHq++MohWb/nOq3r3+VLJaf+OJtuIaE0g3NVA5 prDg== X-Gm-Message-State: AOJu0Yz581U85oxXut6eXCstiuK8SPbjXBxZ/HFp0vtr6LXjTIuRvc/9 gal/lC0HOIOzgvBwaW1XlZwiKKeYUfNUfF8S9StKjBQf4W/0Qm9z+oMvOF3pPb0= X-Gm-Gg: ASbGncvQqbYbW6OIkl5GbqUa+fUzSTThDVSZM0psci13alie1LaLCc/VTgWyBcgh+Fi aPx9XFdKpmy5uiSJzovKMJ/v14UpknO5niiOhE6TewtEIbA0jeFi6SlM+1eB2hxbvXPojN9YtNQ /NGnYfMEMXIYSDi1/gyfmvx6OKrddIVEMg2Edd0JXjxl7qFjNY1cJs9BvyEHl5xFKvC6OkH6n19 Bxf3wxHjnegVre+/Kcy4tDOAYeuk38VyAF87ISqQ9Go+lanbQupbkRt5xH2rXRdQ4tt1smuARGr YNlwHv128RMkpi9GAzy1WirdmSBZ0EDGvUyb X-Google-Smtp-Source: AGHT+IE7t84m4BCM4TmWOCmFQFqdsr7GUqDmKXu9j93tWoANi+DXx582k3Ij4BI18U0vfQc//0bwWQ== X-Received: by 2002:a05:6512:3186:b0:545:f9c:a80f with SMTP id 2adb3069b0e04-54838edd978mr5618553e87.1.1740477320805; Tue, 25 Feb 2025 01:55:20 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id 2adb3069b0e04-548514b261esm132867e87.24.2025.02.25.01.55.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 25 Feb 2025 01:55:20 -0800 (PST) From: Linus Walleij Date: Tue, 25 Feb 2025 10:55:05 +0100 Subject: [PATCH v5 18/31] ARM: entry: Drop the superfast ret_fast_syscall MIME-Version: 1.0 Message-Id: <20250225-arm-generic-entry-v5-18-2f02313653e5@linaro.org> References: <20250225-arm-generic-entry-v5-0-2f02313653e5@linaro.org> In-Reply-To: <20250225-arm-generic-entry-v5-0-2f02313653e5@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250225_015522_437566_08C24CF7 X-CRM114-Status: GOOD ( 14.82 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org When not using any irqflags tracing, context tracking or rseq debugging, the ARM kernel uses a "superfast" ret_fast_syscall with a very few assembly instructions. With generic entry, such local optimizations are not really possible, we need to call C unconditionally and take the (hopefully small) overhead. With generic entry in C, most of the tracing (etc) calls are replaced by stubs and optimized out by the compiler. Signed-off-by: Linus Walleij --- arch/arm/kernel/entry-common.S | 29 +---------------------------- 1 file changed, 1 insertion(+), 28 deletions(-) diff --git a/arch/arm/kernel/entry-common.S b/arch/arm/kernel/entry-common.S index 3559e0a9f5e079f1af91392bf000d39f9b45906c..dddbd6ba1eb9fcbc2c0d81f24e1a9b3231558d7f 100644 --- a/arch/arm/kernel/entry-common.S +++ b/arch/arm/kernel/entry-common.S @@ -30,34 +30,8 @@ saved_pc .req lr .section .entry.text,"ax",%progbits .align 5 -#if !(IS_ENABLED(CONFIG_TRACE_IRQFLAGS) || IS_ENABLED(CONFIG_CONTEXT_TRACKING_USER) || \ - IS_ENABLED(CONFIG_DEBUG_RSEQ)) /* - * This is the fast syscall return path. We do as little as possible here, - * such as avoiding writing r0 to the stack. We only use this path if we - * have tracing, context tracking and rseq debug disabled - the overheads - * from those features make this path too inefficient. - */ -ret_fast_syscall: - UNWIND(.fnstart ) - UNWIND(.cantunwind ) - disable_irq_notrace @ disable interrupts - ldr r1, [tsk, #TI_FLAGS] @ re-check for syscall tracing - movs r1, r1, lsl #16 - bne fast_work_pending - - restore_user_regs fast = 0, offset = S_OFF - UNWIND(.fnend ) -ENDPROC(ret_fast_syscall) - - /* Ok, we need to do extra processing, enter the slow path. */ -fast_work_pending: - add sp, sp, #(S_R0 + S_OFF) - /* fall through to work_pending */ -#else -/* - * The "replacement" ret_fast_syscall for when tracing, context tracking, - * or rseq debug is enabled. As we will need to call out to some C functions, + * As we will need to call out to some C functions, * we save r0 first to avoid needing to save registers around each C function * call. */ @@ -75,7 +49,6 @@ ret_fast_syscall: ENDPROC(ret_fast_syscall) /* Slower path - fall through to work_pending */ -#endif tst r1, #_TIF_SYSCALL_WORK beq slow_work_pending From patchwork Tue Feb 25 09:55:06 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13989771 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 63630C021B2 for ; Tue, 25 Feb 2025 10:38:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=SK4XvykyJ6Zn+vIxSK9vSx/zpAi4l4ILYAtxYCxqygE=; b=dALFm9T1euU9J77mZnOBbTkeP4 CbBFOGYD4xSy3pTej7IvWrA37VN2mEKvUNwRwPdtwOAKtdIH/NmX1aprQnPhZFQm2ABHGV7maEer+ jZ8mDfDvofoFKc1A/oyOvEEESF2ftm6+9tmA8Y1J4tUzoEV56A8Ury7G9KjXJY1dkgsW7ZbfBGg0K hgKOUiduj4fCBiwmGyt/tTAl782xMTETcYTD4ixbDJhjTBev1HjFvvi3QVFaNDhsoaOWGRgaRgqp9 MrTIy2jlIP1+zJ6y7zx1gUlCjpsTMIBzHO9EDaBDLFad4l4M3zLx5DdIeRQzWb0GiGRSw7LtFVNnl iUke3ckg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tmsKK-0000000GrIL-3fU9; Tue, 25 Feb 2025 10:38:12 +0000 Received: from mail-lf1-x129.google.com ([2a00:1450:4864:20::129]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tmret-0000000GhkG-3WbE for linux-arm-kernel@lists.infradead.org; Tue, 25 Feb 2025 09:55:24 +0000 Received: by mail-lf1-x129.google.com with SMTP id 2adb3069b0e04-548878c6a5aso293991e87.3 for ; Tue, 25 Feb 2025 01:55:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1740477322; x=1741082122; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=SK4XvykyJ6Zn+vIxSK9vSx/zpAi4l4ILYAtxYCxqygE=; b=FPcXIfODQwlfwoOet4GZJS1RnG7nEgajEmxMTqg4JDrVdBPpGOOLQoBABmsrxTtcG0 umCE50fP7kocSbuPBo0OeSTbJOQ5kCDV0ey0RBCaq317m7zVUkWWlusYXAwAVJHAuwhu VcPWoCGoi/E2+MLnOoZ2PVjM9mNv06HirWcPCQhJe/btLJ9uE2qWqdAF3dBB/KlI9Mdt y7a8oCTBKYTiqN/4rKp1cfFjM7v8+7bwMC/DQABcQK3Ty7gpJrnW9iUibqMEgb4hlMHL 9S/mRmzoyu5+IC36UpjRBEG+tFnEV11O164pPd59iI2OqGvAmYrRqmleHN4zaQYjJQT9 Dwrw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740477322; x=1741082122; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=SK4XvykyJ6Zn+vIxSK9vSx/zpAi4l4ILYAtxYCxqygE=; b=RZKe9DPfWBaLpIKNvaT+9t8i5o4H7y2k0KIZoymN1h9aAH6uN3Q6ugKuyd3d142ARK xMv5E4tSRrkbqvcLURBv96nxfaq1fV+AJawZwVN+/I4g1K0WX3ZZ+u5X0WBja5FjvaLV jRi2fYQgFXRt4evft7nYqEChjAoji5qOeVTS7rJBXYJhdtqZki9uRQWi+1wmbiLOuC+6 x/dJL06aNwd7OYUuUDiIkQZG3Re6gBLdTVdPnwllvjcC1MIK/4gdgt0+h3r3FgTSpZEn 5oNI66RkVqZeFk0EqgyjpiCvP/mnOYzULX0tVSH7hVOYLgLaMrBBHcwcw4b8fVbnJSa8 9nCw== X-Gm-Message-State: AOJu0YziVRDn/ZU4kgFNe/5Lmonr0zhxxs3rF/5mq2i8AhwXSBdy6bJJ QRcaVVfLbSzRT1bhTJCg5N+HOQwiG7n18anIXt1JOt9Zj9cHPukH1SCDuFSVNRY= X-Gm-Gg: ASbGnctnKILjBds4f47D+jl/i73LRyc9l0nenjtyLydz4NE/MSivhxxIx/IvMrOWsY0 CwOr+gmcJPIAlxNdQdmUFkKEecsZ+weF2CNq7n4zrthZzG4E+CFlGWY9JfQQ6cDylePJ6mt2r09 9idZM+E/YE6Rd2/+qOmgGp8dBVsKY3cKr1e8Bo4SXIzXblKJuJ7tAxyXXp6t5GPtsi+fdrVb5Dd /FZnKFE3Gh99IWc9ZBUYCtKw0ldsj+M94D7WtQgoI+mx2jpVA8Dq1kJxZr11hBzn33+3feZwttc VZSZ9XaJ9PQGsWhu0/PqsQ11uEhGAXo3HgLO X-Google-Smtp-Source: AGHT+IHCtJvTxVRz26fQjj6kCjgqj3b1/2GhW8qtHYMwDMQCiPhoAVv8hPhhcX8Zp/SFXBZFx0kpow== X-Received: by 2002:a05:6512:3f17:b0:545:2ddf:1189 with SMTP id 2adb3069b0e04-5483912fc63mr6223975e87.6.1740477321934; Tue, 25 Feb 2025 01:55:21 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id 2adb3069b0e04-548514b261esm132867e87.24.2025.02.25.01.55.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 25 Feb 2025 01:55:21 -0800 (PST) From: Linus Walleij Date: Tue, 25 Feb 2025 10:55:06 +0100 Subject: [PATCH v5 19/31] ARM: entry: Remove fast and offset register restore MIME-Version: 1.0 Message-Id: <20250225-arm-generic-entry-v5-19-2f02313653e5@linaro.org> References: <20250225-arm-generic-entry-v5-0-2f02313653e5@linaro.org> In-Reply-To: <20250225-arm-generic-entry-v5-0-2f02313653e5@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250225_015523_886622_DBE1F827 X-CRM114-Status: GOOD ( 13.89 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Now all remaining invocations of restore_user_regs has fast = 0 and offset = 0 set, so drop these parameters to the macro. Signed-off-by: Linus Walleij --- arch/arm/kernel/entry-armv.S | 2 +- arch/arm/kernel/entry-common.S | 2 +- arch/arm/kernel/entry-header.S | 29 +++++++++-------------------- 3 files changed, 11 insertions(+), 22 deletions(-) diff --git a/arch/arm/kernel/entry-armv.S b/arch/arm/kernel/entry-armv.S index 460aa92c3c1f50de905550acf363c58f509bfe0a..3cd5a80b47758867d1db6c3971030370bca1551a 100644 --- a/arch/arm/kernel/entry-armv.S +++ b/arch/arm/kernel/entry-armv.S @@ -497,7 +497,7 @@ __fiq_usr: mov r0, sp @ struct pt_regs *regs bl handle_fiq_as_nmi get_thread_info tsk - restore_user_regs fast = 0, offset = 0 + restore_user_regs UNWIND(.fnend ) ENDPROC(__fiq_usr) diff --git a/arch/arm/kernel/entry-common.S b/arch/arm/kernel/entry-common.S index dddbd6ba1eb9fcbc2c0d81f24e1a9b3231558d7f..d27e34d7e97a079d62e7b778cf7cf97c302e4763 100644 --- a/arch/arm/kernel/entry-common.S +++ b/arch/arm/kernel/entry-common.S @@ -81,7 +81,7 @@ no_work_pending: #ifdef CONFIG_GCC_PLUGIN_STACKLEAK bl stackleak_erase_on_task_stack #endif - restore_user_regs fast = 0, offset = 0 + restore_user_regs ENDPROC(ret_to_user_from_irq) ENDPROC(ret_to_user) diff --git a/arch/arm/kernel/entry-header.S b/arch/arm/kernel/entry-header.S index 50c0b55adc7421e7be123c9d00f94b1ebb93ff9e..51c7f93c1ded1db56f173c7aaed8094ef011ff5a 100644 --- a/arch/arm/kernel/entry-header.S +++ b/arch/arm/kernel/entry-header.S @@ -290,7 +290,7 @@ .endm - .macro restore_user_regs, fast = 0, offset = 0 + .macro restore_user_regs #if defined(CONFIG_CPU_32v6K) && \ (!defined(CONFIG_CPU_V6) || defined(CONFIG_SMP)) #ifdef CONFIG_CPU_V6 @@ -309,8 +309,8 @@ ALT_UP_B(.L1_\@) #ifndef CONFIG_THUMB2_KERNEL @ ARM mode restore mov r2, sp - ldr r1, [r2, #\offset + S_PSR] @ get calling cpsr - ldr lr, [r2, #\offset + S_PC]! @ get pc + ldr r1, [r2, S_PSR] @ get calling cpsr + ldr lr, [r2, S_PC]! @ get pc tst r1, #PSR_I_BIT | 0x0f bne 1f msr spsr_cxsf, r1 @ save in spsr_svc @@ -318,31 +318,24 @@ ALT_UP_B(.L1_\@) @ We must avoid clrex due to Cortex-A15 erratum #830321 strex r1, r2, [r2] @ clear the exclusive monitor #endif - .if \fast - ldmdb r2, {r1 - lr}^ @ get calling r1 - lr - .else ldmdb r2, {r0 - lr}^ @ get calling r0 - lr - .endif mov r0, r0 @ ARMv5T and earlier require a nop @ after ldm {}^ - add sp, sp, #\offset + PT_REGS_SIZE + add sp, sp, #PT_REGS_SIZE movs pc, lr @ return & move spsr_svc into cpsr 1: bug "Returning to usermode but unexpected PSR bits set?", \@ #elif defined(CONFIG_CPU_V7M) @ V7M restore. @ Note that we don't need to do clrex here as clearing the local @ monitor is part of the exception entry and exit sequence. - .if \offset - add sp, #\offset - .endif - v7m_exception_slow_exit ret_r0 = \fast + v7m_exception_slow_exit ret_r0 = 0 #else @ Thumb mode restore mov r2, sp - load_user_sp_lr r2, r3, \offset + S_SP @ calling sp, lr - ldr r1, [sp, #\offset + S_PSR] @ get calling cpsr - ldr lr, [sp, #\offset + S_PC] @ get pc - add sp, sp, #\offset + S_SP + load_user_sp_lr r2, r3, S_SP @ calling sp, lr + ldr r1, [sp, S_PSR] @ get calling cpsr + ldr lr, [sp, S_PC] @ get pc + add sp, sp, S_SP tst r1, #PSR_I_BIT | 0x0f bne 1f msr spsr_cxsf, r1 @ save in spsr_svc @@ -350,11 +343,7 @@ ALT_UP_B(.L1_\@) @ We must avoid clrex due to Cortex-A15 erratum #830321 strex r1, r2, [sp] @ clear the exclusive monitor - .if \fast - ldmdb sp, {r1 - r12} @ get calling r1 - r12 - .else ldmdb sp, {r0 - r12} @ get calling r0 - r12 - .endif add sp, sp, #PT_REGS_SIZE - S_SP movs pc, lr @ return & move spsr_svc into cpsr 1: bug "Returning to usermode but unexpected PSR bits set?", \@ From patchwork Tue Feb 25 09:55:07 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13989772 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3E8EEC021B2 for ; Tue, 25 Feb 2025 10:39:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=hLPL0cxSmhQjzTuSazqL5ULZpLc4BXlPwpKFYiv7qfk=; b=EA30SbCGJKM6hUvuMU245ik8je 3575R8EmU66Ktwu+lchuRrqTaVifK3eHPU9GREk1gzUkgJaqDnJhyvigni4nVJJgyhIB+b8XkvluM kgRmYX+3dA7VIreSWSD4spdA2sdunHBEyt0GL3h7iZ2nt9ANiFBChWmIJIvP8SvYpuVzFV14onwum DjwVcEPgy7ZvBntNpBPC1cLiYicjudWn+3q9eAqJYRKGx9ak3oXTx2Vv0Kd/hZ+0zau3+JqOGqOKC t4cT/2odOVByP+GdsDkS/0EBJHHI7mouWps56b+hlXWYYKJo96V7KsEIu6j9Y0ghwKle81NtYbmKK 27bEqDVQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tmsLr-0000000GrXV-2I96; Tue, 25 Feb 2025 10:39:47 +0000 Received: from mail-lf1-x136.google.com ([2a00:1450:4864:20::136]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tmreu-0000000Ghki-3KHp for linux-arm-kernel@lists.infradead.org; Tue, 25 Feb 2025 09:55:26 +0000 Received: by mail-lf1-x136.google.com with SMTP id 2adb3069b0e04-5454f00fc8dso5037286e87.0 for ; Tue, 25 Feb 2025 01:55:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1740477323; x=1741082123; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=hLPL0cxSmhQjzTuSazqL5ULZpLc4BXlPwpKFYiv7qfk=; b=QTkEhf98nw/pF8ILsFseCiGNDlsS/bV2FVUb2le7p8y6JmMVXuF1aMcU+XI7L13Td1 /ZOn9OXtGSqBVqr2fEXva0ktH9ZAPCfhZcqSZXjrSZUFN81sulva8GCONp70ct6FcnZs APjG+MgPh0yyQQQJj3FsedL64apFDy6L5GFEYpI9XjrNZV02v1euK8UMNHSZ4rxd6huS do1qsuMxO0J22dBSvGAco02q6wTIfEU+F5aNbdqomoFDP/GhT5kIWbBfgRtDhgouvxCG ++qotarEW3d2/fzpLedgB+vG4pt+KMicmn4ewpo60h1B9he1VZF8zFfMT5eOf3P75yxn qtzQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740477323; x=1741082123; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=hLPL0cxSmhQjzTuSazqL5ULZpLc4BXlPwpKFYiv7qfk=; b=irJM1U49Y+VwlRthd1mULU0rffgqojTm2xoeu9it7OjbmH6chqyEEbi8D/us0mwlOA notxggHBbtHaBq62ecuY6Ts2W1T0jKCsoEYqhet/oU8fkryomyTZn5fqRvD3SxPq65vW O74CP9NWHZdMzF5MJkX3ny4FywdKKzxEm3rwtGCT9RbSJx5/7ZhmINSF7GPbvrPINTZ7 nMo24FKlSpPmuF6VgHOPx1jcWMgXBhqyRHMXiI/Xu06NTkErkIS+NsmU+DvORoLLRB0Y hD1VjeGFKc7ruE5/i9ycyh5NoAJhuODHukuw+2bKnOv+5AuHhK2PEzVapuNkuDciG/9+ wgVA== X-Gm-Message-State: AOJu0YzvqlFpEPxI1pkMnMa7wRAPCdnL5vM+SKKCAGYbCVKpHQo+Vdrl NsQNPbkBhexf5g/t/1iHtZxy5xUWOF9xFPnskP3IgjQVlFKJGHj+wzN1jzFVKdY= X-Gm-Gg: ASbGncujuYjckm6daWNPOLBtw+g3HMnr/nliqtAKoF2dcNPTnv9JGk6IoedbM9R34O1 4XbgicR0c+DMRt1KfNaNeeD2UK7oRufdoDrpawFtKry3W32n3wboJuyvK0D2LCiP4NBDiYjskhd qQjEeHLMrzbj2L2p+/IQFnGYfberY2oDfLwl7otWxe53sDOMwnSYxOwqBDJGKM/P3FjS4r8NSog DkJTIIQpgwNSD2vREdBnArZbyDGiR3o+wSBsT7HHWs+cnMTuoprckgmS4EbUZUDvqmSddO21gEi uQsrutFdzSlmfIyWvWpWyd4MlZo8YjBj8nOS X-Google-Smtp-Source: AGHT+IG4gnndhO6d56pPyJzHQkvZAwyB0xsXUyc/MHVXGstc6/S0wQ2MFyd0xpUKVgQgFI8a8Tye1Q== X-Received: by 2002:a05:6512:2809:b0:545:a2f:22b4 with SMTP id 2adb3069b0e04-54838f5a9d3mr5459397e87.40.1740477323227; Tue, 25 Feb 2025 01:55:23 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id 2adb3069b0e04-548514b261esm132867e87.24.2025.02.25.01.55.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 25 Feb 2025 01:55:22 -0800 (PST) From: Linus Walleij Date: Tue, 25 Feb 2025 10:55:07 +0100 Subject: [PATCH v5 20/31] ARM: entry: Untangle ret_fast_syscall/to_user MIME-Version: 1.0 Message-Id: <20250225-arm-generic-entry-v5-20-2f02313653e5@linaro.org> References: <20250225-arm-generic-entry-v5-0-2f02313653e5@linaro.org> In-Reply-To: <20250225-arm-generic-entry-v5-0-2f02313653e5@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250225_015524_835021_4B528CAD X-CRM114-Status: GOOD ( 12.26 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The return to userspace is a bit hard to follow. To prepare the ground for further refactorings we rewrite the functions to ret_fast_syscall and ret_to_user are mostly stand-alone symbols. Signed-off-by: Linus Walleij --- arch/arm/kernel/entry-common.S | 25 +++++++++++++++---------- 1 file changed, 15 insertions(+), 10 deletions(-) diff --git a/arch/arm/kernel/entry-common.S b/arch/arm/kernel/entry-common.S index d27e34d7e97a079d62e7b778cf7cf97c302e4763..ece9215717143427de7572a18a6dbe620a9193e9 100644 --- a/arch/arm/kernel/entry-common.S +++ b/arch/arm/kernel/entry-common.S @@ -44,20 +44,23 @@ ret_fast_syscall: bl syscall_exit_to_user_mode ldr r1, [tsk, #TI_FLAGS] @ re-check for syscall tracing movs r1, r1, lsl #16 - beq no_work_pending - UNWIND(.fnend ) -ENDPROC(ret_fast_syscall) - - /* Slower path - fall through to work_pending */ + beq 2f tst r1, #_TIF_SYSCALL_WORK - beq slow_work_pending + beq 1f + b ret_to_user -slow_work_pending: - mov r0, sp @ 'regs' +1: mov r0, sp @ 'regs' bl do_work_pending - b no_work_pending + +2: asm_irqentry_exit_to_user_mode + +#ifdef CONFIG_GCC_PLUGIN_STACKLEAK + bl stackleak_erase_on_task_stack +#endif + restore_user_regs + UNWIND(.fnend ) ENDPROC(ret_fast_syscall) /* @@ -74,7 +77,9 @@ ENTRY(ret_to_user) ENTRY(ret_to_user_from_irq) ldr r1, [tsk, #TI_FLAGS] movs r1, r1, lsl #16 - bne slow_work_pending + beq no_work_pending + mov r0, sp @ 'regs' + bl do_work_pending no_work_pending: asm_irqentry_exit_to_user_mode From patchwork Tue Feb 25 09:55:08 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13989773 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6E0CBC021B2 for ; Tue, 25 Feb 2025 10:41:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=GFYj8M8sWUilipZ79moNvoJmNP0x1CtymAnRzXm2Cm0=; b=ZWLY2QN0FZmHEjN3axejJrtjWG QghXRVkYurfDvONLj5usq/tFEDY9svQGf4sQjuHsqAWSL88bfvF06KvFs3t4YOemQbTInF256zCsY 0IjgyKuctNUx5tqtl25h6BfFEwz2aKkuDBUS/oV2NaWq2x0eHj3ERyVr4s/oAlS9yHjl3LLoZApvw i+cJp99egBdrHyOklzjk9hw6wfu8yuRfYa2gX8Ls5ZRMrrZJb9ddDFIPGr15muhmltBD8yYx2qTPF 0Cv9l0eXgcz5EE0nv3YaajLrMKPw3yL6EWoif0yPq2OeX0Wn1M+meWFlQZm+aqitvlFoOQMgqx+an 2c6qqd8g==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tmsNO-0000000Grsz-18xD; Tue, 25 Feb 2025 10:41:22 +0000 Received: from mail-lf1-x129.google.com ([2a00:1450:4864:20::129]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tmrev-0000000GhlI-3pUz for linux-arm-kernel@lists.infradead.org; Tue, 25 Feb 2025 09:55:27 +0000 Received: by mail-lf1-x129.google.com with SMTP id 2adb3069b0e04-5484fa1401cso1187358e87.1 for ; Tue, 25 Feb 2025 01:55:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1740477324; x=1741082124; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=GFYj8M8sWUilipZ79moNvoJmNP0x1CtymAnRzXm2Cm0=; b=jQQUH2coHZJOElb/v/4RwhPiDlM4ahiJ4vU5JZ4m5E6qjmCiGq849UVPaokSaiZzJk B3JzUFE6vZ4EcEwtp51Az89OVVSAaGe8ZhTDNpOLLuuNbnGoeVOsKICNh51+dcDLlYiM wuO9Ov8K61uoevRbwbI6Zjbs7hd/SX+TdtdcdSCMjoiVklxDGddQXtmCuD/n+nqQAhzE yRvH5Zpq9s2WnmWsALBdaLjdchPGUB8DUgBqO63mRw/i6yu+YAfAQhNczZzBq+Lw7TX3 7QqHnOy7x29Aa4L1FTffjWIwULspTiOP8qzsbkYyVW+r+ps3ZPqmeoKXlbhPfYpntX9f C3kg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740477324; x=1741082124; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=GFYj8M8sWUilipZ79moNvoJmNP0x1CtymAnRzXm2Cm0=; b=m/tatdHrSxkWAUlcBmDz/Dt9aJAUxl8HW9Mzy+5Q7HbShT4afoYTPmI/w1XY3PTJkQ 1Id8ROmF+DrNqfViDv5AASKlInl7kNYtXkF3rVxXuns8QRotZPcJJ8Iju3P7NMuWhnh+ 1fRsbEJqHY1juxMz8MM58lSL1FdmQXcrI5qKftfPd1/1UQXOdLY8qZvyqL2OgcEXT4DG rcYTxuQjYuPmTqujnsqKF9TjmgM6eAd8qcRfI9WKNnm4xp+wSEIlX5ePm4MzIhjnyUb2 LDMDqUTzKyKxkcNHvYa904dO61sKRyNrBwqKJ35X8NUXNJhqMW9rDCHKPmm4B6HfJ0cn u2aQ== X-Gm-Message-State: AOJu0YzAsxwPbeMvBqDA98WxdqaCZpYR8EetvVAqp4ErE+N/9MGcCMw5 w2+L+KyK1HylIiqs5TYsWhIpq+4tosVHeJynbqUFr/TTM5xK5zsRDJtrMMyf9mM= X-Gm-Gg: ASbGncubm9bCId1SzGDoOUmmZ9wct1+OllMhQSd9hJWES4+kgFjE2fGKDpJmv6SdhZh w/G5ukygUPKN490Dux5ZNEKo4LTEPnRx6e4OmoEu5kg3bu92F7Z5W87ZgtEzQ03tBIqby0JrIiw v2Zw0SL8QyigBNeRsgFPn4VlXY54QA1L+MMWKk7zK9T8SGiVB2QMKOSmt5OG4/pBo6mNt3hRfJH Q/BQLg7LTXORfh29HPYS1lLQx71rYc12h/PjyM+ZUK5CciwCLLXvQ5cjCDD+LJ2Jw4dfInFXGPs KFK17cGAy1SJAlttR7WUIGw4PNdQinPLNpfw X-Google-Smtp-Source: AGHT+IEXZnOHohV1r4Mfp6ZZX3DbBx2GvFu2h5njrZLMFHCmGZ4MEwYftDaVvCGo6wOkCls5ru72YA== X-Received: by 2002:a05:6512:3e0d:b0:545:a89:4dc7 with SMTP id 2adb3069b0e04-54838f5ecf5mr6710865e87.52.1740477324315; Tue, 25 Feb 2025 01:55:24 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id 2adb3069b0e04-548514b261esm132867e87.24.2025.02.25.01.55.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 25 Feb 2025 01:55:23 -0800 (PST) From: Linus Walleij Date: Tue, 25 Feb 2025 10:55:08 +0100 Subject: [PATCH v5 21/31] ARM: entry: Do not double-call exit functions MIME-Version: 1.0 Message-Id: <20250225-arm-generic-entry-v5-21-2f02313653e5@linaro.org> References: <20250225-arm-generic-entry-v5-0-2f02313653e5@linaro.org> In-Reply-To: <20250225-arm-generic-entry-v5-0-2f02313653e5@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250225_015525_975494_18ABA99C X-CRM114-Status: GOOD ( 17.57 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org This is a semantic change to ret_fast_syscall: if there is no work pending the function used to jump to ret_to_user, instead we proceed to return from the syscall. If we jump to ret_to_user, IRQs are enabled and we call syscall_exit_to_user_mode a second time; at this point a this just means a second call to rseq_syscall(), but it seems wrong and we want syscall_exit_to_user_mode to be called exactly once. Apparently the ARM kernel has under some circumstances called rseq_syscall() twice on the exit to userspace without side effects. ret_to_user also checks for pending work a second time under ret_to_user_from_irq, but this isn't even needed any more on the fast return path: this was needed because the syscall invocation enabled interrupts when doing syscall tracing, and all tracing invocation has been moved over to C in earlier patches. Signed-off-by: Linus Walleij --- arch/arm/kernel/entry-common.S | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/arm/kernel/entry-common.S b/arch/arm/kernel/entry-common.S index ece9215717143427de7572a18a6dbe620a9193e9..33bc9e7c5b32f62474724f1d47f97af24dba53bf 100644 --- a/arch/arm/kernel/entry-common.S +++ b/arch/arm/kernel/entry-common.S @@ -49,7 +49,7 @@ ret_fast_syscall: tst r1, #_TIF_SYSCALL_WORK beq 1f - b ret_to_user + b 2f 1: mov r0, sp @ 'regs' bl do_work_pending From patchwork Tue Feb 25 09:55:09 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13989774 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 96AE5C021B6 for ; Tue, 25 Feb 2025 10:43:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=0l05ZDEo5bKH98lcf/z2wVpVbfg6jDUAnt9TMH5HSeU=; b=hNzAKLks9JveSW0W6Ojvnep733 VKOGWQp16vacovtA/Sun/3yb0s6Y4G1gKDqf6Lpm2GVzUaQ09nkvQW0hxJliOgyiHeO+QmOS92rme 4AXDzlp2t2L4MWDR7v04pqiKygZMJ0mbT2QDyucMEt3EhqmygyfzdH6WiBULR3Af8KwpW3GWHvps6 vfQ02tOds3lhoagOv/wEzEvBfWKraPWcSKaaiLB0xV655WDiXoVXWqXQisbZ6oM5dNJn43JTLRWn+ TD1Uff4kaB1zTmXFQMQfV5j1FHYQQPgNafAKqaHLGk/I4fn5mkuUboeeuPjxFZVJYmis/h3A0A8/u 5VjbroHQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tmsOt-0000000GsAv-49bi; Tue, 25 Feb 2025 10:42:55 +0000 Received: from mail-lf1-x132.google.com ([2a00:1450:4864:20::132]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tmrex-0000000Ghm9-02ed for linux-arm-kernel@lists.infradead.org; Tue, 25 Feb 2025 09:55:28 +0000 Received: by mail-lf1-x132.google.com with SMTP id 2adb3069b0e04-54524740032so5581796e87.3 for ; Tue, 25 Feb 2025 01:55:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1740477325; x=1741082125; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=0l05ZDEo5bKH98lcf/z2wVpVbfg6jDUAnt9TMH5HSeU=; b=DfG5eImTxMkxDZtW5AQrHcRb/dlqHxUQ4GqHqSHUQ0OauwaWwRD4LVfmTl5KGBEZIE 2e1s5QUcFeN+eHpX4ZoH4fAh7QS+FZiIWdDQqINerTIBI5cu+mHyQ5fnD3JSwY5zylQg 0CQD2W/o1rCdD+scvBlGdtxI/OaRbSKEFxgcDAwYtikmbhYBiohJEyBhUegVZyWIQSzB DhADMrxci9tBb2bscApFAaoAuyNp38VpD1fXs+3FjEvyB7SqdHJf1ffLLHVSjt+vFXSI GneUOUC4qMmn2mprC7qp1gHtO8Q7qpceJxRGILCbZRjsfr3bTBPnL9wm6tCJFjtszS5Y /95A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740477325; x=1741082125; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=0l05ZDEo5bKH98lcf/z2wVpVbfg6jDUAnt9TMH5HSeU=; b=G2SrudWHobJzp8MDYOO2rbXIeLQJSJNN8cucp5euQSnRuzKegR3QJLntIC/6zctHws 4eJdFGh4OspC3wdDsEm4n4q7qqeT/ebSDEqNgoAf85LbuGhw9+rN/+wAVsHOmn8Ug02I tDoRbsLBmmGTE2NR/gGcu+TYBWsvnxqZgag6POaxhhzwu/ImMrFsr8BZgGOve2zLio5e NF8h30kyvrgETTGNAWHF7Xwb9jRu2U0pxjf3Fjo0A9YB88E8IRfuD+swpELVg8wmz5JN 4IkWZezQir0xP1B/930Z8IL9oNR81+pRz/alswMd+BdFVy4PEa/uhA4YSf7UvJhcApby h12A== X-Gm-Message-State: AOJu0YzwvstvWRJpTkXKr7PnZTMcAJRZPhzOczvONWQ02foJZyGkMGvG U0Kif+ZhyjQ4ShpuxVyD53LHvgIxYgHoEx9Xd0RatCPyiSdL689JJwwDo8Fd2KQ= X-Gm-Gg: ASbGncvdmxEB650iis7O+stNLhHB7NPvDKmjVOzYNQ8N0gbvPICzaVajahdMMw/NG/J oGO5c+CVgU7tnDX9fnNkIs/wVV92aDCRrDxbzC1NBCdl4C3UGpOXs3Ko04om19GavTrmt7gohTr gNNbkNa3TJVn1Vf6R3PJKsNyab4GzVd9d6OXGN9YGVslqIvSnSXJT9SRZ2LlTzloTw4qdVC3b1I d1dAA6nmhol7OQEakjp7OC+3TzMiksBIVcgQm3rBZ1SbesLk1Z6Ive3uJ1M9NX6QQ9EuctYOpGD TB2W0X77CU2BsVS9FuGGQ38WSjlKVxn7DCET X-Google-Smtp-Source: AGHT+IGv9PupW5hC43RAbr/CGZZGm/FFBN/oPYbjQj0aMhZodLXr8EOafbq0GSLZY9dixj7FM743WQ== X-Received: by 2002:a05:6512:15a7:b0:545:a2f:22ba with SMTP id 2adb3069b0e04-548510eb0b2mr957604e87.37.1740477325442; Tue, 25 Feb 2025 01:55:25 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id 2adb3069b0e04-548514b261esm132867e87.24.2025.02.25.01.55.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 25 Feb 2025 01:55:24 -0800 (PST) From: Linus Walleij Date: Tue, 25 Feb 2025 10:55:09 +0100 Subject: [PATCH v5 22/31] ARM: entry: Move work processing to C MIME-Version: 1.0 Message-Id: <20250225-arm-generic-entry-v5-22-2f02313653e5@linaro.org> References: <20250225-arm-generic-entry-v5-0-2f02313653e5@linaro.org> In-Reply-To: <20250225-arm-generic-entry-v5-0-2f02313653e5@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250225_015527_067746_0CCFC08F X-CRM114-Status: GOOD ( 14.30 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org At the end of a syscall software interrupt and every time we exit an interrupt handler we check if there is work to do, i.e. if the current thread (userspace or kernel daemon) has set the _TIF_SYSCALL_WORK flag. Move this processing over to the C entry code. Signed-off-by: Linus Walleij --- arch/arm/kernel/entry-common.S | 14 +------------- arch/arm/kernel/entry.c | 10 ++++++++++ 2 files changed, 11 insertions(+), 13 deletions(-) diff --git a/arch/arm/kernel/entry-common.S b/arch/arm/kernel/entry-common.S index 33bc9e7c5b32f62474724f1d47f97af24dba53bf..5385574c4339714f3d8b5ab6894f27466b89f932 100644 --- a/arch/arm/kernel/entry-common.S +++ b/arch/arm/kernel/entry-common.S @@ -42,19 +42,7 @@ ret_fast_syscall: /* do_rseq_syscall needs interrupts enabled. */ mov r0, sp @ 'regs' bl syscall_exit_to_user_mode - ldr r1, [tsk, #TI_FLAGS] @ re-check for syscall tracing - movs r1, r1, lsl #16 - beq 2f - - tst r1, #_TIF_SYSCALL_WORK - beq 1f - - b 2f - -1: mov r0, sp @ 'regs' - bl do_work_pending - -2: asm_irqentry_exit_to_user_mode + asm_irqentry_exit_to_user_mode #ifdef CONFIG_GCC_PLUGIN_STACKLEAK bl stackleak_erase_on_task_stack diff --git a/arch/arm/kernel/entry.c b/arch/arm/kernel/entry.c index 47233804733752aef0f2fec6f54badeab18e98d0..0e3960844b94b14325eb00ce7dd791b36c43ac3c 100644 --- a/arch/arm/kernel/entry.c +++ b/arch/arm/kernel/entry.c @@ -1,10 +1,16 @@ // SPDX-License-Identifier: GPL-2.0 #include +#include #include #include #include #include +static inline bool has_syscall_work(unsigned long flags) +{ + return unlikely(flags & _TIF_SYSCALL_WORK); +} + long syscall_enter_from_user_mode(struct pt_regs *regs, long syscall) { trace_hardirqs_on(); @@ -18,8 +24,12 @@ long syscall_enter_from_user_mode(struct pt_regs *regs, long syscall) void syscall_exit_to_user_mode(struct pt_regs *regs) { + unsigned long flags = read_thread_flags(); + rseq_syscall(regs); local_irq_disable(); + if (has_syscall_work(flags)) + do_work_pending(regs, flags); } noinstr void irqentry_enter_from_user_mode(struct pt_regs *regs) From patchwork Tue Feb 25 09:55:10 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13989775 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 7B95BC021B2 for ; Tue, 25 Feb 2025 10:44:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=XZE98juq0oGv0Xstj1U7TzgmKhYjLwozMlzxeXVPHlA=; b=n9LY9tlOahoWaF2JGeW8h0DmYa 9FHSFA4tkBAyrIJsU22lklXBPHbyb+RAJYneFK8TAirQF4MfhDY/XNGMEznbIlrs9MfLin36+2Lew jzqfIlbPAHBgOnpdan8E26BovfrWE2XrNrzHvxxATlK8FF/z/H4Hx6vern/N30ZBA2YY/XhnsOgND HQv3u+BJCqnGrUjPbpd5omdri4P3pFFjbavuxduqlMyK5UR+9GDOIYdqsvs370JCIvf7WPkALNohP 0MSrrR+3rTDPThf0aQFPl4Yh8aYoWWIBaGw90DiBDZtsXWp4/0X1keKJsfTwabjOTbN6b2YebUTtR e9RXhL3Q==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tmsQQ-0000000GsWH-3GAy; Tue, 25 Feb 2025 10:44:30 +0000 Received: from mail-lf1-x135.google.com ([2a00:1450:4864:20::135]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tmrey-0000000Ghme-1SxV for linux-arm-kernel@lists.infradead.org; Tue, 25 Feb 2025 09:55:29 +0000 Received: by mail-lf1-x135.google.com with SMTP id 2adb3069b0e04-5452c29bacfso5856803e87.3 for ; Tue, 25 Feb 2025 01:55:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1740477326; x=1741082126; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=XZE98juq0oGv0Xstj1U7TzgmKhYjLwozMlzxeXVPHlA=; b=obPVdw/2BZlFP2FtsbREDcjnAz9IJkaWOhanYqE1yagphIE266fpVVt3UKBTyyDqfm a5bxf+S9nYmC7eCwJ536Hs8JqGbT0Fb5SoYdr4k09DUk4rsaUXvraJLj5G3Q/vZ8kxxX 1OPdn4qD/qqdwaSvXKjKJKHS2WEvdv/jJwKzIVigPArd6YCnbJbwooDtylOni44Q86PO IAIXptH76+BUGAO4X81tZdf4rkeEnyVR/A5ursLVkX8zkGcopPsoYR1qoa+Bo+K15mrg hH2xPajW4PcGz3czQh/box+bPdQ+MjLuJH5ZVICOXfrGwfLebT4SYVYkugpDwvjezOCF zymA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740477326; x=1741082126; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=XZE98juq0oGv0Xstj1U7TzgmKhYjLwozMlzxeXVPHlA=; b=akb5b9f8pdIR6X0Mwxum0KHTtww1iFIZPZ83cT9Kv5cvlSqnGj4iTwX7lNzIJZZsnU fS/9x1F7u2Kg9cfPSLcoozkrnW3gGWZJqX8M9kYRqGmoxog8785SLwQjPeZ66jC/niWu 3Izqn7MZGRNmj9Su0BPr1WDYSj4TJJPbAzx307QHqv6nfgh4HGTK2GOH/zPkD1yTvmQ0 /0RgD9Ep8vVRd7/5mZvOb2fwB4tppF00SUzmcCVxDlg1mlsnA5BHCiyVODo5v1G3InEB fpkyTByPthQ859wsdIo5GWa7fAwBBxfTnAdYgQ+E4I1ghRLxnbxUo+VBdF1a7GjrtX7I LPHw== X-Gm-Message-State: AOJu0YyFqT7LsjL7yIkkUI7o+TBmzKUQUpr076YIdG6JqyHlsb4V1dSv CbC1Xi1HQFwqmZ8ozzzdMeitDk0fJhifDRT+GE/oGEHcfV13ds/94d90SW/CeO4= X-Gm-Gg: ASbGncvXHy6aA5+b4Flkm5HwikmzxiTXCet/65/1b9oMCfkiSbN6gh4sFn3iFEE1qsO 8yI88poQ3jG2kNDD0qg9wq3imEXeucUetXCetHNl4d+Q9wKJonSQ2EneVb3joW531pp+cOm3QUK lIY7PF4rezZlI/fh/MrILYQLpRrK26r/Zp9mvLnM7vjYez/g9Rff5TU6+aVRBjvs6/nSMt8OeIr zMe/JHcp/qDmvj8+mIdotcv42vo3q+bT4WJHijc0noSVtXMkIxOMdTDTVXi8eb8n6unxlHYPZzd GR3U1DMvDJSsOboNd4zGEhkf3KwpkJyFLsBj X-Google-Smtp-Source: AGHT+IHv2LyiSeosodUrcS8c6R5xpQiyNyga8e1Qfb63OSq/UTKN+o0/2yMM03TOk0Z2Siw/VpkP2Q== X-Received: by 2002:a05:6512:3b4:b0:548:d4e0:82bd with SMTP id 2adb3069b0e04-548d4e08399mr321337e87.17.1740477326469; Tue, 25 Feb 2025 01:55:26 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id 2adb3069b0e04-548514b261esm132867e87.24.2025.02.25.01.55.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 25 Feb 2025 01:55:25 -0800 (PST) From: Linus Walleij Date: Tue, 25 Feb 2025 10:55:10 +0100 Subject: [PATCH v5 23/31] ARM: entry: Stop exiting syscalls like IRQs MIME-Version: 1.0 Message-Id: <20250225-arm-generic-entry-v5-23-2f02313653e5@linaro.org> References: <20250225-arm-generic-entry-v5-0-2f02313653e5@linaro.org> In-Reply-To: <20250225-arm-generic-entry-v5-0-2f02313653e5@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250225_015528_391208_BF375097 X-CRM114-Status: GOOD ( 15.02 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The syscalls are issued as software interrupts, and to this point they were sharing code with the other exception handlers, but this does not work with generic entry. Make syscall_exit_to_user_mode do what irqentry_exit_to_user_mode does at it's tail, and drop the invocations of irqentry_exit_to_user_mode from the syscall path so that these are now exception-exclusive. Split ret_to_user and ret_to_user_from_irq into two distinct execution paths. Signed-off-by: Linus Walleij --- arch/arm/kernel/entry-common.S | 14 ++++++++++++-- arch/arm/kernel/entry.c | 4 ++++ 2 files changed, 16 insertions(+), 2 deletions(-) diff --git a/arch/arm/kernel/entry-common.S b/arch/arm/kernel/entry-common.S index 5385574c4339714f3d8b5ab6894f27466b89f932..e2ac6d3216b6828d948d60c424ff95ccdad12331 100644 --- a/arch/arm/kernel/entry-common.S +++ b/arch/arm/kernel/entry-common.S @@ -42,7 +42,6 @@ ret_fast_syscall: /* do_rseq_syscall needs interrupts enabled. */ mov r0, sp @ 'regs' bl syscall_exit_to_user_mode - asm_irqentry_exit_to_user_mode #ifdef CONFIG_GCC_PLUGIN_STACKLEAK bl stackleak_erase_on_task_stack @@ -62,6 +61,18 @@ ENTRY(ret_to_user) enable_irq_notrace @ enable interrupts mov r0, sp @ 'regs' bl syscall_exit_to_user_mode + ldr r1, [tsk, #TI_FLAGS] + movs r1, r1, lsl #16 + beq 1f + mov r0, sp @ 'regs' + bl do_work_pending +1: +#ifdef CONFIG_GCC_PLUGIN_STACKLEAK + bl stackleak_erase_on_task_stack +#endif + restore_user_regs +ENDPROC(ret_to_user) + ENTRY(ret_to_user_from_irq) ldr r1, [tsk, #TI_FLAGS] movs r1, r1, lsl #16 @@ -76,7 +87,6 @@ no_work_pending: #endif restore_user_regs ENDPROC(ret_to_user_from_irq) -ENDPROC(ret_to_user) /* * This is how we return from a fork. diff --git a/arch/arm/kernel/entry.c b/arch/arm/kernel/entry.c index 0e3960844b94b14325eb00ce7dd791b36c43ac3c..88a7a699306ab61153d57333c26266b9161f5466 100644 --- a/arch/arm/kernel/entry.c +++ b/arch/arm/kernel/entry.c @@ -30,6 +30,10 @@ void syscall_exit_to_user_mode(struct pt_regs *regs) local_irq_disable(); if (has_syscall_work(flags)) do_work_pending(regs, flags); + + trace_hardirqs_on(); + /* This context tracking call has inverse naming */ + user_enter_callable(); } noinstr void irqentry_enter_from_user_mode(struct pt_regs *regs) From patchwork Tue Feb 25 09:55:11 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13989778 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 89807C021B2 for ; Tue, 25 Feb 2025 10:46:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=B1th+DnYZSKT2go08vz+Ftvr7FlZ7mKXRvbFUdOlbPI=; b=cpsT5RDPu4UzbZbiZt9sfHETkI w98Pw+COijkHYhED6Lk2++nSO8/nHWyiUIFYoAhS8T3Cqf0sQvkGjY1cFrkVZV7Zgn5PgxNvqt9qi iZCiT2ICxuqcOq/3E/MhQArqe6b7lQxNGXMdrUp4mPrR4VzK6D2TqsZATTmjCNZIdfsoHKNJ5Hjcy 56XbBYjG+hCLL8Njr82j3wqBH//EsUt14VVqqHg8khvrNi3wf8GSF4gEg/YG555Zd6GMv60YjKc7G 20PzBxlGxSsSTHEEVvt+eBpsXlP0v5FIyE7KAB+L+iwFUZ+LHv2RgZl1rjnDC7i+QnwrSpGS0eam1 MJw6WtOw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tmsRx-0000000Gt1P-1ofG; Tue, 25 Feb 2025 10:46:05 +0000 Received: from mail-lj1-x22c.google.com ([2a00:1450:4864:20::22c]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tmrez-0000000Ghmm-0NWE for linux-arm-kernel@lists.infradead.org; Tue, 25 Feb 2025 09:55:30 +0000 Received: by mail-lj1-x22c.google.com with SMTP id 38308e7fff4ca-30a29f4bd43so48187631fa.0 for ; Tue, 25 Feb 2025 01:55:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1740477327; x=1741082127; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=B1th+DnYZSKT2go08vz+Ftvr7FlZ7mKXRvbFUdOlbPI=; b=LQ6g5KnekaTGixcMorl0H5d5lAtszkDRsen0MbFmhf/a+imd5dmMkL4/9kZIFOgDUK 2EThOrr2WFzgVtvC6pLDm9EH1ppivqWUnmxSD4xCLQYQpOomEJoHu+1AsuivbtxVIr+C wvAG6Bxg9Wj+5KP1LQJCTvTFp7yO2E2YCU1kwX88xv9f1XJYDf1npJAC2QyyAkXrKI50 /Y3Y63DFM2bazs4CRyH8Rj/6+scoU95AZXXR0ff4oT9mHMFy33v3s/EE8MCKWv5cDJGM J48f+VnCGjqQzh7IxZcrKmbyDIVQvWdYJtLuY02vjcobPx81wvzUa7N+qALOAtvzR1YW vCRg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740477327; x=1741082127; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=B1th+DnYZSKT2go08vz+Ftvr7FlZ7mKXRvbFUdOlbPI=; b=lUsAdzL9dRnfWLf+QLZV/67J8jEvMMlAyPMDfVsr5Vu8AvZ869nLgVHMi+U/FRCfmE HKj/kUiJ9+KzWNLYL4hMd5B0UUUDVA2T13+YjuRJfXI3zNCwgOL/8EPQMJFJ3YgsEmgn y8DovnHe54knvujZExhRvOoUjfXx5fwFAvz35zI+fkEQehQmZas2bljQW82wQcyhqIC0 uVdeG4o0qK8Z8hNfdop2OAtLgyqFbqcPKEygeFxsNXpRIGQ3TA4YL0XttHXectRTckpe RhzPiF8szW/RCLTwvD6XcVQrOlizSyPg3O8SNp9n/TryGMDEX/F7QPuZ7lecV2kATf18 1M2g== X-Gm-Message-State: AOJu0YxJ2vm516wpMyqwFP68hEyD4ysQ11ACfaBCkzbA0XREXcwQWw9P x1xDunHzpZFNitOgEo+qsTJwzPCUfRJXZ0fUtXQp4SsZyaYOt7wQ0FflzzkhZAM= X-Gm-Gg: ASbGncv0emyCXX9rzzR829WPwpdQKpALggXDpaK6hfDFS1SdWpNC+UZtzGSN07ao4Yk GEQiZl8OewkvjSGFIOxw6ecVNUF/GupuXTIZXmdd4ghbF661QHJ2cdr09MyBL+OWB0xI8tlmBB9 HmCTAZU3mkqQS2MfmgIMXPbzRYNwJI88Ajd8wXbQr9FFwuv88CsvcmNG64x9qMBDiqqSLZFMM8G 2DR/wTwfVor0cTTqpfYc4Ss00q7OCM36HkfQNTLJ/+XbSgVXBWHj4pZR8BJKoxUVUnCtU4fmMPW T2VSsThEJLwypRHAywOuGpbuIUvaI2HyUWwb X-Google-Smtp-Source: AGHT+IFJojS9YcdzZoZZ2a4fWY6El/OS+wlvMSzdOOSRtg7jEpLZV7xetDwt8bSZ007hZm6HmPnT8g== X-Received: by 2002:a05:6512:3b86:b0:545:fad:a747 with SMTP id 2adb3069b0e04-54838ee28e2mr6423382e87.5.1740477327533; Tue, 25 Feb 2025 01:55:27 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id 2adb3069b0e04-548514b261esm132867e87.24.2025.02.25.01.55.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 25 Feb 2025 01:55:26 -0800 (PST) From: Linus Walleij Date: Tue, 25 Feb 2025 10:55:11 +0100 Subject: [PATCH v5 24/31] ARM: entry: Complete syscall and IRQ transition to C MIME-Version: 1.0 Message-Id: <20250225-arm-generic-entry-v5-24-2f02313653e5@linaro.org> References: <20250225-arm-generic-entry-v5-0-2f02313653e5@linaro.org> In-Reply-To: <20250225-arm-generic-entry-v5-0-2f02313653e5@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250225_015529_135002_50DC180E X-CRM114-Status: GOOD ( 17.36 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org This moves over the last few lines of assembly to C. The subtle change is that in return to userspace from syscall (SWI) or interrupt, we need to call do_work_pending() as soon as the thread flags are != 0, just checking for work with _TIF_SYSCALL_WORK is not enough (the machine will freeze if we do that). This is because do_work_pending() does not just handle work: it handles _TIF_NEED_RESCHED, _TIF_SIGPENDING, _TIF_NOTIFY_SIGNAL and _TIF_UPROBE as well. Signed-off-by: Linus Walleij --- arch/arm/kernel/entry-common.S | 13 +------------ arch/arm/kernel/entry.c | 19 +++++++++++++------ arch/arm/kernel/signal.c | 3 +-- 3 files changed, 15 insertions(+), 20 deletions(-) diff --git a/arch/arm/kernel/entry-common.S b/arch/arm/kernel/entry-common.S index e2ac6d3216b6828d948d60c424ff95ccdad12331..6b0f86786a7d9e656ecbe8c741f2edbcbafd2915 100644 --- a/arch/arm/kernel/entry-common.S +++ b/arch/arm/kernel/entry-common.S @@ -61,12 +61,7 @@ ENTRY(ret_to_user) enable_irq_notrace @ enable interrupts mov r0, sp @ 'regs' bl syscall_exit_to_user_mode - ldr r1, [tsk, #TI_FLAGS] - movs r1, r1, lsl #16 - beq 1f - mov r0, sp @ 'regs' - bl do_work_pending -1: + #ifdef CONFIG_GCC_PLUGIN_STACKLEAK bl stackleak_erase_on_task_stack #endif @@ -74,12 +69,6 @@ ENTRY(ret_to_user) ENDPROC(ret_to_user) ENTRY(ret_to_user_from_irq) - ldr r1, [tsk, #TI_FLAGS] - movs r1, r1, lsl #16 - beq no_work_pending - mov r0, sp @ 'regs' - bl do_work_pending -no_work_pending: asm_irqentry_exit_to_user_mode #ifdef CONFIG_GCC_PLUGIN_STACKLEAK diff --git a/arch/arm/kernel/entry.c b/arch/arm/kernel/entry.c index 88a7a699306ab61153d57333c26266b9161f5466..d7fdb9df33312dd2a914b59512b701748f33b597 100644 --- a/arch/arm/kernel/entry.c +++ b/arch/arm/kernel/entry.c @@ -6,11 +6,6 @@ #include #include -static inline bool has_syscall_work(unsigned long flags) -{ - return unlikely(flags & _TIF_SYSCALL_WORK); -} - long syscall_enter_from_user_mode(struct pt_regs *regs, long syscall) { trace_hardirqs_on(); @@ -28,7 +23,11 @@ void syscall_exit_to_user_mode(struct pt_regs *regs) rseq_syscall(regs); local_irq_disable(); - if (has_syscall_work(flags)) + /* + * It really matters that we check for flags != 0 and not + * just for pending work here! + */ + if (flags) do_work_pending(regs, flags); trace_hardirqs_on(); @@ -45,6 +44,14 @@ noinstr void irqentry_enter_from_user_mode(struct pt_regs *regs) noinstr void irqentry_exit_to_user_mode(struct pt_regs *regs) { + unsigned long flags = read_thread_flags(); + + /* + * It really matters that we check for flags != 0 and not + * just for pending work here! + */ + if (flags) + do_work_pending(regs, flags); trace_hardirqs_on(); /* This context tracking call has inverse naming */ user_enter_callable(); diff --git a/arch/arm/kernel/signal.c b/arch/arm/kernel/signal.c index 03b20637a2e1ca6f327223c9132915a97a9c4697..a39ee14ec5b70b3738ecef537709326ef1e8217c 100644 --- a/arch/arm/kernel/signal.c +++ b/arch/arm/kernel/signal.c @@ -598,8 +598,7 @@ static void arch_do_signal_or_restart(struct pt_regs *regs) return; } -asmlinkage void -do_work_pending(struct pt_regs *regs, unsigned int thread_flags) +void do_work_pending(struct pt_regs *regs, unsigned int thread_flags) { /* * The assembly code enters us with IRQs off, but it hasn't From patchwork Tue Feb 25 09:55:12 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13989779 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0DB8EC021B6 for ; Tue, 25 Feb 2025 10:47:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=NA+LzTmhZnGJ5F/6+si6C2L1JDWL6V8/wsE3A9CcZck=; b=xzMpDONZqMiYg9hBX+UGLbcBan qsmSQ0Ov/vmVeyP0Bt2xbjpxS0zKp2LZz/oLdjRlOdbnFzof/kRgEGgT0b2LsKq69DF0hoRSRGwur XfHqbBon8aLMw1z+56AElPWQUYG2F4OuN+FnJapqssqq3fAMqSjvxea9AOYQYCNnMIJTg+xur0rAd 4N5rrS6T2COiksDejZqcvtBq606Puvhs4h1XNnqNEtN840uliuFvkBNjgXZxiP57WA6FgVm8uA/Q7 8LwlF3ISkr8A1Kw+wMiIhS1Wk72owJ2uVLvPZlHi4QFXCMnrsC5/cHTfO/3qIwvYv9hDn3anlbtFS K5yTuN+w==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tmsTT-0000000GtNk-0KXb; Tue, 25 Feb 2025 10:47:39 +0000 Received: from mail-lf1-x12a.google.com ([2a00:1450:4864:20::12a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tmrf0-0000000GhnH-23Ks for linux-arm-kernel@lists.infradead.org; Tue, 25 Feb 2025 09:55:31 +0000 Received: by mail-lf1-x12a.google.com with SMTP id 2adb3069b0e04-5439a6179a7so5997864e87.1 for ; Tue, 25 Feb 2025 01:55:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1740477329; x=1741082129; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=NA+LzTmhZnGJ5F/6+si6C2L1JDWL6V8/wsE3A9CcZck=; b=WeSMfFGNoFBkRM8Db61iTYCHL/MXoXicjrZoAU8+uhYrgHGxWqeYrP/INSieoIhwsC EtkQOb5KjLS2Pdxsi0PNKLUeJBIkWGBvOK7yzo8A7rzPemZJNnR4Uf3B0Lwkh+9Xmff2 6dsB05HrHabKHqCp+uwsdoMJyTBCASYjAuJ0oKSBUSsUSp0VkBNWZshtbgjsFatGKJ37 LGQgYLhtWtrVhiRiBsFN5ygsf1yVCbzuQc2uqvazWJT8uuF3N5asTUaDsoo5N7v2v0vP lrcHIqYx3COKHJLoR5NfDc4VhngwYN5f1dz02wiNFnpUWqiHsaEjMyrqNq6jhIMHWNrC 0lNA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740477329; x=1741082129; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=NA+LzTmhZnGJ5F/6+si6C2L1JDWL6V8/wsE3A9CcZck=; b=JCXUreC7L68ojeJewPc4Nw+fsy9niYEM3WyXnAS6Ffa3ROQxHWKmqnfSD2aEw87GgP GL5afydfkagvveqSpMOT9IBvJf7i28W9lW84X3+ucqLkcpXPTRmBVow0QMitbEnnFjEY 7aALp64VNda7yGm60RgQ0x2k1XweJ24W4TKLFC9zIRCAzoHtLbjQYvJ3XHZb6gssmBsb iblPZEkffdUbKCdqqQu9Pdj9kPPpp4dRuW+xgy25d183wUkQi9yItXmCJ3Fl5BrVL2DB yAA5xgrLLdqpZ9Qtuidr4cLM6EJTZKrb/ep6YL3a/1MJanYdwa4sd8B0rJrjoRthkkpQ HYLw== X-Gm-Message-State: AOJu0Ywe2Vs1TQSK3PFosLn+yEKM96lnO6rpjBehshw7q64qvZXs3Q9L s7EnjsO19uclk96apUerLIU8Rm+sD9j8YaLMEIcnRw3YclEINa7CLgHxj1Zp6Co= X-Gm-Gg: ASbGncv1iMQsPiRp1GpZUdFLlJEvFK8fbs6dpZUhprdD4waXZktfDqyiAauJPa+6wO0 vXzT97mYPehhgbngKQ1hMtAHmZWJaW8GNIv4fa8AHmpKtbokCAXurWXnWCov+baWns72i2TXRMx 76qTtSAVFjUIYiz37vzN0ve6y77s6hXjm01PK4DaBblagW9547Omy6+205qgw6uIXfJZrHbYRfQ XdT01BHLIz+XsTcsCoodmBHUsN20EIYMCBPuxmJRh9bKoHbZaPXiF2hmYG5FKRxDQ2XnsFrejYs L2bcHomd/ZxSzkl8lt2NofW3p6XGJCsXWwLE X-Google-Smtp-Source: AGHT+IHoExRUiD3Ptt8xtogX36HpctNeFu9+M3wDkC7ot0V3ilLsDYUXt+l8IOflXZFx+RQVrnavqw== X-Received: by 2002:a05:6512:6c5:b0:546:2ff7:530d with SMTP id 2adb3069b0e04-546e5f76a64mr6495854e87.25.1740477328590; Tue, 25 Feb 2025 01:55:28 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id 2adb3069b0e04-548514b261esm132867e87.24.2025.02.25.01.55.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 25 Feb 2025 01:55:28 -0800 (PST) From: Linus Walleij Date: Tue, 25 Feb 2025 10:55:12 +0100 Subject: [PATCH v5 25/31] ARM: entry: Create irqentry calls from kernel mode MIME-Version: 1.0 Message-Id: <20250225-arm-generic-entry-v5-25-2f02313653e5@linaro.org> References: <20250225-arm-generic-entry-v5-0-2f02313653e5@linaro.org> In-Reply-To: <20250225-arm-generic-entry-v5-0-2f02313653e5@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250225_015530_533690_583A3E27 X-CRM114-Status: GOOD ( 14.30 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org This adds a few calls to C when exceptions enter/exit from the kernel itself, so these can be used for generic entry. Signed-off-by: Linus Walleij --- arch/arm/include/asm/entry.h | 2 ++ arch/arm/kernel/entry-armv.S | 3 +++ arch/arm/kernel/entry-header.S | 5 +++++ arch/arm/kernel/entry.c | 8 ++++++++ 4 files changed, 18 insertions(+) diff --git a/arch/arm/include/asm/entry.h b/arch/arm/include/asm/entry.h index 8b8cc9a1eefbd00dd75d4e344576fc184a413768..a78bc5054b093120e903f08369fb8b1a9feb7f81 100644 --- a/arch/arm/include/asm/entry.h +++ b/arch/arm/include/asm/entry.h @@ -12,5 +12,7 @@ long syscall_enter_from_user_mode(struct pt_regs *regs, long); void syscall_exit_to_user_mode(struct pt_regs *regs); void irqentry_enter_from_user_mode(struct pt_regs *regs); void irqentry_exit_to_user_mode(struct pt_regs *regs); +void irqentry_enter_from_kernel_mode(struct pt_regs *regs); +void irqentry_exit_to_kernel_mode(struct pt_regs *regs); #endif /* __ASM_ENTRY_H__ */ diff --git a/arch/arm/kernel/entry-armv.S b/arch/arm/kernel/entry-armv.S index 3cd5a80b47758867d1db6c3971030370bca1551a..f64c4cc8beda6bcd469e6ff1a1f337d52dbbaf9c 100644 --- a/arch/arm/kernel/entry-armv.S +++ b/arch/arm/kernel/entry-armv.S @@ -205,6 +205,9 @@ ENDPROC(__und_invalid) get_thread_info tsk uaccess_entry tsk, r0, r1, r2, \uaccess + mov r0, sp @ 'regs' + bl irqentry_enter_from_kernel_mode + .if \trace #ifdef CONFIG_TRACE_IRQFLAGS bl trace_hardirqs_off diff --git a/arch/arm/kernel/entry-header.S b/arch/arm/kernel/entry-header.S index 51c7f93c1ded1db56f173c7aaed8094ef011ff5a..49a9c5cf6fd5fbb917f2ada6c0d6cc400b7d3fb3 100644 --- a/arch/arm/kernel/entry-header.S +++ b/arch/arm/kernel/entry-header.S @@ -210,6 +210,7 @@ .else @ IRQs off again before pulling preserved data off the stack disable_irq_notrace + #ifdef CONFIG_TRACE_IRQFLAGS tst \rpsr, #PSR_I_BIT bleq trace_hardirqs_on @@ -217,6 +218,10 @@ blne trace_hardirqs_off #endif .endif + + mov r0, sp @ 'regs' + bl irqentry_exit_to_kernel_mode + uaccess_exit tsk, r0, r1 #ifndef CONFIG_THUMB2_KERNEL diff --git a/arch/arm/kernel/entry.c b/arch/arm/kernel/entry.c index d7fdb9df33312dd2a914b59512b701748f33b597..674b5adcec0001b7d075d6936bfb4e318cb7ce74 100644 --- a/arch/arm/kernel/entry.c +++ b/arch/arm/kernel/entry.c @@ -56,3 +56,11 @@ noinstr void irqentry_exit_to_user_mode(struct pt_regs *regs) /* This context tracking call has inverse naming */ user_enter_callable(); } + +noinstr void irqentry_enter_from_kernel_mode(struct pt_regs *regs) +{ +} + +noinstr void irqentry_exit_to_kernel_mode(struct pt_regs *regs) +{ +} From patchwork Tue Feb 25 09:55:13 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13989780 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id EE5A8C021BB for ; Tue, 25 Feb 2025 10:49:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=8L1+xbNej+7+BbJW1uipXHwDEUpYT2HP3XoeW+FBwgo=; b=w1r91g/2uyH+MAKVJIq1dv7Y0U ssT7d8saqX0IJZ6SbP9tf7tL524AzpsEPR2Q8nDjmd7MXqlK4Q3l1d4mcV4IMlxIjcAhvuR7IzwBM yucze+zOCTUCpE+8fVNnx1RdX7/qS0lgtgQc5u04F0tuJIeGeFOJ1TOfSgrswrRcSjcfJkodBmhXw 9a4U5KKO7sjIIYHBxdTMd+VWjuqO8AV9dD1vOlCPx7Bo18ATOsOc62kGJv+6a7eycXzaC1Kd5bgzh tXNmCVEOPfY+vgMn9oaV51ydmQCiXX/bpk4TDWWDqCrqDQfgvKgR24bSGuaeveCvXm6QcHfdid3qS 9R31u3yw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tmsUz-0000000Gte8-2yA1; Tue, 25 Feb 2025 10:49:13 +0000 Received: from mail-lf1-x136.google.com ([2a00:1450:4864:20::136]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tmrf1-0000000GhnS-0d1h for linux-arm-kernel@lists.infradead.org; Tue, 25 Feb 2025 09:55:32 +0000 Received: by mail-lf1-x136.google.com with SMTP id 2adb3069b0e04-5462a2b9dedso6456200e87.1 for ; Tue, 25 Feb 2025 01:55:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1740477330; x=1741082130; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=8L1+xbNej+7+BbJW1uipXHwDEUpYT2HP3XoeW+FBwgo=; b=Wv2nNif+G2HlXR3ZELkDFFclGr2a4y14S0BLMa0O/O+sM0MzT2BMB+f8uAD/OUeQ98 deYfY8rMWczD1ZgXRsMceHqvXknLANzJ0/Bkl2DmvJQ4mVvcgSAiJGKU1ZAldLjxGzGh vejBWwBLpwGRUj9uvnE42A1lfwx7aQKO/kJT2oWdzvKVM20JX12xDdoPATfx9uToYUUp R8EskTvb3eYE2b1xqv7dLJb5De7DdndkwULBlnNcrF4ozGYCfBf7EYnt0O2yyqG/mrCW nO9Xy6TlOjnQgkVK4PdiplLAgUBXizVQ1V1n13J/sIcgGcJ/gyj6luVrzn0Mwcde79qS VoEQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740477330; x=1741082130; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=8L1+xbNej+7+BbJW1uipXHwDEUpYT2HP3XoeW+FBwgo=; b=LGJx7tD55sQJm3ixVuUsDZdt1edeQ7TNCkLgyiMB4PecKaPYtjGea31KJYQXzDAiGP PdXLpcWf+X85v/dhqB8B86jixpacE48ZHK8w0zteFlHdUMUG96OCEnT3pL+8/HaZqshR l4reC1ir/5l/GSlOmR1V9SWlgYzOoCiCA/jhSSJpmIZoAZyG1GuENdxT9vGnVBGNyq25 hnGXYpi9O9ViZoqVxQdGSNgwoFAbv1hTTSKzdxnvasV88S3xj6oOHqpyLE7NjK0or7Hw HkAVrpkv2idfyY65wHk4Ch8mJ5Zr6wgAJRGImRt2tT6c9MKh0Pwr+tBZ8JBPyzYMP+30 vzWw== X-Gm-Message-State: AOJu0YyTitRnQRCeuJH9k+w/eBw2beeX/+Jd4OSxDzW9rIQD6F0WkNP1 ObG1Ub3cTj/+sGa3/Skzf/QrXIBu9qsfnIfwLCrNydN8JncrvP0q99QEt5SsnsY= X-Gm-Gg: ASbGnct1HjGezbPCvivrlkCqXKZM9iPxl4VGQMbwjvPNkY3ToQy/bxcV0AsinTkcVzY BtvFL49Lmf/X5Af8L++eX9L8xVqSslUZPWVnLOt1nEIMTJ/KJgc2xdH+2MLXVa05jaYj8S72x2M KVwJJ69JsWLrOHoi2Lm5yZfmol0OLS5svcPRq0dNqgrXyI67uX/2cSNO6wdllsxElWQIP/Up9+E RTIEOnFxPID2rlICkApfTsUPuUDCuCivhvRpJAthxlH+VzqUCZc6CMmso1yt0LVEArW+RK+1zNx ttSflt3AUnVmVM9uu1Ou+nZCGu2KMywTU5C1 X-Google-Smtp-Source: AGHT+IF7xfl9B7/xJ8k+iPG9dONRxwPTx8ewhI6YDPJcZwPMf++LnRv+shbC0/r1nL1MG22ob6N9GA== X-Received: by 2002:a05:6512:3d8a:b0:545:ea9:1a11 with SMTP id 2adb3069b0e04-54839129bc3mr6152301e87.5.1740477329613; Tue, 25 Feb 2025 01:55:29 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id 2adb3069b0e04-548514b261esm132867e87.24.2025.02.25.01.55.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 25 Feb 2025 01:55:29 -0800 (PST) From: Linus Walleij Date: Tue, 25 Feb 2025 10:55:13 +0100 Subject: [PATCH v5 26/31] ARM: entry: Move in-kernel hardirq tracing to C MIME-Version: 1.0 Message-Id: <20250225-arm-generic-entry-v5-26-2f02313653e5@linaro.org> References: <20250225-arm-generic-entry-v5-0-2f02313653e5@linaro.org> In-Reply-To: <20250225-arm-generic-entry-v5-0-2f02313653e5@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250225_015531_201572_8217EC37 X-CRM114-Status: GOOD ( 14.72 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Move the code tracing hardirqs on/off into the C callbacks for irqentry_enter_from_kernel_mode() and irqentry_exit_to_kernel_mode(). The semantic difference occurred is that we alsways check the PSR_I_BIT to determine if (hard) interrupts were enabled or not. The assembly has a tweak to avoid this if we are exiting an IRQ since it is obvious that IRQs must have been enabled to get there, but for simplicity we just check it for all exceptions. Signed-off-by: Linus Walleij --- arch/arm/kernel/entry-armv.S | 13 ++++--------- arch/arm/kernel/entry-header.S | 19 ++----------------- arch/arm/kernel/entry.c | 5 +++++ 3 files changed, 11 insertions(+), 26 deletions(-) diff --git a/arch/arm/kernel/entry-armv.S b/arch/arm/kernel/entry-armv.S index f64c4cc8beda6bcd469e6ff1a1f337d52dbbaf9c..2a789c8834b93475c32dcb6ba5854e24ddd8d6e9 100644 --- a/arch/arm/kernel/entry-armv.S +++ b/arch/arm/kernel/entry-armv.S @@ -158,7 +158,7 @@ ENDPROC(__und_invalid) #define SPFIX(code...) #endif - .macro svc_entry, stack_hole=0, trace=1, uaccess=1, overflow_check=1 + .macro svc_entry, stack_hole=0 uaccess=1, overflow_check=1 UNWIND(.fnstart ) sub sp, sp, #(SVC_REGS_SIZE + \stack_hole) THUMB( add sp, r1 ) @ get SP in a GPR without @@ -208,11 +208,6 @@ ENDPROC(__und_invalid) mov r0, sp @ 'regs' bl irqentry_enter_from_kernel_mode - .if \trace -#ifdef CONFIG_TRACE_IRQFLAGS - bl trace_hardirqs_off -#endif - .endif .endm .align 5 @@ -239,7 +234,7 @@ __irq_svc: blne svc_preempt #endif - svc_exit r5, irq = 1 @ return from exception + svc_exit r5 @ return from exception UNWIND(.fnend ) ENDPROC(__irq_svc) @@ -303,7 +298,7 @@ ENDPROC(__pabt_svc) .align 5 __fiq_svc: - svc_entry trace=0 + svc_entry mov r0, sp @ struct pt_regs *regs bl handle_fiq_as_nmi svc_exit_via_fiq @@ -321,7 +316,7 @@ ENDPROC(__fiq_svc) @ .align 5 __fiq_abt: - svc_entry trace=0 + svc_entry ARM( msr cpsr_c, #ABT_MODE | PSR_I_BIT | PSR_F_BIT ) THUMB( mov r0, #ABT_MODE | PSR_I_BIT | PSR_F_BIT ) diff --git a/arch/arm/kernel/entry-header.S b/arch/arm/kernel/entry-header.S index 49a9c5cf6fd5fbb917f2ada6c0d6cc400b7d3fb3..cfaf14d71378ba14bbb2a42cd36d48a23838eee1 100644 --- a/arch/arm/kernel/entry-header.S +++ b/arch/arm/kernel/entry-header.S @@ -199,26 +199,11 @@ .endm - .macro svc_exit, rpsr, irq = 0 - .if \irq != 0 - @ IRQs already off -#ifdef CONFIG_TRACE_IRQFLAGS - @ The parent context IRQs must have been enabled to get here in - @ the first place, so there's no point checking the PSR I bit. - bl trace_hardirqs_on -#endif - .else + .macro svc_exit, rpsr + @ IRQs off again before pulling preserved data off the stack disable_irq_notrace -#ifdef CONFIG_TRACE_IRQFLAGS - tst \rpsr, #PSR_I_BIT - bleq trace_hardirqs_on - tst \rpsr, #PSR_I_BIT - blne trace_hardirqs_off -#endif - .endif - mov r0, sp @ 'regs' bl irqentry_exit_to_kernel_mode diff --git a/arch/arm/kernel/entry.c b/arch/arm/kernel/entry.c index 674b5adcec0001b7d075d6936bfb4e318cb7ce74..1e1284cc4caed6e602ce36e812d535e6fe324f34 100644 --- a/arch/arm/kernel/entry.c +++ b/arch/arm/kernel/entry.c @@ -59,8 +59,13 @@ noinstr void irqentry_exit_to_user_mode(struct pt_regs *regs) noinstr void irqentry_enter_from_kernel_mode(struct pt_regs *regs) { + trace_hardirqs_off(); } noinstr void irqentry_exit_to_kernel_mode(struct pt_regs *regs) { + if (interrupts_enabled(regs)) + trace_hardirqs_on(); + else + trace_hardirqs_off(); } From patchwork Tue Feb 25 09:55:14 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13989782 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id EBDBAC021B2 for ; Tue, 25 Feb 2025 10:50:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=PgPYwUz2C3jq2K678fDOOKopXFthUCF1z0KBZvQ8kgo=; b=RkFwSyMPBhFTeCuf/O0Ic3e9jx t7a0iHQKrDfODEh0sHzbvHndLiYcxRiQoIV5B3gIkYslUCWmkwFivjSsXWkHEodzzSpMSuTOkXKMl 3U5eWcKMgh3NRBxguNNw+dL2EOMCzZI6yYuBEYt9N/EyqhkdfaVV295mVmQELyc26xwZkxczjGn+r RNHFTCzswLwuw9JNCkBGFg88g67FrRAo8UwizXqA+cGfKW/Mt0usgWvWHB2BCEOoEKQAhG53JrTPn IXql2Y+w2qNLfBEi/oyGKTxwu2BTVG8ewHsTqHnT3rHY44LOtMrfsBU4fh3yGEAzybwNQIXsHw7P6 yL4Efz8Q==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tmsWW-0000000GtvG-29rE; Tue, 25 Feb 2025 10:50:48 +0000 Received: from mail-lf1-x134.google.com ([2a00:1450:4864:20::134]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tmrf2-0000000GhoB-27QT for linux-arm-kernel@lists.infradead.org; Tue, 25 Feb 2025 09:55:33 +0000 Received: by mail-lf1-x134.google.com with SMTP id 2adb3069b0e04-5452efeb87aso5417847e87.3 for ; Tue, 25 Feb 2025 01:55:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1740477331; x=1741082131; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=PgPYwUz2C3jq2K678fDOOKopXFthUCF1z0KBZvQ8kgo=; b=pb95KdxV7x+jLXqjtKdGQlmH/NQuQtT6zkWTGFJUJ/IshDg5IZ5cuN1GLuvFyxX6KF Ca9r9xboVIfnw2E4kWUiQ7vvVAt+Fhi4unuzwjDTuCYFofRjygvL+X7rIx/dHbrWURea KSX2T3u6bB1cpfdXAe8gQ0peTNaVCUw6vwTacvmNr/A+cqqRgoAPRMK93cthoiAIrwJT n2EDqwlgyaS+KYmtwek20mFH7BQdBZ1VPvg8iqEqR2EMd+0hQImPvhLUqLQGbUF4XBQ0 fNML/K2n4otme/hGlbxut3t0jwfeV9dfyNQ6p9QiEeZ6YxMyC8Mf2xYswjnGX9U5YeAp YyPA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740477331; x=1741082131; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=PgPYwUz2C3jq2K678fDOOKopXFthUCF1z0KBZvQ8kgo=; b=dnYUM4639z8jvPELtAoMX7UtwDogM6sjk6cBIPGzWRYamsjW4odkSZFm8xIu+KEryv iNfKV3XhHrIlTVw1PyAyNjOQv8Fg4VSYWOX4NeEwzi250EPJPyE/2pnHc1G2OfxD2kBF qrpvvBtVFDeJC9C49Wl3b8SHgmIu4wl1R7KeypnKeFYL0VEeVrNhO5K0hkp69iaEzSuv zzB671UxFVtRBgl+nSuKVmomxrYKiddaw5mcqZ0nqsEBwP5AZe0tjgM8AMrUcBXC9qZp BEMvnw2j9DGlAp6IdJAcsy0QzC0pbeNQzW0IBPVhjZygJ0Ddg7O51fJicxglby9R6oNd mQ5w== X-Gm-Message-State: AOJu0YxL2ApCjs5VTIxiKg27SXwYabfYL541tX7KiDv8FleY8DHAGidv Er+QImqWHV29yVbRF1F6t7guRboZuE189lGtSNV6l8iqQoscZn2PKOpnjHEfJjA= X-Gm-Gg: ASbGncsHaonazKzlRYgJIZOBVGO0nAn6zGT4r2jeC0NQrzAZSGTyZnNdgFnjj/gUFFW 1hcU/BjubN7aCz55gWZQ/zy/I7mrIFlRSQmzdIoymhwgw+pDk9dw6Pv63EB/shZSIMVdGOfGpkf 8MSxM82r2B9Bgq+M23C4kxx/hVHoVqXoDSjt/qUNR9zCIC0vTmfj8iwXF+70SIklnVdE6zdxJbe q79RY5NHMT6uZP2O0QYlh3QAkqn00uSBnUiEbQxV5GIZ1dDYS6zznSuNb8p8MjMnvdDDLIQRaA8 maD8BFhWWpeSzif/FD+rZDhG/UF8Sxz3k5XR X-Google-Smtp-Source: AGHT+IHrYBKEgwrOsqtabCf77riVA6wTLLPZltwvy4MPKuhYyJsmmbuvXTyEaXfB9oGzNDCBGFjJZw== X-Received: by 2002:a05:6512:3093:b0:545:2f9b:7589 with SMTP id 2adb3069b0e04-54838f79d15mr5814906e87.50.1740477330711; Tue, 25 Feb 2025 01:55:30 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id 2adb3069b0e04-548514b261esm132867e87.24.2025.02.25.01.55.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 25 Feb 2025 01:55:30 -0800 (PST) From: Linus Walleij Date: Tue, 25 Feb 2025 10:55:14 +0100 Subject: [PATCH v5 27/31] ARM: irq: Add irqstack helper MIME-Version: 1.0 Message-Id: <20250225-arm-generic-entry-v5-27-2f02313653e5@linaro.org> References: <20250225-arm-generic-entry-v5-0-2f02313653e5@linaro.org> In-Reply-To: <20250225-arm-generic-entry-v5-0-2f02313653e5@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250225_015532_549178_046978DD X-CRM114-Status: GOOD ( 14.91 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Add a helper to dispatch IRQ execution to the IRQ stack: call_on_irq_stack() so we can explicitly issue handle_irq() on the IRQ stack from a C program. Cc: Ard Biesheuvel Signed-off-by: Linus Walleij --- arch/arm/kernel/irq.c | 6 ++++++ arch/arm/kernel/irq.h | 2 ++ 2 files changed, 8 insertions(+) diff --git a/arch/arm/kernel/irq.c b/arch/arm/kernel/irq.c index e1993e28a9ecfd80b55b2677253ac582467e6c14..f99d6b24d8ff56f901e4a817c61a1e57b7e532cc 100644 --- a/arch/arm/kernel/irq.c +++ b/arch/arm/kernel/irq.c @@ -43,6 +43,7 @@ #include #include +#include "irq.h" #include "reboot.h" unsigned long irq_err_count; @@ -71,6 +72,11 @@ static void __init init_irq_stacks(void) } } +void call_on_irq_stack(void (*fn)(void *), void *arg) +{ + call_with_stack(fn, arg, __this_cpu_read(irq_stack_ptr)); +} + #ifdef CONFIG_SOFTIRQ_ON_OWN_STACK static void ____do_softirq(void *arg) { diff --git a/arch/arm/kernel/irq.h b/arch/arm/kernel/irq.h new file mode 100644 index 0000000000000000000000000000000000000000..80dd5bfe6403d68fc62fab30666d5645748d329f --- /dev/null +++ b/arch/arm/kernel/irq.h @@ -0,0 +1,2 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +void call_on_irq_stack(void (*fn)(void *), void *arg); From patchwork Tue Feb 25 09:55:15 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13989783 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3EB0AC021B2 for ; Tue, 25 Feb 2025 10:52:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=f3uDcPAUn17u5guwAmKffFv3WeYri61KKtnRD7KLFFU=; b=fpNbaK9xPiZCkejWz/9mOOfD9g JVWK8VGC1dbJPFiT7j66iDcicMmKofkzlGxp8Lj0bniPot7/dDTIB5CrYs1zgBIoUyZt5LUzHy/9m Kpr5VK/XANQMu/EUrF55TbsBert1zkvsy/Y5Msyj91QHGxHuIIHls6ykQc9r6Yms6Ie7QW1pGGRBk LXfHhrtqjSIA9vSeqd6CW1GJobRwKALhuiwMmREacqyyBRmu89pZD8VL/xDSZz3K2gO4QF3/mtiIu XOtwU0ZVu9gGYgAi3laE1dXf6sO0KPPSm1bVogvBE6qXeFXWVf7ZbeEuu+ksNavbjhqGFD6vTx6NO FW9+oghw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tmsY2-0000000GuDm-0nB0; Tue, 25 Feb 2025 10:52:22 +0000 Received: from mail-lf1-x12d.google.com ([2a00:1450:4864:20::12d]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tmrf3-0000000Ghob-2X8t for linux-arm-kernel@lists.infradead.org; Tue, 25 Feb 2025 09:55:35 +0000 Received: by mail-lf1-x12d.google.com with SMTP id 2adb3069b0e04-54298ec925bso7642794e87.3 for ; Tue, 25 Feb 2025 01:55:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1740477332; x=1741082132; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=f3uDcPAUn17u5guwAmKffFv3WeYri61KKtnRD7KLFFU=; b=HW5Uh3onLed6ZQsAX3f4YKR8yEOqrCsiVIwEYGkx6g2tGf5flTO2WadqbHu0ZH3g0Q NNbMQqNAiZh//yejvGW5WexCPzLyV4pgDaqF1J6nBKM3avdys33OtXtqgxGff6yhsOSA 5AtM+WAF63mHhyTtBc8bEzf9/xi4ixgZoxkGyMRyRzImKC37T599+wlG3AWEc/KRWIbu JsBwjKMWUNPnYL0O/+dpYw8YWT90qdO4apTQjQV8ws/C6fl63PedSft0l42mM2XdUaDO YmTryP/8IxAkATt58wYY+eOS7UsG6eS8/oMR/cMieSU5KOYaGH6OGJJ/UdbFiwnJhwfM 7rDQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740477332; x=1741082132; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=f3uDcPAUn17u5guwAmKffFv3WeYri61KKtnRD7KLFFU=; b=tqboiSIQa3I9kI9PzGYMTWlJVPRy75fk3W6uwodPQasN3GXVZrwU7GitM/0Dp1k9TE TZ1qdNzcVUMziMGKLhXzvYxw5OyC/IuuZTScmGQl8th94NMR1yVaUUK0iA5ww7/r1gcl PXa9/JVqBXxHBeK4E9kRkX4I7+ZJzDHUBgujHQ8WY4AusA9a+andCurFw9JlSSdERfc9 M5l7ytvg55kq6stQtP4oHPMfEd2N6WlwCLalln4IUG+OwvIusCHvxFbQ9xbcYeIxJ3dr m2oP6r4wdBMznukxdmkbZavtHgDM0Kzg/VYHPy9VRpN6zMGp6L0A5cyGebLEqI5fiBic p/xw== X-Gm-Message-State: AOJu0Yx8MOP8UXScCRPfszYLwpP+3b0H5pRgXTpR8HnxentuNWiex0Pi 33LgNp23eUkL8NED90BdvhXL2L2QddMl3wNRWZdmPl4hp+9XsbqPyx91JkBixHw= X-Gm-Gg: ASbGncvUs9oQbzNEV00TrI+HceXBXWy5b9wunlMacxonW974n8jSMff/xxrrLxf43Hg uYTIOx/5JBbKAqlAqolFIJe0C9NxgqzIXzHNqNLXQop0gF5KqovYNirvPvv4Hm3gJ2yuOj4iRFP chXlGoSazf2FT4fQCU8L1cdE0YNBk+dOXLcjEn6MO+5N7WFqz3JW7XVXCj6dpTxB/HelPpUqRp4 epB49ugPp8hcNG/tFgWI8sNbTk2po/4pQrj2C6CsW13N/ei9yV3dSDKJ44+kAZFGPMIrniX3BoH QpTtI0UpHyb63h3NeCG0srdWaIrfOHEIbKqY X-Google-Smtp-Source: AGHT+IGZSyqBfVWqqe5J4Zp72BCqTmEE8EDYl5esUBqbgbQwzfkt9iMaTDDcYx6uNEGX+3vdzFMHVg== X-Received: by 2002:a05:6512:1152:b0:545:271d:f85 with SMTP id 2adb3069b0e04-54838ef8816mr5740468e87.29.1740477331850; Tue, 25 Feb 2025 01:55:31 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id 2adb3069b0e04-548514b261esm132867e87.24.2025.02.25.01.55.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 25 Feb 2025 01:55:31 -0800 (PST) From: Linus Walleij Date: Tue, 25 Feb 2025 10:55:15 +0100 Subject: [PATCH v5 28/31] ARM: entry: Convert to generic entry MIME-Version: 1.0 Message-Id: <20250225-arm-generic-entry-v5-28-2f02313653e5@linaro.org> References: <20250225-arm-generic-entry-v5-0-2f02313653e5@linaro.org> In-Reply-To: <20250225-arm-generic-entry-v5-0-2f02313653e5@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250225_015533_896825_1617A07A X-CRM114-Status: GOOD ( 28.91 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org This rewrites ARM to use the generic entry. All of the irqentry_* callbacks are collapsed and made to call into the generic entry code that handle the interrupts storing regs context before entry and restoring it after. The syscall_enter_from_user_mode(), syscall_exit_to_user_mode(), do_work_pending() constructs are replaced with generic code. The syscall invocation now enables syscall tracing "work" using the generic callbacks and no extra actions are needed to enable/disable syscall tracing. The old context tracking with user_exit_callable() user_enter_callable() is no longer used. Only CSKY and Xtensa use this deprecated mechanism now. handle_fiq_as_nmi() is retired as the generic entry provides the same functionality in irqentry_nmi_enter() and irqentry_nmi_exit(). Like with the ARM64 proposed patch for generic entry, it is necessary to create a small call "arm_exit_to_user_mode()" that is used for things such as ret_from_fork, i.e. when we create a completely new execution context with a botched return from interrupt. The elaborate checking and switching to IRQ stack in the irq_handler macro was reimplemented in C for this exercise: this was easiest and there was already code making use of the IRQ stack from C for soft interrupts. I have dropped the context tracking (i.e. calling irqentry_enter() and looking for trace conditions) on dabt/pabt i.e. do_DataAbort and do_PrefetchAbort code paths. These are addressed in a separate patch for development/review reasons. I added stubs for PTRACE_SYSEMU and PTRACE_SYSEMU_SINGLESTEP returning -EIO like the UM arch does, since those syscalls are required by generic entry. If someone wants to implement them, the placeholders are there. Signed-off-by: Linus Walleij --- arch/arm/Kconfig | 1 + arch/arm/include/asm/entry-common.h | 66 ++++++++++++++++++++++++++ arch/arm/include/asm/entry.h | 13 ++--- arch/arm/include/asm/ptrace.h | 3 -- arch/arm/include/asm/signal.h | 2 - arch/arm/include/asm/syscall.h | 7 ++- arch/arm/include/asm/thread_info.h | 17 +------ arch/arm/include/asm/traps.h | 1 - arch/arm/include/uapi/asm/ptrace.h | 2 + arch/arm/kernel/entry-armv.S | 50 ++++--------------- arch/arm/kernel/entry-common.S | 34 +++---------- arch/arm/kernel/entry-header.S | 18 ------- arch/arm/kernel/entry.c | 94 ++++++++++++++++++------------------ arch/arm/kernel/process.c | 5 +- arch/arm/kernel/ptrace.c | 95 ++----------------------------------- arch/arm/kernel/signal.c | 31 +----------- arch/arm/kernel/syscall.c | 30 ++++-------- arch/arm/kernel/traps.c | 25 ---------- 18 files changed, 161 insertions(+), 333 deletions(-) diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig index 835b5f100e926e54bab4be7802aeada4af37a145..564575f52cf7ec8296938944fa5296a4acc2a598 100644 --- a/arch/arm/Kconfig +++ b/arch/arm/Kconfig @@ -69,6 +69,7 @@ config ARM select GENERIC_CPU_AUTOPROBE select GENERIC_CPU_DEVICES select GENERIC_EARLY_IOREMAP + select GENERIC_ENTRY select GENERIC_IDLE_POLL_SETUP select GENERIC_IRQ_MULTI_HANDLER select GENERIC_IRQ_PROBE diff --git a/arch/arm/include/asm/entry-common.h b/arch/arm/include/asm/entry-common.h new file mode 100644 index 0000000000000000000000000000000000000000..7b1f108512db0001a65911af0df4f48197d99e75 --- /dev/null +++ b/arch/arm/include/asm/entry-common.h @@ -0,0 +1,66 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +#ifndef _ASM_ARM_ENTRY_COMMON_H +#define _ASM_ARM_ENTRY_COMMON_H + +#include + +#include + +enum ptrace_syscall_dir { + PTRACE_SYSCALL_ENTER = 0, + PTRACE_SYSCALL_EXIT, +}; + +static inline unsigned long +arch_prepare_report_syscall_entry(struct pt_regs *regs) +{ + unsigned long ip; + + /* + * IP is used to denote syscall entry/exit: + * IP = 0 -> entry + */ + ip = regs->ARM_ip; + regs->ARM_ip = PTRACE_SYSCALL_ENTER; + + return ip; +} +#define arch_prepare_report_syscall_entry arch_prepare_report_syscall_entry + +static inline void +arch_post_report_syscall_entry(struct pt_regs *regs, + unsigned long saved_reg, long ret) +{ + regs->ARM_ip = saved_reg; +} +#define arch_post_report_syscall_entry arch_post_report_syscall_entry + + +static inline unsigned long +arch_prepare_report_syscall_exit(struct pt_regs *regs, + unsigned long work) +{ + unsigned long ip; + + /* + * IP is used to denote syscall entry/exit: + * IP = 1 -> exit + */ + ip = regs->ARM_ip; + regs->ARM_ip = PTRACE_SYSCALL_EXIT; + + return ip; +} +#define arch_prepare_report_syscall_exit arch_prepare_report_syscall_exit + +static inline void +arch_post_report_syscall_exit(struct pt_regs *regs, + unsigned long saved_reg, + unsigned long work) +{ + regs->ARM_ip = saved_reg; +} +#define arch_post_report_syscall_exit arch_post_report_syscall_exit + +#endif /* _ASM_ARM_ENTRY_COMMON_H */ diff --git a/arch/arm/include/asm/entry.h b/arch/arm/include/asm/entry.h index a78bc5054b093120e903f08369fb8b1a9feb7f81..df2dd14ab51a586b83b6da6bbf9bd99858c1ddf4 100644 --- a/arch/arm/include/asm/entry.h +++ b/arch/arm/include/asm/entry.h @@ -4,15 +4,8 @@ struct pt_regs; -/* - * These are copies of generic entry headers so we can transition - * to generic entry once they are semantically equivalent. - */ -long syscall_enter_from_user_mode(struct pt_regs *regs, long); -void syscall_exit_to_user_mode(struct pt_regs *regs); -void irqentry_enter_from_user_mode(struct pt_regs *regs); -void irqentry_exit_to_user_mode(struct pt_regs *regs); -void irqentry_enter_from_kernel_mode(struct pt_regs *regs); -void irqentry_exit_to_kernel_mode(struct pt_regs *regs); +void arm_irq_handler(struct pt_regs *regs, int mode); +void arm_fiq_handler(struct pt_regs *regs); +void arm_exit_to_user_mode(struct pt_regs *regs); #endif /* __ASM_ENTRY_H__ */ diff --git a/arch/arm/include/asm/ptrace.h b/arch/arm/include/asm/ptrace.h index a01c66f0ad907882af2a383e53de8aca68c551e5..8a7de4020cbc016f743dcf515b9fa83b9bd3e3f0 100644 --- a/arch/arm/include/asm/ptrace.h +++ b/arch/arm/include/asm/ptrace.h @@ -200,8 +200,5 @@ static inline unsigned long it_advance(unsigned long cpsr) return cpsr; } -int syscall_trace_enter(struct pt_regs *regs); -void syscall_trace_exit(struct pt_regs *regs); - #endif /* __ASSEMBLY__ */ #endif diff --git a/arch/arm/include/asm/signal.h b/arch/arm/include/asm/signal.h index bb753cd3e29fc54186543c48b2aee7b221301a37..45885383646c9f166b0d7ee80080b78776b515e7 100644 --- a/arch/arm/include/asm/signal.h +++ b/arch/arm/include/asm/signal.h @@ -23,6 +23,4 @@ typedef struct { #include -void do_work_pending(struct pt_regs *regs, unsigned int thread_flags); - #endif diff --git a/arch/arm/include/asm/syscall.h b/arch/arm/include/asm/syscall.h index b94fd7ce17f82f0baf06b74800245112e1f13002..a6c8f0e799bb8bf3833161dec70fc7a1aaa513f6 100644 --- a/arch/arm/include/asm/syscall.h +++ b/arch/arm/include/asm/syscall.h @@ -19,7 +19,12 @@ extern const unsigned long sys_call_table[]; -int invoke_syscall(void *table, struct pt_regs *regs, int scno); +void invoke_syscall(void *table, struct pt_regs *regs, int scno); + +static inline bool arch_syscall_is_vdso_sigreturn(struct pt_regs *regs) +{ + return false; +} static inline int syscall_get_nr(struct task_struct *task, struct pt_regs *regs) diff --git a/arch/arm/include/asm/thread_info.h b/arch/arm/include/asm/thread_info.h index d8a45c5a10496aaf806bfeaa0353d5e8985bd6f5..84e58a9cdab63ad264c2cd2bad64239d1912cbe7 100644 --- a/arch/arm/include/asm/thread_info.h +++ b/arch/arm/include/asm/thread_info.h @@ -74,6 +74,7 @@ struct thread_info { #ifdef CONFIG_ARM_THUMBEE unsigned long thumbee_state; /* ThumbEE Handler Base register */ #endif + unsigned long syscall_work; /* SYSCALL_WORK_ flags */ }; #define INIT_THREAD_INFO(tsk) \ @@ -149,30 +150,14 @@ extern int vfp_restore_user_hwstate(struct user_vfp *, #define TIF_USING_IWMMXT 17 #define TIF_MEMDIE 18 /* is terminating due to OOM killer */ #define TIF_RESTORE_SIGMASK 19 -#define TIF_SYSCALL_TRACE 20 /* syscall trace active */ -#define TIF_SYSCALL_AUDIT 21 /* syscall auditing active */ -#define TIF_SYSCALL_TRACEPOINT 22 /* syscall tracepoint instrumentation */ -#define TIF_SECCOMP 23 /* seccomp syscall filtering active */ - #define _TIF_SIGPENDING (1 << TIF_SIGPENDING) #define _TIF_NEED_RESCHED (1 << TIF_NEED_RESCHED) #define _TIF_NOTIFY_RESUME (1 << TIF_NOTIFY_RESUME) #define _TIF_UPROBE (1 << TIF_UPROBE) -#define _TIF_SYSCALL_TRACE (1 << TIF_SYSCALL_TRACE) -#define _TIF_SYSCALL_AUDIT (1 << TIF_SYSCALL_AUDIT) -#define _TIF_SYSCALL_TRACEPOINT (1 << TIF_SYSCALL_TRACEPOINT) -#define _TIF_SECCOMP (1 << TIF_SECCOMP) #define _TIF_NOTIFY_SIGNAL (1 << TIF_NOTIFY_SIGNAL) #define _TIF_USING_IWMMXT (1 << TIF_USING_IWMMXT) -/* Checks for any syscall work in entry-common.S */ -#define _TIF_SYSCALL_WORK (_TIF_SYSCALL_TRACE | _TIF_SYSCALL_AUDIT | \ - _TIF_SYSCALL_TRACEPOINT | _TIF_SECCOMP) - -/* - * Change these and you break ASM code in entry-common.S - */ #define _TIF_WORK_MASK (_TIF_NEED_RESCHED | _TIF_SIGPENDING | \ _TIF_NOTIFY_RESUME | _TIF_UPROBE | \ _TIF_NOTIFY_SIGNAL) diff --git a/arch/arm/include/asm/traps.h b/arch/arm/include/asm/traps.h index b888912c2450e0e8eec8139166c7292f283545c7..f8695104c72f2f450092839c7c50920e540e594b 100644 --- a/arch/arm/include/asm/traps.h +++ b/arch/arm/include/asm/traps.h @@ -38,7 +38,6 @@ extern void *vectors_page; asmlinkage void dump_backtrace_stm(u32 *stack, u32 instruction, const char *loglvl); asmlinkage void do_undefinstr(struct pt_regs *regs); -asmlinkage void handle_fiq_as_nmi(struct pt_regs *regs); asmlinkage void bad_mode(struct pt_regs *regs, int reason); int arm_syscall(int no, struct pt_regs *regs); asmlinkage void baddataabort(int code, unsigned long instr, struct pt_regs *regs); diff --git a/arch/arm/include/uapi/asm/ptrace.h b/arch/arm/include/uapi/asm/ptrace.h index 8896c23ccba78ed7e10c7d1b9955124314496dc6..336a5beb282f360e875703045ee833974bbd68b2 100644 --- a/arch/arm/include/uapi/asm/ptrace.h +++ b/arch/arm/include/uapi/asm/ptrace.h @@ -33,6 +33,8 @@ #define PTRACE_GETHBPREGS 29 #define PTRACE_SETHBPREGS 30 #define PTRACE_GETFDPIC 31 +#define PTRACE_SYSEMU 32 +#define PTRACE_SYSEMU_SINGLESTEP 33 #define PTRACE_GETFDPIC_EXEC 0 #define PTRACE_GETFDPIC_INTERP 1 diff --git a/arch/arm/kernel/entry-armv.S b/arch/arm/kernel/entry-armv.S index 2a789c8834b93475c32dcb6ba5854e24ddd8d6e9..498a22f5fa8a6516d1eab584f0523c3d6c6e4926 100644 --- a/arch/arm/kernel/entry-armv.S +++ b/arch/arm/kernel/entry-armv.S @@ -36,35 +36,6 @@ #define RELOC_TEXT_NONE #endif -/* - * Interrupt handling. - */ - .macro irq_handler, from_user:req - mov r1, sp - ldr_this_cpu r2, irq_stack_ptr, r2, r3 - .if \from_user == 0 - @ - @ If we took the interrupt while running in the kernel, we may already - @ be using the IRQ stack, so revert to the original value in that case. - @ - subs r3, r2, r1 @ SP above bottom of IRQ stack? - rsbscs r3, r3, #THREAD_SIZE @ ... and below the top? -#ifdef CONFIG_VMAP_STACK - ldr_va r3, high_memory, cc @ End of the linear region - cmpcc r3, r1 @ Stack pointer was below it? -#endif - bcc 0f @ If not, switch to the IRQ stack - mov r0, r1 - bl generic_handle_arch_irq - b 1f -0: - .endif - - mov_l r0, generic_handle_arch_irq - bl call_with_stack -1: - .endm - .macro pabt_helper @ PABORT handler takes pt_regs in r2, fault address in r4 and psr in r5 #ifdef MULTI_PABORT @@ -205,9 +176,6 @@ ENDPROC(__und_invalid) get_thread_info tsk uaccess_entry tsk, r0, r1, r2, \uaccess - mov r0, sp @ 'regs' - bl irqentry_enter_from_kernel_mode - .endm .align 5 @@ -223,7 +191,9 @@ ENDPROC(__dabt_svc) .align 5 __irq_svc: svc_entry - irq_handler from_user=0 + mov r0, sp @ regs + mov r1, #0 @ from kernel mode + bl arm_irq_handler #ifdef CONFIG_PREEMPTION ldr r8, [tsk, #TI_PREEMPT] @ get preempt count @@ -300,7 +270,7 @@ ENDPROC(__pabt_svc) __fiq_svc: svc_entry mov r0, sp @ struct pt_regs *regs - bl handle_fiq_as_nmi + bl arm_fiq_handler svc_exit_via_fiq UNWIND(.fnend ) ENDPROC(__fiq_svc) @@ -329,7 +299,7 @@ __fiq_abt: stmfd sp!, {r1 - r2} add r0, sp, #8 @ struct pt_regs *regs - bl handle_fiq_as_nmi + bl arm_fiq_handler ldmfd sp!, {r1 - r2} ARM( msr cpsr_c, #ABT_MODE | PSR_I_BIT | PSR_F_BIT ) @@ -420,7 +390,6 @@ ENDPROC(__fiq_abt) .align 5 __dabt_usr: usr_entry uaccess=0 - asm_irqentry_enter_from_user_mode kuser_cmpxchg_check mov r2, sp dabt_helper @@ -431,9 +400,10 @@ ENDPROC(__dabt_usr) .align 5 __irq_usr: usr_entry - asm_irqentry_enter_from_user_mode kuser_cmpxchg_check - irq_handler from_user=1 + mov r0, sp @ regs + mov r1, #1 @ from user mode + bl arm_irq_handler get_thread_info tsk mov why, #0 b ret_to_user_from_irq @@ -445,7 +415,6 @@ ENDPROC(__irq_usr) .align 5 __und_usr: usr_entry uaccess=0 - asm_irqentry_enter_from_user_mode @ IRQs must be enabled before attempting to read the instruction from @ user space since that could cause a page/translation fault if the @@ -470,7 +439,6 @@ ENDPROC(__und_usr) .align 5 __pabt_usr: usr_entry - asm_irqentry_enter_from_user_mode mov r2, sp @ regs pabt_helper UNWIND(.fnend ) @@ -493,7 +461,7 @@ __fiq_usr: usr_entry kuser_cmpxchg_check mov r0, sp @ struct pt_regs *regs - bl handle_fiq_as_nmi + bl arm_fiq_handler get_thread_info tsk restore_user_regs UNWIND(.fnend ) diff --git a/arch/arm/kernel/entry-common.S b/arch/arm/kernel/entry-common.S index 6b0f86786a7d9e656ecbe8c741f2edbcbafd2915..a5a0e757ab23d151485092a078af98966e039869 100644 --- a/arch/arm/kernel/entry-common.S +++ b/arch/arm/kernel/entry-common.S @@ -39,10 +39,6 @@ ret_fast_syscall: UNWIND(.fnstart ) UNWIND(.cantunwind ) add sp, sp, #(S_R0 + S_OFF) - /* do_rseq_syscall needs interrupts enabled. */ - mov r0, sp @ 'regs' - bl syscall_exit_to_user_mode - #ifdef CONFIG_GCC_PLUGIN_STACKLEAK bl stackleak_erase_on_task_stack #endif @@ -60,7 +56,7 @@ ENTRY(ret_to_user) /* do_rseq_syscall needs interrupts enabled. */ enable_irq_notrace @ enable interrupts mov r0, sp @ 'regs' - bl syscall_exit_to_user_mode + bl arm_exit_to_user_mode #ifdef CONFIG_GCC_PLUGIN_STACKLEAK bl stackleak_erase_on_task_stack @@ -69,8 +65,6 @@ ENTRY(ret_to_user) ENDPROC(ret_to_user) ENTRY(ret_to_user_from_irq) - asm_irqentry_exit_to_user_mode - #ifdef CONFIG_GCC_PLUGIN_STACKLEAK bl stackleak_erase_on_task_stack #endif @@ -92,9 +86,10 @@ SYM_TYPED_FUNC_START(ret_from_fork_asm) mov r2, r5 mov r3, r4 bl ret_from_fork - mov r0, #0 -1: get_thread_info tsk - b ret_to_user +#ifdef CONFIG_GCC_PLUGIN_STACKLEAK + bl stackleak_erase_on_task_stack +#endif + restore_user_regs SYM_FUNC_END(ret_from_fork_asm) /*============================================================================= @@ -210,29 +205,12 @@ ENTRY(vector_swi) #else str scno, [tsk, #TI_ABI_SYSCALL] #endif - - /* - * Calling out to C to be careful to save and restore registers. - * This call could modify the syscall number. scno is r7 so we - * do not save and restore r7. - */ - mov r0, sp @ regs - mov r1, scno - push {r4 - r6, r8 - r10, lr} - bl syscall_enter_from_user_mode - pop {r4 - r6, r8 - r10, lr} - mov scno, r0 - mov r1, sp @ put regs into r1 stmdb sp!, {r4, r5} @ push fifth and sixth args mov r0, tbl mov r2, scno @ syscall number from r7 bl invoke_syscall - cmp r0, #0 - beq ret_fast_syscall - /* This path taken when tracing */ - add sp, sp, #(S_R0 + S_OFF) - b ret_to_user + b ret_fast_syscall #if defined(CONFIG_OABI_COMPAT) || !defined(CONFIG_AEABI) /* diff --git a/arch/arm/kernel/entry-header.S b/arch/arm/kernel/entry-header.S index cfaf14d71378ba14bbb2a42cd36d48a23838eee1..abd927f1cc3a4d813cc48bc046191f49f3c0e533 100644 --- a/arch/arm/kernel/entry-header.S +++ b/arch/arm/kernel/entry-header.S @@ -203,10 +203,6 @@ @ IRQs off again before pulling preserved data off the stack disable_irq_notrace - - mov r0, sp @ 'regs' - bl irqentry_exit_to_kernel_mode - uaccess_exit tsk, r0, r1 #ifndef CONFIG_THUMB2_KERNEL @@ -340,20 +336,6 @@ ALT_UP_B(.L1_\@) #endif /* !CONFIG_THUMB2_KERNEL */ .endm -/* - * Context tracking and other mode transitions. Used to instrument transitions - * between user and kernel mode. -*/ - .macro asm_irqentry_enter_from_user_mode - mov r0, sp @ regs - bl irqentry_enter_from_user_mode - .endm - - .macro asm_irqentry_exit_to_user_mode - mov r0, sp @ regs - bl irqentry_exit_to_user_mode - .endm - /* * These are the registers used in the syscall handler, and allow us to * have in theory up to 7 arguments to a function - r0 to r6. diff --git a/arch/arm/kernel/entry.c b/arch/arm/kernel/entry.c index 1e1284cc4caed6e602ce36e812d535e6fe324f34..08b8680233e5aae6e477f9bf15d7f06d7fd3e5af 100644 --- a/arch/arm/kernel/entry.c +++ b/arch/arm/kernel/entry.c @@ -1,71 +1,71 @@ // SPDX-License-Identifier: GPL-2.0 #include -#include -#include #include +#include +#include +#include #include +#include #include +#include -long syscall_enter_from_user_mode(struct pt_regs *regs, long syscall) -{ - trace_hardirqs_on(); - local_irq_enable(); - /* This context tracking call has inverse naming */ - user_exit_callable(); - - /* This will optionally be modified later */ - return syscall; -} +#include "irq.h" -void syscall_exit_to_user_mode(struct pt_regs *regs) +static void noinstr handle_arm_irq(void *data) { - unsigned long flags = read_thread_flags(); + struct pt_regs *regs = data; + struct pt_regs *old_regs; - rseq_syscall(regs); - local_irq_disable(); - /* - * It really matters that we check for flags != 0 and not - * just for pending work here! - */ - if (flags) - do_work_pending(regs, flags); + irq_enter_rcu(); + old_regs = set_irq_regs(regs); - trace_hardirqs_on(); - /* This context tracking call has inverse naming */ - user_enter_callable(); -} + handle_arch_irq(regs); -noinstr void irqentry_enter_from_user_mode(struct pt_regs *regs) -{ - trace_hardirqs_off(); - /* This context tracking call has inverse naming */ - user_exit_callable(); + set_irq_regs(old_regs); + irq_exit_rcu(); } -noinstr void irqentry_exit_to_user_mode(struct pt_regs *regs) +noinstr void arm_irq_handler(struct pt_regs *regs, int mode) { - unsigned long flags = read_thread_flags(); + irqentry_state_t state = irqentry_enter(regs); /* - * It really matters that we check for flags != 0 and not - * just for pending work here! + * If we are executing in usermode, or kernel process context + * (on the thread stack) then switch to the IRQ stack. Else we + * are already on the IRQ stack (or the overflow stack) and we + * can just proceed to handle the IRQ. */ - if (flags) - do_work_pending(regs, flags); - trace_hardirqs_on(); - /* This context tracking call has inverse naming */ - user_enter_callable(); + if (mode == 1) + call_on_irq_stack(handle_arm_irq, regs); + else if (on_thread_stack()) + call_on_irq_stack(handle_arm_irq, regs); + else + handle_arm_irq(regs); + + irqentry_exit(regs, state); } -noinstr void irqentry_enter_from_kernel_mode(struct pt_regs *regs) +/* + * Handle FIQ similarly to NMI on x86 systems. + * + * The runtime environment for NMIs is extremely restrictive + * (NMIs can pre-empt critical sections meaning almost all locking is + * forbidden) meaning this default FIQ handling must only be used in + * circumstances where non-maskability improves robustness, such as + * watchdog or debug logic. + * + * This handler is not appropriate for general purpose use in drivers + * platform code and can be overrideen using set_fiq_handler. + */ +noinstr void arm_fiq_handler(struct pt_regs *regs) { - trace_hardirqs_off(); + irqentry_state_t state = irqentry_nmi_enter(regs); + + irqentry_nmi_exit(regs, state); } -noinstr void irqentry_exit_to_kernel_mode(struct pt_regs *regs) +asmlinkage void arm_exit_to_user_mode(struct pt_regs *regs) { - if (interrupts_enabled(regs)) - trace_hardirqs_on(); - else - trace_hardirqs_off(); + local_irq_disable(); + irqentry_exit_to_user_mode(regs); } diff --git a/arch/arm/kernel/process.c b/arch/arm/kernel/process.c index 81c1c3f988344185917cf4e53d0d8ee47ae912f4..a59ad5fdaaf51f653c352884bbdd10d1211b2669 100644 --- a/arch/arm/kernel/process.c +++ b/arch/arm/kernel/process.c @@ -25,6 +25,7 @@ #include #include #include +#include #include #include @@ -248,8 +249,8 @@ __visible void ret_from_fork(struct task_struct *prev, struct pt_regs *regs, * execve() syscall. */ } - - /* syscall_exit_to_user_mode(regs); here once we switch to generic entry */ + local_irq_enable(); + syscall_exit_to_user_mode(regs); } int copy_thread(struct task_struct *p, const struct kernel_clone_args *args) diff --git a/arch/arm/kernel/ptrace.c b/arch/arm/kernel/ptrace.c index ac7b98ae47249b4a00fc1cb871bb2d309bfd8c88..d1e5a78c616b0e71f9466c201b74e06ce0d7a08f 100644 --- a/arch/arm/kernel/ptrace.c +++ b/arch/arm/kernel/ptrace.c @@ -27,9 +27,6 @@ #include #include -#define CREATE_TRACE_POINTS -#include - #define REG_PC 15 #define REG_PSR 16 /* @@ -737,6 +734,11 @@ long arch_ptrace(struct task_struct *child, long request, ret = ptrace_write_user(child, addr, data); break; + case PTRACE_SYSEMU: + case PTRACE_SYSEMU_SINGLESTEP: + ret = -EIO; + break; + case PTRACE_GETREGS: ret = copy_regset_to_user(child, &user_arm_view, REGSET_GPR, @@ -821,90 +823,3 @@ long arch_ptrace(struct task_struct *child, long request, return ret; } - -enum ptrace_syscall_dir { - PTRACE_SYSCALL_ENTER = 0, - PTRACE_SYSCALL_EXIT, -}; - -static void report_syscall_enter(struct pt_regs *regs) -{ - unsigned long ip; - - /* - * IP is used to denote syscall entry/exit: - * IP = 0 -> entry - */ - ip = regs->ARM_ip; - regs->ARM_ip = PTRACE_SYSCALL_ENTER; - - if (ptrace_report_syscall_entry(regs)) - current_thread_info()->abi_syscall = -1; - - regs->ARM_ip = ip; -} - -static void report_syscall_exit(struct pt_regs *regs) -{ - unsigned long ip; - - /* - * IP is used to denote syscall entry/exit: - * IP = 1 -> exit - */ - ip = regs->ARM_ip; - regs->ARM_ip = PTRACE_SYSCALL_EXIT; - - ptrace_report_syscall_exit(regs, 0); - - regs->ARM_ip = ip; -} - -asmlinkage int syscall_trace_enter(struct pt_regs *regs) -{ - int scno; - - if (test_thread_flag(TIF_SYSCALL_TRACE)) - report_syscall_enter(regs); - - /* Do seccomp after ptrace; syscall may have changed. */ -#ifdef CONFIG_HAVE_ARCH_SECCOMP_FILTER - if (secure_computing() == -1) - return -1; -#else - /* XXX: remove this once OABI gets fixed */ - secure_computing_strict(syscall_get_nr(current, regs)); -#endif - - /* Tracer or seccomp may have changed syscall. */ - scno = syscall_get_nr(current, regs); - - if (test_thread_flag(TIF_SYSCALL_TRACEPOINT)) - trace_sys_enter(regs, scno); - - audit_syscall_entry(scno, regs->ARM_r0, regs->ARM_r1, regs->ARM_r2, - regs->ARM_r3); - - return scno; -} - -void syscall_trace_exit(struct pt_regs *regs) -{ - /* - * Audit the syscall before anything else, as a debugger may - * come in and change the current registers. - */ - audit_syscall_exit(regs); - - /* - * Note that we haven't updated the ->syscall field for the - * current thread. This isn't a problem because it will have - * been set on syscall entry and there hasn't been an opportunity - * for a PTRACE_SET_SYSCALL since then. - */ - if (test_thread_flag(TIF_SYSCALL_TRACEPOINT)) - trace_sys_exit(regs, regs_return_value(regs)); - - if (test_thread_flag(TIF_SYSCALL_TRACE)) - report_syscall_exit(regs); -} diff --git a/arch/arm/kernel/signal.c b/arch/arm/kernel/signal.c index a39ee14ec5b70b3738ecef537709326ef1e8217c..35d2bb3cd2b442dac164548037262e065fbfe12a 100644 --- a/arch/arm/kernel/signal.c +++ b/arch/arm/kernel/signal.c @@ -4,6 +4,7 @@ * * Copyright (C) 1995-2009 Russell King */ +#include #include #include #include @@ -535,7 +536,7 @@ static void handle_signal(struct ksignal *ksig, struct pt_regs *regs) * the kernel can handle, and then we build all the user-level signal handling * stack-frames in one go after that. */ -static void arch_do_signal_or_restart(struct pt_regs *regs) +void arch_do_signal_or_restart(struct pt_regs *regs) { unsigned int retval = 0, continue_addr = 0, restart_addr = 0; bool syscall = (syscall_get_nr(current, regs) != -1); @@ -598,34 +599,6 @@ static void arch_do_signal_or_restart(struct pt_regs *regs) return; } -void do_work_pending(struct pt_regs *regs, unsigned int thread_flags) -{ - /* - * The assembly code enters us with IRQs off, but it hasn't - * informed the tracing code of that for efficiency reasons. - * Update the trace code with the current status. - */ - trace_hardirqs_off(); - do { - if (likely(thread_flags & _TIF_NEED_RESCHED)) { - schedule(); - } else { - if (unlikely(!user_mode(regs))) - return; - local_irq_enable(); - if (thread_flags & (_TIF_SIGPENDING | _TIF_NOTIFY_SIGNAL)) { - arch_do_signal_or_restart(regs); - } else if (thread_flags & _TIF_UPROBE) { - uprobe_notify_resume(regs); - } else { - resume_user_mode_work(regs); - } - } - local_irq_disable(); - thread_flags = read_thread_flags(); - } while (thread_flags & _TIF_WORK_MASK); -} - struct page *get_signal_page(void) { unsigned long ptr; diff --git a/arch/arm/kernel/syscall.c b/arch/arm/kernel/syscall.c index 6f518ec3870fb36a64d1b671b20f96d63a52bf78..ed3ab51283c06c1398ece2ad3ee1fae16cd03ee8 100644 --- a/arch/arm/kernel/syscall.c +++ b/arch/arm/kernel/syscall.c @@ -1,25 +1,21 @@ // SPDX-License-Identifier: GPL-2.0 +#include #include #include -static inline bool has_syscall_work(unsigned long flags) -{ - return unlikely(flags & _TIF_SYSCALL_WORK); -} - int invoke_syscall_asm(void *table, struct pt_regs *regs, int scno); __ADDRESSABLE(invoke_syscall_asm); -__visible int invoke_syscall(void *table, struct pt_regs *regs, int scno) +__visible void invoke_syscall(void *table, struct pt_regs *regs, int scno) { - unsigned long flags = read_thread_flags(); int ret; - if (has_syscall_work(flags)) { - scno = syscall_trace_enter(regs); - if (scno == -1) - goto trace_exit_nosave; + scno = syscall_enter_from_user_mode(regs, scno); + /* When tracing syscall -1 means "skip syscall" */ + if (scno < 0) { + ret = 0; + goto exit_save; } if (scno < NR_syscalls) { @@ -35,13 +31,7 @@ __visible int invoke_syscall(void *table, struct pt_regs *regs, int scno) ret = sys_ni_syscall(); exit_save: - /* Save return value from syscall */ - regs->ARM_r0 = ret; - if (!has_syscall_work(flags)) - return 0; - -trace_exit_nosave: - local_irq_enable(); - syscall_trace_exit(regs); - return 1; + syscall_set_return_value(current, regs, 0, ret); + + syscall_exit_to_user_mode(regs); } diff --git a/arch/arm/kernel/traps.c b/arch/arm/kernel/traps.c index 942c77767919466d5b89d83d8eb77032b119473e..57e74b49c52533babe177a8a2e0160a83c9bde05 100644 --- a/arch/arm/kernel/traps.c +++ b/arch/arm/kernel/traps.c @@ -502,31 +502,6 @@ asmlinkage void do_undefinstr(struct pt_regs *regs) } NOKPROBE_SYMBOL(do_undefinstr) -/* - * Handle FIQ similarly to NMI on x86 systems. - * - * The runtime environment for NMIs is extremely restrictive - * (NMIs can pre-empt critical sections meaning almost all locking is - * forbidden) meaning this default FIQ handling must only be used in - * circumstances where non-maskability improves robustness, such as - * watchdog or debug logic. - * - * This handler is not appropriate for general purpose use in drivers - * platform code and can be overrideen using set_fiq_handler. - */ -asmlinkage void __exception_irq_entry handle_fiq_as_nmi(struct pt_regs *regs) -{ - struct pt_regs *old_regs = set_irq_regs(regs); - - nmi_enter(); - - /* nop. FIQ handlers for special arch/arm features can be added here. */ - - nmi_exit(); - - set_irq_regs(old_regs); -} - /* * bad_mode handles the impossible case in the vectors. If you see one of * these, then it's extremely serious, and could mean you have buggy hardware. From patchwork Tue Feb 25 09:55:16 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13989784 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3D6D1C021B2 for ; Tue, 25 Feb 2025 10:54:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=hG0/vjAYb5Ulc6jjnKhm2XeOXEYOwE4Q8HmW4VgPla8=; b=raQjESagpOk7CmqjFmdRTy3EeK e59h34RVwm7jRZeHy+VPVmtCkpNj6BGirstzKe6Kg3OtH+GOD1Wx4wOX5QWZuCXrLS46g+C0i2f92 CPcHr+w0nrq4FrLIHwhXiRkVM7ooEzjGAPqFKv55Z/4wfLaDS9loX7aExYzwqXoCGXhfbDo79hm/n SVEj7ZArjEpvyDF2JoARIAjrSHvpne+0kraCaXTosDjHpFCfWe6+2Q9Sb7OG2uoxHFg79WlddC/A7 KJgj4+PdBpG4QSiEyIEEVEIR/1slmplQ6YfSt4opP3NJfcro0tnMt5Xsty1rBEnjtL4AIIgc/mF6O 0i8FwxRw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tmsZY-0000000GuO3-3jWs; Tue, 25 Feb 2025 10:53:56 +0000 Received: from mail-lf1-x134.google.com ([2a00:1450:4864:20::134]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tmrf4-0000000Ghp6-3MxQ for linux-arm-kernel@lists.infradead.org; Tue, 25 Feb 2025 09:55:36 +0000 Received: by mail-lf1-x134.google.com with SMTP id 2adb3069b0e04-54622940ef7so6100526e87.3 for ; Tue, 25 Feb 2025 01:55:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1740477333; x=1741082133; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=hG0/vjAYb5Ulc6jjnKhm2XeOXEYOwE4Q8HmW4VgPla8=; b=wiXJ4EkMp+BWSZZVKLtx3btip1LvHYN7hbvOnO6tOnDW0bDLgjIIAx6V8Wo64pvvSE N0baEHC8XH6Ddyq0yoWi5nWsitewAjmWJaQAeX9jFbgyPzv8xI6pl3XvHw2//n1ndCfK 7v6sZMXx3oPJ6AWzI5OW6PY+3a3eiimzPpSKh4yBn9Ze9UY49bjCFXRf1E11slUOfiTm qBOcMqa2g3w4iVlJ8eipctrBo3mMicv9nB3fT5hLCRCUnYFiFe561IdjIpYvmlsJjFLt dl67GwC0u+KDBGxHXetPUfRr2ZEbR3JNYcRn1xtcs5ymercqPb+YzLPNcVk27Nxdoer0 Vbzw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740477333; x=1741082133; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=hG0/vjAYb5Ulc6jjnKhm2XeOXEYOwE4Q8HmW4VgPla8=; b=CSeBOdIFgoljFQGQO0oCFJCXlrBWEX9cA6vJ+BzHb1Gdt/kJvLc9vjbQyAMgCjWIfl Ry+uITzyl+FgKlQMc+nxHFGtpzxyZc1mF0FVnn5+WYc0z9Qhf+FrGQYxR9hNu8tHeWmQ Wzbm9R9TiuauFS1m+xOTJs7h1g7HRXDym8pbpnmO6ALqqXyWz7OJR4M+LzEAYSTtcUBO HQzzYuz6vi8uc0XKWgLEt8L/i5emnlFCo8BI7g1nGT4rsWtSFdUpgpglELSY/ZiWEzTq 5d0IQzBpgtNJIHwP3WEgK4daMmVFg5a2n/U5FZ3cdGtKFiLSzKoFIfxiMRlJiUaZ7gN4 Srlw== X-Gm-Message-State: AOJu0YzKymcpRV0rMcoe4NjetWwc9/JO/B0rJV0jUnriHoRYUCOnR35R +U+hzaD2lvC62/l/Zyb/uLyWIsFQwzlZNfoF2aH0SqmOTzSSRyO7vL7MdQsRVjI= X-Gm-Gg: ASbGncuu6/sLAV9asyiRhxSq7VzyFHay6PcBWT1qb0QPDpRUaWOzet8wFnaaZg/O2dp iQOqdyAnwm3Ld/UINlpp4eUgtU/FxGNKJOX3JJVuVHiseELW8UFcThE/XW5XbV9E6yFtEWwKkeQ RN6G81P9+iGRTa+CZOb075P4lTTVKd8+R3AmzwEzPk/tVVQgknf5lKRw4Fxp3FBjBVRYV2fp0bB o156NBG//tN+t0wftFFxB74KbyJ2wsEXBzebWi2XPKkaGhFe9XgJDFoYPeJn356GS+9B0HbugRi CyamHWMFnifE+vyhvQHq0JcOo0T2LkGAG4Z9 X-Google-Smtp-Source: AGHT+IH5uBgAMA9dHOegGGT93vBd6r8QwClPslh2DXLtCCBanyA3O0/Gp1EuUpD/mB0oY2I3FDeTFQ== X-Received: by 2002:a05:6512:104e:b0:546:2ea9:6666 with SMTP id 2adb3069b0e04-54838f4e48emr8380278e87.34.1740477333117; Tue, 25 Feb 2025 01:55:33 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id 2adb3069b0e04-548514b261esm132867e87.24.2025.02.25.01.55.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 25 Feb 2025 01:55:32 -0800 (PST) From: Linus Walleij Date: Tue, 25 Feb 2025 10:55:16 +0100 Subject: [PATCH v5 29/31] ARM: entry: Handle dabt, pabt, and und as interrupts MIME-Version: 1.0 Message-Id: <20250225-arm-generic-entry-v5-29-2f02313653e5@linaro.org> References: <20250225-arm-generic-entry-v5-0-2f02313653e5@linaro.org> In-Reply-To: <20250225-arm-generic-entry-v5-0-2f02313653e5@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250225_015535_028478_E94CA270 X-CRM114-Status: GOOD ( 21.68 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org While it isn't entirely intuitive, it appears that any kind of exception such as data or prefetch abort ("page faults") need to be handled as some kind of "interrupts" when using generic entry. At least this is what other platforms are doing. The same goes for undefined instruction handling, i.e. floating point emulation in the kernel (und-exceptions). This is necessary for the context checking to pass: without this patch, a whole slew of warnings start to trigger from syscall_exit_to_user_mode_prepare() CT_WARN_ON(ct_state() != CT_STATE_KERNEL), i.e. syscalls seems to exit from user mode to user mode (not good), because the page faults screws up the context tracker. This patch restores the order. If this seems like the previous patch introduces a regression that is then fixed in this patch, it can simply be squashed into the former: having this rewrite separately surely makes development and review easier. Signed-off-by: Linus Walleij --- arch/arm/include/asm/entry.h | 3 +++ arch/arm/include/asm/traps.h | 2 +- arch/arm/kernel/entry-armv.S | 7 +++---- arch/arm/kernel/entry.c | 31 +++++++++++++++++++++++++++++++ arch/arm/kernel/traps.c | 2 +- arch/arm/mm/abort-ev4.S | 2 +- arch/arm/mm/abort-ev4t.S | 2 +- arch/arm/mm/abort-ev5t.S | 4 ++-- arch/arm/mm/abort-ev5tj.S | 6 +++--- arch/arm/mm/abort-ev6.S | 2 +- arch/arm/mm/abort-ev7.S | 2 +- arch/arm/mm/abort-lv4t.S | 36 ++++++++++++++++++------------------ arch/arm/mm/abort-macro.S | 2 +- arch/arm/mm/abort-nommu.S | 2 +- arch/arm/mm/fault.c | 4 ++-- arch/arm/mm/fault.h | 8 ++++---- arch/arm/mm/pabort-legacy.S | 2 +- arch/arm/mm/pabort-v6.S | 2 +- arch/arm/mm/pabort-v7.S | 2 +- 19 files changed, 77 insertions(+), 44 deletions(-) diff --git a/arch/arm/include/asm/entry.h b/arch/arm/include/asm/entry.h index df2dd14ab51a586b83b6da6bbf9bd99858c1ddf4..de722084d5309194e02cb87db64ec32b9e6cd627 100644 --- a/arch/arm/include/asm/entry.h +++ b/arch/arm/include/asm/entry.h @@ -4,6 +4,9 @@ struct pt_regs; +void arm_und_handler(struct pt_regs *regs); +void arm_dabt_handler(unsigned long addr, unsigned int fsr, struct pt_regs *regs); +void arm_pabt_handler(unsigned long addr, unsigned int ifsr, struct pt_regs *regs); void arm_irq_handler(struct pt_regs *regs, int mode); void arm_fiq_handler(struct pt_regs *regs); void arm_exit_to_user_mode(struct pt_regs *regs); diff --git a/arch/arm/include/asm/traps.h b/arch/arm/include/asm/traps.h index f8695104c72f2f450092839c7c50920e540e594b..0ca98271e10a210225c1714f5b24668cebab2def 100644 --- a/arch/arm/include/asm/traps.h +++ b/arch/arm/include/asm/traps.h @@ -37,7 +37,7 @@ extern void ptrace_break(struct pt_regs *regs); extern void *vectors_page; asmlinkage void dump_backtrace_stm(u32 *stack, u32 instruction, const char *loglvl); -asmlinkage void do_undefinstr(struct pt_regs *regs); +void do_undefinstr(struct pt_regs *regs); asmlinkage void bad_mode(struct pt_regs *regs, int reason); int arm_syscall(int no, struct pt_regs *regs); asmlinkage void baddataabort(int code, unsigned long instr, struct pt_regs *regs); diff --git a/arch/arm/kernel/entry-armv.S b/arch/arm/kernel/entry-armv.S index 498a22f5fa8a6516d1eab584f0523c3d6c6e4926..ae2f952beea7611f0abc7bd299fc944335a21219 100644 --- a/arch/arm/kernel/entry-armv.S +++ b/arch/arm/kernel/entry-armv.S @@ -230,7 +230,7 @@ __und_fault: ldr r2, [r0, #S_PC] sub r2, r2, r1 str r2, [r0, #S_PC] - b do_undefinstr + b arm_und_handler ENDPROC(__und_fault) .align 5 @@ -449,9 +449,8 @@ __pabt_usr: ENTRY(ret_from_exception) UNWIND(.fnstart ) UNWIND(.cantunwind ) - get_thread_info tsk - mov why, #0 - b ret_to_user + disable_irq_notrace + b ret_to_user_from_irq UNWIND(.fnend ) ENDPROC(__pabt_usr) ENDPROC(ret_from_exception) diff --git a/arch/arm/kernel/entry.c b/arch/arm/kernel/entry.c index 08b8680233e5aae6e477f9bf15d7f06d7fd3e5af..01e4339ccdb4951e04a89fb91ad7c6e1991e09af 100644 --- a/arch/arm/kernel/entry.c +++ b/arch/arm/kernel/entry.c @@ -8,8 +8,39 @@ #include #include #include +#include #include "irq.h" +#include "../mm/fault.h" + +noinstr asmlinkage void arm_und_handler(struct pt_regs *regs) +{ + irqentry_state_t state = irqentry_enter(regs); + + do_undefinstr(regs); + + irqentry_exit(regs, state); +} + +noinstr asmlinkage void arm_dabt_handler(unsigned long addr, unsigned int fsr, + struct pt_regs *regs) +{ + irqentry_state_t state = irqentry_enter(regs); + + do_DataAbort(addr, fsr, regs); + + irqentry_exit(regs, state); +} + +noinstr asmlinkage void arm_pabt_handler(unsigned long addr, unsigned int ifsr, + struct pt_regs *regs) +{ + irqentry_state_t state = irqentry_enter(regs); + + do_PrefetchAbort(addr, ifsr, regs); + + irqentry_exit(regs, state); +} static void noinstr handle_arm_irq(void *data) { diff --git a/arch/arm/kernel/traps.c b/arch/arm/kernel/traps.c index 57e74b49c52533babe177a8a2e0160a83c9bde05..d3a689bd05955f1ae46a6341e456bb097e831311 100644 --- a/arch/arm/kernel/traps.c +++ b/arch/arm/kernel/traps.c @@ -449,7 +449,7 @@ int call_undef_hook(struct pt_regs *regs, unsigned int instr) return fn ? fn(regs, instr) : 1; } -asmlinkage void do_undefinstr(struct pt_regs *regs) +noinstr void do_undefinstr(struct pt_regs *regs) { unsigned int instr; void __user *pc; diff --git a/arch/arm/mm/abort-ev4.S b/arch/arm/mm/abort-ev4.S index a10bcb89594dd38ce31ca30bf97d68cf421afc42..68d8ae2f4e9c0955c9f08112b4ff1887867d550a 100644 --- a/arch/arm/mm/abort-ev4.S +++ b/arch/arm/mm/abort-ev4.S @@ -24,4 +24,4 @@ ENTRY(v4_early_abort) bic r1, r1, #1 << 11 | 1 << 10 @ clear bits 11 and 10 of FSR tst r3, #1 << 20 @ L = 1 -> write? orreq r1, r1, #1 << 11 @ yes. - b do_DataAbort + b arm_dabt_handler diff --git a/arch/arm/mm/abort-ev4t.S b/arch/arm/mm/abort-ev4t.S index 14743a2f6997fcae0eeb55f53dec07b7b2ec227c..76c6cfa79e2a32e34b6f35a15c71c1af407956f3 100644 --- a/arch/arm/mm/abort-ev4t.S +++ b/arch/arm/mm/abort-ev4t.S @@ -25,4 +25,4 @@ ENTRY(v4t_early_abort) bic r1, r1, #1 << 11 | 1 << 10 @ clear bits 11 and 10 of FSR tst r3, #1 << 20 @ check write orreq r1, r1, #1 << 11 - b do_DataAbort + b arm_dabt_handler diff --git a/arch/arm/mm/abort-ev5t.S b/arch/arm/mm/abort-ev5t.S index 98c523118820798668bf04d065ad86ea05fb6d2c..fda1e54debc44b380fb3f3af95c1f198a9cc008a 100644 --- a/arch/arm/mm/abort-ev5t.S +++ b/arch/arm/mm/abort-ev5t.S @@ -25,7 +25,7 @@ ENTRY(v5t_early_abort) uaccess_disable ip @ disable user access bic r1, r1, #1 << 11 @ clear bits 11 of FSR teq_ldrd tmp=ip, insn=r3 @ insn was LDRD? - beq do_DataAbort @ yes + beq arm_dabt_handler @ yes tst r3, #1 << 20 @ check write orreq r1, r1, #1 << 11 - b do_DataAbort + b arm_dabt_handler diff --git a/arch/arm/mm/abort-ev5tj.S b/arch/arm/mm/abort-ev5tj.S index fec72f4fbaf508597d826e58d0dc084ee6e58dd0..a786a7d69e6ae23aa1125ccd41deaaaa743601e1 100644 --- a/arch/arm/mm/abort-ev5tj.S +++ b/arch/arm/mm/abort-ev5tj.S @@ -22,12 +22,12 @@ ENTRY(v5tj_early_abort) mrc p15, 0, r0, c6, c0, 0 @ get FAR bic r1, r1, #1 << 11 | 1 << 10 @ clear bits 11 and 10 of FSR tst r5, #PSR_J_BIT @ Java? - bne do_DataAbort + bne arm_dabt_handler do_thumb_abort fsr=r1, pc=r4, psr=r5, tmp=r3 ldreq r3, [r4] @ read aborted ARM instruction uaccess_disable ip @ disable userspace access teq_ldrd tmp=ip, insn=r3 @ insn was LDRD? - beq do_DataAbort @ yes + beq arm_dabt_handler @ yes tst r3, #1 << 20 @ L = 0 -> write orreq r1, r1, #1 << 11 @ yes. - b do_DataAbort + b arm_dabt_handler diff --git a/arch/arm/mm/abort-ev6.S b/arch/arm/mm/abort-ev6.S index 836dc1299243415faaac1a93dce35ac46ae1a7da..2366236053eb86484cb179ff572f06db6559426b 100644 --- a/arch/arm/mm/abort-ev6.S +++ b/arch/arm/mm/abort-ev6.S @@ -42,4 +42,4 @@ ENTRY(v6_early_abort) orreq r1, r1, #1 << 11 @ yes. #endif 1: uaccess_disable ip @ disable userspace access - b do_DataAbort + b arm_dabt_handler diff --git a/arch/arm/mm/abort-ev7.S b/arch/arm/mm/abort-ev7.S index 53fb41c24774db985eddc370904aa7666527f538..72b6eba101555651d7afd749fd5d7715a2964372 100644 --- a/arch/arm/mm/abort-ev7.S +++ b/arch/arm/mm/abort-ev7.S @@ -18,5 +18,5 @@ ENTRY(v7_early_abort) mrc p15, 0, r1, c5, c0, 0 @ get FSR mrc p15, 0, r0, c6, c0, 0 @ get FAR uaccess_disable ip @ disable userspace access - b do_DataAbort + b arm_dabt_handler ENDPROC(v7_early_abort) diff --git a/arch/arm/mm/abort-lv4t.S b/arch/arm/mm/abort-lv4t.S index fbd60a120f6684c56c63cea10b00200765473f1d..3fef9c1479c17bb07652e43e0b8b0683e00c4b2f 100644 --- a/arch/arm/mm/abort-lv4t.S +++ b/arch/arm/mm/abort-lv4t.S @@ -46,8 +46,8 @@ ENTRY(v4t_late_abort) /* 9 */ b .data_arm_ldmstm @ ldm*b rn, /* a */ b .data_unknown /* b */ b .data_unknown -/* c */ b do_DataAbort @ ldc rd, [rn], #m @ Same as ldr rd, [rn], #m -/* d */ b do_DataAbort @ ldc rd, [rn, #m] +/* c */ b arm_dabt_handler @ ldc rd, [rn], #m @ Same as ldr rd, [rn], #m +/* d */ b arm_dabt_handler @ ldc rd, [rn, #m] /* e */ b .data_unknown /* f */ b .data_unknown @@ -60,7 +60,7 @@ ENTRY(v4t_late_abort) .data_arm_ldmstm: tst r8, #1 << 21 @ check writeback bit - beq do_DataAbort @ no writeback -> no fixup + beq arm_dabt_handler @ no writeback -> no fixup str r9, [sp, #-4]! mov r7, #0x11 orr r7, r7, #0x1100 @@ -81,11 +81,11 @@ ENTRY(v4t_late_abort) addeq r7, r7, r6, lsl #2 @ Undo decrement str r7, [r2, r9, lsr #14] @ Put register 'Rn' ldr r9, [sp], #4 - b do_DataAbort + b arm_dabt_handler .data_arm_lateldrhpre: tst r8, #1 << 21 @ Check writeback bit - beq do_DataAbort @ No writeback -> no fixup + beq arm_dabt_handler @ No writeback -> no fixup .data_arm_lateldrhpost: str r9, [sp, #-4]! and r9, r8, #0x00f @ get Rm / low nibble of immediate value @@ -101,14 +101,14 @@ ENTRY(v4t_late_abort) addeq r7, r7, r6 @ Undo decrement str r7, [r2, r9, lsr #14] @ Put register 'Rn' ldr r9, [sp], #4 - b do_DataAbort + b arm_dabt_handler .data_arm_lateldrpreconst: tst r8, #1 << 21 @ check writeback bit - beq do_DataAbort @ no writeback -> no fixup + beq arm_dabt_handler @ no writeback -> no fixup .data_arm_lateldrpostconst: movs r6, r8, lsl #20 @ Get offset - beq do_DataAbort @ zero -> no fixup + beq arm_dabt_handler @ zero -> no fixup str r9, [sp, #-4]! and r9, r8, #15 << 16 @ Extract 'n' from instruction ldr r7, [r2, r9, lsr #14] @ Get register 'Rn' @@ -117,11 +117,11 @@ ENTRY(v4t_late_abort) addeq r7, r7, r6, lsr #20 @ Undo decrement str r7, [r2, r9, lsr #14] @ Put register 'Rn' ldr r9, [sp], #4 - b do_DataAbort + b arm_dabt_handler .data_arm_lateldrprereg: tst r8, #1 << 21 @ check writeback bit - beq do_DataAbort @ no writeback -> no fixup + beq arm_dabt_handler @ no writeback -> no fixup .data_arm_lateldrpostreg: and r7, r8, #15 @ Extract 'm' from instruction ldr r6, [r2, r7, lsl #2] @ Get register 'Rm' @@ -180,10 +180,10 @@ ENTRY(v4t_late_abort) /* 3 */ b .data_unknown /* 4 */ b .data_unknown /* 5 */ b .data_thumb_reg -/* 6 */ b do_DataAbort -/* 7 */ b do_DataAbort -/* 8 */ b do_DataAbort -/* 9 */ b do_DataAbort +/* 6 */ b arm_dabt_handler +/* 7 */ b arm_dabt_handler +/* 8 */ b arm_dabt_handler +/* 9 */ b arm_dabt_handler /* A */ b .data_unknown /* B */ b .data_thumb_pushpop /* C */ b .data_thumb_ldmstm @@ -193,10 +193,10 @@ ENTRY(v4t_late_abort) .data_thumb_reg: tst r8, #1 << 9 - beq do_DataAbort + beq arm_dabt_handler tst r8, #1 << 10 @ If 'S' (signed) bit is set movne r1, #0 @ it must be a load instr - b do_DataAbort + b arm_dabt_handler .data_thumb_pushpop: tst r8, #1 << 10 @@ -217,7 +217,7 @@ ENTRY(v4t_late_abort) subne r7, r7, r6, lsl #2 @ decrement SP if POP str r7, [r2, #13 << 2] ldr r9, [sp], #4 - b do_DataAbort + b arm_dabt_handler .data_thumb_ldmstm: str r9, [sp, #-4]! @@ -234,4 +234,4 @@ ENTRY(v4t_late_abort) sub r7, r7, r6, lsl #2 @ always decrement str r7, [r2, r9, lsr #6] ldr r9, [sp], #4 - b do_DataAbort + b arm_dabt_handler diff --git a/arch/arm/mm/abort-macro.S b/arch/arm/mm/abort-macro.S index bacf53fd0b70c6307e74ef8601d8dcc7db292700..30bb5e2135862902951b5077b0b860fd12ed8c28 100644 --- a/arch/arm/mm/abort-macro.S +++ b/arch/arm/mm/abort-macro.S @@ -20,7 +20,7 @@ orreq \tmp, \tmp, #1 << 11 @ Set L-bit if yes tst \tmp, #1 << 11 @ L = 0 -> write orreq \fsr, \fsr, #1 << 11 @ yes. - b do_DataAbort + b arm_dabt_handler not_thumb: .endm diff --git a/arch/arm/mm/abort-nommu.S b/arch/arm/mm/abort-nommu.S index 6e2366a263219b379f4fdb43cc8e5413fb36e52a..573a09529ac7794f8004680ab350750de7c8e15d 100644 --- a/arch/arm/mm/abort-nommu.S +++ b/arch/arm/mm/abort-nommu.S @@ -17,5 +17,5 @@ ENTRY(nommu_early_abort) mov r0, #0 @ clear r0, r1 (no FSR/FAR) mov r1, #0 - b do_DataAbort + b arm_dabt_handler ENDPROC(nommu_early_abort) diff --git a/arch/arm/mm/fault.c b/arch/arm/mm/fault.c index ab01b51de5590bff6c2acaf446d01c589a5f7987..78afb701b34a17d8d841762944eb5f8af6843dc4 100644 --- a/arch/arm/mm/fault.c +++ b/arch/arm/mm/fault.c @@ -588,7 +588,7 @@ hook_fault_code(int nr, int (*fn)(unsigned long, unsigned int, struct pt_regs *) /* * Dispatch a data abort to the relevant handler. */ -asmlinkage void +void do_DataAbort(unsigned long addr, unsigned int fsr, struct pt_regs *regs) { const struct fsr_info *inf = fsr_info + fsr_fs(fsr); @@ -618,7 +618,7 @@ hook_ifault_code(int nr, int (*fn)(unsigned long, unsigned int, struct pt_regs * ifsr_info[nr].name = name; } -asmlinkage void +void do_PrefetchAbort(unsigned long addr, unsigned int ifsr, struct pt_regs *regs) { const struct fsr_info *inf = ifsr_info + fsr_fs(ifsr); diff --git a/arch/arm/mm/fault.h b/arch/arm/mm/fault.h index e8f8c190254442a015a6cbc179602589cfddb0ea..36a7e7138c310a8311458207d2b694cb44c63b93 100644 --- a/arch/arm/mm/fault.h +++ b/arch/arm/mm/fault.h @@ -37,9 +37,9 @@ static inline int fsr_fs(unsigned int fsr) void do_bad_area(unsigned long addr, unsigned int fsr, struct pt_regs *regs); void early_abt_enable(void); -asmlinkage void do_DataAbort(unsigned long addr, unsigned int fsr, - struct pt_regs *regs); -asmlinkage void do_PrefetchAbort(unsigned long addr, unsigned int ifsr, - struct pt_regs *regs); +void do_DataAbort(unsigned long addr, unsigned int fsr, + struct pt_regs *regs); +void do_PrefetchAbort(unsigned long addr, unsigned int ifsr, + struct pt_regs *regs); #endif /* __ARCH_ARM_FAULT_H */ diff --git a/arch/arm/mm/pabort-legacy.S b/arch/arm/mm/pabort-legacy.S index b2ffce4201062e3ec2045364ddc454cf706bab8d..26c62e568638d37133661f4f5b8dec430593f8eb 100644 --- a/arch/arm/mm/pabort-legacy.S +++ b/arch/arm/mm/pabort-legacy.S @@ -18,5 +18,5 @@ ENTRY(legacy_pabort) mov r0, r4 mov r1, #5 - b do_PrefetchAbort + b arm_pabt_handler ENDPROC(legacy_pabort) diff --git a/arch/arm/mm/pabort-v6.S b/arch/arm/mm/pabort-v6.S index 8686265dc9418b29381942bfd87a937a3234d46e..25abd11a35253bf464fe9fd7fc14fd11bc564dcd 100644 --- a/arch/arm/mm/pabort-v6.S +++ b/arch/arm/mm/pabort-v6.S @@ -18,5 +18,5 @@ ENTRY(v6_pabort) mov r0, r4 mrc p15, 0, r1, c5, c0, 1 @ get IFSR - b do_PrefetchAbort + b arm_pabt_handler ENDPROC(v6_pabort) diff --git a/arch/arm/mm/pabort-v7.S b/arch/arm/mm/pabort-v7.S index 9c70b1a21dc9204f24524df9905fbc077a82f2dc..e05c7d44d307adeba6759213374b2aa328c693da 100644 --- a/arch/arm/mm/pabort-v7.S +++ b/arch/arm/mm/pabort-v7.S @@ -18,5 +18,5 @@ ENTRY(v7_pabort) mrc p15, 0, r0, c6, c0, 2 @ get IFAR mrc p15, 0, r1, c5, c0, 1 @ get IFSR - b do_PrefetchAbort + b arm_pabt_handler ENDPROC(v7_pabort) From patchwork Tue Feb 25 09:55:17 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13989855 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9C863C021B2 for ; Tue, 25 Feb 2025 11:20:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=BLyynCt450Utx2Z4sB12bYa6WPePLKZjdTj9AHDawWQ=; b=ViS0GgBIqeH/acTAjxxtmQTiKF H1Mw20VqFh7oYSjcFamY0wXgx2LRmLoL9Wm2+jySW0ODunOGFbpmxkyON5dRiQu2FtLfEboQVb1dE trqjWlfzq/QOCekKWaiA/dhhOJdWwbibfjruAKrJZcunKpp5uFF9+Qg15Xw6FGFGSCf5bhbkWPdmb zt37nULr0v46bsWMWaAL9pYdpUwkX2+f6qvjlRviCUWTcmvIDpGPFi/NpyxP1T/5Nr+7B/We926/a eWvoh26S1UvWHjLH/JQHrWAQzjt1qq93BRjqW2CpiXzokt40fAZ6Os7t8TfcGJfxZIpJw8kNj2Ksa adBMsaTg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tmsyi-0000000H0fF-3Ioh; Tue, 25 Feb 2025 11:19:56 +0000 Received: from mail-lf1-x132.google.com ([2a00:1450:4864:20::132]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tmrf5-0000000Ghpa-3zds for linux-arm-kernel@lists.infradead.org; Tue, 25 Feb 2025 09:55:37 +0000 Received: by mail-lf1-x132.google.com with SMTP id 2adb3069b0e04-5461cb12e39so5193852e87.2 for ; Tue, 25 Feb 2025 01:55:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1740477334; x=1741082134; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=BLyynCt450Utx2Z4sB12bYa6WPePLKZjdTj9AHDawWQ=; b=Mjsa+PHbFrIbqDYCVgaumbeIsLUnQqk0Uw+7PNmpTq7d/H6Y08PrZRrwmXS3+Q2/jT l22aJA66gWAH58+2qOHhlsM2+vVRBgQtcZv2Vf4DLINJJPWWQ4R9bHqUayi/vrr+mCFO B1puBek4Y390+oeeIiJ8Az08XP90xeH5vViJ6/5xIrP5eIa8X3NIl7c3Jr6IPBYaJdO+ gFZcR+vHpYTn7AJf22h4DiNZhPVFZDv4N3EyA9+//RqmtOxWdbT/JRkthr2BDUgRuvr7 1IcygF14UGl0SVAAaWxIHM7tCi0C179EqzzwHenOH/GL1W1GtbUFOlVrCmb+/TgnlU8w uH/A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740477334; x=1741082134; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=BLyynCt450Utx2Z4sB12bYa6WPePLKZjdTj9AHDawWQ=; b=Y0DTgQv9MQtBaO2hoea0MGTOcpFohxDFQrufubnx0+6+TidhE60DY4uVvveCZY3Vte LFEK8tWQcxd80bP0eS2jJty4lX8gnh7uTY7/da2r/HsDedgGQwFB08OWLq11XTXGu3iM xvgclWkTsVyb4YSoNw35qJGySeZrIa24uSgXANcOpnNMVY8FthhL8xjH/cWJQyaNwZ64 6SLF5mQm83xCxuAyzga7VvRTR0iJuxobNRL608kCo4P16tjZ9C/kXIguChLURQJnsXGo +GRx5F2ooXpoMop15/4uVWHRkK8kYR0YY72GQdCDTlNPa6aV4vUpDFEo0dmTXn8tWqnk KjYg== X-Gm-Message-State: AOJu0YzPd+uc62lg2dASx1xCHV8cNvVHNauFssSTmPMDZgZig2YAIE2Q yCzx7Wr0F8Upqv3kiO3K9d3Pfh2urxwTCcD+Fpab/xbLspnbGmW94CLai+NmFiY= X-Gm-Gg: ASbGnculDi6WzcJDtCvvMPzipOnza9R7vAFMoLXVMT5mzC2vP3M5YK8lmfsyBFUJl90 jngFZcsuDeuct1uzGh0GsFrbqKrgnxMaE0n5yFfHaRBR7Er7G9vSX8ZtWZMObjW6maFL5n5qZ1i 0Lpy5aNG5pHDLaKcAHkpKRAjbGQXFoSXC3X5vHIHRTuWxgcqvlDtJO2k5Cc4BSp7XyXmY1SZI7r FkTu0rJoYDWFAqLcX1/sW8Drq1gdAQO+mP3Lwgy/43J+LM/9jetLlkah7BpZxMItHkTpiuJD8YK VZYRlqb/Og+5KarU5Oa1hNmYYm6DUMVqgk2E X-Google-Smtp-Source: AGHT+IHp3UDsNY1OdXjOXmJRnD8oRCpL6CgaUimOK/pVgJHgRaGWijdZG0cBnNzC1r8eEV5WV+PGEA== X-Received: by 2002:a05:6512:68b:b0:545:cd5:84d9 with SMTP id 2adb3069b0e04-54838ee7b55mr6011950e87.12.1740477334328; Tue, 25 Feb 2025 01:55:34 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id 2adb3069b0e04-548514b261esm132867e87.24.2025.02.25.01.55.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 25 Feb 2025 01:55:33 -0800 (PST) From: Linus Walleij Date: Tue, 25 Feb 2025 10:55:17 +0100 Subject: [PATCH v5 30/31] ARM: entry: Block IRQs in early IRQ context MIME-Version: 1.0 Message-Id: <20250225-arm-generic-entry-v5-30-2f02313653e5@linaro.org> References: <20250225-arm-generic-entry-v5-0-2f02313653e5@linaro.org> In-Reply-To: <20250225-arm-generic-entry-v5-0-2f02313653e5@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250225_015536_007833_F91B8BF1 X-CRM114-Status: GOOD ( 19.83 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org When dabt, pabt or und exceptions occur on ARM, ordinary interrupts (IRQs) can still happen. This isn't nice for the kernels context tracker, which expect (when using generic entry at least) that any nested IRQs happens between irqentry_enter() and irqentry_exit(), else it thinks something is fishy. This change blocks interrupts in the pabt, dabt, und and abt exception paths (all of them really) by unconditionally setting PSR_I_BIT in the early exception handler, until after context has been established with irqentry_enter() and before it is exited with irqentry_exit(). Inside the context-tracked exception handler we enable IRQs again, and once we leave it we disable them while exiting the exception. The local_irq_disable() in bad_mode() can be dropped since we are now disabling IRQs in the early assembly exception handler for all exceptions. This seems like not perfect: it seems an interrupt could still occur right before CPSR is set, or right after the userspace registers are restored in ret_from_exception. I would like to know if there is some way to set up these exceptions to inherently block IRQs when handled, until we explicitly allow them between irqentry_enter() and irqentry_exit() or if this is simply the best we can do on ARM for these exceptions to make the context tracker happy. Signed-off-by: Linus Walleij --- arch/arm/kernel/entry-armv.S | 6 +----- arch/arm/kernel/entry.c | 18 ++++++++++++++++++ arch/arm/kernel/traps.c | 1 - 3 files changed, 19 insertions(+), 6 deletions(-) diff --git a/arch/arm/kernel/entry-armv.S b/arch/arm/kernel/entry-armv.S index ae2f952beea7611f0abc7bd299fc944335a21219..3dae35b0bb3f440ecaf157a45687bf4690fb8f88 100644 --- a/arch/arm/kernel/entry-armv.S +++ b/arch/arm/kernel/entry-armv.S @@ -416,11 +416,6 @@ ENDPROC(__irq_usr) __und_usr: usr_entry uaccess=0 - @ IRQs must be enabled before attempting to read the instruction from - @ user space since that could cause a page/translation fault if the - @ page table was modified by another CPU. - enable_irq - tst r5, #PSR_T_BIT @ Thumb mode? mov r1, #2 @ set insn size to 2 for Thumb bne 0f @ handle as Thumb undef exception @@ -847,6 +842,7 @@ vector_\name: @ mrs r0, cpsr eor r0, r0, #(\mode ^ SVC_MODE | PSR_ISETSTATE) + orr r0, r0, #PSR_I_BIT msr spsr_cxsf, r0 @ diff --git a/arch/arm/kernel/entry.c b/arch/arm/kernel/entry.c index 01e4339ccdb4951e04a89fb91ad7c6e1991e09af..3881670e0987ee40be6fff32f412edcf4f3ec80a 100644 --- a/arch/arm/kernel/entry.c +++ b/arch/arm/kernel/entry.c @@ -17,8 +17,18 @@ noinstr asmlinkage void arm_und_handler(struct pt_regs *regs) { irqentry_state_t state = irqentry_enter(regs); + /* + * IRQs must be enabled before attempting to read the instruction from + * user space since that could cause a page/translation fault if the + * page table was modified by another CPU. + */ + + local_irq_enable(); + do_undefinstr(regs); + local_irq_disable(); + irqentry_exit(regs, state); } @@ -27,8 +37,12 @@ noinstr asmlinkage void arm_dabt_handler(unsigned long addr, unsigned int fsr, { irqentry_state_t state = irqentry_enter(regs); + local_irq_enable(); + do_DataAbort(addr, fsr, regs); + local_irq_disable(); + irqentry_exit(regs, state); } @@ -37,8 +51,12 @@ noinstr asmlinkage void arm_pabt_handler(unsigned long addr, unsigned int ifsr, { irqentry_state_t state = irqentry_enter(regs); + local_irq_enable(); + do_PrefetchAbort(addr, ifsr, regs); + local_irq_disable(); + irqentry_exit(regs, state); } diff --git a/arch/arm/kernel/traps.c b/arch/arm/kernel/traps.c index d3a689bd05955f1ae46a6341e456bb097e831311..b2eaab9cc4977c448f77faa4e6bc22a0749d1f9c 100644 --- a/arch/arm/kernel/traps.c +++ b/arch/arm/kernel/traps.c @@ -515,7 +515,6 @@ asmlinkage void bad_mode(struct pt_regs *regs, int reason) pr_crit("Bad mode in %s handler detected\n", handler[reason]); die("Oops - bad mode", regs, 0); - local_irq_disable(); panic("bad mode"); } From patchwork Tue Feb 25 09:55:18 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13989859 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C4127C021B2 for ; Tue, 25 Feb 2025 11:21:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=oIYvJaonUYyfEumf+Z+WmSCNyMMQcgPh7imvn/7wUEM=; b=qtt5aYLG9j21achttyG2D39la2 Kl2ggefQDtrhq381PiXvuyjA1+6FxlKLFo6x2BGNM7bPCiaDVxfs6SHGPPABAauTNfa/jXHYbg1Q7 tByQMgnAbDPykB1iLJaOVUhm9GRjD+hJD/FYDaLnSfNK9uT1ouCNyTSVHT0yN/FSgADDwiPkfCsxr ZPwKKo8TLhUKjh/Bcd3NkwmXHLsIEqYugA0dQiUiwMlCHf2yUnQ5RUV2ZBcwH1G3HxtXV2CLKF+9l Tr67d14sVOFLKeFbrzs3mDGbb9I+Fo55NcN0lTgJJEV0AbIPmXE4LyRE8L5Mj5GrWIZvdAExZnS1k 8lKn3jbg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tmt0I-0000000H0yj-20zP; Tue, 25 Feb 2025 11:21:34 +0000 Received: from mail-lf1-x129.google.com ([2a00:1450:4864:20::129]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tmrf7-0000000Ghqh-0zZ2 for linux-arm-kernel@lists.infradead.org; Tue, 25 Feb 2025 09:55:38 +0000 Received: by mail-lf1-x129.google.com with SMTP id 2adb3069b0e04-543d8badc30so6249369e87.0 for ; Tue, 25 Feb 2025 01:55:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1740477336; x=1741082136; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=oIYvJaonUYyfEumf+Z+WmSCNyMMQcgPh7imvn/7wUEM=; b=yRaw/5ODgUONgUohX4nC6CpT2xxQCmmaEXFgDOUFRh2Mq83KDocB8czGy6iBtvjIqE O5bqn1GCcyr+GrWMo/1Y1TGOdhguX1O6HInCXpSYRvd4XSJLTxDHnxmzw+rpFvUJWl4T 8YDK1WCzaIm+EevNEUiAXDmM93sWY680ry3P5wpSTapxWauFBgFmDT5Pc/eWA0bTXIzt 0K6PrBg5hB+HBYVmDAaaqaV/CSCyzOUrc3xp+LImMvRiJRVNIVgAUVOYFg08R2GmVuxq weC8EVjtbKuWKrO+RjZ98bQm6/tCq2oSkNpy8nWWdBc9jyT4sjick3o7Q6OPwFB7x7yS cv3Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740477336; x=1741082136; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=oIYvJaonUYyfEumf+Z+WmSCNyMMQcgPh7imvn/7wUEM=; b=Gcbmdk0fNHOD109CSWq5PwX2zwrYSeBrdnVfgR6wTUaHeRaLuPLpWv9MkXOptpEK4D TmtpvR/h1YoZOwW84vVE1PodCgb2eWQu+vT21a796FNh7/FUDuVqmi80ZKpTUyN9UiBH zcmvQdPTQvdB1EAXm8J9U2fZnJPMXixZSuI8UNNdRqGB3Sy9QIbjpy9i0SvgwiZmpN4T NOS7Ali2Xjc8TNpQZcEXKx5JO+zZeDxVGfP11AAXiqj1R55Xj1UO6Mok0ykqTNjPEK/6 l/b1vUiz5yfjIFwFsKyGJ/2scuSLPpLGh1PY8uStoEhr0KsD/YwgWl8euUB7eSaGXLUL ypWQ== X-Gm-Message-State: AOJu0YwLg1EO9oTl/WY+ScXXrzS3W6paNz6IYPsp0o0CoN5nxwWZYKlV PDK9zBfu9G2m5xFah498g7MebtxsLjroOGugpDgNeJTwehDy7o8cZses/SaCPzg= X-Gm-Gg: ASbGncuCOVHD+R57QejKC/muhaCoXsJtknXOu99s9huhBfJQ6ERBrn1l0fMRorg7MMF Ep4QQj8gR98Du9A133WyYhPGo6H6TDTSsQfSA2E5QDHJBZY4Wbh97uR/r6SQQKcMsER2HyBR0xI OP3Sm4WelyukqO7wNZrb8WZOE+Q6NXVAnfxNSbwlGNGQ5rgJqtoBJy2mgoKLcQxc7TSrVEa/5Xw kvUOEMlOfU422NuddK9EgA3e1Tp83xR5Unpe4lUToPGbFWlDlA/jaWtOoY94srHKsxPT1KVUIp8 ior96jZZHwEe5GGbrlXl8rj5ObbleGMRJpif X-Google-Smtp-Source: AGHT+IEadLgBcodjV4MeijK1RxbLNXPOedOFvhDLMaZ6EVsgBHRAKm5NLB+ewxhEdUYnEE+8TXegEw== X-Received: by 2002:a05:6512:224b:b0:545:54b:6a05 with SMTP id 2adb3069b0e04-548510eebcdmr998842e87.45.1740477335666; Tue, 25 Feb 2025 01:55:35 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id 2adb3069b0e04-548514b261esm132867e87.24.2025.02.25.01.55.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 25 Feb 2025 01:55:34 -0800 (PST) From: Linus Walleij Date: Tue, 25 Feb 2025 10:55:18 +0100 Subject: [PATCH v5 31/31] ARM: entry: Reimplement local restart in C MIME-Version: 1.0 Message-Id: <20250225-arm-generic-entry-v5-31-2f02313653e5@linaro.org> References: <20250225-arm-generic-entry-v5-0-2f02313653e5@linaro.org> In-Reply-To: <20250225-arm-generic-entry-v5-0-2f02313653e5@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250225_015537_283227_D09F0095 X-CRM114-Status: GOOD ( 25.09 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The former local restart hack to restart syscalls inside the kernel if we get restart signals while processing a system call was deleted when converting the architecture to generic entry. This makes strace tests fail so the hack is necessary. Now, after the conversion to generic entry, restore the order by reimplementing this with two TIF flags that help us to issue system call restarts immediately in the kernel. This is essentially a reimplementation of commit 81783786d5cf "ARM: 7473/1: deal with handlerless restarts without leaving the kernel" from 2012, but in C, on top of generic entry. Link: http://lists.infradead.org/pipermail/linux-arm-kernel/2012-June/104733.html Link: https://lore.kernel.org/all/1340377626-17075-1-git-send-email-will.deacon@arm.com/ Signed-off-by: Linus Walleij --- arch/arm/include/asm/thread_info.h | 4 ++++ arch/arm/kernel/signal.c | 17 +++++++++++++---- arch/arm/kernel/syscall.c | 22 ++++++++++++++++++++++ 3 files changed, 39 insertions(+), 4 deletions(-) diff --git a/arch/arm/include/asm/thread_info.h b/arch/arm/include/asm/thread_info.h index 84e58a9cdab63ad264c2cd2bad64239d1912cbe7..09dcaeef645a4ab45f40a14f8d7b46f408225f81 100644 --- a/arch/arm/include/asm/thread_info.h +++ b/arch/arm/include/asm/thread_info.h @@ -150,6 +150,8 @@ extern int vfp_restore_user_hwstate(struct user_vfp *, #define TIF_USING_IWMMXT 17 #define TIF_MEMDIE 18 /* is terminating due to OOM killer */ #define TIF_RESTORE_SIGMASK 19 +#define TIF_LOCAL_RESTART 20 +#define TIF_LOCAL_RESTART_BLOCK 21 #define _TIF_SIGPENDING (1 << TIF_SIGPENDING) #define _TIF_NEED_RESCHED (1 << TIF_NEED_RESCHED) @@ -157,6 +159,8 @@ extern int vfp_restore_user_hwstate(struct user_vfp *, #define _TIF_UPROBE (1 << TIF_UPROBE) #define _TIF_NOTIFY_SIGNAL (1 << TIF_NOTIFY_SIGNAL) #define _TIF_USING_IWMMXT (1 << TIF_USING_IWMMXT) +#define _TIF_LOCAL_RESTART (1 << TIF_LOCAL_RESTART) +#define _TIF_LOCAL_RESTART_BLOCK (1 << TIF_LOCAL_RESTART_BLOCK) #define _TIF_WORK_MASK (_TIF_NEED_RESCHED | _TIF_SIGPENDING | \ _TIF_NOTIFY_RESUME | _TIF_UPROBE | \ diff --git a/arch/arm/kernel/signal.c b/arch/arm/kernel/signal.c index 35d2bb3cd2b442dac164548037262e065fbfe12a..a4fc6522124fd3ac3df7149ba38cf4b097196e06 100644 --- a/arch/arm/kernel/signal.c +++ b/arch/arm/kernel/signal.c @@ -541,7 +541,8 @@ void arch_do_signal_or_restart(struct pt_regs *regs) unsigned int retval = 0, continue_addr = 0, restart_addr = 0; bool syscall = (syscall_get_nr(current, regs) != -1); struct ksignal ksig; - int restart = 0; + bool restart = false; + bool restart_block = false; /* * If we were from a system call, check for system call restarting... @@ -557,12 +558,12 @@ void arch_do_signal_or_restart(struct pt_regs *regs) */ switch (retval) { case -ERESTART_RESTARTBLOCK: - restart -= 2; + restart_block = true; fallthrough; case -ERESTARTNOHAND: case -ERESTARTSYS: case -ERESTARTNOINTR: - restart++; + restart = true; regs->ARM_r0 = regs->ARM_ORIG_r0; regs->ARM_pc = restart_addr; break; @@ -593,8 +594,16 @@ void arch_do_signal_or_restart(struct pt_regs *regs) } else { /* no handler */ restore_saved_sigmask(); - if (unlikely(restart) && regs->ARM_pc == restart_addr) + if (unlikely(restart) && regs->ARM_pc == restart_addr) { + /* + * These flags will be picked up in the syscall invocation code, + * and a local restart will be issued without exiting the kernel. + */ + set_thread_flag(TIF_LOCAL_RESTART); + if (restart_block) + set_thread_flag(TIF_LOCAL_RESTART_BLOCK); regs->ARM_pc = continue_addr; + } } return; } diff --git a/arch/arm/kernel/syscall.c b/arch/arm/kernel/syscall.c index ed3ab51283c06c1398ece2ad3ee1fae16cd03ee8..20b59f5dfdc8d2e8c168ac04c2244cb6371e5672 100644 --- a/arch/arm/kernel/syscall.c +++ b/arch/arm/kernel/syscall.c @@ -11,6 +11,7 @@ __visible void invoke_syscall(void *table, struct pt_regs *regs, int scno) { int ret; +local_restart: scno = syscall_enter_from_user_mode(regs, scno); /* When tracing syscall -1 means "skip syscall" */ if (scno < 0) { @@ -34,4 +35,25 @@ __visible void invoke_syscall(void *table, struct pt_regs *regs, int scno) syscall_set_return_value(current, regs, 0, ret); syscall_exit_to_user_mode(regs); + + /* + * Handle local restart: this means that when generic entry + * calls arch_do_signal_or_restart() because a signal to + * restart the syscall arrived while processing a system call, + * we set these flags for the thread so that we don't even + * exit the kernel, we just restart right here and clear + * the restart condition. + * + * This is done because of signal race issues on ARM. + */ + if (test_thread_flag(TIF_LOCAL_RESTART)) { + if (test_thread_flag(TIF_LOCAL_RESTART_BLOCK)) { + scno = __NR_restart_syscall - __NR_SYSCALL_BASE; + /* Make this change visible to tracers */ + task_thread_info(current)->abi_syscall = scno; + clear_thread_flag(TIF_LOCAL_RESTART_BLOCK); + } + clear_thread_flag(TIF_LOCAL_RESTART); + goto local_restart; + } }