From patchwork Tue Jan 7 09:41:17 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13928575 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 8ABABE77197 for ; Tue, 7 Jan 2025 09:44:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=3sy/qewi5QTr4eGa8bqumB+Ls2VPcx0iOCdO37ReQs8=; b=bqqdDKS5JGUI2BadX7pLupTEm0 s/h6eRUJfQ7Oc9loWymz2MENIe6040coCdP48r9xm9kGr/GHWULE40ar5BHGrulPqZl/0yYjHqAma bVeZZFl2AbQuNN6QryPi+xktFpjF8FIJVP77ehq1oG1KzNc/IM550XiDSaEg09efQteyimG1MFNmD dgqDzF5vr6Ckk0+xYvBohSQBNs+SGqksCh5IkLGkFSmQ1L261G+X6TiqayRNHl2vI/1E94zriVv2a YtWRPiw6GTRGA22ijUKiFUYStXg1tpO0N5iYc6zWeMHD5+8Mr1WB95DGGn7p1CpMGyK+2Jc8nRlnk nwovaRVQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tV67x-00000004BBg-3EMI; Tue, 07 Jan 2025 09:43:57 +0000 Received: from mail-ej1-x62d.google.com ([2a00:1450:4864:20::62d]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tV65Y-00000004AJl-3cA2 for linux-arm-kernel@lists.infradead.org; Tue, 07 Jan 2025 09:41:32 +0000 Received: by mail-ej1-x62d.google.com with SMTP id a640c23a62f3a-a9e44654ae3so2555684966b.1 for ; Tue, 07 Jan 2025 01:41:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1736242887; x=1736847687; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=3sy/qewi5QTr4eGa8bqumB+Ls2VPcx0iOCdO37ReQs8=; b=CcUjLT7v9/rzOp8AD34nxZY0RRc0NeqrrLWRlfeGW2wcZ7uFG5WQFuDC0iwRLtbAjZ dNgvk29RFy6qFj7oSt8eD3M7EGXTGULp+B7mQy+43am8IDu/7CwGpdciS6sIpv2Am0jm MuwEpuoHFk9GCK4h5aszNJ7hI664Qva2feYn9o/46vJ74PVouXqx+ZoUxcf8gT5BZHva uGf/0nLaKdDTVauGf1ocFitjK8QWCCd7FKVzsIJIpxpDWZ6miaxWPfVe92Ab4VWCiUkK 297eQSiL9Do+PlFMbEGTsdYHJt0uvfPpztuH/nXm9zarmnZjspOgh/LcUCJiPc9JRUx5 gCkg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736242887; x=1736847687; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3sy/qewi5QTr4eGa8bqumB+Ls2VPcx0iOCdO37ReQs8=; b=BXUJy+4wA4UjwAK3YZ2xvp1+evU2SUVxF0p/Jd51mBA1k0nF6aZdY3lNu+4bzsEfmE 6qTl0CfyrpUtDSB9BsfxAtQyL6Hb/dByEL99pfOiFVF5qL/qrWxv9OUJbjlzOy2GETPt C0Poqhm3LRctBZ5dOog+9Mib3q/zpqBhh+NqIen+KQ4FYtCK906i2cJwC0qlOGJFhG0G +k0kNTNLKgqFtwWJeI2A560MFqyHWoi/Zhsk6IVYVAf3OBS6vnuE2hHtPvsVTUnDl9yQ jIM28njCV5xa1HA9kATdfnFphE0lGtcWfDnncaq6IFNm75xCoK9qPZyblRSgczNIp8SX c+fA== X-Gm-Message-State: AOJu0YyQIW5XcPZHEDbvHHgwlFhftfghcgyvwwPbAPsgOU3dtHBOIZMy vgZVKWdyWF6GmphF4S68iBpxf0WZR81Sg4KzEIYggCnVc/6xI2qkHTx9ODtMCLM= X-Gm-Gg: ASbGncuOXpkMAd8/UqFsc2YAi9vRSXdouLrSzy2No9+FFTsMg/5BX7LYrdKNdInGy71 uLkKO/lF1acY4hzh4elP6R0Yziipj/R+DyY9HL4AZaH3SrBOcPR8Q2Zm2j7MTHPBozAOjl9Iyjb bwkTKoFxbF1qPpPy877VdtGjxS1U2c9Ynrjmn4WjGQgHwyB687sItCvX42RA/xH7ndZBwNmHlgo qfButmi85YM2rlPNP/Fut+Hxzog60gq/JfvnYzrCRwF+gayBqfKGLl/CD0LSXdSzmA= X-Google-Smtp-Source: AGHT+IHFAMDNG0ODw2AINJ6wybX6ZQoPMSvIxzZsPDK4VOdkYw3p0VNup/SMUF/w5GkNHETSOyyRrg== X-Received: by 2002:a17:907:94c2:b0:aa6:19c9:ad08 with SMTP id a640c23a62f3a-aac334f31e9mr5172979766b.48.1736242887317; Tue, 07 Jan 2025 01:41:27 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aac0f012229sm2356901166b.133.2025.01.07.01.41.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jan 2025 01:41:26 -0800 (PST) From: Linus Walleij Date: Tue, 07 Jan 2025 10:41:17 +0100 Subject: [PATCH RFC v3 01/30] ARM: Prepare includes for generic entry MIME-Version: 1.0 Message-Id: <20250107-arm-generic-entry-v3-1-4e5f3c15db2d@linaro.org> References: <20250107-arm-generic-entry-v3-0-4e5f3c15db2d@linaro.org> In-Reply-To: <20250107-arm-generic-entry-v3-0-4e5f3c15db2d@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250107_014128_902009_279FE69E X-CRM114-Status: GOOD ( 13.84 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The generic entry code needs a static inline function for regs_irq_disabled() so add this, in further requires the signature of on_thread_stack() to be __always_inline so add this as well. Signed-off-by: Linus Walleij --- arch/arm/include/asm/ptrace.h | 5 +++++ arch/arm/include/asm/stacktrace.h | 2 +- 2 files changed, 6 insertions(+), 1 deletion(-) diff --git a/arch/arm/include/asm/ptrace.h b/arch/arm/include/asm/ptrace.h index 6eb311fb2da06fa393f8be7caec8d997637a88b2..a01c66f0ad907882af2a383e53de8aca68c551e5 100644 --- a/arch/arm/include/asm/ptrace.h +++ b/arch/arm/include/asm/ptrace.h @@ -52,6 +52,11 @@ struct svc_pt_regs { #define fast_interrupts_enabled(regs) \ (!((regs)->ARM_cpsr & PSR_F_BIT)) +static inline int regs_irqs_disabled(struct pt_regs *regs) +{ + return !interrupts_enabled(regs); +} + /* Are the current registers suitable for user mode? * (used to maintain security in signal handlers) */ diff --git a/arch/arm/include/asm/stacktrace.h b/arch/arm/include/asm/stacktrace.h index f80a85b091d6c4ff365e15ae7100af1c3aed597f..815b5f256af164daa053f3bfd6cb2f54042a8d2b 100644 --- a/arch/arm/include/asm/stacktrace.h +++ b/arch/arm/include/asm/stacktrace.h @@ -26,7 +26,7 @@ struct stackframe { #endif }; -static inline bool on_thread_stack(void) +static __always_inline bool on_thread_stack(void) { unsigned long delta = current_stack_pointer ^ (unsigned long)current->stack; From patchwork Tue Jan 7 09:41:18 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13928578 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 03DA7E77197 for ; Tue, 7 Jan 2025 09:45:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=fxoDZkceuldN2v+AcZoQ7wCap4C6/eLU9DTPkxF/gzU=; b=ux2NwymdLIDS69bsDtQQULd+fY 2HZ+MML23fkVG72+8BPi/LTj5/4W9WWkdNgLNItd+iEh4x1f0zxaSRssfdteXJ43xikDBe4vz76lf XPFSfl3RO+HPhOVANJPMY8bzumqOSoys1ElQ2qzHU5hmgXfCjuVYmAS/r43nU2eqmQhzTHiM07uXG Gm1M8c/d/s4cq2KBYtNrh/+vf+ejVISmBKGVJNfEz1uGFETZlxpe9daZaEVlECfUc8c7fnoVByorB gdlZyRmFQNnwOtzF+VBxQ4Q3rLlv8Layn15ZPcP1rGC3GD384hANAYRjw5l/pKpKubWLg0bPz54sO 7uYp4LyQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tV697-00000004Bag-1eOB; Tue, 07 Jan 2025 09:45:09 +0000 Received: from mail-ej1-x633.google.com ([2a00:1450:4864:20::633]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tV65a-00000004AKN-1Noo for linux-arm-kernel@lists.infradead.org; Tue, 07 Jan 2025 09:41:35 +0000 Received: by mail-ej1-x633.google.com with SMTP id a640c23a62f3a-aaf8f0ea963so3097266b.3 for ; Tue, 07 Jan 2025 01:41:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1736242888; x=1736847688; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=fxoDZkceuldN2v+AcZoQ7wCap4C6/eLU9DTPkxF/gzU=; b=y17qdGivhf9QKhcCiIuC31cm9Pk+MLDJA/lRZM/hibBRWXANKSRUqm4JpcRtYD5MDf F/dq8jkBbZoVGhLAwhys/WjDA/vj81kS51Ho4Tn6OLj+V+i5H14yE100KAft2ppvXva6 RVItp+wZ/0z8TCSlYHELCdf8MRYaFf+BJbontoKLczha2bvaAcfXhz4NJ64tJZpPeCXH acjbX71dpnvRXmfvnFdM06io9h0qw0c4k9eklmtX+vLkHZptFyc9DfIv13XmytkI5lKY mtOKdyDwR8pMe/ZIkBf5UaoIyp0uIQPqHMyghFF/l5czIXKe07V12en3nk8F5x+XYMha p50g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736242888; x=1736847688; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=fxoDZkceuldN2v+AcZoQ7wCap4C6/eLU9DTPkxF/gzU=; b=LP5VXR1pWZ4ROfjwipxONkIhUv8KSxvYSbs12FwdAzCfRt47HssCk5UB62LO/1H9zD XwU+Em7U+D5lWAdF2Yrjo5P6lntxfjDdZzLc9kHo0/BgA+keA27eqsVAAG2xqu5lNcq1 w0huS9vBQQXFiW4lwBTuQ4RcOL5f4T9U3QImujgZ3j9nZfKgbBMiPi+Wn2BLIw00dzQK MpvpldlyC68FtcZx6djpiwGHMk7Mp9hfIucO8wj5rIaQd1kVp9IducjGpjkjVdDYduPl MN4BFDd7GgSNh/WRQWvFe+T4cCFwzHvhsArOnlTkFG7O9UtS+My6Fpn0XsPpnngd+eu8 B0Hw== X-Gm-Message-State: AOJu0Yzs0P5cF7U60fEMkSA2kw8JhCjVNj1juFQmjMbypigpIZ6AsjfK U9oE+ljlsTrDuRNU2+Tdlbn/fnzFCH1KD8bW/YY7ZBApXQ7/I6U0+U4HXMU3k3waJfuhVLTIMFE msRU= X-Gm-Gg: ASbGncuP/h5QEU1yri3qYX4BQw7ZKaBcJDJYgua0aKYhOauVJTRnRGhVEYYs7cR/QP4 Pmtayr41GcGaMBreF2+lK17MJxkgkPW0QPwrL81D8Muo7Gj/KNr1z2ZgaP9i1gNFHyFWnjVeFMQ doBHubve6Klno3qyBeZ1DyutwAKpiv47atBDcQjKTRxCXBHzmyGhnHeBMevs0tPJHoxRo32MLFp HEKunzMaX47LL7FtY1KTckRYXeQVoxRWYxo3ZkVlOitN9T7ectYPtxxJl3k5pDSXj4= X-Google-Smtp-Source: AGHT+IEM28Aop4iPJtDsWQ84LuFu7WxnHVAzJRG9etmyple0xNlzoIv/hKHIlrWa557ZtbVDH/zCCw== X-Received: by 2002:a17:907:7f8e:b0:aa6:9198:75a2 with SMTP id a640c23a62f3a-aac334e51afmr5458619866b.44.1736242888513; Tue, 07 Jan 2025 01:41:28 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aac0f012229sm2356901166b.133.2025.01.07.01.41.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jan 2025 01:41:28 -0800 (PST) From: Linus Walleij Date: Tue, 07 Jan 2025 10:41:18 +0100 Subject: [PATCH RFC v3 02/30] ARM: ptrace: Split report_syscall() MIME-Version: 1.0 Message-Id: <20250107-arm-generic-entry-v3-2-4e5f3c15db2d@linaro.org> References: <20250107-arm-generic-entry-v3-0-4e5f3c15db2d@linaro.org> In-Reply-To: <20250107-arm-generic-entry-v3-0-4e5f3c15db2d@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250107_014130_374781_63D0AF16 X-CRM114-Status: GOOD ( 14.19 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The generic entry code requires that report_syscall() be provided in two parts: report_syscall_enter() and report_syscall_exit() so split the combined function in two. Signed-off-by: Linus Walleij --- arch/arm/kernel/ptrace.c | 30 ++++++++++++++++++++++-------- 1 file changed, 22 insertions(+), 8 deletions(-) diff --git a/arch/arm/kernel/ptrace.c b/arch/arm/kernel/ptrace.c index c421a899fc84c476a672cbfe1c8ece8355d8512d..07b0daf47441f1f76a8af416acc74fa5ed770403 100644 --- a/arch/arm/kernel/ptrace.c +++ b/arch/arm/kernel/ptrace.c @@ -827,31 +827,45 @@ enum ptrace_syscall_dir { PTRACE_SYSCALL_EXIT, }; -static void report_syscall(struct pt_regs *regs, enum ptrace_syscall_dir dir) +static void report_syscall_enter(struct pt_regs *regs) { unsigned long ip; /* * IP is used to denote syscall entry/exit: - * IP = 0 -> entry, =1 -> exit + * IP = 0 -> entry */ ip = regs->ARM_ip; - regs->ARM_ip = dir; + regs->ARM_ip = PTRACE_SYSCALL_ENTER; - if (dir == PTRACE_SYSCALL_EXIT) - ptrace_report_syscall_exit(regs, 0); - else if (ptrace_report_syscall_entry(regs)) + if (ptrace_report_syscall_entry(regs)) current_thread_info()->abi_syscall = -1; regs->ARM_ip = ip; } +static void report_syscall_exit(struct pt_regs *regs) +{ + unsigned long ip; + + /* + * IP is used to denote syscall entry/exit: + * IP = 1 -> exit + */ + ip = regs->ARM_ip; + regs->ARM_ip = PTRACE_SYSCALL_EXIT; + + ptrace_report_syscall_exit(regs, 0); + + regs->ARM_ip = ip; +} + asmlinkage int syscall_trace_enter(struct pt_regs *regs) { int scno; if (test_thread_flag(TIF_SYSCALL_TRACE)) - report_syscall(regs, PTRACE_SYSCALL_ENTER); + report_syscall_enter(regs); /* Do seccomp after ptrace; syscall may have changed. */ #ifdef CONFIG_HAVE_ARCH_SECCOMP_FILTER @@ -892,5 +906,5 @@ asmlinkage void syscall_trace_exit(struct pt_regs *regs) trace_sys_exit(regs, regs_return_value(regs)); if (test_thread_flag(TIF_SYSCALL_TRACE)) - report_syscall(regs, PTRACE_SYSCALL_EXIT); + report_syscall_exit(regs); } From patchwork Tue Jan 7 09:41:19 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13928579 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 900B7E77197 for ; Tue, 7 Jan 2025 09:46:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=hfaMZlHPI0COD+Vx8O9p1xdwXj673vcQcRJJP/RIV4o=; b=tP0ufoBhTOr+e7jkX09POVe6N2 kMHBy9HnEByPm0PMe/Jqs2xswUmJ/uV+Gt5MsmuaYDtJtS+S90IIKms5HWbzpI4NkwM6kGOTEY91o mk0LlcQCojpDDjQWo3cKNOa7jQxMiBBz2Lk6KYwB8JGBgPAXFhMTQnxZ1HOJkZ8lallXYVQGimaR7 rI6YqjTTZF80OUkvsaM0eGUnlEdpwTwKexuia0+Q5Sez39ZDcD5sa5Wt3TvYFOX5VvUu2Ah7Ytf1z f0Rjg5xx9jngpBjRtqJQLGWtt6tVERdhDcvzt/uO8hy7/wNds0d+tQALRxCTH7QEGTd+NZCN0eMGA ABrf8tcw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tV6AI-00000004CGb-0OGX; Tue, 07 Jan 2025 09:46:22 +0000 Received: from mail-ed1-x52a.google.com ([2a00:1450:4864:20::52a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tV65b-00000004AKi-2Y3Z for linux-arm-kernel@lists.infradead.org; Tue, 07 Jan 2025 09:41:35 +0000 Received: by mail-ed1-x52a.google.com with SMTP id 4fb4d7f45d1cf-5d3e6f6cf69so26640926a12.1 for ; Tue, 07 Jan 2025 01:41:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1736242890; x=1736847690; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=hfaMZlHPI0COD+Vx8O9p1xdwXj673vcQcRJJP/RIV4o=; b=jzt5Bt82ly0FVUt5RBFlfd/uxnWKGbxD/qZX83vjvNNmAOSfjXNexUEbnjODY/cqQA 8xJY/7XV9uCUgrKeY9QmvMemRMKwMMSTB2mwRMC6bj/dOEcZ74N3Jf8UfRsNG6e7dzqU wxceRNwtVTzJTz/RCv65zukwZaKWX+0bcZSVkn/XiTu9+vrpIdEr+TjqTmbiSdjCi3dp MBPTdKbByfETrqGzlYhhHbB1V3PcPG5z+VfRYbHj+lrggzVdTlLrVJH6uY9nJYnZPPWa 2M2SGl+7S91XJHUDrNEQZzPFReDobqq1LtV5I4w8U95KHlALwUoX83zq1t4HiY/jsdmG 2IDg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736242890; x=1736847690; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=hfaMZlHPI0COD+Vx8O9p1xdwXj673vcQcRJJP/RIV4o=; b=fR22ik/JcI0cYm69u5a+kq8O5jlrXSTDR3IhkpYLQTHMNTIPscd+5Er60hGmUkAh+n jfVvaqcxIcAa0R4Su2sUdlimH5hJ3XGPoRma4z29glnZvaDAJ2kc7N/wKdiKH5POuj9D atbyfssd8ABEDMKtHsiLlEbWisWm2C37ocHyIAYw4Ku7ejd6YGj5hSD7zIoSAh7jMoF1 C98Cry0mGmN/Dit4lDz0I3tRrt0AYO/Si3jxU0rt7AkjjcKzPsTmz+r69wWHWITpIPuG zcmRrWIZLzKZw7BrHiVvV610zGCd7yTJVIrSQSUo55yhOksCB5d3UmRIhAN22O9fpK8X KsYg== X-Gm-Message-State: AOJu0Yztug4mWf8mA9a41mlckgyvZej6g/K3H54lVafX0RnxYcA0/Znw uSFMlO++yyjBf5Tm4NNTZHjOguYyr9JTxrtG5Yx8HELIaRxlqZ+fu1WHDVYwZyo= X-Gm-Gg: ASbGncturYtZZ61MY7Kx6/+YMsIgEVS+/kgI8nbigjcDgbolYGRaaydp7snU/2T+Gh6 FQ9LRs0sum2Y6AaXDQUeJ7fhLwGoHh/hxUVP4oo8LqidthGL8v+P16dzXMTDHQ2jnRhTQJsl2Vr LsjsOyxnPMI4HY6tq6q9KdJY1zU8uMIwBPlVlDyqen8nRCjAw2dhQJBjjaXiqbAtu+cdtuTAT7u wU83p1LL/bqwnNJYpKajl9MIamOc6n/Zq9BuTnM6fgwskmBOs7knbSobxJvPTfao+4= X-Google-Smtp-Source: AGHT+IGOLuMW5s2DxWSu/ld9IZ3q3TUEkPd6kSV+QxLv7icvBj7L7t/dnghaUDBa0m/DN4kHcsfcyA== X-Received: by 2002:a17:907:6d0c:b0:aac:212d:a883 with SMTP id a640c23a62f3a-aac34451c4fmr4709173966b.45.1736242890050; Tue, 07 Jan 2025 01:41:30 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aac0f012229sm2356901166b.133.2025.01.07.01.41.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jan 2025 01:41:29 -0800 (PST) From: Linus Walleij Date: Tue, 07 Jan 2025 10:41:19 +0100 Subject: [PATCH RFC v3 03/30] ARM: entry: Skip ret_slow_syscall label MIME-Version: 1.0 Message-Id: <20250107-arm-generic-entry-v3-3-4e5f3c15db2d@linaro.org> References: <20250107-arm-generic-entry-v3-0-4e5f3c15db2d@linaro.org> In-Reply-To: <20250107-arm-generic-entry-v3-0-4e5f3c15db2d@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250107_014131_648752_55914045 X-CRM114-Status: GOOD ( 11.04 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Don't introduce this extra label anymore now that we have a clearly defined entry point for ret_to_user. Signed-off-by: Linus Walleij --- arch/arm/kernel/entry-common.S | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/arch/arm/kernel/entry-common.S b/arch/arm/kernel/entry-common.S index f379c852dcb72fa5e960bba5621cc2152f0df0d3..2f93c2f6d5b8fdf6aaf3dda0192bde144fcf238d 100644 --- a/arch/arm/kernel/entry-common.S +++ b/arch/arm/kernel/entry-common.S @@ -102,7 +102,6 @@ ENDPROC(ret_fast_syscall) * do_work_pending() will update this state if necessary. */ ENTRY(ret_to_user) -ret_slow_syscall: #if IS_ENABLED(CONFIG_DEBUG_RSEQ) /* do_rseq_syscall needs interrupts enabled. */ enable_irq_notrace @ enable interrupts @@ -136,7 +135,7 @@ ENTRY(ret_from_fork) badrne lr, 1f retne r5 1: get_thread_info tsk - b ret_slow_syscall + b ret_to_user ENDPROC(ret_from_fork) /*============================================================================= @@ -310,13 +309,13 @@ __sys_trace_return_nosave: enable_irq_notrace mov r0, sp bl syscall_trace_exit - b ret_slow_syscall + b ret_to_user __sys_trace_return: str r0, [sp, #S_R0 + S_OFF]! @ save returned r0 mov r0, sp bl syscall_trace_exit - b ret_slow_syscall + b ret_to_user .macro syscall_table_start, sym .equ __sys_nr, 0 From patchwork Tue Jan 7 09:41:20 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13928580 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id CBE33E77197 for ; Tue, 7 Jan 2025 09:47:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=XcwyQnV8bm55i5QCWMqzeBKQ01GSNdA6efjZ9lsGDLQ=; b=jYmZpCYwD7vt0U73Wlx7yz5GY+ QrHoJqQ3SA6RJaC3T/d0arcOSpPFKo32jgC1zN/fuVW/VGozQz0z8QGpEiq2qCDLYG68U6gUaKL8g +sqM4gA1b6JGatdCycr1611/UJTo7leOMLOyVxUjTZMZyyA9Onede3WlEPoW2OoKjCi9cJT7sdgoG +TmlVLM67MDa62o8MQz6M/wjPQRpfAopIolvzScGGgtx/CCdzck5U0knaCoDPYC6941klNYyBk4Zz W580svNPqzBBiunErSBM8hmsLtq27/QIOGrGCJVt4MpLnmVbrSzGCsKaQVyMuRWJYImi4Hu4dII9Y +QnPkKow==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tV6BR-00000004Cbx-35H7; Tue, 07 Jan 2025 09:47:33 +0000 Received: from mail-ej1-x62d.google.com ([2a00:1450:4864:20::62d]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tV65c-00000004ALF-34B2 for linux-arm-kernel@lists.infradead.org; Tue, 07 Jan 2025 09:41:36 +0000 Received: by mail-ej1-x62d.google.com with SMTP id a640c23a62f3a-aabfb33aff8so2739304066b.0 for ; Tue, 07 Jan 2025 01:41:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1736242891; x=1736847691; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=XcwyQnV8bm55i5QCWMqzeBKQ01GSNdA6efjZ9lsGDLQ=; b=KlXqTNPcZf8M+LF8rmVJIio3hbIhOklxNd6tuiU19zKxhTz0pX3v3lXo0Khw+PeHFq FkD90a8n6iT/8SxcV1pWpOpRG+529fssA+pAzEJokjb9XjeBU9V8Xc+v6ZniNUlyyvQL CX5caENtF+9xTRRnikaTGBIW0iI3PSSIV/7Vw1f5FaR0ih6nEZmnwMVE53xemxMA3PXE LvJSpdDMFenQshIdGVJ5N1jiqj0H9YDMka7Je9WO/JJ9okb7lCVMqYXp2RvqN5mOtXxw gvvE+VMLhSRYaTV4G+6e6kXetcnKNa1o9exoRbDJiB+2po0NPR8ptQvPWwPLdHW1b4mf f5GQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736242891; x=1736847691; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=XcwyQnV8bm55i5QCWMqzeBKQ01GSNdA6efjZ9lsGDLQ=; b=Sfxv19qacoOrljyRDeU3T+mnN1lrnVXihb8B7UzzmSpEPR0igiqs6Vzjquw99ed3Zw yBsvBlrAzZTb8vmnIJ/u8DM+ez8RnQJopQd7qPOCHcH6ZEB1yXOFUd9GIUkwcer4Eb+d WhleVkxgJMY2OZJwnkz02MuZPjAoTiOa71ZtjtEmNeinbphbxBJXmC/91OCRtfy1AmRm Rzq3s+ulaVqTjGH6E4UPenqBIJqpzqGEMaHu+kasV/HEWjJasrKBULi5OeKe4cHlxhCq SUSabRtJwwPZV/8Un0+kNMZ2UTYxhRm+MpeL60Mtv3ybEOduldJXq4T75QhZQPwzrHCb 0YTQ== X-Gm-Message-State: AOJu0YxiuiUsggU6zIyjmAPbt/Eg4/bW9Awg/Sco3rI3WL38sCafVCPI 0gb0ZKlj2Y5ET8WYjv6lvsopkqi0oGi6GGUfpRVWcR46vENYEL+ArKHtD3F7BWQ= X-Gm-Gg: ASbGncuouj0089CAkZAEAeXLFDGyo2CBEumFPN5TyIwMCrwVs1AR/UoGxuSCqpSIow3 wlvErmCS055YFxiLFlQLW/QGfqRyzcQLJ5/XWHzg934CBRdyCYbee0JiqRCagL7He+QWL2kE5cr pc8m1SME92T2ja3bn4ykOrlNk5LEMfMOY0E1s7BxRxHWKpouTqLJ94uwDEdIW99NwEneSf5cQep 5+n3qqaZLHPHWqgfyzSfa9oy/4G5B3b+xbVtf0CURJlWMsRTMhjIoluCvwKCsq+Yow= X-Google-Smtp-Source: AGHT+IEjUOoCAFXZRMy0I9gsQuvhr/I63Fh4EkcJ8plhvbLKJ7YnsQGNvkDK0foSGl9cEXw6e8AViQ== X-Received: by 2002:a17:907:3d8f:b0:aae:83c7:6ce7 with SMTP id a640c23a62f3a-ab28d07ba93mr230343166b.0.1736242891330; Tue, 07 Jan 2025 01:41:31 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aac0f012229sm2356901166b.133.2025.01.07.01.41.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jan 2025 01:41:30 -0800 (PST) From: Linus Walleij Date: Tue, 07 Jan 2025 10:41:20 +0100 Subject: [PATCH RFC v3 04/30] ARM: process: Rewrite ret_from_fork i C MIME-Version: 1.0 Message-Id: <20250107-arm-generic-entry-v3-4-4e5f3c15db2d@linaro.org> References: <20250107-arm-generic-entry-v3-0-4e5f3c15db2d@linaro.org> In-Reply-To: <20250107-arm-generic-entry-v3-0-4e5f3c15db2d@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250107_014132_786026_1EA1CD39 X-CRM114-Status: GOOD ( 18.96 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Move the direct assembly call to schedule_tail() and the call starting a kernel thread into a C version of ret_from_fork() in process.c and rename the old ret_from_fork to ret_from_fork_asm following the pattern in x86. Leave a comment on what we will do when we get to switching to generic entry with this as a base. Signed-off-by: Linus Walleij --- arch/arm/include/asm/switch_to.h | 4 ++++ arch/arm/kernel/entry-common.S | 22 +++++++++++++++------- arch/arm/kernel/process.c | 24 ++++++++++++++++++++++-- 3 files changed, 41 insertions(+), 9 deletions(-) diff --git a/arch/arm/include/asm/switch_to.h b/arch/arm/include/asm/switch_to.h index 9372348516ce9b80fa713966943d0bc622e86066..18d084aff5a4faa7cc5e40d48fbc4aa97a89b0c6 100644 --- a/arch/arm/include/asm/switch_to.h +++ b/arch/arm/include/asm/switch_to.h @@ -17,6 +17,10 @@ #define __complete_pending_tlbi() #endif +asmlinkage void ret_from_fork_asm(void); +__visible void ret_from_fork(struct task_struct *prev, struct pt_regs *regs, + int (*fn)(void *), void *fn_arg); + /* * switch_to(prev, next) should switch from task `prev' to `next' * `prev' will never be the same as `next'. schedule() itself diff --git a/arch/arm/kernel/entry-common.S b/arch/arm/kernel/entry-common.S index 2f93c2f6d5b8fdf6aaf3dda0192bde144fcf238d..df6961a1006b7a0f3522728812bf52ccb849e511 100644 --- a/arch/arm/kernel/entry-common.S +++ b/arch/arm/kernel/entry-common.S @@ -5,6 +5,7 @@ * Copyright (C) 2000 Russell King */ +#include #include #include #include @@ -128,15 +129,22 @@ ENDPROC(ret_to_user) /* * This is how we return from a fork. */ -ENTRY(ret_from_fork) - bl schedule_tail - cmp r5, #0 - movne r0, r4 - badrne lr, 1f - retne r5 +SYM_TYPED_FUNC_START(ret_from_fork_asm) + /* + * ret_from_fork() expects: + * r0: prev + * r1: regs + * r2: fn() for kernel process or NULL + * r3: fn_arg for kernel process or N/A + */ + mov r1, sp + mov r2, r5 + mov r3, r4 + bl ret_from_fork + mov r0, #0 1: get_thread_info tsk b ret_to_user -ENDPROC(ret_from_fork) +SYM_FUNC_END(ret_from_fork_asm) /*============================================================================= * SWI handler diff --git a/arch/arm/kernel/process.c b/arch/arm/kernel/process.c index e16ed102960cb01d625e2ccc484f238b824b03b3..81c1c3f988344185917cf4e53d0d8ee47ae912f4 100644 --- a/arch/arm/kernel/process.c +++ b/arch/arm/kernel/process.c @@ -29,6 +29,7 @@ #include #include #include +#include #include #include #include @@ -230,7 +231,26 @@ void flush_thread(void) thread_notify(THREAD_NOTIFY_FLUSH, thread); } -asmlinkage void ret_from_fork(void) __asm__("ret_from_fork"); +/* The assembly stub sets up the parameters and calls ret_from_fork() */ +asmlinkage void ret_from_fork_asm(void) __asm__("ret_from_fork_asm"); + +__visible void ret_from_fork(struct task_struct *prev, struct pt_regs *regs, + int (*fn)(void *), void *fn_arg) +{ + schedule_tail(prev); + + /* Is this a kernel thread? */ + if (unlikely(fn)) { + fn(fn_arg); + /* + * A kernel thread is allowed to return here after successfully + * calling kernel_execve(). Exit to userspace to complete the + * execve() syscall. + */ + } + + /* syscall_exit_to_user_mode(regs); here once we switch to generic entry */ +} int copy_thread(struct task_struct *p, const struct kernel_clone_args *args) { @@ -263,7 +283,7 @@ int copy_thread(struct task_struct *p, const struct kernel_clone_args *args) thread->cpu_context.r5 = (unsigned long)args->fn; childregs->ARM_cpsr = SVC_MODE; } - thread->cpu_context.pc = (unsigned long)ret_from_fork; + thread->cpu_context.pc = (unsigned long)ret_from_fork_asm; thread->cpu_context.sp = (unsigned long)childregs; clear_ptrace_hw_breakpoint(p); From patchwork Tue Jan 7 09:41:21 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13928581 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id DAC7EE77197 for ; Tue, 7 Jan 2025 09:49:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=c0xk90iCJuCR8FFWwSxF5kQuJOeLnT5yLRORiP5UC0w=; b=TTpqSqQKl8/6eQDCtl1NCOlF44 1cG6iVvTMRPfzqPhPHt7uGo+rqOcl86l+QHx5INReGRz64opWcUo3kLtBJ83ew+4exnwkEVXAThUm 1p5MWT6Hfru+OvhScU+LDL3uHEpdLgCLrQo+nyVKELCB9YMNqycVn602fO5GlKOenwhPWY2pKuxpY VYsPHqgBj9VnRo0teDrWF0C3NWNf33GiAPk9txlgLPwOE+2pTf1uB4+lc8GZEFEPl/UlLiYLjIt2S LORUXbNO/ySq/Jz8ID2G5aaUwKej9HeE2/glyioR1mgFbMeuOdj/KeigVJTbZhHcf5jsE6u2PvOFg khF3fyfA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tV6Cb-00000004Cvl-1SdY; Tue, 07 Jan 2025 09:48:45 +0000 Received: from mail-ej1-x62a.google.com ([2a00:1450:4864:20::62a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tV65e-00000004ALn-1AKY for linux-arm-kernel@lists.infradead.org; Tue, 07 Jan 2025 09:41:38 +0000 Received: by mail-ej1-x62a.google.com with SMTP id a640c23a62f3a-aaf900cc7fbso866224366b.3 for ; Tue, 07 Jan 2025 01:41:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1736242892; x=1736847692; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=c0xk90iCJuCR8FFWwSxF5kQuJOeLnT5yLRORiP5UC0w=; b=WKC85lPD/3ATr5kL/RU/spOv39uLjfrthSgEjWVSqpCCPj6oyOj9PSd1OuKcbAoslf sZ8OIVKP250fWjXUyLlyRgAE9szKjdtz4dvfBWxCcb0FWeTRfkb83zyRtQ/cEKrDiJgC SjPrhGzS07bunu3cCInPyY1v1nAxPuo+1qm4b405pQjSsDcMUVrj4JomCxviZJs2bt3D YRM25JeXx/oAACTIhVXeTi6Kucmxt3AeNIZAVELLHsoRz8Bh2js3scOUZDCpI2yU8ZZs wPAGD6H4IvuDd7yWHzsoYQlCM4gz1KJaQREojdiQCMYeHDRbiWU6rZA3bZiVQO1dh/in pQnQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736242892; x=1736847692; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=c0xk90iCJuCR8FFWwSxF5kQuJOeLnT5yLRORiP5UC0w=; b=KG2rKG+V/A0fdJBg6esRdpUfuHtVEt+7Xmn5vK66V+meP2apN40wbwZiyuYaBwGOmG E0hwoxjWnfuwqzs/9t9w7QbuddHF5egkq3FW4jhQae22YWPC7dbM9Cnm8OQ0pIlN+jyM 2ZJ3XZNu5r/t61tpYjm1hlsG8vRSu0iZwbNweFqOcqF+qwQJbXn4UCGHOnHsUyLNrvng 4/uGXqNbybwHyoLkyRM7gyHATQH8hddBrWh+9fFCK0vjKxcgzoJVbT1lP9F5gJzagpZC aaXutCROPlDKe6wQtvxZ40iMOcDhNs+HycH6yVmzOa5NPS0Pw/fYFymMQTe9ZWF4MsuK Wwig== X-Gm-Message-State: AOJu0YxsmhFLbbU4+RqWwyHvhHOd1q4YnRz91DftGo3MtrBAb72rAP1t EEJypmGoairmGtzfX007HaTR392IgHszBMWgcDGpUT1CpX0TMprInw85taQ5J7xyBt08vKzR+7S rT4c= X-Gm-Gg: ASbGncvbojIMcgJ6jt3LLdGirL4kVjq17rK1PKnkNIa46ReDf8p9PB7EQoZAT08HXKd pcK/5iJI4bMyC5ARqwAxLw+/W9AmX6srTxOjZxZf/OVjTQ2e2M7sP+BOXBKH850klGuDNX5xxsv ZcMibk57x1+6wxq3dwEyR9XJeAyH2hI6X+1XikPnzcEfqXrqFZ8B43Lp6gyyLHu1raty+rkNTy6 gVaeCw+KoZjJ1f+o5StFT0a3lLMVhgr9BgVBfp3Yi97mSEqy4HpULyPnGAuegRzTyo= X-Google-Smtp-Source: AGHT+IH7GnKHose+SCGU4IDmzQsxlhuuWzS7qxw8Naw7AVRLHNfhriDPca5ZUr9UKS5W4G3xsHXuZw== X-Received: by 2002:a17:907:c247:b0:aae:85ad:4997 with SMTP id a640c23a62f3a-aae85ad4b0amr5075746566b.40.1736242892461; Tue, 07 Jan 2025 01:41:32 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aac0f012229sm2356901166b.133.2025.01.07.01.41.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jan 2025 01:41:32 -0800 (PST) From: Linus Walleij Date: Tue, 07 Jan 2025 10:41:21 +0100 Subject: [PATCH RFC v3 05/30] ARM: process: Remove local restart MIME-Version: 1.0 Message-Id: <20250107-arm-generic-entry-v3-5-4e5f3c15db2d@linaro.org> References: <20250107-arm-generic-entry-v3-0-4e5f3c15db2d@linaro.org> In-Reply-To: <20250107-arm-generic-entry-v3-0-4e5f3c15db2d@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250107_014134_323073_7A61C580 X-CRM114-Status: GOOD ( 22.55 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The ARM kernel contains a quirk to handle syscall restarts inside the kernel without exiting to userspace. The generic entry cannot handle this. Rename do_signal() to arch_do_signal_or_restart() to fit with the upcoming generic entry conversion. This is essentially a revert of commit 81783786d5cf "ARM: 7473/1: deal with handlerless restarts without leaving the kernel" from 2012. Other solutions may be possible, such as checking the PC after do_work_pending and assume it is a local restart if that address is inside the kernel, or modifying the generic entry code to track local restarts like ARM does and pass that information back. Signed-off-by: Linus Walleij --- arch/arm/include/asm/signal.h | 3 +-- arch/arm/kernel/entry-common.S | 9 +-------- arch/arm/kernel/signal.c | 28 +++++++++------------------- 3 files changed, 11 insertions(+), 29 deletions(-) diff --git a/arch/arm/include/asm/signal.h b/arch/arm/include/asm/signal.h index 8b84092d151800cb4076672d1c31f0a83777d0d8..7acccc96840c8a17744cc3f2894f19d142aba4fd 100644 --- a/arch/arm/include/asm/signal.h +++ b/arch/arm/include/asm/signal.h @@ -24,7 +24,6 @@ typedef struct { #include void do_rseq_syscall(struct pt_regs *regs); -int do_work_pending(struct pt_regs *regs, unsigned int thread_flags, - int syscall); +void do_work_pending(struct pt_regs *regs, unsigned int thread_flags); #endif diff --git a/arch/arm/kernel/entry-common.S b/arch/arm/kernel/entry-common.S index df6961a1006b7a0f3522728812bf52ccb849e511..da5c2d4b62e56cce644b15e329ab4149d69c5b79 100644 --- a/arch/arm/kernel/entry-common.S +++ b/arch/arm/kernel/entry-common.S @@ -86,14 +86,8 @@ ENDPROC(ret_fast_syscall) bne __sys_trace_return_nosave slow_work_pending: mov r0, sp @ 'regs' - mov r2, why @ 'syscall' bl do_work_pending - cmp r0, #0 - beq no_work_pending - movlt scno, #(__NR_restart_syscall - __NR_SYSCALL_BASE) - str scno, [tsk, #TI_ABI_SYSCALL] @ make sure tracers see update - ldmia sp, {r0 - r6} @ have to reload r0 - r6 - b local_restart @ ... and off we go + b no_work_pending ENDPROC(ret_fast_syscall) /* @@ -266,7 +260,6 @@ ENTRY(vector_swi) */ TRACE( ldmia sp, {r0 - r3} ) -local_restart: ldr r10, [tsk, #TI_FLAGS] @ check for syscall tracing stmdb sp!, {r4, r5} @ push fifth and sixth args diff --git a/arch/arm/kernel/signal.c b/arch/arm/kernel/signal.c index 79a6730fa0eb7d6581343bdbbeb6834dcaabb9f8..7b1a16e86b236575efdc29cda9b751e8e2a3f64e 100644 --- a/arch/arm/kernel/signal.c +++ b/arch/arm/kernel/signal.c @@ -18,6 +18,7 @@ #include #include #include +#include #include #include "signal.h" @@ -534,9 +535,10 @@ static void handle_signal(struct ksignal *ksig, struct pt_regs *regs) * the kernel can handle, and then we build all the user-level signal handling * stack-frames in one go after that. */ -static int do_signal(struct pt_regs *regs, int syscall) +static void arch_do_signal_or_restart(struct pt_regs *regs) { unsigned int retval = 0, continue_addr = 0, restart_addr = 0; + bool syscall = (syscall_get_nr(current, regs) != -1); struct ksignal ksig; int restart = 0; @@ -590,16 +592,14 @@ static int do_signal(struct pt_regs *regs, int syscall) } else { /* no handler */ restore_saved_sigmask(); - if (unlikely(restart) && regs->ARM_pc == restart_addr) { + if (unlikely(restart) && regs->ARM_pc == restart_addr) regs->ARM_pc = continue_addr; - return restart; - } } - return 0; + return; } -asmlinkage int -do_work_pending(struct pt_regs *regs, unsigned int thread_flags, int syscall) +asmlinkage void +do_work_pending(struct pt_regs *regs, unsigned int thread_flags) { /* * The assembly code enters us with IRQs off, but it hasn't @@ -612,19 +612,10 @@ do_work_pending(struct pt_regs *regs, unsigned int thread_flags, int syscall) schedule(); } else { if (unlikely(!user_mode(regs))) - return 0; + return; local_irq_enable(); if (thread_flags & (_TIF_SIGPENDING | _TIF_NOTIFY_SIGNAL)) { - int restart = do_signal(regs, syscall); - if (unlikely(restart)) { - /* - * Restart without handlers. - * Deal with it without leaving - * the kernel space. - */ - return restart; - } - syscall = 0; + arch_do_signal_or_restart(regs); } else if (thread_flags & _TIF_UPROBE) { uprobe_notify_resume(regs); } else { @@ -634,7 +625,6 @@ do_work_pending(struct pt_regs *regs, unsigned int thread_flags, int syscall) local_irq_disable(); thread_flags = read_thread_flags(); } while (thread_flags & _TIF_WORK_MASK); - return 0; } struct page *get_signal_page(void) From patchwork Tue Jan 7 09:41:22 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13928588 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 2B084E77198 for ; Tue, 7 Jan 2025 09:50:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=Mrz9pc85brY24UNXnbG/br6V/oQ7kbdLPycDpNkgqOA=; b=xzJQUTPIWupKQDX4GProgLGKNd 4Fhn8LIrHBpgLJh8kNpTsNg3EfLhOBZzuDpEcIwY48O0qnGX4n+iaMpeF5KVakDK2lsPw9zX3oQyD /H7E3Z3T4UhR2cPCFKLP9pjZajPNOTtpep7/e5UZGp4jVTZTJeqo4AxDiWxO3vFsyP3VpIBp51z4M MnZ8o/JyoH6iuzVV+oj3hDnPTZB0+ZKHhzkwG4704gByBO6dCGZ2GiIGN46N7Cv+9Vn/AHJIp2SRu 5839O17ad0NBI7+gXQIazEoBgl4N3Fq8D4pXOdOK+/n8y3axgPIz4Ch5Ytju6e3hVBQEQUsJseI56 M8PPeJmQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tV6Dk-00000004DDT-487O; Tue, 07 Jan 2025 09:49:56 +0000 Received: from mail-ej1-x62c.google.com ([2a00:1450:4864:20::62c]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tV65f-00000004AME-0HkB for linux-arm-kernel@lists.infradead.org; Tue, 07 Jan 2025 09:41:39 +0000 Received: by mail-ej1-x62c.google.com with SMTP id a640c23a62f3a-aa692211331so2940850266b.1 for ; Tue, 07 Jan 2025 01:41:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1736242894; x=1736847694; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=Mrz9pc85brY24UNXnbG/br6V/oQ7kbdLPycDpNkgqOA=; b=POKUctrygjanc8h45HKQUWHqr3yYzdMkyaUa8p/I+R8ZXo6iY8y2Lkgx4eY5ZrJCc+ mWl5CiwWfBTJqnftcUzmzx6uiv/O/mQyShqBHl7lCPn0u+QGCUUXK1uOXounCG4b6OeO /g6pNYV6oPvy3TaAFMsiv2MqXxdhH3zK/tuvm1mGOX1f7Se1twQ+HN3HLRYQVlp8NOpH 4c2ruAENojyMLM1ngS8qLI6QvABP4ZiuNj1jEw+BHMt/00HIWfv2NB8hgkdf23bfQZgz B11hQk5BZY1nJPHF6LK9vHb7ATq7yKH+3PLsMsERxuqu1qbWl0goGcsK8m6V/lsRo44N YnGw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736242894; x=1736847694; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Mrz9pc85brY24UNXnbG/br6V/oQ7kbdLPycDpNkgqOA=; b=F056RYM3JygncYDfLMXrp/bJepI1wnMZt/PX1D/5G0b0MnHUNsOJFMNq5hh0pV4+6G cIcC+9IK1KnuPvuCEW6wPCyQts/zbZo3dqe7H0O1Q3cCGqIkwd6IKymuuiwxG8ign1AG RuMo5W3RcSHmHLp0vntXyHNkOoZeNC5GSmYA0PP88Ci4Je2q8HLB7y/9/rLN6ygC5/Ar L2Q2lsIo/Tjvi+Q52Mw7HLeAbKFLm11m514Q6jcmavfOu4vQs/rWhQjNYzQZv2HI258L FFhPTkuiF16TA0+4DzxCHx/RuHU2rVTr/SLARVPT/ceFxMZu3Yl94LMArBgOjG+K/DVg 6bIw== X-Gm-Message-State: AOJu0Yz3LDfY776AI8yVDVPjgweyNQHjHnTeYsPrvME/fAHOBUN90UJi Oapr7kADoJCMjYpjcqcmEK/pkDUmnsttMhQKH7bTrxv2OfIaipK2jnnQqAUHmWQ= X-Gm-Gg: ASbGncsA6BejRQBJToWzam0EjFwKdRvw9l4ULMmSr0u+DN5e0dhyAXX8U1XtXcDCt3k tYxR/4ht8Soqf4dbyLDH1BzFq0WYyzZHwgMwgIVljavAu2PqSm5UhUIhEt6Snt/OsVyqGMmOJGg oXlYhNjkbFkMeZjYN6Rf2baeb6wFJmhB2lxRsK1V5S+YS3nkdgHa6rvQ0Dqi/an30vV0LJoq9pr E9sUPI4okRAMgWu7grI2mPhnBUx/IKk15yIejwBlrV0G2YRE77hmBZzKhS7MMudtf8= X-Google-Smtp-Source: AGHT+IE738Tqr3jv6Gy2r2AS56pcvxRwCYsE6u9XLiSLTVsU2J8/19plrBtKkqKUhsaBw0bxCxZ+nw== X-Received: by 2002:a17:907:36c8:b0:aa6:88a2:cfbd with SMTP id a640c23a62f3a-ab2918b5170mr192603366b.22.1736242893618; Tue, 07 Jan 2025 01:41:33 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aac0f012229sm2356901166b.133.2025.01.07.01.41.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jan 2025 01:41:33 -0800 (PST) From: Linus Walleij Date: Tue, 07 Jan 2025 10:41:22 +0100 Subject: [PATCH RFC v3 06/30] ARM: entry: Invoke syscalls using C MIME-Version: 1.0 Message-Id: <20250107-arm-generic-entry-v3-6-4e5f3c15db2d@linaro.org> References: <20250107-arm-generic-entry-v3-0-4e5f3c15db2d@linaro.org> In-Reply-To: <20250107-arm-generic-entry-v3-0-4e5f3c15db2d@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250107_014135_134919_50F56EA3 X-CRM114-Status: GOOD ( 28.39 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org This makes the assembly invoking syscalls switch over to doing this from C with a small assembly stub to actually jump into the syscall. Split the syscall invocation into two paths: one for plain invocation and one for tracing (also known as "reload" as it was reloading the registers from regs). We rename this path with the infix "trace" as that code will be trace-specific as we move code over to C. Some registers such as r1 and lr get cobbled during the C calls and need to be restored when we return. Right now the part in C doesn't do much more than check the syscall number to be valid (a test previously done with a cmp r #NR_syscalls inside the invoke_syscall macro) but we will gradually factor over more assembly to C that can then be switched to the generic entry code so the exercise gets a point. Since both functions are called from a C file that is in turn only called from assembly, the functions have to be tagged with __ADDRESSABLE() in order not to upset CFI. Tested with a full system boot and by issuing some command line tools with strace to make sure the tracing path still works. Signed-off-by: Linus Walleij --- arch/arm/include/asm/syscall.h | 3 ++ arch/arm/kernel/Makefile | 3 +- arch/arm/kernel/entry-common.S | 65 ++++++++++++++++++++++++++++++++++++------ arch/arm/kernel/entry-header.S | 25 ---------------- arch/arm/kernel/syscall.c | 28 ++++++++++++++++++ 5 files changed, 90 insertions(+), 34 deletions(-) diff --git a/arch/arm/include/asm/syscall.h b/arch/arm/include/asm/syscall.h index fe4326d938c18efb54c96930e6b52ab7da9b2942..1d21f26ecf510ac00a878b4d51a5753ee49475f5 100644 --- a/arch/arm/include/asm/syscall.h +++ b/arch/arm/include/asm/syscall.h @@ -19,6 +19,9 @@ extern const unsigned long sys_call_table[]; +int invoke_syscall(void *table, struct pt_regs *regs, int scno, void *retp); +int invoke_syscall_trace(void *table, struct pt_regs *regs, int scno, void *retp); + static inline int syscall_get_nr(struct task_struct *task, struct pt_regs *regs) { diff --git a/arch/arm/kernel/Makefile b/arch/arm/kernel/Makefile index aaae31b8c4a5dba25b4cd362fe95e5ad2fa0df6c..d5a128a4228dbdf5825aca7385deb94b473716a8 100644 --- a/arch/arm/kernel/Makefile +++ b/arch/arm/kernel/Makefile @@ -20,7 +20,8 @@ CFLAGS_REMOVE_return_address.o = -pg obj-y := elf.o entry-common.o irq.o opcodes.o \ process.o ptrace.o reboot.o io.o \ setup.o signal.o sigreturn_codes.o \ - stacktrace.o sys_arm.o time.o traps.o + stacktrace.o sys_arm.o time.o traps.o \ + syscall.o KASAN_SANITIZE_stacktrace.o := n KASAN_SANITIZE_traps.o := n diff --git a/arch/arm/kernel/entry-common.S b/arch/arm/kernel/entry-common.S index da5c2d4b62e56cce644b15e329ab4149d69c5b79..57aa1084a0476e9c218cd100ce4fdf4aaa748234 100644 --- a/arch/arm/kernel/entry-common.S +++ b/arch/arm/kernel/entry-common.S @@ -254,21 +254,23 @@ ENTRY(vector_swi) #else str scno, [tsk, #TI_ABI_SYSCALL] #endif - /* - * Reload the registers that may have been corrupted on entry to - * the syscall assembly (by tracing or context tracking.) - */ - TRACE( ldmia sp, {r0 - r3} ) - + mov r1, sp @ put regs into r1 ldr r10, [tsk, #TI_FLAGS] @ check for syscall tracing stmdb sp!, {r4, r5} @ push fifth and sixth args tst r10, #_TIF_SYSCALL_WORK @ are we tracing syscalls? bne __sys_trace - invoke_syscall tbl, scno, r10, __ret_fast_syscall + mov r0, tbl + /* r1 already contains regs */ + mov r2, scno @ syscall number from r7 + badr r3, __ret_fast_syscall + bl invoke_syscall + /* Restore regs into r1 and lr after C call */ + badr lr, __ret_fast_syscall add r1, sp, #S_OFF + 2: cmp scno, #(__ARM_NR_BASE - __NR_SYSCALL_BASE) eor r0, scno, #__NR_SYSCALL_BASE @ put OS number back bcs arm_syscall @@ -301,7 +303,16 @@ __sys_trace: add r0, sp, #S_OFF bl syscall_trace_enter mov scno, r0 - invoke_syscall tbl, scno, r10, __sys_trace_return, reload=1 + mov r2, r0 @ scno into r2 + add r1, sp, #S_R0 + S_OFF @ pointer to regs + mov r0, tbl + badr r3, __sys_trace_return + bl invoke_syscall_trace + + /* Restore regs into r1 and lr after C call */ + add r1, sp, #S_R0 + S_OFF @ pointer to regs + badr lr, __sys_trace_return + cmp scno, #-1 @ skip the syscall? bne 2b add sp, sp, #S_OFF @ restore stack @@ -415,6 +426,44 @@ sys_mmap2: b sys_mmap_pgoff ENDPROC(sys_mmap2) +/* + * This call wants: + * r0: syscall table + * r1: regs + * r2: syscall number + * r3: pointer to return function + */ +SYM_TYPED_FUNC_START(invoke_syscall_asm) +#ifdef CONFIG_CPU_SPECTRE + csdb +#endif + mov tbl, r0 + mov scno, r2 + mov lr, r3 @ return address + ldmia r1, {r0 - r3} @ reload r0-r3 + /* Arguments 5 and 6 are (hopefully) on the stack */ + ldr pc, [tbl, scno, lsl #2] @ call sys_* routine +SYM_FUNC_END(invoke_syscall_asm) + +/* + * This call wants: + * r0: syscall table + * r1: regs + * r2: syscall number + * r3: pointer to return function + */ +SYM_TYPED_FUNC_START(invoke_syscall_trace_asm) +#ifdef CONFIG_CPU_SPECTRE + csdb +#endif + mov tbl, r0 + mov scno, r2 + mov lr, r3 @ return address + ldmia r1, {r0 - r6} @ reload r0-r6 + stmia sp, {r4, r5} @ update stack arguments + ldr pc, [tbl, scno, lsl #2] @ call sys_* routine +SYM_FUNC_END(invoke_syscall_trace_asm) + #ifdef CONFIG_OABI_COMPAT /* diff --git a/arch/arm/kernel/entry-header.S b/arch/arm/kernel/entry-header.S index 99411fa913501cad8b55ef04a2b8ab3d44f3e39e..52b4fa97226dbfa4b55aca8eaf74ae24e1e220f8 100644 --- a/arch/arm/kernel/entry-header.S +++ b/arch/arm/kernel/entry-header.S @@ -389,31 +389,6 @@ ALT_UP_B(.L1_\@) #endif .endm - .macro invoke_syscall, table, nr, tmp, ret, reload=0 -#ifdef CONFIG_CPU_SPECTRE - mov \tmp, \nr - cmp \tmp, #NR_syscalls @ check upper syscall limit - movcs \tmp, #0 - csdb - badr lr, \ret @ return address - .if \reload - add r1, sp, #S_R0 + S_OFF @ pointer to regs - ldmiacc r1, {r0 - r6} @ reload r0-r6 - stmiacc sp, {r4, r5} @ update stack arguments - .endif - ldrcc pc, [\table, \tmp, lsl #2] @ call sys_* routine -#else - cmp \nr, #NR_syscalls @ check upper syscall limit - badr lr, \ret @ return address - .if \reload - add r1, sp, #S_R0 + S_OFF @ pointer to regs - ldmiacc r1, {r0 - r6} @ reload r0-r6 - stmiacc sp, {r4, r5} @ update stack arguments - .endif - ldrcc pc, [\table, \nr, lsl #2] @ call sys_* routine -#endif - .endm - /* * These are the registers used in the syscall handler, and allow us to * have in theory up to 7 arguments to a function - r0 to r6. diff --git a/arch/arm/kernel/syscall.c b/arch/arm/kernel/syscall.c new file mode 100644 index 0000000000000000000000000000000000000000..edc0ac88ec60ce3f23149a526de7dc1205906552 --- /dev/null +++ b/arch/arm/kernel/syscall.c @@ -0,0 +1,28 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include +#include + +int invoke_syscall_asm(void *table, struct pt_regs *regs, int scno, void *retp); +__ADDRESSABLE(invoke_syscall_asm); + +__visible int invoke_syscall(void *table, struct pt_regs *regs, int scno, void *retp) +{ + if (scno < NR_syscalls) + /* Doing this with return makes sure the stack gets pop:ed */ + return invoke_syscall_asm(table, regs, scno, retp); + + return 0; +} + +int invoke_syscall_trace_asm(void *table, struct pt_regs *regs, int scno, void *retp); +__ADDRESSABLE(invoke_syscall_trace_asm); + +__visible int invoke_syscall_trace(void *table, struct pt_regs *regs, int scno, void *retp) +{ + if (scno < NR_syscalls) + /* Doing this with return makes sure the stack gets pop:ed */ + return invoke_syscall_trace_asm(table, regs, scno, retp); + + return 0; +} From patchwork Tue Jan 7 09:41:23 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13928589 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id CF02CE77197 for ; Tue, 7 Jan 2025 09:51:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=KCIOsAtrOIwVuMz8AAsW+bZvUjPbPot+cHp/DueUyE0=; b=ADfNCQK4AQuR0TRDRu+WsJImjU f8b/6AAMSs66DZ+/8WN0VC2yWGaBCl6LRBT2xpToG/8nxDW3SmVkDqD299EIrh+WdYDmYlXTV5E7t Wl6lv3G1weuXxO2U4QeiW0Pz9kz7+dg6CxrBPSS4/eUBOZH9T7TGYNSW0iYjQeTmCC9bu6uYYfm35 JoWPYdy1h0moPxIeyG543+B+SjfvoQn5wu+2hg9S78spP5+XcyyzOHfIXdbU62enUphRCbZAkWnVU DKYT+JAWZbRAO8XUwQ828NptxD0YZuLwAVeoRG30I9Hk9n4/xlFmyKmRTieRKYQ9dFEP04M5ATNVb /CzqZJ6g==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tV6Ev-00000004DWh-2nUH; Tue, 07 Jan 2025 09:51:09 +0000 Received: from mail-ed1-x52c.google.com ([2a00:1450:4864:20::52c]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tV65g-00000004AMq-1KyJ for linux-arm-kernel@lists.infradead.org; Tue, 07 Jan 2025 09:41:40 +0000 Received: by mail-ed1-x52c.google.com with SMTP id 4fb4d7f45d1cf-5d3d143376dso22532579a12.3 for ; Tue, 07 Jan 2025 01:41:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1736242895; x=1736847695; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=KCIOsAtrOIwVuMz8AAsW+bZvUjPbPot+cHp/DueUyE0=; b=ACJRm+wkAWnI6VQpXkr3Txvd3BHPK1HCYPuqhcPsxrDbNRQbkbMC/OukYW8YkLIU1X QtN7v+o7sZGh51NE00eW/YMbOXMD9SYvpEgfxOEB1mRFvCASGsApVIDVyK8isoKnDMA9 tHtr5qB4RbuUtTOU+lUs7CcQSQ9CqT+BqREpgw2S5LJRJX5td0uy+jwpw5znUOEuW3qu WZSmO3/rf/iPo9YEsQytw5C6rVl5hBd+UOZPY/srdKkVb5StKufvLTTojtllyDSvJM19 eihTD97163lz8g6/Lnx7przntuK7lbaXEP9u7cLSUSrsxIbOrjzPVK8/6h5HBiEN2Kst NxaA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736242895; x=1736847695; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=KCIOsAtrOIwVuMz8AAsW+bZvUjPbPot+cHp/DueUyE0=; b=bP0sNeKOe8VlxKMYqaUEmf43qQu4XOiMcjvuRcn2BGpsISy37UfZobtX9mNrL1b1Zc sbZu2uRlpDDiv/YIFokxCBh613Cn1i10Ech/YwYrQsEQMLNeit6c8nvRbDyEEIkHiu9L oTBdarpTLoU1HCvNNRZHwUV1mUrfl9YUgE9SjWUAjER/LVodxCn6nTxR8m1fu7oQ89XE bbE910UNi8wr+ugEPc/tzjfOr+IyM4z4OIJuZh7eVmgyejV6WDzWRuNb5xY7vXPvkK/b vNxQKk61DHCJYR0TUSvWjrzsKCVwB+Lm1g56F+HvqGNQ54xLZNuOoieCv0RFPodZE5s1 Gmyw== X-Gm-Message-State: AOJu0Yy9IoEw8eHuBurm1y0K1XwzCFI3tfrmpV1zTQCCKTtIJDaS2Tdy tbhtlZ+NVzunZ1vbkM1n+sOnVMKx26OVoF4ip2MwBz5rdKMQGhd+ZxIC7ZsaHA0= X-Gm-Gg: ASbGncuvXaFNJ9hfeZ9jhbVnTPQnzuOixHjnt+IXX2SE/PjDO0VgmJSr2pW1IcDbcBj 12SyACNcCDAxCRgmbfxx0ftHG+K3Guw6/ZacqHcrfm3nnpHoZ6k58ZL6QWga2yktOHPYTcwZwok Z6mzp4O3m6/IdXfuSORi4m8rVwh0x0Z/jyTL3suk/44g/ig+HtBNF7Ho5lD9fX9dybh0ZlJTDJV MnFLOmMkCfI305/3pGdefF+UKwQPxbfrelAIiLz0USdvKqqbXOVFEKPSShxPHf2Ui8= X-Google-Smtp-Source: AGHT+IGsYLeEIV9Svcr+n7wuZvUGXSmBLy0vRkLLhWqpBA9vLSf8rLeHtiJVrepCoMzwbdEyg/jczw== X-Received: by 2002:a17:907:9345:b0:aae:ea52:3a5 with SMTP id a640c23a62f3a-aaeea5204bemr4516811766b.44.1736242894733; Tue, 07 Jan 2025 01:41:34 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aac0f012229sm2356901166b.133.2025.01.07.01.41.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jan 2025 01:41:34 -0800 (PST) From: Linus Walleij Date: Tue, 07 Jan 2025 10:41:23 +0100 Subject: [PATCH RFC v3 07/30] ARM: entry: Rewrite two asm calls in C MIME-Version: 1.0 Message-Id: <20250107-arm-generic-entry-v3-7-4e5f3c15db2d@linaro.org> References: <20250107-arm-generic-entry-v3-0-4e5f3c15db2d@linaro.org> In-Reply-To: <20250107-arm-generic-entry-v3-0-4e5f3c15db2d@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250107_014136_366530_44A15ED9 X-CRM114-Status: GOOD ( 17.31 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The normal and trace entry code calls out to arm_syscall() and sys_ni_syscall() from assembly, but these calls can be moved over to the new C implementation. Signed-off-by: Linus Walleij --- arch/arm/include/asm/traps.h | 2 +- arch/arm/kernel/entry-common.S | 23 +++++------------------ arch/arm/kernel/syscall.c | 13 +++++++++++-- arch/arm/kernel/traps.c | 2 +- 4 files changed, 18 insertions(+), 22 deletions(-) diff --git a/arch/arm/include/asm/traps.h b/arch/arm/include/asm/traps.h index 2621b9fb9b19b064aa40a8ab4a3a89067b98a010..b888912c2450e0e8eec8139166c7292f283545c7 100644 --- a/arch/arm/include/asm/traps.h +++ b/arch/arm/include/asm/traps.h @@ -40,7 +40,7 @@ asmlinkage void dump_backtrace_stm(u32 *stack, u32 instruction, const char *logl asmlinkage void do_undefinstr(struct pt_regs *regs); asmlinkage void handle_fiq_as_nmi(struct pt_regs *regs); asmlinkage void bad_mode(struct pt_regs *regs, int reason); -asmlinkage int arm_syscall(int no, struct pt_regs *regs); +int arm_syscall(int no, struct pt_regs *regs); asmlinkage void baddataabort(int code, unsigned long instr, struct pt_regs *regs); asmlinkage void __div0(void); asmlinkage void handle_bad_stack(struct pt_regs *regs); diff --git a/arch/arm/kernel/entry-common.S b/arch/arm/kernel/entry-common.S index 57aa1084a0476e9c218cd100ce4fdf4aaa748234..77801d039b5f6562b231a6fd4979e318c7e736a6 100644 --- a/arch/arm/kernel/entry-common.S +++ b/arch/arm/kernel/entry-common.S @@ -264,18 +264,10 @@ ENTRY(vector_swi) mov r0, tbl /* r1 already contains regs */ mov r2, scno @ syscall number from r7 - badr r3, __ret_fast_syscall - bl invoke_syscall - - /* Restore regs into r1 and lr after C call */ + /* We return here no matter what, also pass this as an argument */ badr lr, __ret_fast_syscall - add r1, sp, #S_OFF - -2: cmp scno, #(__ARM_NR_BASE - __NR_SYSCALL_BASE) - eor r0, scno, #__NR_SYSCALL_BASE @ put OS number back - bcs arm_syscall - mov why, #0 @ no longer a real syscall - b sys_ni_syscall @ not private func + mov r3, lr + b invoke_syscall #if defined(CONFIG_OABI_COMPAT) || !defined(CONFIG_AEABI) /* @@ -308,13 +300,8 @@ __sys_trace: mov r0, tbl badr r3, __sys_trace_return bl invoke_syscall_trace - - /* Restore regs into r1 and lr after C call */ - add r1, sp, #S_R0 + S_OFF @ pointer to regs - badr lr, __sys_trace_return - - cmp scno, #-1 @ skip the syscall? - bne 2b + cmp r0, #-1 + bne __sys_trace_return add sp, sp, #S_OFF @ restore stack __sys_trace_return_nosave: diff --git a/arch/arm/kernel/syscall.c b/arch/arm/kernel/syscall.c index edc0ac88ec60ce3f23149a526de7dc1205906552..d637dc74b1c70415933898fbcadbedc71df5f654 100644 --- a/arch/arm/kernel/syscall.c +++ b/arch/arm/kernel/syscall.c @@ -12,7 +12,10 @@ __visible int invoke_syscall(void *table, struct pt_regs *regs, int scno, void * /* Doing this with return makes sure the stack gets pop:ed */ return invoke_syscall_asm(table, regs, scno, retp); - return 0; + if (scno >= __ARM_NR_BASE) + return arm_syscall(scno, regs); + + return sys_ni_syscall(); } int invoke_syscall_trace_asm(void *table, struct pt_regs *regs, int scno, void *retp); @@ -20,9 +23,15 @@ __ADDRESSABLE(invoke_syscall_trace_asm); __visible int invoke_syscall_trace(void *table, struct pt_regs *regs, int scno, void *retp) { + if (scno == -1) + return -1; + if (scno < NR_syscalls) /* Doing this with return makes sure the stack gets pop:ed */ return invoke_syscall_trace_asm(table, regs, scno, retp); - return 0; + if (scno >= __ARM_NR_BASE) + return arm_syscall(scno, regs); + + return sys_ni_syscall(); } diff --git a/arch/arm/kernel/traps.c b/arch/arm/kernel/traps.c index 6ea645939573fb65ed36f5435a9c94e98828f45b..942c77767919466d5b89d83d8eb77032b119473e 100644 --- a/arch/arm/kernel/traps.c +++ b/arch/arm/kernel/traps.c @@ -609,7 +609,7 @@ do_cache_op(unsigned long start, unsigned long end, int flags) * 0x9f0000 - 0x9fffff are some more esoteric system calls */ #define NR(x) ((__ARM_NR_##x) - __ARM_NR_BASE) -asmlinkage int arm_syscall(int no, struct pt_regs *regs) +int arm_syscall(int no, struct pt_regs *regs) { if ((no >> 16) != (__ARM_NR_BASE>> 16)) return bad_syscall(no, regs); From patchwork Tue Jan 7 09:41:24 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13928590 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 26FE3E77197 for ; Tue, 7 Jan 2025 09:52:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=eMg93yBwVVTokRKyzFlWLmvMqpyEbrII8V8DSY/WZf4=; b=q6qT3XzCD719mtLZahc0+8ab1b Ab8fWIqW6DU9aJSsezIyYjczf4mhaYVqlcDny/CClpOaSzZKelY8LDIOzvCXTZwAaTBDTQLMQmzVs f8JhMosCgpVDl/u/p4eIXbV7FZKTB5aaCmGnJHDwiNal98rT9/OuAZuW8Q39cXW9UQCGqYOUqT+MQ //lCeOHNUPmuR9LY0iVag8M2zDvZap1VWKfFk8EbkMJYmEdamkQaKaSDmelSyYaIy3Pcf4R9kyULY TKhE+8y4d/QXQyHHvtVwVPGRbbftV27jMl1uk25bP+AMB+b9XXs2vERyqkGbdLu31exJe9ErP+PaN nFoq147w==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tV6G4-00000004Dfh-2LPN; Tue, 07 Jan 2025 09:52:20 +0000 Received: from mail-ej1-x635.google.com ([2a00:1450:4864:20::635]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tV65i-00000004ANX-0Hx0 for linux-arm-kernel@lists.infradead.org; Tue, 07 Jan 2025 09:41:41 +0000 Received: by mail-ej1-x635.google.com with SMTP id a640c23a62f3a-aaec111762bso2491393666b.2 for ; Tue, 07 Jan 2025 01:41:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1736242896; x=1736847696; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=eMg93yBwVVTokRKyzFlWLmvMqpyEbrII8V8DSY/WZf4=; b=lju2eRTCCCWpcKfLPYQ1gPWHCWoKH4s/Vnh38TxRrag6uLcHZhV2eBP3tG4J62R27J L6d1ffCKol/L3NnKW9yhpe0+5Rg6xvwYjhl6zT8yebWRxmgDi8UuNafDZF2RPhdfxdxz 7Y9nj+Rpy0AWLBvFSp5mrkzROA7InN9e7v5sTEkZvScERcLFVIIrvJjoPtgXUB0zAbe1 W1i3aa0K4ESqQgIk4K2BekB5/IlGjumpuo0BbPwAJwtcHiioCzhNvyyNfDpiXvf72+nI RsVxuZjm179pdrvoBH701m/ocz+o+t7hcXxWToaNvrXfPIccSWbTOJ10RZ9hMxAQI8iA LNuQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736242896; x=1736847696; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=eMg93yBwVVTokRKyzFlWLmvMqpyEbrII8V8DSY/WZf4=; b=dstRH/iDN5w433vWwVJ1Z7PxVFetjQes56poKpY/RlgynPJx5/3hNyGwYAG3tcA09N AYOjJeY5t0l6KWmQw/UZyUCqnClREesnFOkIMlyVR9XtC5XKwoWx9rMnovOu1JBorpCn quK3wTlFwwmamrvIBFvA2fxFh9mP1gjKjgDW4h/CaXjPjl1ARi+GHhYl5rrTA8D2d93+ fqo2lwiBmjhILUcKE2IewEHrz5RzCfPY6wwrhOvOsNCvMoa+2Qxx1V5sXlNqsxO19vHo WsGc8+yUitGFXqF17a1btiDOKcH0sojJ+ve3luKcSJpi2f22YYXgQ0bl9Wb2EbT9Is3+ juqA== X-Gm-Message-State: AOJu0Yy2KY44580OaByRGAOy92uVEGNbw0tRxz2p6myhsIoKe3jMD45x eOsu1a+ncm61OAx9JfXrLW27fNta2tMkCE5dpjHzf5Mq8EvIjb+R+1e2isyN5Vw= X-Gm-Gg: ASbGnctJof0zjINX0fc5TBPDGI0aiHAIk07hWEcwB1S3CoJ4wwkr+7KLAFTyhVNlq7+ H/cD5ob87oVHnGBIiFdJsYiidMXbxm3cw6CPgg6cVXtDEGrqe41EBcrlpkp9fKpIqeRfK0lqMha ZIRdbLmm7oCc6EvgeW2sScNQ5ThCS7MJfbg5alr8d7dCapbxB9XS+2dW4YyKaFFHdYZ9MmssfIV g4yIgOqX8MYPBPqOtvWrVWpWCBTimoKYtoXnP40IM98ypppjzg+N5hAsEkusWcSnjc= X-Google-Smtp-Source: AGHT+IGWBlZrGMzIZx/5MGyqW+jdyguusLjBOk7dr/oDrWgqG28clwNqww9qx9LhDjGwhVlgBeZiHg== X-Received: by 2002:a17:907:7208:b0:aa5:43c4:da78 with SMTP id a640c23a62f3a-aac33685dfemr5301778966b.51.1736242896331; Tue, 07 Jan 2025 01:41:36 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aac0f012229sm2356901166b.133.2025.01.07.01.41.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jan 2025 01:41:35 -0800 (PST) From: Linus Walleij Date: Tue, 07 Jan 2025 10:41:24 +0100 Subject: [PATCH RFC v3 08/30] ARM: entry: Move trace entry to C function MIME-Version: 1.0 Message-Id: <20250107-arm-generic-entry-v3-8-4e5f3c15db2d@linaro.org> References: <20250107-arm-generic-entry-v3-0-4e5f3c15db2d@linaro.org> In-Reply-To: <20250107-arm-generic-entry-v3-0-4e5f3c15db2d@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250107_014138_109436_7DFE02E8 X-CRM114-Status: GOOD ( 15.17 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org This moves over the code entering into tracing into the C syscall invocation code. As syscall_trace_enter() will provide the current syscall number from the regs we need not provide the syscall number to the invocation function any more. Signed-off-by: Linus Walleij --- arch/arm/include/asm/syscall.h | 2 +- arch/arm/kernel/entry-common.S | 6 +----- arch/arm/kernel/syscall.c | 5 ++++- 3 files changed, 6 insertions(+), 7 deletions(-) diff --git a/arch/arm/include/asm/syscall.h b/arch/arm/include/asm/syscall.h index 1d21f26ecf510ac00a878b4d51a5753ee49475f5..66067d165ba3fbbe0f840a89ae396eb1a311bacc 100644 --- a/arch/arm/include/asm/syscall.h +++ b/arch/arm/include/asm/syscall.h @@ -20,7 +20,7 @@ extern const unsigned long sys_call_table[]; int invoke_syscall(void *table, struct pt_regs *regs, int scno, void *retp); -int invoke_syscall_trace(void *table, struct pt_regs *regs, int scno, void *retp); +int invoke_syscall_trace(void *table, struct pt_regs *regs, void *retp); static inline int syscall_get_nr(struct task_struct *task, struct pt_regs *regs) diff --git a/arch/arm/kernel/entry-common.S b/arch/arm/kernel/entry-common.S index 77801d039b5f6562b231a6fd4979e318c7e736a6..3cfc6d952ff99be9c4c1be4481ac3039260e3e57 100644 --- a/arch/arm/kernel/entry-common.S +++ b/arch/arm/kernel/entry-common.S @@ -292,13 +292,9 @@ ENDPROC(vector_swi) * context switches, and waiting for our parent to respond. */ __sys_trace: - add r0, sp, #S_OFF - bl syscall_trace_enter - mov scno, r0 - mov r2, r0 @ scno into r2 add r1, sp, #S_R0 + S_OFF @ pointer to regs mov r0, tbl - badr r3, __sys_trace_return + badr r2, __sys_trace_return bl invoke_syscall_trace cmp r0, #-1 bne __sys_trace_return diff --git a/arch/arm/kernel/syscall.c b/arch/arm/kernel/syscall.c index d637dc74b1c70415933898fbcadbedc71df5f654..377e16c8c53c8e809e3f8b157a889ef115d5c043 100644 --- a/arch/arm/kernel/syscall.c +++ b/arch/arm/kernel/syscall.c @@ -21,8 +21,11 @@ __visible int invoke_syscall(void *table, struct pt_regs *regs, int scno, void * int invoke_syscall_trace_asm(void *table, struct pt_regs *regs, int scno, void *retp); __ADDRESSABLE(invoke_syscall_trace_asm); -__visible int invoke_syscall_trace(void *table, struct pt_regs *regs, int scno, void *retp) +__visible int invoke_syscall_trace(void *table, struct pt_regs *regs, void *retp) { + int scno; + + scno = syscall_trace_enter(regs); if (scno == -1) return -1; From patchwork Tue Jan 7 09:41:25 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13928593 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 90DD6E77197 for ; Tue, 7 Jan 2025 09:55:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=sQAp8VACnKHObFrcN6BKQWBu0lC+h+9xnjqKvK7FcXs=; b=SuFsZosk2yBjGjj56/lor85Ptr rM+3y0ddrOfHF+VaJvDrnIsHdNBIk5+lmhNOCJdG65OKaoK0keLKgas81Wqn+oiByFb1rAdFyiXod 9dkpySqD1m1JEIEKUVYVkZyEpAQ/WiuBZDItk3mKNuVzS7ZmcuXhXC72swixhiDAXwv8wCQdhDsDg EeREarYjotVz71gTNIWDhrJR9iZ7DqM3ThQSoMMglzIAz3leQXRL/rKKsKsw8yFYcCaCcjdXhUOSS vvVWbotPYAK351K6UEVhy+mW8hXFigehW6dv7ZhLaT/ibhlLdvbnag8z9wJ2ANY7QeDVtx2ECfBnm kxahlseQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tV6IN-00000004E58-3aXV; Tue, 07 Jan 2025 09:54:43 +0000 Received: from mail-ej1-x634.google.com ([2a00:1450:4864:20::634]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tV65j-00000004AOV-1dD0 for linux-arm-kernel@lists.infradead.org; Tue, 07 Jan 2025 09:41:43 +0000 Received: by mail-ej1-x634.google.com with SMTP id a640c23a62f3a-aa684b6d9c7so2540118666b.2 for ; Tue, 07 Jan 2025 01:41:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1736242898; x=1736847698; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=sQAp8VACnKHObFrcN6BKQWBu0lC+h+9xnjqKvK7FcXs=; b=WeacyQrKCmNgPMMjZ+izN9G5nZe166jFR8bqniMwKtTahStQEu1L9LXGptLApZrFTy O3K3zQIQsMIOPG06iXemkZ3bpYnx00dUEvaqvWwqR/AWzag7AtweRmw0gn0dFVA4BK10 9zTvNz/0N9dvOxM/3k4Rc0D8Fl1EjbXOLvHVO2hI2QpwkNrLmk/C+fMoe3IsWW0jVxfi WcaCkFVjxgsbmlfcmxpEj8uwqrxkCP1t6ISWma4yVVOYrwmUo0rpLUondxF/qL/oU/i5 Wt0KSSqJIi2mNrajVjH9d+8iBQx3pyEroJvEB47l3lvjEYNXBqM9lyzdvNBcYc5k52J1 d81w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736242898; x=1736847698; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=sQAp8VACnKHObFrcN6BKQWBu0lC+h+9xnjqKvK7FcXs=; b=QBHFOl7fVnm3IQgT5MXSfj6bUjxh84Q7e7hMR4dVxUw4WK1M16RtET1pMNp4esdMqS WBceL79xLMCdR51/caEcgL3X3uzr38y8+VpL0iKzmfAGqp8ILilYGpSjIiag29vQ2JU0 DYLmeG4CgJxFiDHl3eaxGfZV0v6TEXnZjN8wHjb6iZ8ZkY1voG2WKTMte3kS5jzAFZe3 7y7clCNWeSKcbFW+8nBlfyFzLUSFM6RVp5jdw0NM/l1f+djrDsP2tH/R/pGcnTB31rIq 5H2Z4in8EN3CrnIWA1NZ2aCqDQNcEgdYAhk+LxlZuJFV97ASwNeH0tIIHCTOxXXvVnz4 E0mQ== X-Gm-Message-State: AOJu0YwVExhdHMoK1AUY8lKCOr8HNO/uC5tN1H33xojzdiD8fS/4k52T 191EoKpsdciuIRD56kI9h302ySERcLEJ3AH6IuDAEwmUIYpmU7/UcqBZKpc2mgw= X-Gm-Gg: ASbGncuLcN/Nui6Itr25RsKmLdY0rJbZBBixqzV3LgGCJBkMZGAifITgqUO4ivbH9xe H1MxMOktmvSt2vsY1wFfrbGSJu1ox54WBFeG41gm6yHx3eZFbSewCjOQQUCk3hErVooHX5SpQGI Z2tPNNVpTU+X23lAWTREmAQpv5lyKX8tmSjke3zUkWKs1aVM2gihdoB2qcNJhsIskIWBUl+ym4j OvWbhWDC6QtKpSru0Dj7Cago6yYHMekDUOM7KzojhfuSYd74UarG2YfX1+dJ3YXuLQ= X-Google-Smtp-Source: AGHT+IFbK5JNMY/DlBPttO03FrsecsUcG94kP+7J7Uhk6bjBGJqZw7Ul1vEZtVACwonxWgRUxG1J2Q== X-Received: by 2002:a17:906:6a27:b0:aa6:9fad:3c1d with SMTP id a640c23a62f3a-aac3464881amr4406657766b.39.1736242897835; Tue, 07 Jan 2025 01:41:37 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aac0f012229sm2356901166b.133.2025.01.07.01.41.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jan 2025 01:41:37 -0800 (PST) From: Linus Walleij Date: Tue, 07 Jan 2025 10:41:25 +0100 Subject: [PATCH RFC v3 09/30] ARM: entry: save the syscall sp in thread_info MIME-Version: 1.0 Message-Id: <20250107-arm-generic-entry-v3-9-4e5f3c15db2d@linaro.org> References: <20250107-arm-generic-entry-v3-0-4e5f3c15db2d@linaro.org> In-Reply-To: <20250107-arm-generic-entry-v3-0-4e5f3c15db2d@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250107_014139_431131_8A82B029 X-CRM114-Status: GOOD ( 18.75 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org We are going to rewrite the syscall handling in C, which means that the stack used by the call code is no longer predicatably 8 bytes (for syscall arguments r4 and r5) but a varying number of bytes depending on how nested the C code is. However the current code is just assuming it can rewind the stack by adding 8 to sp if a syscall is interrupted by a sigreturn call. Solve this by storing the entry sp in the per-task struct thread_info and use that in the sigreturn wrapper instead. We already have the thread info available in the SWI entry and sigreturn is probably not so common that retrieveing a pointer to thread_info should affect anything very much. Storing this per-task in thread_info makes the solution SMP robust. Signed-off-by: Linus Walleij --- arch/arm/include/asm/thread_info.h | 1 + arch/arm/kernel/asm-offsets.c | 1 + arch/arm/kernel/entry-common.S | 8 ++++++-- 3 files changed, 8 insertions(+), 2 deletions(-) diff --git a/arch/arm/include/asm/thread_info.h b/arch/arm/include/asm/thread_info.h index 943ffcf069d29cf4a035964d20d56f7ebdd6d602..d8a45c5a10496aaf806bfeaa0353d5e8985bd6f5 100644 --- a/arch/arm/include/asm/thread_info.h +++ b/arch/arm/include/asm/thread_info.h @@ -67,6 +67,7 @@ struct thread_info { __u32 cpu_domain; /* cpu domain */ struct cpu_context_save cpu_context; /* cpu context */ __u32 abi_syscall; /* ABI type and syscall nr */ + __u32 sp_syscall; /* SP when entering syscall */ unsigned long tp_value[2]; /* TLS registers */ union fp_state fpstate __attribute__((aligned(8))); union vfp_state vfpstate; diff --git a/arch/arm/kernel/asm-offsets.c b/arch/arm/kernel/asm-offsets.c index 4853875740d0fe61c6bbc32ddd9a16fa8d1fb530..c9525cbb26b73827821aa746030e56b037f49556 100644 --- a/arch/arm/kernel/asm-offsets.c +++ b/arch/arm/kernel/asm-offsets.c @@ -49,6 +49,7 @@ int main(void) DEFINE(TI_CPU_DOMAIN, offsetof(struct thread_info, cpu_domain)); DEFINE(TI_CPU_SAVE, offsetof(struct thread_info, cpu_context)); DEFINE(TI_ABI_SYSCALL, offsetof(struct thread_info, abi_syscall)); + DEFINE(TI_SP_SYSCALL, offsetof(struct thread_info, sp_syscall)); DEFINE(TI_TP_VALUE, offsetof(struct thread_info, tp_value)); DEFINE(TI_FPSTATE, offsetof(struct thread_info, fpstate)); #ifdef CONFIG_VFP diff --git a/arch/arm/kernel/entry-common.S b/arch/arm/kernel/entry-common.S index 3cfc6d952ff99be9c4c1be4481ac3039260e3e57..8baab7f97f59c434396f30b08ddd3029c5f9c0e5 100644 --- a/arch/arm/kernel/entry-common.S +++ b/arch/arm/kernel/entry-common.S @@ -232,6 +232,8 @@ ENTRY(vector_swi) uaccess_disable tbl get_thread_info tsk + /* Save a per-task copy of SP for sigreturn */ + str sp, [tsk, #TI_SP_SYSCALL] adr tbl, sys_call_table @ load syscall table pointer @@ -377,13 +379,15 @@ sys_syscall: ENDPROC(sys_syscall) sys_sigreturn_wrapper: - add r0, sp, #S_OFF + get_thread_info tsk + ldr r0, [tsk, #TI_SP_SYSCALL] @ read back SP mov why, #0 @ prevent syscall restart handling b sys_sigreturn ENDPROC(sys_sigreturn_wrapper) sys_rt_sigreturn_wrapper: - add r0, sp, #S_OFF + get_thread_info tsk + ldr r0, [tsk, #TI_SP_SYSCALL] @ read back SP mov why, #0 @ prevent syscall restart handling b sys_rt_sigreturn ENDPROC(sys_rt_sigreturn_wrapper) From patchwork Tue Jan 7 09:41:26 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13928591 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 8AB23E7719A for ; Tue, 7 Jan 2025 09:53:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=OKWM4cPPHTUloQ8W+d7SfJ+6Vh0jRzzpCF+pKqqjEZA=; b=nrH1oMYFxTSknC5bQ7oy/lan3b D8KrUQgaKz+L0a3G69CRifAM4PezPE3hChlDDTrP/gsgu6yEi4qe699T/5IERA1IR8cyaVHB1UzGU iWYf2VV0tVRFmlRpvPTHeZmVMr3+XGxzEhQ4GpEgc63jQf/RN1yzJ08PmZSDQLJdIMho8WhXiU6HD 3pE8jPicoe1rcvPzpIzFXuhMP6RamdQboKwqIPHahYZAKeo1k9CU4SA9XP7ySCT+BFdZdd2czBhY8 aZYF0l0cFGv+GtUH/wrttmAWjh1tgY2dl2avZLbtAwSgA5wq0GeSg99TIdAuiq8x+Q+3+n6NNOBkn kaSJcY/A==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tV6HE-00000004Dq1-0lDj; Tue, 07 Jan 2025 09:53:32 +0000 Received: from mail-ej1-x62d.google.com ([2a00:1450:4864:20::62d]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tV65k-00000004AOy-3StI for linux-arm-kernel@lists.infradead.org; Tue, 07 Jan 2025 09:41:41 +0000 Received: by mail-ej1-x62d.google.com with SMTP id a640c23a62f3a-aab6fa3e20eso2519865666b.2 for ; Tue, 07 Jan 2025 01:41:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1736242899; x=1736847699; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=OKWM4cPPHTUloQ8W+d7SfJ+6Vh0jRzzpCF+pKqqjEZA=; b=eTvkL9WM0hYZY0bEzKtvyIDHUINAImubQJfXvEgU5Iy1OB6+8dW+1LL7Fon1+bDVvg AicNP39Aq9+qedNGUxYbKFvma+oQCDD4uzLddtHZBOmdxqsx+D25rqIHXB14QwHK7q3T 9cB8HywVLtzPPZTz9AbctatQQhIHWV1LfeII4+ugeMSwjd2MSRRMkFf8CyRf6CgIoKLq 6UVLDZvdiWZu4QxQXcWUO6WBN2upey2sBE6EjwDva6NQhlcjiAqviu4SOh3cbXd0vbS6 7CcHLsf+62nk11ahGlsfrTcrVpI6PoT1Gm9NdWrfF4m/T+QbzJZ26mVINLltjkiviZ8K b+Dg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736242899; x=1736847699; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=OKWM4cPPHTUloQ8W+d7SfJ+6Vh0jRzzpCF+pKqqjEZA=; b=NpcWyW9kXVXRpYzL1Z+gsbSMotKJRLieduycs1nUEG2L0NIKaDnBBFsDAx6lXm/mxh Z6TfDcAyeDAd5VL4ge5usfwYGpf36M2QaWSfKouuQKIydreRDWlGbGQI+1AAd7O+Anfm 8ffC004E12I6tlyOUknzDyyDyLwLkrvPtumScXbqqR2nMjvfd+LNyTw3Lk6bqT2OXRBB 6OKGL89t62W5xktF+OWwHLtqG6RJlaAdyJiPyiQfww9Ji+UlGjioT6mbbiLUG3/3ONgj v2ZYelCe98st+UtsDqLRxHJIIDtHWYcy11QGZi2w7cwPewU1NYqlw2cgPzLDKiZSTv+u BBMQ== X-Gm-Message-State: AOJu0YwSuHHGJ16ea1TqxOiwFjmEd8yOmIqVdrU+5O0sCf1pRCEbCWFM P/ghQ9UUzMERxyuWScJxh6HOEgMr4jrQt9DuL9hSoIoI8EXOKK6PU6b9EjAOv7c= X-Gm-Gg: ASbGncuQ67xrD/xr6lCX/anWTLWNEbUI+wv4aavKvO8LBHGOtzXy6tvJYo6weIWGRns /IGtKdhUAhJfPWlaukVVEyhlCO8/TlwVp8z2Q2cB+dmru9A5oWoFs9XXwqq3NwOQaoxWL2YFbFo 5Cf/brcGrIMV/K7ZZXe49yomuU2814FZPnfL6d8YVOu/jS1AcOKtcfaT6LtFKq865xpHKzsWJHb 5SK9FXttPlo7rqASe+kw065eRKpaupMD3/qejVwBZlt+0Ys+E4npwa9xdlcp1mDSfQ= X-Google-Smtp-Source: AGHT+IGORu1sJ9gjYMgOVIE8V4cB1mpEs+RP8HyomPhOZJkI2q6RHOVgjiyQlXfFj0xR8o3+R65swg== X-Received: by 2002:a17:906:c106:b0:aa6:7cae:db98 with SMTP id a640c23a62f3a-aac2d447333mr5019449766b.10.1736242899387; Tue, 07 Jan 2025 01:41:39 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aac0f012229sm2356901166b.133.2025.01.07.01.41.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jan 2025 01:41:38 -0800 (PST) From: Linus Walleij Date: Tue, 07 Jan 2025 10:41:26 +0100 Subject: [PATCH RFC v3 10/30] ARM: entry: move all tracing invocation to C MIME-Version: 1.0 Message-Id: <20250107-arm-generic-entry-v3-10-4e5f3c15db2d@linaro.org> References: <20250107-arm-generic-entry-v3-0-4e5f3c15db2d@linaro.org> In-Reply-To: <20250107-arm-generic-entry-v3-0-4e5f3c15db2d@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250107_014140_868881_31EC8407 X-CRM114-Status: GOOD ( 21.17 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The traced invocation of syscall is rewritten in C, moving over also the call to syscall_trace_exit() to C. To do this we cannot have invoke_syscall_trace_asm() be a leaf call, which is the mechanism we hithereto relied on to make sure the stack is pulled back to the state we were at before we called out to the code written in C. Push the registers potentially used by C on the stack, and make a copy of the two stack-pushed syscall arguments on the top of the stack before invoking the syscall and then drop these copied arguments and pop back the registers before returning from invoke_syscall_trace_asm(). As we are now calling from and returning to C, we no longer need to pass the regs pointer around so drop it. Signed-off-by: Linus Walleij --- arch/arm/include/asm/syscall.h | 2 +- arch/arm/kernel/entry-common.S | 32 +++++++++++++++----------------- arch/arm/kernel/ptrace.c | 2 +- arch/arm/kernel/syscall.c | 30 +++++++++++++++++++++--------- 4 files changed, 38 insertions(+), 28 deletions(-) diff --git a/arch/arm/include/asm/syscall.h b/arch/arm/include/asm/syscall.h index 66067d165ba3fbbe0f840a89ae396eb1a311bacc..cb0073c4151bf70a82d15e17a95b6b6f48b245d6 100644 --- a/arch/arm/include/asm/syscall.h +++ b/arch/arm/include/asm/syscall.h @@ -20,7 +20,7 @@ extern const unsigned long sys_call_table[]; int invoke_syscall(void *table, struct pt_regs *regs, int scno, void *retp); -int invoke_syscall_trace(void *table, struct pt_regs *regs, void *retp); +void invoke_syscall_trace(void *table, struct pt_regs *regs); static inline int syscall_get_nr(struct task_struct *task, struct pt_regs *regs) diff --git a/arch/arm/kernel/entry-common.S b/arch/arm/kernel/entry-common.S index 8baab7f97f59c434396f30b08ddd3029c5f9c0e5..dbc947d301ec11bb13007cb4bb161c035ede5c10 100644 --- a/arch/arm/kernel/entry-common.S +++ b/arch/arm/kernel/entry-common.S @@ -296,22 +296,11 @@ ENDPROC(vector_swi) __sys_trace: add r1, sp, #S_R0 + S_OFF @ pointer to regs mov r0, tbl - badr r2, __sys_trace_return bl invoke_syscall_trace - cmp r0, #-1 - bne __sys_trace_return - add sp, sp, #S_OFF @ restore stack - -__sys_trace_return_nosave: - enable_irq_notrace - mov r0, sp - bl syscall_trace_exit + add sp, sp, #S_OFF @ restore stack pointer b ret_to_user -__sys_trace_return: - str r0, [sp, #S_R0 + S_OFF]! @ save returned r0 - mov r0, sp - bl syscall_trace_exit +__sys_trace_return_nosave: b ret_to_user .macro syscall_table_start, sym @@ -437,18 +426,27 @@ SYM_FUNC_END(invoke_syscall_asm) * r0: syscall table * r1: regs * r2: syscall number - * r3: pointer to return function */ SYM_TYPED_FUNC_START(invoke_syscall_trace_asm) #ifdef CONFIG_CPU_SPECTRE csdb #endif + /* Save registers because we are being called from C */ + push {r4 - r10, lr} mov tbl, r0 + /* Make space to copy the two syscall stack arguments */ + sub sp, sp, #S_OFF mov scno, r2 - mov lr, r3 @ return address - ldmia r1, {r0 - r6} @ reload r0-r6 - stmia sp, {r4, r5} @ update stack arguments + badr lr, __invoke_syscall_trace_ret @ return right here + ldmia r1, {r0 - r6} @ reload r0-r6 from regs + stmia sp, {r4, r5} @ copy stack arguments ldr pc, [tbl, scno, lsl #2] @ call sys_* routine +__invoke_syscall_trace_ret: + /* Drop the copied stack arguments */ + add sp, sp, #S_OFF + pop {r4 - r10, lr} + ARM( mov pc, lr ) + THUMB( bx lr ) SYM_FUNC_END(invoke_syscall_trace_asm) #ifdef CONFIG_OABI_COMPAT diff --git a/arch/arm/kernel/ptrace.c b/arch/arm/kernel/ptrace.c index 07b0daf47441f1f76a8af416acc74fa5ed770403..ac7b98ae47249b4a00fc1cb871bb2d309bfd8c88 100644 --- a/arch/arm/kernel/ptrace.c +++ b/arch/arm/kernel/ptrace.c @@ -888,7 +888,7 @@ asmlinkage int syscall_trace_enter(struct pt_regs *regs) return scno; } -asmlinkage void syscall_trace_exit(struct pt_regs *regs) +void syscall_trace_exit(struct pt_regs *regs) { /* * Audit the syscall before anything else, as a debugger may diff --git a/arch/arm/kernel/syscall.c b/arch/arm/kernel/syscall.c index 377e16c8c53c8e809e3f8b157a889ef115d5c043..e90f04f35b3485968a1e9046cf0889f56cca92e5 100644 --- a/arch/arm/kernel/syscall.c +++ b/arch/arm/kernel/syscall.c @@ -18,23 +18,35 @@ __visible int invoke_syscall(void *table, struct pt_regs *regs, int scno, void * return sys_ni_syscall(); } -int invoke_syscall_trace_asm(void *table, struct pt_regs *regs, int scno, void *retp); +int invoke_syscall_trace_asm(void *table, struct pt_regs *regs, int scno); __ADDRESSABLE(invoke_syscall_trace_asm); -__visible int invoke_syscall_trace(void *table, struct pt_regs *regs, void *retp) +__visible void invoke_syscall_trace(void *table, struct pt_regs *regs) { int scno; + int ret; scno = syscall_trace_enter(regs); if (scno == -1) - return -1; + goto trace_exit_nosave; - if (scno < NR_syscalls) - /* Doing this with return makes sure the stack gets pop:ed */ - return invoke_syscall_trace_asm(table, regs, scno, retp); + if (scno < NR_syscalls) { + ret = invoke_syscall_trace_asm(table, regs, scno); + goto trace_exit_save; + } - if (scno >= __ARM_NR_BASE) - return arm_syscall(scno, regs); + if (scno >= __ARM_NR_BASE) { + ret = arm_syscall(scno, regs); + goto trace_exit_save; + } - return sys_ni_syscall(); + ret = sys_ni_syscall(); + +trace_exit_save: + /* Save return value from syscall */ + regs->ARM_r0 = ret; + +trace_exit_nosave: + local_irq_enable(); + syscall_trace_exit(regs); } From patchwork Tue Jan 7 09:41:27 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13928594 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 276DAE77197 for ; Tue, 7 Jan 2025 09:56:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=obIcK06NDnAV5qCp4HaVvaBhtHVYu9G8V01JmlqBhdo=; b=mz8qeLVVwAqvnycUXEBYrjjpw3 TKe/K/hAEdYPET3I6J0Bix+OFyDKjzDlPQBLR1X8TSmUzjLAgCrlH/bvqVCciUMIV4f8WMqd9SEjU 4EtC9nriS53HVDlgoHDTfsTCQ/lj3XCfugzerOMtzTrLLWqctobEC+u4r4tvESPC58fJ0b7oYNgEG kzA3JfkOles8BqaZ64CHgs1NA7u4hGCWy4ng6Srbct7nsH1polcSk7iLxAfBceKF/J4ljj9mUQy02 4WxpNZk7eY/+SC3kkoIo0tLZOW8e9J0IRllJz0Jlq2xC4LmjWFCn69W9puehPERQAVqUNwflyYbCZ oxgqrIfw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tV6JX-00000004EHp-2xbi; Tue, 07 Jan 2025 09:55:55 +0000 Received: from mail-ej1-x633.google.com ([2a00:1450:4864:20::633]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tV65m-00000004APy-1iCH for linux-arm-kernel@lists.infradead.org; Tue, 07 Jan 2025 09:41:46 +0000 Received: by mail-ej1-x633.google.com with SMTP id a640c23a62f3a-aa67ac42819so2212120966b.0 for ; Tue, 07 Jan 2025 01:41:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1736242901; x=1736847701; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=obIcK06NDnAV5qCp4HaVvaBhtHVYu9G8V01JmlqBhdo=; b=BWla7339edEFAImFESrBJ+7jY17fVmN8cx5rVLi8n+E0h8XuqSLXKEZmYIU2Rdlng5 zGhTYxxpT+wFfB95KTaDGvwwwmWIVehMl5a4NC0WdwKM2rVswFEHfy2gP7nJTE4/cbGu zlIpQi0QiG9WmRk2KuNWMaI+kSzFMQd6yAjyI7VE2ee5+4tCxJV4F6r5NmgKn0VuI4KX Ogdxl+tdSvZXBFuOwxemgc+aAAYj+y3J6XcO4ydNlQv8G1BsqEwLLSUtI9f+aR547P2+ B6IYTY6+ChEo5+Wyr4CqOdq/97gJOGr6rdUGfON0AJPWEgeGNov7jY5/fgTd/CmR1dDX rShA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736242901; x=1736847701; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=obIcK06NDnAV5qCp4HaVvaBhtHVYu9G8V01JmlqBhdo=; b=T5o9wNqeKUDn8FKwR7/tgh9es5eV9iK75vf+xaHmxZlH3OqZBRKMWZZ6DOmTh2DluP IwH+y9idKhQJEX3CW5e6dcUXG/rT122zP2yyZ6++9pik1DVKWPeuNiYQskYYfnTF5Tjp 1YUwndRi1CsZIIpdh1Kq3jhYToQhWr2pRS58A83PEwNUH6GhF99hgpVeSKrR8rJvjU8a 7eO/BGm4t0wvqzjU8K4JoiS7uL5sptJo7YxHJ7GK9FUHzcBHjAQy0c8LPRszw3kd0WZp XeAim7WEfFoKY2us7eTd7qh5FRvyEgLRUEkH1d0q1L8W9H0BZ6HMXoKbgp6DKkwZNBVn sLmg== X-Gm-Message-State: AOJu0YxjKbRWxjSIQumYJOIjzqJO9xe0CYptfepQdKj2tOFpKbExp8xg QJPyW5nROyCwJdRDDCRZvdQNZ4/gGeL8qE52GS9n+90WpzE809LZ4QMA/7ZMvtA= X-Gm-Gg: ASbGncvw9emlk7QckuOZpCESw7w4uyzxw1DTk/xkSBBX6Mxrz0De1rl1vgrx409hqN1 iG2MllBQ8n1sKf29FGFPsl/2xoRoLDeEIERGhKy34y7y6ofvWfcHZAZbhUtIfptSprIac6Z9nE9 qhr98klFc5ZWHHSSljIXLcBprmyQv1purfa74wxNUuQQuhK6pJPWVUQv+krV8H5vwZhYTI0C+fK afFiw23deTwgeLY/29RMMZTvQv+4BV1fgvu4fzxffpadzAqhIovg2SnN2rN7h+TXz0= X-Google-Smtp-Source: AGHT+IEZWsPMIYjNv7jxdVj3hAXsMC1BoKsd7O3ruCr9H4/wtm3VV0KAm67GFUnre39yZsgN9rJvNA== X-Received: by 2002:a17:907:944d:b0:aa6:84d4:8021 with SMTP id a640c23a62f3a-aac34a07e84mr5339810166b.61.1736242900971; Tue, 07 Jan 2025 01:41:40 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aac0f012229sm2356901166b.133.2025.01.07.01.41.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jan 2025 01:41:40 -0800 (PST) From: Linus Walleij Date: Tue, 07 Jan 2025 10:41:27 +0100 Subject: [PATCH RFC v3 11/30] ARM: entry: Merge the common and trace entry code MIME-Version: 1.0 Message-Id: <20250107-arm-generic-entry-v3-11-4e5f3c15db2d@linaro.org> References: <20250107-arm-generic-entry-v3-0-4e5f3c15db2d@linaro.org> In-Reply-To: <20250107-arm-generic-entry-v3-0-4e5f3c15db2d@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250107_014142_451547_55547B80 X-CRM114-Status: GOOD ( 24.25 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The trace entry code now can handle recursive and complex calls in C using stack. Move the common code over to using that approach. We now use the ret_fast_syscall return path also when tracing, which appears to work just fine. Signed-off-by: Linus Walleij --- arch/arm/include/asm/syscall.h | 3 +-- arch/arm/kernel/entry-common.S | 61 +++++++++--------------------------------- arch/arm/kernel/syscall.c | 35 +++++++++++------------- 3 files changed, 28 insertions(+), 71 deletions(-) diff --git a/arch/arm/include/asm/syscall.h b/arch/arm/include/asm/syscall.h index cb0073c4151bf70a82d15e17a95b6b6f48b245d6..9c664d8c5718f58034f442c20a030b59faf65dd9 100644 --- a/arch/arm/include/asm/syscall.h +++ b/arch/arm/include/asm/syscall.h @@ -19,8 +19,7 @@ extern const unsigned long sys_call_table[]; -int invoke_syscall(void *table, struct pt_regs *regs, int scno, void *retp); -void invoke_syscall_trace(void *table, struct pt_regs *regs); +int invoke_syscall_trace(void *table, struct pt_regs *regs, int scno); static inline int syscall_get_nr(struct task_struct *task, struct pt_regs *regs) diff --git a/arch/arm/kernel/entry-common.S b/arch/arm/kernel/entry-common.S index dbc947d301ec11bb13007cb4bb161c035ede5c10..f0f1f8723965313b287158fb2a238908db72fcd2 100644 --- a/arch/arm/kernel/entry-common.S +++ b/arch/arm/kernel/entry-common.S @@ -39,7 +39,6 @@ saved_pc .req lr * from those features make this path too inefficient. */ ret_fast_syscall: -__ret_fast_syscall: UNWIND(.fnstart ) UNWIND(.cantunwind ) disable_irq_notrace @ disable interrupts @@ -47,13 +46,13 @@ __ret_fast_syscall: movs r1, r1, lsl #16 bne fast_work_pending - restore_user_regs fast = 1, offset = S_OFF + restore_user_regs fast = 0, offset = S_OFF UNWIND(.fnend ) ENDPROC(ret_fast_syscall) /* Ok, we need to do extra processing, enter the slow path. */ fast_work_pending: - str r0, [sp, #S_R0+S_OFF]! @ returned r0 + add sp, sp, #(S_R0 + S_OFF) /* fall through to work_pending */ #else /* @@ -63,10 +62,9 @@ fast_work_pending: * call. */ ret_fast_syscall: -__ret_fast_syscall: UNWIND(.fnstart ) UNWIND(.cantunwind ) - str r0, [sp, #S_R0 + S_OFF]! @ save returned r0 + add sp, sp, #(S_R0 + S_OFF) #if IS_ENABLED(CONFIG_DEBUG_RSEQ) /* do_rseq_syscall needs interrupts enabled. */ mov r0, sp @ 'regs' @@ -83,7 +81,9 @@ ENDPROC(ret_fast_syscall) #endif tst r1, #_TIF_SYSCALL_WORK - bne __sys_trace_return_nosave + beq slow_work_pending + b ret_to_user + slow_work_pending: mov r0, sp @ 'regs' bl do_work_pending @@ -257,19 +257,15 @@ ENTRY(vector_swi) str scno, [tsk, #TI_ABI_SYSCALL] #endif mov r1, sp @ put regs into r1 - ldr r10, [tsk, #TI_FLAGS] @ check for syscall tracing stmdb sp!, {r4, r5} @ push fifth and sixth args - - tst r10, #_TIF_SYSCALL_WORK @ are we tracing syscalls? - bne __sys_trace - mov r0, tbl - /* r1 already contains regs */ mov r2, scno @ syscall number from r7 - /* We return here no matter what, also pass this as an argument */ - badr lr, __ret_fast_syscall - mov r3, lr - b invoke_syscall + bl invoke_syscall_trace + cmp r0, #0 + beq ret_fast_syscall + /* This path taken when tracing */ + add sp, sp, #(S_R0 + S_OFF) + b ret_to_user #if defined(CONFIG_OABI_COMPAT) || !defined(CONFIG_AEABI) /* @@ -289,20 +285,6 @@ ENTRY(vector_swi) ENDPROC(vector_swi) .ltorg - /* - * This is the really slow path. We're going to be doing - * context switches, and waiting for our parent to respond. - */ -__sys_trace: - add r1, sp, #S_R0 + S_OFF @ pointer to regs - mov r0, tbl - bl invoke_syscall_trace - add sp, sp, #S_OFF @ restore stack pointer - b ret_to_user - -__sys_trace_return_nosave: - b ret_to_user - .macro syscall_table_start, sym .equ __sys_nr, 0 .type \sym, #object @@ -402,25 +384,6 @@ sys_mmap2: b sys_mmap_pgoff ENDPROC(sys_mmap2) -/* - * This call wants: - * r0: syscall table - * r1: regs - * r2: syscall number - * r3: pointer to return function - */ -SYM_TYPED_FUNC_START(invoke_syscall_asm) -#ifdef CONFIG_CPU_SPECTRE - csdb -#endif - mov tbl, r0 - mov scno, r2 - mov lr, r3 @ return address - ldmia r1, {r0 - r3} @ reload r0-r3 - /* Arguments 5 and 6 are (hopefully) on the stack */ - ldr pc, [tbl, scno, lsl #2] @ call sys_* routine -SYM_FUNC_END(invoke_syscall_asm) - /* * This call wants: * r0: syscall table diff --git a/arch/arm/kernel/syscall.c b/arch/arm/kernel/syscall.c index e90f04f35b3485968a1e9046cf0889f56cca92e5..36330023b5484399a4ca2fdb5727b410e2c74b6f 100644 --- a/arch/arm/kernel/syscall.c +++ b/arch/arm/kernel/syscall.c @@ -3,50 +3,45 @@ #include #include -int invoke_syscall_asm(void *table, struct pt_regs *regs, int scno, void *retp); -__ADDRESSABLE(invoke_syscall_asm); - -__visible int invoke_syscall(void *table, struct pt_regs *regs, int scno, void *retp) +static inline bool has_syscall_work(unsigned long flags) { - if (scno < NR_syscalls) - /* Doing this with return makes sure the stack gets pop:ed */ - return invoke_syscall_asm(table, regs, scno, retp); - - if (scno >= __ARM_NR_BASE) - return arm_syscall(scno, regs); - - return sys_ni_syscall(); + return unlikely(flags & _TIF_SYSCALL_WORK); } int invoke_syscall_trace_asm(void *table, struct pt_regs *regs, int scno); __ADDRESSABLE(invoke_syscall_trace_asm); -__visible void invoke_syscall_trace(void *table, struct pt_regs *regs) +__visible int invoke_syscall_trace(void *table, struct pt_regs *regs, int scno) { - int scno; + unsigned long flags = read_thread_flags(); int ret; - scno = syscall_trace_enter(regs); - if (scno == -1) - goto trace_exit_nosave; + if (has_syscall_work(flags)) { + scno = syscall_trace_enter(regs); + if (scno == -1) + goto trace_exit_nosave; + } if (scno < NR_syscalls) { ret = invoke_syscall_trace_asm(table, regs, scno); - goto trace_exit_save; + goto exit_save; } if (scno >= __ARM_NR_BASE) { ret = arm_syscall(scno, regs); - goto trace_exit_save; + goto exit_save; } ret = sys_ni_syscall(); -trace_exit_save: +exit_save: /* Save return value from syscall */ regs->ARM_r0 = ret; + if (!has_syscall_work(flags)) + return 0; trace_exit_nosave: local_irq_enable(); syscall_trace_exit(regs); + return 1; } From patchwork Tue Jan 7 09:41:28 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13928595 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id CF3B7E77197 for ; Tue, 7 Jan 2025 09:57:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=SpsIgg1e/3uFwlFgTsWNBO+zJGIkDDm/DpMGSieQyDA=; b=NHr04rIT9O/nBzaFfNpe/tUSVq JZ8hf1Qyejj8gAGNFzUGpm1Xrm0SKaAKpDwjYSCdZC7nCJcUQBT6vWn7WeKzvhUweBT1+d9FnzyuA dLB6ehjKr3+nkZekn90UnGdHWGeDzPabj6o6Uh9Ue3yVsDO4KbI1Y8mkyPXel4fbygDVbE1oBbZkL 3Ul2ZBN1qPcbuEkh8GN1SYmLnTjor4P4IPAMBX+T+WEOlLaWrFI25oLnnTBfzSH1xr9xGeJOiMjW9 ly5pr0z7nGXF6/zN5a0mP9h1TJDJ3QfETHGg8yRc6UREsEthzbw1SRa25B3TV2WcS3a4Pk8A2TZdw mZ8uxVYA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tV6Kh-00000004ESR-1OMA; Tue, 07 Jan 2025 09:57:07 +0000 Received: from mail-ej1-x62b.google.com ([2a00:1450:4864:20::62b]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tV65n-00000004AQO-3hIN for linux-arm-kernel@lists.infradead.org; Tue, 07 Jan 2025 09:41:47 +0000 Received: by mail-ej1-x62b.google.com with SMTP id a640c23a62f3a-aa692211331so2940875766b.1 for ; Tue, 07 Jan 2025 01:41:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1736242902; x=1736847702; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=SpsIgg1e/3uFwlFgTsWNBO+zJGIkDDm/DpMGSieQyDA=; b=GteJnLAXf8aAdcG8RjhbmxSui5m0YIPrACrhiO29VBqJhFd5ZDsejjb8LUTC+Gpl8e nHQQMJBYgoUTV+IbeNJRYniewiY8XB/BisbXRHW8pYNTWzy2XRKDp6U01S8Co5r90/2f Dp0XRN8hX9+NuuYobp4TskGHvryqnJwlfQn8H6Dh6Hl4L+OJPAcOYJzJP0Sqr4OTv0aI xlgSxHf8PtXOJhfQjlN72uIvPKcIn8k4iuEe3z4+AAfsUyt36IAdNHkt5aKu2rEKQcjQ JPvSoWvNDKt6Fgx7R+km2MXn+XVGYtsVlCp9/hzPV4UnrJlHJd/XyRiWLG8t8kVOCoBL rCWQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736242902; x=1736847702; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=SpsIgg1e/3uFwlFgTsWNBO+zJGIkDDm/DpMGSieQyDA=; b=iFVEPsCMVWNCnJqaCEFSbk/BaQDv8ohB3gWNEN9DGqU/pYUGlz7+LqVEAX4gW+0haO x3WskjWF4wMkWHgIlr0i7ZOcSaq8akQR3quEMgxS7pLcLnhXHv3C8QxvRqbhJpQPpJsO CVoEYn87vpUuXkjqprq2R6GRcRc+3/lf3ThDgtO8NvRlFEB2uF0+yXrxDEcYptf4vKp8 AAgCgQK27syz3lsuRJyeGHrW+nVWdfmoQEhH3x3QnwUP330KUOCGh3F3QJJJhbxxHFbA izmh8NKA9zZxRBtSWBzD4PIGVxtS7K6xap3Z+ylRksvBE8L5Lbcpt+u0/kCLpMicBiz0 3a2w== X-Gm-Message-State: AOJu0Ywg+ylwluZKROtpdRWlB6FxjRxCwdxZDBM3+//Cc3j3rugl+Moe tLeu0zZ5rhRHZd6JpqW+HkAli6TIHMd+vjvQMaRpULqbJNMwKSyYyEUzQor9l8yrxZf1cktwgds fI34= X-Gm-Gg: ASbGncvYoh/DIJkx47ZsLZwkDxp2KB5KYJH6XszOwwEZnUciM9UQL+izAXSldSNehkq 9oxl27PXyxG7Kp5p/0LJGKB/dSPf+vfPcN2uRrBsWyKZNGmW8lLnDfpD76GHCnUhPEYRHv3YEqY ZiFlDFZ1B4fgjmkhEDWRgZwXNDnOr3JTyB/aTUDF1FhtkQ98sjK5vYU2xWydzdwYQF7b27QQtSU 4QnnvjXLVSbJ0D+UrCVC3ZU+osjj4hHosCbM0kwrVUud65XnfjZ/0e7P4O+kU75SKc= X-Google-Smtp-Source: AGHT+IFg90XSIzsHeI+IKESBsSFpkxnaQ4jjexpj6W9kCj39JSV8wQCQI8feJQ8/JhMvrEeNMUEQ8A== X-Received: by 2002:a17:907:7ba5:b0:aab:f971:1f73 with SMTP id a640c23a62f3a-ab2918fdce4mr194300366b.22.1736242902078; Tue, 07 Jan 2025 01:41:42 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aac0f012229sm2356901166b.133.2025.01.07.01.41.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jan 2025 01:41:41 -0800 (PST) From: Linus Walleij Date: Tue, 07 Jan 2025 10:41:28 +0100 Subject: [PATCH RFC v3 12/30] ARM: entry: Rename syscall invocation MIME-Version: 1.0 Message-Id: <20250107-arm-generic-entry-v3-12-4e5f3c15db2d@linaro.org> References: <20250107-arm-generic-entry-v3-0-4e5f3c15db2d@linaro.org> In-Reply-To: <20250107-arm-generic-entry-v3-0-4e5f3c15db2d@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250107_014143_923214_BE7F6BCC X-CRM114-Status: GOOD ( 15.76 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Now that we do not have a separate trace syscall invocation path, rename the invocation functions to be generic. This can be squashed into the previous patch, it is just done here for the previous patch to be easier to get a cleaner diff and be easier to read. Signed-off-by: Linus Walleij --- arch/arm/include/asm/syscall.h | 2 +- arch/arm/kernel/entry-common.S | 10 +++++----- arch/arm/kernel/syscall.c | 8 ++++---- 3 files changed, 10 insertions(+), 10 deletions(-) diff --git a/arch/arm/include/asm/syscall.h b/arch/arm/include/asm/syscall.h index 9c664d8c5718f58034f442c20a030b59faf65dd9..b94fd7ce17f82f0baf06b74800245112e1f13002 100644 --- a/arch/arm/include/asm/syscall.h +++ b/arch/arm/include/asm/syscall.h @@ -19,7 +19,7 @@ extern const unsigned long sys_call_table[]; -int invoke_syscall_trace(void *table, struct pt_regs *regs, int scno); +int invoke_syscall(void *table, struct pt_regs *regs, int scno); static inline int syscall_get_nr(struct task_struct *task, struct pt_regs *regs) diff --git a/arch/arm/kernel/entry-common.S b/arch/arm/kernel/entry-common.S index f0f1f8723965313b287158fb2a238908db72fcd2..f1e48002bd3075ea67b5883178583127fa0055c6 100644 --- a/arch/arm/kernel/entry-common.S +++ b/arch/arm/kernel/entry-common.S @@ -260,7 +260,7 @@ ENTRY(vector_swi) stmdb sp!, {r4, r5} @ push fifth and sixth args mov r0, tbl mov r2, scno @ syscall number from r7 - bl invoke_syscall_trace + bl invoke_syscall cmp r0, #0 beq ret_fast_syscall /* This path taken when tracing */ @@ -390,7 +390,7 @@ ENDPROC(sys_mmap2) * r1: regs * r2: syscall number */ -SYM_TYPED_FUNC_START(invoke_syscall_trace_asm) +SYM_TYPED_FUNC_START(invoke_syscall_asm) #ifdef CONFIG_CPU_SPECTRE csdb #endif @@ -400,17 +400,17 @@ SYM_TYPED_FUNC_START(invoke_syscall_trace_asm) /* Make space to copy the two syscall stack arguments */ sub sp, sp, #S_OFF mov scno, r2 - badr lr, __invoke_syscall_trace_ret @ return right here + badr lr, __invoke_syscall_ret @ return right here ldmia r1, {r0 - r6} @ reload r0-r6 from regs stmia sp, {r4, r5} @ copy stack arguments ldr pc, [tbl, scno, lsl #2] @ call sys_* routine -__invoke_syscall_trace_ret: +__invoke_syscall_ret: /* Drop the copied stack arguments */ add sp, sp, #S_OFF pop {r4 - r10, lr} ARM( mov pc, lr ) THUMB( bx lr ) -SYM_FUNC_END(invoke_syscall_trace_asm) +SYM_FUNC_END(invoke_syscall_asm) #ifdef CONFIG_OABI_COMPAT diff --git a/arch/arm/kernel/syscall.c b/arch/arm/kernel/syscall.c index 36330023b5484399a4ca2fdb5727b410e2c74b6f..6f518ec3870fb36a64d1b671b20f96d63a52bf78 100644 --- a/arch/arm/kernel/syscall.c +++ b/arch/arm/kernel/syscall.c @@ -8,10 +8,10 @@ static inline bool has_syscall_work(unsigned long flags) return unlikely(flags & _TIF_SYSCALL_WORK); } -int invoke_syscall_trace_asm(void *table, struct pt_regs *regs, int scno); -__ADDRESSABLE(invoke_syscall_trace_asm); +int invoke_syscall_asm(void *table, struct pt_regs *regs, int scno); +__ADDRESSABLE(invoke_syscall_asm); -__visible int invoke_syscall_trace(void *table, struct pt_regs *regs, int scno) +__visible int invoke_syscall(void *table, struct pt_regs *regs, int scno) { unsigned long flags = read_thread_flags(); int ret; @@ -23,7 +23,7 @@ __visible int invoke_syscall_trace(void *table, struct pt_regs *regs, int scno) } if (scno < NR_syscalls) { - ret = invoke_syscall_trace_asm(table, regs, scno); + ret = invoke_syscall_asm(table, regs, scno); goto exit_save; } From patchwork Tue Jan 7 09:41:29 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13928596 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id EF569E77197 for ; Tue, 7 Jan 2025 09:58:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=ZVho6GNQ8sHkNW9UnhPXJ73x2YlFfkQj2Bi5lbl2ixw=; b=2K/1DMVwkP/VIpEwzSauIGn6ci ms66MqNFn9qbuZITxw4VURZWm+UmPzqDEjXpYa1AoWTOlkrowpV6X4T0Boub3eIQZ1KbKAHizgr0G JlJZwi2AXETjfZMkW6q+fqGzAnrqOtt0jgxqG7SU2Soz1kaw07hYA0RTrQhLG6DABxkydg3d+30pP LRo4lo1EViyb1hA18uW+j9xGlaVlClrlb4Xu28QWoz2UQy6id224dQERBPJHf58aNASjnmM08TLQq cTHlMKcQGlmEBUrHxdhS7fiw8smecebyRv7JeBHrna4oH/na7NiUoYS3ugBk0wRNs3fvUxkrZw99z 1oQZQMrg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tV6Lq-00000004EdY-1Gz6; Tue, 07 Jan 2025 09:58:18 +0000 Received: from mail-ej1-x62d.google.com ([2a00:1450:4864:20::62d]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tV65o-00000004AQp-32Sd for linux-arm-kernel@lists.infradead.org; Tue, 07 Jan 2025 09:41:49 +0000 Received: by mail-ej1-x62d.google.com with SMTP id a640c23a62f3a-aaeec07b705so1750810066b.2 for ; Tue, 07 Jan 2025 01:41:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1736242903; x=1736847703; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=ZVho6GNQ8sHkNW9UnhPXJ73x2YlFfkQj2Bi5lbl2ixw=; b=e/Rjy+VaGJy163M1fUXwiKtVnPfHYBWf4mBlx+BF4bRP/zyjWoBGuIEOrtBenEW/Df 5K9rRLAj46913f4MaBA+774B3LjC1OKay2c1ZRXMLPxyW8cnsrcCIYHa8PcXvcGbBCk/ 8S1nPzKtfT8SR5DVMC8ZiivXQfC0mA2xK06g3MZ2+YPOwm49FCyJqS3Jc0+5lADfbgS1 CkPWcpzj8sKiF9fuEm0qiaG/V/R0ji/lux+cflDMWpLR9UsL7D0e51/D/1NRyWe3zIKy smekNjYlRCssnlfgBY/O6ReLqH4YyLONkJ1SbyfMkJugo8IgIjaVErP0HpqdyC1AbanF 9kRA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736242903; x=1736847703; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ZVho6GNQ8sHkNW9UnhPXJ73x2YlFfkQj2Bi5lbl2ixw=; b=QUBJflDRsE9vOjsS4Dq93fdt60vbOvODgWZFg2XXBzcaPjHnajFDpZgTQmdGjrFiGk q453BRkmmuhmiImqMD5b1lRUfA2J2FdFMlK9AXb5xAVw97sF0Si5N3kutFzh1n2pGBTQ D9wDBGdPaYhh2rPSg2w08oEmE/7mWeb1RxcF6t3454SZqI5N4Zi1jJy0vC+6Z/LKzRR8 qUjZJUywFLsnVJzVTVM12Nv+LvML0dyIVeuBLCvGOD2CTmnnCY6Y0UDtra9Nw3T0JFhb bzAbyv3+aj3F3fVfFvvppA7mIAtmFhMBPtqrOlkIZYm6dvXXayrU5rvcjlkkLScvmzyh ivtg== X-Gm-Message-State: AOJu0YyOUIY5rGpxAGKgLQ97vs5BFjYBwsx4cEpfhXBd+hEOygjj8lj3 HmWKmQ06JbBMjwEl2mhbpVxaQIbFMxj46xEZXcvLYiQkM+P7HUODzj57eh6ODbg= X-Gm-Gg: ASbGnct8WgUsyw1C62j2sE+TypqJ7Xlaq+kNuHGYJUVQyOzBu54Mdxjv6HY/sBaJhD+ ei0z/hXMHOXazAlA4h0fDfNYlg1LvCgYYQetRzmEqc3L4Q8gvIIrql8yPA0d7vbugiX4Na0aQZ5 IAKHhXZoVxbShcpq9Kbho7/p5k3r6xgnu2AO4JDY/bU7zHH8PJtghobkWOIJeoT93Z4/ucRDtQx jbNXrCmn3vqTSHnRRgqq2xG9iRBI4vhDvmSOCGTAyisj1zeZRDWRjVUNrMnuOTpS3g= X-Google-Smtp-Source: AGHT+IFp0Na7duITz2atjHhj0nkqhhPB21aJ1mPJXyOxSwoqi11/ZptmlR7E7P65WAgLbQ1dpAuC3g== X-Received: by 2002:a17:906:c142:b0:aa6:87e8:1d08 with SMTP id a640c23a62f3a-aac2703377cmr4389365066b.8.1736242903330; Tue, 07 Jan 2025 01:41:43 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aac0f012229sm2356901166b.133.2025.01.07.01.41.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jan 2025 01:41:42 -0800 (PST) From: Linus Walleij Date: Tue, 07 Jan 2025 10:41:29 +0100 Subject: [PATCH RFC v3 13/30] ARM: entry: Create user_mode_enter/exit MIME-Version: 1.0 Message-Id: <20250107-arm-generic-entry-v3-13-4e5f3c15db2d@linaro.org> References: <20250107-arm-generic-entry-v3-0-4e5f3c15db2d@linaro.org> In-Reply-To: <20250107-arm-generic-entry-v3-0-4e5f3c15db2d@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250107_014144_789082_42B01DB2 X-CRM114-Status: GOOD ( 18.95 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The callbacks to the context tracking will be repurposed for several uses that are needed on the IRQ transition to/from userspace. Rename the macro, establish call sites in C calling into the context tracking following the corresponding generic entry function prototypes, despite the assembly macro names become a bit long this makes it clear to readers exactly what is going on and where this call will go. Drop the ifdefs pertaining to context tracking from the macro. The C calls we will use have stubs that will compile these out anyway. The inversion of the signature of the context tracking calls are especially confusing since the generic entry uses the reverse semantics: *enter from* user mode (to kernel mode) and *exit to* user mode (from kernel mode) instead of the other way around as the old context tracker code user_exit_callable() and user_enter_callable() which have inverted semantics. Signed-off-by: Linus Walleij --- arch/arm/include/asm/entry.h | 14 ++++++++++++++ arch/arm/kernel/Makefile | 2 +- arch/arm/kernel/entry-armv.S | 2 +- arch/arm/kernel/entry-common.S | 4 ++-- arch/arm/kernel/entry-header.S | 24 ++++++++++++------------ arch/arm/kernel/entry.c | 15 +++++++++++++++ 6 files changed, 45 insertions(+), 16 deletions(-) diff --git a/arch/arm/include/asm/entry.h b/arch/arm/include/asm/entry.h new file mode 100644 index 0000000000000000000000000000000000000000..e26f369375ca3cf762f92fb499657a666b223ca2 --- /dev/null +++ b/arch/arm/include/asm/entry.h @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +#ifndef __ASM_ENTRY_H__ +#define __ASM_ENTRY_H__ + +struct pt_regs; + +/* + * These are copies of generic entry headers so we can transition + * to generic entry once they are semantically equivalent. + */ +void irqentry_enter_from_user_mode(struct pt_regs *regs); +void irqentry_exit_to_user_mode(struct pt_regs *regs); + +#endif /* __ASM_ENTRY_H__ */ diff --git a/arch/arm/kernel/Makefile b/arch/arm/kernel/Makefile index d5a128a4228dbdf5825aca7385deb94b473716a8..9a6ac09741109367f1bc30ab7f556ea35b30e0f4 100644 --- a/arch/arm/kernel/Makefile +++ b/arch/arm/kernel/Makefile @@ -17,7 +17,7 @@ CFLAGS_REMOVE_return_address.o = -pg # Object file lists. -obj-y := elf.o entry-common.o irq.o opcodes.o \ +obj-y := elf.o entry.o entry-common.o irq.o opcodes.o \ process.o ptrace.o reboot.o io.o \ setup.o signal.o sigreturn_codes.o \ stacktrace.o sys_arm.o time.o traps.o \ diff --git a/arch/arm/kernel/entry-armv.S b/arch/arm/kernel/entry-armv.S index ef6a657c8d130663e68acd07226815ce8a3fff7d..93a37437a9520118eff411f009e8f6bd729412fc 100644 --- a/arch/arm/kernel/entry-armv.S +++ b/arch/arm/kernel/entry-armv.S @@ -406,7 +406,7 @@ ENDPROC(__fiq_abt) #ifdef CONFIG_TRACE_IRQFLAGS bl trace_hardirqs_off #endif - ct_user_exit save = 0 + asm_irqentry_enter_from_user_mode save = 0 .endif .endm diff --git a/arch/arm/kernel/entry-common.S b/arch/arm/kernel/entry-common.S index f1e48002bd3075ea67b5883178583127fa0055c6..ff1dd3169346f3770cad6b7e218f5d74ffc646fe 100644 --- a/arch/arm/kernel/entry-common.S +++ b/arch/arm/kernel/entry-common.S @@ -111,7 +111,7 @@ ENTRY(ret_to_user_from_irq) no_work_pending: asm_trace_hardirqs_on save = 0 - ct_user_enter save = 0 + asm_irqentry_exit_to_user_mode save = 0 #ifdef CONFIG_GCC_PLUGIN_STACKLEAK bl stackleak_erase_on_task_stack @@ -191,7 +191,7 @@ ENTRY(vector_swi) alignment_trap r10, ip, cr_alignment asm_trace_hardirqs_on save=0 enable_irq_notrace - ct_user_exit save=0 + asm_irqentry_enter_from_user_mode save = 0 /* * Get the system call number. diff --git a/arch/arm/kernel/entry-header.S b/arch/arm/kernel/entry-header.S index 52b4fa97226dbfa4b55aca8eaf74ae24e1e220f8..fb5bb019199b2871e29e306a29bea8fdf47dd7f3 100644 --- a/arch/arm/kernel/entry-header.S +++ b/arch/arm/kernel/entry-header.S @@ -362,31 +362,31 @@ ALT_UP_B(.L1_\@) .endm /* - * Context tracking subsystem. Used to instrument transitions + * Context tracking and other mode transitions. Used to instrument transitions * between user and kernel mode. - */ - .macro ct_user_exit, save = 1 -#ifdef CONFIG_CONTEXT_TRACKING_USER +*/ + .macro asm_irqentry_enter_from_user_mode, save = 1 .if \save stmdb sp!, {r0-r3, ip, lr} - bl user_exit_callable + mov r0, sp @ regs + bl irqentry_enter_from_user_mode ldmia sp!, {r0-r3, ip, lr} .else - bl user_exit_callable + mov r0, sp @ regs + bl irqentry_enter_from_user_mode .endif -#endif .endm - .macro ct_user_enter, save = 1 -#ifdef CONFIG_CONTEXT_TRACKING_USER + .macro asm_irqentry_exit_to_user_mode, save = 1 .if \save stmdb sp!, {r0-r3, ip, lr} - bl user_enter_callable + mov r0, sp @ regs + bl irqentry_exit_to_user_mode ldmia sp!, {r0-r3, ip, lr} .else - bl user_enter_callable + mov r0, sp @ regs + bl irqentry_exit_to_user_mode .endif -#endif .endm /* diff --git a/arch/arm/kernel/entry.c b/arch/arm/kernel/entry.c new file mode 100644 index 0000000000000000000000000000000000000000..8b2e8ea66c1376759d6c0c14aad8728895b3ff1e --- /dev/null +++ b/arch/arm/kernel/entry.c @@ -0,0 +1,15 @@ +// SPDX-License-Identifier: GPL-2.0 +#include +#include + +noinstr void irqentry_enter_from_user_mode(struct pt_regs *regs) +{ + /* This context tracking call has inverse naming */ + user_exit_callable(); +} + +noinstr void irqentry_exit_to_user_mode(struct pt_regs *regs) +{ + /* This context tracking call has inverse naming */ + user_enter_callable(); +} From patchwork Tue Jan 7 09:41:30 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13928599 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 997FCE77197 for ; Tue, 7 Jan 2025 09:59:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=v829giFYonEyQ9WTV+P3dJRTtjfI/rZhUUQIvCJakF8=; b=nuxLRe8oAeik8xAsJBd4EHtTn4 gR2FPOA0paJtY3k2dyi/cQVgT8gyVV+Cnlc87fRAa6S90Qs9kMauzvTmINw6NRglM5AnGooffmoi3 kpaJCLZTMmewcoFSRXbrSUBQpi6/gFG2wRYhWWzr8ETPsc2JKr/9F4ULbCQm0LqmNShNwApJPn2TH u4AP/wCW2YBaJPfBevcv9vVCii16Na3AHI1a91+2BIEqA0LAn48wL1VKF5ZPthTyW0xLMmaDJ2cBS hboDYVrgUDaZLm0kIFTy4yZ2HcytLH0Rs/nUv5rW6NZ6yIN4SB1t0gshzE3KGVkWT4TV3WqHhywfN qCrt668g==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tV6Mz-00000004Epu-4BrF; Tue, 07 Jan 2025 09:59:30 +0000 Received: from mail-ej1-x62f.google.com ([2a00:1450:4864:20::62f]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tV65q-00000004ARv-1rXT for linux-arm-kernel@lists.infradead.org; Tue, 07 Jan 2025 09:41:50 +0000 Received: by mail-ej1-x62f.google.com with SMTP id a640c23a62f3a-aa68b513abcso2839245766b.0 for ; Tue, 07 Jan 2025 01:41:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1736242905; x=1736847705; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=v829giFYonEyQ9WTV+P3dJRTtjfI/rZhUUQIvCJakF8=; b=jzxheBUoLMBdrX/CfXfs/R+HAp8HQ0+ZAye6WS5/TeudDv0casZPCZNpl2CffL1M+H c/ba/x9WzGfbHAWKf4Dl1W7yLIM++4HQZ9OVnb5xBNTK9wugcqQCguyFxLnr8DOH8C+M H45BpW9QfvduUVlYdwVlHfoRgjVqKEdwXrKxrcEpIXzS4XqaWWHs5FGwQsKoeu7/gHab fMcVL1HV+gKiiDBww/8XPES266jjN8Sn/J5+TpNN+icZmH9M+/WchLMpxMgjzBzBN/k0 UbWspwAoKZ8LGChqMjRO4LPBkiP64y8EpfdwMXcoyP8lHvDQlrpdgomrpeGvokuTmdDk bFqw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736242905; x=1736847705; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=v829giFYonEyQ9WTV+P3dJRTtjfI/rZhUUQIvCJakF8=; b=Zj/RW8WXvx+lowYffs9bavPv6WBBMGTGfMyTPK1kot3GmcQxIy9PBEOBbJFMIR6lWM vHIEjPlPbJV9RgfoXk5XT9tYyIVsUPvgqJEW4+wHH6Y/kxQ8uZdbXWtdEoO6FluszFeZ xw+Y3uCoi8zMa3N4e5qOnsYpdNomD6b8S8JLlbQz6ep6UB9AxVyp47IhCgd3tOkc8TVD 3chqNG5GA/nbXTyfOcZobCibZ7Z4gTyDVV85fOZ9D+krt8si2Ldf7i+YA8NF5b/npcDz NXC6cdp8brQhA0rOSVTJXpLgWSX1kAPtldOoDpDaYjqqf2MweeiVUZ0HdDSX1ywcrTQY Brlw== X-Gm-Message-State: AOJu0YwzAtMrwLAMsbl5LyjVmIFi8hoJ49DI8DZAibjPSNBvlHVBz97G pa7XFQ5PHUNFRvFzkFVnH0A8WJX2elVTCwIKjj84EmG6I3O/FX8T1Fq2DcEplEQ= X-Gm-Gg: ASbGncu1McMZ0jXoQYTxhjSs4kuQObbLdWRQPtvukciILZMVPhw1nn3XAfs8DBQpd24 TW0uBowYqlTWkBDWIN6aUloBCtCuj9qEIHrk385Ik8dOuzSdlyfmqbCs58UeXFB0kNDiQmC6Wpp 7zWYFrUsUrFstFZ+Ico7Rz4QtwFbaZ1+B5/f93E0Sht4yOKPoZ0Ty7lsPsag+lQNOYSYHiRPo+2 cYSI14T3kIK3vKeWM2ZhPifdZuUDG5S2iMVi6ji8WweiuR/9Aph0GI/yEQQmBLuRlM= X-Google-Smtp-Source: AGHT+IGxL2uvreJSYgRQWypM4h6fpX3X0ht6mED44IiWbS3kunWbdejn+a0zmCmtQsTsupFQ/PTArA== X-Received: by 2002:a17:907:1a4e:b0:aac:4324:977e with SMTP id a640c23a62f3a-aac432497bamr4897636766b.27.1736242904900; Tue, 07 Jan 2025 01:41:44 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aac0f012229sm2356901166b.133.2025.01.07.01.41.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jan 2025 01:41:44 -0800 (PST) From: Linus Walleij Date: Tue, 07 Jan 2025 10:41:30 +0100 Subject: [PATCH RFC v3 14/30] ARM: entry: Drop trace argument from usr_entry macro MIME-Version: 1.0 Message-Id: <20250107-arm-generic-entry-v3-14-4e5f3c15db2d@linaro.org> References: <20250107-arm-generic-entry-v3-0-4e5f3c15db2d@linaro.org> In-Reply-To: <20250107-arm-generic-entry-v3-0-4e5f3c15db2d@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250107_014146_488249_797AAF23 X-CRM114-Status: GOOD ( 12.61 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The trace argument select whether to look into tracing of the hardirqs for the exceptions from userspace, but also selects whether to call further down to establish context for RCU. Split off a second macro that is called on everything except the FIQ since this is the only outlier. Signed-off-by: Linus Walleij --- arch/arm/kernel/entry-armv.S | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/arch/arm/kernel/entry-armv.S b/arch/arm/kernel/entry-armv.S index 93a37437a9520118eff411f009e8f6bd729412fc..c71110126fc105fc6ac2d6cb0f5f399b4c8b1548 100644 --- a/arch/arm/kernel/entry-armv.S +++ b/arch/arm/kernel/entry-armv.S @@ -357,7 +357,7 @@ ENDPROC(__fiq_abt) #error "sizeof(struct pt_regs) must be a multiple of 8" #endif - .macro usr_entry, trace=1, uaccess=1 + .macro usr_entry, uaccess=1 UNWIND(.fnstart ) UNWIND(.cantunwind ) @ don't unwind the user space sub sp, sp, #PT_REGS_SIZE @@ -401,13 +401,14 @@ ENDPROC(__fiq_abt) @ Clear FP to mark the first stack frame @ zero_fp + .endm - .if \trace + /* Called after usr_entry for everything except FIQ */ + .macro usr_entry_enter #ifdef CONFIG_TRACE_IRQFLAGS bl trace_hardirqs_off #endif asm_irqentry_enter_from_user_mode save = 0 - .endif .endm .macro kuser_cmpxchg_check @@ -429,6 +430,7 @@ ENDPROC(__fiq_abt) .align 5 __dabt_usr: usr_entry uaccess=0 + usr_entry_enter kuser_cmpxchg_check mov r2, sp dabt_helper @@ -439,6 +441,7 @@ ENDPROC(__dabt_usr) .align 5 __irq_usr: usr_entry + usr_entry_enter kuser_cmpxchg_check irq_handler from_user=1 get_thread_info tsk @@ -452,6 +455,7 @@ ENDPROC(__irq_usr) .align 5 __und_usr: usr_entry uaccess=0 + usr_entry_enter @ IRQs must be enabled before attempting to read the instruction from @ user space since that could cause a page/translation fault if the @@ -476,6 +480,7 @@ ENDPROC(__und_usr) .align 5 __pabt_usr: usr_entry + usr_entry_enter mov r2, sp @ regs pabt_helper UNWIND(.fnend ) @@ -495,7 +500,7 @@ ENDPROC(ret_from_exception) .align 5 __fiq_usr: - usr_entry trace=0 + usr_entry kuser_cmpxchg_check mov r0, sp @ struct pt_regs *regs bl handle_fiq_as_nmi From patchwork Tue Jan 7 09:41:31 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13928600 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 93D1AE77197 for ; Tue, 7 Jan 2025 10:00:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=S/ahkugHBECRDbWd82a9voFYhr82LbMuPsF4+BSN3Ww=; b=Dn9+u+pHe9Iu54Ps0QLZ/xU88s e93fRrhAGfBmJ2KokMIFZRup1ecd7Ul45mWRzaCy/57gfmLpfmCwVz7dv1HMPZRWlXHX851u32H98 TNvvUouRPIIWDsdNSy7X+37aut5U/ti4wcdQq7j26PTzjt24UySb9aByysWx09QLGLOjIoKkPDSGs S7DeJpT/qZVkDZwXEwztWqQejj/wHULAaxHcK5gtAukeGMrxRKwNbSFCaT0aY7Ev52Ag470Zhb80S NimshuTsVqn+PHELg4oDQYHEBQx8pHNXSJOUCWmEn6MC7cS2BWxFWtLdlinQt0iSCaCwJMdBoMuAs 6YDa7lAg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tV6O9-00000004FPQ-2dA9; Tue, 07 Jan 2025 10:00:41 +0000 Received: from mail-ej1-x62a.google.com ([2a00:1450:4864:20::62a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tV65r-00000004ASj-41pQ for linux-arm-kernel@lists.infradead.org; Tue, 07 Jan 2025 09:41:52 +0000 Received: by mail-ej1-x62a.google.com with SMTP id a640c23a62f3a-aaee0b309adso1866409766b.3 for ; Tue, 07 Jan 2025 01:41:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1736242906; x=1736847706; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=S/ahkugHBECRDbWd82a9voFYhr82LbMuPsF4+BSN3Ww=; b=U1pXYlj6X5uH6aNECQLpwAUCcrSXPmmKe9o+HeifAv4EiMPe5cvU8+NYzPM6TTM1uv lLxNXwEdUBVjD9gJlhe011oydPtwXyl4+9am716+5a2pRlX7b/QM6rVmlpsOEtrHQpR/ G8mkM62DbCCqxkmqc0VdaX7+U+gnjmG2sk6ZDxlkJ5GU63wT3mJ/sRSUPcoHemHt6fQs 0psGgdpx0xeJsPp7Nm1ydHT1OV6WXG0L0+yAdidV15bim1R7XYVwlUVeNsQ6izza/Rui rY8v6fQMP7nmhWQPteExPZhSR/j4SlEn3u3ud/G1Sa1pgczNRjDyyp2adtvdbqvzRiYO brpA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736242906; x=1736847706; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=S/ahkugHBECRDbWd82a9voFYhr82LbMuPsF4+BSN3Ww=; b=WdVPcJ6ZHZAq7GL+EgrJPJL1CjbWJgcrNbJBLPICCfgb/3KJ/j3VVSAv7AaOlGMtiX S9Ew/tNuu27n8K+WIhCPuvUGYldvtB0HsoSrKwQ4K/d4C7zVNviq/oPIVHZVIuSNUY1J 5QBsHix+S0qu3G0dD7HyHZ2tzg+Gz+r5xapRLZghTXgj0BpvajZtgwvUi7wtC+3UoHj9 iuWRC/Fan5g5hw4LNomohdkCpQADk+kKkdEBIJ+vOkTXZBTiEPQcIhoqxcpW2ksZIaNz v2UYHiZSyo2kZPSQdtP+2PK7pl0lmevk2u2sCCCOMJdPPZY7S0KH4bF+t7gQ3cvorZVr 7XaQ== X-Gm-Message-State: AOJu0YybRjmzMi4a/vV6dDBYRQ8+Eu1AeNCrQrOmZGnrYDvjTku/d1Ft tNkdWmq2AjrHwUNlyOf6pIsaurn5owHLm6FB2Q/PrkThF1Hqx7yWH39k3QByjhI= X-Gm-Gg: ASbGncsx0bLkc68BFjnsrKBDvHYRbGZi4xh67Oawn68j6D4uUwYnI8pBY+Rm8BOOxcR 0RYEht2d5ixfn/lUisG9fYKzkY0HuDmO1ZzfTXhpCJ/VB6TizuMZkiwYeX4BPQH4ic8RkWL2q1m kUQLxTVTVI/5H6KoMLrzP7M/e+6htP9KdkL5+S8o6JhSEoHQKv0cCsOxinTkYH50dIrWl3WRQ8v gxtN8OH22JV7tbsm1337+kpKVH360EbaHmSmLbbufcjkSuXJVpFwqsFTTHX8leRMGc= X-Google-Smtp-Source: AGHT+IEu9Gn2Ne7+p3HQxWiL1/TBDYuJ8oWa7mIMc7wjdf15+fO7YiQn1jDslGCNb0rAv0RMub70FQ== X-Received: by 2002:a17:907:360e:b0:aa6:8d51:8fdc with SMTP id a640c23a62f3a-aac348c4e1bmr6296191466b.58.1736242906566; Tue, 07 Jan 2025 01:41:46 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aac0f012229sm2356901166b.133.2025.01.07.01.41.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jan 2025 01:41:45 -0800 (PST) From: Linus Walleij Date: Tue, 07 Jan 2025 10:41:31 +0100 Subject: [PATCH RFC v3 15/30] ARM: entry: Separate call path for syscall SWI entry MIME-Version: 1.0 Message-Id: <20250107-arm-generic-entry-v3-15-4e5f3c15db2d@linaro.org> References: <20250107-arm-generic-entry-v3-0-4e5f3c15db2d@linaro.org> In-Reply-To: <20250107-arm-generic-entry-v3-0-4e5f3c15db2d@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250107_014148_006037_A2B0FC0A X-CRM114-Status: GOOD ( 18.73 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The syscalls (SWIs, software interrupts) are deviating from how any other interrupts are handled as they enable the IRQs again while processing the syscall, while "hard" IRQs disable all interrupts until they are handled. Break out syscall_enter_from_user_mode() into its own function and call it instead of irqentry_enter_from_user_mode(). As we are moving toward generic entry, we use the signature from the generic function. As the generic function requires the syscall number to be determined, we move the call down below the code that figures out the syscall number, the only practical effect should be that interrupts are re-enabled a few instructions later. As we move the trace_hardirqs_on/off() calls into C, we can just get rid of the helper macro usr_entry_enter again and call asm_irqentry_enter_from_user_mode directly. Signed-off-by: Linus Walleij --- arch/arm/include/asm/entry.h | 1 + arch/arm/kernel/entry-armv.S | 16 ++++------------ arch/arm/kernel/entry-common.S | 18 +++++++++++++----- arch/arm/kernel/entry.c | 14 ++++++++++++++ 4 files changed, 32 insertions(+), 17 deletions(-) diff --git a/arch/arm/include/asm/entry.h b/arch/arm/include/asm/entry.h index e26f369375ca3cf762f92fb499657a666b223ca2..e259b074caef75c7f777b18199623f07bebee5b4 100644 --- a/arch/arm/include/asm/entry.h +++ b/arch/arm/include/asm/entry.h @@ -8,6 +8,7 @@ struct pt_regs; * These are copies of generic entry headers so we can transition * to generic entry once they are semantically equivalent. */ +long syscall_enter_from_user_mode(struct pt_regs *regs, long); void irqentry_enter_from_user_mode(struct pt_regs *regs); void irqentry_exit_to_user_mode(struct pt_regs *regs); diff --git a/arch/arm/kernel/entry-armv.S b/arch/arm/kernel/entry-armv.S index c71110126fc105fc6ac2d6cb0f5f399b4c8b1548..6edf362ab1e1035dafebf6fb7c55db71462c1eae 100644 --- a/arch/arm/kernel/entry-armv.S +++ b/arch/arm/kernel/entry-armv.S @@ -403,14 +403,6 @@ ENDPROC(__fiq_abt) zero_fp .endm - /* Called after usr_entry for everything except FIQ */ - .macro usr_entry_enter -#ifdef CONFIG_TRACE_IRQFLAGS - bl trace_hardirqs_off -#endif - asm_irqentry_enter_from_user_mode save = 0 - .endm - .macro kuser_cmpxchg_check #if !defined(CONFIG_CPU_32v6K) && defined(CONFIG_KUSER_HELPERS) #ifndef CONFIG_MMU @@ -430,7 +422,7 @@ ENDPROC(__fiq_abt) .align 5 __dabt_usr: usr_entry uaccess=0 - usr_entry_enter + asm_irqentry_enter_from_user_mode save = 0 kuser_cmpxchg_check mov r2, sp dabt_helper @@ -441,7 +433,7 @@ ENDPROC(__dabt_usr) .align 5 __irq_usr: usr_entry - usr_entry_enter + asm_irqentry_enter_from_user_mode save = 0 kuser_cmpxchg_check irq_handler from_user=1 get_thread_info tsk @@ -455,7 +447,7 @@ ENDPROC(__irq_usr) .align 5 __und_usr: usr_entry uaccess=0 - usr_entry_enter + asm_irqentry_enter_from_user_mode save = 0 @ IRQs must be enabled before attempting to read the instruction from @ user space since that could cause a page/translation fault if the @@ -480,7 +472,7 @@ ENDPROC(__und_usr) .align 5 __pabt_usr: usr_entry - usr_entry_enter + asm_irqentry_enter_from_user_mode save = 0 mov r2, sp @ regs pabt_helper UNWIND(.fnend ) diff --git a/arch/arm/kernel/entry-common.S b/arch/arm/kernel/entry-common.S index ff1dd3169346f3770cad6b7e218f5d74ffc646fe..14b2495cae3c2f95b0dfecd849b4e16ec143dbe9 100644 --- a/arch/arm/kernel/entry-common.S +++ b/arch/arm/kernel/entry-common.S @@ -109,8 +109,6 @@ ENTRY(ret_to_user_from_irq) movs r1, r1, lsl #16 bne slow_work_pending no_work_pending: - asm_trace_hardirqs_on save = 0 - asm_irqentry_exit_to_user_mode save = 0 #ifdef CONFIG_GCC_PLUGIN_STACKLEAK @@ -189,9 +187,6 @@ ENTRY(vector_swi) reload_current r10, ip zero_fp alignment_trap r10, ip, cr_alignment - asm_trace_hardirqs_on save=0 - enable_irq_notrace - asm_irqentry_enter_from_user_mode save = 0 /* * Get the system call number. @@ -256,6 +251,19 @@ ENTRY(vector_swi) #else str scno, [tsk, #TI_ABI_SYSCALL] #endif + + /* + * Calling out to C to be careful to save and restore registers. + * This call could modify the syscall number. scno is r7 so we + * do not save and restore r7. + */ + mov r0, sp @ regs + mov r1, scno + push {r4 - r6, r8 - r10, lr} + bl syscall_enter_from_user_mode + pop {r4 - r6, r8 - r10, lr} + mov scno, r0 + mov r1, sp @ put regs into r1 stmdb sp!, {r4, r5} @ push fifth and sixth args mov r0, tbl diff --git a/arch/arm/kernel/entry.c b/arch/arm/kernel/entry.c index 8b2e8ea66c1376759d6c0c14aad8728895b3ff1e..1973947c7ad753fccd694b3ef334fba1326f58b6 100644 --- a/arch/arm/kernel/entry.c +++ b/arch/arm/kernel/entry.c @@ -1,15 +1,29 @@ // SPDX-License-Identifier: GPL-2.0 #include #include +#include + +long syscall_enter_from_user_mode(struct pt_regs *regs, long syscall) +{ + trace_hardirqs_on(); + local_irq_enable(); + /* This context tracking call has inverse naming */ + user_exit_callable(); + + /* This will optionally be modified later */ + return syscall; +} noinstr void irqentry_enter_from_user_mode(struct pt_regs *regs) { + trace_hardirqs_off(); /* This context tracking call has inverse naming */ user_exit_callable(); } noinstr void irqentry_exit_to_user_mode(struct pt_regs *regs) { + trace_hardirqs_on(); /* This context tracking call has inverse naming */ user_enter_callable(); } From patchwork Tue Jan 7 09:41:32 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13928601 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 07F96E77197 for ; Tue, 7 Jan 2025 10:02:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=0jhFjCJGceq+XYGHJk3Ezf5jn0OXzICEZi9KxOGra7k=; b=V2/ZfEJmDzHQ5zGhlmnmzJWmoI do33oQWRvuLRNDURuqprtuFGkiwJzvL0MN5Z8/BNOzo6/27l/db+bkH5cdhdpMAkBGK9Czp8avpQs x+rDeK58w2bQntYCnl1Abnj3DZHRawxQtU22gJBGOSGAJ3mguELqqIZGiQzaS7VTL4EkQJVMowXe5 ZaNnjM2aZ1KpXmgAeAwjS2w7o8XIpcaaGeiW2K/66lr8PoLK2K9MAxJNTSHQ32Zb4RaWRhEzoqglB W2sJwOajcOaiRoP07K//keshPnawYVq0uGd/ZuPhtTYwTf8btoxasJo6RyoIB3L76SUDCN+bon0jR bx0glqJQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tV6PJ-00000004Fc3-113u; Tue, 07 Jan 2025 10:01:53 +0000 Received: from mail-ed1-x533.google.com ([2a00:1450:4864:20::533]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tV65t-00000004ATj-1vZE for linux-arm-kernel@lists.infradead.org; Tue, 07 Jan 2025 09:41:54 +0000 Received: by mail-ed1-x533.google.com with SMTP id 4fb4d7f45d1cf-5d3e9f60bf4so27459999a12.3 for ; Tue, 07 Jan 2025 01:41:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1736242908; x=1736847708; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=0jhFjCJGceq+XYGHJk3Ezf5jn0OXzICEZi9KxOGra7k=; b=psFKgz7/BEyZKucHMGnfBILVDQHmtnIU7sgfAunQ9nIJPwvFGFnpva/4o9UFZzWgrd f2CFqMXtPN4xkjLEIILRz8l7XDVUFYBRcHXHhz3OfVugCEugzgb7e1SE+Q55kKcQLZNh DjLT63MROPSaTzi5Fq5kcY6wHfv2SeNOfvEpnIKgCdmAEw2j+6ZEC5e74LiYfgCMA9yn SNWVXb8WpYAlet0tCa5hGb6gm9wc632kCCw+fwOS6Ia5faVQd0jd6wx1wnxcI1asPuLX kYInv64wqOin6ji8S39k/iBQ7FurF0Ymh5caw3dSXt2RKf/HMxeA2/c3uDlbFoQALe6h f8bg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736242908; x=1736847708; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=0jhFjCJGceq+XYGHJk3Ezf5jn0OXzICEZi9KxOGra7k=; b=okCY5eoEba3GFClZmc626FI3aY+ma3/5+Dw+un9IerM8wCvos4W3eFEi18C/VdMYSg JBQDYI00sLGcim5K2yPESPQs9Tv5pAPidVkRw0p0q16lLKyAy2fjh8SN3zWmOqbbArPK TRT4UF1R/rs7RQ+6HiVYrgwHw+x8tzT3Fulk49WQwGpPxKBWRUvK196rICaF+CJOiEJJ VTBTRKU44Id0jtEI2T+fHALrwTk9lve5yCKHFLKcAiaJfYhx1VNwOXc5WLVqFaw4L0IA ZDPF73mz3bgFUISgeobxDH3sUu5omMt7PoBYYFDgf9GM2wR7vuoiajhw/kmwb8hdrgJJ F8hg== X-Gm-Message-State: AOJu0YxxP+Av5HzjCFORjEmY2zDn9d/JSmLj5XXLu8MvDWLrczPuQ1/6 yWANIkMgT0+HN+nPsolPLYzrLp2H1RF8eEbooxPQrvndgy7uSdBvTqO0bkWDDsA= X-Gm-Gg: ASbGncumbjHaOYNdCDd84ZQYEV0tjL+OfYM4alQq0rkbLrcOdH3T8eHKDkS+3vQ65uU Ynq52c2rv+WaBWdgZ1uXGcEYj6wNNO/HTn7oMyRMq9r6fCTv7CM8Y1Olr3KETJ2CX0wFk+NoU6/ aL45byqpOsww5vmb51sKZfHgZj5JbrvQaybvDEWqRZpKqXyhIMkpVEskH66PfwZ0/NL7kNkXx+R Z53eRLFDwfc0HiTliBkGOG9OsZ29jDXQP3xaASklvuNxtc3sy04ZgnYIJEHhS5ggW4= X-Google-Smtp-Source: AGHT+IHEH6FJmLL0RIoAIXjSeV1vLr7n0yRDVVeIb2gZ9Zszi7jKVQgNpS1AgWJqppfe+0vNlaJ+NA== X-Received: by 2002:a17:907:a708:b0:aae:8495:e064 with SMTP id a640c23a62f3a-aae8495e124mr5041199166b.40.1736242907918; Tue, 07 Jan 2025 01:41:47 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aac0f012229sm2356901166b.133.2025.01.07.01.41.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jan 2025 01:41:47 -0800 (PST) From: Linus Walleij Date: Tue, 07 Jan 2025 10:41:32 +0100 Subject: [PATCH RFC v3 16/30] ARM: entry: Drop argument to asm_irqentry macros MIME-Version: 1.0 Message-Id: <20250107-arm-generic-entry-v3-16-4e5f3c15db2d@linaro.org> References: <20250107-arm-generic-entry-v3-0-4e5f3c15db2d@linaro.org> In-Reply-To: <20250107-arm-generic-entry-v3-0-4e5f3c15db2d@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250107_014149_723117_4EDE0661 X-CRM114-Status: GOOD ( 13.44 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org asm_irqentry_enter_from_user_mode and asm_irqentry_exit_to_user_mode have a "save" argument that will save and restore registers before the call to the C function. Now all invocations set this argument to 0 so drop the surplus code. Signed-off-by: Linus Walleij --- arch/arm/kernel/entry-armv.S | 8 ++++---- arch/arm/kernel/entry-common.S | 2 +- arch/arm/kernel/entry-header.S | 18 ++---------------- 3 files changed, 7 insertions(+), 21 deletions(-) diff --git a/arch/arm/kernel/entry-armv.S b/arch/arm/kernel/entry-armv.S index 6edf362ab1e1035dafebf6fb7c55db71462c1eae..460aa92c3c1f50de905550acf363c58f509bfe0a 100644 --- a/arch/arm/kernel/entry-armv.S +++ b/arch/arm/kernel/entry-armv.S @@ -422,7 +422,7 @@ ENDPROC(__fiq_abt) .align 5 __dabt_usr: usr_entry uaccess=0 - asm_irqentry_enter_from_user_mode save = 0 + asm_irqentry_enter_from_user_mode kuser_cmpxchg_check mov r2, sp dabt_helper @@ -433,7 +433,7 @@ ENDPROC(__dabt_usr) .align 5 __irq_usr: usr_entry - asm_irqentry_enter_from_user_mode save = 0 + asm_irqentry_enter_from_user_mode kuser_cmpxchg_check irq_handler from_user=1 get_thread_info tsk @@ -447,7 +447,7 @@ ENDPROC(__irq_usr) .align 5 __und_usr: usr_entry uaccess=0 - asm_irqentry_enter_from_user_mode save = 0 + asm_irqentry_enter_from_user_mode @ IRQs must be enabled before attempting to read the instruction from @ user space since that could cause a page/translation fault if the @@ -472,7 +472,7 @@ ENDPROC(__und_usr) .align 5 __pabt_usr: usr_entry - asm_irqentry_enter_from_user_mode save = 0 + asm_irqentry_enter_from_user_mode mov r2, sp @ regs pabt_helper UNWIND(.fnend ) diff --git a/arch/arm/kernel/entry-common.S b/arch/arm/kernel/entry-common.S index 14b2495cae3c2f95b0dfecd849b4e16ec143dbe9..df564388905ee019cd5553f8b37e678da59e3222 100644 --- a/arch/arm/kernel/entry-common.S +++ b/arch/arm/kernel/entry-common.S @@ -109,7 +109,7 @@ ENTRY(ret_to_user_from_irq) movs r1, r1, lsl #16 bne slow_work_pending no_work_pending: - asm_irqentry_exit_to_user_mode save = 0 + asm_irqentry_exit_to_user_mode #ifdef CONFIG_GCC_PLUGIN_STACKLEAK bl stackleak_erase_on_task_stack diff --git a/arch/arm/kernel/entry-header.S b/arch/arm/kernel/entry-header.S index fb5bb019199b2871e29e306a29bea8fdf47dd7f3..50c0b55adc7421e7be123c9d00f94b1ebb93ff9e 100644 --- a/arch/arm/kernel/entry-header.S +++ b/arch/arm/kernel/entry-header.S @@ -365,28 +365,14 @@ ALT_UP_B(.L1_\@) * Context tracking and other mode transitions. Used to instrument transitions * between user and kernel mode. */ - .macro asm_irqentry_enter_from_user_mode, save = 1 - .if \save - stmdb sp!, {r0-r3, ip, lr} + .macro asm_irqentry_enter_from_user_mode mov r0, sp @ regs bl irqentry_enter_from_user_mode - ldmia sp!, {r0-r3, ip, lr} - .else - mov r0, sp @ regs - bl irqentry_enter_from_user_mode - .endif .endm - .macro asm_irqentry_exit_to_user_mode, save = 1 - .if \save - stmdb sp!, {r0-r3, ip, lr} + .macro asm_irqentry_exit_to_user_mode mov r0, sp @ regs bl irqentry_exit_to_user_mode - ldmia sp!, {r0-r3, ip, lr} - .else - mov r0, sp @ regs - bl irqentry_exit_to_user_mode - .endif .endm /* From patchwork Tue Jan 7 09:41:33 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13928602 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3289EE77199 for ; Tue, 7 Jan 2025 10:03:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=MgjerbaRhix+kd50qFGRKR55bWj7bTqB01M6nHY9FI4=; b=qRKzJCRFTp+yStvRdvMYkFN8S6 45uRJ5/20KgBsJkDv3E7kz5VbEsmNL6uMtxNBzCJO3dooI1S7buOVafD13GzIdAkOqYiTQXKqVjul yTowOf+PClaj7AiEiBUaAqpu7DnadPPUBDbZOzXgXmTNujxLYnODs8FeLbn9X9m6QwFWFocKnAcjC 5Ac//sbvUbKC2hktNdb5B2kS5Ofwxdv8UVWA5yG34Sl7GlMnYCHWGh0P82hedDPbVBxbnIgboBEa+ WkzCxvDdJ5uSMJ+CXyZ6tJS3mMhhLj5DmPu5qskkcTpk0IRMv74bTdK0/HxyG2iiUffN6MpW8n8Iy v9MS8Jaw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tV6QS-00000004FqH-3gG3; Tue, 07 Jan 2025 10:03:04 +0000 Received: from mail-ej1-x633.google.com ([2a00:1450:4864:20::633]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tV65u-00000004AUR-3jfO for linux-arm-kernel@lists.infradead.org; Tue, 07 Jan 2025 09:41:54 +0000 Received: by mail-ej1-x633.google.com with SMTP id a640c23a62f3a-aaee0b309adso1866416966b.3 for ; Tue, 07 Jan 2025 01:41:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1736242909; x=1736847709; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=MgjerbaRhix+kd50qFGRKR55bWj7bTqB01M6nHY9FI4=; b=eK3SCLcNR27dIvNnsLUq/8tK+VJScfajFKlmeP7mux+SH1KSpWjx3NPQq1Y5Lr+PRN YXMrUVr1h4xwA/H8gLd9W+lcE8yTEUof98hhBh/NVnwddQ9rSMK/xiwB3TCsUNwxQ+4M idXe4ZsEAwYu34/oTsUfCAYFyPpRCeiUH6aMDPY3B5w3nkEOGxVUs+st/v50shX9Gz7t Us0I5BVtgUkaoOGzwBCiI01H3cMotTOouN8YLTr4Zfb16sDUNj3eKwHLBeUzSUedP01u Ijr+v0TLDg87+WpgsQ7y0kBp/4nZA63jcDaaEsO/BMD7qjZorQv6QB/9/N8YN0H+uJDG PJNg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736242909; x=1736847709; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=MgjerbaRhix+kd50qFGRKR55bWj7bTqB01M6nHY9FI4=; b=A3BG0Q9hjX3lUWpB9W17nuCLNWVAw+0XnuMSGmsbBDsst7PsXB2EYMCqEVBIUl1jlw nTIvhnzR332PlgyMQDADmJo2W9GU5VgqrJo2lmidb/AKv57Ca9sDHV184v+oROlbilGJ 3TktgZGet3dwmL1ZqbzmZA9C9/nSQzYaDg3nEduR2SSzRNca7TzicBQi3iewjJYon39x dIhItmkxfT8hZxpYAzCEPcuk5lcSoXkwrTPAN/JjycGRaW+egEL4pEdJKRMR4Wn8/jCH 2MYu7G9VzsjVC+ij3k5yVbbALiDKjHbow/AEa/8COI63Q45B6qF7rDm/FdydisMoQU2E st8Q== X-Gm-Message-State: AOJu0Yw/WalNeHAhesac4SPi3onoHW0BNAP0S8/zVejZcv/88eeGafTA hvjGlnaKQ6KDGvrswuzYBg+Tsxflf4LleWnQQFhX3gzMCR2upl98JVk8RS7CWfFyNfN0JPxUdFt 3sRk= X-Gm-Gg: ASbGncuENsdGa6nGrTNh8nbOHQJMCYIQY9UntqbvJZd4muO0jR2gmI+VFsTzQrnc890 9HasS0XRXku89BdkSSgTKvyk8oQcK0q0OlMS9j/DZtdMt1s2qLt1r/OqouJUySHjfwkkP8lS+Q0 qHlKawgByg+IsagifPhKt7xEplZlQRsWsGqhkporZ+DKiXvfBD8XWw+mFWpdJEKcWKVKoz1Fy1S rg2VfbYPoXiGgwG/4ZaX1PvsvxayIYnjsonA67yR08NSrmeXkSaPMG8LqHISJ2HH1s= X-Google-Smtp-Source: AGHT+IHeBpDVZuVWnjTw8CL4OhJcL4lUdEaCemS7Kwan56Mqu4ixU2+PMFWEVUVxrqr/Ynh4ykgBdA== X-Received: by 2002:a17:907:3f10:b0:aab:c6d6:62ac with SMTP id a640c23a62f3a-aac26966057mr4407081966b.0.1736242909536; Tue, 07 Jan 2025 01:41:49 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aac0f012229sm2356901166b.133.2025.01.07.01.41.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jan 2025 01:41:48 -0800 (PST) From: Linus Walleij Date: Tue, 07 Jan 2025 10:41:33 +0100 Subject: [PATCH RFC v3 17/30] ARM: entry: Implement syscall_exit_to_user_mode() MIME-Version: 1.0 Message-Id: <20250107-arm-generic-entry-v3-17-4e5f3c15db2d@linaro.org> References: <20250107-arm-generic-entry-v3-0-4e5f3c15db2d@linaro.org> In-Reply-To: <20250107-arm-generic-entry-v3-0-4e5f3c15db2d@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250107_014150_945628_7EFF8B2B X-CRM114-Status: GOOD ( 15.55 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Mirroring syscall_enter_from_user_mode() we create the sibling call syscall_exit_to_user_mode() and start to move some stuff over from assembly, first the rseq call and the interrupt disablement. Signed-off-by: Linus Walleij --- arch/arm/include/asm/entry.h | 1 + arch/arm/include/asm/signal.h | 1 - arch/arm/kernel/entry-common.S | 10 ++-------- arch/arm/kernel/entry.c | 8 ++++++++ arch/arm/kernel/signal.c | 7 ------- 5 files changed, 11 insertions(+), 16 deletions(-) diff --git a/arch/arm/include/asm/entry.h b/arch/arm/include/asm/entry.h index e259b074caef75c7f777b18199623f07bebee5b4..8b8cc9a1eefbd00dd75d4e344576fc184a413768 100644 --- a/arch/arm/include/asm/entry.h +++ b/arch/arm/include/asm/entry.h @@ -9,6 +9,7 @@ struct pt_regs; * to generic entry once they are semantically equivalent. */ long syscall_enter_from_user_mode(struct pt_regs *regs, long); +void syscall_exit_to_user_mode(struct pt_regs *regs); void irqentry_enter_from_user_mode(struct pt_regs *regs); void irqentry_exit_to_user_mode(struct pt_regs *regs); diff --git a/arch/arm/include/asm/signal.h b/arch/arm/include/asm/signal.h index 7acccc96840c8a17744cc3f2894f19d142aba4fd..bb753cd3e29fc54186543c48b2aee7b221301a37 100644 --- a/arch/arm/include/asm/signal.h +++ b/arch/arm/include/asm/signal.h @@ -23,7 +23,6 @@ typedef struct { #include -void do_rseq_syscall(struct pt_regs *regs); void do_work_pending(struct pt_regs *regs, unsigned int thread_flags); #endif diff --git a/arch/arm/kernel/entry-common.S b/arch/arm/kernel/entry-common.S index df564388905ee019cd5553f8b37e678da59e3222..3559e0a9f5e079f1af91392bf000d39f9b45906c 100644 --- a/arch/arm/kernel/entry-common.S +++ b/arch/arm/kernel/entry-common.S @@ -65,12 +65,9 @@ ret_fast_syscall: UNWIND(.fnstart ) UNWIND(.cantunwind ) add sp, sp, #(S_R0 + S_OFF) -#if IS_ENABLED(CONFIG_DEBUG_RSEQ) /* do_rseq_syscall needs interrupts enabled. */ mov r0, sp @ 'regs' - bl do_rseq_syscall -#endif - disable_irq_notrace @ disable interrupts + bl syscall_exit_to_user_mode ldr r1, [tsk, #TI_FLAGS] @ re-check for syscall tracing movs r1, r1, lsl #16 beq no_work_pending @@ -97,13 +94,10 @@ ENDPROC(ret_fast_syscall) * do_work_pending() will update this state if necessary. */ ENTRY(ret_to_user) -#if IS_ENABLED(CONFIG_DEBUG_RSEQ) /* do_rseq_syscall needs interrupts enabled. */ enable_irq_notrace @ enable interrupts mov r0, sp @ 'regs' - bl do_rseq_syscall -#endif - disable_irq_notrace @ disable interrupts + bl syscall_exit_to_user_mode ENTRY(ret_to_user_from_irq) ldr r1, [tsk, #TI_FLAGS] movs r1, r1, lsl #16 diff --git a/arch/arm/kernel/entry.c b/arch/arm/kernel/entry.c index 1973947c7ad753fccd694b3ef334fba1326f58b6..47233804733752aef0f2fec6f54badeab18e98d0 100644 --- a/arch/arm/kernel/entry.c +++ b/arch/arm/kernel/entry.c @@ -1,7 +1,9 @@ // SPDX-License-Identifier: GPL-2.0 #include +#include #include #include +#include long syscall_enter_from_user_mode(struct pt_regs *regs, long syscall) { @@ -14,6 +16,12 @@ long syscall_enter_from_user_mode(struct pt_regs *regs, long syscall) return syscall; } +void syscall_exit_to_user_mode(struct pt_regs *regs) +{ + rseq_syscall(regs); + local_irq_disable(); +} + noinstr void irqentry_enter_from_user_mode(struct pt_regs *regs) { trace_hardirqs_off(); diff --git a/arch/arm/kernel/signal.c b/arch/arm/kernel/signal.c index 7b1a16e86b236575efdc29cda9b751e8e2a3f64e..03b20637a2e1ca6f327223c9132915a97a9c4697 100644 --- a/arch/arm/kernel/signal.c +++ b/arch/arm/kernel/signal.c @@ -659,13 +659,6 @@ struct page *get_signal_page(void) return page; } -#ifdef CONFIG_DEBUG_RSEQ -asmlinkage void do_rseq_syscall(struct pt_regs *regs) -{ - rseq_syscall(regs); -} -#endif - /* * Compile-time assertions for siginfo_t offsets. Check NSIG* as well, as * changes likely come with new fields that should be added below. From patchwork Tue Jan 7 09:41:34 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13928604 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 14C1CE77197 for ; Tue, 7 Jan 2025 10:04:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=rNGuHvTHhzfhTH3CopTS+hOuzXjJxItcv+TieblAYlA=; b=u02jyBYe93mP95uOeBhLqniWb8 BRQcg26VEPUOZ3DPBxloLh+LErQnYyh5kRmNBnXI9XcBCRoOZBd1bEELubeptcHuEYBKskjrGajfk U/+8Zw4h3tIc+zoPRB6fFXb7jTp6FNddQvqoDF6Z4qLKwOm3doCK+I54QO1kaJvwisaj84et5geF+ VOxOSRREpD2i6leYTTxhSlMYC+b2/l9nLLPzBdgRwd3E41eDjdlRzwKdJFB+GINGPe3mTJXodkmTl 21JMVEfKYKwNIwVhEZ5G/kc9VXvynCAlkZ+DVYb96V1ZXy2H7Zku8eLIg6eXpiWxbMXyiPpVTuEj0 nWPmYBGQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tV6Rc-00000004G1E-2QvW; Tue, 07 Jan 2025 10:04:16 +0000 Received: from mail-ed1-x535.google.com ([2a00:1450:4864:20::535]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tV65w-00000004AVM-23ru for linux-arm-kernel@lists.infradead.org; Tue, 07 Jan 2025 09:41:54 +0000 Received: by mail-ed1-x535.google.com with SMTP id 4fb4d7f45d1cf-5d3e6f6cf69so26641454a12.1 for ; Tue, 07 Jan 2025 01:41:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1736242911; x=1736847711; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=rNGuHvTHhzfhTH3CopTS+hOuzXjJxItcv+TieblAYlA=; b=f5MrURz2fCsrDfyYZPzbyWnCYdld+B0+vwu2OxWIfkZQtDpBrTA0jfvhT8pRZy4Zz3 tRGhE5oOvmwDWXw2RaB6YB1kjC2YGyoHvz0TW/sKQTcf/r87/tnRTKZonU/5X0JFLrWF ayKqKZx62XTDDD+CqwDQLEXcfUdrSzNBjDtcocqddJbXGxixonTQ1VwIxS61t2HpT0ib c05N430ncmuAIQC67mBVnXK2PuZp939CFf7vgjQu0pSCGGUWAX7LCJpqsJYZxzH7wFmL gYiSoF34xa3g43mJ9/7ElivHQaKYhRJdhaQ8txJQOveF/pqCfg1sTf+TMW/tOYi+lvwv WX1g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736242911; x=1736847711; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=rNGuHvTHhzfhTH3CopTS+hOuzXjJxItcv+TieblAYlA=; b=Q7tpJ+u2/RfzrH1i0jcvDgzD8sRJLHmpKm8jZcyfrvIphnbU6TlNCKGcGbReEwLkYS FnVXYbGJTtTsfERxHd/mtcGgFYNtMoTlKa0r4kSimDLPAVoPiAv9bF7keJYy/wUsyuQv 1pvRAOBb3/tgXAd6vHvCuVJnolwBu5yJ+8XDbVwuzX9XANvwQiy+JEb6gwNA6en8/1lr nWL+ymtJdadHVlCn0Z0bd9p47qijhVklVwqWDBz07xis7mgRoLnXRSW71TrtH3V0iQBE pVlfLuTo8oZD4NlwDdFlfwkqJh/cjOOLM3/8AxEJhiSepBW25GM7fhUl6U5mkoWuz4hg A3lQ== X-Gm-Message-State: AOJu0YybHEipmf2h1paINoXbHYzkskpizyLU7X6YXaDQZ93G/DWekQ8f Kqv7dX8z0CuX6ySMnvzXHllPuSbDocdgXfgUQcBHkxkWJL14ni0CmDY7Fo1gc5E= X-Gm-Gg: ASbGncs2eNvEqCfuQbMtS1Jn5vDrfVA572zkZZ2ICJUsrronypYrZlPFLnWEGA+OgwB 4FAYCQBdTL0cI0nSgd4u1RiAQxFUFOR6jJgVOd8aDMgqgh3Gcq9LMx7vMd/AcKMY4BXmIJ4DuED R1xDtiT60ddNpy53lI9/UwC8SWXfE/o6pO3eqFaAlqCQfzpHvsz7dh+CpY8VhNs0Qh5Uk+Cilxj wuVqgJM+tXPhNR22CffXKLdxKra4VrPYQe6SX70dIeCQZLgVPHetZYHRfKDRzRVSjw= X-Google-Smtp-Source: AGHT+IFxRx1Uu23VD6LIatzdS+l1ndVVhhtxHZ732JbaJpgqK/hVhhr7XB86kAmbVwbbNYxk+Vt9bw== X-Received: by 2002:a17:907:7291:b0:aae:e784:55c5 with SMTP id a640c23a62f3a-aaee7846124mr3788329666b.33.1736242910656; Tue, 07 Jan 2025 01:41:50 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aac0f012229sm2356901166b.133.2025.01.07.01.41.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jan 2025 01:41:50 -0800 (PST) From: Linus Walleij Date: Tue, 07 Jan 2025 10:41:34 +0100 Subject: [PATCH RFC v3 18/30] ARM: entry: Drop the superfast ret_fast_syscall MIME-Version: 1.0 Message-Id: <20250107-arm-generic-entry-v3-18-4e5f3c15db2d@linaro.org> References: <20250107-arm-generic-entry-v3-0-4e5f3c15db2d@linaro.org> In-Reply-To: <20250107-arm-generic-entry-v3-0-4e5f3c15db2d@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250107_014152_533523_C5E49130 X-CRM114-Status: GOOD ( 14.56 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org When not using any irqflags tracing, context tracking or rseq debugging, the ARM kernel uses a "superfast" ret_fast_syscall with a very few assembly instructions. With generic entry, such local optimizations are not really possible, we need to call C unconditionally and take the (hopefully small) overhead. With generic entry in C, most of the tracing (etc) calls are replaced by stubs and optimized out by the compiler. Signed-off-by: Linus Walleij --- arch/arm/kernel/entry-common.S | 29 +---------------------------- 1 file changed, 1 insertion(+), 28 deletions(-) diff --git a/arch/arm/kernel/entry-common.S b/arch/arm/kernel/entry-common.S index 3559e0a9f5e079f1af91392bf000d39f9b45906c..dddbd6ba1eb9fcbc2c0d81f24e1a9b3231558d7f 100644 --- a/arch/arm/kernel/entry-common.S +++ b/arch/arm/kernel/entry-common.S @@ -30,34 +30,8 @@ saved_pc .req lr .section .entry.text,"ax",%progbits .align 5 -#if !(IS_ENABLED(CONFIG_TRACE_IRQFLAGS) || IS_ENABLED(CONFIG_CONTEXT_TRACKING_USER) || \ - IS_ENABLED(CONFIG_DEBUG_RSEQ)) /* - * This is the fast syscall return path. We do as little as possible here, - * such as avoiding writing r0 to the stack. We only use this path if we - * have tracing, context tracking and rseq debug disabled - the overheads - * from those features make this path too inefficient. - */ -ret_fast_syscall: - UNWIND(.fnstart ) - UNWIND(.cantunwind ) - disable_irq_notrace @ disable interrupts - ldr r1, [tsk, #TI_FLAGS] @ re-check for syscall tracing - movs r1, r1, lsl #16 - bne fast_work_pending - - restore_user_regs fast = 0, offset = S_OFF - UNWIND(.fnend ) -ENDPROC(ret_fast_syscall) - - /* Ok, we need to do extra processing, enter the slow path. */ -fast_work_pending: - add sp, sp, #(S_R0 + S_OFF) - /* fall through to work_pending */ -#else -/* - * The "replacement" ret_fast_syscall for when tracing, context tracking, - * or rseq debug is enabled. As we will need to call out to some C functions, + * As we will need to call out to some C functions, * we save r0 first to avoid needing to save registers around each C function * call. */ @@ -75,7 +49,6 @@ ret_fast_syscall: ENDPROC(ret_fast_syscall) /* Slower path - fall through to work_pending */ -#endif tst r1, #_TIF_SYSCALL_WORK beq slow_work_pending From patchwork Tue Jan 7 09:41:35 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13928605 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 90547E77197 for ; Tue, 7 Jan 2025 10:05:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=tm8Sr1+K14orzlSuTAQl0kAeE85JLIYVrwIxTxn0xos=; b=Wb/ClgQFzoSpPMrfBZiLLbpxU6 f9gviTxqvKFCaxwJw05Yuxj0hOGaL3Qv/zoPGCSNJBV5W5PrkF9Z5cTqHuR8BJ3neVu4zl/ahdFj1 tCyvsYZ20dYPa8nzHa4yjhwGqPtFhOr4qB1Q3q7vP2oIwQB+WlFnI54HX7SzdUSPVBz0MvhppIm7U Zg5uwhpj9O6kRjBzVEu9iChs1GtiX6V81lcp+yxAepRa21l3bwiaNiXjuZSC77eVP3b2ElSjlYs1k dkFApynwpkdX0NJjtuJi+app8zHVi0ZFtwKbcuYWev0u6UEqv7S0MAR1xuf2AHsJbuE1i3HZVu1wW T5FVjbWA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tV6Sm-00000004GEL-0pVq; Tue, 07 Jan 2025 10:05:28 +0000 Received: from mail-ed1-x530.google.com ([2a00:1450:4864:20::530]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tV65x-00000004AWB-3w4s for linux-arm-kernel@lists.infradead.org; Tue, 07 Jan 2025 09:41:54 +0000 Received: by mail-ed1-x530.google.com with SMTP id 4fb4d7f45d1cf-5d0ac27b412so20774719a12.1 for ; Tue, 07 Jan 2025 01:41:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1736242912; x=1736847712; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=tm8Sr1+K14orzlSuTAQl0kAeE85JLIYVrwIxTxn0xos=; b=Xo5iZtEgrMK5O91GEH9K0m0M/IlCVPBmBmhg1ZA+GdEKy3PnMnZXeEhuGk24S9/A5T RTKaoh/5ai83lX1VXe79b9OCC/7y6CxvH9x09hFMvazwUSS8JhzyXOj9ez/8j0P2IbU/ o2G+fpIAH4W/VVY8lfw4JVHbNIQI2hxCee6pRiVKoJcTwTIhJ1Eh5J3ER+oQr/pwzPTC EgbDnR9j88GzWcQLVFmsN01D0TQ/wqB3NFZrSpG9CU0+6xvtwalzIP+wTXzvgLTV/GsL H4DMLi3KbSrWAHfuDYOGWMF0agOqv9eXeZD5OJ3UlvecZGzNmXGxLKZaT+VcJHlAMG24 cveQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736242912; x=1736847712; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=tm8Sr1+K14orzlSuTAQl0kAeE85JLIYVrwIxTxn0xos=; b=wySZK6KxI2m48b6m+zgI3ZGI2YnrrVMoImka8L8mcJg7+f4s5gaYpixLpb3gKK3VIR b+sSCG+jzvaLI3zaTOeXz26bLhmuNpeJMQCzIBnl61LHT+2rA1e2xyG8IcgyVmaXZkMi UNrD5FwKMOp9rN5/W+cyuhgPqr4v4mwGXj97RfWVeGisku5HeMeIHo+MMQHPRuPvtWgn ybvD5xpXynW48Dpi3mfgZBd1U0draLB9z6zIUerFvOaV2rtlWvBU1oo5e/cJbxhsxyqv bS2faIL+asC/PbYxewyYk/Om3vXH7V153R5y7ikGUkc+SQ8Q+2wuXldf3HkbYmg3i4gn 5YGA== X-Gm-Message-State: AOJu0YzbTP71aBNnVZarJerGhetwb1XEyY5tGuHKJE/f+Wy88n9Z2Rpa i8qNNF0E13xP6kfJsDxYbFh4xWeOgDy+7uXC/0xdW3vOY4KkAWZhL8r2VG8uBd4= X-Gm-Gg: ASbGncsMuBn5/XcJd343msY/ZJzFVAAkG2lKMqYAmG6kpcSvfN2s1S+66zSLhC8uqBX WT6ZDpxsglMUuFydzq1KUFi7yplWu37h8HFaSwGe1wDsTkikKuvXjcmJJr9ZX1wDZe+RkMwNzZa kFVLCN0b+UCJ+lDqXLXP0tbpVOFGo7WFVNP7JPpXVMYprRWAPGtcqESBT7f4qopzEVWiaDb5uNU Do1MmPdTQmHxMS8bFjhUoQUiSgNCkbpdExBxo4b5wwQnmaQVwtgAAUYEvEWhYsWlwo= X-Google-Smtp-Source: AGHT+IEoUYCHmiu2LtqOgL3Sk5UkLCViigDDsF2BzL3K9BciTMZDokF+kh6Ppu7b80wGUY1bqQWgoQ== X-Received: by 2002:a17:907:3e1c:b0:aa5:2575:e75d with SMTP id a640c23a62f3a-aac270369b4mr5485696266b.2.1736242912197; Tue, 07 Jan 2025 01:41:52 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aac0f012229sm2356901166b.133.2025.01.07.01.41.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jan 2025 01:41:51 -0800 (PST) From: Linus Walleij Date: Tue, 07 Jan 2025 10:41:35 +0100 Subject: [PATCH RFC v3 19/30] ARM: entry: Remove fast and offset register restore MIME-Version: 1.0 Message-Id: <20250107-arm-generic-entry-v3-19-4e5f3c15db2d@linaro.org> References: <20250107-arm-generic-entry-v3-0-4e5f3c15db2d@linaro.org> In-Reply-To: <20250107-arm-generic-entry-v3-0-4e5f3c15db2d@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250107_014153_984405_882554AC X-CRM114-Status: GOOD ( 13.74 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Now all remaining invocations of restore_user_regs has fast = 0 and offset = 0 set, so drop these parameters to the macro. Signed-off-by: Linus Walleij --- arch/arm/kernel/entry-armv.S | 2 +- arch/arm/kernel/entry-common.S | 2 +- arch/arm/kernel/entry-header.S | 29 +++++++++-------------------- 3 files changed, 11 insertions(+), 22 deletions(-) diff --git a/arch/arm/kernel/entry-armv.S b/arch/arm/kernel/entry-armv.S index 460aa92c3c1f50de905550acf363c58f509bfe0a..3cd5a80b47758867d1db6c3971030370bca1551a 100644 --- a/arch/arm/kernel/entry-armv.S +++ b/arch/arm/kernel/entry-armv.S @@ -497,7 +497,7 @@ __fiq_usr: mov r0, sp @ struct pt_regs *regs bl handle_fiq_as_nmi get_thread_info tsk - restore_user_regs fast = 0, offset = 0 + restore_user_regs UNWIND(.fnend ) ENDPROC(__fiq_usr) diff --git a/arch/arm/kernel/entry-common.S b/arch/arm/kernel/entry-common.S index dddbd6ba1eb9fcbc2c0d81f24e1a9b3231558d7f..d27e34d7e97a079d62e7b778cf7cf97c302e4763 100644 --- a/arch/arm/kernel/entry-common.S +++ b/arch/arm/kernel/entry-common.S @@ -81,7 +81,7 @@ no_work_pending: #ifdef CONFIG_GCC_PLUGIN_STACKLEAK bl stackleak_erase_on_task_stack #endif - restore_user_regs fast = 0, offset = 0 + restore_user_regs ENDPROC(ret_to_user_from_irq) ENDPROC(ret_to_user) diff --git a/arch/arm/kernel/entry-header.S b/arch/arm/kernel/entry-header.S index 50c0b55adc7421e7be123c9d00f94b1ebb93ff9e..51c7f93c1ded1db56f173c7aaed8094ef011ff5a 100644 --- a/arch/arm/kernel/entry-header.S +++ b/arch/arm/kernel/entry-header.S @@ -290,7 +290,7 @@ .endm - .macro restore_user_regs, fast = 0, offset = 0 + .macro restore_user_regs #if defined(CONFIG_CPU_32v6K) && \ (!defined(CONFIG_CPU_V6) || defined(CONFIG_SMP)) #ifdef CONFIG_CPU_V6 @@ -309,8 +309,8 @@ ALT_UP_B(.L1_\@) #ifndef CONFIG_THUMB2_KERNEL @ ARM mode restore mov r2, sp - ldr r1, [r2, #\offset + S_PSR] @ get calling cpsr - ldr lr, [r2, #\offset + S_PC]! @ get pc + ldr r1, [r2, S_PSR] @ get calling cpsr + ldr lr, [r2, S_PC]! @ get pc tst r1, #PSR_I_BIT | 0x0f bne 1f msr spsr_cxsf, r1 @ save in spsr_svc @@ -318,31 +318,24 @@ ALT_UP_B(.L1_\@) @ We must avoid clrex due to Cortex-A15 erratum #830321 strex r1, r2, [r2] @ clear the exclusive monitor #endif - .if \fast - ldmdb r2, {r1 - lr}^ @ get calling r1 - lr - .else ldmdb r2, {r0 - lr}^ @ get calling r0 - lr - .endif mov r0, r0 @ ARMv5T and earlier require a nop @ after ldm {}^ - add sp, sp, #\offset + PT_REGS_SIZE + add sp, sp, #PT_REGS_SIZE movs pc, lr @ return & move spsr_svc into cpsr 1: bug "Returning to usermode but unexpected PSR bits set?", \@ #elif defined(CONFIG_CPU_V7M) @ V7M restore. @ Note that we don't need to do clrex here as clearing the local @ monitor is part of the exception entry and exit sequence. - .if \offset - add sp, #\offset - .endif - v7m_exception_slow_exit ret_r0 = \fast + v7m_exception_slow_exit ret_r0 = 0 #else @ Thumb mode restore mov r2, sp - load_user_sp_lr r2, r3, \offset + S_SP @ calling sp, lr - ldr r1, [sp, #\offset + S_PSR] @ get calling cpsr - ldr lr, [sp, #\offset + S_PC] @ get pc - add sp, sp, #\offset + S_SP + load_user_sp_lr r2, r3, S_SP @ calling sp, lr + ldr r1, [sp, S_PSR] @ get calling cpsr + ldr lr, [sp, S_PC] @ get pc + add sp, sp, S_SP tst r1, #PSR_I_BIT | 0x0f bne 1f msr spsr_cxsf, r1 @ save in spsr_svc @@ -350,11 +343,7 @@ ALT_UP_B(.L1_\@) @ We must avoid clrex due to Cortex-A15 erratum #830321 strex r1, r2, [sp] @ clear the exclusive monitor - .if \fast - ldmdb sp, {r1 - r12} @ get calling r1 - r12 - .else ldmdb sp, {r0 - r12} @ get calling r0 - r12 - .endif add sp, sp, #PT_REGS_SIZE - S_SP movs pc, lr @ return & move spsr_svc into cpsr 1: bug "Returning to usermode but unexpected PSR bits set?", \@ From patchwork Tue Jan 7 09:41:36 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13928606 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 2251BE77197 for ; Tue, 7 Jan 2025 10:06:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=Pj8/SaLudo4NEJ77c2SgPAbc3BlZnt/7rdnxCBQ6AWg=; b=v8J0uMpVfkeyBQTqDcTdoiC/Lp epokors30hNrAJu+79f31lw4iAQwZOoJvxiCUd2E4EZJivr4iWl+et+/4RmAPrF/OpiQT2/ziBOoN 0P0n4T9ne0TrWtx/v3nfXD2kjABjlFRL3FtLR0jtbTCiV0fSn2N2oJY3CEKZaY076q5ZyPuduckdq Bq8AFvR/0gZYx4WtdBatgFc4lfLlizjc0HRxAbcdURmHmtdGT5uU37Yjji0eCeADpPP5IEiFke9yY Qlw/DJSzJ4ardV9mwhEHBaQEnifptkiBSN7lboAWjcMsmoAsF7bBmKjEKGhszUXsQwmoHdj9MPcuK 95ER+Adw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tV6Tv-00000004GPu-3R7P; Tue, 07 Jan 2025 10:06:39 +0000 Received: from mail-ej1-x62b.google.com ([2a00:1450:4864:20::62b]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tV65y-00000004AWv-3Dnt for linux-arm-kernel@lists.infradead.org; Tue, 07 Jan 2025 09:41:54 +0000 Received: by mail-ej1-x62b.google.com with SMTP id a640c23a62f3a-aa6a92f863cso257465366b.1 for ; Tue, 07 Jan 2025 01:41:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1736242913; x=1736847713; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=Pj8/SaLudo4NEJ77c2SgPAbc3BlZnt/7rdnxCBQ6AWg=; b=c6iMcik8wuX4Az8PwNshyna4o0FWrEH4vYePfZocJW/O8NfeKXLuuTl0WLM6R+Yjeh /HY7hvHSViml2Ux+IBzKVCZdInVUPpLeec56PbPUHTsPWytwVm8Xxv2mow6A2KX6NfoN e6uLCtM3PQ5WACt+woH1giQwvqwuOQMlVswHfSis1G0f53kYPoVqkdKgOIAtEEYpVpnz dRbFU75ZZA+6wOsSDw76bw0RQHVMGWY2VsGAXktnvxAvU8qhZk9zFu2HWStcVVo9r5PF dG36yCKGkar9Qpqyv/uK7Yz6UPNxjCjCj7MU9GnMaK9QvihbuvaO1e4fSQ+JUcgSGQ3Z uNmQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736242913; x=1736847713; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Pj8/SaLudo4NEJ77c2SgPAbc3BlZnt/7rdnxCBQ6AWg=; b=ktCNlxIrtCxqV7goQ4muV/lDzZHYm//Tl+0CS0WPZ4TSCuy7CnhbxVv530tIwN8TIE bya5NdjHH7WsTtAIzByxiScqD7PYcQuf8llNBbNKYQylUQoN5V6OcvY9wx0HwM43SIRl McRRoEEkAUt6Gj4TUmHJwkheyCd3apEHQCj2XCe7X1Pk7NQesxSTWT9OM1ERcv1kAJ1n PufJ2uhxZcZn0pz1Y1NXDPcTWG722gLnoE3dRW739lc2W7NB17IGoZNOOq+WbTy21ARD Mpl0MMZzYsZwuOMzZhZAjxhTCmQ4k4IcoZE6lIX1gsPRk7lfbYyqGDU/fkfR103u3o4C jPvw== X-Gm-Message-State: AOJu0YzMRGhMpsICn1tl/pCIFbh+IYqgSn40l5F4h5o4wUtHo+tEoZDh CmUn3UpWjHgLtzEG1qMyaiM/3IbanySZJwzpYcAnA3Vtk/UK/4Pd7o2BAyTO4e8= X-Gm-Gg: ASbGncvL5l1df1HUfCQ0J7A6nHqfqRALMj6ZFrZ3ubsFcaIHOkY5SIl/Gwg4e7yafo6 6f2Djo24Us7AMMC17NjODnNr+5L4TCWLk6yGI4RVtzeuRiZ629rHoLzeS4wGsb8RWolmg1DaIun JlpnvOk246h7spUlDvVwQp/Mc5nkwgiz0SkixL+3EygjfzoNR2EQezopEWhNB3sC1y2hmBicdQ9 axFC9paWobhQOSAen81XI2N+5f0J4EL2DBX929M7Ay++Ak7dz2Kag/88iPehdtNyy4= X-Google-Smtp-Source: AGHT+IHVxLk0EMwczs0Uxehy2LL79n8I1psLDTVXxkeoAqQionVlmgUKTFWz0Wz6ph/R5/p8X73BAg== X-Received: by 2002:a17:907:318c:b0:aaf:ab6f:da49 with SMTP id a640c23a62f3a-aafab6fdc82mr1319882366b.39.1736242913302; Tue, 07 Jan 2025 01:41:53 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aac0f012229sm2356901166b.133.2025.01.07.01.41.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jan 2025 01:41:52 -0800 (PST) From: Linus Walleij Date: Tue, 07 Jan 2025 10:41:36 +0100 Subject: [PATCH RFC v3 20/30] ARM: entry: Untangle ret_fast_syscall/to_user MIME-Version: 1.0 Message-Id: <20250107-arm-generic-entry-v3-20-4e5f3c15db2d@linaro.org> References: <20250107-arm-generic-entry-v3-0-4e5f3c15db2d@linaro.org> In-Reply-To: <20250107-arm-generic-entry-v3-0-4e5f3c15db2d@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250107_014154_817698_D564A030 X-CRM114-Status: GOOD ( 12.11 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The return to userspace is a bit hard to follow. To prepare the ground for further refactorings we rewrite the functions to ret_fast_syscall and ret_to_user are mostly stand-alone symbols. Signed-off-by: Linus Walleij --- arch/arm/kernel/entry-common.S | 25 +++++++++++++++---------- 1 file changed, 15 insertions(+), 10 deletions(-) diff --git a/arch/arm/kernel/entry-common.S b/arch/arm/kernel/entry-common.S index d27e34d7e97a079d62e7b778cf7cf97c302e4763..ece9215717143427de7572a18a6dbe620a9193e9 100644 --- a/arch/arm/kernel/entry-common.S +++ b/arch/arm/kernel/entry-common.S @@ -44,20 +44,23 @@ ret_fast_syscall: bl syscall_exit_to_user_mode ldr r1, [tsk, #TI_FLAGS] @ re-check for syscall tracing movs r1, r1, lsl #16 - beq no_work_pending - UNWIND(.fnend ) -ENDPROC(ret_fast_syscall) - - /* Slower path - fall through to work_pending */ + beq 2f tst r1, #_TIF_SYSCALL_WORK - beq slow_work_pending + beq 1f + b ret_to_user -slow_work_pending: - mov r0, sp @ 'regs' +1: mov r0, sp @ 'regs' bl do_work_pending - b no_work_pending + +2: asm_irqentry_exit_to_user_mode + +#ifdef CONFIG_GCC_PLUGIN_STACKLEAK + bl stackleak_erase_on_task_stack +#endif + restore_user_regs + UNWIND(.fnend ) ENDPROC(ret_fast_syscall) /* @@ -74,7 +77,9 @@ ENTRY(ret_to_user) ENTRY(ret_to_user_from_irq) ldr r1, [tsk, #TI_FLAGS] movs r1, r1, lsl #16 - bne slow_work_pending + beq no_work_pending + mov r0, sp @ 'regs' + bl do_work_pending no_work_pending: asm_irqentry_exit_to_user_mode From patchwork Tue Jan 7 09:41:37 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13928608 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 67509E77197 for ; Tue, 7 Jan 2025 10:08:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=k9gWcnGbM9p5D0ak7vJHUxszZShK1nQ9isvrX6RkWiI=; b=MZbwlCOF4wlRDOEyWl9DCUZR+B w2RhUGxlK8vBCUiPVWGT7/9fAejSTByUBkUXJ/Ti4OsYlBzhD8iRDzYHodp91X2V8bEvvKCCt5taq lXDbOZWGLqdguJDa6wpzPzw02SfqIMlRwWXQMdd0z1Zlfl6yHcit6qTRlxcDY4BR0/R3BBYZCvdd3 Nm9VruoVJ9aTNUJlQFNzBrorhXbfXTQaoZgAThjA/J8q8zu4gbDsaRw6vZuK6ofJO2VsQV0fUOBEQ Nh8v3xB2azCM45H+7GYwCrKjMAIbnOs/UxnL+B1m/DdGklU0tZLlu/qTB3sdU3MtW3/Y9CjKXLlcE MvSxRIEQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tV6V6-00000004GnX-0Hgb; Tue, 07 Jan 2025 10:07:52 +0000 Received: from mail-ej1-x62e.google.com ([2a00:1450:4864:20::62e]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tV660-00000004AXz-05Hi for linux-arm-kernel@lists.infradead.org; Tue, 07 Jan 2025 09:42:01 +0000 Received: by mail-ej1-x62e.google.com with SMTP id a640c23a62f3a-aabfb33aff8so2739369266b.0 for ; Tue, 07 Jan 2025 01:41:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1736242914; x=1736847714; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=k9gWcnGbM9p5D0ak7vJHUxszZShK1nQ9isvrX6RkWiI=; b=Y90REn3cIlEX+pvLYpmzVTkvbkc1fs5gzVi0rUf4gyBgMBrd7OsztuP1s7480tZshA lcG+d9zQwYX9uX7Bb3uIXCyhpv7AYm/a/qweX++LlaE4pST+1z3F2MkI02o/S7G4Rymx y4cIhI+5RLbS6Ic74LiqcuCjcJX2dBm+cdHo/CbI1dskG0HQ6uXkifd3+mF4gkl3DYVz q3ZtRG8I24ECBpJ0z5ef+E6wxsd5jhJvejZ4oDwb7sx6m4IPB7zjz2vqhcTVAgAM5Uyd d6tA9EvEC6/4lVWUYGPDEeHI6zPPJ11PYKWl1SHfpq01j6VGhLk/F5/DSMxR6QaOylo3 7OLA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736242914; x=1736847714; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=k9gWcnGbM9p5D0ak7vJHUxszZShK1nQ9isvrX6RkWiI=; b=o2MFpRBmQVt06Sp4F1Q9AoDL976dRAsro4U+2v96qiOGQZPK8r3heIbsgm/SH2o1M4 s0DJdzdN8GJzoVrSCk61x2A9OmndyL2ouUh8ymd6Gz2O8pat9lsHTre4f3MT24/FA33W SyFUgelBCBu32vkfUpNCKVnCgz3c4k+eOIYJ9I8dAo5uQFBN5zjBKsMhHRF/LvMEPc2m S0LQ1ZSheHyL3dU+HQkrAGabCh/x67Fbdppnqiqmn6hX6JNPi9a3zxipQwyHOlOqJ92g w4tTQnRA+ECcMVSq+7xTBhmBH6r/2MKOAh9914lLCh3Oa4DVGYncINzzVAixcYBg/pfU XQmg== X-Gm-Message-State: AOJu0Yz6I4LVJlx4Iw7/OV+nHsMj+6al3Rf09okSiEOts4vTDabhuymy puOKaLQv+yLUVvAiMycHOD/Rt0ePlys+q86NhPrwNLfABryhpqXyt5+YmD2Nt2E= X-Gm-Gg: ASbGncver4qqdFjIdxnHcimLEo0xZD/KtKexD6UomiIXbtVY0AQf2RBCX+K0OTmmvEY Ah6yGw5MQpMAcO+FLM+B0MbhVMH7zFxuYZIEENe9QPFn2lmp7mwmpdIB+Pt5YkPvG2fgGi61I6z 04xJT+r2rqD5f/WN3kEJWo7/EVJxiQEol+01YKjvo2SvNU9KP51I54pnU7FPZOPqIDPAwiHOonN 1sLEH+EYkEbGOnlE8HOTVhawKB+IJMaRzEtQ+M/iG6u/kPgoqp7UunlPXGvJA6iGkA= X-Google-Smtp-Source: AGHT+IEHxw4mJqEFIqYXG/sglqgavssOFXM+rzspDasKvBGAFEnARnh+FvuNyCDWBSoQ/jKx1bXiCw== X-Received: by 2002:a17:907:1b05:b0:a9a:6c41:50a8 with SMTP id a640c23a62f3a-ab2910990f2mr261206866b.17.1736242914486; Tue, 07 Jan 2025 01:41:54 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aac0f012229sm2356901166b.133.2025.01.07.01.41.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jan 2025 01:41:54 -0800 (PST) From: Linus Walleij Date: Tue, 07 Jan 2025 10:41:37 +0100 Subject: [PATCH RFC v3 21/30] ARM: entry: Do not double-call exit functions MIME-Version: 1.0 Message-Id: <20250107-arm-generic-entry-v3-21-4e5f3c15db2d@linaro.org> References: <20250107-arm-generic-entry-v3-0-4e5f3c15db2d@linaro.org> In-Reply-To: <20250107-arm-generic-entry-v3-0-4e5f3c15db2d@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250107_014156_060384_0A70DA0F X-CRM114-Status: GOOD ( 17.42 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org This is a semantic change to ret_fast_syscall: if there is no work pending the function used to jump to ret_to_user, instead we proceed to return from the syscall. If we jump to ret_to_user, IRQs are enabled and we call syscall_exit_to_user_mode a second time; at this point a this just means a second call to rseq_syscall(), but it seems wrong and we want syscall_exit_to_user_mode to be called exactly once. Apparently the ARM kernel has under some circumstances called rseq_syscall() twice on the exit to userspace without side effects. ret_to_user also checks for pending work a second time under ret_to_user_from_irq, but this isn't even needed any more on the fast return path: this was needed because the syscall invocation enabled interrupts when doing syscall tracing, and all tracing invocation has been moved over to C in earlier patches. Signed-off-by: Linus Walleij --- arch/arm/kernel/entry-common.S | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/arm/kernel/entry-common.S b/arch/arm/kernel/entry-common.S index ece9215717143427de7572a18a6dbe620a9193e9..33bc9e7c5b32f62474724f1d47f97af24dba53bf 100644 --- a/arch/arm/kernel/entry-common.S +++ b/arch/arm/kernel/entry-common.S @@ -49,7 +49,7 @@ ret_fast_syscall: tst r1, #_TIF_SYSCALL_WORK beq 1f - b ret_to_user + b 2f 1: mov r0, sp @ 'regs' bl do_work_pending From patchwork Tue Jan 7 09:41:38 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13928609 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B39E2E77197 for ; Tue, 7 Jan 2025 10:09:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=deVa2FvUEOHV8lxaYOobNw6K9tuy+tyk/0Of0h0Ki9g=; b=J3vYmfLr3aLgS//4orqnaHIhSA TpAn9WiXn3V/AUjiIeFkkyY6HazMQqquWhPIMAQQv26FlzJC3tUTBh0nX24Dut7L8bIssDP4R16CB yXiunCa1aMMjNG/nremVln5bH7lIGYMyxWjMpjFXnpU66DR9xtQ2XCEOftfa66VH7JqEyl9YGyxqP eHe2CK4vqej54ievf+Z37pqcFNvOdjEPF1xbDCG9QJjx8RtkQ90QvBqbdfZ/J8I49q+Vvwamqpna/ Jky1AWD4wouybfv/7B5Thr/XODYxF9+8jlhYpf8PFDYxBFxkUomz4DFEMIf1P/yyM6rSmURvZzkIY zN96yoFQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tV6WG-00000004H2j-31Kf; Tue, 07 Jan 2025 10:09:04 +0000 Received: from mail-ej1-x636.google.com ([2a00:1450:4864:20::636]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tV661-00000004AYx-0hOa for linux-arm-kernel@lists.infradead.org; Tue, 07 Jan 2025 09:42:05 +0000 Received: by mail-ej1-x636.google.com with SMTP id a640c23a62f3a-aaeef97ff02so1919384666b.1 for ; Tue, 07 Jan 2025 01:41:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1736242916; x=1736847716; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=deVa2FvUEOHV8lxaYOobNw6K9tuy+tyk/0Of0h0Ki9g=; b=V/OkEKvzgWJuIstQj8zE9WoG7qioZJcrOaegQveJJyWqEHl9i6Ki75pBRI+6pGaduO sNQSy0SiZYlQUlPqhtscmq9askCrcWKHBSslIkSP9kCAcT4u9oMuzsA6/8I6CeEJJl6y /PxVr4uRjpHPxM5WcMa4Yl1pYUN20yHT8XzLwVlCZuVPuv2x7Im3CnrgK0SOzQP2Vh/D /7ibxzQn9tVndzzX+Nldxn7zg7va8EJNA56l6Ru+yqDn2SBxgZ/feOLiEbO5nos4GBxY wN1wqoLJprs/vVzLtMOVd3h09o9NBsW2Xb3D7zXweG8RcyBzfLtwXfyi7aZBYPBIv45s 81cg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736242916; x=1736847716; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=deVa2FvUEOHV8lxaYOobNw6K9tuy+tyk/0Of0h0Ki9g=; b=SDW9DgvGN9EgC1rvooq7ER8vN1iawvm17VWio4S14xbISsZ143ZDynJo7jzIigaTUj B21PX7apne3o5zr4mrq/KJhAU4NS2MD7FrV/FOz2fQH1B1/mdS4SkPMAdxhftWpEyvMP SQZ9fR7iMjxH/uWG0Jvjouf6cg5X7i7WKCQ7NqhTCWAQI2n8MPe/RbK1a1nzjSBKFBvY kc8TxJMtOfDsOFMJkxiO5oPydd8wqJzd9IG8ufsamSnaiY9fQ1piGcEFhiLFL93i486e IpA9NgDYE9VLY+Ao9UVc8KXl0Xhcv/rJ78uvO1PQ2WwCQFDUxEsW8nnHJr6W4W2jJKii QuYw== X-Gm-Message-State: AOJu0YwowokIf9uiR5XKG2aFmhnQBL2DGGYBkokBA5/1FxgXraEYsvD/ a5IvWI10WEi5YZFgpn4FWlw/ooxYBqQ/uqmI/tMEsXONl0tmzHaQsqM25bCun5k= X-Gm-Gg: ASbGnct4cTUPjLMKsPbc1M1wFR2MO3sns6OWWk61mvDtujgWVJYg7XG0xjRe+pukDZ1 W/DxwT9Wi8neIqkkrrIeFxGVR4rRSRRd1+JX9tLKeB/wHDnNK6wHV/RnZa3ssumba+oXH6sEy5g tRc6dzrRZfuq0doBuWpgmJ6k07sMrYsSCwZSBJW+l9w+6GjwlDJtPLAarCkCxp6Gu8+owcF9zZO ZqVlGfHZWFU9nuBWlwQew/smrE6PUZZR4tZlXbtk6CUjwx22bN1ldNX+i9gYyBOajc= X-Google-Smtp-Source: AGHT+IEI7jiLjIgZysIlEX73yDDpgFYYpjbOqnnBoDztE8NoLaevmNECfxbufOxFIll7IK31jTg9iA== X-Received: by 2002:a17:907:7f22:b0:aa6:7c36:3423 with SMTP id a640c23a62f3a-aac2d4472f0mr5322819166b.1.1736242915603; Tue, 07 Jan 2025 01:41:55 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aac0f012229sm2356901166b.133.2025.01.07.01.41.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jan 2025 01:41:55 -0800 (PST) From: Linus Walleij Date: Tue, 07 Jan 2025 10:41:38 +0100 Subject: [PATCH RFC v3 22/30] ARM: entry: Move work processing to C MIME-Version: 1.0 Message-Id: <20250107-arm-generic-entry-v3-22-4e5f3c15db2d@linaro.org> References: <20250107-arm-generic-entry-v3-0-4e5f3c15db2d@linaro.org> In-Reply-To: <20250107-arm-generic-entry-v3-0-4e5f3c15db2d@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250107_014157_207829_E0D76483 X-CRM114-Status: GOOD ( 14.25 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org At the end of a syscall software interrupt and every time we exit an interrupt handler we check if there is work to do, i.e. if the current thread (userspace or kernel daemon) has set the _TIF_SYSCALL_WORK flag. Move this processing over to the C entry code. Signed-off-by: Linus Walleij --- arch/arm/kernel/entry-common.S | 14 +------------- arch/arm/kernel/entry.c | 10 ++++++++++ 2 files changed, 11 insertions(+), 13 deletions(-) diff --git a/arch/arm/kernel/entry-common.S b/arch/arm/kernel/entry-common.S index 33bc9e7c5b32f62474724f1d47f97af24dba53bf..5385574c4339714f3d8b5ab6894f27466b89f932 100644 --- a/arch/arm/kernel/entry-common.S +++ b/arch/arm/kernel/entry-common.S @@ -42,19 +42,7 @@ ret_fast_syscall: /* do_rseq_syscall needs interrupts enabled. */ mov r0, sp @ 'regs' bl syscall_exit_to_user_mode - ldr r1, [tsk, #TI_FLAGS] @ re-check for syscall tracing - movs r1, r1, lsl #16 - beq 2f - - tst r1, #_TIF_SYSCALL_WORK - beq 1f - - b 2f - -1: mov r0, sp @ 'regs' - bl do_work_pending - -2: asm_irqentry_exit_to_user_mode + asm_irqentry_exit_to_user_mode #ifdef CONFIG_GCC_PLUGIN_STACKLEAK bl stackleak_erase_on_task_stack diff --git a/arch/arm/kernel/entry.c b/arch/arm/kernel/entry.c index 47233804733752aef0f2fec6f54badeab18e98d0..0e3960844b94b14325eb00ce7dd791b36c43ac3c 100644 --- a/arch/arm/kernel/entry.c +++ b/arch/arm/kernel/entry.c @@ -1,10 +1,16 @@ // SPDX-License-Identifier: GPL-2.0 #include +#include #include #include #include #include +static inline bool has_syscall_work(unsigned long flags) +{ + return unlikely(flags & _TIF_SYSCALL_WORK); +} + long syscall_enter_from_user_mode(struct pt_regs *regs, long syscall) { trace_hardirqs_on(); @@ -18,8 +24,12 @@ long syscall_enter_from_user_mode(struct pt_regs *regs, long syscall) void syscall_exit_to_user_mode(struct pt_regs *regs) { + unsigned long flags = read_thread_flags(); + rseq_syscall(regs); local_irq_disable(); + if (has_syscall_work(flags)) + do_work_pending(regs, flags); } noinstr void irqentry_enter_from_user_mode(struct pt_regs *regs) From patchwork Tue Jan 7 09:41:39 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13928612 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 8CCC9E77197 for ; Tue, 7 Jan 2025 10:11:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=1fSpPlBqZE2GxPZ8aqxBgZ4u22dj0xDyR7PMf5d8jHM=; b=RoZThgQo+61T6qRZfCvi1agKc6 NqeFMq4RGRG8mBL0erTvR35/tZqddTo4D93O2qaoqIzTCJ8OSiazYYq2bPywWbOrBYSv++08/9c2c 5bYdtd/SBCFfORQ5MbPP0wEo5/BvYl8tOXoW5tJokTGck0l1kBWU9aTkd8NuxLnvo0sOP81MZQkC0 c5bXPKx2dntyL1QynphE5keeMMAmxcuSPw0a6HY04XJEOa5BV1fkp1jKMKcKe0OjrxIDgA4/jRqIR uG1D1Jm9PPMKC0sOq9RXSu30AIzfe1HAZQPkDhB3pPk5R1/OjlNFBdwjvPa5UZ7PfIgU6J6+fVnKP wC5xRkTA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tV6Ya-00000004HX9-0M5o; Tue, 07 Jan 2025 10:11:28 +0000 Received: from mail-ed1-x52f.google.com ([2a00:1450:4864:20::52f]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tV662-00000004AZK-1QYz for linux-arm-kernel@lists.infradead.org; Tue, 07 Jan 2025 09:42:05 +0000 Received: by mail-ed1-x52f.google.com with SMTP id 4fb4d7f45d1cf-5d3e6f6cf69so26641631a12.1 for ; Tue, 07 Jan 2025 01:41:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1736242917; x=1736847717; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=1fSpPlBqZE2GxPZ8aqxBgZ4u22dj0xDyR7PMf5d8jHM=; b=U8tAgFJyVcFvE2Jfw8uXGV379bRHzHOM5pk7Tfs00hD1dTaOFNfkSjZJkExsdaCi1E 0clVv+ZsR5p7guxvM+SPb0JXpxGVdXxxUSOVbSKuePL5rJOW+/v4cb0Cfl1g+AJ+O9JO 6sy4WP/wHl6ZAVFP7NDo9icUttwxU3/Cwq14adxFIc9upmu51z2CWPoRgyvaLbgDbrZ2 3ZywQFkTQkgCw0sCvMJuFil3MACytoprINEO4Ebyxa0QzF9hWmXWfwpMH4k/h4hLpBCi DYOcf4sQJpC9esISnIT2Req+QIEY24MF0LUUtocTjoVVEILHyv0mWwN7pTFm4eZpsg9f xqHw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736242917; x=1736847717; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=1fSpPlBqZE2GxPZ8aqxBgZ4u22dj0xDyR7PMf5d8jHM=; b=LVhVe9+bshnnuKArgYvAdNyJBfQy9bUBLsK4v70e85YUjkaPSsxxeSfAXEGGLZfo7s D8qiAyVcqCwaEh1HdILRpLaRmLqHjG3wPEP4sUyaXe4IhF/gmLFcssCNjsVF0KpZTnKJ EHe3zk8vMQNSdHPpp2wHXo0MP22AvOf+lgHL+rfrLlGSStmpcHJ3/6gZGMu6uKsTHnMe fiVl5kSmoxb7RHCxE26mJB3RfcveTQleGAgaZimRa7GG1hiy1FJ+q+4HWKhNIwwTyS0G 5YceDJmcE7Y6ppV8/IIcJGV66MzHTELtmCs8UKrnus99Hy3RllDEFxb9zTee3deFq9em 4JBg== X-Gm-Message-State: AOJu0Yyvukf7BgglAvpbWA+Fqk/DctUskTVwDkozFHlRd+JOMSpUXSvz vJnMmGQqAF1HuxrrdMCwFfBIrPos/OKykZW8/IZwYRIE3SSoA8ZPkSa7SQaPk08= X-Gm-Gg: ASbGncth+GHnF6386OXKsWChVsWEKKRS9JoNOqjDY15A25KBAskejeJwcRM36RizUHq ih/duiVFwcnQjXLM1+B2W+GXkGlNYosYPFLhJoriYnxNTecmj8KH4HNL/obhrsKe431fgVtGBYm rBKgxIveuEs8gxvymJIsUmYefO8OIS3uIhzSxwv/xlkY5hxabcPs+2hGHNclXVxNjuijSCAJ7/u MdNkerlXSzbW6MhW1IAEjSqu5+WPk8d/ogNoahLm38I2BaspS6WnKruA8dVq/cu42g= X-Google-Smtp-Source: AGHT+IH/dI1PxCW+yp4QARSJfd4dkKI+h1pdEwCvQ+1RIosRFpfUhuopq0xYZv8q0GQqsFrKty0/6A== X-Received: by 2002:a17:907:96a7:b0:aa6:3223:19e6 with SMTP id a640c23a62f3a-aac345fabc9mr4954124166b.60.1736242916794; Tue, 07 Jan 2025 01:41:56 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aac0f012229sm2356901166b.133.2025.01.07.01.41.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jan 2025 01:41:56 -0800 (PST) From: Linus Walleij Date: Tue, 07 Jan 2025 10:41:39 +0100 Subject: [PATCH RFC v3 23/30] ARM: entry: Stop exiting syscalls like IRQs MIME-Version: 1.0 Message-Id: <20250107-arm-generic-entry-v3-23-4e5f3c15db2d@linaro.org> References: <20250107-arm-generic-entry-v3-0-4e5f3c15db2d@linaro.org> In-Reply-To: <20250107-arm-generic-entry-v3-0-4e5f3c15db2d@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250107_014158_382834_B7B9A0A2 X-CRM114-Status: GOOD ( 14.87 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The syscalls are issued as software interrupts, and to this point they were sharing code with the other exception handlers, but this does not work with generic entry. Make syscall_exit_to_user_mode do what irqentry_exit_to_user_mode does at it's tail, and drop the invocations of irqentry_exit_to_user_mode from the syscall path so that these are now exception-exclusive. Split ret_to_user and ret_to_user_from_irq into two distinct execution paths. Signed-off-by: Linus Walleij --- arch/arm/kernel/entry-common.S | 14 ++++++++++++-- arch/arm/kernel/entry.c | 4 ++++ 2 files changed, 16 insertions(+), 2 deletions(-) diff --git a/arch/arm/kernel/entry-common.S b/arch/arm/kernel/entry-common.S index 5385574c4339714f3d8b5ab6894f27466b89f932..e2ac6d3216b6828d948d60c424ff95ccdad12331 100644 --- a/arch/arm/kernel/entry-common.S +++ b/arch/arm/kernel/entry-common.S @@ -42,7 +42,6 @@ ret_fast_syscall: /* do_rseq_syscall needs interrupts enabled. */ mov r0, sp @ 'regs' bl syscall_exit_to_user_mode - asm_irqentry_exit_to_user_mode #ifdef CONFIG_GCC_PLUGIN_STACKLEAK bl stackleak_erase_on_task_stack @@ -62,6 +61,18 @@ ENTRY(ret_to_user) enable_irq_notrace @ enable interrupts mov r0, sp @ 'regs' bl syscall_exit_to_user_mode + ldr r1, [tsk, #TI_FLAGS] + movs r1, r1, lsl #16 + beq 1f + mov r0, sp @ 'regs' + bl do_work_pending +1: +#ifdef CONFIG_GCC_PLUGIN_STACKLEAK + bl stackleak_erase_on_task_stack +#endif + restore_user_regs +ENDPROC(ret_to_user) + ENTRY(ret_to_user_from_irq) ldr r1, [tsk, #TI_FLAGS] movs r1, r1, lsl #16 @@ -76,7 +87,6 @@ no_work_pending: #endif restore_user_regs ENDPROC(ret_to_user_from_irq) -ENDPROC(ret_to_user) /* * This is how we return from a fork. diff --git a/arch/arm/kernel/entry.c b/arch/arm/kernel/entry.c index 0e3960844b94b14325eb00ce7dd791b36c43ac3c..88a7a699306ab61153d57333c26266b9161f5466 100644 --- a/arch/arm/kernel/entry.c +++ b/arch/arm/kernel/entry.c @@ -30,6 +30,10 @@ void syscall_exit_to_user_mode(struct pt_regs *regs) local_irq_disable(); if (has_syscall_work(flags)) do_work_pending(regs, flags); + + trace_hardirqs_on(); + /* This context tracking call has inverse naming */ + user_enter_callable(); } noinstr void irqentry_enter_from_user_mode(struct pt_regs *regs) From patchwork Tue Jan 7 09:41:40 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13928611 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3E86FE77197 for ; Tue, 7 Jan 2025 10:10:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=xYkNHQuv+SACgxNERp8CxEKakLXgFnXCL7ekBiageuE=; b=bACB8ejhhbaymWgzPvm/E1CTm4 T+Yu0ZsRTp77n3pWV41GVJRH5ULol9mkR1+NzxAtgDCs7xJTm++3DvuXgJHLDQ3Xsfqa5ULmVs6Oc WjDMcbh81iq/wu2rIJTKThxWbyvBnvaWrZTzJw+bV2WlXtbFJIBr8Lho85WbIE2yhBjuU/r4FhJ/4 +N0JV+jZ3gGMUtallCphjhXhZcvNWXkFAWknSVN3RkAtqNWPRrN0Q4BQo7WGJQnd1cKprmCWM1GRb 4e4MSo9XfJvrhLQyx5Zhmq+UDXpwwQGXR1GUKsDXBsUH7eHUaebXv3i+cXYbfqU8nxCuRW6q6rJAR 0ewW+Cgg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tV6XP-00000004HEh-1rxL; Tue, 07 Jan 2025 10:10:15 +0000 Received: from mail-ej1-x62f.google.com ([2a00:1450:4864:20::62f]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tV663-00000004AZj-1Tpa for linux-arm-kernel@lists.infradead.org; Tue, 07 Jan 2025 09:42:05 +0000 Received: by mail-ej1-x62f.google.com with SMTP id a640c23a62f3a-aaec111762bso2491465366b.2 for ; Tue, 07 Jan 2025 01:41:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1736242918; x=1736847718; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=xYkNHQuv+SACgxNERp8CxEKakLXgFnXCL7ekBiageuE=; b=kikjV6ALzeytR7AyeKXaG2GffuEYppdoxSAUfyEMdUkDjrQMTaFvcCLe+T3SxRPR2z btohd0+hd+3z9wx55aQkdvIFM2VH8worEIoFgXT9UoeIEVt9TD4ASwX6E2MyRC0pw3Ki bDTKOSPeI69pdtV1T8mZBzcZDb5+WgMNSIAV7kEznF+Ievuh7B6RlRchLwShvFYRMXLE LPmHTW8jO+/908HfUtGP6NtF9MP2JuZFuNSVLIus9nmsQEm/unmDywrfattyAZD0JVgk 7ktRHAFdXnL6T2jV388yiiS3kIpRPfsc4ykQpTCGOxds1oxdj/bWuFsKelfuVplLBqEi UXXw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736242918; x=1736847718; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=xYkNHQuv+SACgxNERp8CxEKakLXgFnXCL7ekBiageuE=; b=cASyRBSSb7i3YCaajwyFEyPKCfL5cw2Z+eurKCFaF8vEww4gFJklxDsVFnWZlgfB3m m3kX/GMsw0RWVmOom+XkF2A7dBXDLl1TUpBbC80XOuozoxxp7G/j81xYt3nFAJKFvxhA oYpL0vivitah9r8I5Brot92fD4IqX8BKoxg5BP0LcnU44Kbbi25ZxL2OxeHNStxkXG+P kt9GoMZ7TGGsHJxrtIT93GdxsP9Eoa9gAn+Fr5lowXLWTMhLJWAqjxMqE7WOvi2y8lMX s+96YD3jECu6XUAcXBKeL44ePKuZ4eKedKkqZ8nC4hKiPolzMfFesiyCYRthsA0Gl5Zk ZNjw== X-Gm-Message-State: AOJu0YzEfgNvZrPNHo8IEQ6ctq0O5rGlNKKcizdjauVr+igSPIkbgcHf Xtvx3M9/ohw7bwQZAC4HQrBt411oXi7pXoWn44mISw99rzDfseWSkdncML3wHs4= X-Gm-Gg: ASbGncvjoKE1BJmPLnEcjSz7gAU2+rdOKL84cnXiApvG67rS1TNPGtfB0VtLZyG6Unf NX/FqnqRyZYzZ1VrnKosnhWYzmrJ85Zegz8Sr7jS6aUpjY9T+eMz9vJVnfZXJxKhmQzp2Fex5cM 6zj1kuRPeArrmUoi7JbI3FzBonWd8hmRai0Ukw5xJSOknemxCnbXC14bea6XIpeKrbsk4jTgVql DewFky7fprOMfmw1ZaseCw0z0iP/+8VCat77D3lwa7te6WN/FUv5qAE4qFsUo58tzI= X-Google-Smtp-Source: AGHT+IGS1h0tqhv8PFF2VyP7s0fGDYjnRN6TwLkjLD1tgNRKoA7OPaiv3ifCCBh2WK52GnioqRk31A== X-Received: by 2002:a17:907:968a:b0:aab:f014:fc9a with SMTP id a640c23a62f3a-aac2b383cb3mr4464284266b.22.1736242917900; Tue, 07 Jan 2025 01:41:57 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aac0f012229sm2356901166b.133.2025.01.07.01.41.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jan 2025 01:41:57 -0800 (PST) From: Linus Walleij Date: Tue, 07 Jan 2025 10:41:40 +0100 Subject: [PATCH RFC v3 24/30] ARM: entry: Complete syscall and IRQ transition to C MIME-Version: 1.0 Message-Id: <20250107-arm-generic-entry-v3-24-4e5f3c15db2d@linaro.org> References: <20250107-arm-generic-entry-v3-0-4e5f3c15db2d@linaro.org> In-Reply-To: <20250107-arm-generic-entry-v3-0-4e5f3c15db2d@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250107_014159_394954_6CE94316 X-CRM114-Status: GOOD ( 17.21 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org This moves over the last few lines of assembly to C. The subtle change is that in return to userspace from syscall (SWI) or interrupt, we need to call do_work_pending() as soon as the thread flags are != 0, just checking for work with _TIF_SYSCALL_WORK is not enough (the machine will freeze if we do that). This is because do_work_pending() does not just handle work: it handles _TIF_NEED_RESCHED, _TIF_SIGPENDING, _TIF_NOTIFY_SIGNAL and _TIF_UPROBE as well. Signed-off-by: Linus Walleij --- arch/arm/kernel/entry-common.S | 13 +------------ arch/arm/kernel/entry.c | 19 +++++++++++++------ arch/arm/kernel/signal.c | 3 +-- 3 files changed, 15 insertions(+), 20 deletions(-) diff --git a/arch/arm/kernel/entry-common.S b/arch/arm/kernel/entry-common.S index e2ac6d3216b6828d948d60c424ff95ccdad12331..6b0f86786a7d9e656ecbe8c741f2edbcbafd2915 100644 --- a/arch/arm/kernel/entry-common.S +++ b/arch/arm/kernel/entry-common.S @@ -61,12 +61,7 @@ ENTRY(ret_to_user) enable_irq_notrace @ enable interrupts mov r0, sp @ 'regs' bl syscall_exit_to_user_mode - ldr r1, [tsk, #TI_FLAGS] - movs r1, r1, lsl #16 - beq 1f - mov r0, sp @ 'regs' - bl do_work_pending -1: + #ifdef CONFIG_GCC_PLUGIN_STACKLEAK bl stackleak_erase_on_task_stack #endif @@ -74,12 +69,6 @@ ENTRY(ret_to_user) ENDPROC(ret_to_user) ENTRY(ret_to_user_from_irq) - ldr r1, [tsk, #TI_FLAGS] - movs r1, r1, lsl #16 - beq no_work_pending - mov r0, sp @ 'regs' - bl do_work_pending -no_work_pending: asm_irqentry_exit_to_user_mode #ifdef CONFIG_GCC_PLUGIN_STACKLEAK diff --git a/arch/arm/kernel/entry.c b/arch/arm/kernel/entry.c index 88a7a699306ab61153d57333c26266b9161f5466..d7fdb9df33312dd2a914b59512b701748f33b597 100644 --- a/arch/arm/kernel/entry.c +++ b/arch/arm/kernel/entry.c @@ -6,11 +6,6 @@ #include #include -static inline bool has_syscall_work(unsigned long flags) -{ - return unlikely(flags & _TIF_SYSCALL_WORK); -} - long syscall_enter_from_user_mode(struct pt_regs *regs, long syscall) { trace_hardirqs_on(); @@ -28,7 +23,11 @@ void syscall_exit_to_user_mode(struct pt_regs *regs) rseq_syscall(regs); local_irq_disable(); - if (has_syscall_work(flags)) + /* + * It really matters that we check for flags != 0 and not + * just for pending work here! + */ + if (flags) do_work_pending(regs, flags); trace_hardirqs_on(); @@ -45,6 +44,14 @@ noinstr void irqentry_enter_from_user_mode(struct pt_regs *regs) noinstr void irqentry_exit_to_user_mode(struct pt_regs *regs) { + unsigned long flags = read_thread_flags(); + + /* + * It really matters that we check for flags != 0 and not + * just for pending work here! + */ + if (flags) + do_work_pending(regs, flags); trace_hardirqs_on(); /* This context tracking call has inverse naming */ user_enter_callable(); diff --git a/arch/arm/kernel/signal.c b/arch/arm/kernel/signal.c index 03b20637a2e1ca6f327223c9132915a97a9c4697..a39ee14ec5b70b3738ecef537709326ef1e8217c 100644 --- a/arch/arm/kernel/signal.c +++ b/arch/arm/kernel/signal.c @@ -598,8 +598,7 @@ static void arch_do_signal_or_restart(struct pt_regs *regs) return; } -asmlinkage void -do_work_pending(struct pt_regs *regs, unsigned int thread_flags) +void do_work_pending(struct pt_regs *regs, unsigned int thread_flags) { /* * The assembly code enters us with IRQs off, but it hasn't From patchwork Tue Jan 7 09:41:41 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13928613 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 7CC0EE77197 for ; Tue, 7 Jan 2025 10:12:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=GA5CIdQ0lbXH4n0pAtvx2LNmm2vuWTvTqeYMmiBEiDg=; b=QEXC2SERJTk5Q71KzM94FZ69C1 wLBEhSMJM8Il/Lvj6SKIZ4SmgKn51ejDLRGuNWYooVd4c31liqdNgaVZorsHL4dMzZCKDJ49F7yn3 iUdx6NeAaK1RQrQuzUUB7l1rJJnPL7Q8aYOkYxJQMxtbpCRllFsVwAOg0bf7Sjyd/0uUxKYv8FsKx AkkisEQxSzLqgV3eNlNV2Pc3qBZmAnz8u9OP1zLAuFyD5qCRG8CcKVWUZ2YwRjYCvClgu/DvkeMTC O0GdU/zMB7NNVi6EWHwZPlREvmT3w8zWJh83nzSaHOrExqHePeikilX9EuUcQWvBUrnQjUGrdGeCx KG6pZcAg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tV6Zj-00000004HoM-31h8; Tue, 07 Jan 2025 10:12:40 +0000 Received: from mail-ed1-x52e.google.com ([2a00:1450:4864:20::52e]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tV664-00000004AaD-30h2 for linux-arm-kernel@lists.infradead.org; Tue, 07 Jan 2025 09:42:05 +0000 Received: by mail-ed1-x52e.google.com with SMTP id 4fb4d7f45d1cf-5d34030ebb2so10625462a12.1 for ; Tue, 07 Jan 2025 01:42:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1736242919; x=1736847719; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=GA5CIdQ0lbXH4n0pAtvx2LNmm2vuWTvTqeYMmiBEiDg=; b=CfZPghWdRszWcNikseFar2d63ja4wsvUk8S42VJ6FEmd4aCAmU4cINrcuLzb0lkL+f u9qie0Pp6BRFksTHwxQbvH4clWFGbKCA0n/9wRH1zuViXXsCb2If+xuG48n3jhVlSlsO GhNBsmogPGeKlZFsW+4E121ZlZDFS9jZiezT1lWGeSi/eAbSfSK5DKM7fjkCGpUcZj4Q iFEZvgEZa4mxXv2O/L1eTNTfq4n7TxqxLPyc90KV7rQQxUt0YbKltlDGzTUInrvBUVzq T7nXVydnyDKWjDWEyYwmaWl2uBdkMF1f0+irSUrq1u4Sz0iXj5ZogA+3gpmCxXf3NSiB T5tQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736242919; x=1736847719; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=GA5CIdQ0lbXH4n0pAtvx2LNmm2vuWTvTqeYMmiBEiDg=; b=JQ6x4ENnsIwUplDYtZPAEA/yyvyfqQz73bDaETh8vvN+Jj+obCceJax/IPUy/lmxnh vrDR+Baxykvgfq9E4EwC+hMxkZyGtnwy2YqrfJwpC8sjxTyAx9aTSSngYQQSMy3I8roB pHHtId5yMPW6EMtKfNBReXEJerKBaVdt5/h8bM16ZBQ4+3JMNgkfsSmzAL0v8ZtsU1kD 8mGY0dgOFjC6lCQpvAuyucWZBoogbbbHoTspQ0yapIXe4Aep1C1O0Z7eNJIO6x5t1hmz jzN3AUd6bOopfZOF+0WvGod9OiK1GP5V7MLtGFNxNz/vL+3sKc0xq9+wOD8Oe46/vNIn v5aw== X-Gm-Message-State: AOJu0YwQzbzFuHZF7G4wU3wlGgY2937veyxzU1oGpwOUx056M57brnD4 Qh8js4Y9hqn+wbqapedUAQqqnL7/FoAZQADRylOjKouuuF7S1Nlw3qry5kEF84I= X-Gm-Gg: ASbGncsYEMQd87FWZVAcTghtb92z+FnnNSUzpIqAAyq4bn4UgXyhdLOuW0KKwFDX4eb tUeOP7ZEwHksHFUM/Xju+LONnm9Ih4XO6XAMAacLJ/i3oCM/wrLuZzowjPVKs2JeAAq6Mlu+xmX tOEQfJ5pmygp4/B/4RjdSMPvAW0dD9fpOCptkmPkYOGSwnmSuJB9+loEMnv3ng8w9TAW+FCchia +pcJNUpJaHSg4ACLxh3uTG7tMReqo4JC+Mv6jFOfeC6/zOBkZRUe65fRe2G4btsYUY= X-Google-Smtp-Source: AGHT+IHjNcm3SJgRDE9sTXW5p/Q+t40ZGJWcXhVcRyYilmutwpMo3+fZgXXoT+2r7xFg0aaeFgkyQA== X-Received: by 2002:a17:907:72ce:b0:aa6:25c6:d94f with SMTP id a640c23a62f3a-aac2ba3f469mr5899210166b.31.1736242919089; Tue, 07 Jan 2025 01:41:59 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aac0f012229sm2356901166b.133.2025.01.07.01.41.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jan 2025 01:41:58 -0800 (PST) From: Linus Walleij Date: Tue, 07 Jan 2025 10:41:41 +0100 Subject: [PATCH RFC v3 25/30] ARM: entry: Create irqentry calls from kernel mode MIME-Version: 1.0 Message-Id: <20250107-arm-generic-entry-v3-25-4e5f3c15db2d@linaro.org> References: <20250107-arm-generic-entry-v3-0-4e5f3c15db2d@linaro.org> In-Reply-To: <20250107-arm-generic-entry-v3-0-4e5f3c15db2d@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250107_014200_764996_756599E0 X-CRM114-Status: GOOD ( 14.07 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org This adds a few calls to C when exceptions enter/exit from the kernel itself, so these can be used for generic entry. Signed-off-by: Linus Walleij --- arch/arm/include/asm/entry.h | 2 ++ arch/arm/kernel/entry-armv.S | 3 +++ arch/arm/kernel/entry-header.S | 5 +++++ arch/arm/kernel/entry.c | 8 ++++++++ 4 files changed, 18 insertions(+) diff --git a/arch/arm/include/asm/entry.h b/arch/arm/include/asm/entry.h index 8b8cc9a1eefbd00dd75d4e344576fc184a413768..a78bc5054b093120e903f08369fb8b1a9feb7f81 100644 --- a/arch/arm/include/asm/entry.h +++ b/arch/arm/include/asm/entry.h @@ -12,5 +12,7 @@ long syscall_enter_from_user_mode(struct pt_regs *regs, long); void syscall_exit_to_user_mode(struct pt_regs *regs); void irqentry_enter_from_user_mode(struct pt_regs *regs); void irqentry_exit_to_user_mode(struct pt_regs *regs); +void irqentry_enter_from_kernel_mode(struct pt_regs *regs); +void irqentry_exit_to_kernel_mode(struct pt_regs *regs); #endif /* __ASM_ENTRY_H__ */ diff --git a/arch/arm/kernel/entry-armv.S b/arch/arm/kernel/entry-armv.S index 3cd5a80b47758867d1db6c3971030370bca1551a..f64c4cc8beda6bcd469e6ff1a1f337d52dbbaf9c 100644 --- a/arch/arm/kernel/entry-armv.S +++ b/arch/arm/kernel/entry-armv.S @@ -205,6 +205,9 @@ ENDPROC(__und_invalid) get_thread_info tsk uaccess_entry tsk, r0, r1, r2, \uaccess + mov r0, sp @ 'regs' + bl irqentry_enter_from_kernel_mode + .if \trace #ifdef CONFIG_TRACE_IRQFLAGS bl trace_hardirqs_off diff --git a/arch/arm/kernel/entry-header.S b/arch/arm/kernel/entry-header.S index 51c7f93c1ded1db56f173c7aaed8094ef011ff5a..49a9c5cf6fd5fbb917f2ada6c0d6cc400b7d3fb3 100644 --- a/arch/arm/kernel/entry-header.S +++ b/arch/arm/kernel/entry-header.S @@ -210,6 +210,7 @@ .else @ IRQs off again before pulling preserved data off the stack disable_irq_notrace + #ifdef CONFIG_TRACE_IRQFLAGS tst \rpsr, #PSR_I_BIT bleq trace_hardirqs_on @@ -217,6 +218,10 @@ blne trace_hardirqs_off #endif .endif + + mov r0, sp @ 'regs' + bl irqentry_exit_to_kernel_mode + uaccess_exit tsk, r0, r1 #ifndef CONFIG_THUMB2_KERNEL diff --git a/arch/arm/kernel/entry.c b/arch/arm/kernel/entry.c index d7fdb9df33312dd2a914b59512b701748f33b597..674b5adcec0001b7d075d6936bfb4e318cb7ce74 100644 --- a/arch/arm/kernel/entry.c +++ b/arch/arm/kernel/entry.c @@ -56,3 +56,11 @@ noinstr void irqentry_exit_to_user_mode(struct pt_regs *regs) /* This context tracking call has inverse naming */ user_enter_callable(); } + +noinstr void irqentry_enter_from_kernel_mode(struct pt_regs *regs) +{ +} + +noinstr void irqentry_exit_to_kernel_mode(struct pt_regs *regs) +{ +} From patchwork Tue Jan 7 09:41:42 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13928614 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6EACAE77198 for ; Tue, 7 Jan 2025 10:14:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=JMT4PCSSjbqf8W79ja5QYvNX8ZzwiayCa6wY1VSVH3k=; b=sxo5+slIFzEBjmFnwNjdN5wYBf SLUNEHAP4v9KdgSLDo+Y5c9UPZgXhkX0AaIMd4SPTtp3yhbJxjfU8fDMwOnx6+2tC9edWD50/w/Vw 0MYzYDh40dMjdvbA1jn9jTABQK6dIiTqgjsWXqKT0i69YbuDt3V+efJBNe1MFEoTOzgx8XhHeRNIu SEwoHUi9VlPQLS9ioaJN/UXtb1NTHIQtPKIQ6/tjsRYNTIfDWQz2i80avaphiIHo5nzr3lnRFBUGW ZgSoXVMirsWpC0uQ6NyDIpAvc4RsBfwVQJGG0TACjgDrS+TkHePLpvDoz8di2wg79wsvhI9yen8F5 i0zODh6A==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tV6at-00000004HyC-0pu2; Tue, 07 Jan 2025 10:13:51 +0000 Received: from mail-ed1-x52d.google.com ([2a00:1450:4864:20::52d]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tV666-00000004Aaj-1AEZ for linux-arm-kernel@lists.infradead.org; Tue, 07 Jan 2025 09:42:08 +0000 Received: by mail-ed1-x52d.google.com with SMTP id 4fb4d7f45d1cf-5d88c355e0dso8521639a12.0 for ; Tue, 07 Jan 2025 01:42:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1736242921; x=1736847721; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=JMT4PCSSjbqf8W79ja5QYvNX8ZzwiayCa6wY1VSVH3k=; b=tfxeczEPukmpdl7JYSEqXS6r1MkRIMn1cLfMW8MrCVZeu2xXP8q8TaRsQE0VDjyY1G 2X7uYCnyd2r0yiNMPJO98U9F6Tihq6OvMao1B57NEWydTlJrJnh2tYKwmgyTT1xz10IR hjPI6IrwS3zGfSU0Lbst3P/PGyqeDHMZTiqwAudQaw/hRnn6WLGOQvamX2PxEvAC5qAz uZMFUE42gdQ5nyR60LtO+sektg+ZEkrYKi+QDRx7oKepCBKFGPEpRwnv73N/k9n81Gl3 qbwGBbmpoMMF+SSrXxmTerrBqfl2TZAYWXw/03b53jdg0YFzv/uYZrooVmspiFFtYTWk f88A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736242921; x=1736847721; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=JMT4PCSSjbqf8W79ja5QYvNX8ZzwiayCa6wY1VSVH3k=; b=L88GqxAnaj8Y9xY4aTDBH5r9fmpFD+DT7wa3t8rRCeygJu33Qj45e/oNybsam5gUeY Jep5Ads2H92XFZMa+VDyQCcTwWnPpYteoyHH3k3Y313JyD3jFjGTRvVZtMpF38FkG4L+ 1G4Z+fmIB+idxAQxSx/TLgIurFFiu7U6cEe3e17tUPpzNwe9o+DuOXaeK9AAPv6cwUjQ amOFnuzXyPwWRQ5TgAi85EO4JxLapjjoO9dRbOqkC7Opv7HbqjMyNnavrnhs0bvp2Kqu n/huKRK4F36l0lsvx8LkvX30O84YKpkwZgDXbL1XuU8t1EUx9L9Jx8khkcILx/Vm0hES f5GA== X-Gm-Message-State: AOJu0Yzi3kaWhQeN/c9BoD7lI/+er9N+WSkFsZ60Ja13U4cCR73QrVOi Uz+31/giCBHdUZvGjCt0hQr5dqYOZpYKW79B7/5B7/aYKO/bzP7gX3GFUojG2ZM= X-Gm-Gg: ASbGncuR6ZfiyzyhRN2AjPb88puTNnIyzOPMbol48n1XmydlRXKblv9sAoTBFzMe7kS wJZ+NqzKkH7BSjpyxQUHmSymtIfO/qOlPoPbemXZvYLZNEGeOpqaOIMxWvfh/6OS1yVHrxnKmQY ihv8JPVxuEcd5Wp8iBCyMbypzwR9zrzFF0LrzX2kHchqs7FmiQworI+fWTwPjuUnsXy24vsLZho bb7OjeQPP67hpAK2HuCWf3NT6iArDIAra90FICKiFs7ms8QJXWw02P7m/ycsz9KJs8= X-Google-Smtp-Source: AGHT+IGHMqa0/ZiWplVpsWLK3xTllHfRhzv23loJzX9+Mz5N8MCuV6AGOtz4HX0TG5tn5XTLx0fc0A== X-Received: by 2002:a17:907:1c91:b0:aa6:5d30:d976 with SMTP id a640c23a62f3a-aac270278edmr6059701866b.10.1736242920686; Tue, 07 Jan 2025 01:42:00 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aac0f012229sm2356901166b.133.2025.01.07.01.41.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jan 2025 01:41:59 -0800 (PST) From: Linus Walleij Date: Tue, 07 Jan 2025 10:41:42 +0100 Subject: [PATCH RFC v3 26/30] ARM: entry: Move in-kernel hardirq tracing to C MIME-Version: 1.0 Message-Id: <20250107-arm-generic-entry-v3-26-4e5f3c15db2d@linaro.org> References: <20250107-arm-generic-entry-v3-0-4e5f3c15db2d@linaro.org> In-Reply-To: <20250107-arm-generic-entry-v3-0-4e5f3c15db2d@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250107_014202_333877_84826E13 X-CRM114-Status: GOOD ( 14.57 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Move the code tracing hardirqs on/off into the C callbacks for irqentry_enter_from_kernel_mode() and irqentry_exit_to_kernel_mode(). The semantic difference occurred is that we alsways check the PSR_I_BIT to determine if (hard) interrupts were enabled or not. The assembly has a tweak to avoid this if we are exiting an IRQ since it is obvious that IRQs must have been enabled to get there, but for simplicity we just check it for all exceptions. Signed-off-by: Linus Walleij --- arch/arm/kernel/entry-armv.S | 13 ++++--------- arch/arm/kernel/entry-header.S | 19 ++----------------- arch/arm/kernel/entry.c | 5 +++++ 3 files changed, 11 insertions(+), 26 deletions(-) diff --git a/arch/arm/kernel/entry-armv.S b/arch/arm/kernel/entry-armv.S index f64c4cc8beda6bcd469e6ff1a1f337d52dbbaf9c..2a789c8834b93475c32dcb6ba5854e24ddd8d6e9 100644 --- a/arch/arm/kernel/entry-armv.S +++ b/arch/arm/kernel/entry-armv.S @@ -158,7 +158,7 @@ ENDPROC(__und_invalid) #define SPFIX(code...) #endif - .macro svc_entry, stack_hole=0, trace=1, uaccess=1, overflow_check=1 + .macro svc_entry, stack_hole=0 uaccess=1, overflow_check=1 UNWIND(.fnstart ) sub sp, sp, #(SVC_REGS_SIZE + \stack_hole) THUMB( add sp, r1 ) @ get SP in a GPR without @@ -208,11 +208,6 @@ ENDPROC(__und_invalid) mov r0, sp @ 'regs' bl irqentry_enter_from_kernel_mode - .if \trace -#ifdef CONFIG_TRACE_IRQFLAGS - bl trace_hardirqs_off -#endif - .endif .endm .align 5 @@ -239,7 +234,7 @@ __irq_svc: blne svc_preempt #endif - svc_exit r5, irq = 1 @ return from exception + svc_exit r5 @ return from exception UNWIND(.fnend ) ENDPROC(__irq_svc) @@ -303,7 +298,7 @@ ENDPROC(__pabt_svc) .align 5 __fiq_svc: - svc_entry trace=0 + svc_entry mov r0, sp @ struct pt_regs *regs bl handle_fiq_as_nmi svc_exit_via_fiq @@ -321,7 +316,7 @@ ENDPROC(__fiq_svc) @ .align 5 __fiq_abt: - svc_entry trace=0 + svc_entry ARM( msr cpsr_c, #ABT_MODE | PSR_I_BIT | PSR_F_BIT ) THUMB( mov r0, #ABT_MODE | PSR_I_BIT | PSR_F_BIT ) diff --git a/arch/arm/kernel/entry-header.S b/arch/arm/kernel/entry-header.S index 49a9c5cf6fd5fbb917f2ada6c0d6cc400b7d3fb3..cfaf14d71378ba14bbb2a42cd36d48a23838eee1 100644 --- a/arch/arm/kernel/entry-header.S +++ b/arch/arm/kernel/entry-header.S @@ -199,26 +199,11 @@ .endm - .macro svc_exit, rpsr, irq = 0 - .if \irq != 0 - @ IRQs already off -#ifdef CONFIG_TRACE_IRQFLAGS - @ The parent context IRQs must have been enabled to get here in - @ the first place, so there's no point checking the PSR I bit. - bl trace_hardirqs_on -#endif - .else + .macro svc_exit, rpsr + @ IRQs off again before pulling preserved data off the stack disable_irq_notrace -#ifdef CONFIG_TRACE_IRQFLAGS - tst \rpsr, #PSR_I_BIT - bleq trace_hardirqs_on - tst \rpsr, #PSR_I_BIT - blne trace_hardirqs_off -#endif - .endif - mov r0, sp @ 'regs' bl irqentry_exit_to_kernel_mode diff --git a/arch/arm/kernel/entry.c b/arch/arm/kernel/entry.c index 674b5adcec0001b7d075d6936bfb4e318cb7ce74..1e1284cc4caed6e602ce36e812d535e6fe324f34 100644 --- a/arch/arm/kernel/entry.c +++ b/arch/arm/kernel/entry.c @@ -59,8 +59,13 @@ noinstr void irqentry_exit_to_user_mode(struct pt_regs *regs) noinstr void irqentry_enter_from_kernel_mode(struct pt_regs *regs) { + trace_hardirqs_off(); } noinstr void irqentry_exit_to_kernel_mode(struct pt_regs *regs) { + if (interrupts_enabled(regs)) + trace_hardirqs_on(); + else + trace_hardirqs_off(); } From patchwork Tue Jan 7 09:41:43 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13928634 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E9F90E77197 for ; Tue, 7 Jan 2025 10:15:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=33vNghdTGkjvHLzGCYW2qoDNw0mlPHElmOb2schp32s=; b=jxP0IpZi77zUhahgCrxHvJ+TgW 0SfSU8OsKCFTeB5JMfiXQ90FdRROZ0ZGvshLgBbl/n5R1FpkXdsHRNQJgpZ9bAytUHeDoWMpwOyH0 ClvaXpMmkWL/GMSZ3WfNgJrcB9YDReeXJwltLdu3tM7duzjnJjX3hp0SmB/Dw32vBtuartN1G/ov7 fnkTibN/GilQDM4LpMV7V4fd4DJb06QlRXcyjOWhoPScNxHqX1XKJjnv/EHy5KvaH61dt+gcVmjjF qsKxEivNLe3jAqsMJYWi8PmbNbN0bfUh/0kkWIeIxfLHvlGyo+90T995t4P8E3fU0OT64HQtgKRqv 4+XqCYlg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tV6c3-00000004IDI-3aCM; Tue, 07 Jan 2025 10:15:03 +0000 Received: from mail-ej1-x62a.google.com ([2a00:1450:4864:20::62a]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tV667-00000004Abi-1Fjw for linux-arm-kernel@lists.infradead.org; Tue, 07 Jan 2025 09:42:09 +0000 Received: by mail-ej1-x62a.google.com with SMTP id a640c23a62f3a-aa6c0dbce1fso2145121866b.2 for ; Tue, 07 Jan 2025 01:42:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1736242922; x=1736847722; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=33vNghdTGkjvHLzGCYW2qoDNw0mlPHElmOb2schp32s=; b=SfCz5P55KdwepC/u+pO8jtOdVXJnv3SynLCDVlqN00u86adUTwR+BROW3d+3J26AeZ 6jz5GzaMMB1L1wJL9pOSHRgp5N2r1oe5DdDuE35JveC5rewPs44SOyMMmVWLio4KCdti P6io/OmtddukCAonuRDT4DbmKrIyO6wBcPFvqOq7XhQnpuVwqjhfSI2AepsXlbQm+rck eCw1HXmbfHjDUky9GMql/3ybUSTOUgoMHRGMBnW6O9TtfetlfxoDCEZPmiB5lRhVJBFU jI1gDI72r+dIn0PdvYrTkea0QDscjsSnd+uXpexo/gLfzqdzV8PakUF/v4b8crAJYbnp roEA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736242922; x=1736847722; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=33vNghdTGkjvHLzGCYW2qoDNw0mlPHElmOb2schp32s=; b=HPKIQH61dic8g/4sqRTUr/H1z1WxFYaspCFlx/Tgzx3S6yA0f7D9dH8enPS3Y9YzX1 Xfx66AN1Tk4SfXVHTZP0H7oSQOiDHdxHodnTVr0P9F0ifyl5MlXY9/l2kFiKRWxiiSeE PMH+wLB7mIgon/JLzyotokJWXEEy2hgkqsdq0dt4FxefiTXtNhCeYLgtDCUOqhWHkAgQ Ib6js4L6gdOEIWk5IL4dqSvngn31c3mK0G9iLAThLzxJEXwHHPe8Y/9aSX8pyB7xySQN HSSLCqWGEq+uPiwPmbZ8naj7pRhYcfn5/btKaOhsNFUorYenH/P/FgP3RZZ7OlxyJs1s IfIg== X-Gm-Message-State: AOJu0YyksPKgJmnP2lV7v70+8XLBc5UyccTKqtXKj8um4EgHl9ThG32f M3Jy2ung7Mj7xPUROgwgZsJYMfliFE2CAWwVePlSTTyaveAlX+aGs0BUqnbuxVk= X-Gm-Gg: ASbGncuMb3tP7meGOrs7MLxoCYK45fJthVQbXmDdJY+Jrl8xPIY3gTFgJ7ZKpeDfBBS Md8XbZRbrpu1RkE2SjebkKcWFNJQxIYkEnZhyhFMp89UEIdh8+WYy4zO4jv6AVIlG7q91u1erlH IosrXcR/w1fxa+7AtC5twVZXg2mh++WgXMVZenoyOiN09UfvVN9CfuIPQXgB3BfbewCuT2UnjSg pfyCSwBzx9KEWTY+gGaWksOE1+d6b58/wlYJ78CpASa8RFrhM2/TMz4D+sKXxDqXx4= X-Google-Smtp-Source: AGHT+IEXSqMDdLWEVOLm+/ypfhPzotrgNQdcTuiEWGVxTX1sQzs6KiT8mG4S8teEme0iKBSwiJOMbA== X-Received: by 2002:a05:6402:35c2:b0:5d0:ea4f:972f with SMTP id 4fb4d7f45d1cf-5d81dd9af30mr132628782a12.8.1736242921817; Tue, 07 Jan 2025 01:42:01 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aac0f012229sm2356901166b.133.2025.01.07.01.42.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jan 2025 01:42:01 -0800 (PST) From: Linus Walleij Date: Tue, 07 Jan 2025 10:41:43 +0100 Subject: [PATCH RFC v3 27/30] ARM: irq: Add irqstack helpers MIME-Version: 1.0 Message-Id: <20250107-arm-generic-entry-v3-27-4e5f3c15db2d@linaro.org> References: <20250107-arm-generic-entry-v3-0-4e5f3c15db2d@linaro.org> In-Reply-To: <20250107-arm-generic-entry-v3-0-4e5f3c15db2d@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250107_014203_355963_2E2E4985 X-CRM114-Status: GOOD ( 18.92 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Add two helpers that essentially re-implement some IRQ dispatch code in C: on_irq_stack() that checks if a regs context is on the IRQ stack, and call_on_irq_stack() so we can explicitly issue handle_irq() on the IRQ stack from a C program. Cc: Ard Biesheuvel Signed-off-by: Linus Walleij --- arch/arm/kernel/irq.c | 31 +++++++++++++++++++++++++++++++ arch/arm/kernel/irq.h | 3 +++ 2 files changed, 34 insertions(+) diff --git a/arch/arm/kernel/irq.c b/arch/arm/kernel/irq.c index e1993e28a9ecfd80b55b2677253ac582467e6c14..4c1a678b72b326e42c871b994c09d1fa4b0863f6 100644 --- a/arch/arm/kernel/irq.c +++ b/arch/arm/kernel/irq.c @@ -43,6 +43,7 @@ #include #include +#include "irq.h" #include "reboot.h" unsigned long irq_err_count; @@ -71,6 +72,36 @@ static void __init init_irq_stacks(void) } } +/* + * on_irq_stack() - check if a regs context is using the IRQ stack + * @regs: the context to check + * Returns true if regs is using the IRQ stack + */ +bool on_irq_stack(struct pt_regs *regs) +{ + u8 *high = __this_cpu_read(irq_stack_ptr); + u8 *low = high - THREAD_SIZE; + u8 *sp = (u8 *)regs->ARM_sp; + bool on_stack; + + on_stack = (low <= sp && sp <= high); + + if (on_stack && IS_ENABLED(CONFIG_VMAP_STACK)) + /* + * Also check that SP is inside the linear kernel memory + * if using VMAP:ed stacks. + * TODO: Ask Ard to explain why we need to do this + */ + on_stack = (sp > (u8 *)high_memory); + + return on_stack; +} + +void call_on_irq_stack(void (*fn)(void *), void *arg) +{ + call_with_stack(fn, arg, __this_cpu_read(irq_stack_ptr)); +} + #ifdef CONFIG_SOFTIRQ_ON_OWN_STACK static void ____do_softirq(void *arg) { diff --git a/arch/arm/kernel/irq.h b/arch/arm/kernel/irq.h new file mode 100644 index 0000000000000000000000000000000000000000..c5186b67242a2976fe702154ed4f2a22eca51add --- /dev/null +++ b/arch/arm/kernel/irq.h @@ -0,0 +1,3 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +bool on_irq_stack(struct pt_regs *regs); +void call_on_irq_stack(void (*fn)(void *), void *arg); From patchwork Tue Jan 7 09:41:44 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13928635 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id BDED0E77198 for ; Tue, 7 Jan 2025 10:16:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=X0GqWkC04Ij+z0Sk491t2uiJzeYpC3kG6U0zCzcls1U=; b=fr4YECRMcTCujdRr5U+9gMbmcs W6LaLyTd2detQDhNfK38PdZxMFPHx9nzMSDuiZF8K8ZHYh1tWi4042XssA0W0OwLeJM18OhWMLdIF lIZl3KCxoeAFGc0vJQLNxfZsCJXINtjLm4RyPJWs0QHL4evLDzIeqMCC/uCYcHy1q2/OGdXVkvO4h 7jUIg7DQ1lvWVKkq6sN+5218l/v2sZQgSoPco1DS1779G2ZWGMtvx14ExhNFd4ETMwawibMi30rW1 rAa6AaxAz3ccTeyjWEeNxQuXnIpwEiYiHeDnq/8nmEbmDQ6P6TDrgrOMVuxwne9kTRMlJOUTvoeen kN3ktE6A==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tV6dD-00000004IQa-2R84; Tue, 07 Jan 2025 10:16:15 +0000 Received: from mail-ed1-x536.google.com ([2a00:1450:4864:20::536]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tV669-00000004Acl-2FTO for linux-arm-kernel@lists.infradead.org; Tue, 07 Jan 2025 09:42:12 +0000 Received: by mail-ed1-x536.google.com with SMTP id 4fb4d7f45d1cf-5d3e9a88793so2507633a12.1 for ; Tue, 07 Jan 2025 01:42:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1736242924; x=1736847724; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=X0GqWkC04Ij+z0Sk491t2uiJzeYpC3kG6U0zCzcls1U=; b=s+DtAvW3VfaIKk+qH003/dtTBS8/KZuCzX3We0OeKa2Ax+xiRjwIBm1O1zWsB0QPAH NxMWlO8w0wTR/dV89xFox0KVd6T+brGZge36rWa0E75QpFBxV9Ux8Gee4jAcCk1eZFS9 rJbsWzjCUFz3b8BN12P88ecLt3HxdC4bUsv3Xcnbd4vSin8h+zKV6yDemIma94FkFWw2 +AKgaDjfMwX2QQf0ja93kf1D9O1ehI3pEDQIaILNs8sjTXmmpNg/vgQquTMSzkrTbU2a X4vIt1O+xz27IPWddVMJLyK9B2ZwkWdPvQbqUWD0Bhguf8zvsDDEJxrnU2d38GObxakb 8EGw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736242924; x=1736847724; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=X0GqWkC04Ij+z0Sk491t2uiJzeYpC3kG6U0zCzcls1U=; b=weL8k7NQCbfGFDa+hwvVpfLXKwi1SUxfBzior3tncXa71RfxSue13wA+gfsQB9Vz5g ixa5bugyJorZil5cfuYmpsSRNHh0lj/Ug4bHAXNQsre8PdiiFc+Msq10ewg2+QRw7gSP OMHpKKlxHOqgzS1HX/O8cwtyOOAUBqQSity2UtMa56s3vPBNe4cOU/eHIVPJFiK8MGu6 6jsEilOjHZS2Psa9JBE9Cm6/z9SMVwuvZ43Fa9IzHJ3j4xioW579/6SoYwX6FnNc4m8H WtdrbD86muXAWVPdx64fMk8Ggbg4n01K3WWgKzqdpf77rO9JlHoCxDINW+2+rHiT2XvX PMdA== X-Gm-Message-State: AOJu0YxKhQnxXTj+u2iaNMs8ge0Z/WD1k/aRL5UCUzZEgGEhpHqSIznU dQBlbAOD/N8MZlDhiowKUBy7Td8s2NG30siV2wDtSG59vSTQ6Q5f0YynKwA+W6M= X-Gm-Gg: ASbGncvykvIAYxic+1rX//xAwSuBRTkqo1ErZfY6nPIYDdsnFnfbbuOmYYMLPzoEFIJ 8ZgA6Fyxh/QgnV97oN/8nxu3yihQiZtFIo+ltKB6lthWdZHAlqv2YFOEDc8tGYPFY2ZF4ayrIuD yGbPFoitL/Shw2ZLCLR8qoHsE/GoCN8h9xz3GOxYtdyfAqPohheyc0LHFreLqtoauVhDmoBNfYK kPRXNeM76UtYI7DSeE51JUPSwLh2jyrsbuLEYL/eUrvY6rg1qlcGBgX5PIUFA7xedA= X-Google-Smtp-Source: AGHT+IG5LvSzQR0OLaYYCvZwsUk4tV7TB7tsqGlzcR8FTjXXaPVimVUQVz/2PvJNCKKaTEqUQoYPRQ== X-Received: by 2002:a17:907:3607:b0:aa6:88f5:5fef with SMTP id a640c23a62f3a-aac2c9948d6mr6249682166b.32.1736242923527; Tue, 07 Jan 2025 01:42:03 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aac0f012229sm2356901166b.133.2025.01.07.01.42.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jan 2025 01:42:02 -0800 (PST) From: Linus Walleij Date: Tue, 07 Jan 2025 10:41:44 +0100 Subject: [PATCH RFC v3 28/30] ARM: entry: Convert to generic entry MIME-Version: 1.0 Message-Id: <20250107-arm-generic-entry-v3-28-4e5f3c15db2d@linaro.org> References: <20250107-arm-generic-entry-v3-0-4e5f3c15db2d@linaro.org> In-Reply-To: <20250107-arm-generic-entry-v3-0-4e5f3c15db2d@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250107_014205_637603_2EF78AC1 X-CRM114-Status: GOOD ( 30.41 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org This rewrites ARM to use the generic entry. All of the irqentry_* callbacks are collapsed and made to call into the generic entry code that handle the interrupts storing regs context before entry and restoring it after. The syscall_enter_from_user_mode(), syscall_exit_to_user_mode(), do_work_pending() constructs are replaced with generic code. The syscall invocation now enables syscall tracing "work" using the generic callbacks and no extra actions are needed to enable/disable syscall tracing. The old context tracking with user_exit_callable() user_enter_callable() is no longer used. Only CSKY and Xtensa use this deprecated mechanism now. handle_fiq_as_nmi() is retired as the generic entry provides the same functionality in irqentry_nmi_enter() and irqentry_nmi_exit(). Like with the ARM64 proposed patch for generic entry, it is necessary to create a small call "arm_exit_to_user_mode()" that is used for things such as ret_from_fork, i.e. when we create a completely new execution context with a botched return from interrupt. The elaborate checking and switching to IRQ stack in the irq_handler macro was reimplemented in C for this exercise: this was easiest and there was already code making use of the IRQ stack from C for soft interrupts. I have dropped the context tracking (i.e. calling irqentry_enter() and looking for trace conditions) on dabt/pabt i.e. do_DataAbort and do_PrefetchAbort code paths. These are addressed in a separate patch for development/review reasons. Open questions: - I had to add defines for PTRACE_SYSEMU and PTRACE_SYSEMU_SINGLESTEP and I have no idea what those are or why they are required. Sorry for my ignorance, I need help to figure out how we deal with this. Signed-off-by: Linus Walleij --- arch/arm/Kconfig | 1 + arch/arm/include/asm/entry-common.h | 66 +++++++++++++++++++++++++++ arch/arm/include/asm/entry.h | 13 ++---- arch/arm/include/asm/ptrace.h | 3 -- arch/arm/include/asm/signal.h | 2 - arch/arm/include/asm/syscall.h | 7 ++- arch/arm/include/asm/thread_info.h | 17 +------ arch/arm/include/asm/traps.h | 1 - arch/arm/include/uapi/asm/ptrace.h | 2 + arch/arm/kernel/entry-armv.S | 50 ++++---------------- arch/arm/kernel/entry-common.S | 34 +++----------- arch/arm/kernel/entry-header.S | 18 -------- arch/arm/kernel/entry.c | 91 ++++++++++++++++++------------------- arch/arm/kernel/process.c | 5 +- arch/arm/kernel/ptrace.c | 90 ------------------------------------ arch/arm/kernel/signal.c | 31 +------------ arch/arm/kernel/syscall.c | 27 +++-------- arch/arm/kernel/traps.c | 25 ---------- 18 files changed, 149 insertions(+), 334 deletions(-) diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig index 202397be76d8037b531b34dee16c7dfcfd0124ef..7c9b7255e940f9cf08647c3228b7165795781a4b 100644 --- a/arch/arm/Kconfig +++ b/arch/arm/Kconfig @@ -66,6 +66,7 @@ config ARM select GENERIC_CPU_AUTOPROBE select GENERIC_CPU_DEVICES select GENERIC_EARLY_IOREMAP + select GENERIC_ENTRY select GENERIC_IDLE_POLL_SETUP select GENERIC_IRQ_MULTI_HANDLER select GENERIC_IRQ_PROBE diff --git a/arch/arm/include/asm/entry-common.h b/arch/arm/include/asm/entry-common.h new file mode 100644 index 0000000000000000000000000000000000000000..7b1f108512db0001a65911af0df4f48197d99e75 --- /dev/null +++ b/arch/arm/include/asm/entry-common.h @@ -0,0 +1,66 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +#ifndef _ASM_ARM_ENTRY_COMMON_H +#define _ASM_ARM_ENTRY_COMMON_H + +#include + +#include + +enum ptrace_syscall_dir { + PTRACE_SYSCALL_ENTER = 0, + PTRACE_SYSCALL_EXIT, +}; + +static inline unsigned long +arch_prepare_report_syscall_entry(struct pt_regs *regs) +{ + unsigned long ip; + + /* + * IP is used to denote syscall entry/exit: + * IP = 0 -> entry + */ + ip = regs->ARM_ip; + regs->ARM_ip = PTRACE_SYSCALL_ENTER; + + return ip; +} +#define arch_prepare_report_syscall_entry arch_prepare_report_syscall_entry + +static inline void +arch_post_report_syscall_entry(struct pt_regs *regs, + unsigned long saved_reg, long ret) +{ + regs->ARM_ip = saved_reg; +} +#define arch_post_report_syscall_entry arch_post_report_syscall_entry + + +static inline unsigned long +arch_prepare_report_syscall_exit(struct pt_regs *regs, + unsigned long work) +{ + unsigned long ip; + + /* + * IP is used to denote syscall entry/exit: + * IP = 1 -> exit + */ + ip = regs->ARM_ip; + regs->ARM_ip = PTRACE_SYSCALL_EXIT; + + return ip; +} +#define arch_prepare_report_syscall_exit arch_prepare_report_syscall_exit + +static inline void +arch_post_report_syscall_exit(struct pt_regs *regs, + unsigned long saved_reg, + unsigned long work) +{ + regs->ARM_ip = saved_reg; +} +#define arch_post_report_syscall_exit arch_post_report_syscall_exit + +#endif /* _ASM_ARM_ENTRY_COMMON_H */ diff --git a/arch/arm/include/asm/entry.h b/arch/arm/include/asm/entry.h index a78bc5054b093120e903f08369fb8b1a9feb7f81..df2dd14ab51a586b83b6da6bbf9bd99858c1ddf4 100644 --- a/arch/arm/include/asm/entry.h +++ b/arch/arm/include/asm/entry.h @@ -4,15 +4,8 @@ struct pt_regs; -/* - * These are copies of generic entry headers so we can transition - * to generic entry once they are semantically equivalent. - */ -long syscall_enter_from_user_mode(struct pt_regs *regs, long); -void syscall_exit_to_user_mode(struct pt_regs *regs); -void irqentry_enter_from_user_mode(struct pt_regs *regs); -void irqentry_exit_to_user_mode(struct pt_regs *regs); -void irqentry_enter_from_kernel_mode(struct pt_regs *regs); -void irqentry_exit_to_kernel_mode(struct pt_regs *regs); +void arm_irq_handler(struct pt_regs *regs, int mode); +void arm_fiq_handler(struct pt_regs *regs); +void arm_exit_to_user_mode(struct pt_regs *regs); #endif /* __ASM_ENTRY_H__ */ diff --git a/arch/arm/include/asm/ptrace.h b/arch/arm/include/asm/ptrace.h index a01c66f0ad907882af2a383e53de8aca68c551e5..8a7de4020cbc016f743dcf515b9fa83b9bd3e3f0 100644 --- a/arch/arm/include/asm/ptrace.h +++ b/arch/arm/include/asm/ptrace.h @@ -200,8 +200,5 @@ static inline unsigned long it_advance(unsigned long cpsr) return cpsr; } -int syscall_trace_enter(struct pt_regs *regs); -void syscall_trace_exit(struct pt_regs *regs); - #endif /* __ASSEMBLY__ */ #endif diff --git a/arch/arm/include/asm/signal.h b/arch/arm/include/asm/signal.h index bb753cd3e29fc54186543c48b2aee7b221301a37..45885383646c9f166b0d7ee80080b78776b515e7 100644 --- a/arch/arm/include/asm/signal.h +++ b/arch/arm/include/asm/signal.h @@ -23,6 +23,4 @@ typedef struct { #include -void do_work_pending(struct pt_regs *regs, unsigned int thread_flags); - #endif diff --git a/arch/arm/include/asm/syscall.h b/arch/arm/include/asm/syscall.h index b94fd7ce17f82f0baf06b74800245112e1f13002..a6c8f0e799bb8bf3833161dec70fc7a1aaa513f6 100644 --- a/arch/arm/include/asm/syscall.h +++ b/arch/arm/include/asm/syscall.h @@ -19,7 +19,12 @@ extern const unsigned long sys_call_table[]; -int invoke_syscall(void *table, struct pt_regs *regs, int scno); +void invoke_syscall(void *table, struct pt_regs *regs, int scno); + +static inline bool arch_syscall_is_vdso_sigreturn(struct pt_regs *regs) +{ + return false; +} static inline int syscall_get_nr(struct task_struct *task, struct pt_regs *regs) diff --git a/arch/arm/include/asm/thread_info.h b/arch/arm/include/asm/thread_info.h index d8a45c5a10496aaf806bfeaa0353d5e8985bd6f5..84e58a9cdab63ad264c2cd2bad64239d1912cbe7 100644 --- a/arch/arm/include/asm/thread_info.h +++ b/arch/arm/include/asm/thread_info.h @@ -74,6 +74,7 @@ struct thread_info { #ifdef CONFIG_ARM_THUMBEE unsigned long thumbee_state; /* ThumbEE Handler Base register */ #endif + unsigned long syscall_work; /* SYSCALL_WORK_ flags */ }; #define INIT_THREAD_INFO(tsk) \ @@ -149,30 +150,14 @@ extern int vfp_restore_user_hwstate(struct user_vfp *, #define TIF_USING_IWMMXT 17 #define TIF_MEMDIE 18 /* is terminating due to OOM killer */ #define TIF_RESTORE_SIGMASK 19 -#define TIF_SYSCALL_TRACE 20 /* syscall trace active */ -#define TIF_SYSCALL_AUDIT 21 /* syscall auditing active */ -#define TIF_SYSCALL_TRACEPOINT 22 /* syscall tracepoint instrumentation */ -#define TIF_SECCOMP 23 /* seccomp syscall filtering active */ - #define _TIF_SIGPENDING (1 << TIF_SIGPENDING) #define _TIF_NEED_RESCHED (1 << TIF_NEED_RESCHED) #define _TIF_NOTIFY_RESUME (1 << TIF_NOTIFY_RESUME) #define _TIF_UPROBE (1 << TIF_UPROBE) -#define _TIF_SYSCALL_TRACE (1 << TIF_SYSCALL_TRACE) -#define _TIF_SYSCALL_AUDIT (1 << TIF_SYSCALL_AUDIT) -#define _TIF_SYSCALL_TRACEPOINT (1 << TIF_SYSCALL_TRACEPOINT) -#define _TIF_SECCOMP (1 << TIF_SECCOMP) #define _TIF_NOTIFY_SIGNAL (1 << TIF_NOTIFY_SIGNAL) #define _TIF_USING_IWMMXT (1 << TIF_USING_IWMMXT) -/* Checks for any syscall work in entry-common.S */ -#define _TIF_SYSCALL_WORK (_TIF_SYSCALL_TRACE | _TIF_SYSCALL_AUDIT | \ - _TIF_SYSCALL_TRACEPOINT | _TIF_SECCOMP) - -/* - * Change these and you break ASM code in entry-common.S - */ #define _TIF_WORK_MASK (_TIF_NEED_RESCHED | _TIF_SIGPENDING | \ _TIF_NOTIFY_RESUME | _TIF_UPROBE | \ _TIF_NOTIFY_SIGNAL) diff --git a/arch/arm/include/asm/traps.h b/arch/arm/include/asm/traps.h index b888912c2450e0e8eec8139166c7292f283545c7..f8695104c72f2f450092839c7c50920e540e594b 100644 --- a/arch/arm/include/asm/traps.h +++ b/arch/arm/include/asm/traps.h @@ -38,7 +38,6 @@ extern void *vectors_page; asmlinkage void dump_backtrace_stm(u32 *stack, u32 instruction, const char *loglvl); asmlinkage void do_undefinstr(struct pt_regs *regs); -asmlinkage void handle_fiq_as_nmi(struct pt_regs *regs); asmlinkage void bad_mode(struct pt_regs *regs, int reason); int arm_syscall(int no, struct pt_regs *regs); asmlinkage void baddataabort(int code, unsigned long instr, struct pt_regs *regs); diff --git a/arch/arm/include/uapi/asm/ptrace.h b/arch/arm/include/uapi/asm/ptrace.h index 8896c23ccba78ed7e10c7d1b9955124314496dc6..336a5beb282f360e875703045ee833974bbd68b2 100644 --- a/arch/arm/include/uapi/asm/ptrace.h +++ b/arch/arm/include/uapi/asm/ptrace.h @@ -33,6 +33,8 @@ #define PTRACE_GETHBPREGS 29 #define PTRACE_SETHBPREGS 30 #define PTRACE_GETFDPIC 31 +#define PTRACE_SYSEMU 32 +#define PTRACE_SYSEMU_SINGLESTEP 33 #define PTRACE_GETFDPIC_EXEC 0 #define PTRACE_GETFDPIC_INTERP 1 diff --git a/arch/arm/kernel/entry-armv.S b/arch/arm/kernel/entry-armv.S index 2a789c8834b93475c32dcb6ba5854e24ddd8d6e9..498a22f5fa8a6516d1eab584f0523c3d6c6e4926 100644 --- a/arch/arm/kernel/entry-armv.S +++ b/arch/arm/kernel/entry-armv.S @@ -36,35 +36,6 @@ #define RELOC_TEXT_NONE #endif -/* - * Interrupt handling. - */ - .macro irq_handler, from_user:req - mov r1, sp - ldr_this_cpu r2, irq_stack_ptr, r2, r3 - .if \from_user == 0 - @ - @ If we took the interrupt while running in the kernel, we may already - @ be using the IRQ stack, so revert to the original value in that case. - @ - subs r3, r2, r1 @ SP above bottom of IRQ stack? - rsbscs r3, r3, #THREAD_SIZE @ ... and below the top? -#ifdef CONFIG_VMAP_STACK - ldr_va r3, high_memory, cc @ End of the linear region - cmpcc r3, r1 @ Stack pointer was below it? -#endif - bcc 0f @ If not, switch to the IRQ stack - mov r0, r1 - bl generic_handle_arch_irq - b 1f -0: - .endif - - mov_l r0, generic_handle_arch_irq - bl call_with_stack -1: - .endm - .macro pabt_helper @ PABORT handler takes pt_regs in r2, fault address in r4 and psr in r5 #ifdef MULTI_PABORT @@ -205,9 +176,6 @@ ENDPROC(__und_invalid) get_thread_info tsk uaccess_entry tsk, r0, r1, r2, \uaccess - mov r0, sp @ 'regs' - bl irqentry_enter_from_kernel_mode - .endm .align 5 @@ -223,7 +191,9 @@ ENDPROC(__dabt_svc) .align 5 __irq_svc: svc_entry - irq_handler from_user=0 + mov r0, sp @ regs + mov r1, #0 @ from kernel mode + bl arm_irq_handler #ifdef CONFIG_PREEMPTION ldr r8, [tsk, #TI_PREEMPT] @ get preempt count @@ -300,7 +270,7 @@ ENDPROC(__pabt_svc) __fiq_svc: svc_entry mov r0, sp @ struct pt_regs *regs - bl handle_fiq_as_nmi + bl arm_fiq_handler svc_exit_via_fiq UNWIND(.fnend ) ENDPROC(__fiq_svc) @@ -329,7 +299,7 @@ __fiq_abt: stmfd sp!, {r1 - r2} add r0, sp, #8 @ struct pt_regs *regs - bl handle_fiq_as_nmi + bl arm_fiq_handler ldmfd sp!, {r1 - r2} ARM( msr cpsr_c, #ABT_MODE | PSR_I_BIT | PSR_F_BIT ) @@ -420,7 +390,6 @@ ENDPROC(__fiq_abt) .align 5 __dabt_usr: usr_entry uaccess=0 - asm_irqentry_enter_from_user_mode kuser_cmpxchg_check mov r2, sp dabt_helper @@ -431,9 +400,10 @@ ENDPROC(__dabt_usr) .align 5 __irq_usr: usr_entry - asm_irqentry_enter_from_user_mode kuser_cmpxchg_check - irq_handler from_user=1 + mov r0, sp @ regs + mov r1, #1 @ from user mode + bl arm_irq_handler get_thread_info tsk mov why, #0 b ret_to_user_from_irq @@ -445,7 +415,6 @@ ENDPROC(__irq_usr) .align 5 __und_usr: usr_entry uaccess=0 - asm_irqentry_enter_from_user_mode @ IRQs must be enabled before attempting to read the instruction from @ user space since that could cause a page/translation fault if the @@ -470,7 +439,6 @@ ENDPROC(__und_usr) .align 5 __pabt_usr: usr_entry - asm_irqentry_enter_from_user_mode mov r2, sp @ regs pabt_helper UNWIND(.fnend ) @@ -493,7 +461,7 @@ __fiq_usr: usr_entry kuser_cmpxchg_check mov r0, sp @ struct pt_regs *regs - bl handle_fiq_as_nmi + bl arm_fiq_handler get_thread_info tsk restore_user_regs UNWIND(.fnend ) diff --git a/arch/arm/kernel/entry-common.S b/arch/arm/kernel/entry-common.S index 6b0f86786a7d9e656ecbe8c741f2edbcbafd2915..a5a0e757ab23d151485092a078af98966e039869 100644 --- a/arch/arm/kernel/entry-common.S +++ b/arch/arm/kernel/entry-common.S @@ -39,10 +39,6 @@ ret_fast_syscall: UNWIND(.fnstart ) UNWIND(.cantunwind ) add sp, sp, #(S_R0 + S_OFF) - /* do_rseq_syscall needs interrupts enabled. */ - mov r0, sp @ 'regs' - bl syscall_exit_to_user_mode - #ifdef CONFIG_GCC_PLUGIN_STACKLEAK bl stackleak_erase_on_task_stack #endif @@ -60,7 +56,7 @@ ENTRY(ret_to_user) /* do_rseq_syscall needs interrupts enabled. */ enable_irq_notrace @ enable interrupts mov r0, sp @ 'regs' - bl syscall_exit_to_user_mode + bl arm_exit_to_user_mode #ifdef CONFIG_GCC_PLUGIN_STACKLEAK bl stackleak_erase_on_task_stack @@ -69,8 +65,6 @@ ENTRY(ret_to_user) ENDPROC(ret_to_user) ENTRY(ret_to_user_from_irq) - asm_irqentry_exit_to_user_mode - #ifdef CONFIG_GCC_PLUGIN_STACKLEAK bl stackleak_erase_on_task_stack #endif @@ -92,9 +86,10 @@ SYM_TYPED_FUNC_START(ret_from_fork_asm) mov r2, r5 mov r3, r4 bl ret_from_fork - mov r0, #0 -1: get_thread_info tsk - b ret_to_user +#ifdef CONFIG_GCC_PLUGIN_STACKLEAK + bl stackleak_erase_on_task_stack +#endif + restore_user_regs SYM_FUNC_END(ret_from_fork_asm) /*============================================================================= @@ -210,29 +205,12 @@ ENTRY(vector_swi) #else str scno, [tsk, #TI_ABI_SYSCALL] #endif - - /* - * Calling out to C to be careful to save and restore registers. - * This call could modify the syscall number. scno is r7 so we - * do not save and restore r7. - */ - mov r0, sp @ regs - mov r1, scno - push {r4 - r6, r8 - r10, lr} - bl syscall_enter_from_user_mode - pop {r4 - r6, r8 - r10, lr} - mov scno, r0 - mov r1, sp @ put regs into r1 stmdb sp!, {r4, r5} @ push fifth and sixth args mov r0, tbl mov r2, scno @ syscall number from r7 bl invoke_syscall - cmp r0, #0 - beq ret_fast_syscall - /* This path taken when tracing */ - add sp, sp, #(S_R0 + S_OFF) - b ret_to_user + b ret_fast_syscall #if defined(CONFIG_OABI_COMPAT) || !defined(CONFIG_AEABI) /* diff --git a/arch/arm/kernel/entry-header.S b/arch/arm/kernel/entry-header.S index cfaf14d71378ba14bbb2a42cd36d48a23838eee1..abd927f1cc3a4d813cc48bc046191f49f3c0e533 100644 --- a/arch/arm/kernel/entry-header.S +++ b/arch/arm/kernel/entry-header.S @@ -203,10 +203,6 @@ @ IRQs off again before pulling preserved data off the stack disable_irq_notrace - - mov r0, sp @ 'regs' - bl irqentry_exit_to_kernel_mode - uaccess_exit tsk, r0, r1 #ifndef CONFIG_THUMB2_KERNEL @@ -340,20 +336,6 @@ ALT_UP_B(.L1_\@) #endif /* !CONFIG_THUMB2_KERNEL */ .endm -/* - * Context tracking and other mode transitions. Used to instrument transitions - * between user and kernel mode. -*/ - .macro asm_irqentry_enter_from_user_mode - mov r0, sp @ regs - bl irqentry_enter_from_user_mode - .endm - - .macro asm_irqentry_exit_to_user_mode - mov r0, sp @ regs - bl irqentry_exit_to_user_mode - .endm - /* * These are the registers used in the syscall handler, and allow us to * have in theory up to 7 arguments to a function - r0 to r6. diff --git a/arch/arm/kernel/entry.c b/arch/arm/kernel/entry.c index 1e1284cc4caed6e602ce36e812d535e6fe324f34..c0083e4fc5c3730afd8c5d7ce583fdda513abe3f 100644 --- a/arch/arm/kernel/entry.c +++ b/arch/arm/kernel/entry.c @@ -1,71 +1,68 @@ // SPDX-License-Identifier: GPL-2.0 #include -#include -#include #include +#include +#include +#include #include +#include #include -long syscall_enter_from_user_mode(struct pt_regs *regs, long syscall) -{ - trace_hardirqs_on(); - local_irq_enable(); - /* This context tracking call has inverse naming */ - user_exit_callable(); - - /* This will optionally be modified later */ - return syscall; -} +#include "irq.h" -void syscall_exit_to_user_mode(struct pt_regs *regs) +static void noinstr handle_arm_irq(void *data) { - unsigned long flags = read_thread_flags(); + struct pt_regs *regs = data; + struct pt_regs *old_regs; - rseq_syscall(regs); - local_irq_disable(); - /* - * It really matters that we check for flags != 0 and not - * just for pending work here! - */ - if (flags) - do_work_pending(regs, flags); + irq_enter_rcu(); + old_regs = set_irq_regs(regs); - trace_hardirqs_on(); - /* This context tracking call has inverse naming */ - user_enter_callable(); -} + handle_arch_irq(regs); -noinstr void irqentry_enter_from_user_mode(struct pt_regs *regs) -{ - trace_hardirqs_off(); - /* This context tracking call has inverse naming */ - user_exit_callable(); + set_irq_regs(old_regs); + irq_exit_rcu(); } -noinstr void irqentry_exit_to_user_mode(struct pt_regs *regs) +noinstr void arm_irq_handler(struct pt_regs *regs, int mode) { - unsigned long flags = read_thread_flags(); + irqentry_state_t state = irqentry_enter(regs); /* - * It really matters that we check for flags != 0 and not - * just for pending work here! + * If we are executing in kernel context and we are already on + * the IRQ stack (i.e. we get interrupted in interrupt context) + * then just handle the IRQ, else switch to the IRQ stack and + * handle the interrupt using the IRQ stack. */ - if (flags) - do_work_pending(regs, flags); - trace_hardirqs_on(); - /* This context tracking call has inverse naming */ - user_enter_callable(); + if ((mode == 0) && on_irq_stack(regs)) + handle_arm_irq(regs); + else + call_on_irq_stack(handle_arm_irq, regs); + + irqentry_exit(regs, state); } -noinstr void irqentry_enter_from_kernel_mode(struct pt_regs *regs) +/* + * Handle FIQ similarly to NMI on x86 systems. + * + * The runtime environment for NMIs is extremely restrictive + * (NMIs can pre-empt critical sections meaning almost all locking is + * forbidden) meaning this default FIQ handling must only be used in + * circumstances where non-maskability improves robustness, such as + * watchdog or debug logic. + * + * This handler is not appropriate for general purpose use in drivers + * platform code and can be overrideen using set_fiq_handler. + */ +noinstr void arm_fiq_handler(struct pt_regs *regs) { - trace_hardirqs_off(); + irqentry_state_t state = irqentry_nmi_enter(regs); + + irqentry_nmi_exit(regs, state); } -noinstr void irqentry_exit_to_kernel_mode(struct pt_regs *regs) +asmlinkage void arm_exit_to_user_mode(struct pt_regs *regs) { - if (interrupts_enabled(regs)) - trace_hardirqs_on(); - else - trace_hardirqs_off(); + local_irq_disable(); + irqentry_exit_to_user_mode(regs); } diff --git a/arch/arm/kernel/process.c b/arch/arm/kernel/process.c index 81c1c3f988344185917cf4e53d0d8ee47ae912f4..a59ad5fdaaf51f653c352884bbdd10d1211b2669 100644 --- a/arch/arm/kernel/process.c +++ b/arch/arm/kernel/process.c @@ -25,6 +25,7 @@ #include #include #include +#include #include #include @@ -248,8 +249,8 @@ __visible void ret_from_fork(struct task_struct *prev, struct pt_regs *regs, * execve() syscall. */ } - - /* syscall_exit_to_user_mode(regs); here once we switch to generic entry */ + local_irq_enable(); + syscall_exit_to_user_mode(regs); } int copy_thread(struct task_struct *p, const struct kernel_clone_args *args) diff --git a/arch/arm/kernel/ptrace.c b/arch/arm/kernel/ptrace.c index ac7b98ae47249b4a00fc1cb871bb2d309bfd8c88..89afe714cedc86bbe486f20671433fbe099621c3 100644 --- a/arch/arm/kernel/ptrace.c +++ b/arch/arm/kernel/ptrace.c @@ -27,9 +27,6 @@ #include #include -#define CREATE_TRACE_POINTS -#include - #define REG_PC 15 #define REG_PSR 16 /* @@ -821,90 +818,3 @@ long arch_ptrace(struct task_struct *child, long request, return ret; } - -enum ptrace_syscall_dir { - PTRACE_SYSCALL_ENTER = 0, - PTRACE_SYSCALL_EXIT, -}; - -static void report_syscall_enter(struct pt_regs *regs) -{ - unsigned long ip; - - /* - * IP is used to denote syscall entry/exit: - * IP = 0 -> entry - */ - ip = regs->ARM_ip; - regs->ARM_ip = PTRACE_SYSCALL_ENTER; - - if (ptrace_report_syscall_entry(regs)) - current_thread_info()->abi_syscall = -1; - - regs->ARM_ip = ip; -} - -static void report_syscall_exit(struct pt_regs *regs) -{ - unsigned long ip; - - /* - * IP is used to denote syscall entry/exit: - * IP = 1 -> exit - */ - ip = regs->ARM_ip; - regs->ARM_ip = PTRACE_SYSCALL_EXIT; - - ptrace_report_syscall_exit(regs, 0); - - regs->ARM_ip = ip; -} - -asmlinkage int syscall_trace_enter(struct pt_regs *regs) -{ - int scno; - - if (test_thread_flag(TIF_SYSCALL_TRACE)) - report_syscall_enter(regs); - - /* Do seccomp after ptrace; syscall may have changed. */ -#ifdef CONFIG_HAVE_ARCH_SECCOMP_FILTER - if (secure_computing() == -1) - return -1; -#else - /* XXX: remove this once OABI gets fixed */ - secure_computing_strict(syscall_get_nr(current, regs)); -#endif - - /* Tracer or seccomp may have changed syscall. */ - scno = syscall_get_nr(current, regs); - - if (test_thread_flag(TIF_SYSCALL_TRACEPOINT)) - trace_sys_enter(regs, scno); - - audit_syscall_entry(scno, regs->ARM_r0, regs->ARM_r1, regs->ARM_r2, - regs->ARM_r3); - - return scno; -} - -void syscall_trace_exit(struct pt_regs *regs) -{ - /* - * Audit the syscall before anything else, as a debugger may - * come in and change the current registers. - */ - audit_syscall_exit(regs); - - /* - * Note that we haven't updated the ->syscall field for the - * current thread. This isn't a problem because it will have - * been set on syscall entry and there hasn't been an opportunity - * for a PTRACE_SET_SYSCALL since then. - */ - if (test_thread_flag(TIF_SYSCALL_TRACEPOINT)) - trace_sys_exit(regs, regs_return_value(regs)); - - if (test_thread_flag(TIF_SYSCALL_TRACE)) - report_syscall_exit(regs); -} diff --git a/arch/arm/kernel/signal.c b/arch/arm/kernel/signal.c index a39ee14ec5b70b3738ecef537709326ef1e8217c..35d2bb3cd2b442dac164548037262e065fbfe12a 100644 --- a/arch/arm/kernel/signal.c +++ b/arch/arm/kernel/signal.c @@ -4,6 +4,7 @@ * * Copyright (C) 1995-2009 Russell King */ +#include #include #include #include @@ -535,7 +536,7 @@ static void handle_signal(struct ksignal *ksig, struct pt_regs *regs) * the kernel can handle, and then we build all the user-level signal handling * stack-frames in one go after that. */ -static void arch_do_signal_or_restart(struct pt_regs *regs) +void arch_do_signal_or_restart(struct pt_regs *regs) { unsigned int retval = 0, continue_addr = 0, restart_addr = 0; bool syscall = (syscall_get_nr(current, regs) != -1); @@ -598,34 +599,6 @@ static void arch_do_signal_or_restart(struct pt_regs *regs) return; } -void do_work_pending(struct pt_regs *regs, unsigned int thread_flags) -{ - /* - * The assembly code enters us with IRQs off, but it hasn't - * informed the tracing code of that for efficiency reasons. - * Update the trace code with the current status. - */ - trace_hardirqs_off(); - do { - if (likely(thread_flags & _TIF_NEED_RESCHED)) { - schedule(); - } else { - if (unlikely(!user_mode(regs))) - return; - local_irq_enable(); - if (thread_flags & (_TIF_SIGPENDING | _TIF_NOTIFY_SIGNAL)) { - arch_do_signal_or_restart(regs); - } else if (thread_flags & _TIF_UPROBE) { - uprobe_notify_resume(regs); - } else { - resume_user_mode_work(regs); - } - } - local_irq_disable(); - thread_flags = read_thread_flags(); - } while (thread_flags & _TIF_WORK_MASK); -} - struct page *get_signal_page(void) { unsigned long ptr; diff --git a/arch/arm/kernel/syscall.c b/arch/arm/kernel/syscall.c index 6f518ec3870fb36a64d1b671b20f96d63a52bf78..bb72e29252fa10875d09f41c9a21825e6d60249f 100644 --- a/arch/arm/kernel/syscall.c +++ b/arch/arm/kernel/syscall.c @@ -1,26 +1,17 @@ // SPDX-License-Identifier: GPL-2.0 +#include #include #include -static inline bool has_syscall_work(unsigned long flags) -{ - return unlikely(flags & _TIF_SYSCALL_WORK); -} - int invoke_syscall_asm(void *table, struct pt_regs *regs, int scno); __ADDRESSABLE(invoke_syscall_asm); -__visible int invoke_syscall(void *table, struct pt_regs *regs, int scno) +__visible void invoke_syscall(void *table, struct pt_regs *regs, int scno) { - unsigned long flags = read_thread_flags(); int ret; - if (has_syscall_work(flags)) { - scno = syscall_trace_enter(regs); - if (scno == -1) - goto trace_exit_nosave; - } + scno = syscall_enter_from_user_mode(regs, scno); if (scno < NR_syscalls) { ret = invoke_syscall_asm(table, regs, scno); @@ -35,13 +26,7 @@ __visible int invoke_syscall(void *table, struct pt_regs *regs, int scno) ret = sys_ni_syscall(); exit_save: - /* Save return value from syscall */ - regs->ARM_r0 = ret; - if (!has_syscall_work(flags)) - return 0; - -trace_exit_nosave: - local_irq_enable(); - syscall_trace_exit(regs); - return 1; + syscall_set_return_value(current, regs, 0, ret); + + syscall_exit_to_user_mode(regs); } diff --git a/arch/arm/kernel/traps.c b/arch/arm/kernel/traps.c index 942c77767919466d5b89d83d8eb77032b119473e..57e74b49c52533babe177a8a2e0160a83c9bde05 100644 --- a/arch/arm/kernel/traps.c +++ b/arch/arm/kernel/traps.c @@ -502,31 +502,6 @@ asmlinkage void do_undefinstr(struct pt_regs *regs) } NOKPROBE_SYMBOL(do_undefinstr) -/* - * Handle FIQ similarly to NMI on x86 systems. - * - * The runtime environment for NMIs is extremely restrictive - * (NMIs can pre-empt critical sections meaning almost all locking is - * forbidden) meaning this default FIQ handling must only be used in - * circumstances where non-maskability improves robustness, such as - * watchdog or debug logic. - * - * This handler is not appropriate for general purpose use in drivers - * platform code and can be overrideen using set_fiq_handler. - */ -asmlinkage void __exception_irq_entry handle_fiq_as_nmi(struct pt_regs *regs) -{ - struct pt_regs *old_regs = set_irq_regs(regs); - - nmi_enter(); - - /* nop. FIQ handlers for special arch/arm features can be added here. */ - - nmi_exit(); - - set_irq_regs(old_regs); -} - /* * bad_mode handles the impossible case in the vectors. If you see one of * these, then it's extremely serious, and could mean you have buggy hardware. From patchwork Tue Jan 7 09:41:45 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13928636 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A78C0E77198 for ; Tue, 7 Jan 2025 10:17:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=mAfkic6+criyOuanDZL0bvo4PgikxSotKrfedELEMNs=; b=l6HQR0xw/vmdVGsUgjVz0DITCz F4d/eq7Vsv+sAdZ9kcvSvJLfN5gCeAIzl6lg2ip0iAnYBQOwR4uy8Gjv9Ir4qX9AvfL/RbyITGuou 1nPVIiS8kUM0oliimIwJplBTQXmvfZFrOb/WjK6Wot4FMtcvqHswvlca9WRL2UrVF7y4WFYwpxt00 LqKOIKkdE2hJsUdKiDU5mc+Ccq/GaJAlvknvCP5og2yurM3rTzbskStQZGyNMnhuZT/3Ww5j2k4eo sBR00SODOdzro7Qz4e6myLScVxX4x7MKf2xN36KQcEBrhNBzzxWrEk3taN6FOSBsbJouJhIb4vDdD hvM4L+oA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tV6eN-00000004Ibo-10Ro; Tue, 07 Jan 2025 10:17:27 +0000 Received: from mail-ej1-x636.google.com ([2a00:1450:4864:20::636]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tV66A-00000004Adn-35x4 for linux-arm-kernel@lists.infradead.org; Tue, 07 Jan 2025 09:42:13 +0000 Received: by mail-ej1-x636.google.com with SMTP id a640c23a62f3a-aaedd529ba1so1711066766b.1 for ; Tue, 07 Jan 2025 01:42:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1736242925; x=1736847725; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=mAfkic6+criyOuanDZL0bvo4PgikxSotKrfedELEMNs=; b=mdrPGypXOXoYt0ZtCwtQTIa2SCH49GRkcyQF2x/s6r3CGev6OkKfIxBz5L2dI9LrHf AIZXljVXhLKeXs+C9LrlGX15QeYc7KlXKqcDeoVKuTp/GkbVY6oOhLtjyAEu/S3T04i4 ab9vl3Y4R6boMFZZEQXOy/FgQ2j8YfToScgJN/KUkC0HV220hs1vyrbNtomoiV0pexa9 YhzWtX2QZBlCb2HK1j6OYCDHpGemlwx1PvkVuVXMg8Hx4ieb+rGAjb3Fo8Epzi5lBHUs gDfAKIyk/LhoP7VtERb+Ooyd5ChXz7ci5PAEV0C83r6E0bRCzF52Kt2TkC4oOiO5psh3 9XGA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736242925; x=1736847725; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=mAfkic6+criyOuanDZL0bvo4PgikxSotKrfedELEMNs=; b=lmkW89/zqyvB0sJXYqjhGNS42WXgXg34TkL5M/dK8oPrHva6c2uvYi3BvvPHGOzUEu 2hJhGbrOKfmPE+zZBRF1WMTVBG0etVQI0tqFuI2C3KHIRaWAspdeNbT5E6gSexdRQ8F1 fuRgaPbPYp2WaO4CByc5nxyxL4aBceBt6wuuy56xpL0uqQsle0M7KrGDvEqUdbU4fIg5 3k5k2t9p+jtHBARjBLrRXlPVm+3RqFXdW1B/OtqWh2wrxfcAZgbiCcKYn497pMlnTEpV OeK/bo8uZZaIBrXH7015JzNOiLDGKioXeuCGGi55rAFZc19A15p071Ipa4NaNYb+tX2W RHgQ== X-Gm-Message-State: AOJu0Yxs2hdxLrKN0EyOPSfu8aS1u4/7F19ddUGzLmt6ygVMm5t5Ymvj yhJWWNy7SfMSYkme73/0MRjGUVOCrZfclxu9GD+92jRn9dtWQWiCl3IQQ9/zAnE= X-Gm-Gg: ASbGncv6lg20Z2hJWyzJ95ackruNIzC2qYjw7rgA3jCbocptLy/CJD/9gE/WfqhMbAn nba77b3kTfLPRhIwuB8sa2ThhUxnanZGLHfMJ6yPegfHHpVzBJIj8F/1CE0lDRPLMfkDXXIPN/p CGz+oDvAvnvsThC0gSayBJeFrJhiEXLQagrGiqE/jxsElcgbSifTMU1G+CGWp2X3ZbMj8cL6ay1 xTlPCg8k0oFvR4MYw7gqu0u/XiSyK8TeZDXyJxuc+/lDHQcj8PzWytrNXl8YzOw9ho= X-Google-Smtp-Source: AGHT+IHhvCoKRjSlNZ4hKUdfpY7ibxFwjoAu7q2CKeJKd0NbWOQ32M7LzDZ0LHE+yZMuUWOy+L5SQg== X-Received: by 2002:a05:6402:5204:b0:5d0:ced8:d22d with SMTP id 4fb4d7f45d1cf-5d81de05c3fmr157108448a12.22.1736242925154; Tue, 07 Jan 2025 01:42:05 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aac0f012229sm2356901166b.133.2025.01.07.01.42.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jan 2025 01:42:04 -0800 (PST) From: Linus Walleij Date: Tue, 07 Jan 2025 10:41:45 +0100 Subject: [PATCH RFC v3 29/30] ARM: entry: Handle prefetch and data aborts as interrupts MIME-Version: 1.0 Message-Id: <20250107-arm-generic-entry-v3-29-4e5f3c15db2d@linaro.org> References: <20250107-arm-generic-entry-v3-0-4e5f3c15db2d@linaro.org> In-Reply-To: <20250107-arm-generic-entry-v3-0-4e5f3c15db2d@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250107_014206_931556_73A88E45 X-CRM114-Status: GOOD ( 21.37 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org While it isn't entirely intuitive, it appears that any kind of exception such as data or prefetch abort ("page faults") need to be handled as some kind of "interrupts" when using generic entry. At least this is what other platforms are doing. The same goes for undefined instruction handling, i.e. floating point emulation in the kernel (und-exceptions). This is necessary for the context checking to pass: without this patch, a whole slew of warnings start to trigger from syscall_exit_to_user_mode_prepare() CT_WARN_ON(ct_state() != CT_STATE_KERNEL), i.e. syscalls seems to exit from user mode to user mode (not good), because the page faults screws up the context tracker. This patch restores the order. If this seems like the previous patch introduces a regression that is then fixed in this patch, it can simply be squashed into the former: having this rewrite separately surely makes development and review easier. Signed-off-by: Linus Walleij --- arch/arm/include/asm/entry.h | 3 +++ arch/arm/include/asm/traps.h | 2 +- arch/arm/kernel/entry-armv.S | 7 +++---- arch/arm/kernel/entry.c | 29 +++++++++++++++++++++++++++++ arch/arm/kernel/traps.c | 2 +- arch/arm/mm/abort-ev4.S | 2 +- arch/arm/mm/abort-ev4t.S | 2 +- arch/arm/mm/abort-ev5t.S | 4 ++-- arch/arm/mm/abort-ev5tj.S | 6 +++--- arch/arm/mm/abort-ev6.S | 2 +- arch/arm/mm/abort-ev7.S | 2 +- arch/arm/mm/abort-lv4t.S | 36 ++++++++++++++++++------------------ arch/arm/mm/abort-macro.S | 2 +- arch/arm/mm/abort-nommu.S | 2 +- arch/arm/mm/fault.c | 4 ++-- arch/arm/mm/fault.h | 8 ++++---- arch/arm/mm/pabort-legacy.S | 2 +- arch/arm/mm/pabort-v6.S | 2 +- arch/arm/mm/pabort-v7.S | 2 +- 19 files changed, 75 insertions(+), 44 deletions(-) diff --git a/arch/arm/include/asm/entry.h b/arch/arm/include/asm/entry.h index df2dd14ab51a586b83b6da6bbf9bd99858c1ddf4..de722084d5309194e02cb87db64ec32b9e6cd627 100644 --- a/arch/arm/include/asm/entry.h +++ b/arch/arm/include/asm/entry.h @@ -4,6 +4,9 @@ struct pt_regs; +void arm_und_handler(struct pt_regs *regs); +void arm_dabt_handler(unsigned long addr, unsigned int fsr, struct pt_regs *regs); +void arm_pabt_handler(unsigned long addr, unsigned int ifsr, struct pt_regs *regs); void arm_irq_handler(struct pt_regs *regs, int mode); void arm_fiq_handler(struct pt_regs *regs); void arm_exit_to_user_mode(struct pt_regs *regs); diff --git a/arch/arm/include/asm/traps.h b/arch/arm/include/asm/traps.h index f8695104c72f2f450092839c7c50920e540e594b..0ca98271e10a210225c1714f5b24668cebab2def 100644 --- a/arch/arm/include/asm/traps.h +++ b/arch/arm/include/asm/traps.h @@ -37,7 +37,7 @@ extern void ptrace_break(struct pt_regs *regs); extern void *vectors_page; asmlinkage void dump_backtrace_stm(u32 *stack, u32 instruction, const char *loglvl); -asmlinkage void do_undefinstr(struct pt_regs *regs); +void do_undefinstr(struct pt_regs *regs); asmlinkage void bad_mode(struct pt_regs *regs, int reason); int arm_syscall(int no, struct pt_regs *regs); asmlinkage void baddataabort(int code, unsigned long instr, struct pt_regs *regs); diff --git a/arch/arm/kernel/entry-armv.S b/arch/arm/kernel/entry-armv.S index 498a22f5fa8a6516d1eab584f0523c3d6c6e4926..ae2f952beea7611f0abc7bd299fc944335a21219 100644 --- a/arch/arm/kernel/entry-armv.S +++ b/arch/arm/kernel/entry-armv.S @@ -230,7 +230,7 @@ __und_fault: ldr r2, [r0, #S_PC] sub r2, r2, r1 str r2, [r0, #S_PC] - b do_undefinstr + b arm_und_handler ENDPROC(__und_fault) .align 5 @@ -449,9 +449,8 @@ __pabt_usr: ENTRY(ret_from_exception) UNWIND(.fnstart ) UNWIND(.cantunwind ) - get_thread_info tsk - mov why, #0 - b ret_to_user + disable_irq_notrace + b ret_to_user_from_irq UNWIND(.fnend ) ENDPROC(__pabt_usr) ENDPROC(ret_from_exception) diff --git a/arch/arm/kernel/entry.c b/arch/arm/kernel/entry.c index c0083e4fc5c3730afd8c5d7ce583fdda513abe3f..10714dda5753684c7a2b8960562748525de774a8 100644 --- a/arch/arm/kernel/entry.c +++ b/arch/arm/kernel/entry.c @@ -7,8 +7,37 @@ #include #include #include +#include #include "irq.h" +#include "../mm/fault.h" + +noinstr asmlinkage void arm_und_handler(struct pt_regs *regs) +{ + irqentry_state_t state = irqentry_enter(regs); + + do_undefinstr(regs); + + irqentry_exit(regs, state); +} + +noinstr asmlinkage void arm_dabt_handler(unsigned long addr, unsigned int fsr, struct pt_regs *regs) +{ + irqentry_state_t state = irqentry_enter(regs); + + do_DataAbort(addr, fsr, regs); + + irqentry_exit(regs, state); +} + +noinstr asmlinkage void arm_pabt_handler(unsigned long addr, unsigned int ifsr, struct pt_regs *regs) +{ + irqentry_state_t state = irqentry_enter(regs); + + do_PrefetchAbort(addr, ifsr, regs); + + irqentry_exit(regs, state); +} static void noinstr handle_arm_irq(void *data) { diff --git a/arch/arm/kernel/traps.c b/arch/arm/kernel/traps.c index 57e74b49c52533babe177a8a2e0160a83c9bde05..d3a689bd05955f1ae46a6341e456bb097e831311 100644 --- a/arch/arm/kernel/traps.c +++ b/arch/arm/kernel/traps.c @@ -449,7 +449,7 @@ int call_undef_hook(struct pt_regs *regs, unsigned int instr) return fn ? fn(regs, instr) : 1; } -asmlinkage void do_undefinstr(struct pt_regs *regs) +noinstr void do_undefinstr(struct pt_regs *regs) { unsigned int instr; void __user *pc; diff --git a/arch/arm/mm/abort-ev4.S b/arch/arm/mm/abort-ev4.S index a10bcb89594dd38ce31ca30bf97d68cf421afc42..68d8ae2f4e9c0955c9f08112b4ff1887867d550a 100644 --- a/arch/arm/mm/abort-ev4.S +++ b/arch/arm/mm/abort-ev4.S @@ -24,4 +24,4 @@ ENTRY(v4_early_abort) bic r1, r1, #1 << 11 | 1 << 10 @ clear bits 11 and 10 of FSR tst r3, #1 << 20 @ L = 1 -> write? orreq r1, r1, #1 << 11 @ yes. - b do_DataAbort + b arm_dabt_handler diff --git a/arch/arm/mm/abort-ev4t.S b/arch/arm/mm/abort-ev4t.S index 14743a2f6997fcae0eeb55f53dec07b7b2ec227c..76c6cfa79e2a32e34b6f35a15c71c1af407956f3 100644 --- a/arch/arm/mm/abort-ev4t.S +++ b/arch/arm/mm/abort-ev4t.S @@ -25,4 +25,4 @@ ENTRY(v4t_early_abort) bic r1, r1, #1 << 11 | 1 << 10 @ clear bits 11 and 10 of FSR tst r3, #1 << 20 @ check write orreq r1, r1, #1 << 11 - b do_DataAbort + b arm_dabt_handler diff --git a/arch/arm/mm/abort-ev5t.S b/arch/arm/mm/abort-ev5t.S index 98c523118820798668bf04d065ad86ea05fb6d2c..fda1e54debc44b380fb3f3af95c1f198a9cc008a 100644 --- a/arch/arm/mm/abort-ev5t.S +++ b/arch/arm/mm/abort-ev5t.S @@ -25,7 +25,7 @@ ENTRY(v5t_early_abort) uaccess_disable ip @ disable user access bic r1, r1, #1 << 11 @ clear bits 11 of FSR teq_ldrd tmp=ip, insn=r3 @ insn was LDRD? - beq do_DataAbort @ yes + beq arm_dabt_handler @ yes tst r3, #1 << 20 @ check write orreq r1, r1, #1 << 11 - b do_DataAbort + b arm_dabt_handler diff --git a/arch/arm/mm/abort-ev5tj.S b/arch/arm/mm/abort-ev5tj.S index fec72f4fbaf508597d826e58d0dc084ee6e58dd0..a786a7d69e6ae23aa1125ccd41deaaaa743601e1 100644 --- a/arch/arm/mm/abort-ev5tj.S +++ b/arch/arm/mm/abort-ev5tj.S @@ -22,12 +22,12 @@ ENTRY(v5tj_early_abort) mrc p15, 0, r0, c6, c0, 0 @ get FAR bic r1, r1, #1 << 11 | 1 << 10 @ clear bits 11 and 10 of FSR tst r5, #PSR_J_BIT @ Java? - bne do_DataAbort + bne arm_dabt_handler do_thumb_abort fsr=r1, pc=r4, psr=r5, tmp=r3 ldreq r3, [r4] @ read aborted ARM instruction uaccess_disable ip @ disable userspace access teq_ldrd tmp=ip, insn=r3 @ insn was LDRD? - beq do_DataAbort @ yes + beq arm_dabt_handler @ yes tst r3, #1 << 20 @ L = 0 -> write orreq r1, r1, #1 << 11 @ yes. - b do_DataAbort + b arm_dabt_handler diff --git a/arch/arm/mm/abort-ev6.S b/arch/arm/mm/abort-ev6.S index 836dc1299243415faaac1a93dce35ac46ae1a7da..2366236053eb86484cb179ff572f06db6559426b 100644 --- a/arch/arm/mm/abort-ev6.S +++ b/arch/arm/mm/abort-ev6.S @@ -42,4 +42,4 @@ ENTRY(v6_early_abort) orreq r1, r1, #1 << 11 @ yes. #endif 1: uaccess_disable ip @ disable userspace access - b do_DataAbort + b arm_dabt_handler diff --git a/arch/arm/mm/abort-ev7.S b/arch/arm/mm/abort-ev7.S index 53fb41c24774db985eddc370904aa7666527f538..72b6eba101555651d7afd749fd5d7715a2964372 100644 --- a/arch/arm/mm/abort-ev7.S +++ b/arch/arm/mm/abort-ev7.S @@ -18,5 +18,5 @@ ENTRY(v7_early_abort) mrc p15, 0, r1, c5, c0, 0 @ get FSR mrc p15, 0, r0, c6, c0, 0 @ get FAR uaccess_disable ip @ disable userspace access - b do_DataAbort + b arm_dabt_handler ENDPROC(v7_early_abort) diff --git a/arch/arm/mm/abort-lv4t.S b/arch/arm/mm/abort-lv4t.S index fbd60a120f6684c56c63cea10b00200765473f1d..3fef9c1479c17bb07652e43e0b8b0683e00c4b2f 100644 --- a/arch/arm/mm/abort-lv4t.S +++ b/arch/arm/mm/abort-lv4t.S @@ -46,8 +46,8 @@ ENTRY(v4t_late_abort) /* 9 */ b .data_arm_ldmstm @ ldm*b rn, /* a */ b .data_unknown /* b */ b .data_unknown -/* c */ b do_DataAbort @ ldc rd, [rn], #m @ Same as ldr rd, [rn], #m -/* d */ b do_DataAbort @ ldc rd, [rn, #m] +/* c */ b arm_dabt_handler @ ldc rd, [rn], #m @ Same as ldr rd, [rn], #m +/* d */ b arm_dabt_handler @ ldc rd, [rn, #m] /* e */ b .data_unknown /* f */ b .data_unknown @@ -60,7 +60,7 @@ ENTRY(v4t_late_abort) .data_arm_ldmstm: tst r8, #1 << 21 @ check writeback bit - beq do_DataAbort @ no writeback -> no fixup + beq arm_dabt_handler @ no writeback -> no fixup str r9, [sp, #-4]! mov r7, #0x11 orr r7, r7, #0x1100 @@ -81,11 +81,11 @@ ENTRY(v4t_late_abort) addeq r7, r7, r6, lsl #2 @ Undo decrement str r7, [r2, r9, lsr #14] @ Put register 'Rn' ldr r9, [sp], #4 - b do_DataAbort + b arm_dabt_handler .data_arm_lateldrhpre: tst r8, #1 << 21 @ Check writeback bit - beq do_DataAbort @ No writeback -> no fixup + beq arm_dabt_handler @ No writeback -> no fixup .data_arm_lateldrhpost: str r9, [sp, #-4]! and r9, r8, #0x00f @ get Rm / low nibble of immediate value @@ -101,14 +101,14 @@ ENTRY(v4t_late_abort) addeq r7, r7, r6 @ Undo decrement str r7, [r2, r9, lsr #14] @ Put register 'Rn' ldr r9, [sp], #4 - b do_DataAbort + b arm_dabt_handler .data_arm_lateldrpreconst: tst r8, #1 << 21 @ check writeback bit - beq do_DataAbort @ no writeback -> no fixup + beq arm_dabt_handler @ no writeback -> no fixup .data_arm_lateldrpostconst: movs r6, r8, lsl #20 @ Get offset - beq do_DataAbort @ zero -> no fixup + beq arm_dabt_handler @ zero -> no fixup str r9, [sp, #-4]! and r9, r8, #15 << 16 @ Extract 'n' from instruction ldr r7, [r2, r9, lsr #14] @ Get register 'Rn' @@ -117,11 +117,11 @@ ENTRY(v4t_late_abort) addeq r7, r7, r6, lsr #20 @ Undo decrement str r7, [r2, r9, lsr #14] @ Put register 'Rn' ldr r9, [sp], #4 - b do_DataAbort + b arm_dabt_handler .data_arm_lateldrprereg: tst r8, #1 << 21 @ check writeback bit - beq do_DataAbort @ no writeback -> no fixup + beq arm_dabt_handler @ no writeback -> no fixup .data_arm_lateldrpostreg: and r7, r8, #15 @ Extract 'm' from instruction ldr r6, [r2, r7, lsl #2] @ Get register 'Rm' @@ -180,10 +180,10 @@ ENTRY(v4t_late_abort) /* 3 */ b .data_unknown /* 4 */ b .data_unknown /* 5 */ b .data_thumb_reg -/* 6 */ b do_DataAbort -/* 7 */ b do_DataAbort -/* 8 */ b do_DataAbort -/* 9 */ b do_DataAbort +/* 6 */ b arm_dabt_handler +/* 7 */ b arm_dabt_handler +/* 8 */ b arm_dabt_handler +/* 9 */ b arm_dabt_handler /* A */ b .data_unknown /* B */ b .data_thumb_pushpop /* C */ b .data_thumb_ldmstm @@ -193,10 +193,10 @@ ENTRY(v4t_late_abort) .data_thumb_reg: tst r8, #1 << 9 - beq do_DataAbort + beq arm_dabt_handler tst r8, #1 << 10 @ If 'S' (signed) bit is set movne r1, #0 @ it must be a load instr - b do_DataAbort + b arm_dabt_handler .data_thumb_pushpop: tst r8, #1 << 10 @@ -217,7 +217,7 @@ ENTRY(v4t_late_abort) subne r7, r7, r6, lsl #2 @ decrement SP if POP str r7, [r2, #13 << 2] ldr r9, [sp], #4 - b do_DataAbort + b arm_dabt_handler .data_thumb_ldmstm: str r9, [sp, #-4]! @@ -234,4 +234,4 @@ ENTRY(v4t_late_abort) sub r7, r7, r6, lsl #2 @ always decrement str r7, [r2, r9, lsr #6] ldr r9, [sp], #4 - b do_DataAbort + b arm_dabt_handler diff --git a/arch/arm/mm/abort-macro.S b/arch/arm/mm/abort-macro.S index bacf53fd0b70c6307e74ef8601d8dcc7db292700..30bb5e2135862902951b5077b0b860fd12ed8c28 100644 --- a/arch/arm/mm/abort-macro.S +++ b/arch/arm/mm/abort-macro.S @@ -20,7 +20,7 @@ orreq \tmp, \tmp, #1 << 11 @ Set L-bit if yes tst \tmp, #1 << 11 @ L = 0 -> write orreq \fsr, \fsr, #1 << 11 @ yes. - b do_DataAbort + b arm_dabt_handler not_thumb: .endm diff --git a/arch/arm/mm/abort-nommu.S b/arch/arm/mm/abort-nommu.S index 6e2366a263219b379f4fdb43cc8e5413fb36e52a..573a09529ac7794f8004680ab350750de7c8e15d 100644 --- a/arch/arm/mm/abort-nommu.S +++ b/arch/arm/mm/abort-nommu.S @@ -17,5 +17,5 @@ ENTRY(nommu_early_abort) mov r0, #0 @ clear r0, r1 (no FSR/FAR) mov r1, #0 - b do_DataAbort + b arm_dabt_handler ENDPROC(nommu_early_abort) diff --git a/arch/arm/mm/fault.c b/arch/arm/mm/fault.c index ab01b51de5590bff6c2acaf446d01c589a5f7987..78afb701b34a17d8d841762944eb5f8af6843dc4 100644 --- a/arch/arm/mm/fault.c +++ b/arch/arm/mm/fault.c @@ -588,7 +588,7 @@ hook_fault_code(int nr, int (*fn)(unsigned long, unsigned int, struct pt_regs *) /* * Dispatch a data abort to the relevant handler. */ -asmlinkage void +void do_DataAbort(unsigned long addr, unsigned int fsr, struct pt_regs *regs) { const struct fsr_info *inf = fsr_info + fsr_fs(fsr); @@ -618,7 +618,7 @@ hook_ifault_code(int nr, int (*fn)(unsigned long, unsigned int, struct pt_regs * ifsr_info[nr].name = name; } -asmlinkage void +void do_PrefetchAbort(unsigned long addr, unsigned int ifsr, struct pt_regs *regs) { const struct fsr_info *inf = ifsr_info + fsr_fs(ifsr); diff --git a/arch/arm/mm/fault.h b/arch/arm/mm/fault.h index e8f8c190254442a015a6cbc179602589cfddb0ea..36a7e7138c310a8311458207d2b694cb44c63b93 100644 --- a/arch/arm/mm/fault.h +++ b/arch/arm/mm/fault.h @@ -37,9 +37,9 @@ static inline int fsr_fs(unsigned int fsr) void do_bad_area(unsigned long addr, unsigned int fsr, struct pt_regs *regs); void early_abt_enable(void); -asmlinkage void do_DataAbort(unsigned long addr, unsigned int fsr, - struct pt_regs *regs); -asmlinkage void do_PrefetchAbort(unsigned long addr, unsigned int ifsr, - struct pt_regs *regs); +void do_DataAbort(unsigned long addr, unsigned int fsr, + struct pt_regs *regs); +void do_PrefetchAbort(unsigned long addr, unsigned int ifsr, + struct pt_regs *regs); #endif /* __ARCH_ARM_FAULT_H */ diff --git a/arch/arm/mm/pabort-legacy.S b/arch/arm/mm/pabort-legacy.S index b2ffce4201062e3ec2045364ddc454cf706bab8d..26c62e568638d37133661f4f5b8dec430593f8eb 100644 --- a/arch/arm/mm/pabort-legacy.S +++ b/arch/arm/mm/pabort-legacy.S @@ -18,5 +18,5 @@ ENTRY(legacy_pabort) mov r0, r4 mov r1, #5 - b do_PrefetchAbort + b arm_pabt_handler ENDPROC(legacy_pabort) diff --git a/arch/arm/mm/pabort-v6.S b/arch/arm/mm/pabort-v6.S index 8686265dc9418b29381942bfd87a937a3234d46e..25abd11a35253bf464fe9fd7fc14fd11bc564dcd 100644 --- a/arch/arm/mm/pabort-v6.S +++ b/arch/arm/mm/pabort-v6.S @@ -18,5 +18,5 @@ ENTRY(v6_pabort) mov r0, r4 mrc p15, 0, r1, c5, c0, 1 @ get IFSR - b do_PrefetchAbort + b arm_pabt_handler ENDPROC(v6_pabort) diff --git a/arch/arm/mm/pabort-v7.S b/arch/arm/mm/pabort-v7.S index 9c70b1a21dc9204f24524df9905fbc077a82f2dc..e05c7d44d307adeba6759213374b2aa328c693da 100644 --- a/arch/arm/mm/pabort-v7.S +++ b/arch/arm/mm/pabort-v7.S @@ -18,5 +18,5 @@ ENTRY(v7_pabort) mrc p15, 0, r0, c6, c0, 2 @ get IFAR mrc p15, 0, r1, c5, c0, 1 @ get IFSR - b do_PrefetchAbort + b arm_pabt_handler ENDPROC(v7_pabort) From patchwork Tue Jan 7 09:41:46 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13928637 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A5910E77197 for ; Tue, 7 Jan 2025 10:18:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=YE0gSDhwSaFrDMzYlVMvfGElRXcDHBg9PKYpzyMYroQ=; b=roF3GDaDCVg/Qn1oWXyaQvjNN2 ZByz9YBagaLEGc+P8/XwXEozjtXWMt2ITj4A29upBbt2sMtU/WH9RSJyumPotghKkT8gGePUCyLPN lJxh3eCKo8EyDji6xJydQQ7K4gfc5tYtMVSi+mNdvl/Ecf0UHlBvU7rZJc3a91r0XQIgtZ1r9VaLj TnWqzZZ/hdYLjQN6X8OfRX6RxKMcfuvJXWvihg8CiDulu/SrKxNipXTT5Uvo90gXin5lIWobokLNN wrx1Gq7QonGyJXhs21NvmD62bHHlWPiG7ilkV1Tr2nmMQSseqzQfFokpfzBpOZL1Doer7YLqfW3fm lXORZJ2w==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tV6fX-00000004Ivv-3kVJ; Tue, 07 Jan 2025 10:18:39 +0000 Received: from mail-ej1-x631.google.com ([2a00:1450:4864:20::631]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tV66C-00000004AeK-3Hzn for linux-arm-kernel@lists.infradead.org; Tue, 07 Jan 2025 09:42:13 +0000 Received: by mail-ej1-x631.google.com with SMTP id a640c23a62f3a-aaf900cc7fbso866317866b.3 for ; Tue, 07 Jan 2025 01:42:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1736242927; x=1736847727; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=YE0gSDhwSaFrDMzYlVMvfGElRXcDHBg9PKYpzyMYroQ=; b=Qq4r3cQae1eOp0vdtsp7TH4ZP6n3rZ5p6ReFatij+HXzkIfsEnvmSuEOIjTU6UzApY n0amNbQtG5y8pJjU1XlOm+oMsKrTwI1AB+oUIz02OtpHHfvdXlkDiBH21qWDLGUf+BT1 4yOsIUGUV2q3ecggeW04Ow5IN1JM0xnoDsfYh76Sy4nlPqDriBh9ByXmnIWxhWL1aDiN 4JiyNeVTkflL+LObVrwlKU4vteUzPhLGPy8pdFPpk1RN9jXO4SYHL3EN9u2963OcDip6 4+aN8BFZrbFk6F6/fGRfB96LVowSHVkOrGMSiXXm9L978A0UW1vN1Sc+hJQUdRJcoO24 o/ig== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736242927; x=1736847727; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=YE0gSDhwSaFrDMzYlVMvfGElRXcDHBg9PKYpzyMYroQ=; b=PQx869Wlt39nSernWJdF9SwhDLrXL3DPsgKklJS2VFfiLfRCW2rteWKEwHa0CrAALT B6ScZ7QqtxOLrSdJtfM0tHAAVMVw8rNt9YhfMcHgk0bsRo5/+RX95qqpvmHUgvxuPnyb H2CVwVk+DItmWhUmBAH0Uq5yGfxJ23WBvFPmuvNHkMoNd910Gnb1jyZcw0oC8CSnTnnW 1jf7JqmIlwhbdevAp1eI2eb4IyyWgc7RUxMKkrKT6+1kgEtu4VmOla2wQe8XuLr/iBMX UGenbYb8ah/gH8welALZ6cFieNlML24TSEFagICLpyuJsfeU8KG59LG+j8hAhQKOfW8G xlWA== X-Gm-Message-State: AOJu0YzAh+GIHu5rnbk701S+JgPy+IKIkkiTWXmnveTfmb+qjkU4D4xL o+NrzujjWo5qFDgJtatSKTz+I0UmHzMXsOMvrqNu2zDrigaUq5/IAMR1CTw9A80= X-Gm-Gg: ASbGncv+MjArXyP68dQDLLrF8THsF8P0sup+QdFHHMyc6l6fubUwnlR00f78S/rfpgx ayEwZow1D5b1lHv+UEmP5i0bk6uVlc+1TDRSQIwDbI92TxxvjciO6QkGWjjoyS7vO6x7veZ4SlI N5WK/L2Q0DwdrMIA9ukI/vON8BGnahVozL2rH0IVHU4Gd8VR762hZxHQFYxPIlcBWxxLV0olxB/ K6492KKWphoGov/83YN9JdXausOamr7+lfDoNtSjtlTGN7K4YI/vXv3gGr3Qzsp70s= X-Google-Smtp-Source: AGHT+IFTHzUi4Nr/92cQuZP3zFtg0euFy+Bj/7m2UCAv98WxZqhGhnbcyH+vkfzUSa4ZKvGr627g5A== X-Received: by 2002:a17:907:724f:b0:aae:ebfe:cedb with SMTP id a640c23a62f3a-aaeebfecfc4mr3327387766b.51.1736242926410; Tue, 07 Jan 2025 01:42:06 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aac0f012229sm2356901166b.133.2025.01.07.01.42.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 07 Jan 2025 01:42:05 -0800 (PST) From: Linus Walleij Date: Tue, 07 Jan 2025 10:41:46 +0100 Subject: [PATCH RFC v3 30/30] RFC: ARM: entry: Block IRQs in early IRQ context MIME-Version: 1.0 Message-Id: <20250107-arm-generic-entry-v3-30-4e5f3c15db2d@linaro.org> References: <20250107-arm-generic-entry-v3-0-4e5f3c15db2d@linaro.org> In-Reply-To: <20250107-arm-generic-entry-v3-0-4e5f3c15db2d@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250107_014208_837801_0BA6CF4A X-CRM114-Status: GOOD ( 18.58 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org When dabt, pabt or und exceptions occur on ARM, ordinary interrupts (IRQs) can still happen. This isn't nice for the kernels context tracker, which expect (when using generic entry at least) that any nested IRQs happens between irqentry_enter() and irqentry_exit(), else it thinks something is fishy. This workaround blocks interrupts in the pabt, dabt and und exception paths until after context has been established with irqentry_enter() and before it is exited with irqentry_exit(). This seems like far from perfect: it seems an interrupt could still occur right before the first disable_irq_notrace or right after the userspace registers are restored in ret_from_exception. I would like to know if there is some way to set up these exceptions to inherently block IRQs when handled, until we explicitly allow them between irqentry_enter() and irqentry_exit() or if this is simply the best we can do on ARM for these exceptions to make the context tracker happy. Signed-off-by: Linus Walleij --- arch/arm/kernel/entry-armv.S | 7 ++----- arch/arm/kernel/entry.c | 18 ++++++++++++++++++ 2 files changed, 20 insertions(+), 5 deletions(-) diff --git a/arch/arm/kernel/entry-armv.S b/arch/arm/kernel/entry-armv.S index ae2f952beea7611f0abc7bd299fc944335a21219..10f59b2864a43dc4e1b141513f39eb9404ae9fd0 100644 --- a/arch/arm/kernel/entry-armv.S +++ b/arch/arm/kernel/entry-armv.S @@ -389,6 +389,7 @@ ENDPROC(__fiq_abt) .align 5 __dabt_usr: + disable_irq_notrace usr_entry uaccess=0 kuser_cmpxchg_check mov r2, sp @@ -416,11 +417,6 @@ ENDPROC(__irq_usr) __und_usr: usr_entry uaccess=0 - @ IRQs must be enabled before attempting to read the instruction from - @ user space since that could cause a page/translation fault if the - @ page table was modified by another CPU. - enable_irq - tst r5, #PSR_T_BIT @ Thumb mode? mov r1, #2 @ set insn size to 2 for Thumb bne 0f @ handle as Thumb undef exception @@ -438,6 +434,7 @@ ENDPROC(__und_usr) .align 5 __pabt_usr: + disable_irq_notrace usr_entry mov r2, sp @ regs pabt_helper diff --git a/arch/arm/kernel/entry.c b/arch/arm/kernel/entry.c index 10714dda5753684c7a2b8960562748525de774a8..852e03fac6ce8956415e464a69d41328593d2cc6 100644 --- a/arch/arm/kernel/entry.c +++ b/arch/arm/kernel/entry.c @@ -16,8 +16,18 @@ noinstr asmlinkage void arm_und_handler(struct pt_regs *regs) { irqentry_state_t state = irqentry_enter(regs); + /* + * IRQs must be enabled before attempting to read the instruction from + * user space since that could cause a page/translation fault if the + * page table was modified by another CPU. + */ + + local_irq_enable(); + do_undefinstr(regs); + local_irq_disable(); + irqentry_exit(regs, state); } @@ -25,8 +35,12 @@ noinstr asmlinkage void arm_dabt_handler(unsigned long addr, unsigned int fsr, s { irqentry_state_t state = irqentry_enter(regs); + local_irq_enable(); + do_DataAbort(addr, fsr, regs); + local_irq_disable(); + irqentry_exit(regs, state); } @@ -34,8 +48,12 @@ noinstr asmlinkage void arm_pabt_handler(unsigned long addr, unsigned int ifsr, { irqentry_state_t state = irqentry_enter(regs); + local_irq_enable(); + do_PrefetchAbort(addr, ifsr, regs); + local_irq_disable(); + irqentry_exit(regs, state); }