From patchwork Mon Apr 7 16:19:46 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jeremy Linton X-Patchwork-Id: 14041356 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E4BEDC36010 for ; Mon, 7 Apr 2025 17:10:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=/WMHo01xUwdS8oQQrEyS0O+pWy7gsZCa0aWpjGKO1WY=; b=sSVQ57AleD5zzEbLbFJUrAcmVU +LUyadbXLUYYdxYO9LYvctwHOO7Qosn+mqqseGlnE9EQijIJfs4Enp1gDsuRIqyk2QNnCZgtYNOk8 88VuPz7Ssc6is8BodhG4Kgp3X0XdA40sxDz6jPQ0Wda/U2G0jmIND4p3Y8i95kcdmilCC+msvUwL6 SZGD5/rS0Me4Ceh5Qvl6dnVJML/pB1rZYE1kvvdZzUCN2KKZE92bP7Y4J2HZIVPCWT1qyvYEZKoht /IjWMT/1SuOP4VGulk2AcgQjiZHjod5txS3q3LtS29IZ5W0ZVRZhu4NSl2FuEKXemVKj8PgHi8h+K Zmzmd0bA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98.1 #2 (Red Hat Linux)) id 1u1pyz-00000001KNs-1xb5; Mon, 07 Apr 2025 17:10:01 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.98.1 #2 (Red Hat Linux)) id 1u1pCl-0000000197x-0JbX for linux-arm-kernel@lists.infradead.org; Mon, 07 Apr 2025 16:20:12 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id AAECE12FC; Mon, 7 Apr 2025 09:20:09 -0700 (PDT) Received: from u200865.usa.arm.com (usa-sjc-mx-foss1.foss.arm.com [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 1E8EC3F694; Mon, 7 Apr 2025 09:20:08 -0700 (PDT) From: Jeremy Linton To: linux-trace-kernel@vger.kernel.org Cc: linux-perf-users@vger.kernel.org, mhiramat@kernel.org, oleg@redhat.com, peterz@infradead.org, acme@kernel.org, namhyung@kernel.org, mark.rutland@arm.com, alexander.shishkin@linux.intel.com, jolsa@kernel.org, irogers@google.com, adrian.hunter@intel.com, kan.liang@linux.intel.com, thiago.bauermann@linaro.org, broonie@kernel.org, yury.khrustalev@arm.com, kristina.martsenko@arm.com, liaochang1@huawei.com, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Jeremy Linton Subject: [PATCH v2 1/6] arm64/gcs: task_gcs_el0_enable() should use passed task Date: Mon, 7 Apr 2025 11:19:46 -0500 Message-ID: <20250407161951.560865-2-jeremy.linton@arm.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250407161951.560865-1-jeremy.linton@arm.com> References: <20250407161951.560865-1-jeremy.linton@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250407_092011_160084_DAB8FF82 X-CRM114-Status: GOOD ( 13.08 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Mark Rutland noticed that the task parameter is ignored and 'current' is being used instead. Since this is usually what its passed, it hasn't yet been causing problems but likely will as the code gets more testing. But, once this is fixed, it creates a new bug in copy_thread_gcs() since the gcs_el_mode isn't yet set for the task before its being checked. Move gcs_alloc_thread_stack() after the new task's gcs_el0_mode initialization to avoid this. Fixes: fc84bc5378a8 ("arm64/gcs: Context switch GCS state for EL0") Signed-off-by: Jeremy Linton Reviewed-by: Mark Brown --- arch/arm64/include/asm/gcs.h | 2 +- arch/arm64/kernel/process.c | 6 +++--- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/arch/arm64/include/asm/gcs.h b/arch/arm64/include/asm/gcs.h index f50660603ecf..5bc432234d3a 100644 --- a/arch/arm64/include/asm/gcs.h +++ b/arch/arm64/include/asm/gcs.h @@ -58,7 +58,7 @@ static inline u64 gcsss2(void) static inline bool task_gcs_el0_enabled(struct task_struct *task) { - return current->thread.gcs_el0_mode & PR_SHADOW_STACK_ENABLE; + return task->thread.gcs_el0_mode & PR_SHADOW_STACK_ENABLE; } void gcs_set_el0_mode(struct task_struct *task); diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c index 42faebb7b712..68b7021cda77 100644 --- a/arch/arm64/kernel/process.c +++ b/arch/arm64/kernel/process.c @@ -305,13 +305,13 @@ static int copy_thread_gcs(struct task_struct *p, p->thread.gcs_base = 0; p->thread.gcs_size = 0; + p->thread.gcs_el0_mode = current->thread.gcs_el0_mode; + p->thread.gcs_el0_locked = current->thread.gcs_el0_locked; + gcs = gcs_alloc_thread_stack(p, args); if (IS_ERR_VALUE(gcs)) return PTR_ERR((void *)gcs); - p->thread.gcs_el0_mode = current->thread.gcs_el0_mode; - p->thread.gcs_el0_locked = current->thread.gcs_el0_locked; - return 0; } From patchwork Mon Apr 7 16:19:47 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jeremy Linton X-Patchwork-Id: 14041492 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 882D7C36010 for ; Mon, 7 Apr 2025 18:17:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=iqaCfHqeo4kWqXOJJ73fxqo7S6p2haKZ7NkJE4Aj1KI=; b=ryWV7jgwQO+FqiwZ+wJz6vblAe X3LuGuMiMKRBqgB/3NU6zTHJtO3UtSExBcpg+Koyd2bt7gDoapGwI75wZEOYiHd9XiKvjeCMrCp1M Pj3nOI4da7YGW85zFGMaFBGJm/pB37a7OZNGBW0QcU+/eeIuoa62b7QzWWe7KeLSlj12FpoakF4LP HoA6dZAd9aDz93pFbWFCEmTMwbVPdjuSQtWCAmhhVuqPdIJoKunoldwRqaI+kv+WJ8R+yUkdzMTXZ HC323XflhBjznxzvEVY6q1bxzUZG76xvKl0S60nAq48u6E+pv3/J8zM9zVor/ftVbIRnMcHBFZH4+ 7gYXE4cg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98.1 #2 (Red Hat Linux)) id 1u1r1n-00000001Vka-0ELI; Mon, 07 Apr 2025 18:16:59 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.98.1 #2 (Red Hat Linux)) id 1u1pCk-0000000198F-0zSg for linux-arm-kernel@lists.infradead.org; Mon, 07 Apr 2025 16:20:11 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id E04F21424; Mon, 7 Apr 2025 09:20:10 -0700 (PDT) Received: from u200865.usa.arm.com (usa-sjc-mx-foss1.foss.arm.com [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 532783F694; Mon, 7 Apr 2025 09:20:09 -0700 (PDT) From: Jeremy Linton To: linux-trace-kernel@vger.kernel.org Cc: linux-perf-users@vger.kernel.org, mhiramat@kernel.org, oleg@redhat.com, peterz@infradead.org, acme@kernel.org, namhyung@kernel.org, mark.rutland@arm.com, alexander.shishkin@linux.intel.com, jolsa@kernel.org, irogers@google.com, adrian.hunter@intel.com, kan.liang@linux.intel.com, thiago.bauermann@linaro.org, broonie@kernel.org, yury.khrustalev@arm.com, kristina.martsenko@arm.com, liaochang1@huawei.com, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Jeremy Linton Subject: [PATCH v2 2/6] arm64: probes: Break ret out from bl/blr Date: Mon, 7 Apr 2025 11:19:47 -0500 Message-ID: <20250407161951.560865-3-jeremy.linton@arm.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250407161951.560865-1-jeremy.linton@arm.com> References: <20250407161951.560865-1-jeremy.linton@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250407_092010_367344_59C4FDF1 X-CRM114-Status: GOOD ( 10.80 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Prepare for GCS by breaking RET out into its own function, where it makes more sense to encapsulate the new behavior independent from the branch instructions. Signed-off-by: Jeremy Linton --- arch/arm64/kernel/probes/decode-insn.c | 7 ++++--- arch/arm64/kernel/probes/simulate-insn.c | 10 +++++++++- arch/arm64/kernel/probes/simulate-insn.h | 3 ++- 3 files changed, 15 insertions(+), 5 deletions(-) diff --git a/arch/arm64/kernel/probes/decode-insn.c b/arch/arm64/kernel/probes/decode-insn.c index 6438bf62e753..4137cc5ef031 100644 --- a/arch/arm64/kernel/probes/decode-insn.c +++ b/arch/arm64/kernel/probes/decode-insn.c @@ -108,9 +108,10 @@ arm_probe_decode_insn(u32 insn, struct arch_probe_insn *api) aarch64_insn_is_bl(insn)) { api->handler = simulate_b_bl; } else if (aarch64_insn_is_br(insn) || - aarch64_insn_is_blr(insn) || - aarch64_insn_is_ret(insn)) { - api->handler = simulate_br_blr_ret; + aarch64_insn_is_blr(insn)) { + api->handler = simulate_br_blr; + } else if (aarch64_insn_is_ret(insn)) { + api->handler = simulate_ret; } else { /* * Instruction cannot be stepped out-of-line and we don't diff --git a/arch/arm64/kernel/probes/simulate-insn.c b/arch/arm64/kernel/probes/simulate-insn.c index 4c6d2d712fbd..09a0b36122d0 100644 --- a/arch/arm64/kernel/probes/simulate-insn.c +++ b/arch/arm64/kernel/probes/simulate-insn.c @@ -126,7 +126,7 @@ simulate_b_cond(u32 opcode, long addr, struct pt_regs *regs) } void __kprobes -simulate_br_blr_ret(u32 opcode, long addr, struct pt_regs *regs) +simulate_br_blr(u32 opcode, long addr, struct pt_regs *regs) { int xn = (opcode >> 5) & 0x1f; @@ -138,6 +138,14 @@ simulate_br_blr_ret(u32 opcode, long addr, struct pt_regs *regs) set_x_reg(regs, 30, addr + 4); } +void __kprobes +simulate_ret(u32 opcode, long addr, struct pt_regs *regs) +{ + int xn = (opcode >> 5) & 0x1f; + + instruction_pointer_set(regs, get_x_reg(regs, xn)); +} + void __kprobes simulate_cbz_cbnz(u32 opcode, long addr, struct pt_regs *regs) { diff --git a/arch/arm64/kernel/probes/simulate-insn.h b/arch/arm64/kernel/probes/simulate-insn.h index efb2803ec943..9e772a292d56 100644 --- a/arch/arm64/kernel/probes/simulate-insn.h +++ b/arch/arm64/kernel/probes/simulate-insn.h @@ -11,7 +11,8 @@ void simulate_adr_adrp(u32 opcode, long addr, struct pt_regs *regs); void simulate_b_bl(u32 opcode, long addr, struct pt_regs *regs); void simulate_b_cond(u32 opcode, long addr, struct pt_regs *regs); -void simulate_br_blr_ret(u32 opcode, long addr, struct pt_regs *regs); +void simulate_br_blr(u32 opcode, long addr, struct pt_regs *regs); +void simulate_ret(u32 opcode, long addr, struct pt_regs *regs); void simulate_cbz_cbnz(u32 opcode, long addr, struct pt_regs *regs); void simulate_tbz_tbnz(u32 opcode, long addr, struct pt_regs *regs); void simulate_ldr_literal(u32 opcode, long addr, struct pt_regs *regs); From patchwork Mon Apr 7 16:19:48 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jeremy Linton X-Patchwork-Id: 14041357 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5B843C36010 for ; Mon, 7 Apr 2025 17:11:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=HpvRBpGYEnwpfwDmSNpDyV1CEuLroQggbPFTJqL06A0=; b=NHX49IsWSzgGUrStmefY9unLth /vW4nQvyJYb7xXN2U22h5/6vpBdvwgeNJ25BKY00bTCdpjQ50s6Mq82Mn3KfJPY4yrnXxB7bvRRTa IjtyCNIXUXGhYgNnDY4KE3ect7WkpMUwJbLAagTdhY6FG2NZPS9eKmCcBvQLUDnV8mbesjuOhlofD LNmQlR4qTweRuIH2UjE3mCFD0pBG/6ext4F7Z33huSLVmIt6+2E1O5raLF0/Yp+2D15s3yWFSYRYe /Dia83scK4/9upcaIGM9/VXCHTfpkLlZuIE6VfxeVs9diUKK/odwkya1+n8v6LtFAOkE56hhdTRFh Aduh8Xdg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98.1 #2 (Red Hat Linux)) id 1u1q0j-00000001KcL-0T18; Mon, 07 Apr 2025 17:11:49 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.98.1 #2 (Red Hat Linux)) id 1u1pCl-0000000199N-1RAz for linux-arm-kernel@lists.infradead.org; Mon, 07 Apr 2025 16:20:12 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 006691BC0; Mon, 7 Apr 2025 09:20:12 -0700 (PDT) Received: from u200865.usa.arm.com (usa-sjc-mx-foss1.foss.arm.com [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 68C763F694; Mon, 7 Apr 2025 09:20:10 -0700 (PDT) From: Jeremy Linton To: linux-trace-kernel@vger.kernel.org Cc: linux-perf-users@vger.kernel.org, mhiramat@kernel.org, oleg@redhat.com, peterz@infradead.org, acme@kernel.org, namhyung@kernel.org, mark.rutland@arm.com, alexander.shishkin@linux.intel.com, jolsa@kernel.org, irogers@google.com, adrian.hunter@intel.com, kan.liang@linux.intel.com, thiago.bauermann@linaro.org, broonie@kernel.org, yury.khrustalev@arm.com, kristina.martsenko@arm.com, liaochang1@huawei.com, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Jeremy Linton Subject: [PATCH v2 3/6] arm64: uaccess: Add additional userspace GCS accessors Date: Mon, 7 Apr 2025 11:19:48 -0500 Message-ID: <20250407161951.560865-4-jeremy.linton@arm.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250407161951.560865-1-jeremy.linton@arm.com> References: <20250407161951.560865-1-jeremy.linton@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250407_092011_432881_88185CF8 X-CRM114-Status: GOOD ( 12.99 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Uprobes need more advanced read, push, and pop userspace GCS functionality. Implement those features using the existing gcsstr() and copy_from_user(). Its important to note that GCS pages can be read by normal instructions, but the hardware validates that pages used by GCS specific operations, have a GCS privilege set. We aren't validating this in load_user_gcs because it requires stabilizing the VMA over the read which may fault. Signed-off-by: Jeremy Linton --- arch/arm64/include/asm/uaccess.h | 42 ++++++++++++++++++++++++++++++++ 1 file changed, 42 insertions(+) diff --git a/arch/arm64/include/asm/uaccess.h b/arch/arm64/include/asm/uaccess.h index 5b91803201ef..34a8b2cc8935 100644 --- a/arch/arm64/include/asm/uaccess.h +++ b/arch/arm64/include/asm/uaccess.h @@ -20,6 +20,7 @@ #include #include +#include #include #include #include @@ -539,6 +540,47 @@ static inline void put_user_gcs(unsigned long val, unsigned long __user *addr, uaccess_ttbr0_disable(); } +static __always_inline unsigned long __must_check +copy_from_user(void *to, const void __user *from, unsigned long n); + +/* + * Unlike put_user_gcs() above, the use of copy_from_user() may provide + * an opening for non GCS pages to be used to source data. Therefore this + * should only be used in contexts where that is acceptable. + */ +static inline u64 load_user_gcs(unsigned long __user *addr, int *err) +{ + unsigned long ret; + u64 load = 0; + + gcsb_dsync(); + ret = copy_from_user(&load, addr, sizeof(load)); + if (ret != 0) + *err = ret; + return load; +} + +static inline void push_user_gcs(unsigned long val, int *err) +{ + u64 gcspr = read_sysreg_s(SYS_GCSPR_EL0); + + gcspr -= sizeof(u64); + put_user_gcs(val, (unsigned long __user *)gcspr, err); + if (!*err) + write_sysreg_s(gcspr, SYS_GCSPR_EL0); +} + +static inline u64 pop_user_gcs(int *err) +{ + u64 gcspr = read_sysreg_s(SYS_GCSPR_EL0); + u64 read_val; + + read_val = load_user_gcs((unsigned long __user *)gcspr, err); + if (!*err) + write_sysreg_s(gcspr + sizeof(u64), SYS_GCSPR_EL0); + + return read_val; +} #endif /* CONFIG_ARM64_GCS */ From patchwork Mon Apr 7 16:19:49 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jeremy Linton X-Patchwork-Id: 14041360 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D949AC36010 for ; Mon, 7 Apr 2025 17:13:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=gLc5v44BbBjHGH2R/IosxNm2fZiVevhZk8y6Zg923ko=; b=cC6lwzBizQoknWg7PotBbXiCKT /uhI/RyT84RQ6priAycljofd1Begt26gTlX1cPxJAdHmsvQj5/l3ZFXllkO05DS0m0tLt0nXyWYFl JRslfBb3BTLkL9dfvK+nshBN2ZGeo4gJ1DSV+rhx2dfD4gaaGrcG7xBCIOh3lD5U5lYBuGwbdGXwJ 3bURudZfzmb4peXMcT1d3Dl6E9KeInZCf09NTQoYBZinl1UmKNB4ctoW1E64esE0o43QZNHiQYZpF gyjOIwvUjt6J8q4gFZFRqxcKlN/JKn+UiiW53cbDa/1BYbl2JQfQbLkVlxnbZm9qhNIiQmSV00QXQ ifaKWs9g==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98.1 #2 (Red Hat Linux)) id 1u1q2S-00000001KzV-1jLi; Mon, 07 Apr 2025 17:13:36 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.98.1 #2 (Red Hat Linux)) id 1u1pCm-000000019AP-1uCJ for linux-arm-kernel@lists.infradead.org; Mon, 07 Apr 2025 16:20:13 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 0F6E91BCB; Mon, 7 Apr 2025 09:20:13 -0700 (PDT) Received: from u200865.usa.arm.com (usa-sjc-mx-foss1.foss.arm.com [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 784343F694; Mon, 7 Apr 2025 09:20:11 -0700 (PDT) From: Jeremy Linton To: linux-trace-kernel@vger.kernel.org Cc: linux-perf-users@vger.kernel.org, mhiramat@kernel.org, oleg@redhat.com, peterz@infradead.org, acme@kernel.org, namhyung@kernel.org, mark.rutland@arm.com, alexander.shishkin@linux.intel.com, jolsa@kernel.org, irogers@google.com, adrian.hunter@intel.com, kan.liang@linux.intel.com, thiago.bauermann@linaro.org, broonie@kernel.org, yury.khrustalev@arm.com, kristina.martsenko@arm.com, liaochang1@huawei.com, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Jeremy Linton Subject: [PATCH v2 4/6] arm64: probes: Add GCS support to bl/blr/ret Date: Mon, 7 Apr 2025 11:19:49 -0500 Message-ID: <20250407161951.560865-5-jeremy.linton@arm.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250407161951.560865-1-jeremy.linton@arm.com> References: <20250407161951.560865-1-jeremy.linton@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250407_092012_591344_ABF8EFF0 X-CRM114-Status: GOOD ( 13.12 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The arm64 probe simulation doesn't currently have logic in place to deal with GCS and this results in core dumps if probes are inserted at control flow locations. Fix-up bl, blr and ret to manipulate the shadow stack as needed. While we manipulate and validate the shadow stack correctly, the hardware provides additional security by only allowing GCS operations against pages which are marked to support GCS. For writing there is gcssttr() which enforces this, but there isn't an equivalent for reading. This means that uprobe users should be aware that probing on control flow instructions which require reading the shadow stack (ex: ret) offers lower security guarantees than what is achieved without the uprobe active. Signed-off-by: Jeremy Linton --- arch/arm64/kernel/probes/simulate-insn.c | 28 ++++++++++++++++++++---- 1 file changed, 24 insertions(+), 4 deletions(-) diff --git a/arch/arm64/kernel/probes/simulate-insn.c b/arch/arm64/kernel/probes/simulate-insn.c index 09a0b36122d0..1fc9bb69b1eb 100644 --- a/arch/arm64/kernel/probes/simulate-insn.c +++ b/arch/arm64/kernel/probes/simulate-insn.c @@ -13,6 +13,7 @@ #include #include "simulate-insn.h" +#include "asm/gcs.h" #define bbl_displacement(insn) \ sign_extend32(((insn) & 0x3ffffff) << 2, 27) @@ -49,6 +50,18 @@ static inline u32 get_w_reg(struct pt_regs *regs, int reg) return lower_32_bits(pt_regs_read_reg(regs, reg)); } +static inline void update_lr(struct pt_regs *regs, long addr) +{ + int err = 0; + + if (user_mode(regs) && task_gcs_el0_enabled(current)) { + push_user_gcs(addr + 4, &err); + if (err) + force_sig(SIGSEGV); + } + procedure_link_pointer_set(regs, addr + 4); +} + static bool __kprobes check_cbz(u32 opcode, struct pt_regs *regs) { int xn = opcode & 0x1f; @@ -107,9 +120,8 @@ simulate_b_bl(u32 opcode, long addr, struct pt_regs *regs) { int disp = bbl_displacement(opcode); - /* Link register is x30 */ if (opcode & (1 << 31)) - set_x_reg(regs, 30, addr + 4); + update_lr(regs, addr); instruction_pointer_set(regs, addr + disp); } @@ -133,17 +145,25 @@ simulate_br_blr(u32 opcode, long addr, struct pt_regs *regs) /* update pc first in case we're doing a "blr lr" */ instruction_pointer_set(regs, get_x_reg(regs, xn)); - /* Link register is x30 */ if (((opcode >> 21) & 0x3) == 1) - set_x_reg(regs, 30, addr + 4); + update_lr(regs, addr); } void __kprobes simulate_ret(u32 opcode, long addr, struct pt_regs *regs) { + u64 ret_addr; + int err = 0; int xn = (opcode >> 5) & 0x1f; instruction_pointer_set(regs, get_x_reg(regs, xn)); + + if (user_mode(regs) && task_gcs_el0_enabled(current)) { + ret_addr = pop_user_gcs(&err); + if (err || ret_addr != procedure_link_pointer(regs)) + force_sig(SIGSEGV); + } + } void __kprobes From patchwork Mon Apr 7 16:19:50 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jeremy Linton X-Patchwork-Id: 14041361 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 94230C36010 for ; Mon, 7 Apr 2025 17:15:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=MmJI79xOX++xDBcrvPihMsLbiYiuLuD+bYY7rFVQsyw=; b=p09K0ERd1YKWUgBjEF2ILG8xRE 15Om8w6lOg5dyKDGlonEIG9af5B0gvdo16AaUx3VpG7gai+RIf+uNdz0vwZ/MTbAruRSG49NLAfmR lv3QMoA6uXs5WC76+9sIqXmMTai5itz95i/wFc+LQLcYsRLQG0J1Vr2pvAnYM5UFLNVULuQmfye5e 1Hz4c0JsnIvMrRGJVLqEAdxG0svag2ziyyS2WWMz7ndD99+DomPFXHnEVTyvNoq62rjjDxTSW8MjM m0qUVV0dWgWZ9eeDPLpR8KSC000JBusrAbKtYy16M+UmJQUvPdxkBLb1xwahCFXzUOuo5gY5hiwUC FxpKVUXQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98.1 #2 (Red Hat Linux)) id 1u1q4B-00000001LC4-0cEg; Mon, 07 Apr 2025 17:15:23 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.98.1 #2 (Red Hat Linux)) id 1u1pCn-000000019B9-1SSh for linux-arm-kernel@lists.infradead.org; Mon, 07 Apr 2025 16:20:14 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 1EAE21BF3; Mon, 7 Apr 2025 09:20:14 -0700 (PDT) Received: from u200865.usa.arm.com (usa-sjc-mx-foss1.foss.arm.com [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 822873F694; Mon, 7 Apr 2025 09:20:12 -0700 (PDT) From: Jeremy Linton To: linux-trace-kernel@vger.kernel.org Cc: linux-perf-users@vger.kernel.org, mhiramat@kernel.org, oleg@redhat.com, peterz@infradead.org, acme@kernel.org, namhyung@kernel.org, mark.rutland@arm.com, alexander.shishkin@linux.intel.com, jolsa@kernel.org, irogers@google.com, adrian.hunter@intel.com, kan.liang@linux.intel.com, thiago.bauermann@linaro.org, broonie@kernel.org, yury.khrustalev@arm.com, kristina.martsenko@arm.com, liaochang1@huawei.com, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Jeremy Linton , Steve Capper Subject: [PATCH v2 5/6] arm64: uprobes: Add GCS support to uretprobes Date: Mon, 7 Apr 2025 11:19:50 -0500 Message-ID: <20250407161951.560865-6-jeremy.linton@arm.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250407161951.560865-1-jeremy.linton@arm.com> References: <20250407161951.560865-1-jeremy.linton@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250407_092013_417989_36FA97D1 X-CRM114-Status: GOOD ( 15.85 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Ret probes work by changing the value in the link register at the probe location to return to the probe rather than the calling routine. Thus the GCS needs to be updated with this address as well. Since its possible to insert probes at locations where the current value of the LR doesn't match the GCS state this needs to be detected and handled in order to maintain the existing no-fault behavior. Co-developed-by: Steve Capper Signed-off-by: Steve Capper (updated to use new gcs accessors, and handle LR/GCS mismatches) Signed-off-by: Jeremy Linton --- arch/arm64/kernel/probes/uprobes.c | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/arch/arm64/kernel/probes/uprobes.c b/arch/arm64/kernel/probes/uprobes.c index cb3d05af36e3..5e72409a255a 100644 --- a/arch/arm64/kernel/probes/uprobes.c +++ b/arch/arm64/kernel/probes/uprobes.c @@ -159,11 +159,41 @@ arch_uretprobe_hijack_return_addr(unsigned long trampoline_vaddr, struct pt_regs *regs) { unsigned long orig_ret_vaddr; + unsigned long gcs_ret_vaddr; + int err = 0; + u64 gcspr; orig_ret_vaddr = procedure_link_pointer(regs); + + if (task_gcs_el0_enabled(current)) { + gcspr = read_sysreg_s(SYS_GCSPR_EL0); + gcs_ret_vaddr = load_user_gcs((unsigned long __user *)gcspr, &err); + if (err) { + force_sig(SIGSEGV); + goto out; + } + /* + * If the LR and GCS entry don't match, then some kind of PAC/control + * flow happened. Likely because the user is attempting to retprobe + * on something that isn't a function boundary or inside a leaf + * function. Explicitly abort this retprobe because it will generate + * a GCS exception. + */ + if (gcs_ret_vaddr != orig_ret_vaddr) { + orig_ret_vaddr = -1; + goto out; + } + put_user_gcs(trampoline_vaddr, (unsigned long __user *) gcspr, &err); + if (err) { + force_sig(SIGSEGV); + goto out; + } + } + /* Replace the return addr with trampoline addr */ procedure_link_pointer_set(regs, trampoline_vaddr); +out: return orig_ret_vaddr; } From patchwork Mon Apr 7 16:19:51 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jeremy Linton X-Patchwork-Id: 14041363 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B1823C36018 for ; Mon, 7 Apr 2025 17:17:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=A6ZCoBeH0fW0QPxmV5vmWVn1dC4/5hnvYV1OAX1MulU=; b=whoKbL5sUJndHTktUxk31c9jow Ydhozj+s+TUdzZbKyfvbX4n7sZzJ07q5L5ZMpkK3VoSBTdwF60T/GuHIJvPq42TX0k+zvg1zvD7O2 vN6Z7BqyQ3COdTZsO9YpW3BcM+ZjxeEhKHCF0R3UTDG27sbTHH4H54xoLfVTDrgckfXQOoV3bjhZI ouIVqI5QV62tRPfw1M5+R+Klr6i92C6r67YLHGRA/5mjFX65IrLYGI40LihhdzFGUdbvKDet5LrYn O9GUnVcIatwXEU5VcB9xFZz2ub0yTO7J8VEIkE/7r4nJq1z0z4kSQrXkdPXH4Kq3Q6nbVW15pMst1 n0ft1CdA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98.1 #2 (Red Hat Linux)) id 1u1q5t-00000001LSv-3OvL; Mon, 07 Apr 2025 17:17:09 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.98.1 #2 (Red Hat Linux)) id 1u1pCo-000000019AP-2CrL for linux-arm-kernel@lists.infradead.org; Mon, 07 Apr 2025 16:20:15 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 3C4251CC4; Mon, 7 Apr 2025 09:20:15 -0700 (PDT) Received: from u200865.usa.arm.com (usa-sjc-mx-foss1.foss.arm.com [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 9C87B3F694; Mon, 7 Apr 2025 09:20:13 -0700 (PDT) From: Jeremy Linton To: linux-trace-kernel@vger.kernel.org Cc: linux-perf-users@vger.kernel.org, mhiramat@kernel.org, oleg@redhat.com, peterz@infradead.org, acme@kernel.org, namhyung@kernel.org, mark.rutland@arm.com, alexander.shishkin@linux.intel.com, jolsa@kernel.org, irogers@google.com, adrian.hunter@intel.com, kan.liang@linux.intel.com, thiago.bauermann@linaro.org, broonie@kernel.org, yury.khrustalev@arm.com, kristina.martsenko@arm.com, liaochang1@huawei.com, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Jeremy Linton Subject: [PATCH v2 6/6] arm64: Kconfig: Remove GCS restrictions on UPROBES Date: Mon, 7 Apr 2025 11:19:51 -0500 Message-ID: <20250407161951.560865-7-jeremy.linton@arm.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250407161951.560865-1-jeremy.linton@arm.com> References: <20250407161951.560865-1-jeremy.linton@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250407_092014_613271_1343029E X-CRM114-Status: UNSURE ( 8.30 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Now that the uprobe paths have been made GCS compatible drop the Kconfig restriction. Signed-off-by: Jeremy Linton --- arch/arm64/Kconfig | 1 - 1 file changed, 1 deletion(-) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index a182295e6f08..b962a1321ebf 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -2236,7 +2236,6 @@ config ARM64_GCS default y select ARCH_HAS_USER_SHADOW_STACK select ARCH_USES_HIGH_VMA_FLAGS - depends on !UPROBES help Guarded Control Stack (GCS) provides support for a separate stack with restricted access which contains only return