From patchwork Tue Mar 18 20:48:35 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jeremy Linton X-Patchwork-Id: 14021528 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 8957F2054E1; Tue, 18 Mar 2025 20:48:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1742330935; cv=none; b=UNW4bEhVRkUJR6ifx1L/GH0RYUY7O3J2ZJUNQz0lMiQDe/DM4lIt3cAVi9HPypzdBq+nyYXpOcmLVoIZR94dBxO1KlvCggc6Ti7TnZ/J8tkpyiCAfqQNL9nN5JeDM8y6vFaAKfQ4kZFWCEaJ4KkntQO2gWw1tf74p7cBAsKCiBs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1742330935; c=relaxed/simple; bh=F3ytnp5DWEyy9tJeog9eRtAnMiiP3aX2gazyR9/19Ms=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=SxH3P3s0K11fg/XNv85gWNam1Xd2p8hVR42huBvsr1K9cR0p0DIqn2JyufHgvJ8lXZosBzcD30d0oShjY6s+PLukQzFrgsxQZYCjltQ9+P+4T3dRbJOFArLArJO6Nrr5oy5uHdYR2JZhonj7FRBwNVsh81hwKwuapqG4a1lmRsk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id D6B8B150C; Tue, 18 Mar 2025 13:49:00 -0700 (PDT) Received: from u200865.usa.arm.com (usa-sjc-mx-foss1.foss.arm.com [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id E67FB3F694; Tue, 18 Mar 2025 13:48:51 -0700 (PDT) From: Jeremy Linton To: linux-trace-kernel@vger.kernel.org Cc: linux-perf-users@vger.kernel.org, mhiramat@kernel.org, oleg@redhat.com, peterz@infradead.org, acme@kernel.org, namhyung@kernel.org, mark.rutland@arm.com, alexander.shishkin@linux.intel.com, jolsa@kernel.org, irogers@google.com, adrian.hunter@intel.com, kan.liang@linux.intel.com, thiago.bauermann@linaro.org, broonie@kernel.org, yury.khrustalev@arm.com, kristina.martsenko@arm.com, liaochang1@huawei.com, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Jeremy Linton Subject: [PATCH 1/7] arm64/gcs: task_gcs_el0_enable() should use passed task Date: Tue, 18 Mar 2025 15:48:35 -0500 Message-ID: <20250318204841.373116-2-jeremy.linton@arm.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250318204841.373116-1-jeremy.linton@arm.com> References: <20250318204841.373116-1-jeremy.linton@arm.com> Precedence: bulk X-Mailing-List: linux-trace-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Mark Rutland noticed that the task parameter is ignored and 'current' is being used instead. Since this is usually what its passed, it hasn't yet been causing problems but likely will as the code gets more testing. Fixes: fc84bc5378a8 ("arm64/gcs: Context switch GCS state for EL0") Signed-off-by: Jeremy Linton Reviewed-by: Mark Brown --- arch/arm64/include/asm/gcs.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/arm64/include/asm/gcs.h b/arch/arm64/include/asm/gcs.h index f50660603ecf..5bc432234d3a 100644 --- a/arch/arm64/include/asm/gcs.h +++ b/arch/arm64/include/asm/gcs.h @@ -58,7 +58,7 @@ static inline u64 gcsss2(void) static inline bool task_gcs_el0_enabled(struct task_struct *task) { - return current->thread.gcs_el0_mode & PR_SHADOW_STACK_ENABLE; + return task->thread.gcs_el0_mode & PR_SHADOW_STACK_ENABLE; } void gcs_set_el0_mode(struct task_struct *task); From patchwork Tue Mar 18 20:48:36 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jeremy Linton X-Patchwork-Id: 14021529 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 7E9631DE883; Tue, 18 Mar 2025 20:48:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1742330936; cv=none; b=MmlI78fpMy5m4tX4zvt7Ytie9QkxZNLbgqxim94Enwu6Cb/abBgqeNJ5q+4ac8UkptUZOVXaVwfJgURBgr4EaSrzIqyShxetra0kxwZ2qHlg8w6qvcBK+kvpra0NjUMuK2BNIQWqZIxE3RPxQ3+mgN+Q/qcugXz/N0FRrRawd2I= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1742330936; c=relaxed/simple; bh=tTuAWTFCJuczMf2rlYxvUvaUyH5oFIore+Iw+GKc1sc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=HKCM2EJ+wt5O7AtROI11ZgY9Z7TVnF2hRmVVuEz3VbajlvdMtg8Gv1cUQBa9yFjAhZ26TR3MKUsI1I7GC6mRe7+4BGWLp05EkJpO1XdT9Q2r0ePb575501l23bRcUEoYp3f+9X3qGeiBeVxqVVB1ZCQ+ViMWn6G02NUVYhZOV+M= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 1606B1515; Tue, 18 Mar 2025 13:49:02 -0700 (PDT) Received: from u200865.usa.arm.com (usa-sjc-mx-foss1.foss.arm.com [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 2386E3F694; Tue, 18 Mar 2025 13:48:53 -0700 (PDT) From: Jeremy Linton To: linux-trace-kernel@vger.kernel.org Cc: linux-perf-users@vger.kernel.org, mhiramat@kernel.org, oleg@redhat.com, peterz@infradead.org, acme@kernel.org, namhyung@kernel.org, mark.rutland@arm.com, alexander.shishkin@linux.intel.com, jolsa@kernel.org, irogers@google.com, adrian.hunter@intel.com, kan.liang@linux.intel.com, thiago.bauermann@linaro.org, broonie@kernel.org, yury.khrustalev@arm.com, kristina.martsenko@arm.com, liaochang1@huawei.com, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Jeremy Linton Subject: [PATCH 2/7] arm64: probes: Break ret out from bl/blr Date: Tue, 18 Mar 2025 15:48:36 -0500 Message-ID: <20250318204841.373116-3-jeremy.linton@arm.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250318204841.373116-1-jeremy.linton@arm.com> References: <20250318204841.373116-1-jeremy.linton@arm.com> Precedence: bulk X-Mailing-List: linux-trace-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Prepare for GCS by breaking RET out into its own function, where it makes more sense to encapsulate the new behavior independent from the branch instructions. Signed-off-by: Jeremy Linton --- arch/arm64/kernel/probes/decode-insn.c | 7 ++++--- arch/arm64/kernel/probes/simulate-insn.c | 10 +++++++++- arch/arm64/kernel/probes/simulate-insn.h | 3 ++- 3 files changed, 15 insertions(+), 5 deletions(-) diff --git a/arch/arm64/kernel/probes/decode-insn.c b/arch/arm64/kernel/probes/decode-insn.c index 6438bf62e753..4137cc5ef031 100644 --- a/arch/arm64/kernel/probes/decode-insn.c +++ b/arch/arm64/kernel/probes/decode-insn.c @@ -108,9 +108,10 @@ arm_probe_decode_insn(u32 insn, struct arch_probe_insn *api) aarch64_insn_is_bl(insn)) { api->handler = simulate_b_bl; } else if (aarch64_insn_is_br(insn) || - aarch64_insn_is_blr(insn) || - aarch64_insn_is_ret(insn)) { - api->handler = simulate_br_blr_ret; + aarch64_insn_is_blr(insn)) { + api->handler = simulate_br_blr; + } else if (aarch64_insn_is_ret(insn)) { + api->handler = simulate_ret; } else { /* * Instruction cannot be stepped out-of-line and we don't diff --git a/arch/arm64/kernel/probes/simulate-insn.c b/arch/arm64/kernel/probes/simulate-insn.c index 4c6d2d712fbd..09a0b36122d0 100644 --- a/arch/arm64/kernel/probes/simulate-insn.c +++ b/arch/arm64/kernel/probes/simulate-insn.c @@ -126,7 +126,7 @@ simulate_b_cond(u32 opcode, long addr, struct pt_regs *regs) } void __kprobes -simulate_br_blr_ret(u32 opcode, long addr, struct pt_regs *regs) +simulate_br_blr(u32 opcode, long addr, struct pt_regs *regs) { int xn = (opcode >> 5) & 0x1f; @@ -138,6 +138,14 @@ simulate_br_blr_ret(u32 opcode, long addr, struct pt_regs *regs) set_x_reg(regs, 30, addr + 4); } +void __kprobes +simulate_ret(u32 opcode, long addr, struct pt_regs *regs) +{ + int xn = (opcode >> 5) & 0x1f; + + instruction_pointer_set(regs, get_x_reg(regs, xn)); +} + void __kprobes simulate_cbz_cbnz(u32 opcode, long addr, struct pt_regs *regs) { diff --git a/arch/arm64/kernel/probes/simulate-insn.h b/arch/arm64/kernel/probes/simulate-insn.h index efb2803ec943..9e772a292d56 100644 --- a/arch/arm64/kernel/probes/simulate-insn.h +++ b/arch/arm64/kernel/probes/simulate-insn.h @@ -11,7 +11,8 @@ void simulate_adr_adrp(u32 opcode, long addr, struct pt_regs *regs); void simulate_b_bl(u32 opcode, long addr, struct pt_regs *regs); void simulate_b_cond(u32 opcode, long addr, struct pt_regs *regs); -void simulate_br_blr_ret(u32 opcode, long addr, struct pt_regs *regs); +void simulate_br_blr(u32 opcode, long addr, struct pt_regs *regs); +void simulate_ret(u32 opcode, long addr, struct pt_regs *regs); void simulate_cbz_cbnz(u32 opcode, long addr, struct pt_regs *regs); void simulate_tbz_tbnz(u32 opcode, long addr, struct pt_regs *regs); void simulate_ldr_literal(u32 opcode, long addr, struct pt_regs *regs); From patchwork Tue Mar 18 20:48:37 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jeremy Linton X-Patchwork-Id: 14021530 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id D39B32135B7; Tue, 18 Mar 2025 20:48:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1742330937; cv=none; b=h2exm3YLHS2Ga9J2YrmB/krSjYK1KellRp1gzHLSzMDp00E9ngVr+yiv/rQdB2iadOiQPE02IKXQSVA/qzEuQiMRq/+iT8Z/x15G6qJnbUrkznVRfJrqJGkVrvGPGZHEDBBvu02BX6J/ThTlmBcHbkPuPezun15BEeB3gqSkrtE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1742330937; c=relaxed/simple; bh=0+ASMQonCa6VNQmKFdG9ZcQ9NjvWlGD5BV12PUV8YXM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=RIN25qigMNzTkLp2sfAWu9wz56ZKQVED+jyt4nheEvhbmTtL9oguOExQuAf500Xnt+ysogW90z0I9tJYiqtjju2fMlqjZU5BRR01IjTaSHWwKqq7VfGx8cGN3tfkaztHJI3VgHaLFRdOTPBHfBZ/14StfEM738dP3k7ZkA1oo90= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 344CD1A2D; Tue, 18 Mar 2025 13:49:03 -0700 (PDT) Received: from u200865.usa.arm.com (usa-sjc-mx-foss1.foss.arm.com [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 371503F694; Tue, 18 Mar 2025 13:48:54 -0700 (PDT) From: Jeremy Linton To: linux-trace-kernel@vger.kernel.org Cc: linux-perf-users@vger.kernel.org, mhiramat@kernel.org, oleg@redhat.com, peterz@infradead.org, acme@kernel.org, namhyung@kernel.org, mark.rutland@arm.com, alexander.shishkin@linux.intel.com, jolsa@kernel.org, irogers@google.com, adrian.hunter@intel.com, kan.liang@linux.intel.com, thiago.bauermann@linaro.org, broonie@kernel.org, yury.khrustalev@arm.com, kristina.martsenko@arm.com, liaochang1@huawei.com, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Jeremy Linton Subject: [PATCH 3/7] arm64: uaccess: Add additional userspace GCS accessors Date: Tue, 18 Mar 2025 15:48:37 -0500 Message-ID: <20250318204841.373116-4-jeremy.linton@arm.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250318204841.373116-1-jeremy.linton@arm.com> References: <20250318204841.373116-1-jeremy.linton@arm.com> Precedence: bulk X-Mailing-List: linux-trace-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Uprobes need more advanced read, push, and pop userspace GCS functionality. Implement those features using the existing gcsstr() and copy_from_user(). Signed-off-by: Jeremy Linton --- arch/arm64/include/asm/uaccess.h | 42 ++++++++++++++++++++++++++++++++ 1 file changed, 42 insertions(+) diff --git a/arch/arm64/include/asm/uaccess.h b/arch/arm64/include/asm/uaccess.h index 5b91803201ef..c77ab09a01c2 100644 --- a/arch/arm64/include/asm/uaccess.h +++ b/arch/arm64/include/asm/uaccess.h @@ -20,6 +20,7 @@ #include #include +#include #include #include #include @@ -539,6 +540,47 @@ static inline void put_user_gcs(unsigned long val, unsigned long __user *addr, uaccess_ttbr0_disable(); } +static __always_inline unsigned long __must_check +copy_from_user(void *to, const void __user *from, unsigned long n); + +static inline u64 load_user_gcs(unsigned long __user *addr, int *err) +{ + unsigned long ret; + u64 load; + + if (!access_ok((char __user *)addr, sizeof(load))) { + *err = -EFAULT; + return 0; + } + + gcsb_dsync(); + ret = copy_from_user(&load, addr, sizeof(load)); + if (ret != 0) + *err = ret; + return load; +} + +static inline void push_user_gcs(unsigned long val, int *err) +{ + u64 gcspr = read_sysreg_s(SYS_GCSPR_EL0); + + gcspr -= sizeof(u64); + put_user_gcs(val, (unsigned long __user *)gcspr, err); + if (!*err) + write_sysreg_s(gcspr, SYS_GCSPR_EL0); +} + +static inline u64 pop_user_gcs(int *err) +{ + u64 gcspr = read_sysreg_s(SYS_GCSPR_EL0); + u64 read_val; + + read_val = load_user_gcs((unsigned long __user *)gcspr, err); + if (!*err) + write_sysreg_s(gcspr + sizeof(u64), SYS_GCSPR_EL0); + + return read_val; +} #endif /* CONFIG_ARM64_GCS */ From patchwork Tue Mar 18 20:48:38 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jeremy Linton X-Patchwork-Id: 14021531 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 880B920468F; Tue, 18 Mar 2025 20:48:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1742330938; cv=none; b=ocWJ9vXujTAnR0yZk15TCzQQCtcnTerGrb5YREtglUDqtk3i9z52DaFTWOl5rzYjMA0YeR/bAutBb00A4NxQrdcplyOk/rBEeiNQH43fF6msc3zo2GfgRTOZ8UQnrre1PZe5bun1XtKbGanNgEoYayK9umx0wEGFNbABGaVenUo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1742330938; c=relaxed/simple; bh=NmNliEr7qlockhahAe07m+QPos+G1IBiNIM5VnzmLew=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=C2YOsJOtjwMcLgLAxoh0GyLOaJfuzraOgYKf+n0990V30klIq56j1UpzsQl1leziz3G0YhvjGRyckDaFl/8XhxFT9qMrE4bvCa0T49mo1XilQtUxKYjJWQrOJZ1mO8Z+HJPAod138Xh108Gzjk/xIgGjR+iJQBsRv0f+W/CkAsA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 4141B1BB2; Tue, 18 Mar 2025 13:49:04 -0700 (PDT) Received: from u200865.usa.arm.com (usa-sjc-mx-foss1.foss.arm.com [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 501D23F694; Tue, 18 Mar 2025 13:48:55 -0700 (PDT) From: Jeremy Linton To: linux-trace-kernel@vger.kernel.org Cc: linux-perf-users@vger.kernel.org, mhiramat@kernel.org, oleg@redhat.com, peterz@infradead.org, acme@kernel.org, namhyung@kernel.org, mark.rutland@arm.com, alexander.shishkin@linux.intel.com, jolsa@kernel.org, irogers@google.com, adrian.hunter@intel.com, kan.liang@linux.intel.com, thiago.bauermann@linaro.org, broonie@kernel.org, yury.khrustalev@arm.com, kristina.martsenko@arm.com, liaochang1@huawei.com, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Jeremy Linton Subject: [PATCH 4/7] arm64: probes: Add GCS support to bl/blr/ret Date: Tue, 18 Mar 2025 15:48:38 -0500 Message-ID: <20250318204841.373116-5-jeremy.linton@arm.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250318204841.373116-1-jeremy.linton@arm.com> References: <20250318204841.373116-1-jeremy.linton@arm.com> Precedence: bulk X-Mailing-List: linux-trace-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 The arm64 probe simulation doesn't currently have logic in place to deal with GCS and this results in core dumps if probes are inserted at control flow locations. Fix-up bl, blr and ret to manipulate the shadow stack as needed. Signed-off-by: Jeremy Linton --- arch/arm64/kernel/probes/simulate-insn.c | 28 ++++++++++++++++++++---- 1 file changed, 24 insertions(+), 4 deletions(-) diff --git a/arch/arm64/kernel/probes/simulate-insn.c b/arch/arm64/kernel/probes/simulate-insn.c index 09a0b36122d0..1fc9bb69b1eb 100644 --- a/arch/arm64/kernel/probes/simulate-insn.c +++ b/arch/arm64/kernel/probes/simulate-insn.c @@ -13,6 +13,7 @@ #include #include "simulate-insn.h" +#include "asm/gcs.h" #define bbl_displacement(insn) \ sign_extend32(((insn) & 0x3ffffff) << 2, 27) @@ -49,6 +50,18 @@ static inline u32 get_w_reg(struct pt_regs *regs, int reg) return lower_32_bits(pt_regs_read_reg(regs, reg)); } +static inline void update_lr(struct pt_regs *regs, long addr) +{ + int err = 0; + + if (user_mode(regs) && task_gcs_el0_enabled(current)) { + push_user_gcs(addr + 4, &err); + if (err) + force_sig(SIGSEGV); + } + procedure_link_pointer_set(regs, addr + 4); +} + static bool __kprobes check_cbz(u32 opcode, struct pt_regs *regs) { int xn = opcode & 0x1f; @@ -107,9 +120,8 @@ simulate_b_bl(u32 opcode, long addr, struct pt_regs *regs) { int disp = bbl_displacement(opcode); - /* Link register is x30 */ if (opcode & (1 << 31)) - set_x_reg(regs, 30, addr + 4); + update_lr(regs, addr); instruction_pointer_set(regs, addr + disp); } @@ -133,17 +145,25 @@ simulate_br_blr(u32 opcode, long addr, struct pt_regs *regs) /* update pc first in case we're doing a "blr lr" */ instruction_pointer_set(regs, get_x_reg(regs, xn)); - /* Link register is x30 */ if (((opcode >> 21) & 0x3) == 1) - set_x_reg(regs, 30, addr + 4); + update_lr(regs, addr); } void __kprobes simulate_ret(u32 opcode, long addr, struct pt_regs *regs) { + u64 ret_addr; + int err = 0; int xn = (opcode >> 5) & 0x1f; instruction_pointer_set(regs, get_x_reg(regs, xn)); + + if (user_mode(regs) && task_gcs_el0_enabled(current)) { + ret_addr = pop_user_gcs(&err); + if (err || ret_addr != procedure_link_pointer(regs)) + force_sig(SIGSEGV); + } + } void __kprobes From patchwork Tue Mar 18 20:48:39 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jeremy Linton X-Patchwork-Id: 14021532 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 4CF5F213E94; Tue, 18 Mar 2025 20:48:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1742330939; cv=none; b=uqF08VB84KQLyZjZmmt1iwpsUbLyf0uThCdCPDmZmE4vNd3nHQihUnpUYNnK1CJbskuICm69BdmF6Svr3tIbY3LLQ/Qs9yLcM11XS+7uqu6uEdcL5Gg5yQaUYOGX/pifa82Xbq4s+aaKetRWhxTJSWEvKVlsyi8FMhRFEonevBM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1742330939; c=relaxed/simple; bh=7ehPZLOngjd8ImMbsLIXTpmyAaWwJQcZfzQqjt5eY5s=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=XK9AmptzRek2E1YHF5kJVh38wi/fbtj4ChRp0ivVfhuaznOxdl4X5loWsVRqy4DaVhXNhnAzdE3nogxiOgUNynaPApn0bn/0Igj2xMDNEiz1fasUmtnwTqnaJK46G4qcJWK9WZGLOX79JAVA8M6XZ4OzxkT76hkMPHhKJRtrZ9Q= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 5C2331BF7; Tue, 18 Mar 2025 13:49:05 -0700 (PDT) Received: from u200865.usa.arm.com (usa-sjc-mx-foss1.foss.arm.com [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 5E1F13F694; Tue, 18 Mar 2025 13:48:56 -0700 (PDT) From: Jeremy Linton To: linux-trace-kernel@vger.kernel.org Cc: linux-perf-users@vger.kernel.org, mhiramat@kernel.org, oleg@redhat.com, peterz@infradead.org, acme@kernel.org, namhyung@kernel.org, mark.rutland@arm.com, alexander.shishkin@linux.intel.com, jolsa@kernel.org, irogers@google.com, adrian.hunter@intel.com, kan.liang@linux.intel.com, thiago.bauermann@linaro.org, broonie@kernel.org, yury.khrustalev@arm.com, kristina.martsenko@arm.com, liaochang1@huawei.com, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Jeremy Linton , Steve Capper Subject: [PATCH 5/7] arm64: uprobes: Add GCS support to uretprobes Date: Tue, 18 Mar 2025 15:48:39 -0500 Message-ID: <20250318204841.373116-6-jeremy.linton@arm.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250318204841.373116-1-jeremy.linton@arm.com> References: <20250318204841.373116-1-jeremy.linton@arm.com> Precedence: bulk X-Mailing-List: linux-trace-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Ret probes work by changing the value in the link register at the probe location to return to the probe rather than the calling routine. Thus the GCS needs to be updated with this address as well. Since its possible to insert probes at locations where the current value of the LR doesn't match the GCS state this needs to be detected and handled in order to maintain the existing no-fault behavior. Co-developed-by: Steve Capper Signed-off-by: Steve Capper (updated to use new gcs accessors, and handle LR/GCS mismatches) Signed-off-by: Jeremy Linton --- arch/arm64/kernel/probes/uprobes.c | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/arch/arm64/kernel/probes/uprobes.c b/arch/arm64/kernel/probes/uprobes.c index cb3d05af36e3..5e72409a255a 100644 --- a/arch/arm64/kernel/probes/uprobes.c +++ b/arch/arm64/kernel/probes/uprobes.c @@ -159,11 +159,41 @@ arch_uretprobe_hijack_return_addr(unsigned long trampoline_vaddr, struct pt_regs *regs) { unsigned long orig_ret_vaddr; + unsigned long gcs_ret_vaddr; + int err = 0; + u64 gcspr; orig_ret_vaddr = procedure_link_pointer(regs); + + if (task_gcs_el0_enabled(current)) { + gcspr = read_sysreg_s(SYS_GCSPR_EL0); + gcs_ret_vaddr = load_user_gcs((unsigned long __user *)gcspr, &err); + if (err) { + force_sig(SIGSEGV); + goto out; + } + /* + * If the LR and GCS entry don't match, then some kind of PAC/control + * flow happened. Likely because the user is attempting to retprobe + * on something that isn't a function boundary or inside a leaf + * function. Explicitly abort this retprobe because it will generate + * a GCS exception. + */ + if (gcs_ret_vaddr != orig_ret_vaddr) { + orig_ret_vaddr = -1; + goto out; + } + put_user_gcs(trampoline_vaddr, (unsigned long __user *) gcspr, &err); + if (err) { + force_sig(SIGSEGV); + goto out; + } + } + /* Replace the return addr with trampoline addr */ procedure_link_pointer_set(regs, trampoline_vaddr); +out: return orig_ret_vaddr; } From patchwork Tue Mar 18 20:48:40 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jeremy Linton X-Patchwork-Id: 14021533 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 75EC52144B7; Tue, 18 Mar 2025 20:48:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1742330940; cv=none; b=idBoadjsuXpgDv2qH7LAzSFfFLRrv9BqwBV6oyo15CTvyL5H6jHRiyM1Kn0hcj40Wc4b1xGMyDqclk3HSo9H19BxXiVysWlzPhLie6k8AhdPtvzEM+EIQPCIRCNs7k2BxHcsKrdjNZKhOsQTo1H9/wn5DI+iSfK+accMUvPzRV4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1742330940; c=relaxed/simple; bh=gK9egMUzih1a/rK9sRNXsbKWUsRrvAGvkOh19enB7rI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=bh/BZDM2MwwZ6BJzNLImspnQpFekLVTyVDenWyhXPjgCaRnQI3GNlfw2M35/TX52DDSEGh6x/PC3dP1zgkizXvJAVU0kMsZxWPiLaI0XIAGVoMMeDTOdBHu3o09AAdDAbP8nO28lr5SdUoAoh7k1oPGNp8uHgCMOa2TXWoBNPf4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 921121C00; Tue, 18 Mar 2025 13:49:06 -0700 (PDT) Received: from u200865.usa.arm.com (usa-sjc-mx-foss1.foss.arm.com [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id A18823F694; Tue, 18 Mar 2025 13:48:57 -0700 (PDT) From: Jeremy Linton To: linux-trace-kernel@vger.kernel.org Cc: linux-perf-users@vger.kernel.org, mhiramat@kernel.org, oleg@redhat.com, peterz@infradead.org, acme@kernel.org, namhyung@kernel.org, mark.rutland@arm.com, alexander.shishkin@linux.intel.com, jolsa@kernel.org, irogers@google.com, adrian.hunter@intel.com, kan.liang@linux.intel.com, thiago.bauermann@linaro.org, broonie@kernel.org, yury.khrustalev@arm.com, kristina.martsenko@arm.com, liaochang1@huawei.com, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Jeremy Linton Subject: [PATCH 6/7] uprobes: Allow the use of uprobe_warn() in arch code Date: Tue, 18 Mar 2025 15:48:40 -0500 Message-ID: <20250318204841.373116-7-jeremy.linton@arm.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250318204841.373116-1-jeremy.linton@arm.com> References: <20250318204841.373116-1-jeremy.linton@arm.com> Precedence: bulk X-Mailing-List: linux-trace-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 The uprobe_warn function is limited to the uprobe core, but the functionality is useful to report arch specific errors. Drop the static so it can be used in those code paths. Signed-off-by: Jeremy Linton --- arch/arm64/kernel/probes/simulate-insn.c | 8 ++++++-- arch/arm64/kernel/probes/uprobes.c | 4 ++++ include/linux/uprobes.h | 1 + kernel/events/uprobes.c | 2 +- 4 files changed, 12 insertions(+), 3 deletions(-) diff --git a/arch/arm64/kernel/probes/simulate-insn.c b/arch/arm64/kernel/probes/simulate-insn.c index 1fc9bb69b1eb..fe637fec8f36 100644 --- a/arch/arm64/kernel/probes/simulate-insn.c +++ b/arch/arm64/kernel/probes/simulate-insn.c @@ -56,8 +56,10 @@ static inline void update_lr(struct pt_regs *regs, long addr) if (user_mode(regs) && task_gcs_el0_enabled(current)) { push_user_gcs(addr + 4, &err); - if (err) + if (err) { + uprobe_warn(current, "GCS stack push failure"); force_sig(SIGSEGV); + } } procedure_link_pointer_set(regs, addr + 4); } @@ -160,8 +162,10 @@ simulate_ret(u32 opcode, long addr, struct pt_regs *regs) if (user_mode(regs) && task_gcs_el0_enabled(current)) { ret_addr = pop_user_gcs(&err); - if (err || ret_addr != procedure_link_pointer(regs)) + if (err || ret_addr != procedure_link_pointer(regs)) { + uprobe_warn(current, "GCS RET address mismatch"); force_sig(SIGSEGV); + } } } diff --git a/arch/arm64/kernel/probes/uprobes.c b/arch/arm64/kernel/probes/uprobes.c index 5e72409a255a..9349d521316c 100644 --- a/arch/arm64/kernel/probes/uprobes.c +++ b/arch/arm64/kernel/probes/uprobes.c @@ -54,6 +54,7 @@ int arch_uprobe_analyze_insn(struct arch_uprobe *auprobe, struct mm_struct *mm, switch (arm_probe_decode_insn(insn, &auprobe->api)) { case INSN_REJECTED: + uprobe_warn(current, "Unsupported instruction at probe location"); return -EINVAL; case INSN_GOOD_NO_SLOT: @@ -169,6 +170,7 @@ arch_uretprobe_hijack_return_addr(unsigned long trampoline_vaddr, gcspr = read_sysreg_s(SYS_GCSPR_EL0); gcs_ret_vaddr = load_user_gcs((unsigned long __user *)gcspr, &err); if (err) { + uprobe_warn(current, "GCS stack not available for retprobe"); force_sig(SIGSEGV); goto out; } @@ -180,11 +182,13 @@ arch_uretprobe_hijack_return_addr(unsigned long trampoline_vaddr, * a GCS exception. */ if (gcs_ret_vaddr != orig_ret_vaddr) { + uprobe_warn(current, "LR/GCS mismatch, likely due to incorrectly placed retprobe"); orig_ret_vaddr = -1; goto out; } put_user_gcs(trampoline_vaddr, (unsigned long __user *) gcspr, &err); if (err) { + uprobe_warn(current, "GCS stack update failure during retprobe"); force_sig(SIGSEGV); goto out; } diff --git a/include/linux/uprobes.h b/include/linux/uprobes.h index b1df7d792fa1..9578ef1ea5a3 100644 --- a/include/linux/uprobes.h +++ b/include/linux/uprobes.h @@ -185,6 +185,7 @@ struct uprobes_state { }; extern void __init uprobes_init(void); +extern void uprobe_warn(struct task_struct *t, const char *msg); extern int set_swbp(struct arch_uprobe *aup, struct mm_struct *mm, unsigned long vaddr); extern int set_orig_insn(struct arch_uprobe *aup, struct mm_struct *mm, unsigned long vaddr); extern bool is_swbp_insn(uprobe_opcode_t *insn); diff --git a/kernel/events/uprobes.c b/kernel/events/uprobes.c index b4ca8898fe17..613c1c76f227 100644 --- a/kernel/events/uprobes.c +++ b/kernel/events/uprobes.c @@ -118,7 +118,7 @@ struct xol_area { unsigned long vaddr; /* Page(s) of instruction slots */ }; -static void uprobe_warn(struct task_struct *t, const char *msg) +void uprobe_warn(struct task_struct *t, const char *msg) { pr_warn("uprobe: %s:%d failed to %s\n", current->comm, current->pid, msg); } From patchwork Tue Mar 18 20:48:41 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jeremy Linton X-Patchwork-Id: 14021534 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id E4D9A2147F7; Tue, 18 Mar 2025 20:48:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1742330941; cv=none; b=GjBYV8KUthHedEr9ti+pz2TFzmMhZte9N7UP6BW8QTDbUjA6+JCRD737N834ju0j9xxnAI/4mubGZBOucJ5eXrl1drCWvRdDaPwsgH4TjaueYU4hSfhUFuWXuBd8KR49qhnId+X1Sg7n9M9IxQjQ5V/BTdQnERbGqLxg+QT5nkk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1742330941; c=relaxed/simple; bh=/PZRRTDVlLur05eM4XoOBcAajZs7m9VrpQi9rIlYK3s=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Rr4BJqxcYW5N6kSed3GQv0Y6+KMxjOdtQ5OaJWseyAcmvqG8MTKd3xpfFUQGH8kANlc8C0kKGuDcOYyohJmqkLvjiXxcuEP2+ZpeetFjf/KDX70u5EJT2z4j1C3q61/f/Lue1P4ZIC+Hk+/oo0llJIU4ITh/28Y9AmtMlfq9lRU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 175701C01; Tue, 18 Mar 2025 13:49:08 -0700 (PDT) Received: from u200865.usa.arm.com (usa-sjc-mx-foss1.foss.arm.com [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 1D52B3F694; Tue, 18 Mar 2025 13:48:59 -0700 (PDT) From: Jeremy Linton To: linux-trace-kernel@vger.kernel.org Cc: linux-perf-users@vger.kernel.org, mhiramat@kernel.org, oleg@redhat.com, peterz@infradead.org, acme@kernel.org, namhyung@kernel.org, mark.rutland@arm.com, alexander.shishkin@linux.intel.com, jolsa@kernel.org, irogers@google.com, adrian.hunter@intel.com, kan.liang@linux.intel.com, thiago.bauermann@linaro.org, broonie@kernel.org, yury.khrustalev@arm.com, kristina.martsenko@arm.com, liaochang1@huawei.com, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Jeremy Linton Subject: [PATCH 7/7] arm64: Kconfig: Remove GCS restrictions on UPROBES Date: Tue, 18 Mar 2025 15:48:41 -0500 Message-ID: <20250318204841.373116-8-jeremy.linton@arm.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: <20250318204841.373116-1-jeremy.linton@arm.com> References: <20250318204841.373116-1-jeremy.linton@arm.com> Precedence: bulk X-Mailing-List: linux-trace-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Now that the uprobe paths have been made GCS compatible drop the Kconfig restriction. Signed-off-by: Jeremy Linton --- arch/arm64/Kconfig | 1 - 1 file changed, 1 deletion(-) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 940343beb3d4..8e6fe551f5fb 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -2223,7 +2223,6 @@ config ARM64_GCS default y select ARCH_HAS_USER_SHADOW_STACK select ARCH_USES_HIGH_VMA_FLAGS - depends on !UPROBES help Guarded Control Stack (GCS) provides support for a separate stack with restricted access which contains only return