From patchwork Sat Feb 3 12:25:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 13544142 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7850FC4828F for ; Sat, 3 Feb 2024 12:31:44 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0BA176B00A2; Sat, 3 Feb 2024 07:31:44 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 0662D6B00A3; Sat, 3 Feb 2024 07:31:43 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E24B06B00A4; Sat, 3 Feb 2024 07:31:43 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id CC8326B00A2 for ; Sat, 3 Feb 2024 07:31:43 -0500 (EST) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id AB5121A0699 for ; Sat, 3 Feb 2024 12:31:43 +0000 (UTC) X-FDA: 81750428886.28.CB3453E Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf03.hostedemail.com (Postfix) with ESMTP id 3D32320013 for ; Sat, 3 Feb 2024 12:31:40 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=Azso4+Yy; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf03.hostedemail.com: domain of broonie@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=broonie@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1706963502; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=HBFMasw6uM5uNsvS4uT7PpEXrKVc3sF3MkqkDHGhO48=; b=ZBKwPcgoVa7hYDxw6ptcrUZoz63wurj9t2WCPGPFNAEUzstnj6+p116jFYtpqk/o2kJpxs BQviRaqvAVHAW7YHHeCjF6QuOmCJMICBc7gngLfMXHnZq4OEslYEO49bzwJp5DRDuoUT6+ 76M76vAXm0PocxCdlsy2dMRB7KP2cHQ= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=Azso4+Yy; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf03.hostedemail.com: domain of broonie@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=broonie@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1706963502; a=rsa-sha256; cv=none; b=fd8ElDa9cXSWVRS40q9fKx09K2+2d2RRbCXcwtscNGzWDHCcU1V+GkRX+jZg3uRapT1gkA 4zyTQz7MepoJYt/IzKStWWlryCzmOceuTkpu8yHehFftUe40PgqqPKEF8UCzurgOfYaz61 o57b0yGr1SnabAIFbkEru17TGGLxWvI= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id BC5C3CE1EFB; Sat, 3 Feb 2024 12:31:37 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id BEDD7C43390; Sat, 3 Feb 2024 12:31:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1706963496; bh=HRCdw6MJRS/rCWztQtxmtqhkYbOj0W+kgtlVYf4SbO0=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=Azso4+YyAzl6+yIFDUNUmT4mFK2oE6rFzieOpcEAd31dy44KjhRbM0MikMRMmH/bt IDJwkyVqV4M9xRtQ7a6uFV+rRQ7jcOki77rkaLxessdsVjowAeul8LYofuitczB+7k uxjexhTTl7YqK+IA6T9pv3KcnMIzu0W6KQ0SvNXHFD57hEB/SjtQkZExvIxwW5ea4N YG+6juBGYl5PnRUrlmsTBDg9FGSZPOItMGzj8iuTun1R2FF7H8L6mxW9vmOsYxZLE9 rWwCcyQY+UHVxCR28wuKmMuR5+NnlznZ8CUJETnLCKJ7XqQP+UixCgNKk89noK4GHx rlSUg+0roOT0Q== From: Mark Brown Date: Sat, 03 Feb 2024 12:25:43 +0000 Subject: [PATCH v8 17/38] arm64/traps: Handle GCS exceptions MIME-Version: 1.0 Message-Id: <20240203-arm64-gcs-v8-17-c9fec77673ef@kernel.org> References: <20240203-arm64-gcs-v8-0-c9fec77673ef@kernel.org> In-Reply-To: <20240203-arm64-gcs-v8-0-c9fec77673ef@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Kees Cook , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , Thiago Jung Bauermann , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.13-dev-a684c X-Developer-Signature: v=1; a=openpgp-sha256; l=6026; i=broonie@kernel.org; h=from:subject:message-id; bh=HRCdw6MJRS/rCWztQtxmtqhkYbOj0W+kgtlVYf4SbO0=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBlvjDeDD8ANoqqbGdDnd/OGuhwpX6BzrLQ36MB5ZLR OGQPZxqJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZb4w3gAKCRAk1otyXVSH0ElXB/ 0cl9F8HCY4OiJaE0HDANmcmaQEbfXcFcXzKfY1VHC6GJR9gFQIpnme9dKTs+LrtwiE3gDY+6xW2Ajw szauDIVoauSS/Ab1v5E1K+om+qvbeYzD8T5V6eucJjrG4sVeIPBaDGOJfivUnZoOp+kuAY8zNpUEvI Q8oPJFZpRy82alEPzkVS5obR8LWL+DGUX17VRoLQUkdAmERuEdoSOvWJtuEgn+bC1gCHX9vMHTnqDQ pjIJXHogMzIZdzuOwZxQMCqeO3lA36Kxb+WtHfxDkpwbEdMJULrJHppQahPmitB7VoNl84uXARwqWD +mZ4SFE///Rh3kh0fH3sZu6xPnK2c0 X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB X-Rspamd-Queue-Id: 3D32320013 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: jt9r5ixhqsj6su68aywekh9wbgqqztjh X-HE-Tag: 1706963500-520824 X-HE-Meta: U2FsdGVkX18y+AaKzuJyItKAXP7ITgmgMtnIef7Z5dP2UbWBCftvuVjz+rmYNtF37s6Uy6V5jPUCmghbH5uDWca1ABTZ7bLa6KL7QwFdTXXfYB5uDPBCOMZX2pq8jW9E5+f3KnFY2km3zH7J/OKIt3+zMQtYTn3vrP2ppraiKY8VMF6RKNReV9ScsKDKoI+OdA1dg7XIsG2Dd+2aBB4LzkhbkZhDPbq9dpw9oJF5dKzig7VXo7vNtgTeW7DAZk+sl0kFvow7MCE13bQTn2HqDZgka9467QrjLtBQYmiswWdOm9TSTJCMcIddpIv3E+c7UDorcNqsJagEIzCuLnapyJ7BwnazP5lWU3Wd15ss+67cef/rnxYPqyGEFTH09HFtTNHffNyPFm475CuH19U5slEBoP03IALB9SnNVeiul3cN45uZckhd5jKpHMyn1frJHS2GklZd9Hbo8TT1AAqt+uliTSQFDUkTfIZ1Kzqvl+aqynAr989+6eK3Zu3JCxeDO5mK685rAeaW0TXlqSs5Ckv5DSandr3kFRl5or6aaK0KKeRRGdokFP9dvsRVjuWP4lCveJyTSO1lNcoapM45x0k38VDpjvS3+blyB1ZeGvbDD9ahe7wqTdHMFW77xZhCU6hOIkrmc5R4UoNxdlr3//eZ/4W22b3Jvjnc3xJZyXrlQg1Yg8fV8bbVWXkdDmrySipKCliZvy98KZp7YNMNXXy3RDiHy1L2lUMQdEZ9ROGun0K6rI/GKPYVsAt4Vik7PO5I1AhMBbZauXNBfyOEC6A/ondhs9F1CUC9dwPdEzbyY29a2+eZ+/nMu+5wxeFerQYf+1M1eclGmkt8TJ4VPI2tjbPIJhcF3p4C8JKvj+L43M1pAjCwXNcPHderNmFW4qZTLneT/QpAWnw8nBWY+2wN04FkEcWP+1IJLIknD5tWr9dcxB9JNr9SszdhJG9W7VtN1/fqy7/yT7i+KxX k7u2Adv6 x0+SoDUtjL5vtv9RKxEW4jJu3vhHfLWbhBM8FAL3XxlsqigwlSwNn9seVVmUM6et8EVZhBq/rtjXRXSSxWNEUdH/akNqlZ2aOFFwEcHOfO5A5+hGt0DbuI44VGLzcL5YCCSbHt8rYaApaI8DK9R48MD6X5TPgSQnCV7CcjiU06+BMZkR5ed6X8tvy8PGpoucaWJyYRXMwqW9tb2hcWVPXGDoiZEdcUTgowi/R8xIYZXowKYV8aeCQM/H0tH+fkmAP9RkE6QZy2vq7yWY3uJGxnN60BZW091P4ikOgqpMJSeiaYZ51I2KvdY3qKDUkmBHLR7wY X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: A new exception code is defined for GCS specific faults other than standard load/store faults, for example GCS token validation failures, add handling for this. These faults are reported to userspace as segfaults with code SEGV_CPERR (protection error), mirroring the reporting for x86 shadow stack errors. GCS faults due to memory load/store operations generate data aborts with a flag set, these will be handled separately as part of the data abort handling. Since we do not currently enable GCS for EL1 we should not get any faults there but while we're at it we wire things up there, treating any GCS fault as fatal. Signed-off-by: Mark Brown --- arch/arm64/include/asm/esr.h | 28 +++++++++++++++++++++++++++- arch/arm64/include/asm/exception.h | 2 ++ arch/arm64/kernel/entry-common.c | 23 +++++++++++++++++++++++ arch/arm64/kernel/traps.c | 11 +++++++++++ 4 files changed, 63 insertions(+), 1 deletion(-) diff --git a/arch/arm64/include/asm/esr.h b/arch/arm64/include/asm/esr.h index 353fe08546cf..20ee9f531864 100644 --- a/arch/arm64/include/asm/esr.h +++ b/arch/arm64/include/asm/esr.h @@ -51,7 +51,8 @@ #define ESR_ELx_EC_FP_EXC32 (0x28) /* Unallocated EC: 0x29 - 0x2B */ #define ESR_ELx_EC_FP_EXC64 (0x2C) -/* Unallocated EC: 0x2D - 0x2E */ +#define ESR_ELx_EC_GCS (0x2D) +/* Unallocated EC: 0x2E */ #define ESR_ELx_EC_SERROR (0x2F) #define ESR_ELx_EC_BREAKPT_LOW (0x30) #define ESR_ELx_EC_BREAKPT_CUR (0x31) @@ -382,6 +383,31 @@ #define ESR_ELx_MOPS_ISS_SRCREG(esr) (((esr) & (UL(0x1f) << 5)) >> 5) #define ESR_ELx_MOPS_ISS_SIZEREG(esr) (((esr) & (UL(0x1f) << 0)) >> 0) +/* ISS field definitions for GCS */ +#define ESR_ELx_ExType_SHIFT (20) +#define ESR_ELx_ExType_MASK GENMASK(23, 20) +#define ESR_ELx_Raddr_SHIFT (10) +#define ESR_ELx_Raddr_MASK GENMASK(14, 10) +#define ESR_ELx_Rn_SHIFT (5) +#define ESR_ELx_Rn_MASK GENMASK(9, 5) +#define ESR_ELx_Rvalue_SHIFT 5 +#define ESR_ELx_Rvalue_MASK GENMASK(9, 5) +#define ESR_ELx_IT_SHIFT (0) +#define ESR_ELx_IT_MASK GENMASK(4, 0) + +#define ESR_ELx_ExType_DATA_CHECK 0 +#define ESR_ELx_ExType_EXLOCK 1 +#define ESR_ELx_ExType_STR 2 + +#define ESR_ELx_IT_RET 0 +#define ESR_ELx_IT_GCSPOPM 1 +#define ESR_ELx_IT_RET_KEYA 2 +#define ESR_ELx_IT_RET_KEYB 3 +#define ESR_ELx_IT_GCSSS1 4 +#define ESR_ELx_IT_GCSSS2 5 +#define ESR_ELx_IT_GCSPOPCX 6 +#define ESR_ELx_IT_GCSPOPX 7 + #ifndef __ASSEMBLY__ #include diff --git a/arch/arm64/include/asm/exception.h b/arch/arm64/include/asm/exception.h index ad688e157c9b..99caff458e20 100644 --- a/arch/arm64/include/asm/exception.h +++ b/arch/arm64/include/asm/exception.h @@ -57,6 +57,8 @@ void do_el0_undef(struct pt_regs *regs, unsigned long esr); void do_el1_undef(struct pt_regs *regs, unsigned long esr); void do_el0_bti(struct pt_regs *regs); void do_el1_bti(struct pt_regs *regs, unsigned long esr); +void do_el0_gcs(struct pt_regs *regs, unsigned long esr); +void do_el1_gcs(struct pt_regs *regs, unsigned long esr); void do_debug_exception(unsigned long addr_if_watchpoint, unsigned long esr, struct pt_regs *regs); void do_fpsimd_acc(unsigned long esr, struct pt_regs *regs); diff --git a/arch/arm64/kernel/entry-common.c b/arch/arm64/kernel/entry-common.c index 0fc94207e69a..52d78ce63a4e 100644 --- a/arch/arm64/kernel/entry-common.c +++ b/arch/arm64/kernel/entry-common.c @@ -429,6 +429,15 @@ static void noinstr el1_bti(struct pt_regs *regs, unsigned long esr) exit_to_kernel_mode(regs); } +static void noinstr el1_gcs(struct pt_regs *regs, unsigned long esr) +{ + enter_from_kernel_mode(regs); + local_daif_inherit(regs); + do_el1_gcs(regs, esr); + local_daif_mask(); + exit_to_kernel_mode(regs); +} + static void noinstr el1_dbg(struct pt_regs *regs, unsigned long esr) { unsigned long far = read_sysreg(far_el1); @@ -471,6 +480,9 @@ asmlinkage void noinstr el1h_64_sync_handler(struct pt_regs *regs) case ESR_ELx_EC_BTI: el1_bti(regs, esr); break; + case ESR_ELx_EC_GCS: + el1_gcs(regs, esr); + break; case ESR_ELx_EC_BREAKPT_CUR: case ESR_ELx_EC_SOFTSTP_CUR: case ESR_ELx_EC_WATCHPT_CUR: @@ -650,6 +662,14 @@ static void noinstr el0_mops(struct pt_regs *regs, unsigned long esr) exit_to_user_mode(regs); } +static void noinstr el0_gcs(struct pt_regs *regs, unsigned long esr) +{ + enter_from_user_mode(regs); + local_daif_restore(DAIF_PROCCTX); + do_el0_gcs(regs, esr); + exit_to_user_mode(regs); +} + static void noinstr el0_inv(struct pt_regs *regs, unsigned long esr) { enter_from_user_mode(regs); @@ -732,6 +752,9 @@ asmlinkage void noinstr el0t_64_sync_handler(struct pt_regs *regs) case ESR_ELx_EC_MOPS: el0_mops(regs, esr); break; + case ESR_ELx_EC_GCS: + el0_gcs(regs, esr); + break; case ESR_ELx_EC_BREAKPT_LOW: case ESR_ELx_EC_SOFTSTP_LOW: case ESR_ELx_EC_WATCHPT_LOW: diff --git a/arch/arm64/kernel/traps.c b/arch/arm64/kernel/traps.c index 215e6d7f2df8..fb867c6526a6 100644 --- a/arch/arm64/kernel/traps.c +++ b/arch/arm64/kernel/traps.c @@ -500,6 +500,16 @@ void do_el1_bti(struct pt_regs *regs, unsigned long esr) die("Oops - BTI", regs, esr); } +void do_el0_gcs(struct pt_regs *regs, unsigned long esr) +{ + force_signal_inject(SIGSEGV, SEGV_CPERR, regs->pc, 0); +} + +void do_el1_gcs(struct pt_regs *regs, unsigned long esr) +{ + die("Oops - GCS", regs, esr); +} + void do_el0_fpac(struct pt_regs *regs, unsigned long esr) { force_signal_inject(SIGILL, ILL_ILLOPN, regs->pc, esr); @@ -838,6 +848,7 @@ static const char *esr_class_str[] = { [ESR_ELx_EC_MOPS] = "MOPS", [ESR_ELx_EC_FP_EXC32] = "FP (AArch32)", [ESR_ELx_EC_FP_EXC64] = "FP (AArch64)", + [ESR_ELx_EC_GCS] = "Guarded Control Stack", [ESR_ELx_EC_SERROR] = "SError", [ESR_ELx_EC_BREAKPT_LOW] = "Breakpoint (lower EL)", [ESR_ELx_EC_BREAKPT_CUR] = "Breakpoint (current EL)",