From patchwork Thu Aug 22 01:15:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 13776851 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 42C56C5472C for ; Sun, 25 Aug 2024 18:06:52 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D5D288D001A; Sun, 25 Aug 2024 14:06:50 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CFDC68D0018; Sun, 25 Aug 2024 14:06:50 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9A2448D0016; Sun, 25 Aug 2024 14:06:50 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 6D4C48D0018 for ; Sun, 25 Aug 2024 14:06:50 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 1661680835 for ; Sun, 25 Aug 2024 18:06:50 +0000 (UTC) X-FDA: 82491548580.27.523D54F Received: from nyc.source.kernel.org (nyc.source.kernel.org [147.75.193.91]) by imf30.hostedemail.com (Postfix) with ESMTP id 5442280004 for ; Sun, 25 Aug 2024 18:06:48 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=Pbtd03YP; dmarc=pass (policy=quarantine) header.from=kernel.org; spf=pass (imf30.hostedemail.com: domain of broonie@kernel.org designates 147.75.193.91 as permitted sender) smtp.mailfrom=broonie@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724609145; a=rsa-sha256; cv=none; b=m8KoUpmpQhHTqHmLKAdqXVkotgt/5vqmr5Hc3llvvjMSZb01P2QKDYjukZlobvVjIoEZb2 KXP9SBxPn4vkHzI45BupCnVz+AfH26vIcdt/XX3X6kW9p4TS9byNWTwvf2e7Yr4TtuzOXq Oow8c+Yw3TITK6aiinaKrCkj1tx/G+g= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=Pbtd03YP; dmarc=pass (policy=quarantine) header.from=kernel.org; spf=pass (imf30.hostedemail.com: domain of broonie@kernel.org designates 147.75.193.91 as permitted sender) smtp.mailfrom=broonie@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724609145; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=aCcddeKQgbpVCZ8gdJeklAiNqjTPhcrj/cmJQo+qFOI=; b=KshMy7biRwv0Fae1IR7h3SJIepZkjWtBpRuzmAr/gtMJVOz/dhVL8O5Ja/0IJfeWC+3DrR /BFY4CVcq6uYEkHyshTuYPDeW/pucevAB6+JUCsLNZF9bwWaJEE6Zt8aditq4WYFHk8oLx Y3sj531GPNgLxY3kyjbYY1JdiVI93IY= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by nyc.source.kernel.org (Postfix) with ESMTP id 8A374A4210D; Thu, 22 Aug 2024 01:19:38 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id D83A7C32781; Thu, 22 Aug 2024 01:19:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1724289584; bh=e44VTA4PnVMnteUpPeHLUcAac2ATi0WZ4XE1rzWONBc=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=Pbtd03YPcs1kyLM4sBb0xDIyW2yq92U7tOZQ9fabgPGYa3s5xtQ6I40erqzcf22XJ xYieHlP7yB2U+mdBvsSaUnso5fxL7sMS6IO48Y0VLjNX6Sss+pouZus1XoYl/MIe1N dDl+uUaI0X21QWpoSgzYX03wdx20X+DEWCXRDn+7qj+Nfp3F8ftHUzSm3Yxs6QlIVR bo8sKzPlkxyZgmDFhB1Mx94j9gcgXuqr2ZU3+ReGVUBwaQr/O9DFV8onQN/RbRwm5O zdfw341jowqrUFuoeB13x34D0kZfRIndMSQgxUIIgGlfWcsegzcg8AsaMrSEhr5vnX uT6y0Hm5E4OOA== From: Mark Brown Date: Thu, 22 Aug 2024 02:15:21 +0100 Subject: [PATCH v11 18/39] arm64/traps: Handle GCS exceptions MIME-Version: 1.0 Message-Id: <20240822-arm64-gcs-v11-18-41b81947ecb5@kernel.org> References: <20240822-arm64-gcs-v11-0-41b81947ecb5@kernel.org> In-Reply-To: <20240822-arm64-gcs-v11-0-41b81947ecb5@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy , Kees Cook Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , Thiago Jung Bauermann , Ross Burton , Yury Khrustalev , Wilco Dijkstra , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.15-dev-37811 X-Developer-Signature: v=1; a=openpgp-sha256; l=6092; i=broonie@kernel.org; h=from:subject:message-id; bh=e44VTA4PnVMnteUpPeHLUcAac2ATi0WZ4XE1rzWONBc=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBmxpExKYuMVBeBz23oWOsSU1k9OAWwiLzazhRA1qWu Bx+hqyKJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZsaRMQAKCRAk1otyXVSH0NmsB/ 9EjPsQ/EeMfFBiLaIOkUOjhs28bhufGcXkuzr7C+Wbxopk+/XOgczp4HeDvZTP36PelXAY0aXHqMX1 5ydSxbyO+L1JPycTk0EClSE3uP6kIfLGkterYUUQAbOvueBiyayr6eNZjwqze4yUpl/upmbnyga6p4 x8DSnhLLsxeLKXkSfN7mCnagujT55MkIKpGbmlmcgkj0FVJpEz2cEaTBxaQk9RhBINzcBAGXUULwtV LIWb58IVPo0B+BX6en+13U18Fb7pxhovGl8MbndGIeewt3ZQcP2Yc7yQwf9CoZDJ6LP6/nqI34sxBz GBWmbU2MJqvPm5CiPuxTItmSUk4+FC X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB X-Rspamd-Queue-Id: 5442280004 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: oa37um59x1dr97h84tqwwesfauuyn6hn X-HE-Tag: 1724609208-289116 X-HE-Meta: U2FsdGVkX18Q0zCZaDbmwW0S1j6i6NiKIycIEkPzwzMFAgOdWw0VtL3kqlaWe+1NC8V+H206J5HTDuIsyP2TecfD4ZH0/y+2M4tNRMOulgOaRyx9+0RNJpIMcnh4mxncrNJRSzKC59srnVlLijZnOiQW/6FOd3i0jnIoA2ZLC3wprX34aAAUQEUmgVLrUDpmcwarfdxU1AL4tbycp8Pd/6apxfk7ryDfU9w9IYomaY4xQg0y7TmJ68l834zGI4Qp3CchzPR8SksGSalds/BwmexsbF7LL+Tk9Xp8xbII4NW1QbTZarKe1PgzfajBWJ661y4oaOwAzGkvrKX5UqutixbEmFVklFmn3LKl+5HJM60Mn4XExBn1xUAr8idxFmtGKDFjTTL98mQnhop6nn2RCZoVUPFax8BtT2+9bj2DK+Ag42CXGg7JLDK4pYP6orsUFzLb4Sgq2s7Fm9RoQz2iHspLFr+HilZ7JiToTmb9ztK873o09uNJGMfQHtnHA9KJf5cwMvu/4fMMc8ogIStQNZWBDbL3Lrj7i3OIs0L8KyZY/JI83frNLZBZRIBZ+ZOiONWY133GyK8mipGf2QZQzCijOqBubSrP6B2g2q8mFtA1+hqCwsGnBuPmlXLHcltpo6r1LGJiAfRUMwAkvpQydNYjNgpI9JzLRPRJbZdyzUciZjR/ek9hG4V9z8kPF9FZa0ST3zoyVvooRRwvLBtdo1P/Wfk9LecnU7OmyDbwZsbMvHtan4LoCD3r8M+BqAhqHfWvfSpUcFl/VwskaxjSzn/7mx0Q/1VDn5Bn1zbTqKuvNeb0TsAI/8EaYlvAcHpZS/K+5pdB2LU/TQ24k20IkWNr5fr15X3VKZPpjtyK0iiMRIjUGQqY+ZeUVIjb3/1f82iT5U+DMJLdT8eOjbaSq7RsK6jqI26/E7K1DZrC2u00Caa8j0ViiW2/fRsOLO/aQz9za2bPbsLKT5tHZii KYkXyjK9 B7hH3l4jbwJHIyNUCrO5Kma27Z6DCqH+8EtXjEw5byROHsvfu8stfMfrvHEx0sMO9nNU1jnoZfSo9Y9BZcMvCgAarbrDqZZZ8XRdRbHHbv/Tu6vX6Vvi1XzG1IV/6a4qTLBepK0ELy/6m1Gy1CZlah8ndWKj9MA4ItmW9xxqZ6fG2kiyKNkAUirSnTI6kf5EqkSgcKCqID/8wVOJtsaEjoF/WbytOQ8pTSM9OJR7kz9l8LIuWrWpOG+RlRJ/wKWHJ+hbncyGTHhbhQbpIY4M83DkKYnbRP5jDVI7EEJrV4E1Fxp57/got1p/IayxUfq/nAs6DxpjOjcnvVVcLNn4SjiqVXvlrcbc6MT3Fp+E87AyS0oYl9xmk9AU3G95yJ8RSUNPGYKWD5b5gU+VA2kr1Aw+RH6ofNs88fTM+8MOkm896PPg= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: A new exception code is defined for GCS specific faults other than standard load/store faults, for example GCS token validation failures, add handling for this. These faults are reported to userspace as segfaults with code SEGV_CPERR (protection error), mirroring the reporting for x86 shadow stack errors. GCS faults due to memory load/store operations generate data aborts with a flag set, these will be handled separately as part of the data abort handling. Since we do not currently enable GCS for EL1 we should not get any faults there but while we're at it we wire things up there, treating any GCS fault as fatal. Reviewed-by: Thiago Jung Bauermann Signed-off-by: Mark Brown --- arch/arm64/include/asm/esr.h | 28 +++++++++++++++++++++++++++- arch/arm64/include/asm/exception.h | 2 ++ arch/arm64/kernel/entry-common.c | 23 +++++++++++++++++++++++ arch/arm64/kernel/traps.c | 11 +++++++++++ 4 files changed, 63 insertions(+), 1 deletion(-) diff --git a/arch/arm64/include/asm/esr.h b/arch/arm64/include/asm/esr.h index 56c148890daf..0c231adf3867 100644 --- a/arch/arm64/include/asm/esr.h +++ b/arch/arm64/include/asm/esr.h @@ -51,7 +51,8 @@ #define ESR_ELx_EC_FP_EXC32 (0x28) /* Unallocated EC: 0x29 - 0x2B */ #define ESR_ELx_EC_FP_EXC64 (0x2C) -/* Unallocated EC: 0x2D - 0x2E */ +#define ESR_ELx_EC_GCS (0x2D) +/* Unallocated EC: 0x2E */ #define ESR_ELx_EC_SERROR (0x2F) #define ESR_ELx_EC_BREAKPT_LOW (0x30) #define ESR_ELx_EC_BREAKPT_CUR (0x31) @@ -385,6 +386,31 @@ #define ESR_ELx_MOPS_ISS_SRCREG(esr) (((esr) & (UL(0x1f) << 5)) >> 5) #define ESR_ELx_MOPS_ISS_SIZEREG(esr) (((esr) & (UL(0x1f) << 0)) >> 0) +/* ISS field definitions for GCS */ +#define ESR_ELx_ExType_SHIFT (20) +#define ESR_ELx_ExType_MASK GENMASK(23, 20) +#define ESR_ELx_Raddr_SHIFT (10) +#define ESR_ELx_Raddr_MASK GENMASK(14, 10) +#define ESR_ELx_Rn_SHIFT (5) +#define ESR_ELx_Rn_MASK GENMASK(9, 5) +#define ESR_ELx_Rvalue_SHIFT 5 +#define ESR_ELx_Rvalue_MASK GENMASK(9, 5) +#define ESR_ELx_IT_SHIFT (0) +#define ESR_ELx_IT_MASK GENMASK(4, 0) + +#define ESR_ELx_ExType_DATA_CHECK 0 +#define ESR_ELx_ExType_EXLOCK 1 +#define ESR_ELx_ExType_STR 2 + +#define ESR_ELx_IT_RET 0 +#define ESR_ELx_IT_GCSPOPM 1 +#define ESR_ELx_IT_RET_KEYA 2 +#define ESR_ELx_IT_RET_KEYB 3 +#define ESR_ELx_IT_GCSSS1 4 +#define ESR_ELx_IT_GCSSS2 5 +#define ESR_ELx_IT_GCSPOPCX 6 +#define ESR_ELx_IT_GCSPOPX 7 + #ifndef __ASSEMBLY__ #include diff --git a/arch/arm64/include/asm/exception.h b/arch/arm64/include/asm/exception.h index f296662590c7..674518464718 100644 --- a/arch/arm64/include/asm/exception.h +++ b/arch/arm64/include/asm/exception.h @@ -57,6 +57,8 @@ void do_el0_undef(struct pt_regs *regs, unsigned long esr); void do_el1_undef(struct pt_regs *regs, unsigned long esr); void do_el0_bti(struct pt_regs *regs); void do_el1_bti(struct pt_regs *regs, unsigned long esr); +void do_el0_gcs(struct pt_regs *regs, unsigned long esr); +void do_el1_gcs(struct pt_regs *regs, unsigned long esr); void do_debug_exception(unsigned long addr_if_watchpoint, unsigned long esr, struct pt_regs *regs); void do_fpsimd_acc(unsigned long esr, struct pt_regs *regs); diff --git a/arch/arm64/kernel/entry-common.c b/arch/arm64/kernel/entry-common.c index b77a15955f28..54f2d16d82f4 100644 --- a/arch/arm64/kernel/entry-common.c +++ b/arch/arm64/kernel/entry-common.c @@ -463,6 +463,15 @@ static void noinstr el1_bti(struct pt_regs *regs, unsigned long esr) exit_to_kernel_mode(regs); } +static void noinstr el1_gcs(struct pt_regs *regs, unsigned long esr) +{ + enter_from_kernel_mode(regs); + local_daif_inherit(regs); + do_el1_gcs(regs, esr); + local_daif_mask(); + exit_to_kernel_mode(regs); +} + static void noinstr el1_dbg(struct pt_regs *regs, unsigned long esr) { unsigned long far = read_sysreg(far_el1); @@ -505,6 +514,9 @@ asmlinkage void noinstr el1h_64_sync_handler(struct pt_regs *regs) case ESR_ELx_EC_BTI: el1_bti(regs, esr); break; + case ESR_ELx_EC_GCS: + el1_gcs(regs, esr); + break; case ESR_ELx_EC_BREAKPT_CUR: case ESR_ELx_EC_SOFTSTP_CUR: case ESR_ELx_EC_WATCHPT_CUR: @@ -684,6 +696,14 @@ static void noinstr el0_mops(struct pt_regs *regs, unsigned long esr) exit_to_user_mode(regs); } +static void noinstr el0_gcs(struct pt_regs *regs, unsigned long esr) +{ + enter_from_user_mode(regs); + local_daif_restore(DAIF_PROCCTX); + do_el0_gcs(regs, esr); + exit_to_user_mode(regs); +} + static void noinstr el0_inv(struct pt_regs *regs, unsigned long esr) { enter_from_user_mode(regs); @@ -766,6 +786,9 @@ asmlinkage void noinstr el0t_64_sync_handler(struct pt_regs *regs) case ESR_ELx_EC_MOPS: el0_mops(regs, esr); break; + case ESR_ELx_EC_GCS: + el0_gcs(regs, esr); + break; case ESR_ELx_EC_BREAKPT_LOW: case ESR_ELx_EC_SOFTSTP_LOW: case ESR_ELx_EC_WATCHPT_LOW: diff --git a/arch/arm64/kernel/traps.c b/arch/arm64/kernel/traps.c index 9e22683aa921..d410dcc12ed8 100644 --- a/arch/arm64/kernel/traps.c +++ b/arch/arm64/kernel/traps.c @@ -500,6 +500,16 @@ void do_el1_bti(struct pt_regs *regs, unsigned long esr) die("Oops - BTI", regs, esr); } +void do_el0_gcs(struct pt_regs *regs, unsigned long esr) +{ + force_signal_inject(SIGSEGV, SEGV_CPERR, regs->pc, 0); +} + +void do_el1_gcs(struct pt_regs *regs, unsigned long esr) +{ + die("Oops - GCS", regs, esr); +} + void do_el0_fpac(struct pt_regs *regs, unsigned long esr) { force_signal_inject(SIGILL, ILL_ILLOPN, regs->pc, esr); @@ -838,6 +848,7 @@ static const char *esr_class_str[] = { [ESR_ELx_EC_MOPS] = "MOPS", [ESR_ELx_EC_FP_EXC32] = "FP (AArch32)", [ESR_ELx_EC_FP_EXC64] = "FP (AArch64)", + [ESR_ELx_EC_GCS] = "Guarded Control Stack", [ESR_ELx_EC_SERROR] = "SError", [ESR_ELx_EC_BREAKPT_LOW] = "Breakpoint (lower EL)", [ESR_ELx_EC_BREAKPT_CUR] = "Breakpoint (current EL)",