From patchwork Fri Apr 11 09:16:20 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 14047905 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 29D39C36010 for ; Fri, 11 Apr 2025 09:17:29 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C016128019C; Fri, 11 Apr 2025 05:17:27 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B8A6728019B; Fri, 11 Apr 2025 05:17:27 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 93E6128019C; Fri, 11 Apr 2025 05:17:27 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 708DD28019B for ; Fri, 11 Apr 2025 05:17:27 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id E0723818C0 for ; Fri, 11 Apr 2025 09:17:27 +0000 (UTC) X-FDA: 83321209734.07.5EDA388 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf22.hostedemail.com (Postfix) with ESMTP id 5136DC0002 for ; Fri, 11 Apr 2025 09:17:26 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf22.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1744363046; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=WSd0yigUChVpRcAhfJwe0J3l1gFAvojA/sDA8zvgUxg=; b=dn5f0Xz115Wc2wb3N9Z4sn7PmlukcRv8BHQnHjeDjlcOybp/BfNgiHG2tR2hdKLJp1u5h5 +FsHONJovXhaU21DWIIg5KxTssEDcTgar0px8iWcKZ2LYgyzoiUZeHXKsica0HhSX7IhwV qmWqvUfCuzG3e7x0HvWj5eCIJgH0cD0= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1744363046; a=rsa-sha256; cv=none; b=aTUHP2elyi2HOmfE6+eAneUdGiwbgPB7RiNX2emDjHVUOJwdhFpkCGnOjvuysm/L6DiKq/ cVk71h83rqJQsRfw+FcciI8PpqVNzaWpO3b6fLrdOXPIIpujYxSrWegASDO4GrQD4lkmM3 Wn3SY30wmF8MrKpiE4Xiy2RbwQlf5LU= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf22.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 783311596; Fri, 11 Apr 2025 02:17:25 -0700 (PDT) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id E5EB13F6A8; Fri, 11 Apr 2025 02:17:21 -0700 (PDT) From: Kevin Brodsky To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , Andrew Morton , Mark Brown , Catalin Marinas , Dave Hansen , David Hildenbrand , Ira Weiny , Jann Horn , Jeff Xu , Joey Gouly , Kees Cook , Linus Walleij , Andy Lutomirski , Marc Zyngier , Peter Zijlstra , Pierre Langlois , Quentin Perret , Rick Edgecombe , "Mike Rapoport (IBM)" , Ryan Roberts , Thomas Gleixner , Will Deacon , Matthew Wilcox , Qi Zheng , linux-arm-kernel@lists.infradead.org, x86@kernel.org Subject: [RFC PATCH v4 07/18] arm64: Reset POR_EL1 on exception entry Date: Fri, 11 Apr 2025 10:16:20 +0100 Message-ID: <20250411091631.954228-8-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250411091631.954228-1-kevin.brodsky@arm.com> References: <20250411091631.954228-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 5136DC0002 X-Stat-Signature: 3d845ds569zmh4p4s6eead83yrjd8jwz X-Rspam-User: X-HE-Tag: 1744363046-555879 X-HE-Meta: U2FsdGVkX1/GuRrH5u9OUsQRSobpygPFGFXK0KZcW2Few+RAKo0NegrYu0uvMC6CSz2z61qafNcVDqiuyco8FbXvP99Yrxme7fIy9QW59l4NZCjYWdnbZq7WsoEmIjGRzU0NtqrMw6wyQgbV//u8iaiyOKdKsGZrLFeH5STFjw1gxYajRv5QFJo+DKajoppV1GfhrWI2wpOCftRF8NHWB4W9JkftT073XqbsjlycFY41099MIdEQyk1vwSu6hKVko+7RDa0VTiufvsPvl3S/fzlSv4OzDk8HI+BLF0VIqkja3Z+rjDW5V+qOx6xOJ8SOKISYQLzh/8WTKGYhd0VXA9HqzK1mUVX9yO7HStYbJ6KtDk2WALh1TTHC2ZmLmUtZwejVExRhvRzj3QjUNJlJkN9dvK175v4sxphnQ/BzqplVk/PuhpqTVX0B5w67YKRO16/k/t5689AnG1imcYZ896f32+luJaPdFIraDNRRtahXmrSVDlc4oZVE7jIMhRlFrWK2AzAoxhVpg0Ij3hZDjOpozoa2FAfwoURCCljSb/CjtyX9e3br8PR1rEnoVnjVJEDdqmk+WFqJ20Eae9Id815q8k36IpGCf1TzdkTNzNPcYSOfPN9Om1rw1qlKDzN/Vu16kweTaZ5bDntCruvgRHXMb+2rC9fMI/fhBdBmOPHLAqIbeSldQT0C74wPwtkDbR8RAsXZzERGF0/KBuJvxCKv4mERgJdWSRM517SFktlQhK1zJbgezi1ZXgDxenUsvPEQddzciSPts21Qso+4WBs40opKSAcVuxJu9NEri9tbN8tnwHNCNTi/PBf7elUinZECbUtBC59Ls9r3zW/G+mD9YeveAIKDbLoy9KExYEux40Fx/I5OXznYDquTTtIfcouJVHubUZfsAtkV3tgkuaLqE7oh17wePFfVS7Zie624iWIu8Iwo4vUpw8B8caiQ1oQxKEACU0NuC+LijVW f/4mZPkl QsiQjEew4vVLRAxPnG8nMCK0vPZllLhsOtsJxVoyr7jVtpEfS8WYp0VimEJBSrosakrEjvvAwx+KXExSsgWcqRfi8+wbVGOzACs1BspwWg+GBMjZxpNKxf6y6mW4N6wTyrZyoXZ1d9twrkfJC9/uhrpu2ThdQhrn2E7gt+bzTEQ5jixWFZpjN9skQyaKm2HgFBImbrXGAVHki0Ss= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: POR_EL1 will be modified, through the kpkeys framework, in order to grant temporary RW access to certain keys. If an exception occurs in the middle of a "critical section" where POR_EL1 is set to a privileged value, it is preferable to reset it to its default value upon taking the exception to minimise the amount of code running at higher kpkeys level. This patch implements the reset of POR_EL1 on exception entry, storing the original value in a new pt_regs field and restoring on exception return. To avoid an expensive ISB, the register is only reset if the interrupted value isn't the default. No check is made on the return path as an ISB occurs anyway as part of ERET. Signed-off-by: Kevin Brodsky --- arch/arm64/include/asm/kpkeys.h | 10 ++++++++++ arch/arm64/include/asm/por.h | 4 ++++ arch/arm64/include/asm/ptrace.h | 4 ++++ arch/arm64/kernel/asm-offsets.c | 3 +++ arch/arm64/kernel/entry.S | 24 +++++++++++++++++++++++- 5 files changed, 44 insertions(+), 1 deletion(-) diff --git a/arch/arm64/include/asm/kpkeys.h b/arch/arm64/include/asm/kpkeys.h index 3b0ab5e7dd22..79ae33388088 100644 --- a/arch/arm64/include/asm/kpkeys.h +++ b/arch/arm64/include/asm/kpkeys.h @@ -8,6 +8,14 @@ #include +/* + * Equivalent to por_set_kpkeys_level(0, KPKEYS_LVL_DEFAULT), but can also be + * used in assembly. + */ +#define POR_EL1_INIT POR_ELx_PERM_PREP(KPKEYS_PKEY_DEFAULT, POE_RWX) + +#ifndef __ASSEMBLY__ + static inline bool arch_kpkeys_enabled(void) { return system_supports_poe(); @@ -46,4 +54,6 @@ static __always_inline void arch_kpkeys_restore_pkey_reg(u64 pkey_reg) #endif /* CONFIG_ARM64_POE */ +#endif /* __ASSEMBLY__ */ + #endif /* __ASM_KPKEYS_H */ diff --git a/arch/arm64/include/asm/por.h b/arch/arm64/include/asm/por.h index bffb4d2b1246..58dce4b8021b 100644 --- a/arch/arm64/include/asm/por.h +++ b/arch/arm64/include/asm/por.h @@ -10,6 +10,8 @@ #define POR_EL0_INIT POR_ELx_PERM_PREP(0, POE_RWX) +#ifndef __ASSEMBLY__ + static inline bool por_elx_allows_read(u64 por, u8 pkey) { u8 perm = POR_ELx_PERM_GET(pkey, por); @@ -38,4 +40,6 @@ static inline u64 por_elx_set_pkey_perms(u64 por, u8 pkey, u64 perms) return (por & ~(POE_MASK << shift)) | (perms << shift); } +#endif /* __ASSEMBLY__ */ + #endif /* _ASM_ARM64_POR_H */ diff --git a/arch/arm64/include/asm/ptrace.h b/arch/arm64/include/asm/ptrace.h index 47ff8654c5ec..e907df4225d4 100644 --- a/arch/arm64/include/asm/ptrace.h +++ b/arch/arm64/include/asm/ptrace.h @@ -166,6 +166,10 @@ struct pt_regs { u64 orig_x0; s32 syscallno; u32 pmr; +#ifdef CONFIG_ARM64_POE + u64 por_el1; + u64 __unused; +#endif u64 sdei_ttbr1; struct frame_record_meta stackframe; diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c index eb1a840e4110..a69c8ba0e085 100644 --- a/arch/arm64/kernel/asm-offsets.c +++ b/arch/arm64/kernel/asm-offsets.c @@ -75,6 +75,9 @@ int main(void) DEFINE(S_SYSCALLNO, offsetof(struct pt_regs, syscallno)); DEFINE(S_SDEI_TTBR1, offsetof(struct pt_regs, sdei_ttbr1)); DEFINE(S_PMR, offsetof(struct pt_regs, pmr)); +#ifdef CONFIG_ARM64_POE + DEFINE(S_POR_EL1, offsetof(struct pt_regs, por_el1)); +#endif DEFINE(S_STACKFRAME, offsetof(struct pt_regs, stackframe)); DEFINE(S_STACKFRAME_TYPE, offsetof(struct pt_regs, stackframe.type)); DEFINE(PT_REGS_SIZE, sizeof(struct pt_regs)); diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index 5ae2a34b50bd..bc042eabd730 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -20,6 +20,7 @@ #include #include #include +#include #include #include #include @@ -279,6 +280,19 @@ alternative_else_nop_endif .else add x21, sp, #PT_REGS_SIZE get_current_task tsk +#ifdef CONFIG_ARM64_POE +alternative_if_not ARM64_HAS_S1POE + b 1f +alternative_else_nop_endif + mrs_s x0, SYS_POR_EL1 + str x0, [sp, #S_POR_EL1] + mov x1, #POR_EL1_INIT + cmp x0, x1 + b.eq 1f + msr_s SYS_POR_EL1, x1 + isb +1: +#endif /* CONFIG_ARM64_POE */ .endif /* \el == 0 */ mrs x22, elr_el1 mrs x23, spsr_el1 @@ -409,7 +423,15 @@ alternative_else_nop_endif mte_set_user_gcr tsk, x0, x1 apply_ssbd 0, x0, x1 - .endif + .else +#ifdef CONFIG_ARM64_POE +alternative_if ARM64_HAS_S1POE + ldr x0, [sp, #S_POR_EL1] + msr_s SYS_POR_EL1, x0 + /* No explicit ISB; we rely on ERET */ +alternative_else_nop_endif +#endif /* CONFIG_ARM64_POE */ + .endif /* \el == 0 */ msr elr_el1, x21 // set up the return data msr spsr_el1, x22